Test Suites
A test suite is a tool for checking that your system does what you think it should. The good news is that every application has at least one test suite. The bad news is that test suites vary greatly in quality.
Your test site will come in one of three flavours:
- User-driven
- Manual
- Automated
User-Driven Suite
User-driven test suites are the default most of the time and are the highest risk. User-driven test suites work by releasing the update, and waiting for users to complain.
This has the obvious (only?) advantage of being easy. It has the downside of being terrible for your customers. User-driven testing leads to dissatisfied users, a lack of confidence in being able to release updates, and old bugs that just keep on returning.
I don’t recommend user-driven testing in any but the most trivial of circumstances. If you are looking to do custom development you will certainly want to do at least manual testing.
Manual Test Suite
A manual test suite means you have a formal specification to test your application. This will typically look like a set of instructions detailing specific screens, entered values, and actions to take in the application. Complex or mission-critical workflows, like completing a shopping cart, will have multiple paths specified, detailing success and error conditions.
Manual test suites reflect a tradeoff between thoroughness and time to complete. Since real people are clicking through the tests one at a time, testing every possible situation is often not feasible. Additionally, manual testing is often left till the end of a project and completed in a batch. This increases the risk that work will not be completed on schedule, or that unforeseen issues will arise.
Automated Test Suite
Automated tests often function very similarly to manual tests with the key difference that humans are not involved. Instead, the computer does all the manual work, but performs it with complete focus and in a dozen browsers at the same time, taking a fraction of the time.
This has the clear advantage of saving time and labour costs but has another huge benefit. With manual testing, you typically do all the testing once the changes are “ready”. This means you don’t catch issues early. Instead of doing final polishing when getting ready to launch, you are putting out fires that could have been caught much earlier in the process.
With automated testing, the entire suite is typically done multiple times per day. A subset of the tests related to the feature being worked on is often run continuously to provide instant feedback. This is taken to a logical extreme with test driven development (TDD) where you write the test before you write any of the code. Once the test passes, by definition, the code is complete.
This means that errors are caught early and regressions are avoided.
Having a solid test suite allows the development team to make changes more effectively, improves reliability and speed, and helps business owners to forward with greater confidence.
If you don’t have a test suite in place, look into adding one. It can be a big job, but it will have huge benefits!