Don’t Start with Unit Tests!

The dev team may have forgotten to write unit tests but unless you are at the bleeding edge of continuous integration (most companies are not!) this is not your biggest product quality issue. Unit tests are often over-relied upon, for example, validating that a method accepts only specified types is not going to mitigate the risk that changing where that method is called will cause a customer credit card payment process to fail.

Whether teams have no unit tests or whether they have lots, a daily build is not credible just because it compiles and passes the unit tests. Don’t start there.

To have a credible CI environment, the build needs to be automatically deployed to a ‘primary’ product environment and automatically smoke tested (at the functional level) on a daily basis. Start with this. Once that process is up and running for a few days (at least!) and the smoke tests are passing, it’s time for the next step.

When smoke tests pass on the ‘primary’ environment, automatically promote the build to the ‘secondary’ test environment (if your CI system is decent, this is easy). Automatically execute the longer duration suites on the secondary environment including core & full functional regression test suites and non-functional (performance, load and stress) test suites. These are suites that could take days to execute and we need to abstract these suites away from the ‘primary’ daily build environment or face the inevitable issue of test ‘traffic’ congestion occurring. We then lose the value of the daily build process.

Simply set up the two environments and execute the smoke tests daily on the primary, simple.

There is a crux to this climb. Never promote a build to the secondary if it fails the smoke tests on the primary, it is an argument worth having. If you lose, you will end up shipping bad product, guaranteed.

Book your FREE one hour consultancy session right now