I prefer pictures to words especially nice pictures that take problems I’d been thinking about and present a solution in an elegant manner, much more elegant than anything I could ever have achieved. For example take James Bach’s Heuristic Test Strategy Model, Alister Scott’s ice cream cone anti-pattern or Brian Marick’s agile testing quadrants. They just make sense.
It’s a bit more difficult to see what to do and how to do it if you’re testing an API. Is testing a single API end point a unit test or is it a functional test? If it’s considered to be a unit test then does your unit testing cover mandatory and optional parameters as well as boundary values, out-of-range parameters, exception handling and sensible default values for every end point? These kinds of tests are essential but they often fall through the cracks because they require a very particular devious-but-detail-focused mindset with the ability to balance pragmatism, coverage and value and irrespective of who writes the tests, they take a lot of time.
If we look at what are often called the business-facing tests, they assume that the individual end points have been sufficiently tested i.e. a functional test of an API often requires a piece of code that calls multiple API end-points in order to achieve a particular business outcome. The same is true of exploratory tests and other tests that are listed in the manual agile testing quadrant.
For me, the difficulties that I see in testing APIs is that in order to achieve appropriate coverage in each quadrant, we need a testing mindset, an understanding of business value in terms of APIs and strong Development skills. While the types of tests listed in the quadrants are still very relevant, the lines between the quadrants are much more blurred and types of tests don’t really fit into nice boxes any more (if they ever really did).
If manual testing has no place in a new agile API testing quadrant then what implication does that have for our existing test teams? At the moment, technical testers are very much in demand and command high salaries but will APIs totally redefine the role of the tester? I guess time will tell.
Let us help with some static analysis of your API definition file. If you pass a poor Swagger file to a test team then you’re wasting time; reduce the feedback loop and get this right first. You can use our Swagger / OAS validator for free, just click here:
You can help! Please share this tool by clicking here:
Within seconds you will have a list of errors, warnings and useful suggestions for your dev team to work on to ensure your definition file is not just compliant but solid. This is the first step to RESTful API happiness. Please use the tool regularly and please share this useful tool with your network and let us know what your thoughts are.