May 17, 2016
Ace, an up-and-coming application developer, woke up this morning with a headache. He spent several hours late last night troubleshooting problems with integration tests between his application and another application called “The Hub”. The tests were failing because for some reason the data coming back from the Hub API was not what his application was expecting. As it turned out, there was another application running integration tests with The Hub at the same time and changing the very same data! Before going to bed that night Ace wished for his own copy of the Hub API he could use exclusively for his testing needs…
We do not know if there was a shooting star falling from the sky that night, but the next day Ace read this article. “Wow,” Ace couldn’t believe it, “Wishes do come true, after all!”
API Dependencies Problems
More often than not, applications nowadays use a bunch of APIs in their workflows. These are direct, first-degree API dependencies. Those APIs may in turn depend on a good number of other APIs, which in turn may have their own dependencies, and so on. These are transitive API dependencies.
“Yep,” noted Ace. “Applications are more and more interconnected via APIs”.
The unavailability of these direct and transitive dependencies during development and testing are cause for missed deadlines, long hours wasted troubleshooting false positive defects, and altogether skipped testing which results in poor quality, real defects, and rework. How come? Gerard Meszaros sums it up very well in his book “xUnit Test Patterns: Refactoring Test Code”:
“…Sometimes it is just plain hard to test the SUT [System Under Test] because it depends on other components that cannot be used in the test environment. Such a situation may arise because those components aren’t available, because they will not return the results needed for the test, or because executing them would have undesirable side effects”.
“Yes, yes,” nodded Ace. “My team and I, unfortunately, experience these problems quite often”.
Here are some typical issues with API dependencies:
- An API runs in a shared test environment. Updates by one API consuming application are visible to all other API consumers and cause unreliable tests and false positive defects.
- Depended-on API does not have enough test environments to support parallel development and testing of the same application.
- An API dependency goes down and it takes hours to find somebody from that team to bring it back up.
- API dependency of an API dependency of an API dependency goes down and it takes hours to pinpoint the failed API and find somebody from that team to bring it back up.
- An API doesn’t exist yet but new development needed it “yesterday”.
- Front-end development – like for a web or mobile app – needs a back-end API with data and lightweight logic to iterate over and get early feedback from customers on a new user interface design.
- Seeding the data store of a depended-on API with the right data before an integration test can start requires challenging coordination across teams and long lead times.
- A third-party external API requires pay-per-transaction fees even in non-production environments. Testing can get expensive, not to mention performance load tests.
- Third-party external APIs impose rate limiting with low quotas in non-production environments. Testing at your speed and frequency can be challenging, not to mention performance testing.
“Hmm…” thought Ace. “We use mocks and spies in our automated unit tests. They work great for us and help us to get around some of the problems with remote API dependencies. Unit tests, though, are executed on developers’ workstations and in server build environments only, not beyond that”.
“Ugh!” continued Ace. ” We also sometimes write stubs with complicated ‘if…else if…else’ statements to provide canned data to remote API calls. Coding a stub to return dynamic output for the same input adds a lot of complexity and everyone seems to be doing it differently” Ace got all fired up. “To add to all that, some of the testers can’t write code and that takes time away from our developers”.
API Simulations
The good news is that there is a solution (of course :-)). In test and development, applications can achieve autonomy and get in control by using API simulations to mimic the data and behavior of first-degree API dependencies.
“API simulations?” wondered Ace. “What is that?”
API simulations are stand-ins that imitate real APIs or APIs not built yet.
For all intent and purposes, the application talking to an API simulation during a test does not know the difference – is it the real API or a simulation? The application under test does not care for as long as the simulation is realistic.
Here is another benefit – transitive dependencies can do no harm anymore. By mimicking the first-degree dependencies, API simulations effectively remove the reliance on transitive dependencies as well! Rather powerful, no?
It is also a no brainer when it comes to an API that does not exist yet – an API simulation can play-act how the real API will behave and what data it would return for a given input.
“Cool,” Ace was starting to warm up to the idea of API simulations. “How do I go about API simulations?”
Without a doubt, there is no substitute to using the right tool for a job. Make sure you pick an easy to use, feature-rich, reasonably priced tool (why not free?) to model and run API simulations.
Check out API Simulator™ – a free, configuration-driven tool for testers and developers that will get you on the road to success.
In subsequent articles we will explore API Simulator and the world of API simulations and simlets in more details.
“I got it!” exclaimed Ace aloud. People around him smiled – he looked happy and excited. That night Ace went to bed on time and dreamed of his upcoming vacation.
@