What is an API?
API stands for Application Programming Interface.
API is an interface that is used by the programmers to make communication from your system’s application to other applications/third party servers. The developers connect to the APIs to make requests and get responses from other applications as required.
APIs in Action:
How is it possible to make online flight booking from your credit card? Your bank must have provided the APIs for online transaction that is internally called by the flight booking website. The flight booking websites makes REQUEST and receives RESPONSE; based on the response (successful/failure of transaction) the ticket gets booked.
If your bank’s APIs does not work properly, transaction becomes unreliable and can cause potential loss of business. Hence, APIs used in such cases should be tested thoroughly before rolling out.
Testing an API is no different than testing for anything else. You still follow same steps of the test method, but some of them have a different flavour.
The GUIs would have items like menus, buttons, check boxes, and combo lists that would trigger the event or would require an action to be taken. Similarly, the APIs have input parameters as request and output as response; both need to be tested.
Each API carries its own set of input parameters / request and based on the combination of input parameters/request we send, the API responds accordingly.
While testing APIs, input/output schema (request/response) coverage becomes important. We still probably care about all the same coverage used to do with non-API testing (scenario coverage, requirements coverage, and application data coverage), but in API testing schema coverage comes into picture.
Test coverage is simple when the API carries less number of parameters. For example, consider an API named MULTIPLY with 2 parameters.
[table id=3 /]
These are relatively simple to test since the input can be defined with various value pairs and results can be validated against expected return value.
The challenges come around when we deal with APIs which got heavy request / response with more number of parameters, especially the optional parameters.
Challenges in Test Coverage:
Inadequate test coverage might cause huge problems. The quality of the product will sufferand product will become bug prone. There are quite a few challenges that are ecountered to ensure enough test coverage in the API based applications. Following is the list:
- Schema Updates
- Parameter Combination
- API Call Sequencing
- Request data combination
The input/output schema (request/response) skeleton may vary time to time due to the business requirements. The coverage should be extended for that. For example, If a new parameter getting added in the request, the coverage has to increase with all the request combination using the new parameter and possible data combination.
Each API will take various input parameters and there will be huge number of combinations of input params. Verification on missing mandatory parameter is quite simple. The combination will get increased with more number of optional parameters. There might be Boolean input params which may carry value 1 or 0 and each of the combination among these parameters might make difference in the business case and the response which is to be covered and tested.
API Call Sequencing:
Apart from testing of each API, the sequence of APIs to be called also a challenge and may end up with inadequate coverage. For example consider the sample shopping cart APIs as below:
Apart from testing of each API, there can various sequences of flow can be made out of it. The output of an API can be used as an input of another API. All the sequence flow should be covered to deliver bug free application.
Request data combination:
Each parameter might be designed to carry a specific value and it should be validated. It can be a number range, max length restrictions, acceptance of specific string / value. These kinds of parameters might get less attention while test case preparation and ends with lack of coverage. It will become problematic from end user perspective.
Overcoming the Challenges:
Whenever there is a change in the request/response of the API, the related test cases should be revisited for modification/including more coverage. The change might in addition/removal of few parameters, request body change, response body change.
For example, let us take the MULITPLY API got an optional parameter int C:
You can find the below table the parameter combination has been increased as there is a new parameter got introduced in the API.
[table id=4 /]
In above example we increased the coverage for 1 additional parameter. What if there are 50 parameters getting added to the API? We should come up with test coverage that includes parameters with all the permutation of combinations so that it will help us to discover all the failures. We can use any tool such as “All Pairs” to derive all the combinations.
The input parameters should be tested with the valid and invalid values using strategies like the boundary value analysis and equivalence partitioning. Shuffle the parameters with all possible combinations to find the failures.
Grasp the purpose of the APIs, the business need in which they are to be used. How the wrapper UI going to pull the data from the APIs? How it will be real time scenarios? Keep all such things in mind. Once all this parameter selections and combinations are designed, different call sequences need to be explored.
- 4.trainType (default type express)
- 5.quotaCode (default quota General)
- 6.seniorCitizen (default as No)
- 7.gender (default as Male)
* are the mandatory parameters.
The fare is going to be calculated differently on each parameter combination you choose. Hence it is important to cover all the possible combinations.
[table id=5 /]
Refer the above table. We added the test cases for verification of missing mandatory parameter (TC-2,3,4) and still we can add more few combos.
The test cases for various optional parameter combinations are added in TC-4,5,6. By this way we can track the coverage for all the combinations of the parameters. We can design separate table for the parameters for which boundary values need to be analysed.
The test coverage is really important as APIs play vital roles in business modules and is exposed to third parties as well. An API which fails to meet the expected functionality, reliability, performance can end up with issues which can cause major business impact.