Functional testing is the first pillar of API testing. As a part of this test, we will set up and run a number of tests that make up a test suite. We will also learn how to configure success criteria to determine PASS/FAIL for each test case in the test suite.
Our Test API
US Census Bureau has begun to provide Open API access to the Economic Census is the U.S. To use the census API you will need to register and request a Key. For more information on this please visit census website
Let’s say we would like to test that API correctly returns the current population of California. From documentation on the site, California is state:06 and we need to issue a get request for P0010001 with the above key
1. Configure REST test cases in SOAPSonar we first need to create a new Test Group to hold all Test cases – Click File->New->Test Group as shown in the Figure below.
2. Right click on the Tests folder and Select New REST Test. Right click on New Test and name it California.
3. Now we need to provide Protocol, Host, Path, URI Parameters and the request Method.
There are two ways to do it – One you can simply type it all in URI Window. The second is to enter individual values in the boxes provided. Either way you do it, SOAPSonar will make sure that all values are in Sync:
In the URI window enter http://api.census.gov/data/2010/sf1?key=xxxxxxxxx&get=P0010001&for=state:06 (please enter your own key) and GET
4. COMMIT the values for test case and SAVE your project.
5. You are now ready to send your first REST request in SOAPSonar. RUN the test case The API response will be processed and displayed on the bottom Response tab.
In this case response will be send in a JSON format and should look something like
[[“P0010001″,”state”], [“37253956″,”06”]]. Population of state 06 (California) is 37,253,956
If you don’t get this, something is wrong. On occasion, if the service is down, you may get a timeout after 10 seconds.
For every request, the response from the web service API must be evaluated. To determine whether a response is valid or invalid, a tester should setup pre-defined filters that examine HTTP return codes or any business specific content contained in a response.
6. Next, click to the Success Criteria Tab, to define evaluation criteria that SOAPSonar will use to determine if the TC Pass or Fail:
Click Add Criteria Rule-> Document->Xpath Match:
7. Click on the XPath Match rule to edit it.
SOAPSonar will automatically recognize that response is in JSON format. ( Tip: click on UPDATE TREE if the tree is not populated)
8. To validate that State and Population values returned as expected Right-click on the JSON value 06 node and select Compare Element Value( to validate State).
SOAPSonar will automatically create a rule that this element must contain) 06 value and COMMIT your changes.
9. Repeat the same for node value 37253956( to validate Population) COMMIT your changes.
10. Now lets add a test case for Maryland. We could repeat steps 2-9 using State:24. The faster way to do it is to clone California by Right Click on California and select Clone. Rename it Maryland and modify State:24. COMMIT your changes and RUN the GET request. Go to success criteria, and refresh . Under tab Criteria Rules make sure 1st. Element is 24 and 2nd Element is 5773552 (The expected responses.
All test configuration and results validation setup is complete and you are ready to run the test.
11. Once the test cases are defined, the user can select any combination of test cases and build a Test Suite. To build a test suite, click on the Run View as shown in the Figure below. From the left-most navigation panel, Drag-and-Drop “California” and “Maryland” test cases into the Default Test Suite. Commit your changes. Test cases that appear under DefaultGroup are selected to run as the Default Suite.
12. RUN the test case and then select Analyze Results in Report View on the Real-time Run Monitor.
13. As seen in the Report View Panel below, all tests PASS (as seen in the Test Result Column) with the responses from the Service operations as expected.
If for any reason Test Case fails you may go to “Success Criteria Evaluation” Tab to learn the reason why.
14. By selecting the appropriate report and clicking on the GENERATE icon a PDF test report is automatically generated detailing the tests passed, response times etc.
The default success criteria is HTTP 200 = PASS, however Unlike web-site testing, you cannot rely on HTTP codes alone for web services API testing. Deep content inspection is critical for evaluating whether a service is behaving as expected.