Wednesday, September 15, 2021

[New post] APIs Unleashed 07: Data Driven Testing

APIs Unleashed 07: Data Driven Testing

by Nithin

APIs Unleashed 07:
Data Driven Testing

API unleashed

What is Data driven Testing?

The concept of data-driven testing is about separating the test data from the tests. Data-driven testing uses placeholders for the values, which can be varied to define different test cases. During the test execution, the placeholders will be replaced by specific values provided in the external files like JSON, CSV etc.

Also in some cases, client will be providing the data file and they are used for execution.

When should we use Data driven Testing?

Data-driven testing is most useful for tests that have a similar structure but using different data. This helps to test your APIs efficiently.

We can implement data-driven for the following scenarios:

- Same API request to run N numbers of times, with different set of data.
- To check the behavior of the API by providing different flavors/combinations of data.
- To perform some basic load/performance Tests for your APIs.
- To cover specific scenarios based on the requirements.

For today's illustration we will be using the dummy API point, POST: https://jsonplaceholder.typicode.com/posts and the request body is in JSON format as below:

{
"title": "foo",

"body": "bar",

"userId": 1
}

KarateDSL: Data driven Testing

Let's see how data driven testing can be done using KarateDSL

When we need to test the same scenario with a different set of data, the data driven testing technique plays an integral role.

It helps in maintaining our code more effectively by eliminating code redundancy.

Do you want to know how to do it?

Let's look at some of the techniques offered by KarateDSL-

Method 1 - Using "Examples" Keyword — Cucumber style

The "Scenario Outline with Examples" keyword will be used in this approach; the code snippet below demonstrates how the "Examples" keyword aids in accomplishing data driven functionality.

Two scenarios are depicted in the above screenshot, each with a different set of data. (indicated by a red rectangle)

Method 2 - Reading Request body from a JSON File

In this method, we'll read the request body from a JSON file.

This technique is highly recommended over CSV file, to know more refer this section of the documentation.

Sample JSON File consist of following data:

The syntax for accepting a JSON request body from a JSON file is "read(your json file.json)".

If you look at the report above, you'll notice that the scenario is executed three times for each json object in the array.

Method 3 - Using JSON Function Data Source i.e. Dynamic method

This technique is an add-on offered by Karate. It is extremely powerful way of generating test data without needing to load a huge number of data rows into memory.

The generator function in the code snippet below is used to auto-increment the value of "i."

In the above example we did not use any JSON file or Examples keyword, we are rather using JS function to construct the request-body for various data dynamically.

We hope you found this information to be interesting, as it covers a wide range of techniques in achieving data (along with examples) driven in KarateDSL.

POSTMAN: Data driven Testing

Let's see how data driven testing can be done using POSTMAN

In Postman, when we run the collection through collection runner/newman, we have the ability to feed an external data file in JSON/CSV format. Postman is smart enough to read the file and process it, all you have to do is just match the name of the field in the request and in the file.

Let's see in detail with an example below.

Create a new request, fill in the endpoint and provide the body as mentioned.

Save and hit "Send".

Step 1:

Since it's going to be a data driven the body values are replaced with variables like below:

Step 2:

Based on the value for the three fields, you can build a file manually with N number of data. You can also use Mockaroo tool here, please read this blog for more details on this. Here, we created a file with 10 different set of data for body section.

Step 3:

Now, this file goes as input for our request. But before this, it is better to have few collection level test cases for validation. Here, we will just add the basic assertion for validating the response code.

Step 4:

Select the folder/collection and click "Run".

Step 5:

Below window is launched.

Step 6:

Select the data driven file which we created, under "Data" section.

File is read and automatically few field are populated. You can override the value based on your requirement. For example, instead of 10 iteration, if you just need 4 we can edit it here. The first four data set is picked for execution. If you need to pick any specific data set, then we need to handle that logic in our scripting part.

Also if the file type is as CSV it will be displayed as below.

Step 7:

After making the configuration changes as per our needs, we can click Run.

You can see all the iterations and their test cases here.

Tada , it is done!

When working with newman along with the collection link/path we need to provide the data file along with the command (-d datafile). Refer this link for more details on running with newman along with the different commands and examples.

This is one of the heavily used feature of Postman. Currently there are feature requests open for supporting few other data files too!

Note: For now Postman supports only two formats: JSON and CSV, also once you use the data variable you will not be able to run as stand alone request. You should either do it through collection runner or newman.

Read this for more details: https://learning.postman.com/docs/running-collections/working-with-data-files/

We discussed about data driven testing for your APIs using Postman and KarateDSL. Testing with different combinations will help to test your APIs better. Also this approach will be useful when some UAT users need to run some tests from their end.

So please try with multiple combinations and explore more around data driven approach.

Dear readers, thank you for your time. See you in the next few days with our next topic!!