Reports

Introduction

In our previous post, we explored Logging and Debugging in REST Assured, learning how to troubleshoot API tests effectively. Now, we’ll dive into Test Reporting, a crucial technique for documenting and presenting test results in a clear, visual format. This guide uses Allure, a powerful reporting framework, integrated with REST Assured and JUnit to generate detailed test reports. It’s designed for beginners and experienced developers, providing clear explanations and practical examples.

Key Point: Test reporting with Allure enhances visibility of REST Assured test results, making it easier to analyze test outcomes, failures, and trends for teams and stakeholders.

What is Test Reporting?

Test Reporting involves generating structured reports that summarize test execution results, including passed/failed tests, execution times, and error details. Reports help:

• Track test coverage and quality.
• Communicate results to developers, QA, and managers.
• Identify patterns in test failures.

REST Assured doesn’t have built-in reporting, so we use Allure, which integrates seamlessly with JUnit and provides interactive HTML reports with features like test steps, attachments, and timelines.

Setting Up Allure with REST Assured

To generate reports, we’ll set up Allure with REST Assured and JUnit, using the public API https://jsonplaceholder.typicode.com for examples.

Ensure your pom.xml includes dependencies for REST Assured, JUnit, Allure, and the Allure Maven plugin:

    2.27.0
    1.9.22


    
        io.rest-assured
        rest-assured
        5.4.0
        test
    
    
        org.junit.jupiter
        junit-jupiter
        5.10.2
        test
    
    
        org.hamcrest
        hamcrest
        2.2
        test
    
    
        io.qameta.allure
        allure-junit5
        ${allure.version}
        test
    


    
        
            org.apache.maven.plugins
            maven-surefire-plugin
            3.2.5
            
                
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                
                
                    
                        allure.results.directory
                        target/allure-results
                    
                
            
            
                
                    org.aspectj
                    aspectjweaver
                    ${aspectj.version}
                
            
        
        
            io.qameta.allure
            allure-maven
            2.12.0
            
                ${allure.version}
            
        
    

Additional Setup:

• Install the Allure command-line tool from Allure’s documentation to generate and view reports.
• Ensure Java and Maven are configured on your system.

Generating Test Reports with Allure

Let’s create REST Assured tests with Allure annotations to generate detailed reports.

Example 1: Basic Test with Allure Annotations

Use Allure annotations to add metadata and steps to a test.

import io.qameta.allure.Description;
import io.qameta.allure.Feature;
import io.qameta.allure.Step;
import io.restassured.RestAssured;
import org.junit.jupiter.api.Test;
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

@Feature("Post API Tests")
public class BasicAllureTest {

    @Test
    @Description("Verify retrieval of a single post by ID")
    public void testGetPost() {
        RestAssured.baseURI = "https://jsonplaceholder.typicode.com";

        sendGetRequest(1)
            .then()
                .statusCode(200)
                .body("id", equalTo(1))
                .body("title", notNullValue());
    }

    @Step("Send GET request for post ID {postId}")
    private Response sendGetRequest(int postId) {
        return given()
            .pathParam("postId", postId)
            .when()
                .get("/posts/{postId}");
    }
}

Explanation:

• @Feature: Groups tests under a feature (e.g., “Post API Tests”) in the report.
• @Description: Adds a description to the test case.
• @Step: Marks a method as a reportable step, showing detailed execution in the report.
• The test validates a GET request for a post.

Run and Generate Report:

1. Run tests: mvn clean test
2. Generate report: mvn allure:serve (opens the report in a browser)

The report shows test status, steps, and response details in an interactive HTML interface.

Example 2: Attaching Request and Response Details

Attach request and response data to the Allure report for debugging.

import io.qameta.allure.Allure;
import io.qameta.allure.Description;
import io.qameta.allure.Feature;
import io.restassured.RestAssured;
import io.restassured.response.Response;
import org.junit.jupiter.api.Test;
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

@Feature("Post API Tests")
public class AttachDetailsTest {

    @Test
    @Description("Verify post creation with request and response logging")
    public void testPostCreation() {
        RestAssured.baseURI = "https://jsonplaceholder.typicode.com";

        String requestBody = "{\"title\": \"Test Post\", \"body\": \"Test Body\", \"userId\": 1}";
        Allure.addAttachment("Request Body", "application/json", requestBody, ".json");

        Response response = given()
            .contentType("application/json")
            .body(requestBody)
            .when()
                .post("/posts")
            .then()
                .statusCode(201)
                .body("title", equalTo("Test Post"))
                .extract().response();

        Allure.addAttachment("Response Body", "application/json", response.asString(), ".json");
    }
}

Explanation:

• Allure.addAttachment: Attaches the request and response bodies to the report as JSON files.
• Attachments appear in the report, allowing you to inspect payloads during debugging.
• The test simulates creating a post and validates the response.

Pro Tip: Attach request/response data for failed tests to simplify debugging, but avoid attaching large payloads to keep reports lightweight.

Example 3: Handling Test Failures

Create a test that intentionally fails to demonstrate Allure’s failure reporting.

import io.qameta.allure.Description;
import io.qameta.allure.Feature;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;
import io.restassured.RestAssured;
import org.junit.jupiter.api.Test;
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

@Feature("Error Handling Tests")
public class FailureTest {

    @Test
    @Description("Verify error response for non-existent post")
    @Severity(SeverityLevel.CRITICAL)
    public void testNotFoundError() {
        RestAssured.baseURI = "https://jsonplaceholder.typicode.com";

        given()
            .pathParam("postId", 9999)
            .when()
                .get("/posts/{postId}")
            .then()
                .statusCode(200); // Intentionally incorrect to cause failure
    }
}

Explanation:

• @Severity: Marks the test as critical, highlighting its importance in the report.
• The test expects a 200 status but gets 404, causing a failure.
• Allure’s report shows the failure reason, stack trace, and logged details.

Using Request Specification with Allure

Combine a Request Specification with Allure for consistent test setup and reporting.

import io.qameta.allure.Description;
import io.qameta.allure.Feature;
import io.qameta.allure.Step;
import io.restassured.RestAssured;
import io.restassured.builder.RequestSpecBuilder;
import io.restassured.specification.RequestSpecification;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

@Feature("Post API Tests")
public class AllureSpecTest {

    private static RequestSpecification requestSpec;

    @BeforeAll
    public static void setup() {
        requestSpec = new RequestSpecBuilder()
            .setBaseUri("https://jsonplaceholder.typicode.com")
            .addHeader("Accept", "application/json")
            .log(LogDetail.ALL) // Enable logging for Allure attachments
            .build();
    }

    @Test
    @Description("Verify retrieval of a post using request specification")
    public void testGetPostWithSpec() {
        sendGetRequest(1)
            .then()
                .statusCode(200)
                .body("id", equalTo(1))
                .body("userId", equalTo(1));
    }

    @Step("Send GET request for post ID {postId}")
    private Response sendGetRequest(int postId) {
        return given()
            .spec(requestSpec)
            .pathParam("postId", postId)
            .when()
                .get("/posts/{postId}");
    }
}

Explanation:

• requestSpec: Defines the base URI, headers, and logging.
• spec(requestSpec): Applies the specification to the test.
• @Step: Logs the request as a step in the Allure report.

Generating and Viewing Allure Reports

After running tests, generate and view the Allure report:

1. Run Tests: mvn clean test generates results in target/allure-results.
2. Generate Report: mvn allure:serve to open the report in a browser, or mvn allure:report to create a static report in target/site/allure-maven-plugin.

The report includes:

• Overview: Summary of passed/failed tests.
• Suites: Test cases grouped by feature.
• Graphs: Visuals for test status and execution time.
• Attachments: Request/response details for debugging.

Important: Ensure the Allure command-line tool is installed and added to your system PATH to use mvn allure:serve.

Tips for Beginners

• Use Annotations: Add @Feature, @Description, and @Step to organize and document tests clearly.
• Attach Relevant Data: Include request/response bodies or logs for failed tests to aid debugging.
• Keep Reports Lightweight: Avoid attaching large payloads or excessive logs to maintain report performance.
• Integrate with CI/CD: Use Allure with tools like Jenkins to automate report generation.

Troubleshooting Tip: If the Allure report doesn’t generate, verify the target/allure-results directory contains JSON files after running tests. Check Maven plugin configurations and ensure Allure is installed.

What’s Next?

In the next post, we’ll cover Integration with CI/CD, exploring how to integrate REST Assured tests into continuous integration pipelines using tools like Jenkins or GitHub Actions. Stay tuned for more hands-on examples!