Managing System State in Contract Testing with Pact

Contract testing is a software testing technique focused on ensuring that interactions between different components or services in a system adhere to a predefined contract, so that changes in one do not negatively impact the others.

However, one of the most significant challenges in implementing contract testing is effectively managing the initial state of the system before tests are executed.

This article explores the importance of this aspect and various techniques to address it effectively.

Why It Matters?

Managing system state in contract testing is crucial for several reasons:

1- Reproducibility: Ensures that tests are consistent and reproducible, regardless of the environment or time of execution.

2- Isolation: Allows components under test to be isolated, reducing interference from external factors.

3- Reliability: Increases confidence in test results, as they are based on a known and controlled state.

4- Efficiency: Facilitates automation and rapid test execution by not relying on complex environment configurations.

Going Deeper

One of the greatest advantages of using Pact for contract testing is its ability to generate HTTP traffic on both the provider and consumer sides, thanks to its request mocking system. This allows us to validate both sides independently.

For these checks to be performed on the provider's side, the provider must be ready to receive requests at the time of testing, meaning it needs to be up and running. Pact will then validate that the provider's responses meet the pre-established contract.

This is where the challenge of properly managing data and system state during testing arises, as these factors are crucial for contract validation. 

In this context, we will explore various techniques to address these challenges.

A key aspect is determining the level of realism needed in the system's state. On one hand, a more realistic state can lead to more reliable tests; on the other hand, it may slow down the tests and make them harder to maintain.

Context

To clarify the specific scenario we are discussing in this article, let's look at an example in Java where the consumer expects a student with ID 1 to exist, and the code that the provider needs to implement.

Consumer

@Pact(consumer \= "Student", provider \= "Provider")

public RequestResponsePact getStudent(PactDslWithProvider builder) {

   return builder.given("Student 1 exists")

        .uponReceiving("get student with ID 1")

        .path("/student/1")

        .method("GET")

        .willRespondWith()

        .status(200)

        .body(newJsonBody(object \-> {

           object.stringType("id", "1");

           object.stringValue("name", "Sara");

           object.stringType("email", "sara.test@test.com");

           object.numberType("studentNumber", 23);

   }).build()).toPact();

}

Provider

@State("student 1 exist")

 public void noStudentExist() {

 }

*Pact always requires us to declare a function to manage the state that the consumer sets, but we’re not obligated to implement it.

In the following cases, we’ll see what modifications need to be made to this function for it to have an effect.

Techniques

Now, we'll evaluate different approaches for managing the system's initial state before testing, each with its own pros and cons.

• Preliminary Considerations

As we've mentioned in another article, there are conceptual differences between contract testing and functional testing. The goal of contract testing is to validate, early and automatically, that the consumer and provider can communicate correctly with each other by adhering to the agreed-upon rules.

However, functional testing goes a bit further, as it not only validates that communication is correct, but also verifies that the components behave as expected, meaning they “do what they’re supposed to do.” This includes checking the content of responses, data flows, and expected side effects.

This difference may seem like a minor detail, but it significantly changes the testing strategy. Contract testing focuses less on the specific values of the messages and more on their “shape”, such as schema, data types, etc. 

This distinction plays a crucial role in evaluating the pros and cons of each approach.

• Using Real Data Sources

Let’s assume our service consumes information from a development database. With this approach, contract validations would be done using that same data source.

✅Pros:

• No specific configuration needed to run the tests.

• A pre-load script might not be necessary.

• High fidelity with the production environment.

• Can identify issues related to real data.

❌ Cons:

• Tests are tightly coupled to the existing data in the database.

• Possible collisions with other users or tests.

• The initial state may be unknown, making test reproducibility difficult.

• Harder to manage or delete data.

• Dependency on resource availability.

• Potential performance and execution speed issues.

• Difficulty in controlling edge cases or specific scenarios.

Provider Code

As in the example, since the application will start with the appropriate configuration to point to the correct database, no specific modifications would be needed.

@State("student 1 exist")

 public void noStudentExist() {

 }

Conclusion

While this approach requires the least configuration, it's the least advisable. Over time, the problems it creates far outweigh the initial benefits.

• Dockerized Dependencies or In-Memory Database

Dockerized dependencies use container technology to encapsulate and isolate the system components needed for contract testing. This technique allows for the creation of consistent and reproducible environments, where each service or dependency runs in its own container. Docker makes it easier to configure and precisely control the system state, offering a balance between realism and manageability.

Although in-memory databases and dockerized environments are not entirely equivalent solutions, in this context, they have similar pros and cons.

✅Pros:

• Consistent and reproducible environment.

• Easy to configure and share across teams.

• Good balance between realism and control.

• Isolated and portable environment.

• Simplifies test debugging.

• Login system.

❌Cons:

• Initial data loading can be complicated.

• Maintaining the initial data script can be laborious.

• Requires constant synchronization between database changes and the data load script.

• May require significant hardware resources.

• Possible complexity in orchestrating multiple containers.

• Potentially long startup time for quick tests.

Provider Code

For this example, we can use TestContainers. This way, for each test, we would:

1- Generate a MongoDB instance.

2- Inject dependencies into the controller.

3- Configure the connection with the container.

4- Insert a student with ID 1 before the test.

@Testcontainers

class ProviderContractTest {

    @Container

    static MongoDBContainer mongoDBContainer \= new MongoDBContainer("mongo:4.4.6");

    @Autowired

    private StudentRepository studentRepository;

    @Autowired

    private StudentController studentController;

    @DynamicPropertySource

    static void setProperties(DynamicPropertyRegistry registry) {

        registry.add("spring.data.mongodb.uri", mongoDBContainer::getReplicaSetUrl);

    }

    @BeforeEach

    void setUp(PactVerificationContext context) {

        context.setTarget(new MockMvcTestTarget(studentController));

        studentRepository.deleteAll();

    }

    @State("Student 1 exists")

    void student1Exists() {

        Student student \= new Student();

        student.setId("1");

        student.setName("Sara");

        student.setEmail("sara.test@example.com");

        student.setStudentNumber(99);

        studentRepository.save(student);

    }

Conclusion

This hybrid approach can be very convenient when a representative dataset is needed for testing, as it avoids the "surgical" configuration required by mocking. However, it also necessitates constant synchronization of the code and database changes.

• Code-mocked dependencies

Code-mocked dependencies involve creating simulated or "mock" versions of the external services and components that the system under test interacts with. These simulations are programmed to respond in a predefined manner to requests, allowing for complete control over the behavior of the dependencies. This technique is particularly useful for testing edge cases and scenarios that are difficult to reproduce with real systems.

✅Pros:

• Complete control over dependency behavior.

• Fast and resource-efficient.

• Facilitates testing of edge cases and hard-to-reproduce situations.

• No collisions with other users.

❌Cons:

• If our code isn’t prepared for dependency injection, implementation can be complex.

• If the response requires interaction from multiple controllers, programming all the mocks can be laborious.

• Less realistic than other options.

• Risk of divergence between the mock and actual behavior.

• May require constant maintenance to stay in sync with changes in real dependencies.

Provider Code

If, for example, we use Spring Boot, we can declare the Mock and inject it into the controller before starting each test. This way, we could define specific behaviors and data for the test quickly, easily, and in isolation.

class ProviderContractTest {

    @Mock

    private StudentRepository studentRepository;

    @InjectMocks

    private StudentController studentController;

    @BeforeEach

    void setUp(PactVerificationContext context) {

        val testTarget \= new MockMvcTestTarget();

        testTarget.setControllers(studentController);

        context.setTarget(testTarget);

    }

    @State("Student 1 exists")

    public void student1Exists() {

        Student expected \= new Student();

        expected.setId("6");

        expected.setName("Sara");

        expected.setEmail("fran.test@sngular.com");

        expected.setStudentNumber(99);

        given(studentRepository.findById("1"))

                .willReturn(java.util.Optional.of(expected));

    }

Conclusion

If our code supports it, this would be the ideal option as it gives us the most control in testing, generates less data coupling, and runs faster. It is especially useful for validating edge cases.

Closing Thoughts

Effective system state management is crucial for successful contract testing. Each technique has its own strengths and weaknesses, and the choice will depend on factors like system complexity, number of dependencies, code quality, available resources, and specific testing goals.

In practice, many teams opt for a hybrid approach, combining different techniques depending on the needs of each component or service. The important thing is to maintain a balance between reproducibility, test efficiency, and maintenance costs.

Regardless of the chosen technique, it's essential to document and automate the system state configuration. This not only facilitates consistent test execution but also helps new team members understand and contribute to the testing process.

Ultimately, proper system state management in contract testing leads to more reliable tests, greater confidence in service integration, and, consequently, more robust and higher-quality systems.