Contract Testing: The Journey to Confidence

Once we’ve decided to test on the branch — as described in Test Before You Merge — the next question is what is the right way to do it? Testing in isolated environments sounds appealing, but when systems rely on classic end-to-end testing, the approach quickly falls apart. Why? Because E2E tests require the entire system to be deployed and integrated. Feature branches rarely have that luxury.

As we move testing earlier in the software development lifecycle, we must not only shift left — we must also shift down the testing pyramid. Instead of validating everything through expensive end-to-end tests, we push verification closer to unit and component levels. But if we move functional tests lower, an important question appears:

How do we validate integrations between services?

This is where contract testing becomes extremely powerful.

---

Classic E2E vs Contract Testing

Classic end-to-end tests validate the system by running everything together — all services deployed, all dependencies live. That works in a stable shared environment, but it creates several problems:

• Slow feedback. A full E2E suite can take hours to complete.
• Fragile results. Any service in the chain can cause an unrelated failure.
• Hard to isolate. When a test fails, it is not obvious which service or change caused it.
• Impossible on branches. Running the whole system for every feature branch is expensive and logistically complex.

Contract testing introduces a different model. Instead of testing entire systems together, each service validates its communication boundaries independently — without requiring the rest of the system to be running. The result is faster, more focused feedback that can run on any branch, at any point in development.

---

Shifting Down the Testing Pyramid

Testing on the branch becomes possible when we shift validation lower in the pyramid.

Figure: Contract tests reduce the need for expensive end-to-end testing.

In this model:

• Unit tests validate small pieces of logic
• Component tests validate service behavior in isolation
• Contract tests validate communication between services
• E2E tests become minimal smoke checks

Contract tests effectively replace most integration testing traditionally performed at the top of the pyramid.

---

Services Communication as a Testable Unit

In contract testing we treat service interaction itself as a unit of testing.

Think of it this way: a service may work perfectly in isolation, but still fail in production because it sends a field the downstream service doesn’t expect, or because the response it receives has changed shape. These are integration failures — and they are invisible to unit tests.

Instead of verifying the entire system at once, we validate the structure of requests, the structure of responses, and compatibility between services. Each service still maintains its own functional tests. But contract tests add integration guarantees without requiring the entire system to run.

This changes the debugging experience too. When a contract test fails, the failure points directly at the communication boundary — not somewhere inside a 20-step E2E scenario.

---

Two Sides of Communication

Communication between services is always bidirectional. Even if a service only performs a request, it still depends on the response format.

In contract terminology:

Role	Responsibility
Consumer	Defines expectations about the API
Provider	Guarantees compatibility

The consumer is the service that initiates the interaction — it knows what it needs from the other side. The provider is the service that responds — it is responsible for honouring what consumers depend on.

This split is important because it separates ownership. Consumers own their expectations. Providers own the guarantee. Neither side needs to know the internal implementation of the other — they only need to agree on the interface.

Contract testing verifies outgoing requests, incoming responses, and schema compatibility between both sides. If either side drifts, the contract test catches it before the change reaches a shared environment.

---

What Exactly Is a Contract?

A contract is a formal definition of the expected interaction between two services — what the consumer sends, and what the provider returns.

For an HTTP API, that typically includes the method, endpoint path, headers, request body schema, response body schema, field types, and validation rules. For events or messages, it defines the schema of the payload. For database interactions, it defines the structure of the records being read or written.

In all cases, the contract answers one question: is the shape of the data compatible?

What a contract does not cover is business logic. It does not say what the provider should do with the data, or whether the returned values are correct for a given situation. For example: a contract can verify that a response contains a field called status with a string value. It cannot verify that the status is "active" when the user has completed onboarding. That is the responsibility of functional tests.

This boundary is what makes contracts lightweight and stable — they change only when the interface changes, not every time internal logic does.

---

Single Source of Truth

Consistency is achieved by having one shared contract definition.

Without a shared source, each team defines their own understanding of the interface. Providers write mocks based on what they think they return. Consumers write tests based on what they think they receive. The two definitions drift apart silently — and the gap only surfaces in production.

Both API providers and consumers rely on the same contracts. In practice this usually means a shared contracts repository, library, or versioned contract package. This repository contains request and response schemas, validation rules, and optional test data generators. Every service validates against the same definitions.

When the contract changes, everyone sees the change. There is no ambiguity about what the current interface looks like.

Figure: Service repositories reference a shared contracts package — the single source of truth for schemas and test data generation.

---

How Contract Testing Enables Testing on Branches

Consumers write their tests against the shared contract — not against a live provider. This means consumer tests can run at any time, independently, on any branch.

When the provider changes the contract, the responsibility shifts to the provider. It is their change, and it is their job to verify it does not break any of the dependent consumers. The provider runs the consumer test suites against the updated contract and their new implementation.

Figure: Contract changes trigger consumer verification before code is merged.

Consider a concrete example: Service A provides an API, services B and C consume it. When Service A changes the contract:

1. The contract is updated in a feature branch.
2. Consumer tests from B and C run against the new contract and provider implementation.
3. If all tests pass — the change is safe to go.
4. If any test fails, there are two possible outcomes:
   • The consumer needs to update first, adapting to the new contract before the provider change can go live.
   • The contract change needs to be revised, because the proposed interface breaks consumers in a way that cannot be resolved immediately.

Either way, the problem is surfaced before any code is merged or deployed. The provider never has to guess whether their change is safe — the tests give a definitive answer.

---

Organizing Branches and Pipelines

To support this workflow, the branching strategy must align with the testing strategy.

Figure: When a contract branch is updated, consumer tests run automatically against the new provider implementation — catching incompatibilities before merge.

Tests evolve with code

Tests should always be written in the same branch as the application code. Even if they live in a separate repository, it should still be a mapped branch. Once the change is merged, tests merge automatically as well. Keeping the main branch green is critical — more on this in Green or Gone: Why We Stop Trusting Tests.

Contracts evolve within branches

Contracts represent future system behavior, so they must exist in the same feature branch as the implementation. A contract merged ahead of the code creates a false expectation that the system already supports. A contract merged after the code means there was a window where the implementation was unverified. Keeping them together eliminates both problems.

Consumer tests run automatically

When a contract changes, consumer tests trigger automatically. They run against the new contract and provider implementation — simulating how the system will behave once the new provider version is deployed. With the results, the provider knows whether the change is safe to merge.

---

Contract Testing with surety

One of the biggest practical challenges is organizing a framework that supports contract definitions as first-class objects. The surety test framework was built to solve exactly this problem.

Surety introduces schemas and contracts as Python classes — replacing scattered assertions with a single definition shared by both sides.

A contract binds request and response schemas to an HTTP endpoint:

from surety import Dictionary, String, Int
from surety.api import ApiContract, HttpMethod

class CreateOrderRequest(Dictionary):
    ProductId = Int(name='product_id')
    Quantity = Int(name='quantity', min_val=1, max_val=100)

class OrderResponse(Dictionary):
    OrderId = Int(name='order_id')
    Status = String(name='status', default='pending')

class CreateOrder(ApiContract):
    method = HttpMethod.POST
    url = '/api/v2/orders'
    req_body = CreateOrderRequest
    resp_body = OrderResponse

The provider uses the same contract to verify that the implementation responds correctly:

# Provider test — verify the endpoint returns the expected shape
CreateOrder().call()
CreateOrder().verify_response()

The consumer uses it to set up a mock and verify their code handles the response:

# Consumer test — mock the provider, verify consumer behaviour
CreateOrder.reply(status=201)
# ... trigger the consumer logic ...
CreateOrder.verify_called()

Both sides reference the same CreateOrder contract from the shared package. When the contract changes, both tests are immediately affected — there is no drift.

Surety supports API testing via surety-api, database interaction via surety-db, and structured comparison via surety-diff. Full documentation is available at surety.readthedocs.io.

---

What Contract Testing Won’t Do For You

Contract testing is powerful, but it comes with two honest caveats.

It requires discipline. The workflow only works when development happens on branches, contracts evolve with the code, and merging without passing tests is not an option. The temptation to bypass is real: a deadline is close, the failing test looks unrelated, someone argues the change is “just a small tweak.” Each exception creates a gap between what the contract says and what the system actually does. Over time, those gaps accumulate into exactly the kind of silent incompatibilities contract testing was meant to prevent. The process only has value when it is non-negotiable.

It verifies structure, not behavior. A response can match the schema perfectly and still contain logically incorrect data that causes the consumer to break. Contracts do not cover what the provider does internally — only what it returns. Teams that need deeper guarantees sometimes layer state-aware validation on top: modeling the system states that produce certain data, and verifying that state transitions behave as expected. This sits closer to model-based testing, and it complements rather than replaces contract testing.

Neither caveat undermines the approach. They define its scope. Combined with testing on feature branches, contract testing moves integration confidence to where it belongs — before the code reaches production, not after.