Contract Testing vs Integration Testing: Where Each One Fits

A contract test checks one side of an integration against a shared description of a message. An integration test runs two or more implementations together and watches the result. Schema validation checks a message against a schema. The three are easy to collapse into “we tested the API,” and they fail in different places. This note uses Pact’s published description of contract testing, then says where a joined run is still doing different work. It does not claim that one technique finds more bugs. This desk has no such count.

What Pact means by a contract test

Pact’s documentation defines contract testing as checking each application in isolation so the messages it sends or receives match a shared understanding written down as a contract. For HTTP, the messages are the request and the response. For a queue, the message is the one on the queue.

Pact is consumer-driven in a specific sense: the contract is generated when the consumer tests run. The consumer registers the request it expects to send and the minimal response it needs. A mock provider checks that request and returns that response. The consumer test fails if the consumer does not send the expected request or does not understand the response. On the provider, verification replays each request and checks that the real response contains at least what the consumer asked for. The two sides do not have to be deployed together for that pair of checks.

The documentation draws a line that matters here. A schema or an OpenAPI document describes possible states of a resource. A Pact interaction is one concrete example. Only the parts the consumer uses are fixed. Provider behavior no current consumer uses can change without failing those tests. That is a feature when the extra fields are free to evolve, and a gap when a field is important and no consumer has listed it yet.

The same documentation uses “provider contract testing” for a different technique: checking a provider against its own documented contract, such as an OpenAPI document, without a consumer in the loop. That keeps the provider and its document aligned. It does not show that any consumer calls the provider correctly. Both uses of the phrase “contract test” appear in the literature. Say which one you mean.

What each check can show

The questions are editorial distinctions drawn from the Pact documentation and from the schema-validation note. They are not pass rates.

Schema checks and the gaps they leave explains why a valid body can still be the wrong effect. A contract test as Pact defines it gets closer to “the consumer can use the message,” because the expected response is the one the consumer code actually reads. It still does not start the consumer and the provider in one network namespace unless you add that run yourself.

Where a joined run still earns its cost

Run systems together when the risk lives in the seam the message schema does not describe:

• Authentication and network policy between the two deployments.
• A transaction that must commit in the provider and become visible to a later read from the consumer.
• Timeouts, retries, and duplicate delivery. Event payloads that deserve their own contract can be contract-tested as messages. The decision to acknowledge before the slow work is a behavior you only see when a retry happens.
• An operational rollback that depends on both processes being on stated versions.

Those runs are slower and more brittle. Pact’s introduction describes integration tests that way, as the expensive alternative when you have no contract test. The editorial conclusion this desk draws is narrower than “never write an integration test.” Use the joined run for the seam risks above. Do not use it as the only proof that a renamed field is still present. A contract test fails faster and points at the interaction.

Compatibility rules a test can pin down belong in the contract examples: an old request without a new field, a response that includes a new field the consumer ignores, an enum value the consumer must tolerate. If those examples are missing, the compatibility note is prose with no check.

Which test to run

This list is a decision aid. It is not a coverage metric.

1. If the question is “does this JSON match the published schema,” validate. Do it in CI against the artifact clients use.
2. If the question is “does this consumer still understand this provider,” add or update a consumer-driven contract test for the interactions that consumer uses. Include messages on a queue if that is the transport.
3. If the question is “does the provider match its OpenAPI document,” that is the other kind of contract check. Keep the name straight so nobody deletes it thinking the consumer tests replaced it.
4. If the question is “does the effect happen when both sides run, including auth and retries,” keep a joined scenario. Make the scenario assert the effect, not the entire JSON tree.
5. When a contract test and a joined test disagree, believe the one that matches the question. A green end-to-end scenario that never sends the old request does not prove compatibility.
6. Revisit the interactions when a client moves. What to retest while clients move is the migration side of the same list.

Collaboration is part of the definition

Pact’s documentation warns that “consumer-driven” does not mean the consumer team dictates a contract and throws it over a wall. The provider verification fails when the consumer asks for something the provider does not do, even if the consumer is the side that is wrong. A contract test is evidence of a conversation that was written down. It is not a substitute for agreeing the change. Use it that way, and it will show a break before a shared environment does. Use it as a surprise at the end of a release, and it will only show that the conversation did not happen.