Factor #2: Flexibility - Assert What Matters, Ignore What Doesn't

Good tests focus on the contract, not the implementation details. When your API evolves with new fields or restructured data, your tests should continue passing as long as the essential behavior remains unchanged.

"Is there a subset of the json that my service needs to do its job or am I just trying to catch someone making an API change at all? I think for an external entity just asserting on the fields I care about is less brittle." —Paul Thomson on flexible API testing

The Problem: Over-Specific Assertions

The most common mistake in integration testing is asserting on the exact structure of API responses. When you use expect(response.body).toEqual({...}) with a complete object, you're saying "this response must match exactly, forever." But APIs evolve. Teams add new fields, include metadata, or restructure responses for performance.

Your test shouldn't care if the API team adds a last_login field or includescache_metadata in the response. It should only care about the fields your code actually uses. This is the difference between testing the contract (what you depend on) versus testing the implementation (how the data happens to be structured today).

The Core Principle
Bad Example
// ❌ BAD: Exact matching breaks when API evolves
expect(response.body).toEqual({
  id: 123,
  name: 'Alice',
  email: 'alice@example.com'
});
Good Example
// ✅ GOOD: Assert only what you actually need
expect(response.body).toEqual(expect.objectContaining({
  id: 123,
  name: 'Alice',
  email: 'alice@example.com'
}));

The second approach passes when the API adds new fields

Common Over-Specification Traps

Beyond exact object matching, there are several other ways tests become overly brittle:

Exact Array Lengths
expect(results.length).toBe(12) — Why exactly 12? What happens when inventory changes?

Instead, test the constraints that matter: expect(results.length).toBeGreaterThan(0)and expect(results.length).toBeLessThanOrEqual(pageSize)

Timestamp Precision
expect(user.created_at).toBe(exactTime) — Millisecond differences will break this

Instead, test reasonableness: expectRecentTimestamp(user.created_at)or verify it's within a reasonable range.

Order Dependencies
Assuming specific ordering unless order is part of the business requirement

Test that important items are present, not their exact positions. If order matters for business reasons, test that specifically.

The Right Way: Focus on Your Contract

The key insight is to test what your service depends on, not what the API happens to return today. Think of it like a contract: you're asserting that the other party will provide certain guarantees, but you don't care about the extra details they might include.

Use Partial Matching
Assert on specific fields while ignoring extras
expect(order).toEqual(expect.objectContaining({
  id: expect.any(Number),
  customerId: 123,
  status: 'pending'
}));
Create Contract Validators
Reusable functions that validate the shape and types of your data
function expectValidUser(user) {
  expect(user.id).toEqual(expect.any(Number));
  expect(user.email).toMatch(/^[^\s@]+@[^\s@]+\.[^\s@]+$/);
  expect(user.name.length).toBeGreaterThan(0);
}
Test Business Rules, Not Structure
Focus on the behavior that matters to users
// Don't care how shipping is calculated internally
expect(result.shippingCost).toBe(0);
expect(result.reason).toContain('premium');

When to Be Specific vs Flexible

Be Specific About:
Fields your service actually depends on
  • Business rules: Discount calculations, pricing logic, validation rules
  • Security constraints: Authentication requirements, permission checks
  • User experience: Error messages, status codes, required fields
  • Data your code uses: Fields that your service reads, processes, or displays
  • Contract guarantees: Required fields, data types, value constraints
Be Flexible About:
Implementation details and additive changes
  • Extra fields: New optional fields, metadata, internal IDs that you don't use
  • Ordering: Unless order is part of the business requirement
  • Exact counts: Unless the count is a business constraint
  • Timestamps: Exact precision, timezone representation
  • Internal structure: How data is organized internally
  • Additive changes: New fields shouldn't break existing functionality (semver-style)

Practical Techniques

Here are the most effective approaches for writing flexible assertions:

Use Schema Validation Libraries
Tools like Zod, Joi, or Yup handle structure validation and allow extra fields by default

Define the shape your service expects, then let the schema library validate it. Most schema validators ignore unknown fields, giving you flexibility automatically.

Create Contract Helpers
Reusable functions that validate the essential shape of your data

Write expectValidUser(), expectValidOrder() functions that check required fields and types. Use these across tests instead of inline assertions.

The Semver Mindset
Treat API testing like semantic versioning: additive changes shouldn't break tests

Just like adding new fields to an API is a minor version bump (not breaking), your tests should pass when new fields are added. Focus on the contract you depend on.

Further Discussion

This principle sparked a great discussion on X/Twitter about the balance between catching API changes and maintaining test flexibility. You can read the full conversation here, where developers discuss the semver approach to API testing and when to be strict vs flexible.