What is a data contract?

Your sales dashboard broke again, and the warehouse was fine. A team upstream renamed customer_id to cust_id, made a timestamp column wider, and started sending empty values in a field that marketing still treats as required. Everyone acted in good faith, yet nobody ever wrote down what a good feed looks like. That missing written agreement is what people now call a data contract.

This deep dive explains data contracts in plain language for analysts, the engineers who prepare data for them, and anyone who owns a metric. It covers what belongs in a contract and what does not. You will also see a filled example and learn how contracts connect to data quality, pipelines, and stewardship.

What a data contract is

A data contract is a written agreement between the team that makes a dataset and the teams that use it. It has a version number, so everyone knows which copy applies. The agreement covers the structure of the data, what each field means and how clean it must be. It also says when the data arrives, who owns it and how breaking changes are announced. A file that lists column names and nothing else, with no owners and no change rules, is only a partial contract.

Rule of thumb: Suppose a producer can change a field tomorrow and consumers only find out when a dashboard breaks. Then you do not have a contract. You have hope.

Contracts caught on among analytics engineers and platform teams because warehouses made it easy to share tables and hard to share promises about them. A contract is not magic. It is the same idea as the agreed shape of a software interface, applied to data, so both sides know what to expect.

Why contracts became popular

Older enterprise systems came with heavy interface specifications. Modern analytics often went the other way. Teams dumped tables into a data lake (a big shared storage area), wrote the documents later and fixed problems as they appeared. That works until one of these things happens:

• Multiple consumers depend on the same feed for money, compliance, or exec metrics.
• Producers ship product changes weekly and treat analytics as a side effect.
• AI features and reverse ETL make bad fields travel farther and faster (ETL stands for extract, transform, load. Reverse ETL pushes warehouse data back into business tools such as a CRM).
• Answering the question “why is revenue empty?” at odd hours becomes a routine part of someone’s job.

A contract moves the check earlier. It happens before a change is merged, before it ships and before the board meeting, instead of after. It pairs naturally with checks from the data quality series and ownership habits from the data stewardship series.

Producers and consumers: two jobs, one document

Producers generate the data: application services, operational databases, event pipelines, third-party extracts. They want to ship product features without a permanent freeze on changes.

Consumers use the data: analytics, finance, machine learning (ML) features, reverse ETL, other services. They want stable meaning and on-time delivery.

The contract is the record of that negotiation. Producers accept some limits on how they change the data. Consumers accept some leftover risk and the rules for moving to a new version. Neither side gets unlimited freedom.

What goes in a data contract

Formats vary. Some teams write the contract in YAML (a plain text format for settings) or JSON files. Some follow the Open Data Contract Standard. Others use an internal wiki plus a schema registry, which is a service that stores the approved shape of each dataset. Whatever the format, the contents cover the same six areas.

1. Identity and ownership

• Dataset or stream name, domain, environment (prod/stage).
• Producer team and on-call path.
• Primary consumer stewards (not “everyone with a warehouse login”).
• Version number and last review date.

2. Schema

• Field names, types, nullability
• Primary keys and important foreign keys.
• Allowed enums or coded values.
• Compatibility policy: adding a new field is safe, while renaming an old one breaks things.

3. Semantics (meaning)

• What one row or event represents (grain).
• Business definitions for critical fields.
• Timezone and unit conventions (cents vs dollars, UTC vs local).
• Known caveats (late events, partial backfills).

4. Quality expectations

• Freshness targets, such as data available by a set hour each morning.
• Completeness thresholds for required fields.
• Uniqueness of keys
• Acceptable ranges or referential checks.
• What happens when checks fail (page producer, quarantine, continue with flag).

5. Delivery and interface

• Location (table, topic, file path, API).
• Update pattern (batch daily, microbatch, streaming).
• Partitioning and retention
• Security and PII classification

6. Change management

• How consumers are notified
• Deprecation windows for breaking changes.
• Dual-publish periods when renaming fields.
• Who can approve exceptions

If that list feels like a lot, start with identity, schema, grain and two quality checks. Add more when a real problem shows up. A perfect contract that never gets written down helps nobody.

Contracts vs related ideas

Think of the contract as the product requirements for a dataset. Tests and registries are the machinery that carries out those requirements, and a catalog is the storefront where people find the data. Most teams end up using several of these together rather than betting on a single buzzword.

Worked example: orders_v1 contract

Imagine the team that runs your online checkout builds a table called analytics.orders_v1 every night for the finance and growth teams to use.

Here is a readable excerpt of the contract, written in a simplified YAML style:

dataset: analytics.orders_v1
version: 1.4.0
domain: commerce
producer:
  team: checkout-platform
  contact: checkout-oncall
consumers:
  - finance-analytics
  - growth-analytics
grain: One row per order_id for completed checkout attempts that created an order
schema:
  - name: order_id
    type: string
    required: true
    unique: true
  - name: customer_id
    type: string
    required: true
  - name: order_ts_utc
    type: timestamp
    required: true
    description: Order creation time in UTC
  - name: currency
    type: string
    required: true
    allowed: ["USD", "EUR", "GBP"]
  - name: amount_cents
    type: integer
    required: true
    description: Gross merchandise value in minor units before tax
  - name: status
    type: string
    required: true
    allowed: ["paid", "pending", "cancelled", "refunded"]
quality:
  freshness:
    max_delay_hours: 6
    ready_by_local: "06:00 America/New_York"
  checks:
    - type: not_null
      fields: [order_id, customer_id, order_ts_utc, amount_cents]
    - type: unique
      fields: [order_id]
    - type: range
      field: amount_cents
      min: 0
delivery:
  location: warehouse.analytics.orders_v1
  mode: batch_nightly
  pii: customer_id is indirect identifier; join to customers under policy
change_policy:
  additive_fields: allowed with notice in #data-changes
  breaking_changes: 30 day deprecation; dual publish when renaming
  approvals: producer lead + one primary consumer steward

Look at what this contract prevents. Renaming amount_cents without publishing both names for a while counts as a breaking change. Negative amounts fail the range check. Delivering the table at noon without a declared exception breaks the freshness promise. None of that needs a 100-page governance program, only written expectations and a place to enforce some of them automatically.

Enforcement sketch

Producers can validate events before they load them, and consumers can add tests inside the warehouse. These are example SQL checks a consumer could run after each load:

-- uniqueness
SELECT order_id, COUNT(*) AS n
FROM analytics.orders_v1
GROUP BY 1
HAVING COUNT(*) > 1;

-- nulls on required fields
SELECT
  COUNT(*) AS rows,
  COUNT(order_id) AS order_id_nonnull,
  COUNT(customer_id) AS customer_id_nonnull,
  COUNT(amount_cents) AS amount_nonnull
FROM analytics.orders_v1
WHERE order_ts_utc >= CURRENT_DATE - INTERVAL '1 day';

-- freshness proxy: max timestamp should be recent for daily batch
SELECT MAX(order_ts_utc) AS max_order_ts
FROM analytics.orders_v1;

A schema-diff tool compares the old and new shape of a table. It can run in continuous integration (CI), which means the automatic checks that run on every proposed code change. It can block a pull request (PR) that removes customer_id without a version bump. The human part still remains, because someone has to care when a check goes red. A contract with no response path turns into an unread PDF.

How contracts connect to metrics and pipelines

A metric definition such as “net revenue excludes pending orders” sits on top of reliable inputs. If the values in status drift, the metric definition cannot rescue you. A perfect table contract does not define the business metric either. You need both layers.

Each stage of a pipeline should know which of its outputs are covered by a contract. Raw data that just landed can be messier, while the tables finance relies on should be stricter. You are setting rules at the handoff points, not demanding that every raw feed be perfect. For pipeline thinking, see the data pipelines series. For metric definitions, see the metrics series.

When you need a contract (and when you do not)

Strong candidates:

• Shared sources for certified executive metrics.
• Feeds powering money movement, compliance, or customer communications.
• Cross-team interfaces that already break quarterly.
• ML features that silently skew when null rates climb.

Weak candidates (at least at first):

• One-off extracts for a single analyst experiment.
• Throwaway sandbox tables
• Rapid product prototypes with one consumer who sits next to the producer.

Start where outages hurt the most and expand when the pain moves somewhere else. For learning paths around these skills, browse Learn.

Common mistakes

• Schema-only contracts: types without grain and meaning still produce wrong metrics.
• No owners: a YAML file cannot attend the incident call.
• Unbounded consumers: if everyone is a consumer, nobody prioritizes changes.
• Freshness promises nobody can keep: promising 30-second delivery on a job that only runs nightly.
• Breaking changes without dual publish: renames that wipe out historical dashboards.
• Contracts as blame weapons: the goal is fewer surprises, not a courtroom.
• Trying to cover everything: 200 fields polished while the three money fields stay unchecked.
• Ignoring PII: delivery location and access rules belong in the agreement.

Practice: draft a thin contract in 40 minutes

Pick one table or feed that already causes tickets.

1. Write the grain sentence.
2. List the five most important fields with types and nullability.
3. Name producer and one primary consumer steward.
4. Add two quality checks you could run this week.
5. Write a one-paragraph change policy (how breaking changes are announced).
6. Share the draft with the producer team as a conversation starter, not a surprise mandate.

If the producer team pushes back, negotiate the scope. A thinner contract that both sides accept beats a perfect document that sits alone in a shared drive folder.

Quick recap

• A data contract is a versioned producer-consumer agreement on structure, meaning, quality, delivery, and change.
• Schema alone is not enough; grain and semantics carry the business truth.
• Owners and change policy make the document operational.
• Enforce what you can with automatic checks and warehouse tests, and give the rest a clear response path.
• Start with high-pain shared feeds. Expand with evidence, not fashion.
• Contracts complement catalogs, SLAs, and metric definitions; they do not replace them.

Sources

• Open Data Contract Standard (community effort toward portable contract structure): https://bitol-io.github.io/open-data-contract-standard/.
• Apache Avro specification (schema evolution concepts widely used in data interfaces): https://avro.apache.org/docs/current/specification/.
• Confluent Schema Registry documentation (compatibility modes for evolving schemas): https://docs.confluent.io/platform/current/schema-registry/index.html.
• dbt Labs, data tests documentation (enforcement patterns in analytics engineering): https://docs.getdbt.com/docs/build/data-tests.
• Data Management Association International (DAMA), Data Management Body of Knowledge (DMBOK) resources (governance and stewardship context for agreements): https://www.dama.org/cpages/body-of-knowledge.
• Google SRE Workbook / SRE Book material on SLOs (useful analogy for freshness and reliability targets): https://sre.google/books/.

More definitions in Key Terms →

Merch on Etsy

Open store →

You're On Mute Coffee Mug – Data Science Gift View on Etsy → Git Version Control Shirt View on Etsy → Apache Hadoop T-Shirt View on Etsy → You're On Mute Coffee Mug View on Etsy →