Building a Data Contract System
That Teams Actually Follow

01 · Why Contract Programs Die in Quarter Three

The lifecycle is so consistent you can schedule it. Q1: platform team ships contract tooling, mandates specs on every table, demo goes well. Q2: producers treat the YAML as paperwork; violations alert a channel nobody triages; the data team quietly fixes breaches itself to keep dashboards alive. Q3: a reorg moves one sponsor, the checks get muted "temporarily," and the program joins the wiki of initiatives past.

Each failure has the same root: the contract described an obligation nobody had agreed to bear. Mandated contracts create checkbox compliance; alerts routed to the data team make breaches the victim's problem; consequences improvised per incident make enforcement political. The fixes are cultural, and they are Section 06 — but the tooling must be in place first, because culture without enforcement is just a memo.

02 · The Architecture: Contract as Code, Enforcement as Pipeline

03 · The Violation Data Flow: From Breach to Fix

The flow's two non-obvious choices: code-path and data-path violations are different events with different audiences (a blocked merge teaches at review time; a post-load anomaly demands triage), and the alert payload always names the affected consumers — abstract breaches get deprioritised, but "you broke finance_mart and ml_churn" has a constituency.

04 · Layer 1–2: Schema, Then Everything Schema Misses

the contract spec — one file, both layers (YAML)
contract: orders_v2
owner: team-payments            # a Slack handle, never a person
tier: 1                         # breach SLA: same-day
consumers: [finance_mart, exec_dashboard, ml_churn]
schema:                         # → Layer 1, enforced in CI via dbt
  model: fct_orders
  contract: {enforced: true}
  columns:
    - {name: order_id,    type: string,  tests: [not_null, unique]}
    - {name: order_total, type: numeric(18,2),
       tests: [{accepted_range: {min: 0}}]}
    - {name: currency,    type: string,
       tests: [{accepted_values: [USD, EUR, GBP, INR]}]}
semantics:                      # the clauses schema validators can't check
  order_total: "gross, includes tax, transaction currency"
  grain: one row per order line
behaviour:                      # → Layer 2, post-load via GE/Soda
  freshness: {landed_by: "06:00 Europe/Dublin"}
  volume:    {min_rows_daily: 1.8M, vs: trailing-28d-dow}
  null_rate: {customer_id: "< 0.5%"}

Layer 1 runs in CI on every PR — shape breaks are caught before they ship, when the fix costs a code review comment. Layer 2 runs post-load against learned baselines, because schema tests pass while the data goes wrong: volume collapses, 100× drifts, staleness, and null-creep are behavioural failures only baselines catch. The semantics block is the most-skipped and highest-value section — most expensive breaches are two teams agreeing on a column name while disagreeing about what the number means.

05 · Layer 3: Routing — Violations Reach the Producer

The single most important configuration in the entire system: violations page the producing team, not the data team. Every contract declares an owner — a team handle that survives personnel churn — and the alert lands in that team's channel with the contract link, the expected-vs-observed diff, and the named consumers affected. The data platform team is cc'd for awareness, never assigned for resolution.

This one routing decision encodes the entire philosophy: the contract is an obligation the producer accepted toward named consumers, and the breach is theirs to fix. Estates that route violations to the data team have built an expensive monitoring system for the data team's own anxiety — producers never feel the feedback, behaviour never changes, and the program dies in the usual quarter.

06 · The Cultural Mechanics: The Actual Product

1 — Contracts are created at the consumer's request, never imposed. Platform-mandated contracts on every table produce resentment and checkbox YAML. The working pattern: when a consumer — finance, ML, an exec dashboard — depends on a dataset, they request the contract, and the negotiation (columns, freshness, breach behaviour) is a 30-minute meeting between the two teams. The contract documents a relationship that already exists; that is why it gets honored. Coverage grows organically along the lines where breakage actually costs money — which is exactly where it should.

2 — Breaches have a pre-agreed, boring consequence. Not punishment: process. A breached contract auto-creates a ticket on the producer's board with an SLA matched to the tier (Tier 1 same-day, Tier 2 this-sprint), and the escalation path is agreed when the contract is signed — not improvised mid-incident, when it becomes political. Boring is the design goal: the consequence machine runs identically for every team, including the loud ones, and the quarterly breach ledger replaces blame meetings with a trend line.

3 — Contracts are code, reviewed where the producer works. The spec lives in the producer's repo, versioned in git, changed via PR with consumers as reviewers. A contract change is a negotiation made visible: the producer proposes, affected consumers approve, CI enforces from merge. The artifact teams actually maintain is the one in their own workflow — external contract registries become stale wikis with API access.

07 · Production Evidence: Contracts in Regulated Estates

This is the mechanism — visibility, ownership, automated consequence — behind Vipra's Fortune 500 governance engagement, which cut reconciliation effort 40% and enabled self-service BI without governance chaos. The same contract pattern carries our clinical quality metrics (FHIR profiles and value sets as contract clauses, CI-blocked on violation), the supplier mesh (federated governance enforced by contract validation rather than committee), and the inventory reconciliation playbook (where contract semantics blocks — quantity meaning, units, timezones — caught more money than every type check combined). One pattern, four industries, because the failure it fixes — producers and consumers disagreeing silently — is universal.

08 · Lessons Learned: The Hard Truths

• Routing is destiny. Every dead contract program we've audited alerted the data team; every living one paged producers. Decide nothing else first.
• Mandates produce YAML, requests produce contracts. The platform-mandated rollout generated 200 specs and zero behaviour change. The consumer-requested restart generated 30 contracts that all still hold.
• Semantics clauses pay the bills. Type checks catch typos; the "gross, includes tax, transaction currency" line catches the six-figure misunderstanding. Write the English, not just the types.
• Consequences must be boring before the first breach. The first improvised escalation becomes precedent and politics simultaneously. Pre-agree the ticket-SLA-escalation machine while everyone is still friends.
• The data team must refuse to quietly fix breaches. Every silent fix teaches producers that violations are someone else's problem. It feels unhelpful for a month and transforms the estate within two quarters.
• The breach ledger ends the meetings. Breaches-created and MTTR, trending by team, replaced the quarterly blame ritual with a two-line review. Evidence beats anecdotes, especially about people.

09 · Key Takeaways for Practitioners

The applied versions: inventory reconciliation (dollar-denominated), clinical quality metrics (regulated), supplier federation (cross-border). The documented result: Fortune 500 governance, 40% reconciliation cut.

FAQ · Frequently Asked Questions