← All perspectives

Perspective / Data & platform engineering

Data contracts that protect commercial decisions: identity, quality and business meaning

A technically valid dataset can still produce the wrong commercial answer. Explore executable data contracts that define customer identity, record grain, quality rules and failure handling, supported by an illustrative customer-and-order model.

10 min readReviewed

Aligned data connectors with precision gold contacts ready to connect.
Conceptual editorial image.

Data contracts are useful when they connect business meaning to enforceable technical checks. A list of column names and types is insufficient if teams disagree about what a customer represents, when revenue is recognised or whether a row describes an order or an order line. For executives funding reporting, integration or AI initiatives, the objective is dependable decision-making. A practical contract makes assumptions explicit, identifies the evidence required to trust an output and defines what happens when that evidence fails.

The executive view

  • Define the entity, record grain and business meaning before combining datasets or automating decisions.
  • Validate contract structure and production data separately, with executable checks and owned responses.
  • Evaluate identity matching by false merges and missed matches, and version semantic changes explicitly.
01

Define the commercial decision and data grain

Consider an illustrative commercial platform that combines customer records and orders from several operating systems. Finance asks for revenue by customer; account management wants an account hierarchy; operations needs the destination of each delivery. Those questions can require different entities and different levels of detail.

Write down the decision before designing a combined table. Define whether customer means a person, contracting organisation, billing account or delivery location. Define whether revenue includes tax, refunds, cancellations and currency conversion. State the time basis used for each calculation.

Then identify record grain: one row per order, order line, invoice or adjustment. A technically correct join between two tables can multiply values if their grains differ. Recording grain alongside keys and relationships gives reviewers a concrete basis for challenging a transformation before its results enter a board report or automated workflow.

A useful discovery session traces one executive number backwards. Ask which operational action creates it, which corrections can change it and which team can explain an exception. If a sales dashboard counts booked orders while finance reports posted invoices, neither definition is inherently defective. The defect is using one as evidence for a question answered by the other. Record the intended decision and prohibited interpretations in the contract. This also helps an AI analyst: the presence of a column called revenue should never be the sole reason it selects that measure for a board-level question.

02

Validate the data contract and its delivered dataset separately

Bitol’s September 2026 ODCS release extends its open data contract format with richer semantics, context and lifecycle metadata. That gives teams more expressive ways to describe intended meaning. Description still needs implementation: a schema-valid contract does not establish that the data behind it satisfies the declared requirements.

Separate three questions. Does the contract itself follow its chosen format? Does the delivered dataset match its declared structure? Do its records satisfy the business rules required for use? Each question needs different evidence.

For the illustrative model, make this distinction visible in delivery status. A dataset might pass structural validation while failing a reconciliation check. A consumer should not receive a single green badge that hides that difference. Define which checks are mandatory, their scope and the action that follows a failure.

For implementation, attach evidence to a dataset version or processing run, rather than only to the current contract. Store the contract revision, producer build, extraction interval, test results and publication decision together. A consumer investigating last month's report needs to know what passed at that time, not what passes against today's revised rules. Distinguish a warning from a waiver: a warning follows an agreed policy, whereas a waiver is an explicit exception with an owner, reason and expiry. Repeated waivers indicate a definition or delivery problem that needs investment, even when no individual run is blocked.

03

Make identity rules explicit and inspectable

Use stable internal identifiers while retaining source-system references. A record linking two source IDs should state why they were joined and how the decision can be reviewed. An email address can be an attribute used in matching, but shared addresses, changes and recycled accounts make it a poor universal definition of identity.

An illustrative identity contract could contain these elements:

ElementMeaning
Canonical entity IDStable reference within the agreed entity model
Source ID and source systemTraceable origin of each contributing record
Entity typePerson, organisation, account or location
Match method and rule versionReproducible explanation of the relationship
Review stateProposed, accepted, disputed or superseded
Effective intervalPeriod during which a relationship is valid

Keep identity resolution distinct from access control. Concluding that records refer to the same entity does not authorise every consumer to see all their attributes.

Identity changes also need reversibility. Suppose two trading accounts are merged because they share a telephone number, then later evidence shows that one number belongs to a switchboard serving unrelated companies. Retain the original source records and relationship history so the merge can be split without inventing missing provenance. Decide which downstream aggregates must be restated and whether already-issued communications require review. Never make a canonical ID encode a mutable business attribute. Stable references, separate relationships and effective dates allow the entity model to evolve while preserving the explanation of earlier decisions.

04

Turn quality expectations into executable assertions

The first useful checks are often precise and inexpensive: required identifiers, uniqueness at the declared grain, valid status values and relationships to existing entities. dbt’s data testing documentation illustrates assertions that return failing rows, including these common integrity checks.

Add domain rules that protect the intended decision. For an order model, test whether line totals reconcile to the order value under the documented rounding policy. Check that cancelled and refunded amounts are treated consistently. Evaluate currency and date assumptions before combining values.

Every test needs an operational response. A small number of missing optional descriptions may produce a warning; orphaned financial records might block publication. Set those actions deliberately. Store enough failure evidence for diagnosis without creating an uncontrolled secondary copy of sensitive data. A test that nobody owns is a recurring notification, rather than a control over data use.

Set tolerances in business units. A rule allowing one per cent of missing order values can hide a serious loss if the missing records contain the largest contracts. Evaluate both record count and financial exposure, segmented by product or source where that affects interpretation. Critical tests should fail on intentionally corrupted fixtures before launch. Include duplicated joins, late refunds, null identifiers, unexpected currencies and invalid effective intervals. A dataset can pass every declared rule and still be unsuitable for a new use, so require consumers to declare materially different decisions rather than silently expanding the original assurance boundary.

05

Evaluate matching by the cost of being wrong

Matching more records is not automatically an improvement. A false merge can combine distinct customers; a missed match can fragment a genuine account. Their costs differ by use case. An outreach workflow, financial report and access decision should not inherit the same tolerance simply because they share a matching service.

AWS added transitive matching support to Entity Resolution in April 2026. Its technical explanation describes how records can connect groups across rule levels. That capability makes the design of matching rules and evaluation examples especially important; connectivity is not independent proof of real-world identity.

Create a labelled evaluation set with straightforward matches, difficult non-matches, shared contact details and changed attributes. Measure false merges and missed matches separately. Preserve examples that caused previous errors. Changes to a match rule should be evaluated against that evidence before existing identity relationships are rewritten.

For a hypothetical evaluation, 1,000 proposed links contain 970 genuine matches and 30 false merges. That is 97 per cent precision for those proposals; it says nothing about recall until the total set of true links is known. If the labelled population contains 1,100 true links, recall is about 88.2 per cent. A rule change that finds twenty additional true links but creates fifty false merges might improve coverage while making consolidated billing unacceptable. Evaluate difficult groups separately and investigate how labels were established. An apparently precise score can conceal uncertainty in the reference set itself.

06

Version meaning alongside structure

A field can keep its name and type while its meaning changes. For example, an active account might move from a contractual definition to a recent-usage definition. A schema comparison will not reveal the impact on a retention report.

Treat definitions, transformation rules and identity policies as versioned dependencies. Record the consumers affected by a proposed change, how historical data will be interpreted and whether a transition period is required. A deprecation flag is communication; a removal plan also needs dates, ownership and evidence of consumer readiness.

In the illustrative platform, retain the previous definition while a new measure is evaluated. Run both against the same period and explain material differences. The purpose is not indefinite duplication. It is to make a change in reported performance distinguishable from a change in measurement. That distinction matters when the result influences investment, targets or customer treatment.

Use compatibility reviews that include semantic scenarios. If a producer changes a timestamp from local time to UTC, a type-compatible release may move orders between reporting days. If a status gains a new value, a consumer's default branch may classify it incorrectly. Keep representative queries and expected results alongside the contract, including boundary dates and reversals. Run consumers against proposed revisions before removing old semantics. When a definition changes deliberately, publish the interpretation and effective period so reporting teams can explain discontinuities rather than quietly rewriting historical performance.

Retained snapshots also need an access and deletion policy. A reporting reproducibility requirement does not justify copying every identifying attribute indefinitely. Separate the lineage and aggregate evidence needed to explain a result from personal details needed only for a particular operation. Record when historical datasets expire and what happens when corrections or permitted deletions affect a previously published version. A consumer must be able to distinguish a reproducible business calculation from an unrestricted archive of source records. Contract ownership therefore includes the lifecycle of evidence, not only its arrival at the quality gate.

07

Worked example: reconcile order grain before publishing revenue

Suppose an illustrative dataset contains 1,000 orders at an average net value of £120, giving £120,000 before later adjustments. Each order has exactly three lines. Joining the order header to its lines and summing the header value produces £360,000 even though the SQL executes successfully and every identifier is populated. A contract that only checks types and non-null values will approve the wrong answer.

Protect the model with an explicit grain assertion and a reconciliation to the authoritative order population. If line amounts are the intended basis, sum those amounts once and compare them with headers under the agreed tax, discount and rounding rules. If headers are the intended basis, aggregate line-level attributes before joining or keep separate measures at their declared grains. A query should make its denominator inspectable.

Now add a refund of £600 received after the initial reporting cutoff. Decide whether the original period is restated or the adjustment belongs to the later period. Preserve both the occurrence and processing dates so the policy is executable. The acceptance evidence should demonstrate the expected value before and after the adjustment, and explain the handling of an unavailable refund feed. The example is deliberately small: its value is that finance and engineering can independently predict the correct result and challenge the implementation before the same error affects a much larger portfolio.

08

Operate a small contract before expanding the programme

Begin with one dataset that supports a consequential decision. Agree its meaning, identity model and critical checks with the people who use it. Deliver a contract, a test report and a defined response to exceptions. Use an intentionally flawed fixture to demonstrate that failures are caught and handled.

An initial operating review should ask whether the checks find meaningful defects, whether failures are resolved and whether consumers understand the remaining limitations. Measure the time spent investigating ambiguity as well as the number of passing tests. Increasing test volume without improving decisions adds maintenance rather than assurance.

The executive value is a clearer chain from definition to data to decision. A useful data contract makes that chain inspectable, while its runtime checks and operating responsibilities make it actionable. It also exposes where judgement remains necessary, so a technically polished output is not mistaken for an unquestionable account of the business.

Assign one party to accept the dataset for its declared use, while retaining producer responsibility for delivery. Acceptance should include evidence that a failed critical rule prevents consumption, that a corrected run can replace an earlier version and that consumers can locate the applicable definition. Test an unavailable upstream source as well as malformed rows: missing delivery is a contract failure too. The operating review should distinguish defects fixed at source from defects repeatedly patched downstream. That distinction reveals whether the programme is reducing ambiguity or simply funding a more sophisticated repair team.

Further questions

Questions this raises

How is a data contract different from a database schema?

A schema describes structure. A useful data contract also documents meaning, expected quality, relationships, responsibilities and service expectations. Runtime checks are still required to verify delivered data.

Can a data contract prevent every incorrect report?

No. It makes assumptions and checks explicit, but coverage, implementation and interpretation can still fail. Reconciliation, review and representative evaluation remain necessary.

Why does customer identity require its own rules?

Different systems may describe people, organisations, billing accounts and locations as customers. Explicit identity and matching rules prevent those concepts being combined without a defensible explanation.

References

Sources & further reading

  1. Bitol: Data Contracts & Data Products for AI 8 September 2026
  2. dbt: Add data tests to your DAG
  3. AWS Entity Resolution: Using transitive matching
  4. AWS Entity Resolution documentation history Transitive matching release: 27 April 2026

Connected capabilities

Software & CRMStrategy & digital transformationAI solutions

From thinking to doing

What could this mean
for your business?

Your next chapter

What could your
business do better?

Tell us where you want to go, what’s getting in the way, or what you need to deliver. We’ll start there.

Start a conversation

Start a conversation

Tell us what
you have in mind.

Interactive preview. Your information stays in this page and is not sent.