Reading from an authoritative source¶

Use the record Defra already holds, by reference, instead of keeping your own copy that slowly drifts out of date.

Context¶

Defra services keep asking for the same things: who the customer is, which business they work for, which land parcels and holdings they manage, where something is, which species is involved. Each of these has an authoritative source - see Defra on a page.

When services copy this data into their own databases, the copies drift. Users have to tell Defra the same thing many times, and decisions get made on out-of-date information.

Solution¶

Store the identifier of the shared record, not the record itself. Read the current data from the authoritative source's API when you need it. Where you must keep a copy - for speed, resilience or a legal record of what the decision was based on - keep it deliberately: record where it came from, when, and how it is refreshed.

flowchart LR
    accTitle: Reading from an authoritative source
    accDescr: The service stores only identifiers of shared records, such as a customer or land parcel id. When it needs current data it calls the authoritative source's API. A short-lived cache may hold recent answers. If the service must keep a copy, for example a snapshot of what a decision was based on, it records the source, the time and how it is refreshed. Changes in the authoritative source can be published as events that the service subscribes to.
    SVC["Service"] -->|"store ids only"| DB[("Service data")]
    SVC -->|"read current data"| API["Authoritative source API"]
    API --- AS[("Authoritative<br/>source")]
    SVC -.-> CACHE["Short-lived cache"]
    AS -.->|"change events"| SVC
    SVC -->|"snapshot when needed:<br/>source, time, refresh"| SNAP[("Decision snapshot")]

How it works:

Find the authoritative source for each shared entity in Defra on a page, and the identifiers it uses in the data standards.
Store those identifiers. Do not create your own ids for things that already have one.
Read through the source's documented API, authenticated as your service.
Cache only for a short time, and design for the source being unavailable: show what you can, and let users carry on where it is safe to.
If a decision depends on the data at a point in time - such as the land area a payment was calculated on - keep a snapshot with the decision, saying where and when it came from.
If the source publishes change events, subscribe, rather than polling.

What users see¶

To be confirmed

TODO: what users see when data from the authoritative source is out of date or unavailable, the content for those situations, and what to test with users. Write these three sections with a designer and a user researcher.

Content to design¶

To be written - see the box above.

What to test with users¶

To be written - see the box above.

Guardrails it helps you meet¶

Guardrail	Level	What the guardrail asks
GR-DATA-02 Use authoritative sources	Should	Use the authoritative source for shared entities - customers, organisations, land parcels, holdings, locations, species - rather than creating local copies that drift. See Defra on a page.
GR-DATA-03 Use agreed data standards and identifiers	Should	Use the data standards for dates, addresses, locations, identifiers and code lists, so data can be joined across services.
GR-DATA-04 Collect once, share safely	Should	Do not ask users for information Defra already holds. Share data between services through APIs or governed data products, with data sharing agreements where required.
GR-API-05 No integration through shared databases	Should	Services do not read or write another service's database directly. Integrate through APIs, events or governed data products.
GR-API-07 Secure every API	Must	All APIs authenticate callers (OAuth 2.0 / OpenID Connect or mutual TLS), authorise every request, validate input and apply rate limiting. No API is "internal so it's safe".
GR-OPS-04 Health checks and graceful degradation	Should	Expose health endpoints, set timeouts and retries on dependencies, and degrade gracefully (for example save progress and tell the user) when a dependency fails.

When not to use it¶

There is no authoritative source yet. Talk to the enterprise data architecture team. You may become the source, which brings duties under GR-DATA-01.
Analytical use at scale. Use governed data products on the data platform, not high-volume calls to an operational API. See the data and analytics service pattern.