Artificial intelligence¶
AI can help Defra do more with less - from classifying species in images to drafting responses. These guardrails help teams use it safely, lawfully and transparently.
Doctrine: 4. Data is an enterprise asset, 5. Assume AI until proven otherwise
Principles: 7. Empower to innovate, 4. Clean data, clear decisions
Who these guardrails apply to
The core department. Whether the artificial intelligence guardrails also apply to Defra's arm's length bodies is still to be confirmed.
Applies the DDTS doctrine assume AI until proven otherwise, and builds on the AI Playbook for the UK Government.
For practical guidance - which tools teams use, what data you can put into them, keeping data safe, working with AI agents and reporting an AI incident - use the AI digital toolkit in the Defra Digital Service Manual, run by the AI Capability and Enablement (AICE) team. Contact details for AICE are on the toolkit's home page. These guardrails set the architecture boundaries; the toolkit tells you how. Technology capability Artificial intelligence and machine learning.
GR-AI-01 Consider AI first¶
Should When designing a service or process, actively explore whether AI can improve quality, productivity, user experience or outcomes before choosing a traditional approach, and record the reasoning either way.
Why: the DDTS doctrine says assume AI until proven otherwise. That does not mean AI everywhere; it means deliberately considering the opportunity before dismissing it.
How to meet it: in discovery and alpha, look for repetitive tasks, triage, summarising, classification or decision support that AI could help with. Note in your ADR what you considered and why you did or did not use AI.
In the AI digital toolkit: start with check if AI is right for your idea, five questions about the problem, users, data, benefits and what you have tried.
GR-AI-02 Use approved AI services and tenancies¶
Must Services you build run their AI in Defra's cloud tenancies or under approved enterprise agreements. When anyone uses an AI tool, they put Defra data into it only as the using data with AI rules in the AI digital toolkit allow - for example, never OFFICIAL-SENSITIVE or personal data in a public consumer tool.
In the AI digital toolkit: choosing a tool and keeping data safe.
GR-AI-03 Keep a human accountable¶
Must Decisions with legal or significant effects on people or organisations - such as licensing, enforcement or payments - have meaningful human oversight, and the ability to challenge.
GR-AI-04 Be transparent¶
Must Publish an Algorithmic Transparency Recording Standard record for algorithmic tools that meet its scope, and tell users when they are interacting with AI.
GR-AI-05 Evaluate, monitor and threat model¶
Should Evaluate models for accuracy, bias and safety before release, monitor them in live, and include AI-specific threats (prompt injection, data leakage, model abuse) in your threat model.
Cross-government AI tools
OCTO's AI technology enablement resources include an AI risk management toolkit, an AI assurance questionnaire, the public sector AI governance operating model and a service assessment questions navigator. Use them to evidence GR-AI-03, GR-AI-04 and GR-AI-05.
GR-AI-06 Talk to the TDA about novel use¶
Should Novel uses of AI, and any use of generative AI in decision making, are reviewed by the Technical Design Authority.
GR-AI-07 Suppliers use AI coding assistants openly and safely¶
Should Delivery partners who use AI coding assistants on Defra work agree which tools they use with Defra, use them only under terms that meet GR-AI-02, have a person review every change before it is merged, and say in pull requests or delivery reports where AI did significant work.
Why: AI coding assistants can speed up delivery, but they can also leak code, secrets or data to a third party, bring in code with unclear licences, and produce plausible but wrong code. Defra owns the code it pays for and must be able to trust it.
How to meet it:
- Follow choosing a tool and using data with AI in the AI digital toolkit: what matters is the data you put in, and privacy settings must be on.
- Tell the Defra engagement lead which tools you use at mobilisation, and record them in the repository.
- Follow AI security in the AI digital toolkit: AI-written code clears the same Defra security gates as any other code.
- Never put secrets into prompts.
- Review AI-generated code with the same care as any other code (GR-DEV-03), including licences of any suggested code.
This guardrail is a draft proposal. Comment on it by opening an issue.
Agentic AI¶
AI agents do more than answer questions: they plan steps and take actions through tools, such as updating records, sending messages or calling APIs. That makes them useful, and it makes mistakes and attacks more costly. For how to build agents in Defra - identity, connecting tools, evaluations and tracing, and which platforms are ready - see working with AI agents in the AI digital toolkit, and talk to AICE before you choose a platform. These draft guardrails come from the guardrail backlog and apply on top of GR-AI-02 to GR-AI-06.
GR-AI-08 Give agents the least privilege they need¶
Should AI agents act under their own identity, with access only to the tools, data and actions their task needs, and never with a person's full permissions.
Why: an agent can be tricked, or can be wrong. Limiting what it can reach limits the damage (GR-IAM-03).
How to meet it: list each agent's tools and permissions, grant them to a workload identity for the agent, and review them as you would a privileged user's.
In the AI digital toolkit: working with AI agents.
GR-AI-09 Get human approval for consequential actions¶
Should An agent does not take an action with legal, financial or significant effects on people, or one that cannot easily be undone, without approval from an accountable person.
Why: GR-AI-03 keeps a human accountable for decisions. Agents act faster than people can notice, so approval has to be designed in, not assumed.
How to meet it: classify the agent's actions in alpha. Enforce approval in the tool layer, not only in the prompt, and test that the agent cannot bypass it.
In the AI digital toolkit: working with AI agents.
GR-AI-10 Keep an audit trail of what agents do¶
Should Record what each agent was asked, what it read, which tools it called with what inputs, who approved what, and the outcome - and send security-relevant events to security monitoring.
Why: without a full record, nobody can explain, challenge or reverse what an agent did.
How to meet it: log at the tool layer with correlation ids (GR-OPS-02), and keep records for as long as the decisions they support.
In the AI digital toolkit: working with AI agents.
GR-AI-11 Defend agents against prompt injection¶
Should Treat everything an agent reads - documents, emails, web pages, tool results - as untrusted input that may try to change its instructions, and design controls that hold even if it does.
Why: prompt injection is the most common way to make an agent misuse its tools or leak data.
How to meet it: include injection through every input in your threat model (GR-AI-05), separate untrusted content from instructions, restrict tools as in GR-AI-08 and require approval as in GR-AI-09.
In the AI digital toolkit: working with AI agents.