AI security: risks, proportional controls and human oversight
AI security covers the technical and organisational risks specific to AI systems — prompt injection, data leakage through model inputs, poisoning, model and output manipulation, excessive agent permissions — plus the controls and oversight that keep those risks within accepted limits. It complements, and does not replace, existing information security.
Risks that are specific to AI
Existing security programmes rarely cover these by default.
- Sensitive data leaving the organisation through prompts and attachments.
- Prompt injection through content the model reads.
- Manipulated or poisoned training and retrieval data.
- Outputs used as authoritative without verification.
- Agents holding broader permissions than the task requires.
- Dependency on a single provider with no fallback or exit plan.
Proportional controls
Controls should match the impact of the use, not the enthusiasm of the moment. A drafting assistant and an automated credit decision do not need the same regime.
- Approved-tool list and a path for teams to request new ones.
- Rules on what data may enter which class of tool.
- Least privilege for integrations and agents.
- Logging of relevant calls and of who acted on the output.
- A defined human oversight point for consequential decisions.
Oversight that is real
Human oversight only counts when the person has the information, the time and the authority to disagree with the system. Named without those three, it is a formality that fails exactly when it matters.
Frequently asked questions
Is our information security programme enough?
It is the base, not the whole. AI adds risks tied to purpose, data in prompts, output reliability and agent permissions that classic controls do not address.
Should we block public AI tools?
Blanket blocks usually push use underground. An approved list plus a fast request path gives more visibility than a prohibition nobody follows.