AI governance is the set of rules, roles and checks an organisation uses to control how it builds and uses AI: which systems are allowed, what data they may touch, who approves them, how they're tested, and what happens when one gets something wrong. Done well, it's a short list of decisions and a record that they were followed.
What is AI governance?
Strip away the frameworks and governance answers five questions about every AI system you run.
- What is it for, and who owns it?
- What data does it see, and is it allowed to?
- How do we know it works well enough?
- Where does a person stay in the decision?
- What do we do when it fails?
If you can answer those for each system, in writing, with a named owner, you have governance. If you have a forty-page policy and can't answer them, you have a document.
Why it has become urgent
Two things changed. The first is that staff are already using AI, with or without permission. Someone pastes a contract into a public chatbot to summarise it, and confidential data has left the building with no record that it did.
The second is regulation. The EU AI Act entered into force in August 2024, with obligations phasing in over the following years and the strictest rules reserved for high-risk uses. Standards bodies moved too: NIST published its AI Risk Management Framework in January 2023, and ISO/IEC 42001, a management-system standard for AI, followed in December 2023.
Customers are the third pressure, and often the first one felt. Security questionnaires now ask how you use AI. A clear answer wins deals. A vague one stalls them.
The frameworks, briefly
| Framework | What it is | Use it for |
|---|---|---|
| EU AI Act | Law. Sorts AI uses by risk and sets duties for each tier | Knowing what you're legally required to do if you operate in or sell into the EU |
| NIST AI RMF | Voluntary guidance built around govern, map, measure and manage | A structure for your own programme |
| ISO/IEC 42001 | A certifiable management-system standard | Showing customers and auditors a recognised benchmark |
These overlap heavily. Pick one as your spine and map the others to it. This article is engineering guidance and not legal advice, so take the question of which laws bind you to counsel.
How to start: a first ninety days
Most programmes stall because they begin with policy. Begin with an inventory instead.
- List the AI in use. Sanctioned tools, features inside software you already buy, and what staff are using informally. Expect surprises.
- Sort it by risk. A tool that drafts marketing copy and one that influences a claim decision don't deserve the same scrutiny.
- Set data rules. Which classes of data may go to which tools. This single page prevents most incidents.
- Give people an approved option. If the sanctioned assistant is worse than the public one, they'll keep using the public one.
- Name an owner for each system, and a small group that approves new uses quickly.
- Decide how each system is tested and how often.
Speed of approval matters more than people expect. A review board that takes three months teaches the organisation to stop asking.
What governance looks like in the engineering
Policy only counts if the system enforces it. This is where most of our work sits, and it's why we describe our approach as secure first.
- Deployment under your control. We build inside your environment and call models through your own accounts, so your agreements and data terms apply.
- Access checks at retrieval. An assistant answers only from documents the user may open. See private LLMs.
- Evaluation sets. Real questions with agreed answers, run before release and after every change.
- A person in the loop. Where a wrong answer costs money or trust, the system proposes and someone approves.
- Logs. Inputs, outputs, sources and the model version, kept so a decision can be explained later.
Notice that most of this is ordinary software discipline applied to a new kind of component. Teams that already review code, test changes and log decisions are most of the way there.
Common mistakes
Banning AI outright. It doesn't stop use. It stops you seeing it.
Governing the model and ignoring the data. Most real incidents are data incidents: the wrong information sent to the wrong place. Start there.
Treating approval as a one-time event. Models change, prompts change, the documents behind an assistant change. A system approved in spring is a different system by autumn, so recheck on a schedule.
Writing for auditors only. If the people building things can't follow the policy in ten minutes, they won't.
Who should own it
Not IT alone, and not legal alone. The working model is a small group with someone from technology, security, legal or compliance, and the business, with one accountable executive.
Keep it small enough to decide in a week. In regulated sectors the detail differs, and we've written up how our build process meets it for insurance and finance.
If you want an outside view of where you stand, our data and AI assessment includes an inventory and a risk-ranked plan.
