What is AI governance?
AI governance is the set of policies, roles, and controls an organization puts in place to develop and use AI responsibly and in line with the law.
AI governance is the record you can show when someone asks.
AI governance is the set of policies, roles, and controls an organization puts in place to develop and use AI responsibly and in line with the law. It covers questions of accountability, transparency, fairness, privacy, and risk management across an AI system's life. With regulations such as the EU AI Act, sound governance helps organizations stay compliant while building trust in their AI.
AI governance is how an organisation decides which AI systems it runs, who is accountable for each, and what evidence exists that they behave as intended.
Most explanations summarise the regulation. That is not the hard part. The hard part is that governance is a set of documents somebody has to keep current, and knowing which ones is worth more than another summary of the law.
The four artefacts that constitute governance in practice
1. A model register: every AI system in use, what it decides, what data it touches, and who owns it. Most organisations cannot produce this, and discovering the shadow tools is usually the first real finding. 2. A human-oversight record, for each system, what a person can review, override or stop, and who that person is. 3. A logging policy, what is recorded about each decision and for how long, because "the model decided" is not an answer without a log. 4. An incident procedure, what happens when a system produces a harmful or clearly wrong output, written before it does.
Start with the register, not the policy
The common failure is writing a governance policy first. A policy written before the register describes systems nobody has enumerated, and is obsolete on contact with the actual estate. Build the inventory first: walk the departments and ask what tools people use to make or support decisions. Expect to find subscriptions IT does not know about. The register makes the policy writable, because you are then governing things that exist rather than things you imagined.
Shadow AI: the AI tools that are not in your register
Shadow AI is AI use outside what the organisation has approved: an employee pasting client data into a free chatbot, a department buying an AI tool on its own budget, an AI feature that quietly appears in existing software. It is the reason an AI register almost always starts smaller than reality.
The reflex is to block and to measure. In a Dutch organisation with a works council, measuring is the part that needs consent: software that shows AI use per employee quickly becomes a staff-monitoring system under Article 27 of the WOR. The order that works: an approved tool with a processing agreement and one clear rule about which data goes nowhere first, the works council conversation and the DPIA started in parallel, and detection only after that, aggregated and without names at first. How that works step by step is in shadow AI at work: detection needs works council consent.
Two things shrink shadow AI more than a block: training that shows staff what they can do and which data belongs nowhere (the AI literacy training), and an inventory of the tools already in use, with a decision per tool. If you want to run that inventory with an outside party, that is an AI audit and strategy.
Proportionality, not every system needs the same treatment
Governance effort should track consequence. A model that drafts internal meeting summaries and one that screens job applicants do not warrant the same scrutiny, and treating them identically guarantees the framework is ignored. Sort by who is affected and how reversible the outcome is: systems touching people's employment, credit, health or legal standing get the full treatment; internal productivity tools get a register entry and an owner. This is also the logic the EU AI Act uses, so a proportionate internal approach maps onto it reasonably well.
What this looks like for a company without a compliance function
A spreadsheet and a named owner beats an unimplemented framework. The realistic first version is: one sheet listing every AI tool with an owner and a one-line description of what it decides, a short note per system on what a human can override, and a decision about what gets logged. That is perhaps a day of work and it puts you ahead of most mid-sized companies. Governance is a habit, not a document, the register only has value if something makes it get reviewed.
AI governance next to your ICT partner: who owns which document
When the AI supplier is not your ICT partner, one question sits between two parties: who keeps which document? Settle it before the first system goes live.
| Document | Owner | Supplies input |
|---|---|---|
| AI usage policy | Board or CIO | IT (allowed tools), privacy officer |
| AI register | Information manager or risk | Process owners, the AI supplier per system |
| DPIA | Privacy officer or DPO | Process owner, ICT partner (hosting and processors) |
| Model inventory and evaluations | IT department, after handover | The AI supplier (versions, evaluation runs) |
| Incident log | IT department | Service desk, ICT partner, the AI supplier for triage on model and data |
In more detail: writing an AI policy, setting up an AI register, ISO 42001 next to ISO 27001, and the AI literacy training that goes with the policy.
Frequently asked questions
Is AI governance the same as EU AI Act compliance?
No. Compliance is a legal obligation with a defined scope; governance is the internal practice that makes compliance demonstrable and also covers systems the regulation does not reach.
- Plenty of systems fall outside the Act's high-risk categories and still deserve an owner and a log.
- Good governance makes compliance less expensive, because the evidence already exists when someone asks for it.
- Treating governance as purely a legal exercise tends to produce documents nobody uses.
Who should own AI governance in a mid-sized company?
Someone with authority across departments, not the AI team. The AI team cannot meaningfully govern itself, and IT often lacks visibility into tools bought on departmental cards.
- In practice this often sits with operations, risk or the COO: whoever already owns cross-functional process.
- Each individual system still needs a named business owner; central ownership is of the register and the process, not of every model.
- If nobody owns it, the register goes stale within a quarter and the exercise was decorative.
What should be logged about an AI decision?
Enough to reconstruct why a specific output happened, months later, without the original engineer.
- The input, the output, the model version, the timestamp, and whether a human reviewed or overrode it.
- For anything touching personal data, log the decision, not necessarily the full content, retention rules apply to your logs too.
- Agentic systems need every tool call recorded. They fail silently, so an unrecorded action is an undebuggable one.
Want this applied in your business? See how we take it to production:
From concept to working tool?
We build this AI in production, at fixed prices, with one named expert. Start with a free consultation.
Book a free consultation →