Article · AI & compliance
The audit pack: Articles 9 and 15 in practice
The regulation does not require the system never to be wrong. It requires you to know how often, to have decided whether that is acceptable, and to be able to show both.
AI & compliance · documentation · about 4 min
The two articles that most often turn into concrete work for a high-risk system are Article 9 and
Article 15. They are often read as a threat. They are easier to meet than their reputation suggests —
but only if you have measured along the way instead of reconstructing afterwards.
Article 9: risk management as a continuous process
The requirement is a risk management system that runs across the whole life cycle and is
updated systematically. Not a document you write once before you go live.
That is the sharp edge in practice: a risk assessment dated at go-live and untouched
since does not meet its own description. It has to carry traces of having been used —
what you looked for, what you found, what you did about it.
Article 15: accuracy, robustness and cybersecurity
This is the provision people most often misread. Article 15 requires an appropriate
level of accuracy and robustness — and that the level of accuracy and the relevant accuracy metrics
be declared in the accompanying documentation.
It does not say the system has to be perfect. It says the number has to come out. That moves
the job from “make it faultless” to “find out how often it fails, decide whether that is good enough
for the purpose, and write both down”.
A system whose error rate nobody knows cannot be declared appropriate. That is
the problem — not that the error rate is greater than zero.
What the pack contains
- The measurement — how the system performs on representative tasks, set out so it can be
run again by someone else. Including the numbers that count against the system.
- The measurement protocol — the procedure written out, so the result can be verified on your
own material. It is public.
- The decision about what is appropriate — which error rate you have accepted for which
purpose, and why. That is the judgement Article 9 asks for traces of.
- The gate design — which decisions are not made automatically, whatever the model
answers, and how that boundary is drawn.
- The continuing measurement — how the number is kept current when the model, the vendor or
the task changes. See calibration per use case.
One caveat
This is a description of how we work with the requirements — not legal advice. Which
obligations apply to you depends on your role and the system’s classification, and that is a
judgement for a lawyer to make.
The order matters
The expensive way is to build the system, put it into service and only then try to reconstruct
how reliable it was — because by then the numbers that were supposed to be documented do not exist, and the alternative is
to measure from scratch on a system already in operation.
The cheap way is to measure along the way. Then the documentation is a by-product of the work instead
of a project of its own.