A person is not a rubber stamp

European law has a recurring answer when an automated system takes a decision: put a person next to it.

Article 22 GDPR gives data subjects the right not to be subject to a decision based solely on automated processing which produces legal effects concerning them or similarly significantly affects them. And in some of the cases where such a decision is allowed, it grants at least the right to obtain human intervention, to express one's point of view and to contest the decision.

The AI Act goes further. For high-risk systems, Article 14 requires effective human oversight. And Article 26 requires deployers to assign it to natural persons who have the necessary competence, training and authority, as well as the necessary support.

The rubber-stamp problem

Both rules know a risk. A person can be present and count for nothing. They look at the screen, see the recommendation, confirm. A hundred times a day.

The AI Act says so explicitly. Among the capabilities oversight must ensure is remaining aware of the possible tendency of automatically relying or over-relying on the system's output. The rule gives it a name: automation bias.

The other listed capabilities are just as concrete. Understanding the system's capacities and limitations. Correctly interpreting its output. Deciding, in any particular situation, not to use it, or to disregard, override or reverse its output. Interrupting it through a "stop" button or a similar procedure.

Understand before signing

I read these rules as an application of the principle I discuss in Habeas Mentem. The person is useless unless they are put in a position to understand. And understanding takes time, information and the authority to say no.

Human oversight that cannot disagree is a rubber stamp. And a rubber stamp protects nobody.

For those who design these systems, practical questions follow. How many decisions must one person review per hour? Do they see the system's reasons or only the result? Have they ever overturned a recommendation? What happens to them if they do?

These are design questions, not only compliance questions. They can be measured. And the answers tell us whether the person is really there.