When does your AI tool need a compliance process?

An HR team buys an AI assistant to draft job adverts. Seven months later the same assistant is turning candidates away without a human looking at them.

Nobody decides that. It arrives in small steps, each approved the way any other change is approved. The recruiters spend too much of their week on admin, so the assistant starts by answering candidates' routine questions. Then it summarises each CV on one page. A recruiter still opens every application and chooses who goes forward. Nobody thinks to call compliance, and at that stage I would not either.

Six months in, the recruiters ask for a match score on each summary. A month later, applications scoring below a threshold start receiving an automatic polite rejection, to keep the pile manageable. Together those requests produce a tool that a regulator would look at very differently from the one that was bought.

There is a simple way to tell the uses that need heavy rules from the ones that need very few.

Most AI use needs very little

Drafting emails, summarising meetings, searching internal documents, forecasting capacity and writing code with an assistant have something in common. A person reads the output and decides what to do with it, and the people affected by the work never meet the AI directly. The basics still apply. Be careful what data goes in, check what comes back, and tell customers when they are chatting with an AI, which has been a requirement in the EU since August 2026. Beyond that, a heavy approval process costs more than the risk it removes.

One question shows you where the line is

Does the output change how a specific person is treated?

A person here means a customer, an applicant or an employee. Treated means what they are offered, what they pay, whether they get the job, the loan, the claim or the account, and how quickly. If the AI's output feeds any of those decisions, even as a suggestion a human is free to ignore, you are in different territory.

Now the case I would like you to argue with. Go back to the hiring team before the match score exists. The assistant writes a one-page summary of each CV, and the recruiter reads every summary and makes every call. I say that already changes how a candidate is treated, because the summary decides what the recruiter sees first, and a badly written one can drop the single detail that would have earned someone an interview. If you say no, tell me what keeps the recruiter's judgement independent of the summary.

The EU and the UK both draw the line roughly where this question does, in their own ways. The EU AI Act keeps a short list of uses it treats as high-risk, and screening job candidates, checking creditworthiness and pricing life or health insurance are all on it (Annex III). The heavier obligations for those uses start on 2 December 2027, after the EU pushed the date back in July 2026 (Regulation (EU) 2026/1744). The UK has no AI law, but since February 2026 anyone on the receiving end of a significant decision made solely by a machine must be told, can ask for a human to look at it, and can contest it (Data (Use and Access) Act 2025, section 80).

Teams cross the line through ordinary change requests

The hiring team never sets out to build a candidate-screening system. It buys an assistant for admin and approves two change requests. That is the usual route, and one review at the start of a project cannot catch it. The answer can change with any request that asks the tool to rank, rate, flag or route people, and those requests arrive looking like ordinary backlog items. The automatic rejections are the clearest case. In the UK they are exactly the kind of decision made solely by a machine that the candidate must be told about and can contest.

There is one more catch in the EU. A firm that changes what a bought-in tool is used for, so that it lands on the high-risk list, takes on the obligations of the company that built it (Article 25), whatever the vendor contract says.

Once the answer is yes, four habits cover most of it

Write the answer down and give it an owner. A line in the project brief saying whether the tool changes how people are treated, and why, takes five minutes. The owner should be one person with the authority to pause the tool and the knowledge to explain what it does.

Make sure a human can overrule the tool in practice, and give the affected person a route to challenge the result. If reviewers accept the suggestion every single time, the safeguard exists only on paper, so check now and then how often people disagree with it.

Keep a record of what the tool showed and who decided what. Rebuilding that from memory a year later is slow and expensive.

Then add the question to your change process, so every request touching the tool meets it before entering a sprint. One line in the Definition of Ready catches the automatic rejections before the first one goes out.

The question is a first filter and the specialists still matter

Personal data rules apply even when nobody is treated differently, and some sectors, medical devices for one, have their own software rules. Use the question to decide when to bring the specialists in.

This week, list the tools you already have and ask the question of each, starting with the ones nobody has thought about since they were switched on.

Where do you stand on the CV summary? Reply yes or no, with one line on why. I will answer every reply.