The ICO's November 2024 audit of AI recruitment tools found transparency gaps, over-collection and indefinite retention of candidate data, and weak bias testing — including tools that let recruiters filter on protected characteristics or inferred gender and ethnicity from names. UK SMEs can treat those findings as a vendor due-diligence checklist, and should combine data protection duties under UK GDPR with equality duties under the Equality Act 2010, keeping a meaningful human decision for rejections as well as shortlists. This is not legal advice.
A recruiter buys an AI tool that promises to read every CV, score every applicant and hand back a ranked shortlist by lunchtime. The pitch is efficiency: less time reading, more time interviewing. The assumption is that because the tool came from a specialist supplier, someone competent has already dealt with the legal side.
In November 2024 the Information Commissioner's Office (ICO) tested that assumption directly. It carried out consensual audits of several developers and providers of AI-powered sourcing, screening and selection tools, and published what it found. The picture was not reassuring — and for any UK business considering one of these tools, that is quietly good news.
The regulator has already looked inside these tools. Its findings are, in effect, a free due-diligence checklist for the business buying one — not just the developer building it.
The Quick Answer
In November 2024 the ICO published the outcomes of consensual audits into AI recruitment tools. It made almost 300 recommendations, all of which the audited organisations accepted or partially accepted.
The recurring problems it found were:
- Unfair processing — for example, tools that let recruiters filter out candidates with certain protected characteristics
- Inferring sensitive characteristics — some tools guessed gender and ethnicity from a candidate's name rather than asking
- Over-collection — some tools gathered far more personal information than the task required
- Indefinite retention — building large candidate databases without people's knowledge and keeping data with no clear retention period
- Transparency gaps — candidates were not clearly told how AI would use their information
- Weak bias monitoring — insufficient checks to detect and mitigate discrimination over time
The business that buys and uses the tool is a data controller for how candidates are treated, and remains answerable under both UK GDPR and the Equality Act 2010. A polished demo does not transfer that responsibility to the vendor.
Buying from a specialist supplier does not outsource your accountability. If a tool discriminates or over-collects, the employer still answers for the outcome.
Last checked: 4 August 2026. Regulatory guidance, audit follow-ups and the law in this area continue to develop; verify the current position with the ICO and other named sources before acting. This article is general information, not legal advice.
What the ICO actually audited
The ICO describes this work as part of its upstream monitoring of the wider AI ecosystem: rather than waiting for complaints, it engaged directly with the companies building and supplying AI recruitment tools. These were consensual audit engagements with developers and providers of AI-powered sourcing, screening and selection tools — the software that finds potential candidates, summarises CVs and scores applicants.
The output was an "AI in Recruitment Outcomes Report", published on 6 November 2024, alongside a set of key questions for organisations planning to buy these tools. The regulator made almost 300 recommendations across the audited organisations, and reported that all recommendations were accepted or partially accepted, with follow-up confirming that recommended actions were being implemented.
The report is not a naming-and-shaming exercise, and it is worth reading in that spirit. It sets out what good and poor practice looked like, so buyers can recognise the difference. That is exactly what makes it useful as a due-diligence resource.
The findings, translated for a buyer
The findings fall into a few clear themes. Read them as things to check for, not just things that happened to someone else.
| What the ICO found | Why it matters to the buyer |
|---|---|
| Some tools were not processing personal information fairly — for example, allowing recruiters to filter out candidates with certain protected characteristics | Filtering on a protected characteristic can be both unfair processing under UK GDPR and unlawful discrimination under the Equality Act 2010. The employer using the filter is exposed, not just the developer. |
| Some tools inferred characteristics — including gender and ethnicity — from a candidate's name instead of asking | Inferred sensitive data is still personal data, and can be special category data. It is frequently inaccurate, and it can drive biased outcomes the recruiter never intended. |
| Some tools collected far more personal information than necessary | Data minimisation is a core UK GDPR principle. Collecting more than the task needs increases risk, cost and the harm caused if the data is ever breached. |
| Some tools retained data indefinitely to build large candidate databases without people's knowledge | Keeping data with no defined retention period, and without telling candidates, undermines both fairness and storage-limitation duties. |
| Candidates were not always given clear, transparent privacy information about the AI's role | People are entitled to understand how their data is used, including AI screening. Weak transparency is one of the easiest failings for a regulator or an applicant to spot. |
| Bias monitoring was, in places, insufficient | A tool trained on past hiring data can reproduce past bias. Without regular checks, discrimination can persist quietly and at scale. |
The ICO instructed some providers to collect accurate information directly from candidates and to put regular checks in place to monitor and mitigate potential discrimination. If a vendor cannot describe both of those things, that is a gap you can see from the outside.
Where the Equality Act 2010 meets training data
Data protection and equality law are separate regimes, but AI recruitment sits squarely on the seam between them. The Equality Act 2010 applies to recruitment outcomes however they are produced. It does not contain an exception for decisions reached with the help of software. If a process disadvantages people who share a protected characteristic — age, sex, race, disability, religion or belief, sexual orientation, gender reassignment, marriage and civil partnership, or pregnancy and maternity — the employer may be exposed to a discrimination claim regardless of whether a human or an algorithm produced the ranking.
The mechanism is worth understanding, because it is not obvious. Many AI screening tools learn from a company's historic hiring data — who was shortlisted, who was hired, who succeeded. If that history reflects past bias, the model can learn to reproduce it. This can happen even when protected characteristics are never fed into the tool directly, because other data points can act as proxies. A postcode, a school name, a career gap, the wording of a hobby, or a name can all correlate with a protected characteristic. This is broadly what the ICO flagged when it found tools inferring gender and ethnicity from names.
Indirect discrimination can arise where a seemingly neutral rule puts a protected group at a particular disadvantage and cannot be objectively justified. An automated screen that systematically down-ranks a protected group could, depending on the facts, raise exactly that risk. Businesses should consider taking their own advice on how equality law applies to a specific tool and process; the point here is only that "the model did it" is unlikely to be a defence.
Removing the protected-characteristic field from a dataset does not remove bias. Models can rediscover the characteristic through proxies. Bias testing looks at outcomes, not just inputs.
"Meaningful human review" for rejections, not just shortlists
A common instinct is to keep a human in the loop only at the top of the funnel: let the tool rank everyone, then have a person review the shortlist. That feels like meaningful human involvement, but it can miss the point. The candidates who most need a human decision are often the ones the tool has already discarded.
Under UK GDPR, individuals have specific protections in relation to decisions based solely on automated processing that produce legal or similarly significant effects. Being screened out of a job can be a significant effect. Where an automated screen is the practical reason a candidate never reaches a human, waving through only the survivors may not amount to genuine human involvement in the decisions that actually excluded people.
Meaningful human review, in a recruitment context, tends to mean a person who:
- Has the authority and the competence to overturn the tool's output, not merely to rubber-stamp it
- Sees enough information to form an independent view — not just a score, but the underlying application
- Understands what the tool does, its known limitations and where it tends to go wrong
- Applies judgement to rejections and borderline cases, not only to obvious shortlists
- Records the reasons for the decision in a way that could be explained to the candidate later
None of this means AI cannot help. Drafting an advert that a human edits, or summarising an application for a human to read, is low-stakes. The care is needed where the tool ranks, filters or rejects — the higher-stakes uses that carry specific obligations. The dedicated recruitment guide in the AI Governance hub sets out these gradations in more detail.
A short worked example (fictional)
The following scenario is illustrative and fictional. It is used only to show how the findings play out in practice, and does not describe any real business, tool or person.
A 30-person UK firm buys an AI screening tool to handle a spike in applications. The tool scores CVs against past successful hires. Because the firm's historic hires skewed heavily towards one demographic, the model quietly rewards CVs that resemble them — similar universities, similar phrasing, no career breaks. Nobody chose to discriminate. But candidates with caring gaps and non-UK qualifications are consistently ranked low and never reach a human. Six months later, the firm cannot explain why its shortlist looks so uniform, has no record of who was rejected or why, and has kept every applicant's data indefinitely.
Every failing in that story maps to something the ICO's audits highlighted: no bias monitoring on outcomes, no meaningful human review of rejections, no defined retention period, and weak transparency. It also maps to real Equality Act exposure. The fix is not to abandon the tool; it is to add the controls that were missing — a DPIA before purchase, outcome monitoring, a human decision behind rejections, a retention schedule and clear candidate information.
Questions to put to any recruitment-tool vendor
Alongside the audit outcomes, the ICO published key questions for organisations procuring these tools. The list below adapts and extends those, framed as things a buyer can ask before signing. A vendor that answers them clearly is easier to trust; one that deflects is telling you something.
- 1Have you completed a DPIA for this tool, and can we see enough of it to complete our own? A Data Protection Impact Assessment should be done before use, ideally at procurement.
- 2What is the lawful basis for the processing, and how is special category data handled if the tool infers or processes it?
- 3Exactly what personal data does the tool collect, and how is that limited to what the task needs? Ask them to justify anything that looks excessive.
- 4How long is candidate data kept, and can we set the retention period? Beware any answer that amounts to "indefinitely".
- 5Does the tool ever infer characteristics such as gender or ethnicity, directly or from proxies like names? If so, how is that controlled?
- 6How was the tool tested for bias, on what data, and how often is that testing repeated once it is live? Ask about outcome monitoring, not just input controls.
- 7Can a recruiter use the tool to filter on, or against, a protected characteristic? If yes, that is a red flag on its own.
- 8What transparency information can we give candidates about the AI's role, and does the tool support that?
- 9What does the tool actually decide versus recommend, and where is the point of meaningful human review — including for rejections?
- 10Can you explain, in plain language, how a given candidate was scored, so we could justify a decision if challenged?
- 11What happens to our data and the candidates' data if we stop using the tool?
- 12Will you support us if a candidate exercises their data protection rights, or raises a discrimination concern?
Record the answers. If a tool is later challenged, being able to show the due diligence you did — and the assurances you were given — is far better than reconstructing it under pressure. The AI Tool Approval Checklist below turns this into a repeatable step you can apply to any AI tool, not only recruitment software.
What this means for a smaller business
The obligations here do not scale down neatly with headcount. A five-person firm and a fifty-person firm both answer to the same Equality Act and the same UK GDPR principles. What can scale down is the proportionality of the response: a smaller business is not expected to build a research lab, but it is expected to think before it buys, to ask the vendor sensible questions, to run a DPIA where the processing is high-risk, and to keep a genuine human in charge of decisions that affect people's livelihoods.
- Treat the ICO's findings as a checklist of what to ask, not as someone else's problem.
- Run a DPIA before procurement, not after go-live, wherever the processing is likely to be high-risk.
- Insist on a defined retention period and clear candidate privacy information.
- Monitor outcomes for bias over time — a tool that was fair at launch can drift.
- Keep a meaningful human decision behind rejections, not only shortlists, and record the reasons.
This article is general information for UK businesses and is not legal advice. Employment and data protection decisions turn on specific facts; businesses should consider taking professional advice before adopting an AI recruitment tool.
Where to go next
If you are weighing up an AI recruitment tool, start with the dedicated recruitment guide in our AI Governance hub, then use the AI Tool Approval Checklist to structure the vendor conversation.
Using AI in Recruitment — the full guide in our AI Governance hub →
AI Governance hub — policies, guides and templates for using AI safely →
Download: AI Tool Approval Checklist (supplier due-diligence questions) →
Sources and further reading
The following sources were used and verified when this article was last checked on 4 August 2026. The law and guidance in this area continue to develop; confirm the current position before relying on any point.
ICO — AI tools used in recruitment: audit outcomes overview (6 November 2024) →
ICO — AI in Recruitment Outcomes Report (PDF) →
ICO — Thinking of using AI to assist recruitment? Our key data protection considerations →
GOV.UK — Responsible AI in Recruitment guide →
Equality Act 2010 (legislation.gov.uk) →
Equality and Human Rights Commission — Artificial intelligence in public services →
Plain-English Takeaway
The ICO has already audited the AI recruitment tools SMEs are being sold, and its November 2024 findings amount to a free due-diligence list: watch for weak transparency, over-collection and indefinite retention, and thin bias testing. Buyers remain accountable for how a tool treats candidates, so ask hard questions, run a DPIA before procurement, and keep a genuine human decision behind rejections as well as shortlists. This article is general information, not legal advice.
Related Articles
The Small Business That Automated Its Biggest Mistake
Artificial intelligence rarely invents a brand-new risk. More often it takes a weakness the business already had — a habit, a shortcut, an unchecked step — and runs it faster and further than a person ever could. A quoting error that once reached one customer can now reach two hundred in an afternoon, each message personalised, polished and convincing. This article follows a fictional but realistic example to show how the same underlying mistake becomes far more dangerous once it is automated, and it sets out three ordinary controls that would have caught it before it spread.
Read articleWho Owns the Logo Your AI Just Designed?
You typed a prompt, an AI tool produced a logo you love, and the platform's terms say the output is yours. So that settles it — the logo belongs to your business and you can build a brand on it. Except platform permission is only one of five separate ownership questions, and a generous answer to that one does not settle the others. This article unpacks the five questions a UK small business should think through before betting a brand on an AI-generated design, and points to where the law is genuinely unsettled.
Read articleWhy Is AI Computing Power Becoming a Tradable Commodity?
Major financial exchanges are preparing futures contracts linked to benchmark prices for renting AI computing capacity. These products could help large AI companies, cloud providers and investors manage changes in future GPU costs — but they do not buy or sell artificial intelligence itself, and they carry significant financial, benchmark and regulatory risk. This article explains what compute futures are, why they are emerging and what they could mean for businesses.
Read article