AI HR decision output bias checker
When an AI screens CVs, ranks candidates, or recommends promotions, the same disparity tests that apply to human decisions apply to it — and regulators increasingly expect employers to run them. This checker takes your AI-generated decisions, computes the selection rate for each demographic group, and applies the standard four-fifths (80%) rule to flag groups that may be experiencing adverse impact. Everything runs locally in your browser.
How it works
You paste one row per candidate: a group label (gender, age band, or any protected characteristic) and a decision (selected or not). The tool counts favourable outcomes per group, divides by group size to get each group’s selection rate, then divides every rate by the highest group’s rate to get the adverse impact ratio. A ratio below 0.80 is the EEOC’s threshold for potential adverse impact and gets flagged. Because small groups produce unstable ratios, the tool also warns when any group has too few decisions to trust the result.
The four-fifths calculation explained with an example
Suppose an AI screening tool assessed 200 applicants across two groups. Group A had 100 applicants and 40 were selected (a 40% selection rate). Group B had 100 applicants and 28 were selected (a 28% selection rate). The adverse impact ratio is 28 ÷ 40 = 0.70 — below the 0.80 threshold, so Group B is flagged. Put another way: Group B is selected at 70% of the rate of Group A, which the EEOC guidelines treat as evidence of potential adverse impact warranting investigation.
Note that this is a statistical flag, not proof of discrimination. The investigation would look at whether there are legitimate, job-related explanations for the gap, whether the AI system’s features correlate with protected characteristics in ways that introduce proxy bias, and whether the same gap appears across different job roles or departments.
Why this test matters for AI hiring tools specifically
Human selectors make decisions that are hard to aggregate and audit. An AI system makes the same decision repeatedly and at scale, which means that any bias in its outputs is also applied at scale. A disparity that would be a rounding error in a ten-person team becomes a statistically meaningful pattern across thousands of applications. This is why several jurisdictions are beginning to require that employers who use AI in hiring conduct and retain records of disparity testing: the EU AI Act classifies employment-related AI as high-risk and requires bias monitoring, and New York City’s Local Law 144 requires annual bias audits for AI hiring tools covering gender and race/ethnicity.
Running this test regularly — ideally at each stage of a multi-stage hiring process — lets you catch and address problems before they accumulate.
Tips and notes
- The four-fifths rule is a screen, not a verdict. A flag means investigate; a pass means no adverse impact was detected on this metric in this sample.
- Mind sample size. With fewer than roughly 30 decisions per group, ratios are statistically unstable — collect more data before drawing conclusions, and treat the tool’s small-sample warning seriously.
- Test one characteristic at a time. Run gender, then age band, then intersections (such as gender-by-age) separately — pooling groups together hides patterns within them.
- Document every test run. Keeping a record of when disparity tests were conducted, by whom, and what the results were is itself a key part of defensible AI hiring practice under any regulatory framework.