Any AI system that influences decisions about people can produce unequal outcomes across protected groups, even when no group is named in the data. The AI Bias Impact Assessment walks you through the questions regulators and auditors expect you to have answered: who is affected, how harm could occur, what you have done to reduce it, and how you will keep watching.
How it works
You describe the system and the stakes of the decisions it informs, then select which protected characteristics could plausibly be affected. The tool then asks about concrete controls — whether you have measured outcome disparities, tested for proxy variables, built in a human review or appeal path, and committed to ongoing monitoring.
It combines two signals into a residual-risk rating: the decision stakes (a recommendation engine is lower stakes than a hiring or credit decision) and your control coverage (how many mitigation and monitoring measures are in place). High stakes with thin controls produce a high residual risk; the same stakes with disparity testing, an appeal route, and monitoring produce a lower one.
What you get
The output is a structured record: the affected groups, the harm pathways flagged, the controls present and missing, and the residual-risk rating with a short rationale. This maps onto the structure of a UK Equality Impact Assessment and the fundamental-rights impact assessment expected under the EU AI Act for high-risk systems.
Tips and notes
The most commonly missed step is testing for proxy variables — features such as postcode or device type that correlate with a protected characteristic and reintroduce bias you thought you had removed. The second is an appeal or override path, which both reduces harm and is increasingly a regulatory expectation. Treat the assessment as a living document; redo it whenever the model or its context changes. All analysis happens locally in your browser and nothing is uploaded.
Why AI systems produce unequal outcomes
Bias in AI does not require a deliberate design choice to harm a group. It can enter a system at several stages, and a thorough impact assessment looks at each one:
Training data — if historical data reflects past discrimination (for example, loan approvals that historically favoured certain demographics), a model trained on it learns and perpetuates those patterns, even when no protected characteristic is included as a feature.
Feature selection — features that seem neutral can be proxies. Postcode correlates with ethnicity and socioeconomic status. Device type correlates with income. Job title correlates with sex. A model that uses these features can discriminate without ever seeing a protected attribute.
Feedback loops — deployed systems influence future data. A content recommendation algorithm that shows fewer educational opportunities to one group produces less engagement from that group, which the model then interprets as evidence to show even fewer — compounding the gap over time.
Evaluation gaps — if a system is tested only on the majority group, or if performance metrics are averaged across groups, poor performance on a smaller protected group is invisible until deployment.
The regulatory landscape this assessment maps to
In the UK, the Equality Act 2010 requires organisations to consider the differential impact of their policies and practices on groups sharing protected characteristics. A formal Equality Impact Assessment (EIA) is statutory for public authorities and best practice for any organisation using AI to make or influence decisions about people.
The EU AI Act introduces a mandatory fundamental-rights impact assessment for high-risk AI systems — defined by the sector and the nature of the decision, not by the technology itself. High-risk categories include systems used in employment, credit, access to education, and public services. The assessment structure this tool produces mirrors the sections that assessment requires: affected groups, harm pathways, existing controls, residual risk, and a monitoring commitment.
Neither this tool nor its output constitutes legal advice, and organisations deploying high-stakes AI should have their impact assessments reviewed by qualified counsel. The value of the tool is in structuring the thinking before that review — it is far easier to review a well-organised document than a blank page.
High-stakes scenarios where this matters most
- Hiring and recruitment screening — any system that filters or ranks candidates affects access to employment and is high-risk under the EU AI Act.
- Credit and lending decisions — a model that influences loan approvals, credit limits, or interest rates can systematically disadvantage protected groups.
- Healthcare resource allocation — triage algorithms, appointment scheduling tools, or diagnostic aids that allocate scarce resources have direct health outcomes.
- Public-sector eligibility assessment — benefits, housing allocation, or planning decisions that use algorithmic scoring affect access to essential services.