What Is an AI Hiring Bias Audit?
An AI hiring bias audit is a systematic evaluation of automated employment decision tools (AEDTs) to determine whether they produce discriminatory outcomes across protected groups such as race, gender, and age.
The audit examines selection rates, scoring distributions, and statistical patterns to identify whether an AI tool unfairly advantages or disadvantages certain demographic groups in hiring decisions.
Why Are Bias Audits Required?
Regulators worldwide have recognized that AI hiring tools can perpetuate and amplify existing biases. NYC Local Law 144 was the first to mandate independent bias audits, and California, Colorado, and the EU have followed with their own requirements.
The core concern is simple: if an AI tool learns from historical hiring data that reflects past discrimination, it will reproduce that discrimination at scale — faster and more consistently than any human recruiter could.
Who Needs a Bias Audit?
Any organization using AI in hiring decisions should conduct regular bias audits. This includes companies using:
- Resume screening and parsing tools
- AI-powered candidate assessments
- Video interview analysis
- Chatbot-based candidate screening
- Automated candidate ranking systems
How Does a Bias Audit Work?
A bias audit typically follows these steps:
- Data Collection: Gather historical hiring data including candidate demographics and outcomes at each stage
- Impact Ratio Calculation: Calculate selection rates for each demographic group and compare using the four-fifths rule
- Statistical Testing: Apply chi-squared or Fisher's exact tests to determine if disparities are statistically significant
- Intersectional Analysis: Examine outcomes for combinations of protected characteristics
- Score Distribution Analysis: If applicable, analyze how AI scores are distributed across groups
- Reporting: Generate a comprehensive report with findings, compliance status, and remediation recommendations
What Does a Compliant Audit Report Include?
Under NYC Local Law 144, the audit report must include impact ratios for race/ethnicity and sex categories. Best practices also include statistical significance testing, intersectional analysis, and actionable remediation guidance.
How Often Should You Audit?
NYC Local Law 144 requires audits within one year of AEDT use and annual re-audits thereafter. The EU AI Act requires ongoing monitoring. Best practice is quarterly auditing with continuous monitoring between audits.
Getting Started
OnHirely makes bias auditing simple. Upload your hiring data, and our platform automatically calculates impact ratios, runs statistical tests, and generates a compliance report — in minutes, not months.