Purdue University: The Emergence of AI Ethics Auditing

Summary of Read Full Report

Explores the emerging field of artificial intelligence ethics auditing, examining its rapid growth and current state through interviews with 34 professionals. It finds that while AI ethics audits often mirror financial auditing processes, they currently lack robust stakeholder involvement, clear success metrics, and external reporting.

The study highlights a predominant technical focus on bias, privacy, and explainability, often driven by impending regulations like the EU AI Act. Auditors face challenges including regulatory ambiguity, resource constraints, and organizational complexity, yet they play a vital role in developing frameworks and interpreting standards within this evolving landscape.

• AI ethics auditing is an emerging field that mirrors financial auditing in its process (planning, performing, and reporting) but currently lacks robust stakeholder involvement, measurement of success, and external reporting. These audits are often hyper-focused on technical AI ethics principles like bias, privacy, and explainability, potentially neglecting broader socio-technical considerations.
• Regulatory requirements and reputational risk are the primary drivers for organizations to engage in AI ethics audits. The EU AI Act is frequently mentioned as a significant upcoming regulation influencing the field. While reputational concerns can be a motivator, a more sustainable approach involves recognizing the intrinsic value of ethical AI for performance and user trust.
• Conducting AI ethics audits is fraught with challenges, including ambiguity in interpreting preliminary and piecemeal regulations, a lack of established best practices, organizational complexity, resource constraints, insufficient technical and data infrastructure, and difficulties in interdisciplinary coordination. Many organizations are not yet adequately prepared to undergo effective AI audits due to a lack of AI governance frameworks.
• The AI ethics auditing ecosystem is still in development, characterized by ambiguity between auditing and consulting activities, and a lack of standardized measures for quality and accredited procedures. Despite these limitations, AI ethics auditors play a crucial role as "ecosystem builders and translators" by developing frameworks, interpreting regulations, and curating practices for auditees, regulators, and other stakeholders.
• Significant gaps exist in the AI ethics audit ecosystem regarding the measurement of audit success, effective and public reporting of findings, and broader stakeholder engagement beyond technical and risk professionals. There is a need for more emphasis on defining success metrics, increasing transparency through external reporting, and actively involving diverse stakeholders, including the public and vulnerable groups, in the auditing process.