AI Blindspot: A Discovery Process for preventing, detecting, and mitigating bias in AI systems

What are AI Blindspots?

AI Blindspots are oversights in a team’s workflow that can generate harmful unintended consequences. They can arise from our unconscious biases or structural inequalities embedded in society. Blindspots can occur at any point before, during, or after the development of a model. The consequences of blindspots are challenging to foresee, but they tend to have adverse effects on historically marginalized communities. Like any blindspot, AI blindspots are universal -- nobody is immune to them -- but harm can be mitigated if we intentionally take action to guard against them.

Organize an "AL" workshop

Download AI Blindspot cards (PDF)

PLANNING

In the initial stages of your project, it is important to think critically about: why you want to use a particular technology (Purpose); how accurately your data reflects affected communities (Representative Data); what vulnerabilities your system might expose (Abusability); and how to safeguard personal identifiable information (Privacy).

BUILDING

Vulnerable populations can be harmed due to the performance metric you choose (Optimization Criteria) or variables that act as proxies (Discrimination by Proxy). Depending on the sensitivity of the use case, you may need to understand and explain how the algorithm makes determinations (Explainability).

DEPLOYING

You should be vigilant about monitoring for changes that might affect the performance and impact of your system (Generalization Error), and ensure that individuals have mechanisms to challenge decisions (Right to Contest).

MONITORING

Organizations using AI systems should institute inclusive processes for stakeholders input (Consultation) and independent risk assessment (Oversight). The best way to catch blindspots is to genuinely engage with experts and affected communities as equals to define and track progress towards collective goals (Purpose)..

What do we mean by AI?

Artificial intelligence has become a catch-all category of automated decision making systems that derive patterns, insights, and predictions from big datasets. While they might aspire to emulate and automate intelligent human-like judgment, most algorithms referred to as AI are in fact imperfect models susceptible to making erroneous inferences and rendering biased decisions.

The risk of delegating high-stakes social and commercial decisions to AI exposes everyone to unequal treatment because these seemingly impartial algorithms are produced by computer scientists, engineers, and companies whose data and practices may amplify historical biases in society.

Fairness requires thoughtful vigilance across all sectors, especially from researchers inventing, engineers building, organizations deploying, and advocates tracking AI systems. Above all, we need to safeguard and uplift people whose lives are affected by AI.