SXSW speaker: Danger of bias in AI

I spoke at SXSW 2019 on the extreme danger of bias in AI, as Executive Technology Director at R/GA.

The argument turned on scale. A person acting on unconscious bias affects maybe ten people. Encode that same bias into a system and it affects five hundred million. I asked the room to sit inside four cases rather than discuss the idea in the abstract: an Amazon hiring tool that discriminated against women, a tenant screening algorithm standing between you and a home, a self driving car that fails to register you as a human because of the colour of your skin, and being flagged by COMPAS as twice as likely to commit a crime because you are a minority.

The second half was what companies can actually do about it:

• Hire more diverse teams.
• Use better training data and publish it. IBM had released a set of a million images of human faces; Ford was training in-car voice recognition on 50 per cent more female voices than male.
• Add bias testing to the product development cycle, the way you would add security testing.
• Accept that training on historical data reproduces historical mistakes, so the bias has to be found in the data first.
• Use the tooling that was starting to appear, such as IBM’s Fairness 360 kit and independent algorithm auditing.

It followed from what I had been writing since 2017, and I made the same case later that year on an AfroTech panel.