AI and Social Impact: A Review of Current Use Cases and Broader Implications

Background

The importance of not implementing machine learning as one pleases without regard for the consequences is discussed by Bolukbasi, Chang, Zou, Saligrama, and Kalai (2016), where such practices increase bias in data, which can then result in very real physical harm such as jail time or the denial of a loan (Buolamwini & Gebru, 2018). Studies on word bias have highlighted the use of words that increase gender stereotypes (e.g., a word association for machine learning: woman is to homemaker). These pre-labeled data result in biased data; “algorithms trained with biased data have resulted in algorithmic discrimination” (Buolamwini & Gebru, 2018, p.1; Bolukbasi et al., (2016); Caliskan, Bryson & Narayanan, 2017; Elmi, 2021).

The Gender Shades study by Buolamwini and Gebru (2018) compared three commercial facial recognition technologies, IBM, Microsoft, and FACE++, using three facial analysis datasets IJB-A (U.S. government benchmark used by National Institute of Standards and Technology), Adience, and the dataset Pilot Parliaments Benchmark (PPB) developed by Buolamwini and Gebru. IJB-A was composed of 79.6% lighter individuals and Adience was composed of 86.2% lighter individuals and PPB was composed of 53.6% lighter-skinned individuals. Adience consisted of 2194 separate individuals and only included 302 darker individuals, whereas PPB consisted of 1270 people and included 589 individuals with darker skin. Thus, the PPB dataset had a higher number of darker-skinned individuals than the other two datasets. This comparison was designed to determine each facial recognition technology’s accuracy regarding both the shade of one's skin color and gender. The results highlighted the extreme inaccuracies of facial recognition technology regarding the identification of an individual, performing best for light skinned males and least accurate for darker-skinned females. The error rate of the AI facial recognition software for darker-skinned females was between 20.8% - 34.7%, while for lighter-skinned males the error rate ranged between 0.0-0.3%. Darker-skinned males were still misclassified more often than lighter-skinned males (Buolamwini & Gebru, 2018). Researchers concluded that inclusive benchmark datasets are important to increase transparency and accountability in AI.

From the ADAPT Centre at Dublin City University and IBM Research in Dublin, Sen and Ganguly (2020) discussed the use of a learning framework based on multiple objectives to help ensure that predictions derived from the data do not constitute social biases or stereotypes. The resulting model sought to reduce specific associations between pairs of data points that are ethically incorrect (i.e., black people are more likely to be criminals, and fear is associated with women). The research was based on datasets of emotion prediction and illustrated that the proposed bias-aware learning framework could reduce some cognitive biases from data predictions (Sen & Ganguly, 2020).