Diversity, non discrimination and fairness

AI contributors must foster the creation of bias-free AI systems and give themselves the means to reasonably arbitrage between performance and fairness, through the use of proper frameworks. We will discuss an AI project lifecycle through the eyes of fairness, and explain how one can detect discrimination through statistical methods.

The following table shows guidelines for each phase of an AI project, so as to support AI contributors in the creation of bias-free AI systems.

AI Project lifecycle

Project phase	Guideline	Practical consequences
Ideation & design	At the start of the ideation & design phase of an AI project, AI contributors must be aware of potential discrimination risks/breaches in an AI project and avoid initiating a project that discriminates on purpose. Stakeholders should always be involved throughout the whole project and must have complete trust & understanding of the system and its risks.	Be aware of discrimination sensitive AI topics, such as: Police, crime prevention Selection of employees and students Advertising (Targeting specific people based on characteristics) Price discrimination (charge customers differently based on characteristics) Image & Search analysis Translation tools (human biases in translation related to gender stereotypes) Any system involving personal data must be reviewed carefully. AI systems must be built: Through an inclusive design process, so that stakeholders ( business but also individuals whose data is collected ) directly or indirectly affected by the AI system can understand and trust it. Practically: stakeholder’s feedback along the design process as well as during the run phase is necessary. Towards a non-discriminatory usage, namely, it must not be reserved to a subgroup of the society/entity/organisation.
Data collection & preprocessing	AI contributors must anticipate bias and question the risk of discrimination in the data collection and preprocessing of data (origin, gender, age, characteristics related to the brand image, …)	Pay much attention to vulnerable demographics, for example: children, minorities, disabled persons, elderly persons, or immigrants. Bias might come from history, incompleteness and/or bad governance data models. Any identifiable and discriminatory bias should be removed as much as possible in the data collection phase. Ensure that the data is representative of the full population. Be careful when defining class labels and target variable. Poorly defining them might have an unwanted adverse impact on specific groups. Example: AI system with the purpose to predict whether an employee is good or bad. This on its own is already an ill defined target variable. If one of the class labels then contains information on how late employees arrive are at work, then poorer people, living further away from the city, will be discriminated against for not being able to afford living in a city centre and thus having more traffic jams. Do not use proxies that might strongly correlate with fairness-related concepts (like protected characteristics) Do not select specific features that might introduce a bias against certain groups Do not use data (data collection) with a biased sampling procedure Do not use data (labelling examples) with a bias. This might occur when humans are manually labelling data and then feed that into the AI system. Example: Consider an AI system to select good interview candidates. If we want to predict whether to select a candidate, and use data from the selection procedure when humans selected candidates, then there might be a bias in the manual selection procedure already, which an AI system will reproduce.
AI system training	While training (programming) AI models, AI contributors must carefully understand the system its mechanisms, analyze results on fairness and correct bias where needed.	Bias might come from poorly defined constraints, bad decisions, wrong requirements. This can be counteracted by putting in place oversight processes Ensure that individuals and minority groups are free from bias. Example: Not hiring qualified woman or minorities. Positives and negatives resulting from AI should be evenly distributed, avoiding to place vulnerable demographics in a position of greater vulnerability. Example: Crime detection system that differentiates based on skin color or migration status.
AI system industrialisation	While industrializing AI models, AI contributors must ensure that the AI’s system is fair and non-discriminative. Feedback loops should be put in place when the model is deployed and proper actions should be taken when there is a signal of discrimination.	The output of an AI system should be continuously monitored on potential bias and discrimination. Implement feedback loops to detect / anticipate / prevent drifts in the data, to prevent new bias in the data, model’s loss of performance, through the use of monitoring metrics Implement fallback plans to prevent damages. Examples: alerting, switch from statistical to rule-based procedures If harm occurs, AI systems must provide users with effective redress, or effective remedy if data practices are no longer aligned with human beings’ individual or collective preferences. AI contributors must respect high standards of accountability.

The steps in the aforementioned table should be supporting the AI contributor in their project. However, some of these guidelines are subject to human opinion and unfairness might be hard to detect. Therefore, several statistical metrics exist that might help in the identification of discrimination and correction of it. The following table discusses these metrics and shows already existing packages for their implementation.

Identification metrics and ways for correcting discrimination bias and unfairness

Topic	Guideline	Metrics
Identification	Quantify the risk of breach of fairness on identified sensitive populations, through the use of metrics	One might use some of the following metrics for identification of discrimination: Statistical parity difference: the difference of the rate of favorable outcomes received by the unprivileged group to the privileged group. Equal opportunity difference: the difference of the true positive rates between the privileged and unprivileged groups. Average odds difference : the average difference of false positive rates (false positives/negatives) and true positive rate (true positives/positives) between unprivileged and privileged groups. Disparate impact : the ratio of favorable outcome for the unprivileged group to that of the privileged group. Theil index : measures the inequality in benefit allocation for individuals. For a
Correction	After identification of bias, one might correct the bias through the use of algorithms	As an example, one might use some of the following methods for correction of the bias: Reweighting: use to mitigate bias in training data. Modifies weights of different training examples. Optimized pre-processing: mitigates bias in training data. Modifies training data features and labels. Adversarial debiasing: mitigates bias in classifiers. Uses adversarial techniques to maximize accuracy and reduce evidence of protected attributes in predictions. Reject option based classification: mitigates bias in predictions. Changes predictions from a classifier to make them fairer.
Already existing solutions (packages)	Some open sources packages are created that might help in detection & correction of unfairness in an AI system	Please have a look at the following: AI Fairness 360 by IBM Ethical AI Toolkit (Github):

Appendix - Recommendations from the EU

Below are the recommendations directly reported from EU.