What the EU is doing to foster human-centric AI

(Credit: Unsplash)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Saverio Puddu, Managing Associate, Linklaters

  • The EU’s first draft Regulation on AI is part of a wider effort to develop human-centric AI by eliminating mistakes and biases to ensure it is safe and trustworthy.
  • The regulation includes requirements to minimise the risk of algorithmic discrimination, in particular in relation to the quality of data sets used for the development of AI systems.
  • It also introduces human oversight of certain AI systems to prevent or minimise risks to health, safety or fundamental rights that may emerge when an AI system is used.

Far from being science fiction, artificial intelligence is already part of our lives. From self-driving vehicles to virtual personal assistants, AI is a reality.

Beyond simplifying our lives, AI promises to bring an array of economic and social benefits for society. From healthcare to mobility, across public sector and finance, AI is a fast-evolving technology that can solve some of the world’s biggest challenges. It can lead to social and environmental beneficial outcomes and, at the same time, support any kind of businesses by helping them in better knowing and engaging with their customers, improving prediction, speeding up production, and optimising operations and resource allocation.

However, together with countless socio-economic benefits, it is not surprising that AI can also introduce new risks into our society and that, among those challenges, ethical implications are probably the most subtle and insidious.

To enable scientific breakthroughs and to ensure that AI technologies are at the service of all citizens, improving their lives while respecting their rights, the European Commission proposed the first draft Regulation on AI.

The draft legislation is only a part of a major European strategy for AI presented by the European Commission in 2018 to address the opportunities and challenges of AI and promoting, at the same time, European values. However, pursuing the twin objective of promoting the development – and deployment – of AI and of addressing the risks associated with certain uses of this new technology, implies complex evaluation and difficult ethical and regulatory choices, especially when the risk of discrimination arises.

Not immune to mistakes and biases

AI performs functions that previously could only be done by humans. Consequently, we are progressively subject to decisions taken by – or with the assistance of – AI systems. However, believing that AI is by default immune to bias, and thus not able to discriminate against individuals or groups of individuals, is a dangerous mistake: AI can perpetuate certain biases or even amplify them.

As noted by the European Commission in its 2020 White Paper on AI, bias and discrimination are inherent risks of any societal or economic activity, and AI is not invulnerable to mistakes and biases. Indeed, depending on the data input that is used to train and test AI systems, their outputs can be biased.

For instance, certain AI algorithms deployed for predicting criminal recidivism display gender and racial bias, demonstrating different recidivism prediction probability for women versus men or for nationals versus foreigners.

Similarly, certain AI-based facial recognition tools have demonstrated low errors for determining the gender of lighter-skinned men but high errors in determining gender for darker-skinned, thus displaying gender and racial bias.

Undoubtedly, biased results can lead to breaches of our fundamental rights, from our freedom of expression or association (see the effects of indiscriminate surveillance made by facial recognition systems) to non-discrimination, especially when AI biased decisions are based on sex, racial or ethnic origin, religion, disability, age or sexual orientation (for instance in access to employment or being approved a loan).

AI-biased results and discriminatory effects are mainly due to two factors:

  • the use of low-quality training data sets (e.g. the AI system is trained using mostly data from men leading to suboptimal results in relation to women);
  • the lack of transparency of AI (better known as the “opaqueness of AI”) which makes it difficult to identify possible flaws of the AI system’s design.

The EU approach

To mitigate the risk of erroneous or biased AI-assisted decisions in critical areas, such as healthcare and employment, the European Commission envisages a multifaced approach.

The proposed draft Regulation includes requirements that aim to minimise the risk of algorithmic discrimination, in particular in relation to the quality of data sets used for the development of AI systems, accompanied by obligations for testing, risk management, documentation and human oversight throughout the entire AI systems’ lifecycle.

Indeed, when data is gathered, it may reflect socially constructed biases, or contain inaccuracies, errors and mistakes. Data sets used by AI systems – both for training and operating – may also suffer from the inclusion of inadvertent historic bias which could lead to indirect discrimination. Similarly, the way in which AI systems are developed (e.g. the way in which the programming code of an algorithm is written) may also suffer from bias, often inherent in human developers.

Therefore, training, validation and testing data sets should be subject to appropriate data governance and management practices that should take into consideration:

  • the relevant design choices;
  • the data gathering;
  • the relevant data preparation processing operations;
  • the formulation of relevant assumptions, especially with respect to the information that the data are supposed to measure and represent;
  • the prior assessment of the availability, quantity and suitability of the required data sets;
  • the examination in view of possible biases;
  • the identification of any possible data gaps or shortcomings, and potential remediations.

Furthermore, to prevent outcomes entailing prohibited discrimination, each training, validation and testing data set should be relevant, representative, free of errors and complete. They should also have the appropriate statistical properties, including as regards the individuals or groups of individuals on which AI system is intended to be used, especially to ensure that all relevant dimensions of gender, ethnicity and other possible grounds of prohibited discrimination are appropriately reflected in those data sets.

Interestingly, the proposal also provides that, to the extent it is strictly necessary for the purposes of ensuring AI bias monitoring, detection and correction, AI providers can process special categories of personal data – such as health data and those revealing ethnic origin, political opinions, religious beliefs or sexual orientation – subject to appropriate safeguards. These include technical limitations on the re-use and use of state-of-the-art security and privacy-preserving measures, such as pseudonymisation, or encryption where anonymisation may significantly affect the purpose pursued.

In addition to said ex ante testing obligations, the proposal introduces accountability duties aimed at demonstrating that the AI systems comply with the requirements set out by the proposed legislation. Technical documentation of the AI system should be drawn up before that system is placed on the market and kept up-to date. Similarly, AI systems should be designed and developed with capabilities enabling the automatic recording of events (i.e. logs) while the AI system is operating.

Those obligations are evidently intended to allow ex post controls and to facilitate the respect of fundamental rights by ensuring transparency and traceability of the AI system’s functioning throughout its lifecycle. Indeed, to increase transparency and minimise the risk of bias or error, the European Commission suggests that AI systems should be developed in a manner which allows humans to understand the basis of their actions. For the same purposes, it requires high-risk AI systems to be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users to interpret the system’s output and use it appropriately.

In view of the risks that they may pose, the draft Regulation also introduces human oversight of certain AI systems aimed at preventing or minimising the risks to health, safety or fundamental rights that may emerge when an AI system is used, in particular when such risks persist notwithstanding the application of other above-mentioned requirements.

A difficult balance to strike

With its new legislative proposal, the EU has made clear – once again – that AI comes with both the potential to transform our world for the better and, at the same time, the potential to breach our fundamental rights, such as through gender-based or other kinds of discrimination. However, biases, opacity, complexity, unpredictability and the partially autonomous behaviour of AI systems are only a few of the main factors posing risks to citizens’ rights. With this proposal, the EU places a clear and safe framework for companies to develop and implement AI, facilitating innovation and trust for customers and users.

Striking a balance between the protection of citizens’ freedoms and EU economic competitiveness is a hard choice and requires complex evaluations. As recently stated by Margrethe Vestager, Executive Vice-President for a Europe Fit for the Digital Age: “On Artificial Intelligence, trust is a must, not a nice to have”. Trust requires transparency, transparency requires accountability.

With this draft Regulation, the EU is promoting innovation in the area of AI while supporting the development and uptake of ethical and trustworthy AI across its economy. The strategy adopted by the EU is clear and it places people at the centre of the development of AI, thus designing a human-centric AI.

the sting Milestones

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

To protect trust in the 4IR, we need to become experts in distrust

7 ways for businesses to capture the youth dividend

COP21 Breaking News_03 December: Argentina Accepts KP Amendment

Online government services could change your life. But only if you have access to the internet

Progress toward sustainable development is seriously off-track

Statement by Cecilia Malmström, Member of the EC in charge of Trade, on the successful conclusion of the final discussions on the EU-Japan Economic Partnership Agreement (EPA) – Brussels, 08 Dec 2017. (Copyright: European Union; Source: EC - Audiovisual Service; Photo: Georges Boulougouris)

The EU and Japan seal free trade pact that will cover 30% of global GDP

COVID-19: Commission presents guidelines for border measures to protect health and keep goods and essential services available

Strengthening the EU’s autonomy – Commission seeks input on a new anti-coercion instrument

Unemployment is down across the world’s largest economies

“TTIP can boost the European project”; the Sting reports live from EBS 2015 on TTIP

UN climate summit aims to speed up transition to cleaner, greener future

Anti-Money Laundering: Commission decides to refer Austria, Belgium and the Netherlands to the Court of Justice of the EU for failing to fully implement EU anti-money laundering rules

Security Union: Commission receives mandate to start negotiating international rules for obtaining electronic evidence

EU-UK: A deal synonymous to ‘remain’, England pays the Irish price

Fertilisers/cadmium: Parliament and Council negotiators reach provisional deal

Coronavirus: EU guidance for a safe return to the workplace

Education and Training: where do we stand in 2014?

Remittances could fall by $100 billion because of COVID-19 – here’s why that matters

Bank resolutions and recapitalisations by the ESM may end up politically swayed

Japanese banks to move their European HQ from London to Frankfurt after Brexit

It’s getting harder to move data abroad. Here’s why it matters and what we can do

#TakeYourSeat at the UN Climate Change Conference: a way for all people to join the global conversation

St. Vincent and the Grenadines breaks a record, as smallest ever Security Council seat holder

Access to healthcare: what do we lack?

Desires for national independence in Europe bound by economic realities

How can entrepreneurship tackle the migration crisis in the EU?

How multistakeholder platforms can strengthen our multilateral system

On the Global Day of Parents, UNICEF is urging support for parents to give children ‘the best start in life’

The 4 biggest challenges to our higher education model – and what to do about them

The European Green Deal must be at the heart of the COVID-19 recovery

EU unveils plan to accelerate Capital Markets Union ahead of London’s departure from the bloc

Wednesday’s Daily Brief: mental health, conflict prevention, Ebola in Uganda, Sudan protests, child labour

Consumers to be better protected against misleading and unfair practices

State aid: Commission approves €1 billion Danish and Swedish measure to recapitalise SAS

Irish Presidency: Not a euro more for EU budgets

5 facts you might not know about why forest biodiversity matters

China and China-EU Relations in the New Era

To improve women’s access to finance, stop asking them for collateral

EU Facility for Refugees in Turkey: EUR 400 million Special Measure on education adopted

Why responsible consumption is everyone’s business

The ‘American Dream’ can be best achieved in the Nordic nations, says Finland’s PM

The reason the world showed limited empathy to the Orlando victims

These are the challenges facing India’s most sacred river

European Citizens’ Initiative: Commission decides to register ‘Right to Cure’ initiative

Partner countries get €3bn in loans to prop up economies affected by pandemic

External action: more funds for human rights, development and climate change

Human health – litmus paper for the climate change?

Coronavirus containment is the key, as infections tick up: Tedros

D-Day for Grexit is today and not Friday; Super Mario is likely to kill the Greek banks still today

Sweden must urgently implement reforms to boost fight against foreign bribery

Negotiated two-State solution still ‘the only option’ for Palestine: Guterres

7 innovative projects making cities more sustainable

FROM THE FIELD: ‘Race is on’ to halt biodiversity loss in Indonesia

Help African farmers cope with climate change threats, UN food agency urges

The cost of generating renewable energy has fallen – a lot

Varna (Bulgaria) awarded European Youth Capital 2017

Mobile technology saving lives: How simple technological solutions can improve our health care system

Von der Leyen in Ireland: Our mutual solidarity is here to stay

Coronavirus: EU channels critical support to India via EU Civil Protection Mechanism

Trump ostracized by his party and world elites but still remains in course; how can he do it?

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s