How COVID-19 revealed 3 critical AI procurement blindspots that could put lives at risk

covid-19_

(CDC, Unsplash)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Eddan Katz, Project Lead, Protocol Design Networks, World Economic Forum & Conor Sanchez, Project Specialist – L1, World Economic Forum


  • The rush to procure AI technologies during COVID-19 has revealed key governance gaps.
  • Such gaps, if not addressed, could lead to a range of problems, from incorrect diagnoses to eroded public trust.

COVID-19’s global impact has shown the role interconnectedness plays in international cooperation and how outdated technologies are obstacles to effective policy making.

As a result, many entities and governments have found themselves rushing to catch up – identifying, evaluating, and procuring reliable solutions powered by artificial intelligence (AI).

Finding the right AI solutions is an urgent task complicated by the obligation of the public sector to ensure a process guided by fairness, accountability, and transparency (automated processes are especially vulnerable to exacerbating systemic racism and widening existing inequities).

Steps governments have already taken to procure AI have already exposed critical governance gaps. Bridging those gaps will be key to harnessing AI’s transformative potential and guiding economies through a recovery that does not sacrifice privacy and compromise public health.

What is the World Economic Forum doing about the coronavirus outbreak?

Responding to the COVID-19 pandemic requires global cooperation among governments, international organizations and the business community, which is at the centre of the World Economic Forum’s mission as the International Organization for Public-Private Cooperation.

Since its launch on 11 March, the Forum’s COVID Action Platform has brought together 1,667 stakeholders from 1,106 businesses and organizations to mitigate the risk and impact of the unprecedented global health emergency that is COVID-19.

The platform is created with the support of the World Health Organization and is open to all businesses and industry groups, as well as other stakeholders, aiming to integrate and inform joint action.

As an organization, the Forum has a track record of supporting efforts to contain epidemics. In 2017, at our Annual Meeting, the Coalition for Epidemic Preparedness Innovations (CEPI) was launched – bringing together experts from government, business, health, academia and civil society to accelerate the development of vaccines. CEPI is currently supporting the race to develop a vaccine against this strand of the coronavirus.

Here are a few examples of governance opportunities in AI procurement and use:

Chatbots

Many governments have integrated chatbots into their websites to prevent the volume of health questions from the public from overwhelming their resources. When inquiries from the general public overwhelm the staff of government agencies, the use of chatbots by government entities help manage customer service more quickly and more effectively. By responding to common questions and escalating the more complicated ones, significant time is saved and citizens needs are better met.

Still, many of these technologies lack tools and processes to minimize risks. Measures are typically not in place to ensure information accuracy, implement privacy protection or effectively train models agile enough to track the most current knowledge about this rapidly changing disease. This can result in ineffective information dissemination, which at best, does a disservice to the public. At worst, it leads to a misdiagnosis of symptoms, potentially putting thousands or more lives at risk. Governments must put into action plans that help resolve questions over whose voice to amplify and how accurate the information is that is being shared.

 

Diagnostics

Healthcare facilities around the world have been turning to AI systems to spot infections either through people’s voices or with chest x-rays. These tools have significantly helped overburdened hospitals manage the drastic increase in patients. Yet there is little guidance on how to evaluate how well the tools work. As a result, facilities risk making matters worse by incorrectly diagnosing patients.

Governance experts have urged hospitals not to weaken their regulatory protocols and to avoid long-term contracts without doing their due diligence. There is an immediate need for policy frameworks that operationalize the assessment of AI systems prior to being used. One recent report analysing dozens of computer models to diagnose and treat COVID-19 exposed how all models had been trained with unfit and insufficient data. Such assessments should force officials to ask critical questions about risk-mitigation processes, performance and accuracy, and ensure there is robust external oversight of systems through an independent third party.

Tracking the spread of the virus

Dozens of governments have proposed or deployed tracking tools that monitor the virus’ spread person to person. From Asia to Europe, a number of public sector entities have teamed up with the private sector to use AI-based services to monitor the spread of the virus. This includes an array of contact tracing apps that aim to accomplish what public officials have long done manually by interviewing those infected and one-by-one tracking down their recent contacts.

The privacy concerns over how this data may be used in different contexts and by whom suggest that asking the right questions now can help anticipate the inevitable data protection issues later. Establishing processes for thorough documentation can help to assess problems when they do occur and ideally help avoid the harms from taking place. One form of response gaining traction has been the promotion of internal evaluation tools in which companies embrace the multi-stakeholder approach to design an internal assessment questionnaire. Examples include IBM Research’s FactSheets, Google’s framework for internal algorithmic auditing, and Microsoft’s list of AI Principles and the tools to apply them internally. Over time, and if scaled appropriately, these approaches might ensure the responsible deployment of AI in a variety of applications.

“The window of opportunity is brief to establish a set of actionable procurement guidelines that enable good decision-making for the future.”

—Conor Sanchez

Looking ahead

Even before COVID-19, calls for regulation of AI had dramatically increased while the number of AI Ethics principles also proliferated. COVID-19 has accelerated the need to operationalize these principles. In order to help practitioners navigate these challenges, the World Economic Forum’s AI and Machine Learning Platform created the Procurement in a Box package, which aims to unlock public sector adoption of AI through government procurement. The tools released to the general public this week were developed after nearly two years of consultations with procurement officers and private sector companies, several rounds of solicitations of draft comment, comprehensive workshops, individual interviews and international pilots.

The Procurement in a Box effort seeks to operationalize many norm-setting proposals by helping shape the ethical and responsible use of technology. It prioritizes the practical application of these norms in multiple ways, including universalizing the commonalities across different jurisdictions while respecting the differences in law and culture, as well as stages of development of their AI innovation ecosystems.

Such tools will be increasingly relevant for countries globally at various levels, and timely in their utility. The Forum’s AI procurement initiative has notably taken shape at a time of flux, when the social contract of trust between the government, its citizens, and their industries is in suspension. The COVID-19 global crisis has underscored the need for responsibility in innovation and the ethical use of technology. As we embark on this Great Reset, the rules and standards of governance norms are not yet in place. The window of opportunity is brief to establish a set of actionable procurement guidelines that enable good decision-making for the future.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

REACT-EU: EU support to mitigate immediate effects of the COVID-19 crisis

Korea must enhance detection and reinforce sanctions to boost foreign bribery enforcement

Draghi hands over to banks €77.7 billion more

Here’s the secret to financing a greener future

Why EU’s working and unemployed millions remain uncertain or even desperate about their future

Companies that put employees first perform better

38th ACP-EU Assembly: dialogue on cooperation challenges in Kigali

Privatization as a symptom of health inequity

WEF Davos 2016 LIVE: “If we do not do properly the Paris agreement, then all 16 remaining goals will be undermined”, UN Secretary General Ban Ki-moon cautions from Davos

To beat hunger and combat climate change, world must ‘scale-up’ soil health – UN

In visit to hurricane-ravaged Bahamas, UN chief calls for greater action to address climate change

Trade with the United States: Council authorises negotiations on elimination of tariffs for industrial goods and on conformity assessment

Iran: BBC and other broadcast journalists harassed; families threatened – UN experts

Why developing new antibiotics is a matter of life and death

7 shocking statistics that show the cost of corruption

Lithuania finds the ways to maintain its energy security

Migration crisis update: The “Habsburg Empire” comes back to life while EU loses control

Under fire, UN refugee agency evacuates 135 detained in Libya to Niger

State of the Union 2018: The Hour of European Sovereignty

Alarming level of reprisals against activists, human rights defenders, and victims – new UN report

Infrastructural and system barriers to Universal Health Coverage: get in my patient’s shoes

Some truths about the coronavirus, according to the World Health Organization

Visa-free access to the EU for UK nationals and to the UK for EU nationals

Improving Italy’s capital market will boost growth opportunities for Italian companies and savers

Cyber-Risk Assessments: the vaccine for companies in the Fourth Industrial Revolution

These Asian economies invested in their people – and it paid off

Progress made at COP25, despite lack of agreement to increase climate ambition

4 innovation hotspots in the Arab world

This Scottish forest is both a home for wildlife and a boost for the local economy

EU mobilises immediate support for its Western Balkan partners to tackle coronavirus

Boris Johnson’s no-deal Brexit to differ when issued from 10 Downing St.

“There are many converging visions and interests between the One Belt One Road initiative and the Juncker Investment Plan”, Ambassador Yang of the Chinese Mission to EU highlights from Brussels

Humanitarian aid: EU steps up support in Nigeria for conflict victims

The EU Commission openly repudiates the austere economic policies

No hard drivers in sight to remodel the stagnating affairs of the EU

Chart of the day: These countries have the highest share of electric vehicles

Successful carbon removal depends on these 3 conditions

ISIS fighters fleeing Mosul for Syria can topple Assad. Why did the US now decide to uproot them from Iraq?

“Is Europe innovative? Oh, Yes we are very innovative!”, Director General of the European Commission Mr Robert-Jan Smits on another Sting Exclusive

UN must bring more women police officers into the fold to be effective – UN peacekeeping official

The Free World Experience Report – LGBTQI+ health on the spot

Trump badly cornered at home by agribusiness and steel consumer lobbies: Trade

ZTE @ MWC14: ZTE excels in all areas at this year’s Mobile World Congress

Egypt: The road to hell paved with western advices for democracy

UN launches new framework to strengthen fight against terrorism

Syria war: executions condemned as violence continues ‘on both sides’ of border with Turkey

This UK footballer just won free school meals for kids in the summer holidays

MWC 2016 LIVE: Zuckerberg warns mobile industry not to ignore the unconnected

CO2 emissions around the world

COP21 Breaking News: China has promised to cut emissions from its coal power plants by 60% by 2020

Rising landmine blast toll in Afghanistan highlights long-term care needs of survivors

Topic: Mental Health in times of pandemic: What can each individual do to lessen the burden?

25 years on from landmark conference, millions of women and girls still in danger: UN deputy chief

Nigeria: UN chief ‘appalled’ by killing of aid worker; calls for release of remaining hostages

How a new approach to meat can help end hunger

Why social working cultures are happier and more productive

EU: Divided they stand on immigration and Trump hurricanes

TTIP: why it is worth not to pull the covers over your head?

Leading Palestinian legislator calls for ‘new international engagement’ in two-state solution

UN mosquito sterilization technology set for global testing, in battle against malaria, dengue

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s