Emerging legislation on commercial uses of facial recognition shows the work ahead

facial

(Credit: Unsplash)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Karen Silverman, Global AI Council Member, CEO and Founder, The Cantellus Group & Andrea Ortega, Technology and privacy attorney, LLM Law & Technology from UC Berkeley Law


  • Emerging facial recognition legislation shows the need for more clarity and standardization.
  • Developers and users of FRS technologies both should engage in the policy debate to define the coming standards.

Facial recognition has emerged as a powerful biometric technology, both in practice and in our collective imagination. Revelations about its use in the public domain and by law enforcement have fueled discussion about ethical concerns and corresponding legislative efforts to ban or limit its use.

Moreover, IBM has recently announced that it will “no longer offer general purpose IBM facial recognition or analysis software,” and Microsoft and Amazon have called for a year-long pause on police use of their facial recognition technologies and for Congress to take up ethical-standard setting.

These latest developments underscore that the time has come to figure out appropriate uses, limits and safeguards on use-case level challenges – both with the state of the technologies themselves, and how they are deployed within and by people in specific industries and settings.

What is the World Economic Forum doing about the Fourth Industrial Revolution?

The World Economic Forum was the first to draw the world’s attention to the Fourth Industrial Revolution, the current period of unprecedented change driven by rapid technological advances. Policies, norms and regulations have not been able to keep up with the pace of innovation, creating a growing need to fill this gap.

The Forum established the Centre for the Fourth Industrial Revolution Network in 2017 to ensure that new and emerging technologies will help—not harm—humanity in the future. Headquartered in San Francisco, the network launched centres in China, India and Japan in 2018 and is rapidly establishing locally-run Affiliate Centres in many countries around the world.

The global network is working closely with partners from government, business, academia and civil society to co-design and pilot agile frameworks for governing new and emerging technologies, including artificial intelligence (AI), autonomous vehicles, blockchain, data policy, digital trade, drones, internet of things (IoT), precision medicine and environmental innovations.

Learn more about the groundbreaking work that the Centre for the Fourth Industrial Revolution Network is doing to prepare us for the future.

Want to help us shape the Fourth Industrial Revolution? Contact us to find out how you can become a member or partner.

Differences across emerging legislation proposals
US state legislative proposals on commercial uses of facial recognition services (FRS) reveal how much work lies ahead in getting to laws that effectively address human rights and civil liberties (beyond privacy and security interests) and also that effectively instruct and engage business in doing so.

Washington State has made progress on its proposed Washington Privacy Act (WPA), the first of its kind to tackle specifically commercial uses of FRS and to import nondiscrimination requirements beyond traditional privacy ones. California was close on Washington’s heels with Assembly Bill 2261, which would have imposed restrictions on certain commercial FRS activities, but was just blocked as part of a larger legislative effort that would likewise have enabled certain law enforcement uses.

These emerging proposals, however, reveal varying standards highlighting the difficulty of legislating in this arena, the work remaining to define clear, consistent and effective standards, and the uncertainties that will confront businesses as they proceed (at least in the short and medium term).

Existing proposals, for instance, advance different approaches to non-discrimination obligations. The WPA requires third-party testing for detecting “accuracy and unfair performance differences across distinct subpopulations.” It does not, however, define what comprises “unfair performance” and it is not clear whether it extends to guarding against the use of FRS to violate basic civil rights beyond the access to goods and services. (California’s option did offer such guidance, but other laws from other localities in the future may not account for these issues leaving gaps to be bridged.)

Differences extend to issues of oversight as well. The WPA requires controllers to ensure “meaningful human review” and test the FRS in operational conditions before deployment, specifically for FRS intended to make decisions that produce legal or similarly significant effects on consumers. It does not define what might comprise a “meaningful human review” of FRS. California’s measure attempted to address this issue, including review or oversight by one or more individuals who are trained and who “are ultimately responsible for making decisions based, in whole or in part, on the output of a FRS.” Still, varying definitions will complicate matters for businesses trying to comply when measures that don’t align on key definitions.

What businesses can do now
With so much in development, businesses have the opportunity to contribute to the policy debate and to define the coming standards. The World Economic Forum is helping tackle this challenge with its multistakeholder approach and actionable governance framework and is calling for engagement from businesses and stakeholders. A current project, Responsible Limits on Facial Recognition Technology, will help develop a governance framework to ensure safe and trustworthy use of FRS technology. As these and other initiatives take shape, there are some valuable steps that firms can take now:

1. Think about, and beyond, the controller/processor framework

The distinction between controllers and processors, borrowed from the EU’s GDPR, lies at the heart of the WPA and California’s now defunct AB 2261. As currently drafted, any business’ status as one or the other (or neither) will determine its privacy and — importantly — its nondiscrimination obligations. Indeed, the proposals seem to assume that businesses using FRS for identification, verification and persistent tracking of individuals will be controllers, and that businesses developing or supplying FRS to controllers will be processors so long as they process personal data — collect, use, store, or analyze, among other operations — following instructions of the controller.

In practice, however, businesses using or supplying FRS may not always fall into the controller/processor framework, and their roles may change over time. For instance, an FRS supplier may be a controller from the start, or initially operate as a processor but end up a controller if it starts processing personal data outside of the controller’s instructions (e.g. when training a generic AI tool). Others might initially not process personal data but at some point start processing an FRS customer/controller’s specific customer data (e.g. when implementing that tool).

Moreover, FRS developers or suppliers that do not process personal data (including persistent tracking) would remain outside of the legislation’s scope and, therefore, would neither be subject to the privacy nor the nondiscrimination obligations. Under this scenario, a controller would have to contractually require such FRS developer to nonetheless submit in writing to these nondiscrimination obligations — in particular, to third-party testing and implementing mitigation plans — in order to ensure compliance with its own nondiscrimination obligations. In addition, FRS developers or suppliers may also nonetheless wish to contractually require FRS users to comply with applicable federal or state nondiscrimination laws.

Legislation may someday address FRS uses beyond this controller/processer construct, and that will be clarifying. Meanwhile, however, businesses should keep in mind that a) their handling of personal data increasingly defines new obligations, and b) existing nondiscrimination requirements may already apply to their digital activities.

2. Anticipate the most likely requirements and put a diverse team in charge

Under these proposals, both controllers and processors would be subject to broad nondiscrimination requirements through cross-contracting, human oversight, testing and audit requirements. This predominantly human-at-the-end-of-the-loop approach, however, might not be sufficient to address discrimination concerns around the use of FRS, especially as currently drafted. They are, in any case, a starting point.

Businesses contemplating the use of FRS can start building out robust, trustworthy systems by focusing on the similarities between these proposals, including their calls for human oversight, pre-release testing for bias and harmful impacts, and regular audits (and by thinking upfront about their overall AI strategies and trust standards). More broadly, the work to build trustworthy AI systems should be a fundamental part of all the different phases of the product cycle, embedding standards into the design and operation of FRS systems and into the teams operating them. To do this well, at the outset, business needs to consider the different uses in depth, the quality and impacts of these technologies — in particular at the management and oversight levels —and ensure that the responsible teams represent a diversity of lived experience and expertise.

Looking ahead

The differences raised between even just these two legislative approaches would result in a lack of clarity for businesses attempting to standardize or anticipate compliance protocols.

Still, legislation and regulation are inevitable in time Congress could soon enact standards and limitations for specific uses. California will almost certainly — sooner or later — try again to legislate commercial uses of this technology. And Illinois and Texas could also attempt to amend their biometrics privacy laws to incorporate nondiscrimination requirements. Likewise, certain cities (including New York City and San Francisco) are contemplating expanded bills of FRS on private use. Likewise, the Brookings Institute has just issued a report that identifies preemption as one major impediment to regulation in the US, and buried in this issue are inconsistencies among legislative proposals as well as the Constitutional question regarding the balance of federal and State authorities.

As proposals take shape, it is critical for developers and users of these technologies to engage in the policy debate to define the coming standards (and to develop and test their own trust standards). Not doing so could risk longer and more costly market suspensions, business interruptions and reputational damage on the backend — well before any law has anything to say about it.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Building climate resilience and peace, go hand in hand for Africa’s Sahel – UN forum

Myanmar military committed ‘routine, systematic’ sexual violence against ethnic minorities, UN experts find

London is becoming the world’s first National Park City

These are the places with the most climate change deniers

Amending Guatemala ‘reconciliation law’ would lead to unjust amnesty, warns Bachelet

What next for Europe? Three (completely) different Davos views

Afghanistan: UN mission welcomes new polling dates following election delays

Can privatisation be the panacea for the lack of growth in Europe?

Office workers in these economies clock up the most extra hours

A Sting Exclusive: “Asia-Pacific response to COVID-19 and climate emergency must build a resilient and sustainable future”, by the United Nations Under-Secretary-General

‘Being open about my mental health created a better work culture’

Actions not words: what was promised at the UN’s landmark climate summit?

Localized microfactories – the new face of globalized manufacturing

Chart of the day: These countries have the largest carbon footprints

Mexico: Helping refugees go into business, a ‘win-win situation’, says UNHCR’s Grandi

Commission’s feeble response to financial benchmarks fraud

Migration: Better travel safe than sorry

Universal Health Coverage will ‘drive progress’ on 2030 Development Agenda

European Council: Choosing new leaders for the EU betrays efforts for a wider arrangement

The 28 EU leaders don’t touch the thorny issues

How the United States is falling in love with secondhand clothes

Wednesday’s Daily Brief: Libya ‘war crime’ attack, Sudan, Myanmar rights violations continue, ‘xenophobia’ in Assam, South Sudan update

Steps taken to end Saudi ‘guardianship’ system for women, ‘encouraging’ start

25 years on from landmark conference, millions of women and girls still in danger: UN deputy chief

Partner countries get €3bn in loans to prop up economies affected by pandemic

Ukraine’s Poroshenko implicates NATO in his duel with Putin

This is the life of a refugee: the constant destruction and construction of dreams every day

Global leaders and companies pledge to reduce the gender pay gap by 2030

EU to present a “hefty” exit bill to the UK moments before Brexit negotiations

Prisoner executions in Belarus ‘simply unacceptable’, says UN rights body

Four in five adolescents failing to exercise for even 60 minutes a day, UN health agency warns

Central African Republic militia leader and football executive, transferred to ICC

Deeper reforms in Korea will ensure more inclusive and sustainable growth

The vehicles of our future

Coronavirus update: Countries urged to fight ‘controllable’ pandemic

Philippe de Backer of ALDE at European Business Summit 2015 stresses: “Reinvent your business”

The Red Cross’s health chief explains how business must respond to coronavirus

What is the Coral Triangle?

The Challenger Within – Mental Health In Romania During Lockdown

Refugee crisis update: EU seeks now close cooperation with Africa while Schulz is shocked to witness live one single wreck full of immigrants

We must prevent a post-COVID ‘carmageddon’. Here’s how

Millions of Bangladeshi children at risk from climate crisis, warns UNICEF

Bahamas: ‘Clock is ticking’ to help those who lost everything in Hurricane Dorian, says UN

UN chief welcomes prisoner exchange between the Russia and Ukraine

World is closer than ever to seeing polio disappear for good

Major UN aid operation for 650,000 gets underway across Syria-Jordan border

Theresa May expresses her optimism about Britain’s economic success while UK business outlook seems ominous

Draghi: Germany has to spend if Eurozone is to exit recession

Theresa May’s global Britain against Philip Hammond’s Brexit fog

Can we balance conservation and development? Science says yes

Commission challenges Council over EU 2014 budget

Why we need to rethink geo-economics to beat climate change

Hospitals among seven health centres attacked in Syria’s north-east

ITU Telecom World 2017: exploring smart digital transformation

Guterres underlines climate action urgency, as UN weather agency confirms record global warming

Human Rights Council election: 5 things you need to know about it

EU to fail 2050 Green targets due to lack of European citizens’ engagement

G20 LIVE: “Re-envisioning the economy to enable women to reach their full potential” live from Antalya Turkey

India’s Largest Entrepreneurship Event is Back! (23-24th August 2016)

The Japanese have a word to help them be less wasteful – ‘mottainai’

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s