Emerging legislation on commercial uses of facial recognition shows the work ahead

facial

(Credit: Unsplash)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Karen Silverman, Global AI Council Member, CEO and Founder, The Cantellus Group & Andrea Ortega, Technology and privacy attorney, LLM Law & Technology from UC Berkeley Law


  • Emerging facial recognition legislation shows the need for more clarity and standardization.
  • Developers and users of FRS technologies both should engage in the policy debate to define the coming standards.

Facial recognition has emerged as a powerful biometric technology, both in practice and in our collective imagination. Revelations about its use in the public domain and by law enforcement have fueled discussion about ethical concerns and corresponding legislative efforts to ban or limit its use.

Moreover, IBM has recently announced that it will “no longer offer general purpose IBM facial recognition or analysis software,” and Microsoft and Amazon have called for a year-long pause on police use of their facial recognition technologies and for Congress to take up ethical-standard setting.

These latest developments underscore that the time has come to figure out appropriate uses, limits and safeguards on use-case level challenges – both with the state of the technologies themselves, and how they are deployed within and by people in specific industries and settings.

What is the World Economic Forum doing about the Fourth Industrial Revolution?

The World Economic Forum was the first to draw the world’s attention to the Fourth Industrial Revolution, the current period of unprecedented change driven by rapid technological advances. Policies, norms and regulations have not been able to keep up with the pace of innovation, creating a growing need to fill this gap.

The Forum established the Centre for the Fourth Industrial Revolution Network in 2017 to ensure that new and emerging technologies will help—not harm—humanity in the future. Headquartered in San Francisco, the network launched centres in China, India and Japan in 2018 and is rapidly establishing locally-run Affiliate Centres in many countries around the world.

The global network is working closely with partners from government, business, academia and civil society to co-design and pilot agile frameworks for governing new and emerging technologies, including artificial intelligence (AI), autonomous vehicles, blockchain, data policy, digital trade, drones, internet of things (IoT), precision medicine and environmental innovations.

Learn more about the groundbreaking work that the Centre for the Fourth Industrial Revolution Network is doing to prepare us for the future.

Want to help us shape the Fourth Industrial Revolution? Contact us to find out how you can become a member or partner.

Differences across emerging legislation proposals
US state legislative proposals on commercial uses of facial recognition services (FRS) reveal how much work lies ahead in getting to laws that effectively address human rights and civil liberties (beyond privacy and security interests) and also that effectively instruct and engage business in doing so.

Washington State has made progress on its proposed Washington Privacy Act (WPA), the first of its kind to tackle specifically commercial uses of FRS and to import nondiscrimination requirements beyond traditional privacy ones. California was close on Washington’s heels with Assembly Bill 2261, which would have imposed restrictions on certain commercial FRS activities, but was just blocked as part of a larger legislative effort that would likewise have enabled certain law enforcement uses.

These emerging proposals, however, reveal varying standards highlighting the difficulty of legislating in this arena, the work remaining to define clear, consistent and effective standards, and the uncertainties that will confront businesses as they proceed (at least in the short and medium term).

Existing proposals, for instance, advance different approaches to non-discrimination obligations. The WPA requires third-party testing for detecting “accuracy and unfair performance differences across distinct subpopulations.” It does not, however, define what comprises “unfair performance” and it is not clear whether it extends to guarding against the use of FRS to violate basic civil rights beyond the access to goods and services. (California’s option did offer such guidance, but other laws from other localities in the future may not account for these issues leaving gaps to be bridged.)

Differences extend to issues of oversight as well. The WPA requires controllers to ensure “meaningful human review” and test the FRS in operational conditions before deployment, specifically for FRS intended to make decisions that produce legal or similarly significant effects on consumers. It does not define what might comprise a “meaningful human review” of FRS. California’s measure attempted to address this issue, including review or oversight by one or more individuals who are trained and who “are ultimately responsible for making decisions based, in whole or in part, on the output of a FRS.” Still, varying definitions will complicate matters for businesses trying to comply when measures that don’t align on key definitions.

What businesses can do now
With so much in development, businesses have the opportunity to contribute to the policy debate and to define the coming standards. The World Economic Forum is helping tackle this challenge with its multistakeholder approach and actionable governance framework and is calling for engagement from businesses and stakeholders. A current project, Responsible Limits on Facial Recognition Technology, will help develop a governance framework to ensure safe and trustworthy use of FRS technology. As these and other initiatives take shape, there are some valuable steps that firms can take now:

1. Think about, and beyond, the controller/processor framework

The distinction between controllers and processors, borrowed from the EU’s GDPR, lies at the heart of the WPA and California’s now defunct AB 2261. As currently drafted, any business’ status as one or the other (or neither) will determine its privacy and — importantly — its nondiscrimination obligations. Indeed, the proposals seem to assume that businesses using FRS for identification, verification and persistent tracking of individuals will be controllers, and that businesses developing or supplying FRS to controllers will be processors so long as they process personal data — collect, use, store, or analyze, among other operations — following instructions of the controller.

In practice, however, businesses using or supplying FRS may not always fall into the controller/processor framework, and their roles may change over time. For instance, an FRS supplier may be a controller from the start, or initially operate as a processor but end up a controller if it starts processing personal data outside of the controller’s instructions (e.g. when training a generic AI tool). Others might initially not process personal data but at some point start processing an FRS customer/controller’s specific customer data (e.g. when implementing that tool).

Moreover, FRS developers or suppliers that do not process personal data (including persistent tracking) would remain outside of the legislation’s scope and, therefore, would neither be subject to the privacy nor the nondiscrimination obligations. Under this scenario, a controller would have to contractually require such FRS developer to nonetheless submit in writing to these nondiscrimination obligations — in particular, to third-party testing and implementing mitigation plans — in order to ensure compliance with its own nondiscrimination obligations. In addition, FRS developers or suppliers may also nonetheless wish to contractually require FRS users to comply with applicable federal or state nondiscrimination laws.

Legislation may someday address FRS uses beyond this controller/processer construct, and that will be clarifying. Meanwhile, however, businesses should keep in mind that a) their handling of personal data increasingly defines new obligations, and b) existing nondiscrimination requirements may already apply to their digital activities.

2. Anticipate the most likely requirements and put a diverse team in charge

Under these proposals, both controllers and processors would be subject to broad nondiscrimination requirements through cross-contracting, human oversight, testing and audit requirements. This predominantly human-at-the-end-of-the-loop approach, however, might not be sufficient to address discrimination concerns around the use of FRS, especially as currently drafted. They are, in any case, a starting point.

Businesses contemplating the use of FRS can start building out robust, trustworthy systems by focusing on the similarities between these proposals, including their calls for human oversight, pre-release testing for bias and harmful impacts, and regular audits (and by thinking upfront about their overall AI strategies and trust standards). More broadly, the work to build trustworthy AI systems should be a fundamental part of all the different phases of the product cycle, embedding standards into the design and operation of FRS systems and into the teams operating them. To do this well, at the outset, business needs to consider the different uses in depth, the quality and impacts of these technologies — in particular at the management and oversight levels —and ensure that the responsible teams represent a diversity of lived experience and expertise.

Looking ahead

The differences raised between even just these two legislative approaches would result in a lack of clarity for businesses attempting to standardize or anticipate compliance protocols.

Still, legislation and regulation are inevitable in time Congress could soon enact standards and limitations for specific uses. California will almost certainly — sooner or later — try again to legislate commercial uses of this technology. And Illinois and Texas could also attempt to amend their biometrics privacy laws to incorporate nondiscrimination requirements. Likewise, certain cities (including New York City and San Francisco) are contemplating expanded bills of FRS on private use. Likewise, the Brookings Institute has just issued a report that identifies preemption as one major impediment to regulation in the US, and buried in this issue are inconsistencies among legislative proposals as well as the Constitutional question regarding the balance of federal and State authorities.

As proposals take shape, it is critical for developers and users of these technologies to engage in the policy debate to define the coming standards (and to develop and test their own trust standards). Not doing so could risk longer and more costly market suspensions, business interruptions and reputational damage on the backend — well before any law has anything to say about it.

the sting Milestones

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

COVID has shown we can be creative under pressure: Stockholm’s mayor on harnessing a city’s people power

Russia: MEPs deplore military build-up, attack in Czechia and jailing of Navalny

Antitrust: Commission accepts commitments by Transgaz to facilitate natural gas exports from Romania

Business growth is key to post-pandemic recovery

This UK campaign is working to close the politics gender gap

Global Talent – Professional Internships

What’s needed to ensure maternal health for women in vulnerable populations

‘No hope’ global development goals can be achieved without women, says UN Assembly President

Disease slashing global meat output, cereals boom, bananas under watch: FAO

Why digital inclusion must be at the centre of resetting education in Africa

We’re all in the same boat on the SDGs. Here’s how we steer a course

Social inclusion: how much should young people hope from the EU? 

Libya stands at a ‘critical juncture’, UN mission head tells Security Council

EU and U.S. castigate Facebook on Cambridge Analytica scandal as citizens’ data privacy goes down the drain again

Blockchain will make sure green pledges aren’t just greenwash: a new initiative by young leaders at the World Economic Forum

Trade wars won’t fix globalization. Here’s why

Smart cities must pay more attention to the people who live in them

5 key lessons for energy transition from COVID-19 recovery

Why and how did ISIS and Muslim fundamentalism gain momentum this year?

Europe eyes to replace US as China’s prime foreign partner

G20 LIVE: G20 leaders reaffirm OECD’s role in ensuring strong, sustainable and inclusive growth

Sustainable Finance: Commission welcomes the adoption by the European Parliament of the Taxonomy Regulation

ZTE @ MWC14: ZTE excels in all areas at this year’s Mobile World Congress

Providing mental health during pandemic times

Stronger European Border and Coast Guard to secure EU’s borders

Robots and chatbots can help alleviate the mental health epidemic

Digital tokens could transform the economies of the Middle East and North Africa – if the governance keeps up

Future of EU farming: MEPs push for modern common policy with fair funding

Funding boost for sustainable development data agreed at UN conference

The European Parliament declares climate emergency

WHO working to save lives following powerful earthquake in Albania

A renewed agenda for Research and Innovation: Europe’s chance to shape the future

As Dubai switches on its first 5G, what is all the fuss about?

Why Microsoft is a regular to Almunia’s

Trade protectionism and cartels threaten democracy

What the Fifth Industrial Revolution is and why it matters

Third wave of COVID-19: public policy challenges

We must work together to build a new world order. This is how we can do it

Sovereign wealth funds could increase equality in a post-COVID world

Conflict, climate change among factors that increase ‘desperation that enables human trafficking to flourish’, says UN chief

How a teen refugee survived a shipwreck and saved a baby’s life

Is Europe ready to cooperate with the rest of the world? Can Germany change its selfish policies?

EU attempts to make new deal with Turkey as relations deteriorate

UN chief of peace operations honours fallen Chadian ‘blue helmets’ serving in northern Mali

The Mental Health Hero in You

Germany to help China in trade disputes with Brussels

Veteran public official from Portugal elected to lead UN migration agency

These 5 charts show our shifting behaviour around coronavirus

Is deflation a real danger for Eurozone?

How to change the world at Davos

Countries must invest at least 1% more of GDP on primary healthcare to eliminate glaring coverage gaps

Debunked: 5 myths about the future of work

How income-sharing agreements can improve access to education

Why will Paris upcoming “loose” climate change agreement work better than the previous ones?

A day in the life of a refugee: why should we care?

Digital roles top the list of jobs on the rise in 2021

10 predictions for the global economy in 2019

ECB is about to lend trillions to banks

Foreign Investment Screening: new European framework to enter into force in April 2019

The EU accuses Russia of bullying Ukraine to change sides

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s