Emerging legislation on commercial uses of facial recognition shows the work ahead

facial

(Credit: Unsplash)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Karen Silverman, Global AI Council Member, CEO and Founder, The Cantellus Group & Andrea Ortega, Technology and privacy attorney, LLM Law & Technology from UC Berkeley Law


  • Emerging facial recognition legislation shows the need for more clarity and standardization.
  • Developers and users of FRS technologies both should engage in the policy debate to define the coming standards.

Facial recognition has emerged as a powerful biometric technology, both in practice and in our collective imagination. Revelations about its use in the public domain and by law enforcement have fueled discussion about ethical concerns and corresponding legislative efforts to ban or limit its use.

Moreover, IBM has recently announced that it will “no longer offer general purpose IBM facial recognition or analysis software,” and Microsoft and Amazon have called for a year-long pause on police use of their facial recognition technologies and for Congress to take up ethical-standard setting.

These latest developments underscore that the time has come to figure out appropriate uses, limits and safeguards on use-case level challenges – both with the state of the technologies themselves, and how they are deployed within and by people in specific industries and settings.

What is the World Economic Forum doing about the Fourth Industrial Revolution?

The World Economic Forum was the first to draw the world’s attention to the Fourth Industrial Revolution, the current period of unprecedented change driven by rapid technological advances. Policies, norms and regulations have not been able to keep up with the pace of innovation, creating a growing need to fill this gap.

The Forum established the Centre for the Fourth Industrial Revolution Network in 2017 to ensure that new and emerging technologies will help—not harm—humanity in the future. Headquartered in San Francisco, the network launched centres in China, India and Japan in 2018 and is rapidly establishing locally-run Affiliate Centres in many countries around the world.

The global network is working closely with partners from government, business, academia and civil society to co-design and pilot agile frameworks for governing new and emerging technologies, including artificial intelligence (AI), autonomous vehicles, blockchain, data policy, digital trade, drones, internet of things (IoT), precision medicine and environmental innovations.

Learn more about the groundbreaking work that the Centre for the Fourth Industrial Revolution Network is doing to prepare us for the future.

Want to help us shape the Fourth Industrial Revolution? Contact us to find out how you can become a member or partner.

Differences across emerging legislation proposals
US state legislative proposals on commercial uses of facial recognition services (FRS) reveal how much work lies ahead in getting to laws that effectively address human rights and civil liberties (beyond privacy and security interests) and also that effectively instruct and engage business in doing so.

Washington State has made progress on its proposed Washington Privacy Act (WPA), the first of its kind to tackle specifically commercial uses of FRS and to import nondiscrimination requirements beyond traditional privacy ones. California was close on Washington’s heels with Assembly Bill 2261, which would have imposed restrictions on certain commercial FRS activities, but was just blocked as part of a larger legislative effort that would likewise have enabled certain law enforcement uses.

These emerging proposals, however, reveal varying standards highlighting the difficulty of legislating in this arena, the work remaining to define clear, consistent and effective standards, and the uncertainties that will confront businesses as they proceed (at least in the short and medium term).

Existing proposals, for instance, advance different approaches to non-discrimination obligations. The WPA requires third-party testing for detecting “accuracy and unfair performance differences across distinct subpopulations.” It does not, however, define what comprises “unfair performance” and it is not clear whether it extends to guarding against the use of FRS to violate basic civil rights beyond the access to goods and services. (California’s option did offer such guidance, but other laws from other localities in the future may not account for these issues leaving gaps to be bridged.)

Differences extend to issues of oversight as well. The WPA requires controllers to ensure “meaningful human review” and test the FRS in operational conditions before deployment, specifically for FRS intended to make decisions that produce legal or similarly significant effects on consumers. It does not define what might comprise a “meaningful human review” of FRS. California’s measure attempted to address this issue, including review or oversight by one or more individuals who are trained and who “are ultimately responsible for making decisions based, in whole or in part, on the output of a FRS.” Still, varying definitions will complicate matters for businesses trying to comply when measures that don’t align on key definitions.

What businesses can do now
With so much in development, businesses have the opportunity to contribute to the policy debate and to define the coming standards. The World Economic Forum is helping tackle this challenge with its multistakeholder approach and actionable governance framework and is calling for engagement from businesses and stakeholders. A current project, Responsible Limits on Facial Recognition Technology, will help develop a governance framework to ensure safe and trustworthy use of FRS technology. As these and other initiatives take shape, there are some valuable steps that firms can take now:

1. Think about, and beyond, the controller/processor framework

The distinction between controllers and processors, borrowed from the EU’s GDPR, lies at the heart of the WPA and California’s now defunct AB 2261. As currently drafted, any business’ status as one or the other (or neither) will determine its privacy and — importantly — its nondiscrimination obligations. Indeed, the proposals seem to assume that businesses using FRS for identification, verification and persistent tracking of individuals will be controllers, and that businesses developing or supplying FRS to controllers will be processors so long as they process personal data — collect, use, store, or analyze, among other operations — following instructions of the controller.

In practice, however, businesses using or supplying FRS may not always fall into the controller/processor framework, and their roles may change over time. For instance, an FRS supplier may be a controller from the start, or initially operate as a processor but end up a controller if it starts processing personal data outside of the controller’s instructions (e.g. when training a generic AI tool). Others might initially not process personal data but at some point start processing an FRS customer/controller’s specific customer data (e.g. when implementing that tool).

Moreover, FRS developers or suppliers that do not process personal data (including persistent tracking) would remain outside of the legislation’s scope and, therefore, would neither be subject to the privacy nor the nondiscrimination obligations. Under this scenario, a controller would have to contractually require such FRS developer to nonetheless submit in writing to these nondiscrimination obligations — in particular, to third-party testing and implementing mitigation plans — in order to ensure compliance with its own nondiscrimination obligations. In addition, FRS developers or suppliers may also nonetheless wish to contractually require FRS users to comply with applicable federal or state nondiscrimination laws.

Legislation may someday address FRS uses beyond this controller/processer construct, and that will be clarifying. Meanwhile, however, businesses should keep in mind that a) their handling of personal data increasingly defines new obligations, and b) existing nondiscrimination requirements may already apply to their digital activities.

2. Anticipate the most likely requirements and put a diverse team in charge

Under these proposals, both controllers and processors would be subject to broad nondiscrimination requirements through cross-contracting, human oversight, testing and audit requirements. This predominantly human-at-the-end-of-the-loop approach, however, might not be sufficient to address discrimination concerns around the use of FRS, especially as currently drafted. They are, in any case, a starting point.

Businesses contemplating the use of FRS can start building out robust, trustworthy systems by focusing on the similarities between these proposals, including their calls for human oversight, pre-release testing for bias and harmful impacts, and regular audits (and by thinking upfront about their overall AI strategies and trust standards). More broadly, the work to build trustworthy AI systems should be a fundamental part of all the different phases of the product cycle, embedding standards into the design and operation of FRS systems and into the teams operating them. To do this well, at the outset, business needs to consider the different uses in depth, the quality and impacts of these technologies — in particular at the management and oversight levels —and ensure that the responsible teams represent a diversity of lived experience and expertise.

Looking ahead

The differences raised between even just these two legislative approaches would result in a lack of clarity for businesses attempting to standardize or anticipate compliance protocols.

Still, legislation and regulation are inevitable in time Congress could soon enact standards and limitations for specific uses. California will almost certainly — sooner or later — try again to legislate commercial uses of this technology. And Illinois and Texas could also attempt to amend their biometrics privacy laws to incorporate nondiscrimination requirements. Likewise, certain cities (including New York City and San Francisco) are contemplating expanded bills of FRS on private use. Likewise, the Brookings Institute has just issued a report that identifies preemption as one major impediment to regulation in the US, and buried in this issue are inconsistencies among legislative proposals as well as the Constitutional question regarding the balance of federal and State authorities.

As proposals take shape, it is critical for developers and users of these technologies to engage in the policy debate to define the coming standards (and to develop and test their own trust standards). Not doing so could risk longer and more costly market suspensions, business interruptions and reputational damage on the backend — well before any law has anything to say about it.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Terrorism ‘spreading and destabilizing’ entire regions, Guterres warns States, at key Kenya conference

Here’s how we get businesses to harmonize on climate change

Drones are saving lives in Tanzania’s remote communities

Dignified health for all who live here

Austria, Italy, Portugal, Spain receive €279m after natural disasters in 2019

China, forever new adventures

A day that Berlin and Brussels would remember for a long time

I cycled over 6,000km across the United States to document climate change. Here’s what I learned

Climate action ‘both a priority and a driver of the decade’: Guterres

Antitrust: Commission consults stakeholders on guidance for national courts when handling disclosure information

No more lead in PVC to protect public health, say MEPs

Commission reviews relations with China, proposes 10 actions

Available mental health services: is it only about professionals or institutions?

UN rights experts ‘gravely’ concerned at spike in civilian casualties in north-west Myanmar following internet shutdown

COVID-19: Single market must emerge stronger from the crisis, say MEPs

Europe and the tragicomic ‘black sovranismo’

Humanitarian migration falls while labour and family migration rises

Facebook has built an AI-based tool that fixes the social network when it crashes

Africa’s future is innovation rather than industrialization

UN rights expert calls for civilian protection as fighting escalates between military and armed group

East Africa locusts threaten food insecurity across subregion, alerts UN agriculture agency

Migration crisis update: What are the chances of a fair deal at this EU Summit?

2021 EU budget must focus on supporting a sustainable recovery from the pandemic

David McAllister underlines the need for rapid progress in EU-UK negotiations

Prisons are failing. It’s time to find an alternative

This tool shows you which cities will flood as ice sheets melt

How to build a digital infrastructure that benefits emerging economies

A rapid deterioration of the humanitarian situation in the war-torn Yemen

Banking Union: ECOFIN and Parliament ready to compromise

4 ways family businesses can lead the pandemic recovery

Could electric vehicles pose a threat to our power systems?

GSMA Announces Final Event Lineup for Highly Anticipated 2019 “MWC Los Angeles, in Partnership with CTIA”

Summer 2018 Interim Economic Forecast: Resilient Growth amid increased uncertainty

“The winner is who can accelerate the transition to a new digital era”. The Sting reports live from EBS 2015: a Digital Europe 4.0

EU mobilises €21 million to support Palestine refugees via the UN Relief and Works Agency

EU: Divided they stand on immigration and Trump hurricanes

Under-fives’ daily screen time should be kept to 60 minutes only, warns WHO

Preventing the Pandemic of Mental Illness

COVID-19: Emerging technologies are now critical infrastructure – what that means for governance

Mario Draghi quizzed for last time by Economic and Monetary Affairs Committee

MEPs and European Youth Forum call on EU to Invest in Youth

EU to host international donors’ conference for Albania to help with reconstruction after earthquake

Manufacturing is finally entering a new era

Business could learn plenty about cybersecurity from the secret state

In Bali, UN chief Guterres outlines importance of international financial cooperation for sustainable development

Detecting online child sexual abuse requires strong safeguards

The ‘yellow vests’ undermined Macron in France and the EU

UN conference agrees better ways for Global South countries to work together on sustainable development

How India is solving its cooling challenge

Meet Alice, the battery-powered plane that could herald the age of electric air travel

Kids who live in the countryside have better motor skills, a study in Finland has found

Cohesion Policy after 2020: preparing the future of EU investments in health

MEPs demand an end to migrant deaths across the Mediterranean Sea

Search Engine neutrality in Europe in danger: Are 160.000 Google filtering requests good enough?

Our children’s career aspirations have nothing in common with the jobs of the future

World Editors Forum President: Credible media vital in the fight against COVID-19 and fake news epidemic

Coronavirus: here’s what you need to know about face masks

Donor countries set international standard for preventing sexual exploitation, abuse, and harassment in development sector

Can medical students be prepared for Global Health ethical issues?

Merkel refuses to consider the North-South schism of Eurozone

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s