Emerging legislation on commercial uses of facial recognition shows the work ahead

facial

(Credit: Unsplash)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Karen Silverman, Global AI Council Member, CEO and Founder, The Cantellus Group & Andrea Ortega, Technology and privacy attorney, LLM Law & Technology from UC Berkeley Law


  • Emerging facial recognition legislation shows the need for more clarity and standardization.
  • Developers and users of FRS technologies both should engage in the policy debate to define the coming standards.

Facial recognition has emerged as a powerful biometric technology, both in practice and in our collective imagination. Revelations about its use in the public domain and by law enforcement have fueled discussion about ethical concerns and corresponding legislative efforts to ban or limit its use.

Moreover, IBM has recently announced that it will “no longer offer general purpose IBM facial recognition or analysis software,” and Microsoft and Amazon have called for a year-long pause on police use of their facial recognition technologies and for Congress to take up ethical-standard setting.

These latest developments underscore that the time has come to figure out appropriate uses, limits and safeguards on use-case level challenges – both with the state of the technologies themselves, and how they are deployed within and by people in specific industries and settings.

What is the World Economic Forum doing about the Fourth Industrial Revolution?

The World Economic Forum was the first to draw the world’s attention to the Fourth Industrial Revolution, the current period of unprecedented change driven by rapid technological advances. Policies, norms and regulations have not been able to keep up with the pace of innovation, creating a growing need to fill this gap.

The Forum established the Centre for the Fourth Industrial Revolution Network in 2017 to ensure that new and emerging technologies will help—not harm—humanity in the future. Headquartered in San Francisco, the network launched centres in China, India and Japan in 2018 and is rapidly establishing locally-run Affiliate Centres in many countries around the world.

The global network is working closely with partners from government, business, academia and civil society to co-design and pilot agile frameworks for governing new and emerging technologies, including artificial intelligence (AI), autonomous vehicles, blockchain, data policy, digital trade, drones, internet of things (IoT), precision medicine and environmental innovations.

Learn more about the groundbreaking work that the Centre for the Fourth Industrial Revolution Network is doing to prepare us for the future.

Want to help us shape the Fourth Industrial Revolution? Contact us to find out how you can become a member or partner.

Differences across emerging legislation proposals
US state legislative proposals on commercial uses of facial recognition services (FRS) reveal how much work lies ahead in getting to laws that effectively address human rights and civil liberties (beyond privacy and security interests) and also that effectively instruct and engage business in doing so.

Washington State has made progress on its proposed Washington Privacy Act (WPA), the first of its kind to tackle specifically commercial uses of FRS and to import nondiscrimination requirements beyond traditional privacy ones. California was close on Washington’s heels with Assembly Bill 2261, which would have imposed restrictions on certain commercial FRS activities, but was just blocked as part of a larger legislative effort that would likewise have enabled certain law enforcement uses.

These emerging proposals, however, reveal varying standards highlighting the difficulty of legislating in this arena, the work remaining to define clear, consistent and effective standards, and the uncertainties that will confront businesses as they proceed (at least in the short and medium term).

Existing proposals, for instance, advance different approaches to non-discrimination obligations. The WPA requires third-party testing for detecting “accuracy and unfair performance differences across distinct subpopulations.” It does not, however, define what comprises “unfair performance” and it is not clear whether it extends to guarding against the use of FRS to violate basic civil rights beyond the access to goods and services. (California’s option did offer such guidance, but other laws from other localities in the future may not account for these issues leaving gaps to be bridged.)

Differences extend to issues of oversight as well. The WPA requires controllers to ensure “meaningful human review” and test the FRS in operational conditions before deployment, specifically for FRS intended to make decisions that produce legal or similarly significant effects on consumers. It does not define what might comprise a “meaningful human review” of FRS. California’s measure attempted to address this issue, including review or oversight by one or more individuals who are trained and who “are ultimately responsible for making decisions based, in whole or in part, on the output of a FRS.” Still, varying definitions will complicate matters for businesses trying to comply when measures that don’t align on key definitions.

What businesses can do now
With so much in development, businesses have the opportunity to contribute to the policy debate and to define the coming standards. The World Economic Forum is helping tackle this challenge with its multistakeholder approach and actionable governance framework and is calling for engagement from businesses and stakeholders. A current project, Responsible Limits on Facial Recognition Technology, will help develop a governance framework to ensure safe and trustworthy use of FRS technology. As these and other initiatives take shape, there are some valuable steps that firms can take now:

1. Think about, and beyond, the controller/processor framework

The distinction between controllers and processors, borrowed from the EU’s GDPR, lies at the heart of the WPA and California’s now defunct AB 2261. As currently drafted, any business’ status as one or the other (or neither) will determine its privacy and — importantly — its nondiscrimination obligations. Indeed, the proposals seem to assume that businesses using FRS for identification, verification and persistent tracking of individuals will be controllers, and that businesses developing or supplying FRS to controllers will be processors so long as they process personal data — collect, use, store, or analyze, among other operations — following instructions of the controller.

In practice, however, businesses using or supplying FRS may not always fall into the controller/processor framework, and their roles may change over time. For instance, an FRS supplier may be a controller from the start, or initially operate as a processor but end up a controller if it starts processing personal data outside of the controller’s instructions (e.g. when training a generic AI tool). Others might initially not process personal data but at some point start processing an FRS customer/controller’s specific customer data (e.g. when implementing that tool).

Moreover, FRS developers or suppliers that do not process personal data (including persistent tracking) would remain outside of the legislation’s scope and, therefore, would neither be subject to the privacy nor the nondiscrimination obligations. Under this scenario, a controller would have to contractually require such FRS developer to nonetheless submit in writing to these nondiscrimination obligations — in particular, to third-party testing and implementing mitigation plans — in order to ensure compliance with its own nondiscrimination obligations. In addition, FRS developers or suppliers may also nonetheless wish to contractually require FRS users to comply with applicable federal or state nondiscrimination laws.

Legislation may someday address FRS uses beyond this controller/processer construct, and that will be clarifying. Meanwhile, however, businesses should keep in mind that a) their handling of personal data increasingly defines new obligations, and b) existing nondiscrimination requirements may already apply to their digital activities.

2. Anticipate the most likely requirements and put a diverse team in charge

Under these proposals, both controllers and processors would be subject to broad nondiscrimination requirements through cross-contracting, human oversight, testing and audit requirements. This predominantly human-at-the-end-of-the-loop approach, however, might not be sufficient to address discrimination concerns around the use of FRS, especially as currently drafted. They are, in any case, a starting point.

Businesses contemplating the use of FRS can start building out robust, trustworthy systems by focusing on the similarities between these proposals, including their calls for human oversight, pre-release testing for bias and harmful impacts, and regular audits (and by thinking upfront about their overall AI strategies and trust standards). More broadly, the work to build trustworthy AI systems should be a fundamental part of all the different phases of the product cycle, embedding standards into the design and operation of FRS systems and into the teams operating them. To do this well, at the outset, business needs to consider the different uses in depth, the quality and impacts of these technologies — in particular at the management and oversight levels —and ensure that the responsible teams represent a diversity of lived experience and expertise.

Looking ahead

The differences raised between even just these two legislative approaches would result in a lack of clarity for businesses attempting to standardize or anticipate compliance protocols.

Still, legislation and regulation are inevitable in time Congress could soon enact standards and limitations for specific uses. California will almost certainly — sooner or later — try again to legislate commercial uses of this technology. And Illinois and Texas could also attempt to amend their biometrics privacy laws to incorporate nondiscrimination requirements. Likewise, certain cities (including New York City and San Francisco) are contemplating expanded bills of FRS on private use. Likewise, the Brookings Institute has just issued a report that identifies preemption as one major impediment to regulation in the US, and buried in this issue are inconsistencies among legislative proposals as well as the Constitutional question regarding the balance of federal and State authorities.

As proposals take shape, it is critical for developers and users of these technologies to engage in the policy debate to define the coming standards (and to develop and test their own trust standards). Not doing so could risk longer and more costly market suspensions, business interruptions and reputational damage on the backend — well before any law has anything to say about it.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

We had the hottest June ever this year – this is what happened around the world

Force used against protestors in Gaza ‘wholly disproportionate’ says UN human rights chief

A new European banking space is born this year

US – Russia bargain on Syria, Ukraine but EU kept out

Don’t underestimate the power of the fintech revolution

ECB reaches the boundaries of its mandate to revive the entirety of Eurozone

Draghi sees inflationary bubbles

UN-led Yemen ceasefire monitoring team gets ready to begin operations

High-technology manufacturing saves the EU industry

We’ll succeed together

Crimean crisis: not enough to slow down European indices

Why protectionism spells trouble for global economic growth

Three reasons to be optimistic for the future of Asia

Four key challenges for cybersecurity leaders

Eurozone: Disinflation engulfs the industrial goods sector

‘You can and should do more’ to include people with disabilities, wheelchair-bound Syrian advocate tells Security Council in searing speech

5 futuristic ways to fight cyber attacks

Why we need to rethink geo-economics to beat climate change

Mental health and suicide prevention

A day in the life of a refugee: the role of nations and citizens of the world

Governments, businesses ‘walk the talk’ for investment in sustainable development: UN forum

COP24: Huge untapped potential in greener construction, says UN environment agency

Only the private sector can help deliver universal healthcare in Africa

Here’s how we solve the global crisis of tribalism and democratic decay

Want a more inclusive society? Start with mobility

EU and 15 World Trade Organization members establish contingency appeal arrangement for trade disputes

One Health approach to combating Antimicrobial Resistance – how can professionals from different backgrounds unite in this common fight?

Politics is failing to protect the Amazon. It’s time for finance to step up instead

So different, so close – for two twinning cities

Child victims of DRC Ebola outbreak need ‘special attention and care’: UNICEF

Sudan: ‘Violence must stop’, says UNICEF chief, ‘gravely concerned’ over 19 child deaths since military backlash

‘Once lost, hearing doesn’t come back,’ World Health Organization warns on World Hearing Day

COVID-19 outbreak: Commission supports repatriation of EU citizens from cruise ship in Japan

EU-US trade war? EU calls for logic while Trump’s administration is a loose cannon in a dangerous lose-lose situation for global prosperity

Why the way of loving closes doors of health?

Long-term EU budget: MEPs lay down funding priorities for post-2020 budget

European Parliament and Eurovision sign partnership for European Elections

Quality of air in Bucharest-Romania: is it fog or is it smog?

Industrial products: Lifting the last impediments in the EU single market

Cyprus Parliament says no to blackmail

These are New York Public Library’s 10 most borrowed books

These countries give the most aid – and are the most principled about it

3 vital steps to a new gender equality playbook

Why Eurozone needs a bit more inflation

UN postal agency ‘regrets’ US withdrawal

Iraq protests: UN calls for national talks to break ‘vicious cycle’ of violence

JADE Testimonial #3: Sebastian @ Fundraising

Hollande protects the euro from the attacks of extremists

Preparing Africa for ravages of climate change ‘cannot be an afterthought’ – COP24

A 550 km-long mass of rotting seaweed is heading for Mexico’s pristine beaches

UN chief hails ‘positive developments’ towards ending political crisis in Bolivia

Students & Allies Unite Globally To Launch #Students_Against_COVID

Migration crisis update: mutual actions and solidarity needed as anti-migrant policies thrive

Better air pollution data is helping us all breathe easier. Here’s how

Coronavirus: EU Civil Protection Mechanism activated for the repatriation of EU citizens

London wants new skyscrapers to protect cyclists from wind tunnels

80,000 youngsters at risk in DRC after forcible expulsion from Angola: UNICEF

‘We cannot lose momentum’ on the road to peace in Yemen, UN envoy warns

Conflict prevention, mediation: among ‘most important tools’ to reduce human suffering, Guterres tells Security Council

The new North America trade deal USMCA punishes German cars

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s