Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

Advertising

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Execution of juvenile offender in Iran ‘deeply distressing’ – UN rights chief

Crucial medical supplies airlifted to north-east Syria to meet ‘desperate need’

Who and why want the EU-US trade agreement here and now

‘Compelling case’ for urgency around global disarmament, UN-led forum told

E-cigars: Improbable ally or enemy in disguise?

“A Junior Enterprise is run only by students.. there are no professors or managers that can help you solve your problems”

Intervene, don’t overthink – the new mantra of systems design

TTIP’s 11th round major takeaways and the usual “leaked” document

Opposite cultures: Should it be a problem?

Pledging ongoing UN support during visit to cyclone-hit areas, Guterres praises resilience of Mozambicans

Air quality: Commission takes action to protect citizens from air pollution

Sustainable development demands a broader vision, says new OECD Development Centre report

How oysters are cleaning New York’s polluted harbor

EU seeks foreign support on 5G from Mobile World Congress 2015 as the “digital gold rush” begins

SMEs and micro firms sinking together with south Eurozone

“The markets have moved on renewables, policy makers must keep up”, A Sting Exclusive by Erik Solheim, Head of UN Environment

UN Human Rights Council stands firm on LGBTI violence, Syria detainees and Philippines ‘war on drugs’

UK’s PM Theresa May asks for a two-year Brexit transition plan as negotiations round kicks off

Drought in Europe: Member States agree on support measures proposed by the Commission

Thursday’s Daily Brief: Climate crisis and food risks, fresh violence threat for millions of Syrians, calls for calm in Kashmir

Trade in counterfeit goods costs UK economy billions of euros

Refugee crisis update: EU still lacks solidarity as Hungary and Slovakia refuse to accept EU Court’s decision

How to future-proof India’s economy

EU’s judicial cooperation arm, Eurojust, to become more effective with new rules

Our healthcare systems are ailing. Here’s how to make them better

How universities can become a platform for social change

UNICEF backpacks used as a haunting symbol to call for greater protection of children living in conflict

Radioactive nuclear waste is a global threat. These scientists may have a new solution

OECD leading multilateral efforts to address tax challenges from digitalisation of the economy

Iraq protests: UN calls for national talks to break ‘vicious cycle’ of violence

A Europe that protects: EU reports on progress in fighting disinformation ahead of European Council

Commission presents EU-Vietnam trade and investment agreements for signature and conclusion

Blue bonds: What they are, and how they can help the oceans

Obesity rates soar due to dramatic global diet shift, says UN food agency

Coronavirus crisis: “Commission will use all the tools at its disposal to make sure the European economy weathers the storm”

Mexico must increase foreign bribery enforcement: full implementation of anti-corruption reforms could help

Two peacekeepers killed in an attack against UN convoy in Mali

EuroLat plenary in Panama: control of trade talks and fight against crime

Is the English language too powerful?

Trade: First year of the EU-Japan Economic Partnership Agreement shows growth in EU exports

UN agriculture agency digs in to help forests and farms build resilience to climate change

Disaster Medicine in Medical Education: the investment you just can´t afford to ignore

A Sting Exclusive: “Infrastructure can lay the groundwork for the Sustainable Development Goals” by Mr Fulai Sheng, UN Environment Senior Economist

Commission proposes a governance framework for the Budgetary Instrument for Convergence and Competitiveness

Why enterprise risk management is the future for banks

Streaming music isn’t as green as you might think. Here’s why

‘Young people care about peace’: UN Youth Envoy delivers key message to Security Council

Why the ECB prepares to flood the markets with more and free of charge euro; everybody needs that now

Deal on tightening the rules to stop terrorists from using homemade explosives

A Europe that protects: Continued efforts needed on security priorities

International community renews commitment to multilateral efforts to address tax challenges from digitalisation of the economy

EU invests more than €100 million in new LIFE Programme projects to promote a green and climate-neutral Europe

What the car industry has done to help fight climate change – and what it needs to do next

MWC 2016 LIVE: Freemium MVNO model a success, claims FreedomPop head

This company is breeding millions of insects in the heart of London

Millions more migrant workers, means countries lose ‘most productive part’ of workforce

‘Brutal weather’ continues as Rohingya refugee children endure devastating rainfall in Bangladesh

Turkey remains numb while its economy is expected to shrink further due to a cocktail of EU and US sanctions

More taxpayers’ money for the banks

Electronic cigarettes – The alternative we’ve been looking for?

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s