Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Fighting crime: faster EU-wide exchange of non-EU nationals’ criminal records

Nicaragua: MEPs condemn brutal repression and demand elections

Building trust: Here’s what you need to know about The Davos Agenda 2021

The EU has to prove it can remain one piece

Declaring commitment to ‘peace and stability’ for Libya, top UN envoy steps down as stress takes its toll

Draghi proposes timeframe for full Banking Union in five years

5 futuristic ways to fight cyber attacks

‘Real change’ involving women in peace and security, still too slow, Guterres tells Security Council

Rule of law in Hungary: Parliament calls on the EU to act

Coronavirus: Commission starts testing interoperability gateway service for national contact tracing and warning apps

Entrepreneurship in a newly shaped Europe: what is the survival kit for a young Catalan and British entrepreneur in 2018?

Strawberries and child support; a Thai partnership

IMF’s Lagarde to Peoples of the world: You have to work more for the banks!

There isn’t a single country on track to make the UN’s targets for gender equality

Trade marks: Commission decides to refer Romania to the Court of Justice for not transposing the Trade Mark Directive

Phone lines open between Ethiopia and Eritrea, and people are calling strangers

The world is getting angrier, according to a new poll

New study shows close link between GVCs participation and economic development

Deadly violence at Israel-Gaza border escalates dangerously: UN chief condemns in strongest terms

The good news on pensions: sustainable equals profitable

This Danish scheme is offering free kayak rides… for picking up trash

EU Top Jobs summit ended with no agreement: welcome to Europe’s quicksand!

European Democracy Action Plan: making EU democracies stronger

EU citizens disenchanted with Economic and Monetary Union over rising poverty and high unemployment

Accountability in Sudan ‘crucial’ to avoid ‘further bloodshed’, says UN rights office

A woman would have to be born in the year 2255 to get equal pay at work

How Greece was destroyed

These 11 EU states already meet their 2020 renewable energy targets

3 ways Africa can improve the health of women and children

Aluminium can help to build the circular economy. Here’s how

COVID-19: Maintaining the teaching experience for students whilst delivering teaching virtually

Half of the world’s population lack access to essential health services – are we doing enough?

These are the world’s 10 most competitive economies in 2019

Dramatic drop in South Sudan political violence since peace agreement signing

Human health – litmus paper for the climate change?

UN health agency highlights lifestyle choices that can prevent onset of dementia, as millions more succumb each year

Thousands returning to Nigeria’s restive Borno state ‘at risk’; UN ‘gravely concerned’

UNICEF appeals for end to ‘war on children’ in Syria and Yemen

Brexiteer May gets lip-service from Trump and Turkish promises from Erdogan

How to make primary healthcare a favourable career choice for medical students: Strategies and reflections

Reusable packaging: 6 benefits beyond sustainability

A Sting Exclusive: “Youth voice must be heard in climate change negotiations!”, Bérénice Jond Board Member of European Youth Forum demands from Brussels

Can cybersecurity offer value for money?

Chart of the Day: This is how the $88 trillion global economy is spread around the world

Cryptocurrency mining could become the new face of energy storage. Here’s how

As many as 330,000 displaced by heavy fighting in south-west Syria – UN agency

“Leaked” TTIP document breaks post 8th negotiations round silence and opens door to critics

JADE at European Business Summit 2015

Measuring consumer confidence isn’t useful anymore. Here’s what we should do instead

Countries should focus on labour market policies to help refugees and improve coordinated actions to tackle illegal immigration

Financial system risk is elevated and global standards are essential in managing cross-border infrastructure investment

Business leaders join UN to rev up sustainable development investments

Here’s how one business leader is tackling injustice: It starts with personal commitment

Lithuania needs to get rid of the victim mentality

A guide to thriving in the post-COVID-19 workplace

When is Berlin telling the truth about the EU banking union?

Climate crisis could see ‘collapse’ of some economies without diversification, warns UN trade chief

How to make later life happy, healthy and meaningful

These refugee children have danced in the snow for the first time

FROM THE FIELD: South Sudan’s green shoots, highlight environmental recovery from war

More Stings?


Speak your Mind Here

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s