Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Is Universal Health Coverage really available for all in the European Union?

Did young people just kill television?

CLIMATE CHANGE FOCUS: Cutting emissions, one bog at a time

COVID-19: research package welcomed, EU needs to be better equipped in future

South Sudan: UN condemns ‘brutal’ sexual assaults on roads to Bentiu

Brexit preparedness: EU completes preparations for possible “no-deal” scenario on 12 April

Dramatic drop in South Sudan political violence since peace agreement signing

Rohingya emergency one year on: UN says thousands of lives saved, but challenges remain

Conflict prevention, mediation: among ‘most important tools’ to reduce human suffering, Guterres tells Security Council

Mergers: Commission opens in-depth investigation into joint ventures proposed by Boeing and Embraer

Why people with disabilities are your company’s untapped resource

Why the Fourth Industrial Revolution needs more arts graduates

‘Every ventilator becomes like gold’ – a doctor’s stark warning from Italy’s Coronavirus outbreak

Sexual exploitation and abuse: latest UN quarterly update

Mobile technology saving lives: Changing healthcare systems with simple technological solutions

AI has huge potential – but it won’t solve all our problems

Parliament criticises Council’s rejection of money laundering blacklist

Here’s why the world’s recovery from COVID-19 could be doughnut shaped

Statement by President von der Leyen on CureVac

Eurozone: In vicious cycle of disinflation and unemployment?

Brexit: European Commission recommends the European Council (Article 50) to endorse the agreement reached on the revised Protocol on Ireland / Northern Ireland and revised Political Declaration

This Japanese experiment shows how easily coronavirus can spread – and what you can do about it

This is the first ever photo of a black hole

3 reasons why responsibly-deployed technology is key to the COVID recovery

How Google is fighting fire with real-time mapping data

What the next 20 years will mean for jobs – and how to prepare

VAT: EU Member States still losing almost €150 billion in revenues according to new figures

Privatization as a symptom of health inequity

UN chief appeals to G7 leaders for ‘strong commitment’ and political will to tackle climate emergency

Africa’s inspiring innovators show what the future could hold

New UN bullying report calls for ‘safe, inclusive’ schools for all children

FROM THE FIELD: Saving the tree kangaroos of Papua New Guinea

Member States agree to Commission proposal to support Irish beef producers impacted by market uncertainty

Capital Markets Union: Making it easier for insurers to invest in the real economy

Colombia is a Latin American success story, but must pursue new reforms to achieve stronger and more inclusive growth

VP McGuinness on women’s rights: “Not an option, but a duty”

The best and worst parenting advice I’ve heard, by a leading psychologist

EU and Airbus Member States take action to ensure full compliance in the WTO aircraft dispute

“BEUC cautions against TTIP that would seek to align EU and US chemicals management frameworks”

Why ‘video call fatigue’ might be making you tired during lockdown – and how to beat it

How fintech is making investment accessible

The most unlikely innovators are changing ICT for development – it’s time we took notice

Built by a woman: supporting the dreams of mum entrepreneurs

Why do US presidential elections last so long? And 4 other things you need to know

Is corporation tax good or bad for growth?

Some 300,000 Venezuelan children in Colombia need humanitarian assistance; UNICEF looks to boost response funding

An open letter to Europe’s leaders

The relation of deforestation and respiratory diseases

Link between conflict and hunger worldwide, ‘all too persistent and deadly’, says new UN report

This Central Asian lake is a stark reminder of the impact we have on the planet

EU prepares itself to fight back against hostile propaganda

We should treat data as a natural resource. Here’s why

EU to spend €6 billion on youth employment and training futile schemes

Facebook: MEPs demand a full audit by EU bodies to assess data protection

Google succumbs unconditionally to EU’s “right to be forgotten” ruling

Boat made of recycled plastic and flip-flops inspires fight for cleaner seas along African coast

The succesful cooperation

Making Europe’s businesses future-ready: A new Industrial Strategy for a globally competitive, green and digital Europe

China is the first non-EU country to invest in Europe’s €315 billion Plan

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s