Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

What makes Copenhagen the world’s most bike-friendly city?

Denmark plans ‘Silicon Valley’ on 9 artificial islands off Copenhagen

The UN came of age with the nuclear bomb. Time for it to step up to the AI era

India is now the world’s 5th largest economy

This crisis cannot be confronted with statistics

Myanmar: Departing UN rights expert still hopeful for democratic transition

WHO and IFMSA as transcendent pillars for world improvement

Prevent future crises and empower youth – now!

Would you want to live to 150? Top quotes on what it means to grow old

Why this is the year we must take action on mental health

Cancer is a growing global threat and prevention is key, UN study shows

Coronavirus has exposed human rights gaps. We need to fix this

Could the EU’s ban on palm oil in biofuels do more harm than good?

European Innovation Scoreboard 2018: Europe must deepen its innovation edge

3 reasons why business leaders can’t afford to ignore diversity

TTIP 9th Round marked by American disappointment: Will some optimism save this trade agreement?

Turkey needs to step up investment in renewables to curb emissions

What do refugees really need from those who want to help? A refugee explains

‘Protracted crisis’ in Venezuela leads to ‘alarming escalation of tensions’: UN political chief

Von der Leyen on Coronavirus Global Response: World stands united against coronavirus and will win

Africa is helping the drone industry get off the ground. Here’s how

This is how the Western Balkans will become more innovative

Commission goes less than mid-way on expensive euro

EU Commission closer to imposing anti-dumping duties on Chinese solar panel imports?

Are the G20 leaders ready to curb corporate tax-avoidance?

This AI-powered app aims to help people with autism improve their social skills

EU Telecoms deal: Fees on calls across the EU capped and 5G network by 2020

COVID-19: What you need to know about the coronavirus pandemic on 9 April

Air pollution: Most EU Member States not on track to reduce air pollution and its related health impacts by 2030

The fatal consequences of troika’s blind austerity policy

Boris to end up in jail if he loses the next elections?

Are Halloween pumpkins a problem for the planet?

Philanthropy is at a turning point. Here are 6 ways it could go

UN chief praises Malaysia’s death penalty repeal as ‘major step forward’

‘Compelling case’ for urgency around global disarmament, UN-led forum told

How much more political is the new EU leadership? Does this include personal bend?

How to talk about climate change: 5 tips from the front lines

How tomorrow’s buildings will make you – and the planet – healthier

Wednesday’s Daily Brief: Libya ‘war crime’ attack, Sudan, Myanmar rights violations continue, ‘xenophobia’ in Assam, South Sudan update

CLIMATE CHANGE FOCUS: Cows, coffee and sustainable farming

New systemic approach needed to tackle global challenges

We don’t know how autonomous vehicles will depreciate – and that’s a problem

A young European voice on Grexit: too high a bill and too big a deal!

These are the world’s most tree-covered countries

OECD: Rising employment overshadowed by unprecedented wage stagnation

France: New labour laws for more competitiveness

Why we need both science and humanities for a Fourth Industrial Revolution education

Egypt is building one of the world’s largest solar parks

Japan’s agro-food sector would benefit greatly from policies to boost innovation, entrepreneurship and sustainable resource use

How to unleash the potential of regions like São Paulo

Why we need different generations in the workplace

Intergenerational, intercultural, interactive – The 2015 edition of JADE’s Generations Club: Transforming Europe into an entrepreneurial society

Draghi rehabs ECB into a tool to support growth and employment; a departure from Teutonic orthodoxy

Africa shouldn’t have to choose between high growth and low emissions

This project in India helps people and tigers co-exist peacefully

How face masks, gloves and other coronavirus waste is polluting our ocean

Eurozone: Inflation plunge to 0.4% in July may trigger cataclysmic developments

Uganda’s Ebola preparedness ‘will go a long way’ says WHO chief

Coronavirus: 4 tips for parents who are homeschooling

Climate change is speeding up. Our response needs to be even faster

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s