Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

Advertising

Advertising

Advertising

Advertising

Advertising

the European Sting Milestones

Featured Stings

Stopping antimicrobial resistance would cost just USD 2 per person a year

Will Qualcomm avoid Broadcom’s hostile takeover post the 1 bn euro EU antitrust fine?

Draghi joined Macron in telling Germany how Eurozone must be reformed

Plastic Oceans: MEPs back EU ban on polluting throwaway plastics by 2021

The ‘abuse of food relief in Yemen’ must end now

Halt death sentences on children, UN rights expert urge Saudi authorities

Global immunization is having its annual check-up. What can we learn?

As the year closes out, UN political chief talks the art of diplomacy – and crises to watch in 2019.

Ukraine pays the price for lying between Russia and the EU

The EU Commission by serving the banks offers poor support to European mainstream political parties

France sneaks into the Geneva US-Iran talks to claim its business share in Tehran

How to keep our cities cool as temperatures rise

Parliament adopts its position on digital copyright rules

Antitrust: Commission fines Google €4.34 billion for illegal practices regarding Android mobile devices to strengthen dominance of Google’s search engine

The Irish Presidency bullies the Parliament over EU budget

Mental health problems costing Europe heavily

Anti-vaccers: does the empty can rattle the most?

Clean energy will do to gas what gas has done to coal

WHO and IFMSA as transcendent pillars for world improvement

The Oslo model: how to prepare your city for the electric-vehicle surge

Is Eurozone preparing to abandon austerity and stagnation?

Lessons from the Global Entrepreneurship Index

EU Parliament: It takes real banks to fight unemployment and recession

EU-India summit: Will the EU manage to sign a free trade agreement with India before Britain?

UK: Crawley group wins European Citizens’ Prize

Multilateralism: The only path to address the world’s troubles, signals Guterres

The EU Parliament sidesteps the real issues about banks, while the US target the Eurozone lenders

Easing funding woes for UN agency assisting Palestine refugees a ‘wise investment for today and the future’

In Tokyo, UN chief expresses full support for US-Japan dialogue with North Korea

A Sting Exclusive, the European Commissioner for Energy Günther Oettinger writes for the Sting on “EU Industry: a major energizer”

The EU Commission predicts a decimated growth in the next years

There’s a global learning crisis and it’s leaving millions without basic skills

European Union disenchanted with Turkey

Trump aims trade offensive solely to China, renews truce with EU

Education expenditure in the EU not hurt much by crisis

UN chemical weapons watchdog adds new powers to assign blame, following attacks

7 key challenges for the future of ASEAN – and how to solve them

European Youth Vlog

EU-Japan trade agreement enters into force

We don’t need to ban plastic. We just need to start using it properly

Change is happening – and young people are leading the way forward

Neelie Kroes at the European Young Innovators Forum: Unconvention 2014

EU-US trade talks go ahead despite Prism and civil rights breach

EU Ambassadors in the EP: a multilateral approach to global challenges needed

Commission: Raising the social issues that can make or break the monetary union

Want a fairer society? This economist says he has the answer

EU decides “in absentia” of civil society

Oh, well, you are wrong, Google responds to the European Commission

Brexit talks: 2nd round fails to bring the EU and the UK closer on key issues

An ECB banker wants to change the European social model

Large parts of the world are growing more fragile. Here are 5 steps to reverse course

ECB steadily continues monetary easing policy as EU economy gains momentum

Can the next financial crisis be avoided?

Russia to cut gas supplies again: can the EU get back to growth without a solid energy market?

The new European Union of banks is ready

Ceasefire holds in Tripoli, but core problems remain, says UN Libya mission chief

Managers’ pay under fire

French election: Will France vote for a reformed or no EU?

Respect for fundamental rights and freedoms key for peaceful polls in DRC – UN mission chief

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s