Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

Advertising

Advertising

Advertising

Advertising

Advertising

Advertising

Advertising

Featured Stings

Stopping antimicrobial resistance would cost just USD 2 per person a year

The increasing drug prices in Europe

The power of trust and values in the Fourth Industrial Revolution

Margrethe Vestager, EU Commissioner in charge of competition policy, during a recent press conference in Brussels / Berlaymont. (Copyright: EU, 2018 / Source: EC - Audiovisual Service / Photo: Jennifer Jacquemart)

EU opens investigation into Qatar Petroleum over potentially restrictive gas contracts

Germany not famous for easy way outs from political stalemates

Horse meat runs faster than authorities…

Is the English language too powerful?

3 reasons all countries should embrace the Global Compact for Migration

EU Ambassadors in the EP: a multilateral approach to global challenges needed

Multilateralism more vital than ever, as World War centenary looms: Security Council

What our leaders hide from us

The two big uncertainties shaping our future

China hails human rights progress amid calls to close detention camps

Latest leaked TTIP document confirms EU sovereignty may be under threat

Yemen parties underscore ‘strong desire’ for peace, UN Envoy reports

David Attenborough: The planet can’t cope with overpopulation

EU Council: Private web data to be protected by…abusers

These are the cities with the biggest carbon footprints

My ‘’cultural’’ contacts with China

Austria’s EU Presidency: Chancellor Sebastian Kurz aims to “build bridges”

Russia and the West use the same tactics to dismember Ukraine

A renewed agenda for Research and Innovation: Europe’s chance to shape the future

UN chief praises Malaysia’s death penalty repeal as ‘major step forward’

The key takeaways of G7 Summit in Canada

The Bank of China at European Business Summit 2015

IMF to teach Germany a Greek lesson

Afghanistan: Civilian casualties caused by IEDs has reached ‘extreme levels’, UN warns

China repels EU allegations of export subsidies

Are ECB’s €500 billion enough to revive Eurozone? Will the banks pass it to the real economy?

A Sting Exclusive: “Seize the opportunity offered by Africa’s continental free trade area”, written by the Director General of UNIDO

When it comes to envirotech adoption, NGOs can lead us out of the woods

The EU Diplomacy in North Korea promotes peace or war?

UN-based World Summit Award (WSA) presents its master list on digital innovation with impact on society from 24 countries

Matthias in Canada

A Sting Exclusive: “The competitiveness of Europe depends on a digital single market”, EPP President Joseph Daul highlights live from European Business Summit 2015

How bad is the Eurozone economy? The ECB thinks too bad

UN condemns ‘heinous’ suicide attack on education centre in Afghanistan

Afghanistan: UN envoy urges further extension of ceasefire with Taliban, as Eid ul-Fitr gets underway

UN sounds alarm as Venezuelan refugees and migrants passes three million mark

More taxpayers’ money for the banks

The miserables and the untouchables of the economic crisis

EU sets ambitious targets for the Warsaw climate conference

Overseas investment falling, developing countries largely unscathed: UN trade agency

The Franco-German axis considers that all EU needs now is more armaments

MasterCard @ MWC14: Innovation in times of regulatory uncertainty

EU Trust Fund for Africa: Can it be beneficial for Italy and tackle the migration crisis in the Mediterranean?

Trailing the US-EU economic confrontation

Fears for food security and the future of farming families, as Fall Armyworm spreads to Asia

Restoring government control across Central African Republic is ‘key’ to lasting peace, stability – UN envoy

Main results of EU-Japan summit: Tokyo, 17/07/2018

The West and Russia impose a new order on the world

Rehn ready to sacrifice part of the real economy

The EU accuses Russia of bullying Ukraine to change sides

Can Greece’s devastating economy deal with the migration crisis?

Gender Equality as a platform to improve Medicine

5 ways to make your organization a great sustainability partner

EU Commission spends billions without achieving targets

Commissioner sings “Volar-e” but the European driver no “Cantar-e”

EU-US trade deal: Europe to Americanize its social model?

The global liberal order is in trouble – can it be salvaged, or will it be replaced?

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s