Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Next time you fly, could you be boarding a train instead of a plane?

False promises to Small and Medium Enterprises

The Czech economy is thriving but boosting skills and productivity and transitioning to a low-carbon productive model is vital to sustainable and inclusive growth

‘Save Tuvalu; save the world’; UN chief echoes rallying cry from front lines of global climate emergency

This South African lawyer is reading while running marathons – for book donations

‘Proving our worth through action’: 5 things Guterres wants the UN to focus on in 2019

Digital: The EU must set the standards for regulating online platforms, say MEPs

UN Security Council hails ‘courage’ of Afghan voters

Human rights breaches in Azerbaijan and Sudan

There’s a global learning crisis and it’s leaving millions without basic skills

In Christchurch, UN chief calls for tolerance, solidarity to extinguish ‘wildfire’ of hate speech

Building the future of EU strategic foresight – Speech by Vice-President Šefčovič at the annual conference of the European Policy and Strategy Analysis System

Commission launches initiative for more sustainable cocoa production

‘Global care crisis’ set to affect 2.3 billion people warns UN labour agency

1.1 billion people still lack electricity. This could be the solution

‘Think beyond farm jobs’ to reach sustainable development, UN agriculture chief advises African youth

Girls still being treated as aliens in medicine in the 21st century

5 things COVID-19 has taught us about inequality

COVID-19: Commission provides guidance on EU passenger rights

If you want to make progress on all the major global challenges, start with water

We should treat data as a natural resource. Here’s why

These are the top risks for doing business around the world

Germany may prove right rejecting Commission’s bank resolution scheme

How Asia could be the winner in the US and China’s Belt and Road race

7 steps to becoming a ‘CEO Academy’

State aid: Commission approves amended Estonian scheme worth €450 million to support production of electricity from renewable sources

5 facts to know about Africa’s powerhouse – Nigeria

Does May have enough time in Parliament to table a soft Brexit deal?

This man swam under the East Antarctic ice sheet to highlight the impact of climate change

A 3-step path to securing critical infrastructure

FROM THE FIELD: Conversations about Conservation

EU invests more than €100 million in new LIFE Programme projects to promote a green and climate-neutral Europe

‘Starvation’ now a reality for displaced Syrians stranded in camp near Jordanian border

TTIP is not dead as of yet, the 15th round of negotiations in New York shouts

The unique role of business in building social good

The European Union’s Balkan Double Standard

Sustainable finance: Commission welcomes deal on an EU-wide classification system for sustainable investments (Taxonomy)

We’re all in the same boat on the SDGs. Here’s how we steer a course

COP25: MEPs push for CO2 neutrality by 2050

Anti-Semitism ‘toxic to democracy’, UN expert warns, calling for better education

Universal access to energy is a major challenge for the Arab world. Here’s why

“BRI cooperation is entering a new stage: we need a new and more constructive approach rather than waste time on suspicion”, China’s Ambassador to EU Zhang Ming underlines live from European Business Summit 2019 in Brussels

Cameroon: Clear ‘window of opportunity’ to solve crises rooted in violence – Bachelet

Plenary highlights: Commission changes, EU budget and climate law

Why flexible workspaces are the key to winning the talent war

Syria: UN-backed watchdog says chemical weapon ‘likely used’ in February attack

The world needs a circular economy. Help us make it happen

Alcoholic drinks: Commission tables update of rules governing alcohol excise duties

‘Going green’ is good business says private sector at UN’s COP24 climate conference

The US may be “open” to reviving TTIP, while the EU designs the future of trade with China

State aid: Commission approves €1.2 billion French “Fonds de solidarité” scheme for small enterprises in temporary financial difficulties due to coronavirus outbreak

Coronavirus (COVID-19): truth and myth on personal risk perception

These are the ‘positive’ tipping points that could slow global warming

2019 Innovation Scoreboards: The innovation performance of the EU and its regions is increasing

Summer 2018 Interim Economic Forecast: Resilient Growth amid increased uncertainty

At the age of 50, is Davos going through a midlife crisis?

Deepfake democracy: Here’s how modern elections could be decided by fake news

What if we did everything right? This is what the world could look like in 2050

We’re facing a ‘cold crunch,’ and it’s nothing to do with the polar vortex

England’s beavers are back, and they’ve already made a big impact

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s