Algorithmic warfare is coming. Humans must retain control

Army UN 2018

MONUSCO/Sylvain Liechti Special intervention brigade forces from Tanzania, part of the UN peacekeeping mission in the Democratic Republic of the Congo – MONUSCO, on duty in Sake, North Kivu (July 2013).

This article is brought to you thanks to the strategic cooperation of The European Sting with the World Economic Forum.

Author: Peter Maurer, President, International Committee of the Red Cross (ICRC)

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using. Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

Advertising

Advertising

Advertising

Advertising

Advertising

Advertising

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Accelerating a more sustainable industrial revolution with digital manufacturing

Politics still matter in the US but not in Europe

International data flows: Commission launches the adoption of its adequacy decision on Japan

European banking stress tests 2014: A more adverse approach for a shorter banking sector

UN-led Yemen ceasefire monitoring team gets ready to begin operations

Why private investment in deforestation-free commodity production needs to be scaled up

Anti-terror measures against youngsters’ online posts ‘linked to spike in child detention globally’

A Sting Exclusive: “Our ambition is by 2020 Indonesia to become an emerging power of World’s Maritime Access”, reveals the Chargé d’Affaires at the Embassy of Indonesia in Brussels, treating WEF, ASEAN and EU-Indonesia relations on the eve of the World Economic Forum East Asia 2015 in Jakarta

Antisemitism, intolerance, can be unlearned, Guterres tells New York commemoration

President Juncker temporarily transfers portfolio responsibilities following departure of two Members of the European Commission

EU-US to miss 2015 deadline and even lose Germany’s support in TTIP’s darkest week yet

Working Muslim women are a trillion-dollar market

6th Edition of India m2m + iot Forum to open its door on 14th January, in association with The European Sting

Entrepreneurship’s key to success showcased by a serial young entrepreneur

The EU resumes budget support assistance to the Republic of Moldova

Let us keep ‘their spirit of service alive’: Guterres leads tributes to UN workers who died in Ethiopia crash

This is how music festivals are tackling plastic waste

A young doctor from Glasgow reports: in the UK refugees are left to rot

One good reason to feel less blue about the future of our oceans

The European Parliament wants to stay in one place

World Health Organisation and medical students: is there any room for improvement?

What cryptocurrencies will do to the integrity of politics

Plastic is a global problem. It’s also a global opportunity

Guatemala: UN anti-corruption body will continue working, as Constitutional Court blocks Government expulsion

This forgotten chemical element could be the key to our green energy future

‘Bleakest period yet’ in Occupied Palestinian Territory: UN human rights expert

Fleeing violence, Cameroonian refugee arrivals in Nigeria pass 30,000, reports UN refugee agency

Why will Paris upcoming “loose” climate change agreement work better than the previous ones?

‘Multiplicity’ of rights violations in Ukraine as fifth winter of conflict bites

This woman changed the world of work – and you’ve probably never heard of her

Why the agtech boom isn’t your typical tech disruption

World Mental Health Day is a chance to listen to those with the condition – the biggest experts of all

EU legislation protecting home buyers approved in Parliament

WEF Davos 2016 LIVE: “European unity and cooperation is being called on question”, Vice President Joe Biden criticizes from Davos

Palestine refugees’ relief chief warns Security Council money to fund Gaza operations will run out in mid-June

Girls still being treated as aliens in medicine in the 21st century

US-North Korea summit ‘an important milestone’ towards denuclearization, says Guterres

Mental health at stake: A silent epidemic of 21st century

How China Mended My Heart

Over $39 million earmarked by UN-backed fund to combat effects of climate change in Nepal

How India is solving its cooling challenge

Transparency is key to inclusive employment and government integrity

How drones can manage the food supply chain and tell you if what you eat is sustainable

Two shipwrecks add to ‘alarming increase’ in migrant deaths off Libya coast: IOM

India is failing 175 million of its young people. Here’s the solution

UN commission agrees roadmap on ensuring women’s social protection, mobility, safety, and access to economic opportunities

UN investigates systematic sexual violence across South Sudan

Environmental labelling, information and management schemes are central to the circular economy

MWC 2016 LIVE: Qualcomm looks to pick up Hamilton’s winning ways

French election: Will France vote for a reformed or no EU?

Why philanthropy for – and by – Africans is the future

Data show EU Economy in a stubbornly subdued state

The megatrend that will shape our working future

Healing of ozone layer gives hope for climate action: UN report

Why we need artists who strive for social change

Getting African Women into the Boardroom

How start-ups will lead India through the Fourth Industrial Revolution

UN climate summit aims to speed up transition to cleaner, greener future

Industrial producer prices on free fall and stagnant output

How Hawaii plans to be the first US state to run entirely on clean energy

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s