Who should set the rules for AI? The UN is pushing for a safer digital future

This article is published in association with United Nations.

Artificial intelligence (AI) is advancing rapidly, raising questions about how societies and governments can capture its benefits while managing risks that often extend beyond national borders.

Those concerns intensified after a former researcher at the AI company Anthropic warned on 8 September that AI could pose an existential threat to humanity.

For the UN, the best way to address AI’s global risks is to bring countries together to adopt shared principles and commitments on AI governance.

Coordination is ‘indispensable’

“National action is essential, but global coordination is indispensable,” UN Secretary-General António Guterres said on Wednesday.

The UN has worked for several years to provide a forum where countries can develop a shared understanding of AI technologies and discuss how they can cooperate – and provide a regulatory framework – on issues such as safety, human rights and disparities in access.

Rules-based development 

The UN’s current work on AI governance stems from the Global Digital Compact, adopted by Member States in 2024 as part of the UN’s Pact for the Future, exactly two years ago.

The Compact called for international cooperation on AI and proposed two new mechanisms: an independent scientific panel to assess the AI’s risks and opportunities, and an annual global dialogue where governments and other stakeholders could coordinate approaches to AI governance.

In August 2025, the UN General Assembly – UNGA as we call it for short – formally established both.

The Independent International Scientific Panel on AI aims to provide governments with an independent assessment of what is known – and what isn’t known – about the rapidly developing technology.

The Global Dialogue on AI Governance, meanwhile, brings governments and other stakeholders together to discuss international cooperation, share experience and work towards greater compatibility between different approaches to AI governance.

The General Assembly designed the two institutions to complement each other: the Scientific Panel provides the evidence, and countries decide at the Global Dialogue what to do in response.

The Scientific Panel’s first preliminary report, released 1 July, concluded that AI capabilities are advancing rapidly – especially in areas such as reasoning, coding and science – creating significant opportunities. 

It also identified risks such as misinformation, discrimination, privacy violations, cyberattacks and so-called “superintelligence” that could become difficult to control.

“We need a shared understanding of how to advance the safe, secure, and responsible development of AI – while identifying when increasingly powerful systems may require stronger safeguards, or additional measures to manage potential risks,” Mr. Guterres said in his pre-UNGA remarks to the world’s media in New York.

Tweet URL

Working towards AI guardrails

The UN’s work comes as discussions move from whether AI should be regulated, to questions about which safeguards should be implemented.

On Wednesday, Mr. Guterres called for international cooperation to ensure that AI development remains safe, secure and responsible. He added that voluntary efforts to slow AI development will not be sufficient if they are isolated, unverifiable or unevenly applied.

Asked about how the UN could convene a summit as part of a process leading to effective regulation for humankind, he said it could not be done next week – but it should happen soon, with “everybody on board” from the public, private, scientific, and civil society sectors. 

“If we believe AI development should slow when risks become too great, we need more than good intentions,” he said.

The Secretary-General has called for an AI Child Safety Pledge and a Global Fund for AI, which would support developing countries in building AI infrastructure and capacity.

He also stressed the need for guardrails that make AI “safe, transparent, accountable, with human dignity at the centre.”

While national and regional governments make their own decisions about AI regulation, the UN’s role will be to bring these governments together, provide a scientific basis for discussions and identify areas for international cooperation.

AI at UNGA

Some diplomatic sources quoted in news reports speculate that the Security Council will discuss artificial intelligence during the high-level week.

At UN Headquarters on the sidelines of the General Debate  – where AI concerns will no doubt loom large at the podium – the UN’s SDG Media Zone will host an event on 21 September, and the AI and Human Development series will culminate with a discussion on 23 September. Both events will focus on the relationship between AI and human agency.

Mr. Guterres underscored during his press conference Wednesday that AI will be “a major topic” in his discussions with world leaders next week.

Comparing AI development to the Cold War-era arms race between the US and Soviet Union, the Secretary-General called for cooperation between countries at the forefront of AI – including the US and China – to slow down AI development while risks continue to be assessed.

“The countries that have the highest capacity in the frontiers of AI – I think they should establish mechanisms of contact, have an exchange of information and (create) some common guardrails in order to avoid a race to the bottom that could one day lead to a gigantic disaster at the global level,” Mr. Guterres said.

With the scientific and diplomatic mechanisms now in place, the UN will continue to work to turn that cooperation into concrete action as AI becomes increasingly powerful.