Big tech cannot crack down on online hate alone. We need to fund the smaller players

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Melina Sánchez Montañés, Managing Director, Innovation Fund at Alfred Landecker Foundation


  • Online hate can be difficult to identify, given its new and sophisticated forms.
  • Tech companies should move beyond detection and moderation of offensive and defamatory content toprotect impacted communities.
  • More catalytic funding is needed to accelerate early-stage technologies that promise to combat disinformation, hate and extremism in novel ways.

Are you one of the billion active TikTok users? Or are you rather the Twitter type? Either way, chances are you have come across hateful content online.

Hate speech starts offline – and can be accelerated by threats to society. COVID-19 is one such example: the pandemic has fuelled a global wave of social stigma and discrimination against the “other”.

Not surprisingly, anti-Semitism, and more largely racism, is on the rise. A study conducted by the University of Oxford unveiled that around 20% of British adults endorse statements like “Jews have created the virus to collapse the economy for financial gain” or “Muslims are spreading the virus as an attack on Western values.”

The internet is where these beliefs can become mainstream. As the new epicenter of our public and private lives, the digital world has facilitated borderless and anonymous interactions thought impossible a generation ago.

Unlike the physical world, however, the internet has also provided a medium for the exponential dissemination and amplification of false information and hate. And tech companies know it. In 2018, Facebook admitted that its platform was used to inflame ethnic and religious tensions against the Rohingya in Myanmar.

As the lines between online and offline continue to blur, we have a tremendous responsibility: to ensure a safe digital space for all. The opportunity lies in deploying catalytic funding to innovative technologies that combat disinformation, hate and extremism in novel ways.

The butterfly effect of social media

Even if only 1% of Tweets contained offensive or hateful speech, it would be the equivalent to 5 million messages daily. It is not difficult to imagine the consequences of such virality – the Capitol siege on 6 January painfully exemplifies how social media can incite violence so quickly.

Unlike violent extremism, hate speech is often subtle or hidden in between terabytes of content uploaded to the internet every day. Pseudonyms like “juice” (to refer to Jewish people) or covert symbols (like the ones in this database) feature frequently online.

They are also well-documented by advocacy organisations and academic institutions. The Decoding Antisemitism project, funded by the Alfred Landecker Foundation, leverages an interdisciplinary approach – from linguistics to machine learning – to identify both explicit and implicit online hatred by classifying secret codes and stereotypes.

However, the bottleneck is not how to single out defamatory content, but how to scan platforms accurately and at scale. Instagram offers users the option to filter out offensive comments. Twitter acquired Fabula AI to improve the health of online conversations. And TikTok and Facebook have gone so far as to set up Safety Advisory Councils or Oversight Boards that can decide what content should be taken down.

With just these efforts alone though, tech companies have failed to spot and moderate false, offensive or hateful content that is highly context, culture and language dependent.

The dark holes of disinformation

Online hatred is ever-evolving in its shape and form to the extent that it becomes increasingly difficult to uncover. Facebook is only able to detect two-thirds of altered videos (also known as “deepfakes”). Artificial Intelligence (AI) and Natural Language Processing (NLP) algorithms haven’t been quick enough in cracking down on trolls and bots that spread disinformation.

The question is: did technology fail us, or did people fail at using technology?

Identifying altered videos through machine learning Image: Defudger.com

The 4Ps of combating online hate

Unless tech companies want to play catch-up on a constant basis, they should move beyond detection and content moderation to a holistic and proactive approach to how hatred is generated and disseminated online (see chart below).

Such an approach would have the following four target outcomes:

Promoting diversity and anti-bias: Technologies can be designed and developed in an inclusive manner by engaging those who are affected by online hate and discrimination. For example, the Online Hate Index by the Anti-Defamation League uses a human-centered approach that involves impacted communities in the classification of hate speech.

Preventing discrimination and violence: Certain tech designs, like recommendation engines, can accelerate pathways to online radicalisation. Others promote counter-speech or limit the virality of disinformation. We need more of the latter.

The re-direct method employed by social enterprise Moonshot CVE channels internet users who search for violent content towards alternative narratives.

Protecting vulnerable groups: Tech platforms have primarily focused on semi-automated content moderation, through a combination of user reporting and AI flagging. However, new approaches have emerged. Samurai Labs’ reasoning machine can engage in conversations and stop them from evolving into online hate and cyberbullying.

Prompting civic engagement: By ensuring that every voice is heard and that citizens – especially younger ones – are empowered to engage in civic discourse, we can help build more resilient societies that don’t revert back to harmful scapegoating. In the US, New/Mode makes it easier for citizens to affect policy change by leveraging digital advocacy tools.

How is hatred generated and disseminated? Image: The author

Funding early-stage innovation

The common denominator of the 4Ps is that, with the help of technology, they address the root of the problem.

Indeed, it is no longer tech for the sake of tech. From Snapchat’s Evan Spiegel to SAP’s Christian Klein, dozens of CEOs signed President Macron’s Tech for Good call a few months ago.

Beyond pledges, companies are embracing technology’s potential to be a force for good by setting up mission-driven incubators, like Google’s Jigsaw, or by allocating funds to incentivise fundamental research, like WhatsApp’s Award for Social Science and Misinformation.

In conversations with various research organisations, such as the Institute of Strategic Dialogue (ISD), I have learnt firsthand about the demand for tech tools in the online hate and extremism space. Whether it is measuring hate real-time across social media or identifying (with high levels of confidence) troll accounts or deepfakes, there is room for innovation.

But the truth is that the pace and the various ways in which tech is being used to incite and promote online hatred is faster and more intricate than what companies can preempt or police. If the big platforms can’t solve the conundrum, we need smaller tech companies that will. And that is why catalytic funding for risky innovation is key.

Step aside big tech

With the increase of hatred due to COVID and the higher demand for new solutions, it is only natural that funders and investors become interested in scaling for-profit and non-profit early-stage tech tools. Established venture capital players like Seedcamp (investor in Factmata) and companies like Google’s Jigsaw are starting to bridge the gap between supply and demand to combat disinformation, hate and extremism online.

But we need more. And so I invite you to join me.

  • Online hate can be difficult to identify, given its new and sophisticated forms.
  • Tech companies should move beyond detection and moderation of offensive and defamatory content toprotect impacted communities.
  • More catalytic funding is needed to accelerate early-stage technologies that promise to combat disinformation, hate and extremism in novel ways.

Are you one of the billion active TikTok users? Or are you rather the Twitter type? Either way, chances are you have come across hateful content online.

Hate speech starts offline – and can be accelerated by threats to society. COVID-19 is one such example: the pandemic has fuelled a global wave of social stigma and discrimination against the “other”.

Not surprisingly, anti-Semitism, and more largely racism, is on the rise. A study conducted by the University of Oxford unveiled that around 20% of British adults endorse statements like “Jews have created the virus to collapse the economy for financial gain” or “Muslims are spreading the virus as an attack on Western values.”

The internet is where these beliefs can become mainstream. As the new epicenter of our public and private lives, the digital world has facilitated borderless and anonymous interactions thought impossible a generation ago.

Unlike the physical world, however, the internet has also provided a medium for the exponential dissemination and amplification of false information and hate. And tech companies know it. In 2018, Facebook admitted that its platform was used to inflame ethnic and religious tensions against the Rohingya in Myanmar.

As the lines between online and offline continue to blur, we have a tremendous responsibility: to ensure a safe digital space for all. The opportunity lies in deploying catalytic funding to innovative technologies that combat disinformation, hate and extremism in novel ways.

The butterfly effect of social media

Even if only 1% of Tweets contained offensive or hateful speech, it would be the equivalent to 5 million messages daily. It is not difficult to imagine the consequences of such virality – the Capitol siege on 6 January painfully exemplifies how social media can incite violence so quickly.

Unlike violent extremism, hate speech is often subtle or hidden in between terabytes of content uploaded to the internet every day. Pseudonyms like “juice” (to refer to Jewish people) or covert symbols (like the ones in this database) feature frequently online.

They are also well-documented by advocacy organisations and academic institutions. The Decoding Antisemitism project, funded by the Alfred Landecker Foundation, leverages an interdisciplinary approach – from linguistics to machine learning – to identify both explicit and implicit online hatred by classifying secret codes and stereotypes.

However, the bottleneck is not how to single out defamatory content, but how to scan platforms accurately and at scale. Instagram offers users the option to filter out offensive comments. Twitter acquired Fabula AI to improve the health of online conversations. And TikTok and Facebook have gone so far as to set up Safety Advisory Councils or Oversight Boards that can decide what content should be taken down.

With just these efforts alone though, tech companies have failed to spot and moderate false, offensive or hateful content that is highly context, culture and language dependent.

The dark holes of disinformation

Online hatred is ever-evolving in its shape and form to the extent that it becomes increasingly difficult to uncover. Facebook is only able to detect two-thirds of altered videos (also known as “deepfakes”). Artificial Intelligence (AI) and Natural Language Processing (NLP) algorithms haven’t been quick enough in cracking down on trolls and bots that spread disinformation.

The question is: did technology fail us, or did people fail at using technology?

Identifying altered videos through machine learning Image: Defudger.com

The 4Ps of combating online hate

Unless tech companies want to play catch-up on a constant basis, they should move beyond detection and content moderation to a holistic and proactive approach to how hatred is generated and disseminated online (see chart below).

Such an approach would have the following four target outcomes:

Promoting diversity and anti-bias: Technologies can be designed and developed in an inclusive manner by engaging those who are affected by online hate and discrimination. For example, the Online Hate Index by the Anti-Defamation League uses a human-centered approach that involves impacted communities in the classification of hate speech.

Preventing discrimination and violence: Certain tech designs, like recommendation engines, can accelerate pathways to online radicalisation. Others promote counter-speech or limit the virality of disinformation. We need more of the latter.

The re-direct method employed by social enterprise Moonshot CVE channels internet users who search for violent content towards alternative narratives.

Protecting vulnerable groups: Tech platforms have primarily focused on semi-automated content moderation, through a combination of user reporting and AI flagging. However, new approaches have emerged. Samurai Labs’ reasoning machine can engage in conversations and stop them from evolving into online hate and cyberbullying.

Prompting civic engagement: By ensuring that every voice is heard and that citizens – especially younger ones – are empowered to engage in civic discourse, we can help build more resilient societies that don’t revert back to harmful scapegoating. In the US, New/Mode makes it easier for citizens to affect policy change by leveraging digital advocacy tools.

How is hatred generated and disseminated? Image: The author

Funding early-stage innovation

The common denominator of the 4Ps is that, with the help of technology, they address the root of the problem.

Indeed, it is no longer tech for the sake of tech. From Snapchat’s Evan Spiegel to SAP’s Christian Klein, dozens of CEOs signed President Macron’s Tech for Good call a few months ago.

Beyond pledges, companies are embracing technology’s potential to be a force for good by setting up mission-driven incubators, like Google’s Jigsaw, or by allocating funds to incentivise fundamental research, like WhatsApp’s Award for Social Science and Misinformation.

In conversations with various research organisations, such as the Institute of Strategic Dialogue (ISD), I have learnt firsthand about the demand for tech tools in the online hate and extremism space. Whether it is measuring hate real-time across social media or identifying (with high levels of confidence) troll accounts or deepfakes, there is room for innovation.

But the truth is that the pace and the various ways in which tech is being used to incite and promote online hatred is faster and more intricate than what companies can preempt or police. If the big platforms can’t solve the conundrum, we need smaller tech companies that will. And that is why catalytic funding for risky innovation is key.

Step aside big tech

With the increase of hatred due to COVID and the higher demand for new solutions, it is only natural that funders and investors become interested in scaling for-profit and non-profit early-stage tech tools. Established venture capital players like Seedcamp (investor in Factmata) and companies like Google’s Jigsaw are starting to bridge the gap between supply and demand to combat disinformation, hate and extremism online.

But we need more. And so I invite you to join me.

the sting Milestones

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

Better care, stronger laws needed to save 30 million babies on the brink of death

Four in five adolescents failing to exercise for even 60 minutes a day, UN health agency warns

Step up action to protect the planet during wartime: UN environment chief

Strength in unity: Commission makes recommendations for the EU’s next strategic agenda 2019-2024

Hydrogen isn’t the fuel of the future. It’s already here

Cleantech innovation is being stifled. Here’s how to unlock it

European Parliament to commemorate 76 years since the liberation of Auschwitz

COVID-19: National authorities should do more to raise awareness of EU action

Why Eurozone’s problems may end in a few months

Access to health in the developped and developing world

Everybody against Germany over the expensive euro

Financial stability: Commission addresses risks of Libor cessation

Antisemitism, intolerance, can be unlearned, Guterres tells New York commemoration

Agreement reached on digital copyright rules

Banks suffocate the real economy by denying loans

Business is stepping up its fight against climate change. This is how

Climate finance for developing countries reached USD 71 billion in 2017

Will the French let Macron destroy their party political system?

UN chief calls for ‘a fair globalization’ with first-ever Global Goals Summit

What are Asia Pacific countries getting right in the fight against cancer?

AI-powered automation will have an ethnic bias

5 ways the ocean can contribute to a green post-COVID recovery

We need to talk about how we define responsibility online – and how we enforce it

Countries must up their game to reduce low birth weights, warns UN-backed report

Cultural Intelligence: the importance of changing perspectives

‘Stay at home’ UK tells people as global confirmed cases pass 380,000 – Today’s coronavirus updates

This is how AI can help you make sense of the world

Poverty and social exclusion skyrocket with austerity

George Floyd: these are the injustices that led to the protests in the United States

Green Deal: measures to step up the fight against global deforestation

FROM THE FIELD: Changing world, changing families

Brexit: MEPs concerned about citizens’ rights

How a new approach to meat can help end hunger

Assassinations in Ethiopia amidst regional ‘coup’ attempt, condemned by UN chief

SPB TV @ MWC14: The TV of the Future

EU elections update: Can the EU voters vote unaffected from fake news and online disinformation?

ECB offers cheaper money despite reactions from Germany

Global aid appeal targets more than 93 million most in need next year

Sweden has a plan to end all traffic accident deaths

5 charts that show renewable energy’s latest milestone

Why do medical students need to go abroad to become a doctor in 2017?

Afghanistan extends ceasefire with Taliban; UN urges both sides to work towards lasting peace

4 tech innovations that support people with disabilities working from home

Migration crisis update: The “Habsburg Empire” comes back to life while EU loses control

5 things to know about the Western Balkans

Blockchain is becoming key for global trade – but is that a gift for hackers?

How a new encryption technique can help protect privacy amid COVID-19

5 ways students can graduate fully qualified for the Fourth Industrial Revolution

UNICEF backpacks used as a haunting symbol to call for greater protection of children living in conflict

Job automation risks vary widely across different regions within countries

Sanity in times of COVID-19

UN chief urges peaceful, free and fair elections in Cameroon

The refugee crisis seen through the eyes of a young doctor from Turkey

3 charts that show how attitudes to climate science vary around the world

At global health forum, UN officials call for strong, people-focused health systems

International Women’s Day 2021: COVID-19 pandemic is a major challenge for gender equality

Do doctors need to know their patients’ sexual orientation and gender identity?

ILO’s Bureau for Employers´Activities to publish new study on women in business and management

Brexiteer May gets lip-service from Trump and Turkish promises from Erdogan

EU and World Health Organisation team up to boost access to health services in developing countries

More Stings?

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s