Risks to kids online are growing. Here’s what we can do 

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Farah Lalani, Community Lead, Global Coalition for Digital Safety, World Economic Forum

• Apple delayed its rollout of child safety features, and Instagram delayed Instagram for Kids.
• Facing accusations that its platforms harm children, Facebook says it will introduce new safety controls.
• These developments could alter the future of the web for kids.
• Here’s what we know about safety for children online – and how to improve it.

Discussion of the risks to kids online – and the wider implications to society when mitigating these risks – is currently on center stage.

Apple recently delayed its rollout of child safety features on US phones and computers, and Instagram delayed its rollout of Instagram for Kids. In response to accusations that its platforms harm children, Facebook, which owns Instagram, this week said it would introduce new safety controls, including asking kids to take a break when using Instagram and steering them away from content that isn’t conducive to well-being.

With children making up a significant portion of internet users – one in three internet users is a child under 18 years old – the impact of such decisions will notably alter the future of the web for kids.

How risky is it for kids to be online?

The risks to kids online are significant and growing. While the risks vary by age, gender, online exposure and other factors, as it stands right now at an aggregate level, exposure to sexual content is the largest online risk.

UNICEF research shows that risks vary significantly by country and that the extent of harm caused by these risks depends on a number of factors, such as children’s resilience, parental guidance/support, and digital skills (such as ability to manage privacy settings).

The risk to young girls is even more significant. The State of the World’s Girls 2020 report by Plan International, which surveyed over 14,000 girls and young women in 31 countries, found that more than half of respondents had been harassed and abused online, and that one in four felt physically unsafe as a result.

What is the industry doing about it?

Many companies are making improvements to protect children online. For example, TikTok has a version of its app for users under 13 where they can create but not post videos. YouTube’s Kids App has an “Approved content only” mode, so children can only watch videos, channels and/or collections that parents have selected and approved themselves. Instagram is preventing adults from messaging children who do not follow them and defaulting all child accounts to private.

Nevertheless, a number of challenges must be overcome to keep children safe online whilst enabling them to benefit from all the opportunities of digital engagement.

Key tensions and barriers to child safety online

Age Verification

Many kids are clearly being exposed to content they shouldn’t be seeing in places they shouldn’t be at all. The largest major survey of its kind points to a traumatic reality: 70% of respondents first saw child sexual abuse material when they were under 18.

In the US, many platforms do not permit users under 13 years of age to use their services in order to comply with the Children’s Online Privacy Protection Act. Many other parts of the world operate similarly. Nevertheless, it is estimated that a staggering 82% of children aged between 8 and 12 have profiles on social media and messaging apps, according to research from CyberSafeKids.

Studies have found that young people of all ages can sidestep age verification measures when signing-up to popular social media apps like Snapchat, Instagram, TikTok and Facebook by simply lying about their age. While new regulation such as the UK’s Age-Appropriate Design Code sets out requirements for how many digital platforms should protect kids online, it is not prescriptive in dictating how companies should verify age of users.

Doing so in a way that is safe is a major concern. Some believe that innovations in third-party technologies can offer solutions that are not privacy-invasive, but major platforms today mainly still rely on self-declaration. More broadly, verification has become an important safety issue across the tech and media industry as issues of inauthentic behavior seek to be addressed. For example, Tinder has rolled out an ID Verification process to help users feel confident that their matches are real people. Improving age and identity verification processes, technologies and standards will become increasingly critical for user safety given the growing risks across the web.

Privacy

Heightened privacy measures can protect kids from unwanted contact on social applications and help children manage visibility of photos, videos and other content they share with strangers. At the same time, certain privacy measures can complicate children’s safety. For example, detecting illegal material, such as Child Sexual Abuse and Exploitation Material (CSAM), by proactively scanning, monitoring and filtering user content currently cannot work with end‑to‑end encryption.

However, many companies are moving toward encrypted environments nonetheless. The National Center for Missing or Exploited Children (NCMEC) estimates that 70% of child abuse reports could be lost with the roll‑out of end-to-end encryption on the Facebook Messenger platform. While many experts continue to tout the value of encryption, the industry is struggling to come up with solutions that would enable detection of CSAM in these environments.

More broadly, appropriately assessing the balance between privacy and safety in company decisions about product features, content moderation, user profile settings, and other such areas will require a deep understanding of the implications to all stakeholders so that proportionate measures are put in place.

Risk Exposure

Facebook has said its development of Instagram for Kids, which is now on hold, would help ensure that kids use an app designed for them, requiring parental permission to join and devoid of advertising. Sceptics of this plan point to concerning research on the impact of social media; for example, for young girls who struggle with body image, internal Facebook research shows that Instagram makes this worse. Therefore, building an app targeted to kids with the potential of furthering such harm seems unpalatable to many.

Nevertheless, the same research shows the benefits to young girls of using Instagram in other areas. On other apps, such as YouTube, the majority of parents whose children watch videos on these platforms say it helps their children learn new things and keep them entertained; at the same time, 46% of parents have said their child encountered inappropriate videos on YouTube.

Clearly, the benefits of using such apps have a trade-off with the potential risks. The ongoing conversation about these risks and benefits highlights the nuances of this debate and the individual factors that parents may want to consider when making such decisions (to the extent possible) for their kids.

Preventative Interventions

In general, preventative measures have not gained the same traction as measures to detect and remove harmful content once it is already circulating. While a number of interventions could help improve safety for kids online, understanding their effectiveness in real-world settings is key to avoiding unintended consequences.

For example, one of the parts of Apple’s announcement related to warning kids about sharing sexually explicit photos. An analysis of 39 studies found that 12% of young people forwarded a sext, or sexually explicit image or video, without consent, and 8.4% had a sext of themselves forwarded without consent. Clearly, this needs to be addressed. However, research has shown that warnings about these types of privacy violations often backfire and could even have the opposite effect.

This highlights the importance in a multistakeholder approach to such interventions, informed by leading academic research, together with input from civil society and government so that robust interventions are proposed based on a diverse set of expert inputs across the public and private sector.

Information Inequalities

It is not always easy to assess current and proposed safety measures given the different understanding of technologies, processes, and constraints amongst stakeholders outside of industry. When it comes to critically assessing changes, such as what Apple had proposed, regulators, researchers and other key stakeholders often lack the visibility, data, and/or understanding to do so effectively.

Building mechanisms to share information, enable audits, and independently verify claims with such stakeholders is critical to gaining buy-in; doing so in a privacy-preserving way that does not compromise company IP can bridge the information gap, which is necessary to develop more agile, effective, and acceptable policies in the future.

How do we address digital risks?

A holistic approach to safety that considers both the risks and opportunities of digital engagement is needed to move forward. This can only be done with a user-centered approach, when considering the safety tensions associated with new technologies, policies, or products.

The Forum’s Global Coalition for Digital Safety aims to develop principles and practices that help address such trade-offs in a proportionate manner, knowing that solutions will need to be tailored according to a company’s size, role in the digital ecosystem, the type of user it services, and many other factors. While there isn’t a one-size-fits-all solution, there is a path forward to build solutions that will enable kids to take advantage of the internet whilst staying safe.