Why we need global minimum quality standards in EdTech

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: Natalia Kucirkova, Professor of Children’s Reading and Development, The Open University and University of Stavanger

• The lack of universal monitoring standards for EdTech quality and impact hinders progress towards education for all.
• Ensuring that we meet the UN’s Sustainable Development Goal 4 by 2030 requires clear minimum quality standards for EdTech.
• Policy-makers need to move toward a multidimensional impact measurement and management across the EdTech sector.

The UN’s Sustainable Development Goal (SDG) 4 is a commitment to “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.” It is a global imperative. Meeting SDG 4 will rely on scaling the impact of high-quality educational technology (EdTech) the world over.

However, a glance at the standards for impact assessments, such as the IRIS+ metrics catalogue or the Impact-weighted framework, reveals quality metrics tailored for social, environmental and financial impact only — the EdTech sector lacks universal monitoring standards for its quality and impact. Why, and what can be done about it?

EdTech investors rarely ask for measures of learning effects when making their investments. Instead, they favour EdTech solutions with large user bases, prioritizing profitability over educational efficacy. Global agencies, such as UNESCO, report that the industry’s lack of accountability, characterized by scaling without evidence of positive impact, has resulted in the widespread distribution of low-quality products in schools.

After the COVID-19 pandemic and the widespread adoption of online learning, many national government committees, expert groups and reports have been tasked with addressing the question of digital education. Parents and educators require clarity on the best technology for their children, but without robust accountability frameworks, trust in governments’ approaches is elusive.

Meanwhile, the industry defines its own quality standards: numerous commercial and non-profit certifications, quality awards and badges are available, ranging from research-grounded ones like ESSA badges to pedagogical reviews such as ISTE, and even purely commercial options akin to pay-for-play schemes. Adopting these for national EdTech catalogues faces a significant hurdle: the lack of local evidence on the impact of specific solutions.

Much of the existing evidence stems from US-based platforms tailored to US student populations, lacking contextual fit elsewhere. Moreover, smaller startups struggle with the financial burden of rapid evaluation cycles, disadvantaging them against larger curriculum providers with robust infrastructure for data-driven impact insights. This dilemma complicates the balance between local innovation and the demand for rigorous impact standards and high-quality EdTech.

Multiple definitions of quality in EdTech

Globally, the EU has been at the forefront of technology regulations and harmonized systems, but an EU EdTech quality framework is yet to be formulated. Currently, states have varying understandings of quality even for the same hardware, resulting in diverse policies across EU member states. For example, as a precautionary step in the absence of positive evidence, many governments have implemented ad-hoc policies, such as banning entire hardware like phones and tablets from schools.

The lack of shared language on quality EdTech is not confined to EU states. Even within individual clearinghouses, different criteria are followed for evaluating the impact of various interventions, resulting in differing recommendations for what works in education. Furthermore, numerous evaluation frameworks exist: A systematic literature review has identified 65 academic and industry evaluation frameworks, each emphasizing different aspects of EdTech effectiveness and efficacy.

Fragmented global approaches to EdTech standards

The multitude of considerations in EdTech quality may explain the currently disparate efforts by global agencies. The World Bank prioritizes cost-effective scaling of EdTech interventions across borders, the OECD emphasizes the international efficacy of specific EdTech products tailored for distinct learning outcomes (e.g., reading) and UNESCO leads efforts on the ethics of emerging EdTech tools (e.g., Generative AI) modelled after European standards.

Measuring various aspects of impact allows for diverse methods and assessments, but at some stage, consolidation is necessary to systematize the overall impact. This critical moment is long overdue. To tackle the global issue of low EdTech quality, the current disjointed approaches must be consolidated into a unified EdTech quality framework.

To realize this ambitious goal, global agencies should form an inter-agency that launches with the formulation of clear minimum quality standards.

Minimum quality standards of EdTech

Consensus must be reached regarding the minimum quality standards expected of EdTech. For example, all countries agree that manipulative features and advertisements in Edtech products sold to schools are unacceptable. Or that establishing a basic logic model for how the technology generates impact and grounding it in published research is crucial. They agree that involving teachers in the process and ensuring the platform meets certain pedagogical criteria is essential. Moreover, EdTech should not harm the planet and must comply with GDPR and human rights regulations.

The minimum standards can be the first step towards the bigger vision here: a multidimensional index for measuring EdTech impact globally. Such an index should incorporate weighted indicators across multiple impact areas and be interoperable with existing global impact metrics, so that it can inform systemic changes on states-level.

Towards a multidimensional EdTech impact index

Considering the various ways in which technology influences education and learners, at least five areas of quality should be examined:

1. Efficacy: The product’s performance in controlled experiments.

2. Effectiveness: Performance in real classroom settings.

3. Ethics: Implementation with regard to data safety.

4. Equity: Beneficiaries, including marginalized groups.

5. Environment: Local and wider environmental impacts.

These “5Es of impact” are mutually reinforcing. For instance, it is imperative that equity and ethics aspects are embedded into efficacy and effectiveness models from the outset, especially concerning the cost-effectiveness question when technology scales, because this contributes to sustainability and replication across multiple environments.

Mandating data measurement without the necessary infrastructure would be futile. Therefore, the inter-agency’s role should prioritize mobilizing regulators and investors to establish accountability mechanisms that clearly define and prioritize each aspect of impact across the value chain, involving users, investors and founders. Impact creation should be driven by academia-industry partnerships that can generate accurate and verifiable measures of outcomes. These measures can then feed into interoperable databases, facilitating the establishment of research-based certifications and quality badging.

With incentives and infrastructure mechanisms in place to embed impact metrics into products, the EdTech sector can transition from minimum quality standards to desirable, even exceptionally positive, quality in terms of its impact on users. With clear minimum quality standards readily available to educators and parents, the EdTech sector can proactively address emerging risks and act on demands for quality. Crucially, EdTech can move away from improvised impact measures to the development of impactful technologies from the outset.