In 2023, The UNESCO Gem Report underscored the imperative for enhanced evidence in educational technology (EdTech): there's a lack of evidence for EdTech’s effectiveness in children’s learning, prompting the United Nations to advocate for teachers, policymakers, governments, and providers to prioritize research when making decisions about whether and which technology to use with children.

Paradoxically, there is no shortage of EdTech evaluation frameworks -  a UK Department of Education rapid evidence review found 74 quality evaluation frameworks. In our systematic research review, which focused specifically on EdTech effectiveness and efficacy evaluations, we identified another set of 65 frameworks. The substantial number of frameworks might be viewed as either a lack of shared definition of quality or, conversely, as an indication of a strong interest in the quality of EdTech.

To establish potential overlaps and synergies across the frameworks, our review consolidated the various frameworks based on the strength of evidence demanded. We used established rubrics and the EdTech Evidence Evaluation Routine (EVER) to gauge the weight of various evaluation rubrics. We emphasized the importance of focusing on the strength of evidence rather than its type, recognizing the significance of both quantitative and qualitative studies in evaluating what works in learning.

The focus on the strength of evidence rather than its type might be a way forward in a rapidly developing "market of EdTech evidence". Focusing on the quality of evidence is part of nurturing an evidence mindset within the entire EdTech ecosystem. This mindset involves asking questions about the strength of research with a tool, the research study's internal and external validity, and asking questions such as who conducted the study, was this an empirical study testing the solution in classrooms or merely a desktop review. The focus on detailed questions around evidence encourages critical reading and usage of EdTech, collectively pushing for quality.

This responsibility for quality aligns with the 2030 Agenda for Sustainable Development, emphasizing the importance of collective action and outcomes. The 17 Sustainable Development Goals (SDGs) provide a blueprint for peace and prosperity, so EdTech development and uptake should align with the goals.  Quality of education  is one of the SDGs, given that education is the cornerstone of socio-cultural-economic development and plays a crucial role in eradicating poverty and inequality.

In understanding the impact of evidence in EdTech, several frameworks and model exist. The umbrella model of the  5Es—efficacy, effectiveness, ethics, equity, and environment—outlines the essential impact verticals to consider in education and EdTech. These five verticals can be understood as five impact pathways, each requiring rigorous and objective metrics. Efficacy evaluates technology's impact on learning, social, and economic outcomes, while effectiveness assesses the extent to which the EdTech can address cost, pedagogical, local, and infrastructural challenges. The equity vertical examines an EdTech's contribution to social justice and gender equality. Ethical practices, including transparent production methods, form the fourth aspect, while the fifth aspect pertains to environmental considerations.

In 'Evidence of Impact,' the 5E Model serves as an inspirational guide for educational organisations or EdTech companies, directing them towards a holistic understanding of impact. This understanding is built on various pillars that companies must embrace. It is s essential to strike a balance among these pillars, rather than allowing one to dominate over the others. This notion of balance is part of several other popular 5Es Models.  For example, the 5E model for Leadership, which teachers leaders to be envision, express, excite, enable and execute (Dupuis, 2020) or the 5E Model of instruction, which suggests that learning should progress through five phases of Engage, Explore, Explain, Elaborate, and Evaluate.

The 5Es of Evidence map onto all 17 SDGs, with each vertical speaking more strongly to a set of goals. For example, efficacy relates to SDG 3 (ensuring positive health and wellbeing outcomes through the use of an EdTech tool or an educational approach); 4 (ensuring quality education through rigorously tested tools and approach) and 8 (ensuring decent work and economic growth through efficacious technologies and strategies).  Certainly, as observed in other models, there exists a mutual influence among the SDGs. While one SDG may hold prominence in a particular area, it remains relevant and impactful across other areas as well. The key objective is to foster a balance in striving to achieve progress across all SDGs in all areas of impact.

That said, it is interesting to note that the environmental impact pathway is most represented by the SDGs, yet no current EdTech certification specifically addresses the environmental impact. This might change in 2024 as more badges and certifications of EdTech quality emerge on the market. However, relying solely on evaluation frameworks and quality certifications won't drive significant change. Elevating the quality of EdTech and substantiating its positive impact demands collective action. This begins by crafting meaningful impact pathways and measurable metrics across various dimensions of quality, followed by their diligent implementation in practice.