Closing the attainment gap - how we did it (and how we measured it!)
Assessing the impact of any strategy or intervention is a notoriously difficult task, particularly when one considers the external factors at play in any given setting that can dramatically affect the results. When decisions around purchasing or adequately resourcing implementation strategy come into play, the requirement to show evidence becomes increasingly vital to school leaders.
It’s no less of a vexed question for providers: finding evidence of impact is crucial to decision-making processes around the development of products and content, and to be able to assure customers that their investment will prove to be time and money well-spent.
We’ve conducted many trials and experiments over the years to measure impact - and pride ourselves in being a leader in research-led, evidenced EdTech. Having completed an impact study with the IoE into the effect of our videos on knowledge retrieval and retention, Tassomai was awarded an EdWard from UCL. Our correlative analysis on 60,000+ exam results against usage in the product has been well-publicised, showing that over 90% of high users achieved a strong pass in their GCSE.
The challenge, however, was to investigate the effect of our adaptive, personalised spaced retrieval practice on students’ academic progress - we wanted to demonstrate that, whatever the prior attainment, using Tassomai would increase outcomes.
You can find out more detail about the research in this blog post, but in short the experiment was a pre & post test of student subject knowledge on what we term ‘mastery questions’. We looked at the data of around 1500 students around the UK who used Tassomai between September 2018 and May 2019, separated the lower and higher ability based on how they fared on quizzes in their first month of use, and then looked at how their scores in mastery questions improved over time.
Our results showed that, in both groups, students improved in test scores and that this improvement increased with increased usage. Superimposing the graphs for each user type against their respective baselines of attainment, however, showed that the effect of the program was to close the attainment gap between them.
Pre/post test analysis of 1454 students (spread across the UK), measuring initial attainment on Tassomai mastery questions, their progress through Tassomai during the academic year from September 2018 and May 2019, and their final attainment on similar mastery questions. Students were stratified based on their initial attainment, and separated into ‘buckets’ of Tassomai usage. The average usage change in attainment for each bucket was calculated and plotted against the baseline average attainment of the group.
We continue to conduct research on mass cohort data and in collaboration with schools and universities. If you would like your school to participate in a study, please get in touch as we’d love to work with you.