An expert from Edge Hill University who specialises in the science behind big data explains why he believes OfQual’s algorithm failed to make the grade.
Students, teachers and education providers across the UK were up in arms following the Government’s exam results debacle earlier this month which first saw 39.1% of predicted results downgraded by OfQual’s model and was then later replaced with teacher assessed grades.
Dr Marcello Trovati, a Professor in Data Science at Edge Hill, has been following the debate closely and believes the algorithm used failed due to several incorrect assumptions and pre-conceptions.
“Loosely speaking, the OfQual Direct Centre Performance model assumes that a school or college will perform similarly compared to the previous three years,” he said. “In other words, students were marked, with some potential adjustment depending on specific prior GCSEs, based on grades achieved by students of that school over the previous three years.
“If a school scored low in the past it is also assumed to score low in the current year, and vice-versa. Apart from the obvious observation that this approach is extremely likely to perpetuate inequality, and completely ignores any improvement in teaching or student aptitude, the science behind it is rather flimsy, to say the least. This is not a sophisticated machine learning or AI system, which is learning based on actual performance. It just churns out prediction based on a ‘fixed’ set of calculations.
“Most notably, the high calibre contribution which was offered to OfQual by the Royal Statistical Society was not accepted due to a Non-Disclosure Agreement, which is contrary to the Society’s commitment to transparency and public trust.”
Dr Trovati added: “We are often reminded that this is the digital era, where artificial intelligence will soon manage, and hopefully improve our lives. The Government has been urging industries, institutions and individuals to embrace this digitalisation, making this hugely impactful blunder more difficult to comprehend and justify.”
With a background in big data, Dr Trovati’s research includes mathematical modelling, data and text mining, and their applications to multi-disciplinary topics.
For more information about courses in computer science, visit our website.