OfS reports on the potentially inflationary effect of degree classification algorithms on students’ grades

The report sets out what institutions should consider when making changes to the algorithms they use to determine degree classifications for individual students. This includes testing the classes of award given to students against their actual attainment, and taking reasonable steps to ensure any award accurately reflects their academic achievements.

The OfS is concerned that some practices in the design of algorithms make it difficult to show that awards accurately reflect the achievement demonstrated by students at the point of assessment. These include discounting of credits with the lowest marks and determining a class of degree using the best result from multiple algorithms.

The OfS will monitor and, where appropriate, engage further with individual institutions and the sector more widely where it has concerns about the approach to the use of algorithms.

The report is part of the OfS’s work to help preserve public confidence in the value of degrees following a 15-year net increase in the proportion of first and upper second class awards granted to students. It will support institutions to meet the OfS’s condition of registration B4, which requires them to ensure that awards are credible and hold their value over time.

Alongside monitoring sector data and publishing the new guidance, the OfS has carried out detailed assessments of the changes three universities have made to their classification algorithms for bachelors’ degrees over time.

The OfS found that St Mary’s University, Twickenham, the University of West London, and Teesside University were at increased risk of a future breach of condition B4. All three universities have agreed to conduct the review referred to above to test the classifications awarded to students against their actual attainment and consider whether any amendments are as a result needed to their academic regulations, which include their classification algorithms. The OfS welcomes the commitments each university has made to take these further steps and will use the outcomes from these reviews to inform wider discussion with the sector.

The OfS also found breaches of condition B4 for St Mary’s University, Twickenham and the University of West London. In the past these two universities had introduced new degree classification algorithms that would have increased the number of first and upper second class degrees awarded after condition B4 came into force. Before the current condition came into effect and before the OfS opened the investigations about these matters, both universities had taken appropriate steps to remedy the concerns by making changes to their algorithms. As a result, the OfS is not taking any further action.

Jean Arnold, Deputy Director of Quality at the OfS, said:

‘It’s essential that students, graduates, and employers are confident that higher education qualifications represent an accurate assessment of achievement that stands the test of time. We welcome the progress made by institutions in their efforts to curb the risk of unexplained grade inflation. In recent years, the sector has reduced the proportion of students awarded first-class and upper second class degrees, although levels remain above those awarded in 2010-11.

‘Classification algorithms are an important part of measuring the skills and knowledge of students. We would be concerned if, because of how an algorithm is designed or how it has been altered over time, the classes of award being made to students no longer appropriately reflected the achievement demonstrated by those students at the point of assessment. Our aim as the regulator is to build confidence in the value of degrees. Our findings do not seek to question or dispute the classifications awarded to any individual student or groups of students at any institution, now or in the past.

‘We welcome the fact that the institutions we have engaged with have committed to further review of their academic regulations. We think it is likely that the approaches taken by the institutions we assessed reflect practices found more widely in the sector. The report we are publishing today will support all institutions to ensure that the degree classifications they award properly reflect student achievement and help to protect public trust in the high value of a higher education award.

‘We recognise the requirements of condition B4 are relatively new, and we look forward to engaging with the sector, including through Universities UK and GuildHE, to support the implementation of our guidance. We will monitor sector engagement with this report and consider over time whether further guidance or other action is required. We will also continue to engage with the sector about its approach to assessment and awards more generally.’

Notes

1. The Office for Students is the independent regulator for higher education in England. We are shortly due to launch a new strategy for 2025 to 2030, which seeks to ensure that students from all backgrounds benefit from high quality higher education, delivered by a diverse, sustainable sector that continues to improve.
2. Ongoing condition of registration B4: Assessment and awards requires institutions to ensure that their academic regulations are designed to ensure that awards are credible, and for awards to be credible at the point they are granted and when compared to those granted previously. To be credible an award must, in the reasonable opinion of the OfS, reflect students’ knowledge and skills at the time they are granted and hold their value over time.
3. ‘Algorithm’ refers to the sets of calculations institutions use to determine the class of degree to be awarded to individual students.
4. Read the assessment reports for: St Mary’s University, Twickenham, the University of West London, and Teesside University.
5. Read our analysis of degree classifications over time.