Higher education news for libraries and librarians

Educational NLP systems are typically evaluated using engagement metrics and satisfaction surveys, which are at best a proxy for meeting pedagogical goals. We introduce six computational metrics for automated evaluation of pedagogical alignment in student-AI dialogue. We validate our metrics through analysis of 12,650 messages across 500 conversations from four courses. Using our metrics, we identify a fundamental misalignment: educators design conversational tutors for sustained learning dialogue, but students mainly use them for answer-extraction. Deployment context is the strongest predictor of usage patterns, outweighing student preference or system design: when AI tools are optional, usage concentrates around deadlines; when integrated into course structure, students ask for solutions to verbatim assignment questions. Whole-dialogue evaluation misses these turn-by-turn patterns. Our metrics will enable researchers building educational dialogue systems to measure whether they are achieving their pedagogical goals.

We are the voice of the higher education sector in London

Access your digital copy of our latest white paper below Use the button below to download your free digital copy of Educating the AI generation, produced in partnership with Wonkhe. We’ll also send a copy straight to your inbox.

Update on disruption to education systems across Gulf Cooperation Council (GCC) countries and Iraq.

The Russell Group calls for international PhD students to be exempt from the International Student Levy to protect the UK's R&D talent pipeline and continue to attract global talent

This month the C.H.A.S.E consortium has launched a Code of Conduct, a voluntary, human-rights-based framework for media organisations committed to preventing and addressing online gender-based violence. Signatories commit to zero tolerance for gender-based online violence, responsible moderation practices, staff training, transparent complaint mechanisms, and regular public reporting on compliance. The Code is aligned with EU legal standards, including the Digital Services Act,

The emergence of Generative Artificial Intelligence (GenAI) technology and the increasing sophistication of such tools have raised concerns about their imp

The latest results for the English Social Mobility Index, which is compiled by London South Bank University (LSBU) and published by the Higher Education Policy Institute (HEPI), shows the University of Bradford, Aston University and the University of Wolverhampton in the top three places. The annual LSBU English Social Mobility Index is published each year by […]

New research from Wonkhe finds that assessment design – not AI policy – is what determines whether students use AI to learn or to produce. Jim Dickinson and Mack Marshall explain

Jisc Online surveys is a powerful, easy to use tool for creating online surveys. Run by Jisc, online surveys is used by over 300 different organisations in th

Applies to England About this advice note This advice note supports awarding organisations (AOs) in understanding how the existing Conditions of Recognition[footnote 1] and related Guidance apply to the risks of malpractice arising from Learners’ use of artificial intelligence (AI) tools.

For David Mather, student progression may not be a proxy for wellbeing

Universities must scale up collaboration to deliver green economy transitions, new report warns Universities have a pivotal role to play in tackling climate change, but fragmented collaboration and disciplinary boundaries are slowing progress, according to a new Debate Paper from the Higher Education Policy Institute (HEPI), sponsored by the University of Salford. A clear and […]

A report from Prospects and the ISE explores how young people and employers view how AI is shaping the early careers landscape.

This briefing note examines how being in deep poverty in childhood – proxied by eligibility to free school meals at age 16 – continues to shape graduates’ earnings long after university. Using the Longitudinal Education Outcomes dataset to track the education and work pathways of 520,000 graduates...