LLMs and RAG systems show a significant "intervention bias" by over-recommending actions for students, whereas supervised machine learning models provide much more accurate and calibrated advice for educational interventions.

In plain terms

When using AI to advise educators on student support, large language models (LLMs) often recommend too many interventions, even when students don't need them—a problem called "intervention bias." This study tested LLMs like GPT-4o and other AI systems, including retrieval-augmented generation (RAG) systems, on a dataset from the Open University, which tracks student performance and ideal intervention points. They found that zero-shot LLMs and RAG systems had a high false-positive rate, recommending actions for many students who didn't actually need help (e.g., 43% unnecessary recommendations at a certain point). However, traditional supervised machine learning models, like a Decision Transformer and an XGBoost classifier, trained specifically for this task, were much more accurate and didn't show this intervention bias. These supervised models were also very fast and could predict the correct action for nearly all students, even for rare situations, and the study also found that common methods for evaluating LLMs often fail to detect this over-prescription bias.