This paper introduces Edustories, a dataset of real classroom case studies, and uses it to evaluate how well Large Language Models can predict the success of teacher interventions, finding they still lag behind human experts.

In plain terms

Most AI tools in education focus on helping individual students (called individualized student assistance), but teachers often work with groups in collective classroom settings. This paper addresses the gap by creating Edustories, a new dataset of 1,492 real-world descriptions written by elementary and high-school teachers. These case studies detail challenging student behaviors, the specific actions teachers took (called pedagogical interventions), and the resulting outcomes. Researchers used this dataset to test how well powerful AI systems, known as Large Language Models (LLMs), could predict the success of these teacher interventions. They found that while LLMs showed promise, the most advanced AI models achieved 58% accuracy in predicting outcomes, falling short of human experts who reached 64%. This highlights both the current limits and future potential of AI as assistants for practicing teachers.