This study uses Large Language Models (LLMs) to distinguish and diagnose specific grammatical errors from unnatural phrasing in Japanese English as a Foreign Language (EFL) students' writing.

In plain terms

Automated writing evaluation tools often mix up simple grammatical mistakes with phrases that don't sound natural to a native speaker, making it hard for language teachers to provide targeted feedback. This study tackled this issue by developing a system that uses Large Language Models (LLMs), which are advanced AI programs capable of understanding and generating human language. Their system analyzes English writing samples from Japanese junior high school students in two distinct layers: first, it identifies clear grammatical errors, and then it looks for unnatural phrasing or less common ways of expressing ideas. By doing this, they could precisely measure both 'accuracy gaps' (incorrectly used grammar) and 'idiomatic gaps' (underused or overused natural expressions). They found specific areas where students struggled with accuracy (like 'the' or '-s' for verbs) and others where they struggled with idiomatic use (e.g., underusing '-ing' forms but overusing 'can'). This allows teachers to pinpoint whether a student's difficulty comes from making mistakes, avoiding complex structures, or over-relying on patterns from their first language, leading to more effective, personalized teaching.