Correcting or Rewriting? An Expert Evaluation of LLM-Based GEC on Academic Learner Data

Ch. 26. P. 1–10.

Kopylova E.V., Tsegoeva O. G., Berlin V.A., Vyrenkova A.S., Kuvshinskaya Y. M.

This paper investigates how large language models correct complex grammatical errors in Russian academic
learner writing. Unlike traditional minimal-edit GEC systems, LLMs often apply generative rewriting strategies that
may improve fluency, but risk structural overcorrection and semantic drift. We introduce a new expert benchmark
derived from an authentic 3,1M-word learner corpus and construct an evaluation set annotated for error type and
complexity.
We propose an expert-driven evaluation framework combining quantitative scoring, structural-change analysis,
and blind pairwise comparison. Results reveal a consistent minimal-edit vs. generative trade-off across LLMs. This
trade-off has direct implications for evaluation, as purely reference-based metrics may underrepresent structural
overcorrection and fail to capture differences in correction strategies.

Keywords: система оценки, учебный корпус, Learner Corpus(LC), evaluation framework, GEC, LLMs, LLM, Russian academic writing, GEC, русское академическое письмо

Publication based on the results of: Linguistic and cognitive diversity in formal models, computer tools, and educational resources (2025)

In book

Компьютерная лингвистика и интеллектуальные технологии: По материалам ежегодной международной конференции «Диалог». Выпуск 24

Issue 24. , M.: Max press, 2026.