Abstract

Automatic Short Answer Grading (ASAG) is a well-known research task in the field of natural language processing (NLP). Its major purpose is to automatically grade descriptive answers of the students by keeping automatic grading consistent with the evaluation of human graders. Recent developments in Large Language Models (LLMs) have demonstrated a greatly enhanced performance in automated grading; however, the generalizability of the models and accuracy is still quite low because of the absence of dataset-specific grounding. We present EDURAG, a Retrieval-Augmented Generation (RAG) based model to improve contextualization of the LLM-based grading with exemplar-based grading and extra knowledge as generated by QFKE (Question Focused Knowledge Extraction) module. The proposed QFKE module provides extra layer of contextuality for the EDURAG. In contrast to conventional supervised methods, EDURAG does not need model fine-tuning. The suggested framework is tested against the ASAG2024 benchmark that consolidates seven short-answer grading datasets across various domains, educational levels, and grading scales. The benchmark protocol of measuring performance is weighted Root Mean Square Error (wRMSE). The experimental findings show that dual contextuality provided by EDURAG enhances the accuracy of grading significantly when compared to vanilla LLM grading. The ablation study also confirms the significance of dual contextuality provided by EDURAG. Although promising results (almost 15% improvement) have been achieved during experiments, there is still a discrepancy between human grading performance and merit, indicating the potential of hybrid human-AI grading systems. The results indicate that retrieval-enhanced LLMs offer a scalable and generalizable direction for automated assessment.

Statistics