Comparative analysis of large language models as AI assistants for educational Assessment: A case study in steam education

Abstract

This case study explores the effectiveness of various Large Language Models in the domain of educational assessment. In the context of STEAM education, a complex learning scenario was employed to evaluate the AI models. The evaluation focused on their ability to understand the scenario, propose diverse and applicable assessment methods, and evaluate key aspects such as innovation, creativity, and collaboration among students. The results indicate considerable variability in model performance. Notably, certain models demonstrated a strong capacity to adopt holistic approaches and align with learning objectives, whereas most models struggled with differentiating assessments effectively. The findings underscore the potential of Large Language Models as AI assistants in educational assessment, highlighting the importance of careful and supervised application.