Application of Artificial Intelligence powered digital writing assistant in higher education: randomized controlled trial

Abstract

A major challenge in educational technology integration is to engage students with different affective characteristics. Also, how technology shapes attitude and learning behavior is still lacking. Findings from educational psychology and learning sciences have gained less traction in research. The present study was conducted to examine the efficacy of a group format of an Artificial Intelligence (AI) powered writing tool for English second postgraduate students in the English academic writing context. In the present study, (N = 120) students were randomly allocated to either the equipped AI (<i>n</i> = 60) or non-equipped AI (NEAI). The results of the parametric test of analyzing of covariance revealed that at post-intervention, students who participated in the AI intervention group demonstrated statistically significant improvement in the scores, of the behavioral engagement (Cohen's <i>d</i> = .75, 95% CI [0.38, 1.12]), of the emotional engagement Cohen's <i>d</i> = .82, 95% CI [0.45, 1.25], of the cognitive engagement, Cohen's <i>d</i> = .39,95% CI [0.04, .76], of the self-efficacy for writing, Cohen's <i>d</i> = .54, 95% CI [0.18, 0.91], of the positive emotions Cohen's <i>d</i> = . 44, 95% CI [0.08, 0.80], and of the negative emotions, Cohen's <i>d</i> = -.98, 95% CI [-1.36, -0.60], compared with NEAI. The results suggest that AI-powered writing tools could be an efficient tool to promote learning behavior and attitudinal technology acceptance through formative feedback and assessment for non-native postgraduate students in English academic writing.