The integration of Artificial Intelligence (AI) technologies has initiated a new era in language assessment practices, revolutionizing the field with its innovative approaches. This study introduces an advanced Automated Item Generation (AIG) system that utilizes word families as a foundation to automatically generate test items. The primary objective of this research is to investigate the effectiveness of the AIG system in producing high-quality questions through a comprehensive evaluation that combines both quantitative and qualitative measures. The AIG system is developed using cutting-edge machine learning and deep learning techniques, enabling it to generate a substantial number of items, thereby enhancing the language assessment process and facilitating the creation of extensive item banks. To assess the quality of the generated questions, a carefully selected group of 30 experienced English teachers participated in the evaluation process. The participants were provided with a list of items comprising multiple-choice and fill-in-the-blank questions which were automatically generated by the AIG system. Employing a 4-point rating scale that ranged from 1 (acceptable with no revisions required) to 4 (unacceptable), the teachers meticulously evaluated each question. To supplement the quantitative analysis, in-depth interviews were conducted to capture the nuanced perspectives of the teachers concerning the integration of automated item generation in language assessment. The findings of this research demonstrate highly promising outcomes in terms of question quality, validating the efficacy of employing word families as a linguistic basis for generating test items. By shedding light on the advantages and effectiveness of utilizing word families as a fundamental lexical unit for AIG, this study contributes to the field of automated item generation in language assessment. Furthermore, it presents valuable insights for educators and researchers who are seeking innovative approaches to optimize their language assessment practices. This research underscores the transformative potential of word family-based item generation, opening up new horizons for the advancement of language assessment in the era of AI.

---

Abstract Summary
This study presents an innovative approach to language assessment by integrating Artificial Intelligence (AI) technologies. It explains the development and evaluation of an Automated Item Generation (AIG) system that utilizes word families as a foundation for the automatic generation of high-quality test items. Developed using cutting-edge machine learning techniques, the system enhances language assessment and facilitates the creation of extensive item banks. To evaluate item quality, this research employs a comprehensive evaluation strategy, utilizing teacher ratings for quantitative measures and in-depth interviews for qualitative insights. The findings reveal promising outcomes, validating the efficacy of employing word families for generating test items. This study contributes to automated item generation in language assessment by shedding light on the advantages and effectiveness of word family-based AIG. It offers valuable insights for optimizing language assessment practices in the era of AI, benefiting educators and researchers alike.

Authors:
S. Susan Marandi, Alzahra University, Iran
Shaghayegh Hosseini, Alzahra University, Iran

About the Presenter(s)
Ms Shaghayegh Hosseini is a University Postgraduate Student at Alzahra University in Iran

See this presentation on the full schedule – Saturday Schedule

Bookmark