The use of artificial intelligence (AI) in English as a Foreign Language (EFL) learning context has become increasingly popular in recent years. Assessing EFL learners’ English public speaking (EPS) performance in real classroom settings is a challenging issue for frontline teachers, educational researchers, and technological experts. During the public speaking process, multiple modalities of information, such as acoustic, visual, and textual data, are generated, which makes the multimodal evaluation rather complicated.
This study reported the development of a self-designed AI-supported formative assessment platform for evaluating EFL learners’ public speaking performance. Based on a quasi-experimental design, the study further explored the impact of peer and automated feedback on students’ EPS self-efficacy and performance. Fifty-two Chinese college students were randomly assigned into two different groups. The experimental group carried out self-, automated- and teacher- assessment, while the control group undertook self-, peer- and teacher assessment.
The research employed multi-source data to explore and explain the effects of two different modes of formative feedback. Quantitative data included the grades of students’ EPS performance in pre- and post-tests as well as students’ questionnaire responses. An EPS questionnaire (12 items, α = .86) was used as an instrument. Analysis of Covariance (ANCOVA) was conducted to examine the effect of assessment modes on participants’ EPS self-efficacy and performance. Qualitative data included learners’ feedback in semi-structured interviews. Content analysis was used to explore learners’ perceptions of two modes of different formative assessment.
Our results showed that the two modes of formative feedback had positive effect on improving learners’ EPS self-efficacy and performance. However, no significant difference was found in terms of EPS self-efficacy and performance among the two groups. The AI-supported automated feedback seemed to achieve similar effects on assisting the formative assessment of learners’ EPS performance. Qualitative results found that learners showed different concerns and affordances towards the platform and the assessment modes. More nuanced comparisons were carried out in terms of the differences in the effects of the intervention. We then provided related pedagogical implications for educators and technical experts for improving the assessment and instruction of public speaking with AI technologies.

---

Abstract Summary
Assessing EFL learners’ English public speaking (EPS) performance in real classroom settings is a challenging issue. This study reported the development of a self-designed AI-supported formative assessment platform for evaluating EFL learners’ public speaking performance. Based on a quasi-experimental design, the study further explored the impact of peer and automated feedback on students’ EPS self-efficacy and performance. Fifty-two Chinese college students were randomly assigned into two different groups. The experimental group carried out self-, automated- and teacher- assessment, while the control group undertook self-, peer- and teacher assessment.
Our results showed that the two modes of formative feedback had positive effect on improving learners’ EPS self-efficacy and performance. However, no significant difference was found in terms of EPS self-efficacy and performance among the two groups. Qualitative results found that learners showed different concerns and affordances towards the platform and the assessment modes. Related pedagogical implications were provided.

Authors:
Xu Chen, Beijing University of Posts and Telecommunications, China
Tingting Zhang, Beijing University of Posts and Telecommunications, China
Huayang Zhang, Beijing Normal University, China
Chunping Zheng, Beijing University of Posts and Telecommunications, China

About the Presenter(s)
Mr Xu Chen is a University Postgraduate Student at Beijing University of Posts and Telecommunications in China

See this presentation on the full schedule – Sunday Schedule

Bookmark