Learner engagement is a primary driver of the effectiveness of tools and environments for second language learning. In the context of tandem exchange, an important manifestation of engagement is the quality of the personal rapport between students (Bhatti et al, 2020), within a context designed to support the principles of autonomy and reciprocity (Littel & Brammerts 1996). To further explore the influence of this rapport and study the factors that affect it, we seek to measure fluctuations of interpersonal closeness throughout tandem exchange conversations that have taken place in a Spanish-English tandem MOOC (Appel & Pujolà 2021) in which speakers carry out several types of communication tasks. At the same time, recent developments in natural language processing (NLP) have provided powerful new tools for the analysis of language use in the form of Large Language Models (LLMs), a variety of artificial intelligence (AI) based on deep neural networks that have been pre-trained on massive amounts of data, enabling the models to give human-like responses in a variety of general natural language tasks. The discipline of formulating instructions to elicit useful responses from LLMs is known as prompt engineering. In this paper, we present work we have done in engineering prompts for the task of classifying dialog snippets according to the psychological closeness or personal rapport between the speakers. We present promising quantitative comparisons with human-generated classification of the same data and provide a qualitative review of the most interesting results in order to develop a sense of the current and potential usefulness of LLM-based AI for analysis of tandem exchanges.

---

Abstract Summary
We explore the application of large language model (LLM) based artificial intelligence (AI) tools to the analysis of personal rapport between learners in a tandem language exchange environment.

Authors:
Tony Mullen, Northeastern University, United States
Christine Appel, Open University of Catalonia, Spain

About the Presenter(s)
Dr Tony Mullen is a University Associate Professor/Senior Lecturer at Northeastern University in United States

See this presentation on the full schedule – On Demand Schedule

Bookmark