Effectiveness of AI-Based Mobile-Assisted Language Learning (MALL) Using SpeakingPal to Enhance English Speaking Skills of ESL Learners: A Quasi-Experimental Study

Abstract

This study examined the effectiveness of the AI-based SpeakingPal mobile application in improving English speaking proficiency among first-year undergraduate ESL learners. It was delimited to 60 female A2-B1 learners at one public-sector girls' degree college in Hyderabad, Sindh, Pakistan, and focused on fluency, pronunciation accuracy, communicative competence, and coherence and cohesion during an eight-week intervention. AI-supported Mobile-Assisted Language Learning offers repeated speaking practice, immediate feedback, and opportunities for autonomous learning beyond the classroom. However, controlled evidence concerning AI-based speaking applications remains limited in under-resourced South Asian higher-education contexts. This study addressed that gap by evaluating SpeakingPal under authentic classroom conditions. The study employed a quantitatively dominant mixed-method, quasi-experimental pre-test/post-test control-group design. Using purposive sampling, 60 first-year undergraduate ESL learners were assigned through two intact classes to an experimental group (n = 30) and a control group (n = 30). The experimental group used SpeakingPal for four approximately 30-minute sessions per week for eight weeks, whereas the control group received conventional speaking instruction for an equivalent period. Data were collected through a validated 20-point speaking rubric, a 20-item learner-engagement questionnaire, application-use records, and semi-structured interviews with 12 experimental-group participants. Quantitative data were analysed using descriptive statistics, paired- and independent-samples t-tests, ANCOVA, and Cohen's d, while interview data were analysed thematically. Both groups improved, but the experimental group achieved substantially greater gains across all four speaking dimensions. Its mean post-test speaking score (M = 17.43, SD = 1.52) exceeded that of the control group (M = 13.27, SD = 1.84), t(58) = 9.71, p < .001, d = 2.51. The group effect remained significant after controlling for pre-test performance, F(1, 57) = 94.26, p < .001, partial eta squared = .62. Learners reported high engagement and perceived usefulness (M = 4.32/5.00), while interviews highlighted immediate feedback, reduced speaking anxiety, and greater autonomy, alongside connectivity and speech-recognition challenges. SpeakingPal should be integrated as a structured supplement to teacher-led ESL speaking instruction rather than as a replacement for classroom teaching. Institutions should support reliable internet access and learner access to compatible devices, while developers should improve speech-recognition accuracy for regional accents and low-bandwidth settings. Future studies should use larger and more diverse samples, longer interventions, longitudinal follow-up, and detailed usage analytics