Logo image
Back
AI-Driven Verbal Interaction with Virtual Patients in Mixed Reality for Healthcare Education
Conference paper   Open access

AI-Driven Verbal Interaction with Virtual Patients in Mixed Reality for Healthcare Education

Fiona Naumann, David Clark and Golam Sorwar
36th Australasian Conference on Information Systems (ACIS 2025), 36th (Sunshine Coast, Australia, 01/12/2025–05/12/2025)
04/12/2025
pdf
AI-Driv AI-Driven Verbal Inter erbal Interaction with Vir action with Virtual Patients208.92 kBDownloadView
Open Access CC BY-NC V4.0
url
AI-Driven Verbal Interaction with Virtual Patients in Mixed Reality for Healthcare EducationView
Open CC BY-NC V4.0

Metrics

1 Record Views

Abstract

AI-Driven Virtual Patients Mixed Reality Healthcare Education Verbal Interactivity in Simulation Task-Technology Fit and Flow Theory Continued Use Intention in MR Learning
Effective communication is essential to safe, patient-centred healthcare, yet many simulation-based education tools offer limited opportunities for students to practise verbal interaction with patients. This study addresses that gap by developing a bespoke mixed reality (MR) simulation that enables real-time, AI-driven verbal communication with a virtual patient. Designed specifically for this research and deployed on the Meta Quest 3 platform, the simulation supports the development of communication skills by allowing students to speak directly with a virtual patient and receive dynamic responses powered by large language models and natural language processing. The study introduces a novel theoretical model that integrates Task-Technology Fit and Flow Theory to examine how verbal interaction influences user satisfaction, perceived task alignment and continued use intention. A mixedmethod, quasi-experimental design compares healthcare student experiences across two MR conditions: one featuring a non-verbal patient and the other incorporating AI-driven verbal interactivity. Participants will complete structured surveys adapted from validated instruments to assess fluency, absorption, enjoyment, task fit and satisfaction. Qualitative data from focus group discussions will support interpreting learner experience and system usability. This research contributes to designing scalable, interactive MR simulations for healthcare education and offers new insights into how AI-driven communication features affect learner experience and technology adoption. It also provides a foundation for further information systems research into aligning conversational AI with structured learning tasks and immersive environments.

Details

Logo image