Augmented, Virtual, and Mixed Reality Projects

I have explored and led various AR, VR, and MR related projects at the University of Melbourne and Deakin University. Many of these projects are in development or under review now, so I cannot put much details for now. More details to follow later:)

VR Applications as Teaching Platforms

We investigate repurposing existing virtual reality applications as teaching platforms. A pilot study conducted using six freely available social applications on the Oculus Quest platform investigates the practical issues involved in using these applications for distributed learning in a higher education context. Recordings and reports from collaborative sessions with each application yield insights into employing virtual reality as an environment to support teaching.

AR for Intercultural Interactions

In this project, we developed an AR application to assist people from migrant communities and host communities to better understand each others' culture through augmentation of everyday artefacts, bodily movements, etc. We used the lens of immersive performance to investigate how the developed tool and facilitate engagement and learning through doing. Paper currently under review in CHI 2023 Conference.

Santa's Lil Helper

Santa’s Lil Helper (SLH) is a location based, augmented reality (AR) experience. It invited families to weave through Melbourne’s iconic inner city laneways to help track down Santa and save Christmas. Participants use their smartphone and a little Christmas magic to activate AR markers hidden around the city. By finding Santa’s merry crew and triggering 3D and 360 content, the spirit of Christmas takes on a new form across the Melbourne CBD through December.

Augmented Studio

The Augmented Studio platform uses projection mapping to turn a human body into an interactive canvas in real-time. It uses projectors and depth sensors to enable projections of an anatomy model onto a moving body for practical physiotherapy classes, allowing students to observe the dynamic changes in the anatomical configuration of the body throughout a range of movements.

Body Canvas

This projects involves a public demonstration in collaboration with the Melbourne Science Gallery to investigate the use of On Body Projections in public settings. Body as a canvas creates an interaction loop where interaction with information causes changes in the body, which in turn changes the display of information. Our findings show that body as a canvas create connectedness between the body and information.



Publications

We investigate repurposing existing virtual reality applications as teaching platforms. A pilot study conducted using six freely available social applications on the Oculus Quest platform investigates the practical issues involved in using these applications for distributed learning in a higher education context. Recordings and reports from collaborative sessions with each application yield insights into employing virtual reality as an environment to support teaching. First time use involves both adapting to the platform interface, but also ensuring mutual agreement on conventions, control and expressions. Several applications support limited importing and authoring of content. Addressing and locating rooms and participants is a challenge, while the opportunities associated with avatar representation and identity promise enhanced interactions between individuals beyond traditional teacher/student roles.
Facial analysis applications are increasingly being applied to inform decision-making processes. However, as global reports of unfairness emerge, governments, academia and industry have recognized the ethical limitations and societal implications of this technology. Alongside initiatives that aim to formulate ethical frameworks, we believe that the public should be invited to participate in the debate. In this paper, we discuss Biometric Mirror, a case study that explored opinions about the ethics of an emerging technology. The interactive application distinguished demographic and psychometric information from people's facial photos and presented speculative scenarios with potential consequences based on their results. We analyzed the interactions with Biometric Mirror and media reports covering the study. Our findings demonstrate the nature of public opinion about the technology's possibilities, reliability, and privacy implications. We believe that our study indicates an opportunity for case study-based digital ethics research, and we provide practical guidelines for designing future studies.
We present two studies to discuss the design, usability analysis, and educational outcome resulting from our system Augmented Body in physiotherapy classroom. We build on prior user-centric design work that investigates existing teaching methods and discuss opportunities for intervention. We present the design and implementation of a hybrid system for physiotherapy education combining an on-body projection based virtual anatomy supplemented by pen-based tablets to create real-time annotations. We conducted a usability evaluation of this system, comparing with projection only and traditional teaching conditions. Finally, we focus on a comparative study to evaluate learning outcome among students in actual classroom settings. Our studies showed increased usage of visual representation techniques in students’ note taking behavior and statistically significant improvement in some learning aspects. We discuss challenges for designing augmented reality systems for education, including minimizing attention split, addressing text-entry issues, and digital annotations on a moving physical body.
Human body in HCI is often seen as an actuator for issuing commands and providing input to digital systems. We present the concept of the body as a canvas, in which the body acts as both an actuator and a display for information. Body as a canvas creates an interaction loop where interaction with information causes changes in the body, which in turn changes the display of information. Our qualitative study using an on-body projection system in a public exhibition investigates this concept with regards to body characteristics, types of body input, interactions between multiple bodies, and comparison to other display technologies. Our findings show that body as a canvas create connectedness between the body and information. Finally, we discuss how body characteristics and appearances can complement the information, when the body acts as a canvas.
Virtual reality (VR) is now being designed and deployed in diverse sensitive settings, especially for therapeutic purposes. For example, VR experiences are used for diversional therapy in aged care and as therapy for people conditions such as phobias and post-traumatic stress. While these uses of VR offer great promise, they also present significant challenges. Given the novelty of VR, its immersive nature, and its impact on the user’s sense of reality, it can be particularly challenging to engage participants in co-design and predict what might go wrong when implementing these technologies in sensitive settings. This workshop provides a forum for researchers working in this emerging space to share stories about their experiences of designing and evaluating VR applications in settings such as aged care or mental health therapy. The workshop will develop a manifesto for good practice, outlining co-design strategies and ethical issues to consider when designing and deploying VR in sensitive settings.

Our Team

This has been a truely multi disciplinary team!

Image

Hasan Shahid Ferdous


Image

Frank Vetere


Image

Thuong Hoang



Image

Martin Reinoso


Image

Zaher Joukhadar


Image

David Kelly