Virtual Reality Music Therapy for Autism Children
TEAM
Solo
TIME
2 Months
My Role
VR Prototyper
TOOLS
Figma
Unity
Cinema 4D
Adobe Premier Pro
AWARD
Best Student Design, RMIT University
Silver Award, HCII Conference, 2022
‘CUCKOO’ is an accessible, practical, clinically relevant, interactive game with a musical Virtual Reality environment to develop autistic children with impaired Emotional Regulation and Executive Functions. The social scenarios and avatars in VR maintain the enriching interactive experience of music therapy.
INSPIRATION
Why Music Therapy & Gamifications
INSPIRATION
So far, Autism is not curable
Complex cause, no precise test
Deficit in Emotional Regulations
Impaired Executive Functions
INSPIRATION
Music Speaks, When Words Fail
Practical & research-supported application of interactive music treatments
Therapeutic connection embedded
Literature Review: Theory Support
Inter-subjective area / effects attunements
Rhythmic auditory stimuli facilitate
Perception motor
Innate ability of music perception in ASD
DISCOVERY
why Virtual Reality
Feasible Cost
High-stimuli of Interaction
Support Social Scenarios
Data of Habitual Movement Can be Clinically Used
Large Body Movements for Exercising Purposes
IDEATION
Game Mechanism Design
Target ASD symptoms To improve
Cognitive Functions
Improved Emotional Regulations
Improved Inhibitory Control
Executive Functions
Improved Set-shifting Skills
Improved Body Motor Skills
Improved Vocal Communication
Game Mechanism Intervention
Interactive NPC characters
Rhythmic elements: drums, etc.
Freeze Games
Set-shifting Games
Task for improve competence
Vocal Intervention
RESEARCH QUESTION
How can the advantages of high-stimuli interactive experience, embedded Executive Function & Cognitive Function intervention in gaming mechanics of music therapy and make it more accessible without causing extra burden for caregivers?
Design Outcomes
Cuckoo aims at improving Executive Function & Emotional Regulations.
Clinical-approved gaming mechanism is studied and designed in music therapy.
Gaming Mechanism
Set-shifting Skills
Executive Function Training: Changing indicators helps practice set-shifting skills.
Gaming Mechanism
Freezing: Inhibitory Control
When the avatar teacher signals "Stop", the user's interaction with the drum is halted.
Gaming Mechanism
Motor Skills Training
Indicator settings initiate body motor skills training. Using music scores as guides, users with motor skill deficits can practice movements derived from physical rehabilitation techniques.
Gaming Mechanism
Motion Sickness Enhancement
This stabiliser smooths the visuals in VR when the user turns around to minimise the resulting motion sickness. It is used to settle down, ‘turn’ ‘snap turn’ of the virtual view.
Experience Cuckoo at different levels.
Well-designed different levels and roles for children to cope with the spectrum of Autism symptoms.
Cuckoo provides an example for the next paradigm of Music Therapy.
Experienced design for a more accessible music therapy for wide audience. With the high engagement of musical games, children can complete a therapeutic section while playing. This video showcases the whole journey of Cuckoo.
DESIGN
Visual Design
3D SPACE
Party Space
3D SPACE
Party Space
DESIGN DELIVERY
Interface Design
Component Design
UI Design
Visual Reality Inteface
GET IN TOUCH
Contact me if you are interested to understand more how this project is developed!
INSPIRATION
Why Music Therapy & Gamifications
INSPIRATION
So far, Autism is not curable
Complex cause, no precise test
Deficit in Emotional Regulations
Impaired Executive Functions
INSPIRATION
Music Speaks, When Words Fail
Practical & research-supported application of interactive music treatments
Therapeutic connection embedded
Literature Review: Theory Support
Inter-subjective area / effects attunements
Rhythmic auditory stimuli facilitate
Perception motor
Innate ability of music perception in ASD
DISCOVERY
why Virtual Reality
Feasible Cost
High-stimuli of Interaction
Support Social Scenarios
Data of Habitual Movement Can be Clinically Used
Large Body Movements for Exercising Purposes
IDEATION
Game Mechanism Design
Target ASD symptoms To improve
Cognitive Functions
Improved Emotional Regulations
Improved Inhibitory Control
Executive Functions
Improved Set-shifting Skills
Improved Body Motor Skills
Improved Vocal Communication
Game Mechanism Intervention
Interactive NPC characters
Rhythmic elements: drums, etc.
Freeze Games
Set-shifting Games
Task for improve competence
Vocal Intervention
RESEARCH QUESTION
How can the advantages of high-stimuli interactive experience, embedded Executive Function & Cognitive Function intervention in gaming mechanics of music therapy and make it more accessible without causing extra burden for caregivers?
Design Outcomes
Cuckoo aims at improving Executive Function & Emotional Regulations.
Clinical-approved gaming mechanism is studied and designed in music therapy.
Gaming Mechanism
Set-shifting Skills
Executive Function Training: Changing indicators helps practice set-shifting skills.
Gaming Mechanism
Freezing: Inhibitory Control
When the avatar teacher signals "Stop", the user's interaction with the drum is halted.
Gaming Mechanism
Motor Skills Training
Indicator settings initiate body motor skills training. Using music scores as guides, users with motor skill deficits can practice movements derived from physical rehabilitation techniques.
Gaming Mechanism
Motion Sickness Enhancement
This stabiliser smooths the visuals in VR when the user turns around to minimise the resulting motion sickness. It is used to settle down, ‘turn’ ‘snap turn’ of the virtual view.
Experience Cuckoo at different levels.
Well-designed different levels and roles for children to cope with the spectrum of Autism symptoms.
Cuckoo provides an example for the next paradigm of Music Therapy.
Experienced design for a more accessible music therapy for wide audience. With the high engagement of musical games, children can complete a therapeutic section while playing. This video showcases the whole journey of Cuckoo.
DESIGN
Visual Design
3D SPACE
Party Space
3D SPACE
Party Space
DESIGN DELIVERY
Interface Design
Component Design
UI Design
Visual Reality Inteface
GET IN TOUCH
Contact me if you are interested to understand more how this project is developed!
INSPIRATION
Why Music Therapy & Gamifications
INSPIRATION
So far, Autism is not curable
Complex cause, no precise test
Deficit in Emotional Regulations
Impaired Executive Functions
INSPIRATION
Music Speaks, When Words Fail
Practical & research-supported application of interactive music treatments
Therapeutic connection embedded
Literature Review: Theory Support
Inter-subjective area / effects attunements
Rhythmic auditory stimuli facilitate
Perception motor
Innate ability of music perception in ASD
DISCOVERY
why Virtual Reality
Feasible Cost
High-stimuli of Interaction
Support Social Scenarios
Data of Habitual Movement Can be Clinically Used
Large Body Movements for Exercising Purposes
IDEATION
Game Mechanism Design
Target ASD symptoms To improve
Cognitive Functions
Improved Emotional Regulations
Improved Inhibitory Control
Executive Functions
Improved Set-shifting Skills
Improved Body Motor Skills
Improved Vocal Communication
Game Mechanism Intervention
Interactive NPC characters
Rhythmic elements: drums, etc.
Freeze Games
Set-shifting Games
Task for improve competence
Vocal Intervention
RESEARCH QUESTION
How can the advantages of high-stimuli interactive experience, embedded Executive Function & Cognitive Function intervention in gaming mechanics of music therapy and make it more accessible without causing extra burden for caregivers?
Design Outcomes
Cuckoo aims at improving Executive Function & Emotional Regulations.
Clinical-approved gaming mechanism is studied and designed in music therapy.
Gaming Mechanism
Set-shifting Skills
Executive Function Training: Changing indicators helps practice set-shifting skills.
Gaming Mechanism
Freezing: Inhibitory Control
When the avatar teacher signals "Stop", the user's interaction with the drum is halted.
Gaming Mechanism
Motor Skills Training
Indicator settings initiate body motor skills training. Using music scores as guides, users with motor skill deficits can practice movements derived from physical rehabilitation techniques.
Gaming Mechanism
Motion Sickness Enhancement
This stabiliser smooths the visuals in VR when the user turns around to minimise the resulting motion sickness. It is used to settle down, ‘turn’ ‘snap turn’ of the virtual view.
Experience Cuckoo at different levels.
Well-designed different levels and roles for children to cope with the spectrum of Autism symptoms.
Cuckoo provides an example for the next paradigm of Music Therapy.
Experienced design for a more accessible music therapy for wide audience. With the high engagement of musical games, children can complete a therapeutic section while playing. This video showcases the whole journey of Cuckoo.
DESIGN
Visual Design
3D SPACE
Party Space
3D SPACE
Party Space
DESIGN DELIVERY
Interface Design
Component Design
UI Design
Visual Reality Inteface
GET IN TOUCH