Introduction

The application, prototype title “Cogni-gym VR”, is an virtual reality application that tests users logical problem solving by sorting orbs into boxes while on the time limit.

Description of the application

The application is designed to operate with VR headsets such as the Oculus Quest 2. When the application begins the user will find themselves in a room in a skyscraper with 6 boxes surrounding them. The box in front of the player has the instructions to the exercise written out of its side and an orb which spawns on top of it. These orbs have 3 properties:

• A colour (blue, red or yellow)
• A number (0,1,2,3)
• A number colour (blue, red or yellow)

The left and right of the user are 4 boxes labelled 1,2,3 and 4. The user must toss the orbs into the boxes. When the training begins the application will pick one of 4 rule sets. These rule sets determine what type of orb each box will accept. When a orb is thrown into a box the user will either:

• Hear a ding and see green flashing lights. Indicating a correct guess
• Hear a buzzer and see red flashing lights. Indicating an Incorrect guess

Using this feedback the user must rely on logical reasoning to figure out which box each orb should be sorted into. When an orb is thrown into a box, or if it strays out of the reach of the user it is destroyed and a new orb is spawned on the front box. Directly behind the user is the 6th box and on the wall beyond the box are 2 screens with timers. The top timer indicates how long the user has to sort the orbs before the training ends. The lower timer indicates how long before a button will appear on the 6th box which the user can poke with their right index finger to add 30 seconds to the time remaining timer. Doing so will hide the button and reset the cooldown button. This forces the user to frequently break their focus on the task of sorting the orbs and keep track of the 2 timers.

The application keeps track of how many times a user has answered correctly vs incorrectly. 

This information is displayed to the user when time has run out and gives them the option to exit the prototype. Additional features which could be added to this application given sufficient finances and time include:

• Audio and visual distractions to provide cognitive and sensory overload to add further difficulty
• Menu to select difficulty of training
• Additional rulesets
• Additional statistics and tracking so users can see how their cognitive fitness has improved
• Fitness tests in larger locations that could require the user to move via teleportation
• Additional logical tests, particularly ones that take advantage of the VR system (being able to looks and interact with objects in a 3d virtual space)
• High score system to compare results with friends
• A white board the players can interact with and record their findings and deductions to 

This application is designed to aid in the training and measuring cognitive fitness.  (Grosprêtre et al., 2023) argues that “Cognitive-motor multitasking training as provided in a VR environment is thought to be suitable to induce changes in the processing capacity of the brain, for example, synaptic changes and the facilitation of cognitive processing.” By testing the user in a virtual environment we can force the user to multitask by taking advantage of the additional information throughput and spatial interactions of VR. For example, problem solving becomes harder when the user is required to perceive, interact with and decipher information in a 3D space, combined with audio and textual information. VR is also known to consistently stimulate the autonomic nervous system, as evidenced by increases in heart rate and skin resistance, which are two physiological markers of stress and anxiety (Fadeev et al., 2020). Although most IT projects aim to reduce stress, anxiety and mental fatigue. For the purposes of testing a user's cognitive fitness these are positive attributes of the VR medium.

Interaction Design

This application contains the following interaction designs. Users need to physically turn their head to explore the environment and gain information from their environment (E.G. Looking directly behind to interact with the timers). This is done to temporarily distract the user from the primary focus of sorting the orbs. Users point and grab the orbs and physically throw them towards the correct box. User “pushes” the button to add additional time and to exit the application when the time runs out. While not a VR specific interaction the environment provides feedback to the user when an orb is placed in a box. This is intentionally ambiguous feedback so as to properly tax the users cognitive faculties. 

Storyboard

Technical Development

The application is compatible with Oculus Quest 2 VR headset and controllers. The Oculus OVRPlayer controller and other scripts and objects are used to handle the VR camera and interactions with the other game objects. The environment is constructed from unity cubes, free assets and textures found online to make the immediate area the aesthetic of a room that may be found in a gym in a city. Point at an orb and pull the trigger to grab an orb. Make a throwing action and release the trigger to throw the orb into one of the numbered boxes. The boxes have no collision and are there for aesthetics and to hide a collider when the orb collides with a collider it will check if it collided with one of the boxes or one of the out of bounds colliders. If it is in a box then check against the current rule set to see if the orb belongs in that box. If the guess was correct then play the correct sound effect, 1 is added to the user’s correct tally and flash green light. If the guess was incorrect then play the incorrect sound effect, 1 is added to the user’s incorrect tally and flash red light Regardless of whether the guess was correct, incorrect or just out of bounds a new orb is spawned on top of the front box. Orb spawning works by randomly selecting the colour, number and number colour and then spawning an orb with that specific material. In total there are 36 possible orb combinations and there is a material stored in an array on the game manager for each combination.The screens behind the player are models to make the Unity Text mesh pro object fit the theme of the room. The button to add more time only appears when the countdown timer is at 0. Poking the add more time button triggers an event which will add 30 seconds to the time remaining timer and 1 minute to the cooldown timer. The same interaction is done when the user exits the application after the results menu is displayed. Movement using the Oculus joysticks was disabled for this application  to minimise motion sickness and because all relevant objects were designed to be within reach of the player.

Descriptions of 3D Models

• Orbs are a unity sphere with a custom made material and code for tracking their attributes and being able to be grabbed in VR added to them
• Boxes come in 2 varieties
  • Open boxes which players can throw the orb in so it hits a hidden collider inside
  • Closed boxes. Have physics and are used as platforms for the spawned orbs and the add more time button
• 3 buildings, a road and a sidewalk prefab are used to decorate the window and immerse the user in the environment
• Ceiling lamps serve no purpose beyond explaining why the room is lit and adding scenery if the user should look directly up
• 3d model of TV are used for aesthetics and support the timers
• Add more time button is a default unity sphere with text and a script to trigger the add more time event. 
• The door model in the back corner of the room exists for aesthetics and explains how the user got into the room in the virtual world. 

References

https://chat.openai.com/share/8202daa6-e1c4-4a4b-8e7c-db4a62dfbdde

ChatGPT was used for the following:

• Troubleshooting the object Grabbable script
• Writing constructor for Orb class (later rewritten)
• Adding text to a prefab
• get render in child component (later removed)
• on collision event with orb trigger event
• Troubleshooting collision events
• Writing code for timers
• Creating a singleton class (later removed)
• Name ideas for project

GROSPRÊTRE, S. et al. How Exergaming with Virtual Reality Enhances Specific Cognitive and Visuo‐Motor Abilities: An Explorative Study. Cognitive Science, [s. l.], v. 47, n. 4, p. 1–16, 2023. DOI 10.1111/cogs.13278. Disponível em: https://research-ebsco-com.ezproxy.utas.edu.au/linkprocessor/plink?id=19faf603-62b8-356e-ab6c-e06fce84f7da. Acesso em: 24 set. 2023.

FADEEV, K. A. et al. Too Real to Be Virtual: Autonomic and EEG Responses to Extreme Stress Scenarios in Virtual Reality. Behavioural neurology, [s. l.], v. 2020, p. 5758038, 2020. DOI 10.1155/2020/5758038. Disponível em: https://research-ebsco-com.ezproxy.utas.edu.au/linkprocessor/plink?id=e0971473-7aaf-3381-afe9-633c4fd4d2f9. Acesso em: 24 set. 2023.

https://assetstore.unity.com/packages/3d/environments/urban/city-package-107224

https://www.turbosquid.com/3d-models/ceiling-lights-tyson-3d-1556021

https://www.turbosquid.com/3d-models/tv-samsung-f6100-32-max-free/762304

https://assetstore.unity.com/packages/3d/props/interior/door-free-pack-aferar-148411

https://freepbr.com/materials/sprayed-wall-texture1/

https://stock.adobe.com/au/images/black-rubber-tile-for-flooring-texture-and-seamless-background-rubber-tile-floor-pattern-and-background/373642503

https://assetstore.unity.com/packages/3d/environments/medieval-containers-pack-with-wagon-66260

Myinstants (n.d.). Buzzer - Sound Button. [online] myinstants. Available at: https://www.myinstants.com/en/instant/buzzer-89244/ [Accessed 18 Aug. 2023].

Myinstants (n.d.). correct (ding) - Sound Button. [online] myinstants. Available at: https://www.myinstants.com/en/instant/correct-ding/.

www.dafont.com. (n.d.). JMH Typewriter Font | dafont.com. [online] Available at: https://www.dafont.com/jmh-typewriter.font.

www.dafont.com. (n.d.). Digital 7 Font | dafont.com. [online] Available at: https://www.dafont.com/digital-7.font