Proofs of Concept

1. Story World: Jack and the Beanstalk
2. Music World: Whip it

Story World: Jack and the Beanstalk

An interactive 3D audio version of the classic fairy tale where the guest must take certain actions to proceed throught the story.

 

Music World: Whip it

An interactive experience where the guest puts a song back together and can also interact with the different instruments within the song.

 

Prototypes

1. Manipulating sound with an input device
2. Navigation in an audio environment
3. 3D sound
4. Audio UI
5. Combine two platforms (Wii + 3D tracker)

Manipulating sound with an input device

We used a 3D tracker attached to the guest's hand to allow the guest to reach out and hold a sound and then move it to a new position in the virtual 3D space.

Navigation in an audio environment

We used a 3D tracker attached to the guest's head to allow the guest to walk around and rotate their head to listen to different aspects of a virtual 3D environment.

3D sound

We research different DSP plug-ins, such as convolution reverbs and spatialization effects, and determined that they could be integrated into our base code.

Audio UI

We integrated vocie recognition capability into our base code.

Combine two platforms (Wii + 3D trackers)

we used 3D trackers attached to the guest's head and hand and a Wii-mote to allow the guest to move around in 3D virtual environment and interact with sounds with the Wii-mote which gave rumble feedback to the guest when they performed certain actions such as coming in contact with a sound.