Everything Here All at Once is an immersive escape room experience taking place in iconic sets of the movie Everything Everywhere All at Once. Through unconventional interactive elements, they must figure out how to “verse jump” back to reality, using recreations of iconic symbols of the movie to guide them towards clues.
This is two part project - half conceptual and half technical. Learn more and jump to each part of the project below:
Escape room experience inspired by EEAAO sets, where users search for clues in the environment and try to return back to our current universe by performing odd but calculated “verse jumps”.
A working audiospatial interaction in which visitors uncover clues through responsive ambient sounds. I leveraged audio wavefield synthesis and IR motion tracking to create a one-of-a-kind integrated system.
The source material for this project, Everything Everywhere All at Once, is a indie film where an immigrant mother is thrown into the role of saving her world and family from a threat to her world and all its parallel universes.
Everything Everywhere not only focuses on interdimensional action but also themes of generational conflict, communication, and empathy. A physical experience based on such a film must capture both the intense visual style but also the important messages interspersed between.
In-person experiences based on film and writing are not a new concept: the Stranger Things Experience (based on Stranger Things) and Sleep No More (based on Macbeth) are two in a blossoming trend. All of these experience feature some sort of interactive element that help visitors become immersed, whether it be motion gestures or being able to roam freely around a theatrical set.
In the movie, parallel universes have a unique trait to them - in some universes, people have hot dogs as fingers, and in others, you might happen to be an A-list kung fu movie star. People learned to "verse jump" to different universes by performing odd physical actions, gaining skills from the worlds they travel to and from.
I decided to base an immersive experience off of this mechanic - an escape room where visitors are thrown in a universe they must jump back out of.
Everything Everywhere is a sensory overload in imagery, concept, and content, so the escape room would have to have a similar feel where participants must make sense of a chaotic environment that never truly "quiets down".
Everything Here - the experience - takes easter eggs and iconic symbols from the movie and turns them into tools to help participants make sense of the chaotic environments they are thrown in. Visitors must uncover clues with the help of motion-tracked objects and correctly "verse jump" back to our reality.
Since so much of the movie takes place in the humble laundromat and apartment that the Wang family owns, I chose to base an in-person experience on these sets.
Instead of a 100% faithful replica of the set, it would need to be expanded to accommodate larger groups of people at once - see render on the right.
A larger physical space allows visitors to wander freely without being blocked by other visitors, especially through narrow spaces, cabinets, etc. Additionally, Everything Here could be a modular experience, where more rooms could be added or removed to suit the attention span of the audience.
Sonic scavenger hunt is a responsive clue-hunting interaction driven by spatial audio in the Everything Here All at Once escape room. By moving a motion-tracked object around an area, participants hear localized sound sources grow and fade, eventually discovering key audio clues that could help them escape.
I developed responsive spatial audio to be used in Everything Here All at Once - a system where a motion tracked object causes changes in the sonic landscape for a visitor of the experience.
Click through the slides below to see how the system works.
Immersive experiences are currently visual-forwards, but less thought has been put towards cutting edge sonic interactions. Everything Here All at Once was a chance for me to explore how responsive spatial audio could be used for entertainment.
Movie and home theaters mostly use Dolby Atmos (left), which replicates spatial audio well but confines the listener to a smaller "sweet spot" to experience the audio. In contrast, wavefield synthesis setups (right), consisting of rows of tens of speakers, allow for a much greater area for experience.
Wavefield synthesis is intangibly cool - when I first listened to one, it felt as if sounds could pass into and through my ears in a way that other spatial audio has not been able to achieve. Since audio sources can be placed anywhere in the blue area, you are able to physically move around and experience audio sources coming closer to you.
Everything Everywhere is a multiverse movie at heart, and throughout its run, you can tell the characters are mentally and spiritually experiencing worlds simultaneously - with the way they make sense of the world by focusing on parts of the cacophony.
Wavefield synthesis produces less "clean" sounds than an Atmos spatial audio setup, but the "earworm" moments that happen when one moves to the correct spot mimic the way EEAAO characters focus on verse jumping and defracturing their minds - in other words, a perfect parallel to use immersively.
As a quick test for motion-tracked responsive sound, I mocked up a Unity prototype that took data from the Optitrack camera tracking system, moving a player around a digital twin of the tracked room that had virtual audio sources placed within.
Since the wavefield synthesis speaker array was very difficult to setup and iterate with, this early proto helped me understand most of the experience without taking nearly as much time.
A Unity package was used to transmit motion tracking OSC data outwards from Unity towards Max MSP, the software used to output virtual audio sources. To make localized sounds responsive, I started with a wavefield synthesis Max patch and added distance mechanics in order to fade and gain sound sources based on the location of the viewer.
I won't go into detail here about setting up the wavefield synthesis array, but it took a combination of virtual soundcards, Max settings, and network switches to operate. Unity prototyping was done for a reason!
Escape room experience inspired by EEAAO sets, where users search for clues in the environment and try to return back to our current universe by performing odd but calculated “verse jumps”.
A working audiospatial interaction in which visitors uncover clues through responsive ambient sounds. I leveraged audio wavefield synthesis and IR motion tracking to create a one-of-a-kind integrated system.