[Maquette] Scene 3: Reckoning: A Spoken Word x Live Motion Capture Performance

About

Maquette is a live interactive performance leveraging motion capture technology, avatars, and dance in an exploration of the parallel histories of averaging and idealism in art and society, written and directed by Lisa Jamhoury.

Reckoning. the third scene of Maquette, is an 8 minute spoken word and motion captured piece performed solo at New Inc's Demo Day 2023 at ONX Studio in New York. It's a moment of introspection given by movement and speech, visually exploring through the sub conscious space of thoughts on deconstructing the digital self and a reckoning with our physicalities.

I was the main Unreal Engine developer for this scene of Maquette, and it was such a fun experience to work together with Lisa and her team! Our collaboration was cross dimensional, moving fluidly from the physical and digital. We first met eachother at ONX Studio in New York, where Lisa had arranged a three day Maquette hackathon. We had the mornings for tech time, speed prototyping some visual effects and avatar manipulations in Unreal Engine. And afternoon, the four dancers were with us to get in the motion capture suit and play in the scenes! I really enjoyed these days, though they were intense and we didn't see the sun, the duration was perfect to get into the zone. We prototyped some really fun things with the four dancers, blending their motions together, or turning them abstractly into cubes.

Lisa and the four dancers, the blend space between their bodies behind
Behind my computer at ONX
The ritual of an Optitrack calibration, wave the wand over every pixel of 12 camera space and bless.

After this hackathon, we spent the next month working together remotely over Parsec, Zoom, and two more physical rehearsals at ONX. Building out some of the concepts that we enjoyed when we were together, focusing on the moments where we both would look at the projection and go "wow" in unison. Lisa was really pushing for non linear mapping of movement from the performer to the avatar, to match the deconstruction in the text. We came up with some themes and visual concepts in Unreal Engine together -- my favorite being 100 bodies forming a weirdly organic structure by positioning them evenly on a sphere ( where I learned about the Fibonacci Sphere ) , and breaking up the avatar in Houdini and applying a hover effect to each limb in Unreal.

100's of bodies rotated around a sphere, weirdly organic structure
Close up, spoken words are about coming back into our physicality
Wireframe limbs from above, this was one of our "woooow" unison moments that we followed through with
Wireframe limb hover
Wireframe limb disintegration

After reaching an interesting point with the visual themes, we laid out everything onto sequencers and matched the cameras and visuals to a spoken word track in the Unreal Engine sequencer. We broke it out into three sequences that were triggerable by key press, so that Lisa could easily sync the visuals with her speech.

I could not attend the actual show, but kif was in New York and I sent him as a codercat representative to film and enjoy the experience physically! From what it seems from the footage, the visuals, movement, and speech came together in a really interesting way. I loved the stage setup as well, creating an intimate space with a single spotlight and bean bags.

I'm very looking forward to continuing this collaboration, as there will be a full 45 minute showing of Maquette in Nov 2023, including all seven scenes and four dancers. Lisa has mentioned some interesting visuals and concepts for the scenes that still need developing, and I can't wait to explore these with her soon!

Credits

Lisa Jamhoury - Director, Spoken Word

Sneha Belkhale - Unreal Engine Developer

Fran├žoise Voranger - Choreography, Movement, Performance

ONX Studio, NYC - Venue