About
Kiki Yago Live, a realtime face capture and musical performance that was exhibited in 5 different venues in Tbilisi. We explored live stage visuals in Unreal Engine 5, bringing a metahuman to life with the facial expressions of the live performingXenofontova Dasha.
Over the previous 6 months, Dasha and Codercat had been developing the Kiki Yago project. Kiki Yago is a poetess, who creates music and casts high speed spells to actively use the multidimensionality of our mind. She presents the feeling of being uploaded to the Metaverse and recieving every fragment of information in just a few milliseconds.
We expanded on her story and desires by creating music videos and meditation clips. Dasha would record herself singing her tracks rapidly using a Face Capture application on her iPhone, and send the motion data via Telegram for us to then transfer to the metahuman Kiki Yago. If you are curious about the details, check out this blog post I wrote about our first music video here!
After 6 months of collaborating remotely, we were overwhelmed with excitement when Dasha said she would come stay with us in Tbilisi Georgia, for a 3 month cyber camp! The idea was to spend 3 months living, creating, playing, and working together -- with a focus on developing the first live concert/performance of Kiki Yago!
We started planning for the show by creating a track list -- Dasha arranged the songs starting with the slow, vitalizing, birthing moment "Mother Taiga", transitioning into the becoming of self with "Ya Kiki", then into the rapid lyrical madness of "Siroiha" and "Koni", and ending with dance beats like "Utro" to make the crowd get up and (gabber).
Then Codercat matched this progression with a series of visual concepts from previous music videos and new works that could run in a realtime Unreal Engine scene at atleast 30FPS (this quickly eliminated our forest scene from "STR3AM").
After we came up with a scene list, we started to build a live VJ setup in Unreal Engine. We duplicated the chosen scenes from pre-existing videos and refactored them into scenes that could be used live. We built a few controls to VJ live during the show, such as switching scenes, changing camera angles, lighting scenarios, IK Controls, and shader effects.
At the end of this first iteration, we set up our first rehearsal in a local hackerspace,
F0RTHSP4CE just a few minutes from the house. The hackerspace had an awesome basement with sound equipment and a empty white wall for our projection visuals.
This was our moment to rehearse the face capture in a dark club environment. We set up a selfie tripod ( which we somehow manifested on the streets of Tbilisi ), and attached some lights so that Dasha's face would stay lit and trackable. Originally we thought to have Dasha and Kiki both center stage, but weird recursive things happened when the face capture application would see the projection of Kiki and detect it as a face.. :)
Our amazing musician friend and Kiki Yago Collaborator, She Monkey, joined the Team as a DJ / sound mixer -- and we ran through the whole set! With some iterations on placement, lighting, and network issues, the live face capture seemed to be working pretty smoothly and synced to Dasha's vocals! And of course, hearing Dasha sing these songs that we had been working with, live, was really exciting and energizing.
After two weeks of rehearsals and building on our VJ setup, we were ready to hit the venues of Tbilisi:) We played 5 shows, some under bridges, some in basements, and some in strange party houses far away from the city. We cheers with some cha-cha, and brought our energy to each show. Best team ever, and we can't wait to reunite :)
