Using Motion Capture AI to Up-level Strategic Storytelling

As consultants, we use our collaborative Discovery as a Service process to solve complex business problems and unify company messaging. As an agency, we then design the most compelling stories for our clients. No creative technology is off the table, especially since the barriers to utilizing those technologies are vanishing quickly.

Consider motion capture, which essentially refers to the act of capturing someone’s motion. The questions to consider when using this tech: How is that motion being captured, and how is that motion being applied?

When mocap first became mainstream, it was done in large rooms using a series of cameras wrapped 360 degrees around an entire space, pointed to the center of the room. Actors would wear black spandex suits covered in tiny spheres that the cameras would track to approximate the motion of those actors in true 3D space. In fact, 1P did a project with this very technology:

Once this data is collected, it is transferred to digital characters with the goal of reducing artists' animation labor time. While effective, it was only truly cost efficient within large studio pipelines.

Nowadays, motion capture is much different. When it comes to mocap suits, we no longer need large studios or spandex sphere-laden outfits to capture motion. Suits now place sensors directly over actors’ bending joints (knees, elbows, etc.) and capture positional and rotational data in true 3D space.

Traditionally for character animation, artists will record video of themselves acting out scenes in front of a camera, and then use the recordings as reference to create their character animation. With these new systems being so portable, it allows artists the ability to not only record themselves, but to bring motion-capture data almost immediately into a 3D viewport, transferred to their character models. These systems are more cost-efficient than their bulky predecessors, but are still priced out of reach of an everyday consumer. Some of the leaders in this space are Xsens, Rokoko, and Perceptual Neuron.

Character creation is very different than traditional animation setup. You have to create controls for fingers and joints and everything. It can take five weeks jut to animate a hand. On a recent project, 1P saved time and got amazing results by using Live Face, which uses LiDAR (Light Detection and Ranging) from the iPad, allowing an actor to control the character with their own face, with real-time manipulation that is less rigid and more naturalistic than other systems. Here's how it worked and the final result:

We talk about AI a lot in 1P Labs, and as it is elsewhere, this technology is also starting to transform the motion-capture space. You used to need cameras in 360 degrees to capture every angle of someone's body, but now with a single camera, AI can approximate the movement and position of joints that are obstructed or not visible within that camera’s view. And no special suits or equipment are needed—which will make it the most accessible form of mocap.

While AI motion capture is at the point of this writing is not yet fully production ready, it could ultimately present a huge shift in how some animated stories are approached. In theory, using this technology, AI could convert a stock footage or live-action single camera edit into motion capture data applied to animated characters. Some of the leaders in this space are Move.ai, Kinetix, RADiCAL, Plask, DeepMotion, and FrankMocap.

At 1P, we are keeping an eye on this technology and are excited to create new workflows and design innovative stories that will help differentiate our clients.

Our enterprise-level Discovery as a Service method enables us to create multiple customer journeys and experiences, united under one strategic narrative, and we want to make sure nothing is off the table when it comes to realizing those stories in the most effective way possible.