Exploring AI Face Tracking: A Creative Experiment
1. Introduction
At Peters-Fox, we thrive on experimentation, blending creativity and technology to push boundaries. Recently, we turned our focus to AI face tracking—a tool that maps facial expressions and movements with remarkable precision.
We were fortunate to gain early access to this innovative software, which initially sparked our curiosity. Could AI accurately track a video of Katrina speaking and map her expressions onto a still image of a 3D character? These 3D characters, (also generated using AI), became the canvas for our experiment.
One of many characters created.
Our findings were promising: the software delivered highly accurate results, showcasing its potential to transform animation. However, the first release revealed some limitations—it required a human-like face to track successfully, making abstract or animal faces more challenging. We anticipate this capability improving in future iterations.
The implications for future projects are exciting. This technology could revolutionise the creation of brand characters, enabling businesses to bring them to life quickly and cost-effectively. Traditional face-rigging animation, which often demands significant time and budgets, could become a thing of the past.
‘ These same AI-driven characters could potentially engage in interactive tasks, operating in personalised journeys or even acting as guides to relevant knowledge bases for people and employees in their jobs’.
2. The Experiment
Using this software, we aimed to bring more life to our 3D animations by incorporating realistic facial expressions. The AI tracked even the subtlest movements, creating a seamless connection between human input and digital output.
The results were surprisingly accurate. For example, when Katrina's voice became particularly expressive, the AI occasionally attempted to mimic arm movements—showing an unexpected level of initiative! There were a few quirks, like blinking eyeballs instead of eyelids and limited lower-body movement, but these are details we expect to see refined in future releases.
Interestingly, the software could track animal faces in some cases but struggled to locate a “human face” when tested on abstract imagery. This highlights areas for improvement and opportunities for future development, which we’re excited to explore further.
Check out the experiment in action below:
3. Implications for the Industry
This experiment is just a glimpse of what AI face tracking could mean for industries like animation, gaming, and virtual reality. Imagine characters that respond to human emotions in real-time or digital influencers connecting with audiences in deeply authentic ways.
Currently, several live face-tracking (or deepfake) tools are already making waves. Although this software isn’t live yet, it could evolve in that direction. The potential for AI-powered avatars to sing, entertain, inform, or even participate in discussions is fascinating.
Consider a scenario where a remote work colleague is an AI agent. You see them on calls, interacting and contributing, much like a human counterpart. While it may seem futuristic, the face tracking element could provide that essential human—or non-human—visual connection, shaping how we collaborate in virtual environments.
This technology opens possibilities not just for entertainment but also for practical applications like interactive learning, virtual customer service, and dynamic content creation.
4. Closing Thoughts
At Peters-Fox, we’re excited to continue exploring how AI can reshape creativity and technology. What do you think about the potential of AI face tracking? Could it revolutionize industries, improve workflows, or enhance human connection?
We’d love to hear your thoughts or ideas for future experiments—let’s push the boundaries together.
