Researchers have developed a smartphone app that captures whole-body movements without the need for special cameras or expensive equipment.
Engineers at Northwestern University in Illinois, US, have developed a system called MobilePoser that transforms a user’s smartphone into a cinematic-quality full-body motion capture system.
Most movie fans are familiar with motion capture technology, which is often revealed in behind-the-scenes footage.
To create a CGI character, like Gollum, lord of the rings Trilogy or Na’vi avatar — Actors walk around a special room in skintight suits covered in sensors. A computer captures sensor data and displays the actor’s movements and subtle facial expressions.
Traditionally, motion capture has been a complex and expensive process, typically requiring actors to wear sensor-laden body suits and perform in studios equipped with specialized high-tech cameras.
“This is the gold standard for motion capture, but it costs more than $100,000 to run that setup,” Northwestern University professor Karan Ahuja, who led the study, said in a statement.
“We wanted to develop an accessible and democratized version that could be used with equipment that basically everyone already has.”
But researchers at Northwestern University say the MobilePoser system takes advantage of sensors already built into consumer mobile devices such as smartphones, smartwatches and wireless earbuds.
MobilePoser uses a combination of sensor data, machine learning, and physics to accurately track a person’s full-body posture and global movement in space in real time.
The app uses customized AI algorithms to capture and transform this data, enabling full-body motion capture without additional cost or equipment. The algorithm estimates joint positions and joint rotations, walking speed and direction, and contact between the user’s feet and the ground.
While the MobilePoser app doesn’t match the accuracy of expensive motion-capture technology used by major Hollywood studios, its creators suggest that tracking errors can be further reduced by using multiple devices.
However, research published this month also highlights that MobilePoser remains reliable enough to achieve high-quality motion capture with just a single device.
“Accuracy improves when you wear multiple devices, such as a smartwatch on your wrist and a smartphone in your pocket,” Ahuja says.
“But the important part of this system is that it’s adaptive. Even if you don’t have a sundial and only have your phone, it will adapt to figure out the pose of your whole body. You can.”
Researchers also believe that MobilePoser’s technology has potential applications in the entertainment field, potentially unlocking new possibilities in immersive gaming and fitness.
Image credits: Header photo licensed via Depositphotos.
