A team calling itself Akiya Research Institute VR Lab wants to turn phones and webcams into a multi-view body tracking system.
We found no web presence for any “Akiya Research Institute” beyond the Twitter account started this month. We translated the Tweets – the videos have already gone somewhat viral. We’ve not been able to find more information about the team and their direct messages are not open, so we encourage those involved to reach out to tips@uploadvr.com so we can dive deeper into the work and team.
No VR system on the market today comes with body tracking, but some VR enthusiasts use HTC’s Vive Trackers or Microsoft’s Kinect. Social platform VRChat is the most popular use case, as well as mixed reality streaming with an avatar for LIV.
HTC Vive Trackers cost $99-$129 each. Typically three are used for body tracking. If you don’t already have SteamVR Base Stations they’re $149 each, so the total cost approaches $600. Microsoft’s Kinect is no longer sold, but available used for between $50 and $200. It works, but only from a front-facing angle and the tracking quality isn’t always great.
MocapForAll
Akiya says it’s building software for PCs called MocapForAll. The idea is to fuse the perspectives of multiple cameras you already own, like phones and webcams, using deep learning algorithms to track the parts of your body.
You’ll need to use at least two camera sources, but the system can support more for a better result.
It sounds like the tool is meant for regular motion capture, but can also be used for real time body tracking in SteamVR, as demonstrated here:
A Tweet posted today claims the software will be priced around $100, with a time-limited free trial available.
So just how practical could MocapForAll be for VR? A video demonstration showing system resource usage in Windows seems to show it sending an Intel i3-8350K from 80% utilization to 100%. Ayika says this is a debug test in the Unreal Engine editor, not representative of final performance. It mentions “precision-focused and speed-focused modes, with frame rates of around 30fps to 100fps”, suggesting users can choose the balance between performance and quality.
It’s too early yet to draw any conclusions about the software, but we’ll be keeping a close eye on any future demonstrations & announcements.