Face Capture get started

Key takeaways
How to Live Stream Face Capture data to Unreal
How to Live Stream Face Capture data to Unreal
How to use Face Capture Data in Maya
How to use Face Capture Data in Maya
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Cinema4D
How to use Face Capture data in Cinema4D
How to use Face Capture data in Blender (2.80)
How to use Face Capture data in Blender (2.80)
How to Live Stream Face Capture Data in Unity
How to Live Stream Face Capture Data in Unity
Everything you need to know about Rokoko Face Capture
Everything you need to know about Rokoko Face Capture

How to Live Stream Face Capture Data in Unity

Check out our latest tutorial that shows you how to live stream face data to your characters in Unity!

This blog serves as a guide to users who want to learn how to use our Unity Plugin to live stream face capture data to their custom characters within the Unity Engine. Please follow along below as we break down this process!

Please note that the Live Streaming feature within Rokoko Studio is only included under the paid Studio subscription plans. If you are only on the Studio Basic plan and want to take advantage of Live Streaming, check out our subscription plans here. It is also worth mentioning that this tutorial only goes over how to stream face capture data to your character. If you want to learn how to stream Smartsuit Pro data, check out our blog here.

It's also important to note that characters that utilize the standard ARKit 52 blendshapes are compatible with Face Capture. Make sure your character is set up with this method before proceeding. If you're looking for a way to rig your characters to take advantage of face capture, check out Polywink’s solution here.

Setting Up in Studio

  • Within Studio, click on the cogwheel at the top left to access the settings
  • Highlight ‘Data Streaming’:
  • Click the Stream Data’ button to enable it
  • You can also enable Live Streaming by clicking the ‘Start Live Stream’ panel to the right of Studio
  • Make sure that the ‘Forward IP’ is the local host IP (or the IP address for the computer you are streaming the data to)
  • Ensure that the ‘Forward Port’ is set to 14043
  • It is also important that the ‘Forward Port’ must be different from the ‘Smartsuit Listen Port’ otherwise it will cause issues
  • Once these Data Streaming settings are set, open Rokokok Remote app on your smartphone to set up Face Capture
  • As of writing this blog, Face Capture is only available for iPhoneX phones with front-facing TrueDepth cameras
  • Download the Rokoko Remote app for iphone here: https://apps.apple.com/us/app/rokoko-remote/id1465692290
  • Click the ‘Rokoko Remote’ button at the top right of Studio
  • Under “Available devices” you should see the name of your phone
  • To the right of your phone name in Studio, click the ‘Connect’ button
  • On your phone, a ‘Connect to Studio’ window will appear asking if you accept connecting to your computer
  • Tap the ‘Accept’ button to connect your phone to Studio
  • Once you tap ‘Accept’, the screen should show you that your phone is now connected to Studio
  • Tap the slider button to enable ‘Facial capture’ on your phone
  • When you tap the slider button, a prompt will appear asking if you would like “Rokoko Remote” to access the camera on your phone
  • Tap ‘OK’
  • Once Face Capture is set up, minimize Studio and open Unity.

Installing Rokoko Live

  • Click the Unity Launcher to open the Unity program.
  • Open an existing or new Unity Project (if this is your first time, we recommend opening a new project)
  • Import Smartsuit Plug-in from Asset Store
  • Click 'Window', then 'Asset Store' to pull up the Asset Store in-engine
  • Search for ‘Rokoko Live' then download and import it into your project

Setting up Smartsuit Receiver

  • After importing the plugin, within the Rokoko > Smartsuit folders, drag and drop the ‘SmartsuitReceiver’ prefab into your Hierarchy tab
  • This is required within your scene for live streaming to work
  • Make sure that the ‘SmartsuitReceiver’ prefab is highlighted in the Hierarchy so that you can see the attached script components in the Inspector
  • Paste the “Forward Port” number that you copied from Studio into both the “Port Range Start” and “Port Range End”

Setting up Character

  • Import your custom character into your project if you haven't already
  • Right-click on or within the "Assets” folder and click “Import New Asset”
  • Locate the FBX file(s) with your custom character, click “Import”
  • Drag and drop your character FBX (or prefab) into the scene
  • Locate the mesh that contains all of the blendshape data for the character’s face and click it within your Hierarchy to highlight it
  • Within the Inspector, click the ‘Add Component’s button and type ‘Face’ and click on the script named ‘Face’
  • With the Face script component added, you will need to add the Face ID of your face inside of Studio
  • The Face ID is located inside of the Scene Overview to the right of the face icon
  • The ID will be a combination of Upper and Lowercase letters and numbers
  • Type that Face ID inside of the ‘Face ID’ field inside of the Inspector in Unity
  • Once done, click the play button in Unity
  • You should now start to see the character’s face move around with your face inside of the Rokoko Remote app

We hope this guide has helped you learn how to forward your Face Capture data from Rokoko Studio directly to your character in Unity. If you run into any issues during this process, feel free to reach out to our support desk or email support@rokoko.com.

Face Capture
Rokoko Studio
Smartsuit Pro
Unity
Key takeaways
How to Live Stream Face Capture data to Unreal
How to Live Stream Face Capture data to Unreal
How to use Face Capture Data in Maya
How to use Face Capture Data in Maya
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Cinema4D
How to use Face Capture data in Cinema4D
How to use Face Capture data in Blender (2.80)
How to use Face Capture data in Blender (2.80)
How to Live Stream Face Capture Data in Unity
How to Live Stream Face Capture Data in Unity
Everything you need to know about Rokoko Face Capture
Everything you need to know about Rokoko Face Capture
More courses