Face Capture get started

Key takeaways
How to Live Stream Face Capture data to Unreal
How to Live Stream Face Capture data to Unreal
How to use Face Capture Data in Maya
How to use Face Capture Data in Maya
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Cinema4D
How to use Face Capture data in Cinema4D
How to use Face Capture data in Blender (2.80)
How to use Face Capture data in Blender (2.80)
How to Live Stream Face Capture Data in Unity
How to Live Stream Face Capture Data in Unity
Everything you need to know about Rokoko Face Capture
Everything you need to know about Rokoko Face Capture

Everything you need to know about Rokoko Face Capture

Not sure how to set up your character to take advantage of Face Capture? Check out this video on how to get your character ready for Face Capture!

Since the launch of Face Capture for Rokoko Studio, it’s definitely clear to us that this product is something you’ve all been longing for. We are very grateful for the great feedback and positive response. However, some of you are still unsure of how to set up your 3D characters properly to take advantage of Face Capture. This blog contains everything you need to know in order to set up your character, a guide to some ways to add blendshapes, and how to set up Face Capture in Studio to make your characters work in Unreal, Unity, and various other 3D software packages. It also features a guide on how you can get your face rigged with our new amazing partners at Polywink.

Read on below to get started!

How does the Face Capture work?

The Face Capture of Rokoko makes use of the ARKit facial performance capture, a feature found on iPhone X series devices. The performance capture works by analyzing the user’s face, tracking its movements and triggering a special set of 52 corresponding blendshapes. The general head rotation and movement is handled separately, while facial expressions are handled with blendshapes.

What do I need to animate my character?

The ARKit 2 facial performance capture uses a predefined set of 52 blendshapes (or blend shapes), including one for the tongue. In case you are new to blendshapes and facial animation, think of them as the different movements of the face, presented in the form of 52 different “shapes” of the face. These are then triggered in combination and blended together to produce an expression. Blendshapes are usually complemented with joints to handle certain aspects like eye rotation, but in this case the animation is done with blendshapes only.

Create ARKit Blendshapes / Use Polywink

After ensuring that your character has been set up properly for facial animation (more about that below), your next step would be to create blendshapes. You can do this manually if you have the modelling skillset to do so within your preferred 3D modeling software - in short, you must sculpt each of the 52 blendshapes manually, then test them in conjunction to ensure that the set works properly. Since Rokoko’s Face Capture takes advantage of the ARKit’s standard Blendshape set, make sure to closely follow Apple’s documentation on ARKit here.

Alternatively, our friends over at Polywink have developed a solution to automatically generate the blendshapes that you need: the “Animation for iPhone X” service outputs the 52 ARKit blendshapes of any custom character. Their online platform automates the process of creating facial rigs and blendshapes, delivering everything you need to bring your characters to life - you can simply upload your character on their website, and get it back within a couple of hours, ready for animation.

Rokoko has partnered with Polywink and now users can take advantage of a 10% discount off of their ‘Animation for iPhone X’ service with the code ROKOKO-POLYWINK.

To learn more info on how to use their service specifically for Rokoko’s Face Capture, check out their Animation for iPhone X page. If you don’t have the means to create blendshapes on your character, this method is easy, affordable and you’ll have blendshapes added to your character within 24 hours!

If you have a character ready to test, but aren’t sure if it’s setup properly for Polywink’s service, you can send your character to them for free to test for approval before getting it done. To take advantage of this, fill out the form here.

Setting up character for facial animation

First and foremost, it’s important to setup your character properly in order to apply blend shapes or face rigs to it. The following are steps that you should take to ensure that your character’s face is created to take advantage of blendshapes and face rigs:

Freeze the geometry position

  • Ensure that your model has null values in its’ transformation channels. If there are values in its’ channels then freeze them to zero out the values

Check the topology of the head

  • A good clean topology is crucial in order to give correct deformations
  • Messy topology will create deformations when the face is being animated. It’s also best to keep your polygons below 50K
  • Avoid non-manifold geometry and check for unmerged vertices. Keep a unified geometry
  • Don’t cut the geometry into distinct zones (eyes, ears, etc.)

Check the internal mouth

  • Make sure that the size of the internal mouth is not so big that it creates penetrations with the lips and cheeks
  • Keep some space between the external skin and the internal mouth
  • Make sure you have enough loops inside the mouth to follow the lip’s movements

Checking additional geometry

  • If your model has additional geometry (teeth, eyes, eyelashes, etc.) you can combine them with the head mesh if you want
  • Check your UV’s to make sure they are not overlapping

Set up Face Capture in Studio

Once your character is setup and ready to be used for Face Capture, it’s time to set up Face Capture within Rokoko Studio! We have created a helpful tutorial that walks you through this setup step by step. Check out our guide here.

It’s important to note that you will need an iPhone X (or newer) with front-facing TrueDepth cameras to take advantage of Face Capture. You can download our iPhone app, Rokoko Remote here.

You will also need to make sure you have Rokoko Studio 1.13.0 (or newer) to take advantage of Face Capture. You can download it here

Lastly you will also need to subscribe to the Face Capture add-on for Studio. You can do so here.

Test Face Capture on your character in Unreal

After Face Capture is set up and ready to go, you’ll now want to test it’s functionality on your character. If you’re using Unreal, you can test out our Live Streaming feature which allows you to stream your face capture data in real-time to Unreal. Check out how this is done here.

It’s important to note that Live Streaming is a premium Rokoko Studio feature. You will need a Rokoko Studio Plus subscription or higher to take advantage of it. Check out our subscription plans here.

Test Face Capture on your character in Unity

If you’re using Unity, you can test out our Live Streaming feature which allows you to stream your face capture data in real-time to Unity. Check out how this is done here.

Applying recorded face capture to your character in Maya

If you want to use recorded Face Capture data on your character inside of Maya, check out our guide on how to get that setup here.

Applying recorded face capture to your character in Blender

If you want to use recorded Face Capture data on your character inside of Blender, check out our guide on how to get that setup here.

Applying recorded face capture to your character in Cinema4D

If you want to use recorded Face Capture data on your character inside of Blender, check out our guide on how to get that setup here.

Face Capture
Rokoko Studio
Smartsuit Pro
No items found.
Key takeaways
How to Live Stream Face Capture data to Unreal
How to Live Stream Face Capture data to Unreal
How to use Face Capture Data in Maya
How to use Face Capture Data in Maya
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Blender (2.79)
How to use Face Capture data in Cinema4D
How to use Face Capture data in Cinema4D
How to use Face Capture data in Blender (2.80)
How to use Face Capture data in Blender (2.80)
How to Live Stream Face Capture Data in Unity
How to Live Stream Face Capture Data in Unity
Everything you need to know about Rokoko Face Capture
Everything you need to know about Rokoko Face Capture
More courses