Smartsuit Pro get started

Key takeaways
Smartsuit Pro Electronic parts
Smartsuit Pro Electronic parts
How to put on the suit correctly
How to put on the suit correctly
How To Set Up Wifi
How To Set Up Wifi
How To Update Firmware
How To Update Firmware
How To Perform A Straight Pose
How To Perform A Straight Pose
How To Update Location Settings
How To Update Location Settings
What The Colors In The Control Panel Mean
What The Colors In The Control Panel Mean
How To Create A Project
How To Create A Project
How To Export A Recording
How To Export A Recording
How To Use The Playback Panel
How To Use The Playback Panel
How To Forward Real Time Data to MotionBuilder
How To Forward Real Time Data to MotionBuilder
How To Make A Recording
How To Make A Recording
Fast movements with Smartsuit Pro
Fast movements with Smartsuit Pro
Cleaning up Motion Data in Studio
Cleaning up Motion Data in Studio
Working with Character Animations in Unreal
Working with Character Animations in Unreal
Working with Character Animations in Unity
Working with Character Animations in Unity
Rigging in Mixamo and Importing Animations in Unreal
Rigging in Mixamo and Importing Animations in Unreal
Importing Mixamo Characters and Studio Animations in Unity
Importing Mixamo Characters and Studio Animations in Unity
How to Forward Real-Time Data to your Character in Unity
How to Forward Real-Time Data to your Character in Unity
How to Clean up Studio Animations in Maya
How to Clean up Studio Animations in Maya
How to Apply Studio Animations to Your Character in Maya
How to Apply Studio Animations to Your Character in Maya
Applying Studio Animations to Your Character in Cinema4D
Applying Studio Animations to Your Character in Cinema4D
Adding Studio Animations to Your Daz3D Character
Adding Studio Animations to Your Daz3D Character
Working with Smartsuit Pro data from multiple characters in Unity
Working with Smartsuit Pro data from multiple characters in Unity
Working with Smartsuit Pro data from multiple characters in Unreal
Working with Smartsuit Pro data from multiple characters in Unreal
Virtual Production Setup
Virtual Production Setup
Applying Studio Recordings in Blender
Applying Studio Recordings in Blender
Applying Studio recordings to your character in MotionBuilder
Applying Studio recordings to your character in MotionBuilder
How to forward Real-Time data to your character in Unreal
How to forward Real-Time data to your character in Unreal
Cleaning Up Smartsuit Pro Data in Studio
Cleaning Up Smartsuit Pro Data in Studio
How to use Smartsuit Pro data in Blender (2.8)
How to use Smartsuit Pro data in Blender (2.8)
How to Live Stream Data to MotionBuilder
How to Live Stream Data to MotionBuilder
Using the Cinema4D Retarget Tag to Switch Between Rokoko Animations
Using the Cinema4D Retarget Tag to Switch Between Rokoko Animations
How to use the treadmill filter in Rokoko Studio
How to use the treadmill filter in Rokoko Studio
Hands-on Demo of Live Studio Filters (Locomotion 2.0, Locomotion Legacy, Treadmill)
Hands-on Demo of Live Studio Filters (Locomotion 2.0, Locomotion Legacy, Treadmill)
Introduction to Rokoko Studio and Essential Filters (locomotion, loop, treadmill, etc)
Introduction to Rokoko Studio and Essential Filters (locomotion, loop, treadmill, etc)
How to Forward Live Motion Capture Data to Blender
How to Forward Live Motion Capture Data to Blender
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Live Stream Full Performance Capture to Unreal Engine 2.23
Live Stream Full Performance Capture to Unreal Engine 2.23
Organize Rokoko Studio Animations in MotionBuilder
Organize Rokoko Studio Animations in MotionBuilder
Rokoko Studio v1.15: Loop filter (Walk/Run) Cycles, FPS Lock, C3D Format
Rokoko Studio v1.15: Loop filter (Walk/Run) Cycles, FPS Lock, C3D Format
How to Characterize your Animation Skeleton in MotionBuilder
How to Characterize your Animation Skeleton in MotionBuilder
How to characterize your animation skeleton in Maya
How to characterize your animation skeleton in Maya
Rokoko Live / Unreal Engine 4.24 - Overview of what's in the new plugin
Rokoko Live / Unreal Engine 4.24 - Overview of what's in the new plugin
How to clean up motion capture animations in MotionBuilder
How to clean up motion capture animations in MotionBuilder
Rokoko Livestream / Rokoko Studio Filters / Dec. 12, 2019 (locomotion, drift fix, treadmill, footIk)
Rokoko Livestream / Rokoko Studio Filters / Dec. 12, 2019 (locomotion, drift fix, treadmill, footIk)
How to master the Locomotion 2.0 filter in Rokoko Studio
How to master the Locomotion 2.0 filter in Rokoko Studio
How to create loop cycles with your Smartsuit Pro recordings in Rokoko Studio
How to create loop cycles with your Smartsuit Pro recordings in Rokoko Studio
Full performance capture (and face rigging) workflow for Daz3D characters
Full performance capture (and face rigging) workflow for Daz3D characters
Live Stream Full Performance Capture to Unreal Engine 4.23
Live Stream Full Performance Capture to Unreal Engine 4.23

Live Stream Full Performance Capture to Unreal Engine 4.23

This blog will serve as a guide on how to setup real-time streaming of full performance capture, Face and Body, from Rokoko Studio on to your custom characters in the Unreal Engine 4.23 and newer.

In this guide, we show you how to setup real-time streaming of full performance capture, Face and Body, from Rokoko Studio on to your custom characters in the Unreal Engine 4.23 and newer. Let's get started!

Setting your Project in Rokoko Studio

  • Within Studio, click on the cogwheel at the top left to access the settings
  • Click ‘Studio Live’:
  • Enable Unreal Engine Streaming
  • Click the cogwheel on the Unreal Engine Icon.
  • Make sure that the ‘Forward IP’ is the local host IP (or the IP address for the computer you are streaming the data to)
  • Ensure that the ‘Forward Port’ is set to 14043
  1. It is also important that the ‘Forward Port’ must be different from the ‘Smartsuit Listen Port’ otherwise it will cause issues
  • Once these Data Streaming Settings are set, exit Rokoko Studio live and open your Rokoko Studio Project with your actor profile.
  • When your Smartsuit Pro is connected and streaming into Rokoko Studio, add your Actor profile in the Scene Overview window by dragging your profile icon onto you Smartsuit Pro icon.
  • Enable your face capture data but going to your iPhone X (or newer) and download the Rokoko Remote App.
  • Open the App and be sure to have your phone on the same WiFi network as your computer. Go back to Rokoko Studio on your computer. Click on the Rokoko Remote icon. Click Connect. Your phone will promote you to accept. Now that your phone is connected, toggle the Face Capture switch to enable Face Capture.
  • In the Scene Overview Panel, drag your face icon onto your Smartsuit Pro and Actor Profile Icons to attach the face capture onto your character in Rokoko Studio.

Setting your Project in Unreal Engine (4.23 and newer)

Ensure that The Rokoko Live Plugin is installed in your Project and it is enabled. You can download it for free off of the Unreal Marketplace.

Animation Blueprints:

Body Capture Blueprint:

  • Open your project in Unreal and import your Character
  • Right-Click within the Content Browser and select “Import to /Game…”
  1. Find the FBX of your character and select the Open button
  2. If your character isn’t using an existing skeleton, leave Skeleton set to ‘None’
  3. Click “Import”
  • Once your character is imported, right-click on the skeletal mesh of your character, highlight ‘Create’ and click ‘Anim Blueprint’
  • Double-click on the newly created animation blueprint
  • Create the following set of nodes:
  1. Right-click and type “Get Smartsuit Controller” and select that node
  2. Left-click and drag from ‘Return Value’, type “Smartsuit Body Pose” and select that node
  3. Left-click and drag from ‘Animation Pose (Component Space)’, type “Component to Local” and select that node
  4. Left-click and drag from ‘Animation Pose’ and plug it into ‘Result’ on the ‘Final Animation Pose’ node
  • Left-click the ‘Smartsuit Body Pose’ node and expand ‘Bone Map’
  1. If you are using a Mixamo rigged character, match the following character bones to the bone map:
  • Hip = Hips
  • Stomach = Spine
  • Chest = Spine2
  • Neck = Neck
  • Head = Head
  • Head Top = HeadTop_End
  • Left Shoulder = LeftShoulder
  • Left Arm = LeftArm
  • Left Forearm = LeftForearm
  • Left Hand = LeftHand
  • Left Finger Tip = LeftHandMiddle4
  • Right Shoulder = RightShoulder
  • Right Arm = RightArm
  • Right Forearm = RightForearm
  • Right Hand = RightHand
  • Right Finger Tip = RightHandMiddle4
  • Left Upleg = LeftUpLeg
  • Left Leg = LeftLeg
  • Left Foot = LeftFoot
  • Left Toe = LeftToe_End
  • Right Upleg = RightUpLeg
  • Right Leg = RightLeg
  • Right Foot = RightFoot
  • Right Toe = RightTo_End
  • ‘Relative to Start’ checkbox:
  1. When this is True (checked), the animation will keep the starting position on the XY-plane of the character when the game starts
  2. When this is False (unchecked), the animation will jump to the absolute position of your character inside of Rokoko Studio when the game starts
  • ‘Scale Bones’ checkbox:
  1. When this is True (checked), this will scale the bones to match the model’s body dimensions. This can make the animation look more natural on the character, but may cause unusual effects on the mesh
  2. This is usually left set to False, unless you have a character specifically created to take advantage of this feature
  • Once you’re done assigning bones, click the ‘Compile’ button at the top left*
  1. A warning may appear, however, this will not affect performance so you can ignore it
  • Close the Animation Blueprint window
  • Drag and drop the character’s animation blueprint from the Content Browser into the scene
  1. It is important to note that the XYZ axis rotations for your character MUST be set to 0. Otherwise, this could cause some issues with the bones being rotated improperly when the game starts.
  • In the Modes Panel, type in ‘SmartSuit’. Drag and Drop ‘SmartSuitReceiver’ and ‘SmartSuitController’ in your scene.
  • Highlight, ‘SmartsuitReceiver’ in the ‘WorldOutliner’, under Details, double check the Streaming Data Port number and VP Data Port number are the same as that in Rokoko Studio.
  • Highlight ‘SmartsuitController’ in the ‘WorldOutliner’, under Details, type in your SmartSuit Pro name that Rokoko Studio has assigned the Suit. This name can be found in the ‘Scene Overview’ panel in Rokoko Studio.
  • Check ‘Auto-Connect’
  • Press the Play button and watch as your character moves along with the Smartsuit Pro’s movement!

Face Capture Blueprint:

  • Drag and drop your Character's skeletal mesh into your scene in Unreal, highlight your Character.
  • In the Details panel, click ‘Blueprint/Add Script’
  1. Click the folder you want to create a Blueprint (Character folder works)
  2. Click the ‘Create Blueprint’ button
  • The Blueprint should open on it’s own, if not, double-click on your newly added blueprint within the Content Browser
  • Click the ‘Add Component’ button and type ‘LiveLink’
  1. Click on ‘Live Link Skeleton Animation’ from the dropdown to add it as a component
  • Under the ‘My Blueprint’ panel, click the “+” on the ‘Functions’ panel
  1. Type ‘LiveLinkFaceNode’
  2. Within the Live Link Face Node function window, click ‘Live Link Face Node’
  • Under the Details panel, click the “+” on the ‘Inputs’ panel 3 times to add new parameters
  • Name each parameter as follows:
  1. “SubjectName”
  2. “CurveName”
  3. “BlendshapeName”
  • Change all 3 parameters from ‘Boolean’ to ‘String’ by clicking the dropdown and selecting ‘String’
  • Drag out from the Execution Pin on ‘Live Link Face Node’ and search “Evaluate Live Link Frame”
  • On the Evaluate Live Link Frame Node, for the subject name, type in ‘arkit’ (case sensitive). For Role, select AnimationRole.
  • Drag out from Data Result, add and attach it to “Get Curves”
  • Drag out from ‘Curves’ and search ‘Find’
  • Drag out from the green pin on ‘Find’ node and search ‘float / float’
  • Change the float inside of this node from ‘1.0’ to ‘100.0’
  • Drag out from the green pin on ‘float / float’ pin and search ‘Set Morph Target (SkeletalMeshComponent)
  • Go back to LiveLink Face Node, drag from ‘Curve Name’ to ‘Find’.
  • Duplicate the ‘Dot’ node in between ‘Curve Name’ and ‘Find’.
  • Drag out the Blendshape Name to the new ‘Dot’, then attach the ‘Dot’ to ‘Morph Target Name’ on ‘Set Morph Target’ node.
  • Drag out from the ‘Dot’ pin leading from ‘Curve Name’ on ‘Live Link Face Node’ and attach to the purple pin on ‘Find’ node
  • Drag out from the ‘Dot’ pin leading from ‘Blendshape Name’ on ‘Live Link Face Node’ and attach to ‘Morph Target Name’ on ‘Set Morph Target’ node
  • Drag out from Execution Pin of ‘Evaluate LiveLink Frame’ node to the Execution Pin on ‘Set Morph Target’ node
  • Click the ‘Compile’ button
  • Green check mark means it has been set up properly
  • If you don’t get a green check mark, go through these steps again to make sure the nodes were set up properly

Next, click the ‘Event Graph’ tab

  • Drag out from the Execution Pin on ‘Event Tick’ node and search ‘Live Link Face Node’
  • Click the box for ‘Subject Name’ and type ‘arkit’
  • Copy the ‘Live Link Face Node’ and paste one for every blendshape you have for your character (in our case there are 52 blendshapes so we create 52 nodes)
  • Line each of the created nodes up from left to right
  • Next, open your character’s Skeletal Mesh and locate the ‘Morph Target Preview’ tab
  • This tab should list all of your face blendshapes
  • You will need to input the names of each blendshape inside the ‘Curve Name’ and ‘Blendshape Name’ fields inside the ‘Live Link Face Node’ on the Event Graph
  • The quickest way to do this is to right-click and click ‘Copy Name’ within your ‘Morph Target Preview’ and paste the names inside of each node
  • Be careful of extra spaces at the end of the name - make sure to delete them
  • Once all of the names have been added to each of the nodes, drag from the Execution Pin on ‘Event Tick’ node and attach it to the Execution Pin in the first ‘Live Link Face Node’
  • Now drag each of the Execution Pins to the following ‘Live Link Face Node’ in front of it until all nodes are attached to each other
  • The very last ‘Live Link Face Node’ will not have an Execution Pin leading out to anywhere
  • Click the ‘Compile’ button
  • Green check mark means it has been set up properly

Next, Click on the Event Graph tab of your Character’s Blueprint.

  • Drag out from the Execution Pin on ‘Event Tick’ node and search ‘Live Link Face Node’
  • Click the box for ‘Subject Name’ and type ‘arkit’
  • Copy the ‘Live Link Face Node’ and paste one for every blendshape you have for your character (in our case there are 52 blendshapes so we create 52 nodes)
  • Line each of the created nodes up from left to right
  • Next, open your character’s Skeletal Mesh and locate the ‘Morph Target Preview’ tab
  • This tab should list all of your face blendshapes
  • You will need to input the names of each blendshape inside the ‘Curve Name’ and ‘Blendshape Name’ fields inside the ‘Live Link Face Node’ on the Event Graph
  • The quickest way to do this is to right-click and click ‘Copy Name’ within your ‘Morph Target Preview’ and paste the names inside of each node
  • Be careful of extra spaces at the end of the name - make sure to delete them
  • Once all of the names have been added to each of the nodes, drag from the Execution Pin on ‘Event Tick’ node and attach it to the Execution Pin in the first ‘Live Link Face Node’
  • Now drag each of the Execution Pins to the following ‘Live Link Face Node’ in front of it until all nodes are attached to each other
  • The very last ‘Live Link Face Node’ will not have an Execution Pin leading out to anywhere
  • Click the ‘Compile’ button
  • Green check mark means it has been set up properly
  • If you don’t get a green check mark, go through these steps again to make sure the nodes were set up properly
  • Close out of the Blueprint window

Setting up your Scene:

  • Drag and drop your Character’s new blueprint into your scene
  • Highlight your Character, in the ‘Details’ panel, under ‘Animation’, ‘Animation Class’, assign your Body Capture Animation Blueprint.
  • Click the Window dropdown menu, and activate LiveLink.
  1. ‘Add Source’
  2. ‘Rokoko Studio Source’
  3. ‘Studio’
  • Head back to your scene and click ‘Play’

We hope this guide has helped you learn how to forward your SmartSuit Pro Body Capture Data Face Capture data from Rokoko Studio directly to your character in Unreal 4.23 and above. If you run into any issues during this process, feel free to reach out to our support desk or email support@rokoko.com.

Rokoko Guide: Performance Capture Live, How to Stream Body and Face Capture into Unreal Engine 4.23

Link to Face Capture in Unreal 4.22 and earlier versions: https://www.rokoko.com/en/explore/blog/face-capture-unreal

Face Capture
Rokoko Studio
Smartsuit Pro
Unreal
Key takeaways
Smartsuit Pro Electronic parts
Smartsuit Pro Electronic parts
How to put on the suit correctly
How to put on the suit correctly
How To Set Up Wifi
How To Set Up Wifi
How To Update Firmware
How To Update Firmware
How To Perform A Straight Pose
How To Perform A Straight Pose
How To Update Location Settings
How To Update Location Settings
What The Colors In The Control Panel Mean
What The Colors In The Control Panel Mean
How To Create A Project
How To Create A Project
How To Export A Recording
How To Export A Recording
How To Use The Playback Panel
How To Use The Playback Panel
How To Forward Real Time Data to MotionBuilder
How To Forward Real Time Data to MotionBuilder
How To Make A Recording
How To Make A Recording
Fast movements with Smartsuit Pro
Fast movements with Smartsuit Pro
Cleaning up Motion Data in Studio
Cleaning up Motion Data in Studio
Working with Character Animations in Unreal
Working with Character Animations in Unreal
Working with Character Animations in Unity
Working with Character Animations in Unity
Rigging in Mixamo and Importing Animations in Unreal
Rigging in Mixamo and Importing Animations in Unreal
Importing Mixamo Characters and Studio Animations in Unity
Importing Mixamo Characters and Studio Animations in Unity
How to Forward Real-Time Data to your Character in Unity
How to Forward Real-Time Data to your Character in Unity
How to Clean up Studio Animations in Maya
How to Clean up Studio Animations in Maya
How to Apply Studio Animations to Your Character in Maya
How to Apply Studio Animations to Your Character in Maya
Applying Studio Animations to Your Character in Cinema4D
Applying Studio Animations to Your Character in Cinema4D
Adding Studio Animations to Your Daz3D Character
Adding Studio Animations to Your Daz3D Character
Working with Smartsuit Pro data from multiple characters in Unity
Working with Smartsuit Pro data from multiple characters in Unity
Working with Smartsuit Pro data from multiple characters in Unreal
Working with Smartsuit Pro data from multiple characters in Unreal
Virtual Production Setup
Virtual Production Setup
Applying Studio Recordings in Blender
Applying Studio Recordings in Blender
Applying Studio recordings to your character in MotionBuilder
Applying Studio recordings to your character in MotionBuilder
How to forward Real-Time data to your character in Unreal
How to forward Real-Time data to your character in Unreal
Cleaning Up Smartsuit Pro Data in Studio
Cleaning Up Smartsuit Pro Data in Studio
How to use Smartsuit Pro data in Blender (2.8)
How to use Smartsuit Pro data in Blender (2.8)
How to Live Stream Data to MotionBuilder
How to Live Stream Data to MotionBuilder
Using the Cinema4D Retarget Tag to Switch Between Rokoko Animations
Using the Cinema4D Retarget Tag to Switch Between Rokoko Animations
How to use the treadmill filter in Rokoko Studio
How to use the treadmill filter in Rokoko Studio
Hands-on Demo of Live Studio Filters (Locomotion 2.0, Locomotion Legacy, Treadmill)
Hands-on Demo of Live Studio Filters (Locomotion 2.0, Locomotion Legacy, Treadmill)
Introduction to Rokoko Studio and Essential Filters (locomotion, loop, treadmill, etc)
Introduction to Rokoko Studio and Essential Filters (locomotion, loop, treadmill, etc)
How to Forward Live Motion Capture Data to Blender
How to Forward Live Motion Capture Data to Blender
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Rokoko Studio v1.15.1: Our Blender plugin is here! + Custom plugins for all Pro/Plus users!
Live Stream Full Performance Capture to Unreal Engine 2.23
Live Stream Full Performance Capture to Unreal Engine 2.23
Organize Rokoko Studio Animations in MotionBuilder
Organize Rokoko Studio Animations in MotionBuilder
Rokoko Studio v1.15: Loop filter (Walk/Run) Cycles, FPS Lock, C3D Format
Rokoko Studio v1.15: Loop filter (Walk/Run) Cycles, FPS Lock, C3D Format
How to Characterize your Animation Skeleton in MotionBuilder
How to Characterize your Animation Skeleton in MotionBuilder
How to characterize your animation skeleton in Maya
How to characterize your animation skeleton in Maya
Rokoko Live / Unreal Engine 4.24 - Overview of what's in the new plugin
Rokoko Live / Unreal Engine 4.24 - Overview of what's in the new plugin
How to clean up motion capture animations in MotionBuilder
How to clean up motion capture animations in MotionBuilder
Rokoko Livestream / Rokoko Studio Filters / Dec. 12, 2019 (locomotion, drift fix, treadmill, footIk)
Rokoko Livestream / Rokoko Studio Filters / Dec. 12, 2019 (locomotion, drift fix, treadmill, footIk)
How to master the Locomotion 2.0 filter in Rokoko Studio
How to master the Locomotion 2.0 filter in Rokoko Studio
How to create loop cycles with your Smartsuit Pro recordings in Rokoko Studio
How to create loop cycles with your Smartsuit Pro recordings in Rokoko Studio
Full performance capture (and face rigging) workflow for Daz3D characters
Full performance capture (and face rigging) workflow for Daz3D characters
Live Stream Full Performance Capture to Unreal Engine 4.23
Live Stream Full Performance Capture to Unreal Engine 4.23
More courses