Retargeting made easy in Unreal I The Ultimate Retargeting Guide

Below, please find a transcript of the video tutorial. Enjoy!

‍‍

Introduction

Hey, what's going on everyone? Welcome to this Unreal Engine Motion Capture retargeting videos. We're going to start with some motion capture that we've recorded into Rokoko Studio. Then we're going to take a character from Mixamo. Although this workflow will work with any rigged character that you might have, we're gonna bring both the mocap and the character into Unreal Engine and then retarget that motion capture onto our character using the new IK Retargeting System in Unreal Engine. We're also gonna get a little bit into mocap cleanup and tweaking. So before we get started, I wanna mention a few things. One is that this video is going to be for rigged characters other than Metahumans. We're gonna have a whole other video that's about metahumans because they have some particular things to them that don't really fit into this videos. 

The other thing I'll mention specifically for Unreal Engine is that a lot of unreal engine motion capture workflows don't actually retarget motion capture in Unreal. Instead, the retargeting process is handled in programs like Maya or Blender software that has been working with animation specifically for a really long time. So while this workflow totally does work, it's not the most robust retargeting workflow for motion capture, just because it's not really what Unreal is really the best at yet. This is not the only retargeting workflow. There are other ones that are good as well, and we'll put any other links we might think are useful down in the description below, because Unreal is still trying to get its footing a little bit when it comes to retargeting. Without further ado, let's jump into it. 

Exporting recorded mocap into UE

Okay, so here we are in Rokoko Studio. I've already recorded some motion capture. We're, we're not gonna go through how to record mocap in the suit and gloves in this video. We have plenty of tutorials on that. If you, if you want to know how to create motion capture in Rokoko Studio or use Rokoko video instead, I've already done it for the purposes of this video. And if we check it out, you can see that kind of jumping around doing my karate kicks and then I'm going to go and touch my head and you can see that when I touch my head and also when I clap that I don't really make proper contact. And you know, you can see here, and that's pretty typical for inertial motion capture. There may be things that you know, where, where your hands don't line up specifically, or, you know, touch parts of your body. Exactly. And so that's why it's good to be able to edit motion capture after you've retargeted it. And we're gonna, we're gonna talk about that later. But I've recorded, I've gone through and I've cleaned up my, uh, foot contact with the ground following the normal method. 

And now we need to export out this motion capture for Unreal. So I'm going to right click and it was already in the export list, but I'm going to add to export list. And for Unreal, we want to export out an fbx. We need to have body mesh checked because Unreal will not import your mocap without a mesh attached to that mocap data. So we need body mesh checked under skeleton preset. There is an unreal engine preset. We are not going to use that. I'm gonna use a Mixamo preset. Um, we'll update this video in the future. If, if the workflow moves over to this unreal skeleton, we're gonna use the MIMO skeleton for now though, um, for a preset overrides. However, here we can set if we want our character to be in a T pose or in an A pose. And if you are retargeting this motion capture to an unreal character that's in an A pose, then you would set both of these reference pose and rest pose to a pose. 

However, today we're going to be retargeting to a Mixamo character, which is gonna be in a t pose. So in that case, we want to set these both to t pose under more settings. I'm just gonna set it to the most recent FBX version and I'm gonna export it 30 fbs. And, you know, we can choose where we want to export out to. I'm gonna leave it to this default folder and we'll hit export clips. So that's done. Now we need to go and actually grab a character. 

Downloading pre-rigged Mixamo character

And in this case we're gonna be jumping onto Mixamo and just rigging a character on here. You can use this same workflow, uh, with characters that you, you either maybe bought online or maybe you made yourself. Um, again, we're gonna have a separate video for Metahumans, but this workflow will work for any character that you have that has a a typical rig. Um, in this case, we're just using Mixamo cuz it's easy. So we're gonna drag in our character, go through an auto rig, our character on Mixamo. There we go. Our character has been rigged. You can see he's got his own little red beanie just like me. And now we want to download this character in a t pose. So if we go to animations and we search for TPOs, 

There we go. Now we know our character will be in a T pose and we download it. Um, so we're gonna hit download and now we can go ahead and open up Unreal Engine. 

Importing the Mixamo character into UE

So I'm gonna be using Unreal Engine 5.1. We don't actually need any plugins or anything for this workflow. We just need our character in a TPO and our mocap from a Coco Studio, which we already exported in a TPOs, which we took care of when we exported. And other than that, you just need 5.1, although this workflow will will also work with 5.0. So I'm just gonna create a blank project with the, uh, film and video preset and I'll include the starter content, although we won't need it and we'll create, okay, so here we are in Unreal. And now let's just make a couple folders to keep organized. So I'll make a mocap folder and I'll make a MIMO character folder and let's go import our MIMO character first. So I'll just open up that folder, add import to game. Mine is just in downloads. I should have renamed it maybe in this case for our Mixamo character. I don't want to have import animations on, so I'll hit import all. 

There we go. Looks good. Now we'll go into our mocap folder and we'll import our mocap. And this is what we just exported from Rokoko Studio Retargeting demo. And I will open, and for this one, we wanna make sure that sometimes this um, skeleton skeletal mesh field will be filled. Make sure that this is blank and that it doesn't select the character. And then also you want to hit import animations, right? Because this is our mocap, this is, this is our actual animation. So we want to bring in those key frames. You can ignore these kind of warning signs that you might get. But here we go. Now if we open up our animation, you can see we're looking good. 

Retargeting mocap onto the Mixamo character: IK control rig

So how do we get this animation onto our character now, right? How do we retarget this animation onto our character? Well, what retargeting is, you know, when we take mocap, we put it on a rigged character. We are telling the software, Hey, take these key frames from the left arm of your mocap, right? And put it on the left arm of your character. Well, unreal doesn't natively know which bone is which, which bone is the left arm in your character, which bone is the left arm in your mocap. So we need to go in and tell Unreal, hey, this is each bone in my mocap, this is each bone in my character. When I tell you to copy the animation from my mocap, my character, you know where all that animation should go because I've told you. So the way that we do this in Unreal now is using a system called the IK Retargeter. 

And there's a couple different workflows in Unreal for retargeting. This is one we're going to use. Um, and it starts with creating IK rig objects or assets for both our mocap and our character. So I'm gonna create the IK rig asset for my mocap first because I'm in my mocap folder still. So I'll right click, go to animation, and then we're gonna go to IK rig. You could also type this in and IK rig. And we want to pick our mocap, right? And so we're gonna call this mocap IK rig again just to keep everything organized right? And we'll open this up. Okay, so here we have our mesh, our skeleton here, we have all of the different bones in our skeleton. And then over here we have this IK retargeting panel. So the first thing we'll do is we'll go over to hips, which is gonna be the root bone of our mocap, and we'll right click and we will set retarget root. Next we're gonna go make some of those IK chains. So I'll hit add new chain and I'll call this one head. 

I'm not gonna add any goals for now. So I'll hit no goal and I'm gonna keep going. So I'm gonna create another one. I'm gonna call this spine No goal. I'm gonna create another one. I'm gonna call this arm L No goal. I'm gonna create another and I'll call this leg L No goal again. And once we have these uh, chains, what we're gonna do is for the head for instance, we would go and for start bone it would be the neck and end bone would be the head for the spine. The start bone is gonna be the first spine bone, and then the end bone is gonna be spine two. For the arm l this is gonna be our left arm. We're gonna go left shoulder down to left hand. And for leg we're gonna go left up leg all the way down through left foot. 

And we're only doing the left side for now because what we can do is we can go in and if we right click on arm L, I can hit mirror chain and let me just type in arm R And because we've added in these bones, um, it knows to go in and mirror for arm R right, right shoulder to right hand to did it for us. So we only need to do the left side. So we'll do the same thing for leg R. So this is kind of nice that it does this mirroring. And there we go. Everything is rigged. Now what we're also gonna do right now that I'm gonna speed through is I'm gonna go in and I'm gonna do the same thing for all the uh, fingers. So you don't need to do this if you, you know, you're using Rokoko video. There is no finger animation on the mocap in this case we do have glove animation, so we're gonna do it for each finger as well and then mirror it for the right side. And uh, so I'll speed through that right here. 

So there we go, everything is filled out and I actually had to go in and manually add in the right side. I couldn't mirror them. So there might be small changes that will happen in this workflow. I just kind of gotta roll with the punches. But this is done Now, um, our, our mocap is now done. So I'm going to save this and we're gonna just go and do the exact same thing for our character, right? So I'm gonna go into Mixamo character, right click animation, IK rig, ik rig, select our character. This time we're gonna call this MIMO IK rig and we'll just do the exact same process. And I'm gonna name these things the same, which is important because that's how unreal it's going to know, Hey, I saw the same bone chain on the mocap so I know where this animation should go, right? So I'm gonna name it these chains the same way that I named the ones for our mocap. 

We are all done, uh, with our bone chains all set up and we also need to go and for the hips right click set retarget route. Again, we need to set the retarget route for both our mocap and for our character. So now we can save this and now we're ready to actually retarget the motion capture. So to do that we can right click on our main level here and we're gonna go to animation ik rig ik retargeter, and we get this pop-up pick ik rig to copy animation from. So where is the animation coming from? Well, in our case it's from our mocap IK rig. So I'm gonna call this retargeting oh one and we'll open up this asset and you'll see the first thing that that we see is our mocap skeleton, right? And up here we can see source ik rig asset and source preview mesh. These are both from our mocap. So Target, we want to go and we want to select our Mixamo character. So we select Mixamo character here. We can actually offset this character so we can move them 

And we could change the scale if we wanted to. In this case, we don't really need to or want to, but here we go. If we go to chain mapping, you can see that everything target chain, source chain, these all match up, right? So that means that the each, the rig is going to talk to uh, the mocap. However, you can see that we have this problem here where our character for Mixamo actually did come down in this a pose. Um, it's not even an A pose, it's kind of the pose that we, uh, uploaded the character to Mixamo in either way. What we need to do is make sure that our character and our mocap are matching on this frame zero. Because if we go to the asset browser, and again, because we've plugged in the source as our mocap and the target as our character, if we double click on this anim, you can see that we actually get, 

Look, we got our mocap right, however, it's not doing the right thing. The arms are completely messed up and that's because they're offset the same amount that they're offset. When we do the retargeting, you can see this, right? So it's really easy to fix this now. So what we can do is actually just go to edit mode. This used to be very difficult go to edit mode. And once we're in edit mode, we can actually, if we go to bones and show all hierarchy, we can select these bones and move them into place. So in this case, we know that our characters should be in a T pose. And we could also do this the other way, right? We could repose our mocap, it doesn't matter as long as they both end up in a t pose. We could also, you know, we, we might have to fix the, um, for example, the thumbs here it looks like are a little bit messed up. 

You can see that on our mocap, the thumbs are lying flat on our character. The thumbs are, it's kind of hard to see. You can uh, select it only, but the thumb isn't lying flat, right? So we fixed this arm issue. We might still see a thumb issue, but at least this we can exit edit mode, run retargeter and look at that. It looks perfect. If you were having problems be in your, with your character not moving around, that would stem from not setting that root, um, retargeting root bone in your IK rigs. Um, but yeah, this is the basic retargeting process. So if this all looks good, we could just select this animation. And if we go to export selected animations, we'll just put it in our content folder. We could add a prefix prefix, um, called this Anim is the Prefix and Export. 

And now if we go to our main content folder, look at this. We have a character that has our animation on it that is ready to go. That's great. This is the basic, you know, retargeting workflow. But the problem with motion capture always comes kind of here. So you can see we've retargeted our mocap, but for whatever reason, maybe the mocap, it was kind of a, not a the best take initially. Or, um, you know, maybe the proportions of your character are really different from yours and you run into issues like clipping. You know, we can see here the, the hand is passing right through the head or issues where we, you know, need the hands to be in a place where they aren't where we need to. We need to do slight mocap edits or tweaks. And another way that this sometimes manifests is with uh, foot sliding, especially if you have like a character with really short legs or really long legs, you can start getting foot sliding. 

So how can we fix some of these things? So if we jump back to our IK retargeter, remember that this is the actual, you know, this is where we actually did our retargeting. If we go to chain mapping global settings, you can see here that we have all three of these check boxes selected. So let's turn off IK and let's make sure to go to asset browser double click. And whether or not IK is on or off right now, it we see no difference. And that's because we haven't set up any IK handles on our character. So we're gonna go back and do that right now and that will give us a little bit of control to fix some of these problems like this. The other thing we can do is fix foot sliding by using IK handles. But later at the end of the video we'll go through how to do mocap editing in sequencer and also some resources for building control rigs in Unreal. 

But for now, let's go and actually set up some IK handles on our character. So I'm gonna go back into Mixamo character. I'm gonna select Mixamo IK rig. And you can see before we're gonna also go to Character Bones all hierarchy. So I just wanna see them all. You can see here we have IK goal is set to none for all of these. So what if we wanted to create an IK goal for the left arm so we could have done that when we were creating those IK chains, but I prefer to do it this way once I've done all this process, the first thing that we need to do is add a new solver. So this is going to be a limb solver for the left arm. So I'm gonna go to Limb ik. And you can see here it says missing goal. 

So we're gonna go to the left hand again, this is the left arm. We're doing new IK goal and we want to assign goal and it assigns it to the left hand. You can see it puts a little bounding box by that hand joint. And then while this solver is down here, you can see that it says missing root. While this is selected, we're gonna go up to left arm. We could go to the shoulder, but I like to cap it at the arm, otherwise the shoulder starts moving in weird ways. So we're gonna go to left arm and go to set root bone on selected solver. And once we do that, you can see it kind of goes green and this, this doesn't have that warning anymore. We can actually grab this hand. And now we have an IK chain for this hand. And I'm gonna go through and I'm gonna do the same thing for the right hand and it needed a little bit of fiddling, but there we go.

Sometimes you have to go in and it will kind of get these confused. And so you just need to be, you know, maybe delete it and redo it. And eventually you should end up with, uh, you know, ik um, handles that you can use to move the, the arm around from the wrist. So the other thing we're going to do is I'm gonna go and dis make sure nothing is selected. You can see that it's automatically added in ARM R and arm l these goals, which is right, that's what we want. Now we want to add a full body IK solver. And for this we are going to go to the left foot. We're gonna add a new IK goal, assigned goal. We're gonna go to the right foot new IK goal, assign goal. And then for with the full body IK solver selected, we're gonna go up to the hips and set the hips as the root bone on selected solver, right? 

So now we have feet that can be used for uh, IK purposes as well. And so those feet are gonna allow us, if we do have foot slipping, using those feet goals can help, uh, you know, lock the feet into place. So we don't have to use any of these, we've just created them. And when we're retargeting, we can choose whether or not to use them. So let's, let's go use them right now. So I'll hit save again. Now let's go and let's just create a new IK Retargeter. Even though we have one that's already worked, we're gonna leave that alone cuz we used it to create our first animation. Where do we pick animation from? It's gonna be from our mocap again, right? We're gonna call this retargeting oh two, open this up. Same thing as before. Remember we need to go and add in our Mixamo character. 

We can offset that character slightly. We need to edit and fix these arms. And now if we go to global settings under chain mapping first if we load in our animation, but then we go to chain setting global settings. And if we, we can turn on and off ik. And now you can see that we're getting a real difference here in what's happening. So this is turning it off for everything. Let's say that we just wanted to have IK be retargeted for our feet. So we could go to the hands, each arm, turn off the ik, keep the fk, and then we could go back to global settings and turn on this stride warping. And now we can make kind of some adjustments. Maybe we want, oh, we need to actually turn on ik, but now we can make adjustments to our stance. We can offset things a little bit. We can do warping. Um, so th there's a few things we can do on the feet. And also it just has the tendency, again, if you leave these all at default to maybe fix foot sliding issues that you might have. Um, but if we go and we turn the hands back on again, there might be some adjustments that we can make here. Uh, you know, we could adjust the static offset, for example here, this is for the right arm. And you can see the, we're moving that hand around. 

So if we wanted to fix that tap of the head, for instance, tap, tap, right? That clipping, we could go to that point. We could do a static offset. And here you go. Of course this is, this is making this change for the entire animation. So it's gonna script the rest of our animation. We can't key frame in this change on this screen, but this is a way that you could get around maybe making a few changes. 

And there's also a bunch of other really amazing resources, uh, online and videos that go through some of the possibilities of this retargeting system. And we'll have those in the description below, uh, including videos from Unreal itself, which kind of walks through some of why you might want to use these IK goals and why you might not. 

Editing of the mocap data in UE's Sequencer

Now, however, let's actually go and take that again, that animation that we actually exported on our character and let's bring this into sequencer and make a few tweaks to the mocap, um, such as fixing this clipping right here. So I'm going to go and I'm gonna create a new level sequence right here. Add level sequence. I'll call this mocap editing. We will save. And then I'm actually gonna put my character into my scene. I'm gonna delete this player's start, and then I will go into sequencer. 

We're in the right level, right? And I'm going to drag in my character, although we could also, you know, add it in here, boo. And then we go to animation, select the animation that we have, and now we have this controllable animation in sequencer where we, we can make a few tweaks to this animation. So what we're about to go through is from a great tutorial from uh, Jonathan Winbush. We'll have that in the description down below. But let's say we want to fix this issue, right? I think we can get away with just changing the forearm so that it doesn't reach down too far. So the first thing we're going to do is right click on our character here. This, uh, this a m object and we're gonna go to edit with FK control rig and we're not going to do anything here. We're just gonna hit create. 

And as you can see, it kind of goes through and it's creating um, our skeletal mesh and it's kind of not lined up. Yeah, you can see it kind of lines up as soon as we move it. And now what we wanna do is we want to go to FK control rig and we're going to click on add section additive and this layer, and you can see we want this layer to be the entire length here of our, of our FK control rig. But this is a layer where we can do additive animation and it won't affect the original mocap, right? We don't, we want to add something on top of the mocap. We don't wanna replace it. So if we wanted to go and again fix this, this forearm here, you can see as soon as we select forearm, it opens it up in the additive layer, not in the original layer right here and I'm hiding it, but hopefully you can see it's on yeah, right forearm, right? So I can create a key frame right here. Boom. And then at this point, when the arm lands do, do right, right here, we can move this up and I can turn off snapping, we get a little bit more control and let's say we'll move it right here and then we'll add a key frame. 

And then right here I'm gonna add another key frame cuz I want this to end. And then I will go and I will zoom in here and I can grab the original key frame, copy that and it will go back to, you know, it'll reset to where it should be. Well actually, and now we can move these key frames around and we can change the timing. But you can see here, oh and look, we do need to change the timing cuz it does a pat before we even knew it. So now I can grab these key frames, move these over, 

There we go. And now we have done a little bit of mocap editing, um, you know, on the back end. And we could fix this clap the same way. We could go in and we could make, if we wanted to make overall changes for the entire animation, right? We could just go and, uh, maybe not that one, but let's grab an upper spine, just bend our character back that way at a key frame. Now this whole animation is, is like that, right? So this is a very powerful tool, but it's limited by the fact that we can only do this in an FK manner. In other words, what we would wanna do is be able to grab this wrist, just move it around the same way that we set up those IK handles and edit our mocap like that. But we can't do that in sequencer unless we have a control rig. 

So I put a really great link to Woody Devs video in the description below. He goes through kind of the basic idea of making a control rig, a custom control rig for a custom character, right? And that's what you would need to go in and do a more advanced editing. The nice thing about a metahuman is that it comes with that custom control rig already built for you. And so again, you can go check out our Metahuman series when that drops, then that's gonna cover retargeting for Metahumans and also some of the other cool editing things that you can do with that metahuman control rig that you have to build from scratch for, for these custom characters like a Mixamo character. If you have questions, put them in the comments below and we'll try to get to them. Again, we're gonna be updating this series pretty soon cuz there's a few things that we're, that are gonna be changing on the Rokoko side. Um, and so that'll be exciting. But for now, hopefully this workflow will get you through any retargeting and basic mocap tweaking that you might need to do. So thanks for watching. Check out our Instagram for new content, new tutorials happen all the time. And other than that, we'll see you in the next video. Bye everyone.