- Open your Unity project.
- If you have previously installed the old Leap Motion V2 skeletal tracking asset, ensure it is COMPLETELY REMOVED. This includes the various Leap motion DLLs etc.
- Ensure that the Orion version of the Leap Motion tracking software is installed on your computer https://developer.leapmotion.com/orion
- Ensure the official Leap Motion Orion Unity assets package is added into your Unity project https://developer.leapmotion.com/unity
NOTE: The current version of this asset (1.21) is not compatible with the latest Leap Motion Orion Core Assets (4.1.0), please use the older 4.0.2 version of the core assets for the moment.
The pending version on the asset store (1.3) REQUIRES Leap Motion Orion Core Assets 4.1.0.
- Download and import the Avatar Hand Controller for Leap Motion from the asset store.
- That’s it! The package will be installed into a LeapAvatarHands folder. You can check out a demo scene in the Scenes sub-folder.
Configuring your own avatar
Adding this solution onto a player avatar in your Unity scene is quite easy, although does require a fair bit of configuration for the avatar.
- 1. Set the update mode on the Animator component of your character to “Animate Physics”.
- For desk mode: Add a child gameobject to your player gameobject (GameObject->Create Empty Child). The player gameobject must be a properly rigged character with 5 fingered and 3+ jointed hands. I did all my testing with characters generated by Mixamo Fuse, and then rigged with the Mixamo auto-rigger.
- For VR mode: Add a child gameobject to your camera gameobject (GameObject->Create Empty Child).
- For desk mode: Position this child gameobject where you want the virtual Leap controller to sit, generally speaking you will want this to be about 20-40cm out from the avatar’s belly button.
- For VR mode: Position this child gameobject 0.1 z-units forwards of the Camera gameobject, so it sits a little ways in front of the avatar’s face. Then ROTATE the object -90 on the x axis and 180 on the y axis, so that it is facing forwards in the same way that the leap motion itself would face forwards when mounted on a VR headset.
- Add the IKOrionLeapHandController script and the LeapProvider script to this empty gameobject.
- Go to the avatar’s left hand and add the RiggedHand script from the Leap Unity package to it.
- Set the handedness appropriately.
- Add the AutoRigHand component to the hand and use it to automatically set up the hand. You will still need to set finger pointing and palm facing parameters appropriate for your particular avatar, although Mixamo rigged avatars should work with the default settings.
- You will also need to set the finger pointing and palm facing vectors for each hand and finger. For reference with my Mixamo characters I had the finger pointing as 0,1,0, and the palm facing as 0,0,0. For each finger I had 0,1,0 and 0,0,0 except the thumb which was 0,1,0 and either -1,0,0 for the left hand or 1,0,0 for the right hand. However, these settings will depend on how the character is setup.
- Do steps 7-10 for the right hand also.
- Go back to the IKOrionLeapHandController gameobject and set the Avatar Left Hand and Avatar Right Hand fields to use the hands you just configured.
- Go to the root node of the avatar and add the IKActioner script, and tell it which gameobject you put the IKLeapController onto.
The IKActioner script passes the inverse kinematics pass from the animator component on the avatar up to the actual IKLeapHandController.
The IKLeapHandController is an extension of the provided LeapHandController, set up to work with avatar hands and inverse kinematics.
Using rigid hands
So that your hands can interact with physical objects in game, I have included a RigidIKRoundHand_L and RigidIKRoundHand_R prefab which will work with my IK solution.
For desk mode simply add these as children of the IKOrionLeapHandController gameobject and update the physics left hand and physics right hand references in the IKOrionLeapHandController script.
For VR mode place these two gameobjects in root transform (straight into the scene, not as a child node of another transform). Then update the left physics hand and right physics hand references in the IKOrionLeapHandController script to point to them.
NOTE: Physics hands are a little inaccurate still, but can be improved by fine adjustments to the offset and wrist offset parameters in the RigidHands and RiggedHands components. The avatar hands are limited by the range of movement of the avatar’s hands, whereas the physics hands are not. Consequently, it is difficult to get the physics hands and the real hands to sync up perfectly. The offset and palm width parameters of the RigidIKHands can be tweaked to improve accuracy, but it’s still not perfect, particularly when your arms are at full stretch.
Making your own Animator
It is important to note the two different layers I used in my example Animator, with IK Pass enabled on the base layer and blending set to override with a weight of 1.
Also, don’t forget to set the Animator to Animate Physics! This tweaks the order of execution, allowing the Leap data to override the animation data.
If you make your own Animator and it’s not working right, refer to the example to see how it’s meant to be done.
Using Hand Position Data at Runtime
Unity has a very particular order of execution. If you attempt to reference the hand transform in the Update method you will find that the data is incorrect, it will be based on where the currently playing character animation thinks the hand should be, rather than the leap motion hand data. The Animator IK pass happens after this.
As such, if you are trying to use the hand transform data at runtime, you should probably be doing that in LateUpdate(), after the Leap data and IK pass has occurred.