Category Archives: release notes

Installation Instructions for Leap Avatar Hands for Leap Motion Orion

Installation

  1. Open your Unity project.
  2. If you have previously installed the old Leap Motion V2 skeletal tracking asset, ensure it is COMPLETELY REMOVED. This includes the various Leap motion DLLs etc.
  3. Ensure that the Orion version of the Leap Motion tracking software is installed on your computer https://developer.leapmotion.com/orion
  4. Ensure the official Leap Motion Orion Unity assets package is added into your Unity project https://developer.leapmotion.com/unity
  5. NOTE: The current version of this asset (1.21) is not compatible with the latest Leap Motion Orion Core Assets (4.1.0), please use the older 4.0.2 version of the core assets for the moment.
  6. The pending version on the asset store (1.3) REQUIRES Leap Motion Orion Core Assets 4.1.0.
  7. Download and import the Avatar Hand Controller for Leap Motion from the asset store.
  8. That’s it! The package will be installed into a LeapAvatarHands folder. You can check out a demo scene in the Scenes sub-folder.

Configuring your own avatar

Adding this solution onto a player avatar in your Unity scene is quite easy, although does require a fair bit of configuration for the avatar.

  1. 1. Set the update mode on the Animator component of your character to “Animate Physics”.
  2. For desk mode: Add a child gameobject to your player gameobject (GameObject->Create Empty Child). The player gameobject must be a properly rigged character with 5 fingered and 3+ jointed hands. I did all my testing with characters generated by Mixamo Fuse, and then rigged with the Mixamo auto-rigger.
  3. For VR mode: Add a child gameobject to your camera gameobject (GameObject->Create Empty Child).
  4. For desk mode: Position this child gameobject where you want the virtual Leap controller to sit, generally speaking you will want this to be about 20-40cm out from the avatar’s belly button.
  5. For VR mode: Position this child gameobject 0.1 z-units forwards of the Camera gameobject, so it sits a little ways in front of the avatar’s face. Then ROTATE the object -90 on the x axis and 180 on the y axis, so that it is facing forwards in the same way that the leap motion itself would face forwards when mounted on a VR headset.
  6. Add the IKOrionLeapHandController script and the LeapProvider script to this empty gameobject.
  7. Go to the avatar’s left hand and add the RiggedHand script from the Leap Unity package to it.
  8. Set the handedness appropriately.
  9. Add the AutoRigHand component to the hand and use it to automatically set up the hand. You will still need to set finger pointing and palm facing parameters appropriate for your particular avatar, although Mixamo rigged avatars should work with the default settings.
  10. You will also need to set the finger pointing and palm facing vectors for each hand and finger. For reference with my Mixamo characters I had the finger pointing as 0,1,0, and the palm facing as 0,0,0. For each finger I had 0,1,0 and 0,0,0 except the thumb which was 0,1,0 and either -1,0,0 for the left hand or 1,0,0 for the right hand. However, these settings will depend on how the character is setup.
  11. Do steps 7-10 for the right hand also.
  12. Go back to the IKOrionLeapHandController gameobject and set the Avatar Left Hand and Avatar Right Hand fields to use the hands you just configured.
  13. Go to the root node of the avatar and add the IKActioner script, and tell it which gameobject you put the IKLeapController onto.

The IKActioner script passes the inverse kinematics pass from the animator component on the avatar up to the actual IKLeapHandController.

The IKLeapHandController is an extension of the provided LeapHandController, set up to work with avatar hands and inverse kinematics.

Using rigid hands

So that your hands can interact with physical objects in game, I have included a RigidIKRoundHand_L and RigidIKRoundHand_R prefab which will work with my IK solution.
For desk mode simply add these as children of the IKOrionLeapHandController gameobject and update the physics left hand and physics right hand references in the IKOrionLeapHandController script.
For VR mode place these two gameobjects in root transform (straight into the scene, not as a child node of another transform). Then update the left physics hand and right physics hand references in the IKOrionLeapHandController script to point to them.
NOTE: Physics hands are a little inaccurate still, but can be improved by fine adjustments to the offset and wrist offset parameters in the RigidHands and RiggedHands components. The avatar hands are limited by the range of movement of the avatar’s hands, whereas the physics hands are not. Consequently, it is difficult to get the physics hands and the real hands to sync up perfectly. The offset and palm width parameters of the RigidIKHands can be tweaked to improve accuracy, but it’s still not perfect, particularly when your arms are at full stretch.

Making your own Animator

It is important to note the two different layers I used in my example Animator, with IK Pass enabled on the base layer and blending set to override with a weight of 1.
Also, don’t forget to set the Animator to Animate Physics! This tweaks the order of execution, allowing the Leap data to override the animation data.
If you make your own Animator and it’s not working right, refer to the example to see how it’s meant to be done.

Using Hand Position Data at Runtime

Unity has a very particular order of execution. If you attempt to reference the hand transform in the Update method you will find that the data is incorrect, it will be based on where the currently playing character animation thinks the hand should be, rather than the leap motion hand data. The Animator IK pass happens after this.
As such, if you are trying to use the hand transform data at runtime, you should probably be doing that in LateUpdate(), after the Leap data and IK pass has occurred.

Leap Motion Avatar Hand Installation Instructions for V2 Skeletal Tracking System

Installation

  1. Open your Unity project.
  2. First check that you have installed the official Leap motion V2 skeletal tracking asset, available free on the asset store. Optionally you may also want the Leap Motion OVR assets, available from the Leap motion developer website.
  3. Download and import the Avatar Hand Controller for Leap Motion from the asset store.
  4. That’s it! The package will be installed into a LeapAvatarHands folder. You can check out a demo scene in the Scenes sub-folder.
What you should see in your project assets folder if everything has been installed.
What you should see in your project assets folder if everything has been installed.

Configuring your own avatar

Adding this solution onto a player avatar in your Unity scene is quite easy, although does require a little bit of configuration for the avatar.

  1. Set the update mode on the Animator component of your character to “Animate Physics”.
  2. Add a child gameobject to your player gameobject. The player gameobject must be a properly rigged character with 5 fingered and 3+ jointed hands. I did all my testing with characters generated by Mixamo Fuse, and then rigged with the Mixamo auto-rigger.
  3. Position this child gameobject where you want the virtual Leap controller to sit, generally speaking you will want this to be about 10-30cm out from the avatar’s belly button.
  4. Add the IKLeapHandController script to this empty gameobject.
  5. Go to the avatar’s left hand and add the RiggedHand script from the Leap Unity package (v2.0+) to it.
  6. Go to each of the avatar’s fingers and add the RiggedFinger script to them.
  7. Set the finger type on each of the fingers.
  8. Set the bones (transforms) on each of the fingers, you should have 3, with the first one being the transform that represents the base of that finger, the second one being the knuckle, and the third one being the joint between the knuckle and fingernail.
  9. Go back to the avatar’s hand, and set each finger in the Fingers array: Thumb, Index, Middle, Ring, Pinky.
  10. You will also need to set the finger pointing and palm facing vectors for each hand and finger. For reference with my Mixamo characters I had the finger pointing as 0,1,0, and the palm facing as 0,0,0. For each finger I had 0,1,0 and 0,0,0 except the thumb which was 0,1,0 and either -1,0,0 for the left hand or 1,0,0 for the right hand. However, these settings will depend on how the character is setup.
  11. Do steps 4-9 for the right hand also.
  12. Go back to the IKLeapHandController gameobject and set the Avatar Left Hand and Avatar Right Hand fields to use the hands you just configured.
  13. Go to the root node of the avatar and add the IKActioner script, and tell it which gameobject you put the IKLeapController onto.
Leap Hands - Connected to your avatar.
Leap Hands – Connected to your avatar.

The IKActioner script passes the inverse kinematics pass from the animator component on the avatar up to the actual IKLeapHandController.

The IKLeapHandController is an extension of the provided LeapHandController, set up to work with avatar hands and inverse kinematics.

Using rigid hands

So that your hands can interact with physical objects in game, I have included a RigidIKHands prefab which will work with my IK solution. Simply add these into the physics hands slots on the IKLeapHandController component.

How to use rigid hands with the IK Leap Hand Controller.
How to use rigid hands with the IK Leap Hand Controller.

Release Notes – MVEv2

Screenshot

Birds are the major new addition.
Birds are the major new addition in v2.

Download

WIN MAC

New Features

Added flocks of pigeons and sparrows that fly around the joint and land on stuff.

Added a couple of planes that fly back and forth trailing a banner which displays the current song name.

Added the ability to teleport to waypoints, as well as automatic waypoint navigation for the player.

Added ability to switch between various scenes during runtime (although no additional scenes have been created yet).

Added ambient noises.

Fixes

Minor fixes to the weather systems – improved positioning of snow particles, rain particles, and snow dust.

Minor fixes to GUI elements.