Leap Motion Avatar Hand Controller

Purchase on the Unity Asset Store

NOTE: The current version of this asset (1.21) is not compatible with the latest Leap Motion Orion Core Assets (4.1.0), please use the older 4.0.2 version of the core assets for the moment. The latest version (1.3) REQUIRES core assets 4.1.0, and should be available on the asset store in the next few days.

This is a simple extension to the Leap Motion set of assets which allows you to control the actual hands of your properly rigged humanoid character avatar.

This is a more immersive option than the default disembodied HandController provided by Leap Motion.

It is particularly useful when creating first person games, especially when working with the Oculus Rift or HTC Vive, or when playing in a networked multiplayer setting.

The arms of your avatar will be moved via Inverse Kinematics. As such, unfortunately this package is only suitable for users with Unity Pro.

Requires

Leap Motion controller
Unity Pro 5.3+ for Leap Motion Orion
Leap Motion Orion Core Unity assets (4.1.0)

For older Leap Motion V2 and Unity 4.6 support please contact me.

Installation Instructions for Orion (after1.2)

Installation Instructions for V2 (before 1.2)

Frequently Asked Questions

Q: The arms are doing unrealistic things, like clipping through the body. How do I fix this?
A: Try repositioning the IKLeapHandController GameObject relative to the avatar. It may simply be positioned too close/far/high/low. Look at how your arm is positioned/outstretched in the real world and compare it to how it looks in the gameworld… adjust the position of the Game Object until they’re quite similar, and you should get better IK results.

Q: How can I use this system in a networked multiplayer game?
A: I haven’t tried to do this myself, but it should be possible to use network interpolated transforms for each of the arm and finger joints. The problem will be optimisation, because that’s a lot of transform data to be sending on a routine basis. Sufficiently experienced programmers with a good understanding of game networking should be able to solve this problem quite comfortably, but I wouldn’t recommend it for beginner or even intermediate game developers.

Q: Something seems to be broken in the package, but the package isn’t updated yet. How can I get faster access to bug fixes?
A: The Unity Asset Store takes about 1 week to review any updates to assets and approve them for the store. Email me (vanmanivan AT hotmail DOT com) the invoice number from your Asset Store Invoice that you used to purchase the asset. I’ll email updates to you directly, as soon as they’re ready!

Q: How can I get more help?
A: Email vanmanivan AT hotmail DOT com and I will try my best to help you! Fair warning, I am a super busy guy, doing this in my spare time around full time work and a young child, but I usually respond to queries within the day.

Q: You’re my hero. How can I keep up to date with what you’re doing?
A: Follow me on Twitter (see the sidebar), and read my blog (this website)!

Q: It’s giving me all kinds of errors about RiggedHands and whatever else.
A: Make sure you first installed the Leap Motion Orion Core Assets (currently 4.1.0) and then installed this LeapAvatarHands package (currently 1.3). Basically you have to use the right version of the Leap Avatar Hands package matched with the right version of Leap Motion assets. You probably have the wrong version of one of them. This is unavoidable, the Leap Motion guys keep changing their package so I have to keep changing my package to support their changes. Usually this is minor, but sometimes it’s a showstopper and it takes me a little while to get the fix done. Contact me if you’re unsure what’s going on.

14 thoughts on “Leap Motion Avatar Hand Controller”

  1. Hi, will be buying your app tonight from the asset store, appreciate the script. I do have a problem, im trying to control the rest of the avatar with the Kinect, however when i use an avatar mask on the animation in your example the masked joints reset to the bind pose and leaves the kinect fighting the bind pose as it is reset to the bind pose every frame. do you know a way of applying the IK to the arms without affecting the rest of the bones?
    Thanks

    1. Sounds like an ordering/priority problem in terms of when each script/animation system does its thing. Just to be certain, the animator is set to Animate Physics mode? I have had some trouble with the animator masks myself… they don’t quite behave the way you expect them to. Tricky to help without extra information about what you’re doing and how you’re doing it though… might be easier to move to email?

  2. Thanks, I think its like you said I have to control the skeleton with LateUpdate() so that the bones are moved after the animator runs…
    but ive just noticed the rigid hand clone seems to stay in the same place it was instantiated and never moves until the hand is out of tracking range, then it resets.
    I think the issue is with the Kinect script im using, but if you have a Kinect and think you can help that would be great!

  3. Hi thanks, i thought i had replied. It is an issue with the kinect I think now. But your right the avatar mask doesn’t work how i would expect.

    I have just noticed that the rigidikhands instantiate at the first position the hand is in but then don’t move until they are destroyed when the hand is moved out of view…meaning i can’t interact..

    is this the same your end?

    Thanks

    1. Whoops. That’s a bug. Can confirm it’s happening on my demo scene also. Definitely did work in the past though! I’ll figure it out and submit an update. Should be today I expect.

      1. Found it. Fixing it properly now, but in the meantime you can temporarily fix the problem by going into RigidIKHand.cs and removing:
        || targetHand.GetHandOffset() == Vector3.zero

        By doing this you’ll get very intermittent NullReferenceExceptions out of the Leap Motion provided code, but there’s no harm in that apart from it being annoying. I’ll fix it properly soon.

        1. Problem is fixed. Uploading the fix now.

          Turns out you could get orphaned RigidIKHands if your framerate was fast enough, because it would make two gameobjects but only keep track of the last one and thus only destroy the last one when the hand went out of frame.

          Still not happy with how well the rigid hand lines up with the actual hand, but I’ll push through this update anyway, as it’s pretty important. Not sure how long Unity will take to approve the update once it’s uploaded, but hopefully not long.

          1. Great Thanks….Im now noticing if I keep my hand still It seems to flicker from two different positions…and when I build I get two arms, one in the right place the second bent out of position. and it seems to still flicker between the two…
            something you’ve noticed? Its quite bad now, might be something ive done though?

            thanks

          2. If the refresh rate of the Leap Motion gets too slow for whatever reason then you’ll have frames where the controller is saying there’s no hands to track, even though there is. Try reconnecting your Leap Motion to your computer (it seems like they don’t sync so well after a few days of continuous connection?), and also try adjusting the settings of the Leap Motion, and/or the lighting conditions of the room where you’re using it. That should deal with the flickering. If it’s still an issue I can also Lerp the IK so it doesn’t hard switch from 0% to 100% when there’s no longer a hand in frame… that should hugely soften the effect even when it does start to occur.

            I’ll investigate the weirdness in the built version ASAP to see if I can replicate it, but I’ve not experienced that problem.

          3. I’ve just made a build using the demo scene I provided with the package and don’t get the “second arm” issue (or the flickering issue, but as I said, that’s dependent on the condition of the leap motion controller itself). This ghost arm – is it cloned from the avatar, or is it a different arm altogether? (i.e. one from the Leap motion package?) And is it the full arm, or just the hand?

            Can you do a build with my demo scene to see if the issue is present for you once you remove all your own code from the equation? Easiest way to figure out if the problem is on my end or yours.

          4. Thought more about your flickering problem. I believe I’ve solved it comprehensively and categorically. Sending you through a test version via email now.

  4. Hi Ivan!

    I’d like to control the rest of my avatar with kinect. Got Leap and Oculus also. I’ve read all the comments before, but its a bit messy for me. Can You please create a guide, or please highlight the main steps I have to take to do this?
    Thanks in advance

    1. Hi. There’s actually a few of you trying to do this with the Kinect. I’ve got a prototype version that surrenders control of the arms, so you can leave that to the Kinect. However, I’ve still got to re-write the way the physics hands works to make it work properly this way. If you just want it to look right then I can send you the code no problems… but if you want the physics to work properly, then it’s not there yet.

Leave a Reply

Software/game developer, gamer, VR enthusiast.