Saturday, March 30, 2013

Developing for the Leap Motion controller in Scala


I just got a hold of a Leap Motion controller today and I have to admit it's a neat little device with a lot of potential. The first thing I did was figure out how to interface with the device using Scala. Here are some quick pointers on how to get setup.

Before reading further, entire process I outline below is summed up in the following github repo. You can clone it and run ./install then ./run to produce the following results:


Instructions

1- Download the SDK here.

2- Assuming you have maven installed, create a project using the following command
mvn org.apache.maven.plugins:maven-archetype-plugin:2.2:generate \
-DarchetypeGroupId=org.scala-tools.archetypes \
-DarchetypeArtifactId=scala-archetype-simple \
-DarchetypeVersion=1.3 -DgroupId=leap-scala \
-DartifactId=leap-scala -Dversion=1.0.0 -DinteractiveMode=false

3- Install the leap motion sdk jar in your local maven repository (found in the sdk package from step #1 under /LeapSDK/lib/LeapJava.jar). Use the following command:
mvn org.apache.maven.plugins:maven-install-plugin:2.3.1:install-file \
    -Dfile=PATH_TO_JAR \
    -DgroupId=com.leapmotion.leap -DartifactId=leapMotion \
    -Dversion=1.0.0 -Dpackaging=jar

4- Add a reference to this jar file in the leap-scala/pom.xml that was created in step #2.
    
    <dependencies>
        <dependency>
            <groupid>org.scala-lang</groupid>
            <artifactid>scala-library</artifactid>
            <version>${scala.version}</version>
        </dependency>
        <dependency>
            <groupid>com.leapmotion.leap</groupid>
            <artifactid>leapMotion</artifactid>
            <version>1.0.0</version>
        </dependency>
    </dependencies>

5- Write some code for capturing events from the device. Here are some snippets from the leap provided samples I ported to scala:

package com.phyous.leap
import java.io.IOException
import com.leapmotion.leap._
object App {
def main(args: Array[String]) {
val listener = new SampleListener
val controller = new Controller
controller.addListener(listener)
println("Press Enter to quit...")
try {
System.in.read
}
catch {
case e: IOException => {
e.printStackTrace
}
}
controller.removeListener(listener)
}
}
view raw App.scala hosted with ❤ by GitHub
package com.phyous.leap
import java.io.IOException
import java.lang.Math
import com.leapmotion.leap._
import com.leapmotion.leap.Gesture.State
class SampleListener extends Listener {
override def onInit(controller: Controller) {
println("Initialized")
}
override def onConnect(controller: Controller) {
println("Connected")
controller.enableGesture(Gesture.Type.TYPE_SWIPE)
controller.enableGesture(Gesture.Type.TYPE_CIRCLE)
controller.enableGesture(Gesture.Type.TYPE_SCREEN_TAP)
controller.enableGesture(Gesture.Type.TYPE_KEY_TAP)
}
override def onDisconnect(controller: Controller) {
println("Disconnected")
}
override def onExit(controller: Controller) {
println("Exited")
}
override def onFrame(controller: Controller) {
val frame: Frame = controller.frame
println("Frame id: " + frame.id + ", timestamp: " + frame.timestamp + ", hands: " + frame.hands.count + ", fingers: " + frame.fingers.count + ", tools: " + frame.tools.count + ", gestures " + frame.gestures.count)
if (!frame.hands.empty) {
val hand: Hand = frame.hands.get(0)
val fingers: FingerList = hand.fingers
if (!fingers.empty) {
var avgPos: Vector = Vector.zero
import scala.collection.JavaConversions._
for (finger <- fingers) {
avgPos = avgPos.plus(finger.tipPosition)
}
avgPos = avgPos.divide(fingers.count)
println("Hand has " + fingers.count + " fingers, average finger tip position: " + avgPos)
}
println("Hand sphere radius: " + hand.sphereRadius + " mm, palm position: " + hand.palmPosition)
val normal: Vector = hand.palmNormal
val direction: Vector = hand.direction
println("Hand pitch: " + Math.toDegrees(direction.pitch) + " degrees, " + "roll: " + Math.toDegrees(normal.roll) + " degrees, " + "yaw: " + Math.toDegrees(direction.yaw) + " degrees")
}
val gestures: GestureList = frame.gestures()
var i: Int = 0
while (i < gestures.count) {
val gesture: Gesture = gestures.get(i)
gesture.`type` match {
case Gesture.Type.TYPE_CIRCLE =>
val circle = new CircleGesture(gesture)
var clockwiseness: String = null
if (circle.pointable.direction.angleTo(circle.normal) <= Math.PI / 4) {
clockwiseness = "clockwise"
}
else {
clockwiseness = "counterclockwise"
}
var sweptAngle: Double = 0
if (circle.state ne State.STATE_START) {
val previousUpdate = new CircleGesture(controller.frame(1).gesture(circle.id))
sweptAngle = (circle.progress - previousUpdate.progress) * 2 * Math.PI
}
println("Circle id: " + circle.id + ", " + circle.state + ", progress: " + circle.progress + ", radius: " + circle.radius + ", angle: " + Math.toDegrees(sweptAngle) + ", " + clockwiseness)
case Gesture.Type.TYPE_SWIPE =>
val swipe = new SwipeGesture(gesture)
println("Swipe id: " + swipe.id + ", " + swipe.state + ", position: " + swipe.position + ", direction: " + swipe.direction + ", speed: " + swipe.speed)
case Gesture.Type.TYPE_SCREEN_TAP =>
val screenTap = new ScreenTapGesture(gesture)
println("Screen Tap id: " + screenTap.id + ", " + screenTap.state + ", position: " + screenTap.position + ", direction: " + screenTap.direction)
case Gesture.Type.TYPE_KEY_TAP =>
val keyTap = new KeyTapGesture(gesture)
println("Key Tap id: " + keyTap.id + ", " + keyTap.state + ", position: " + keyTap.position + ", direction: " + keyTap.direction)
case _ =>
println("Unknown gesture type.")
}
}
({
i += 1; i - 1
})
if (!frame.hands.empty || !gestures.empty) {
println
}
}
}

6- Once you have some code, you can bundle it along with the leap motion library in a "fat" jar using the maven-assembly-plugin. Check out a sample pom.xml that does this here. You can then bundle the app using the following maven command
 
mvn install

This will produce a jar file called "leap-scala-1.0.0-jar-with-dependencies.jar" under the target folder.

7- Once your code is compiling, it's time to run. You'll need to point the jvm runtime to the native leap motion libraries for interfacing with your OS. On OSX, this is a file called libLeapJava.dylib found in the same directory of the SDK as the jar file. Here is how you would run your app once it's compiled:
 
java -Djava.library.path=".:./libs" -jar target/leap-scala-1.0.0-jar-with-dependencies.jar

Using the code above, you should be able to see input directly form the device on your console.

These instructions were generated while developing on OSX. Things might need a little bit of tweaking if running on other platforms.

3 comments:

  1. I read the post about Developing for the Leap Motion controller in Scala. I was searching about development of Knowledge Games so that i can provide more knowledge to peoples by games.
    Thanks for sharing this.

    ReplyDelete
  2. Hello admin,
    I watch the both videos and also read the code you provide and i copy the codes which you provide then use it later.
    Thanks for providing this.
    web design services

    ReplyDelete
  3. I read your article "Developing for the Leap Motion controller in Scala" and also the coding which is essential for developing this app. Thank you so much for sharing that information with us.
    123movies

    ReplyDelete