That's an L of a demo...

1 post / 0 new
rip
rip's picture
Offline
Last seen: 5 days 22 hours ago
Joined: 04/06/2012
Posts: 321
That's an L of a demo...
LabVIEW,
Lego NXT (Mindstorms)
Leap Motion (motion sensor)
and two Laptops (one Windows, one MacOS)
 

YouTube video

what this "PoC" demo is:

1) Leap Motion SDK (motion sensor).  C++ app that bridges the raw Leap frames via their SDK to DDS on topic "Leap::Hand".  There is also "Leap::Pointable" and "Leap::Gesture", which aren't used here.  Publishes frames at raw frame rate.
 
2) Java app, subscribed to Leap::Hand, bridges the hand "Normal" (palm plane angles) X value for left/right, and "Stabilized" (palm position, smoothed out) Y value (near/far) for faster/slower.  Time-based filter (60Hz) bridging Leap::Hand to SteeringControl using
struct DuoMotorSteerNXT {
    string<128> id; //@key
    float steer; // -100 -- 100, -100 = pivot turn left, 100 = pivot turn right, 0 = full speed based on sign of power
    float power; // -100 -- 100, -100 = full power reverse, 100 = full power forward
};
Some smoothing/filtering done (for example, if delta thresholds not reached, instance is dropped) in the java app. 

3) LabVIEW + Lego NXT + RTI Connext 5.1 toolkits (well, 5.1 EAR).  LabVIEW graph subscribes to SteeringControl and further processes the steer/power fields in order to adjust them to the single "power" input supplied by the NXT motor control.  The algorithm keeps the opposite track at full power (P, as given by the power field), and adjusts the near track's speed downwards.  If Steer is 0, P is set to both tracks.  If Steer is -100, the Left track gets -P and the Right track P (ie, pivot turn in place).  At Steer 50, Left track is P and Right track is 0.
 
Track power levels are passed to the NXT brick via bluetooth.

To do, is specific gestures (the Leap SDK can detect a fist, for example.  I'm going to turn that into steer=0, power=0 [effectively, STOP])

And I have access to one more motor, which my sons want me to mount to a trebuchet, mounted on top of the robot...
 
I plan on writing this up as two longer, more grammatically pleasing posts for both the RTI blog and Leap's Developer Lab newsletter (by their request).  If you have questions and such, please post in reply, and I'll address them here and also keep them in mind for the longer piece.
 
Thanks!
rip
Organization: