It is implemented with GLib and uses plain mathematics to detect the human skeleton and although it does not use any database, it was inspired by Andreas Baak’s paper: A Data-Driven Approach for Real-Time Full Body Pose Reconstruction from a Depth Camera.
Skeltrack devs have recorded very cool videos showing Gnome Shell and Linux games being controlled through gestures.
One or both hands are used control the mouse pointer, perform clicks, drag things around and simulates a pinch gesture which adjusts the zoom level. Hands can be also interpreted as if holding a steering wheel, making racing games so much fun.
[vimeo https://www.vimeo.com/38875885 w=480&h=380]
Skeltrack is device agnostic however you can use Kinect to start with. Full source code and documentation can be found here.