My Intro to Computational Media midterm project used face detection and tracking to manipulate an, admittedly very simple, 3D scene rendered in processing. I used the OpenCV library, and moved the point of view for the scene based on the movement of the subject's face.
It fails if the subject's face isn't fully visible, or if the face is too in profile, or if the face is a black guy wearing glasses. It could also use a lot more smoothing, as what I'm using is fairly rudimentary. And a Microsoft Kinect probably wouldn't hurt matters.