So this summer Søren V. Poulsen and I finished our bachelor thesis on “Human Interaction on Real-time Reconstructed Surfaces”.  The thesis is inspired by the work done by Microsoft Reseach on KinectFusion and expands upon the open source implementation  PCL KinFu. Allowing us to utilize the Kinect to perform foreground-background segmentation in dynamic scenes with a handheld camera. This allows us to detect intersections between foreground and background objects resulting in the creation of a finger painting application that enables users to paint on reconstructed surfaces by directly interacting with their real-life counterparts. As our method is suited for GPU implementation, real-time performance is achieved. We find that our solution supports multi-touch and is surprisingly accurate.

 

User testing the drawing application

We are currently in the process of cleaning up the implementation for an open source release. I will post a new blog post when we have finished the process.

Regarding the project report, feel free to contact me and i will send a PDF version.