News Articles

Augmented Reality with Kinect Fusion

When we open sourced the Kinect Fusion algorithm a few weeks ago, we were not expecting so many people to join our community and start helping co-develop and improve the system. As we said in our initial announcement, the new Kinect Fusion implementation was still experimental, and it still has a lot of rough edges. However, that didn't scare off Weipeng Xu, one of our fearless new users. He saw the potential of this code and quickly integrated it with the XNA framework, creating a foundation to support the development of future augmented reality games and applications.

Augmented Reality is a challenging application domain, and there are a number of difficult tasks that must be performed in any AR system. Notably, for many applications, it's necessary to be able to construct a good 3D model of the environment and accurately track the movements of the camera in real time. Weifeng was able to use a Kinect 3D camera and PCL's experimental implementation of the KinectFusion algorithm to solve both of these problems. The result, as shown in the videos below, is a markerless AR system with the ability to handle occlusion between the real scene and virtual objects.

An open source implementation of KinectFusion

We are happy to announce the development of a new open source implementation of KinectFusion, a simple system for 3D local mapping with an OpenNI-compatible camera. Below you can find the original SIGGRAPH video, together with a complete description of the algorithm presented in KinectFusion: Real-Time Dense Surface Mapping and Tracking.

For a quick comparison, here's a demonstration of our current implementation's capabilities:

The preliminary source code is currently available in our SVN repository's trunk in the CUDA/KinFu module. Since this code is still unreleased and under active development, we won't be able to provide support via our forums yet; however, advanced users are free to check out the code and give it a try. Be advised that this code relies heavily on the NVidia CUDA development libraries for GPU optimizations and will require a compatible GPU for best results.

Moving forward, we continue to refine and improve the system, and we are hoping to improve upon the original…

Large Scale 3D Point Cloud Mapping in PCL

Jochen Sprickerhof from the Knowledge-Based Systems Group of University of Osnabrück (Germany) visited Willow Garage this summer to do an internship on 3D registration and mapping.

Assembling massive datasets from a large number of individual point clouds is an important part of mobile robotics research. This allows robots to see beyond their immediate surroundings, localize in both 2D and 3D, and share large-scale maps built by other robots. One of the challenges here is how to efficiently estimate and correct the pose error in the trajectory of the robot, without sacrificing accuracy. For example, correcting high-dimensional registration data graphs that represent a large building or a city can take a very long time.

During his internship, Jochen ported his registration framework called ELCH (Explicit Loop Closing Heuristic) into the Point Cloud Library framework. ELCH tries to correct collected sensor data by finding loops in the robot trajectory, estimating the pose error the robot accumulated while driving along the loop, using point cloud registration,…

Modular components for point cloud registration

Dirk Holz from the University of Bonn in Germany spent his internship at Willow Garage working on the Point Cloud Library (PCL). He implemented a set of modular components for registering point clouds to create three-dimensional models of objects and environments. The work on the registration part of PCL is a joint effort with other researchers from the PCL community and an ongoing project. Please watch the video above for first demonstrations of what is already achievable or read the slides below (download pdf) for more technical details. The software is available as open source part of the PCL project.

Latest Posts

Latest Comments