News Articles

New Object Segmentation Algorithms

There are several tools available in PCL trunk that may be useful to the upcoming mobile manipulation challenge. Tools for segmentation of organized point cloud data (as from a Kinect or Xtion) have been added in pcl::OrganizedConnectedComponentSegmentation and pcl::OrganizedMultiPlaneSegmentation. These can be used for fast segmentation of all planes in the scene or euclidean clustering. It can also be extended with your own comparison functions for other segmentation tasks.

An example of how to use the plane segmentation is:

// Segment planes
pcl::OrganizedMultiPlaneSegmentation< PointT, pcl::Normal, pcl::Label > mps;
mps.setMinInliers (10000);
mps.setAngularThreshold (0.017453 * 2.0); // 2 degrees
mps.setDistanceThreshold (0.02); // 2cm
mps.setInputNormals (normal_cloud);
mps.setInputCloud (cloud);
std::vector< pcl::PlanarRegion< PointT > > regions;
mps.segmentAndRefine (regions);

for (size_t i = 0; i < regions.size (); i++)
  Eigen::Vector3f centroid = regions[i].getCentroid ();
  Eigen::Vector4f model = regions[i].getCoefficients ();
  pcl::PointCloud boundary_cloud;
  boundary_cloud.points = regions[i].getContour ();
  printf ("Centroid: (%f, %f, %f)\n  Coefficients:…

2nd ICRA Workshop on Semantic Perception, Mapping and Exploration (SPME 2012)

St. Paul, Minnesota, USA, May 14, 2011

Important Dates

Submissions Due:01 April 2012
Notification of Acceptance:16 April 2012
Final Papers Due:01 May 2012
Workshop at ICRA:14 May 2012

Motivation and Objectives

As robots and autonomous systems move away from laboratory setups towards complex real-world scenarios, both the perception capabilities of these systems and their abilities to acquire and model semantic information must become more powerful. The autonomous acquisition of information, the extraction of semantic models, and exploration strategies for deciding where and how to acquire the most relevant information pertinent to a specific semantic model are the research foci of an annual series of workshops at ICRA, called Semantic Perception, Mapping and Exploration (SPME).

Semantic perception for intelligent systems such as robots has seen a lot of progress recently, with many new and interesting techniques being developed in parallel by different research groups. Moreover, with the advent of inexpensive and accurate 3D imaging sensors, there has been an explosion of interest in 3D point clouds across a broad range of people.…

Tracking 3D objects with Point Cloud Library

Tracking 3D objects in continuous point cloud data sequences is an important research topic for mobile robots: it allows robots to monitor the environment and make decisions and adapt their motions according to the changes in the world. An example of such a typical application is visual servoing, with its key challenge to estimate the three dimensional pose of an object in real-time.

During his internship at Willow Garage, Ryohei Ueda from the JSK laboratory at University of Tokyo, worked on a novel 3D tracking library for the Point Cloud Library (PCL) project. The purpose of the library is to provide a comprehensive algorithmic base for the estimation of 3D object poses using Monte Carlo sampling techniques and for calculating the likelihood using combined weighted metrics for hyper-dimensional spaces including Cartesian data, colors, and surface normals. The libpcl_tracking library is optimized to perform computations in real-time, by employing multi CPU cores optimization, adaptive particle filtering (KLD sampling) and other modern techniques.

To find out more about Ryohei's…

Augmented Reality with Kinect Fusion

When we open sourced the Kinect Fusion algorithm a few weeks ago, we were not expecting so many people to join our community and start helping co-develop and improve the system. As we said in our initial announcement, the new Kinect Fusion implementation was still experimental, and it still has a lot of rough edges. However, that didn't scare off Weipeng Xu, one of our fearless new users. He saw the potential of this code and quickly integrated it with the XNA framework, creating a foundation to support the development of future augmented reality games and applications.

Augmented Reality is a challenging application domain, and there are a number of difficult tasks that must be performed in any AR system. Notably, for many applications, it's necessary to be able to construct a good 3D model of the environment and accurately track the movements of the camera in real time. Weifeng was able to use a Kinect 3D camera and PCL's experimental implementation of the KinectFusion algorithm to solve both of these problems. The result, as shown in the videos below, is a markerless AR system with the ability to handle occlusion between the real scene and virtual objects.

JediBot powered by PCL

JediBot is an autonomous robot designed to duel with humans in simulated sword fights. It was created by a group of four students in Stanford University's Experimental Robotics class, who combined a KUKA Light-Weight Robot arm for manipulation and a Microsoft Kinect for 3D sensing, along with custom software that leverages several open source libraries, including the Point Cloud Library (PCL) for 3D perception, the Reflexxes Motion Library for motion planning, and the Fast Research Interface Library for low-level control.

When facing off against a human combatant, the JediBot's software starts by analyzing the point cloud data acquired from its Microsoft Kinect 3D-camera. The Point Cloud Library is used to robustly find the pose of an opponent's sword in real time. The JediBot then quickly plans its response. Depending on the current strategy (attack or defense) an appropriate pose of the robot's sword is calculated. This pose is then sent to the Reflexxes Motion Library, which continuously plans a smooth sequence of arm motions. The resulting robot motions are directly sent to the Fast Research Interface Library, which provides low-level access to the…

Urban Robotics Extends Octree Format to PCL Open Source Community

Urban Robotics Inc., a leading provider of three-dimensional (3D) imaging sensors, software and algorithms, today announced it is making its highly scalable, spatially searchable and colorized 3D octree-based point cloud format available to the Point Cloud Library (PCL) community.

Willow Garage launched PCL in March 2011, to help accelerate 3D algorithmic work related to robotic applications. It is free for research and commercial use. The addition of Urban Robotics's software code to PCL lays the foundation for the creation of a standardized format for large-scale 3D applications. An example of such large-scale dataset representing a reconstruction of Mount St. Helens is shown in the video below.

Urban Robotics developed its octree-based format to efficiently store and manage point cloud data, and to address challenges related to the rapid processing of massive 3D images during daily operations.

"The main challenge with supported LAS and XYZ point cloud file formats is that…

An open source implementation of KinectFusion

We are happy to announce the development of a new open source implementation of KinectFusion, a simple system for 3D local mapping with an OpenNI-compatible camera. Below you can find the original SIGGRAPH video, together with a complete description of the algorithm presented in KinectFusion: Real-Time Dense Surface Mapping and Tracking.

For a quick comparison, here's a demonstration of our current implementation's capabilities:

The preliminary source code is currently available in our SVN repository's trunk in the CUDA/KinFu module. Since this code is still unreleased and under active development, we won't be able to provide support via our forums yet; however, advanced users are free to check out the code and give it a try. Be advised that this code relies heavily on the NVidia CUDA development libraries for GPU optimizations and will require a compatible GPU for best results.

Moving forward, we continue to refine and improve the system, and we are hoping to improve upon the original…

Large Scale 3D Point Cloud Mapping in PCL

Jochen Sprickerhof from the Knowledge-Based Systems Group of University of Osnabrück (Germany) visited Willow Garage this summer to do an internship on 3D registration and mapping.

Assembling massive datasets from a large number of individual point clouds is an important part of mobile robotics research. This allows robots to see beyond their immediate surroundings, localize in both 2D and 3D, and share large-scale maps built by other robots. One of the challenges here is how to efficiently estimate and correct the pose error in the trajectory of the robot, without sacrificing accuracy. For example, correcting high-dimensional registration data graphs that represent a large building or a city can take a very long time.

During his internship, Jochen ported his registration framework called ELCH (Explicit Loop Closing Heuristic) into the Point Cloud Library framework. ELCH tries to correct collected sensor data by finding loops in the robot trajectory, estimating the pose error the robot accumulated while driving along the loop, using point cloud registration,…

Scene Simulation for RGB-D Camera Localization

Maurice Fallon, Hordur Johannsson and John Leonard in MIT have been working on methods for 3D localization using RGB-D data from the Microsoft Kinect - KMCL.

First an entire building floor of the Stata Center was captured using a Kinect. The data was then used to build a simple plane-based 3D model using PCL (and sensor poses from Lidar-based SLAM) of about 1MB in size.

They then developed a robust algorithm for localization within the model in real-time using a particle filter.

The approach works by generating simulated range images using the 3D model for a virtual camera located the different particle poses. This approach correctly simulates the image formation model and allows for a disparity parameterized likelihood function. This allows the filter to correctly utilize the very noisy RGB-D points up to 20 meters away - data that has usually been discarded up to this.

Particles are propagated using the FOVIS…

Latest Posts

Latest Comments