Color-based region growing segmentation

In this tutorial we will learn how to use the color-based region growing algorithm implemented in the pcl::RegionGrowingRGB class. This algorithm is based on the same concept as the pcl::RegionGrowing that is described in the Region growing segmentation tutorial. If you are interested in the understanding of the base idea, please refer to the mentioned tutorial.

There are two main differences in the color-based algorithm. The first one is that it uses color instead of normals. The second is that it uses the merging algorithm for over- and under- segmentation control. Let’s take a look at how it is done. After the segmentation, an attempt for merging clusters with close colors is made. Two neighbouring clusters with a small difference between average color are merged together. Then the second merging step takes place. During this step every single cluster is verified by the number of points that it contains. If this number is less than the user-defined value than current cluster is merged with the closest neighbouring cluster.

The code

This tutorial requires colored cloud. You can use this one. Next what you need to do is to create a file region_growing_rgb_segmentation.cpp in any editor you prefer and copy the following code inside of it:

#include <iostream>
#include <thread>
#include <vector>

#include <pcl/point_types.h>
#include <pcl/io/pcd_io.h>
#include <pcl/search/search.h>
#include <pcl/search/kdtree.h>
#include <pcl/visualization/cloud_viewer.h>
#include <pcl/filters/filter_indices.h> // for pcl::removeNaNFromPointCloud
#include <pcl/segmentation/region_growing_rgb.h>

using namespace std::chrono_literals;

int
main ()
{
  pcl::search::Search <pcl::PointXYZRGB>::Ptr tree (new pcl::search::KdTree<pcl::PointXYZRGB>);

  pcl::PointCloud <pcl::PointXYZRGB>::Ptr cloud (new pcl::PointCloud <pcl::PointXYZRGB>);
  if ( pcl::io::loadPCDFile <pcl::PointXYZRGB> ("region_growing_rgb_tutorial.pcd", *cloud) == -1 )
  {
    std::cout << "Cloud reading failed." << std::endl;
    return (-1);
  }

  pcl::IndicesPtr indices (new std::vector <int>);
  pcl::removeNaNFromPointCloud (*cloud, *indices);

  pcl::RegionGrowingRGB<pcl::PointXYZRGB> reg;
  reg.setInputCloud (cloud);
  reg.setIndices (indices);
  reg.setSearchMethod (tree);
  reg.setDistanceThreshold (10);
  reg.setPointColorThreshold (6);
  reg.setRegionColorThreshold (5);
  reg.setMinClusterSize (600);

  std::vector <pcl::PointIndices> clusters;
  reg.extract (clusters);

  pcl::PointCloud <pcl::PointXYZRGB>::Ptr colored_cloud = reg.getColoredCloud ();
  pcl::visualization::CloudViewer viewer ("Cluster viewer");
  viewer.showCloud (colored_cloud);
  while (!viewer.wasStopped ())
  {
    std::this_thread::sleep_for(100us);
  }

  return (0);
}

The explanation

Now let’s study out what is the purpose of this code.

Let’s take a look at first lines that are of interest:

  pcl::PointCloud <pcl::PointXYZRGB>::Ptr cloud (new pcl::PointCloud <pcl::PointXYZRGB>);
  if ( pcl::io::loadPCDFile <pcl::PointXYZRGB> ("region_growing_rgb_tutorial.pcd", *cloud) == -1 )
  {
    std::cout << "Cloud reading failed." << std::endl;
    return (-1);
  }

They are simply loading the cloud from the .pcd file. Note that points must have the color.

  pcl::RegionGrowingRGB<pcl::PointXYZRGB> reg;

This line is responsible for pcl::RegionGrowingRGB instantiation. This class has two parameters:

• PointT - type of points to use(in the given example it is pcl::PointXYZRGB)

• NormalT - type of normals to use. Insofar as pcl::RegionGrowingRGB is derived from the pcl::RegionGrowing, it can use both tests at the same time: color test and normal test. The given example uses only the first one, therefore type of normals is not used.

  reg.setInputCloud (cloud);
  reg.setIndices (indices);
  reg.setSearchMethod (tree);

These lines provide the instance with the input cloud, indices and search method.

  reg.setDistanceThreshold (10);

Here the distance threshold is set. It is used to determine whether the point is neighbouring or not. If the point is located at a distance less than the given threshold, then it is considered to be neighbouring. It is used for clusters neighbours search.

  reg.setPointColorThreshold (6);

This line sets the color threshold. Just as angle threshold is used for testing points normals in pcl::RegionGrowing to determine if the point belongs to cluster, this value is used for testing points colors.

  reg.setRegionColorThreshold (5);

Here the color threshold for clusters is set. This value is similar to the previous, but is used when the merging process takes place.

  reg.setMinClusterSize (600);

This value is similar to that which was used in the Region growing segmentation tutorial. In addition to that, it is used for merging process mentioned in the beginning. If cluster has less points than was set through setMinClusterSize method, then it will be merged with the nearest neighbour.

  std::vector <pcl::PointIndices> clusters;
  reg.extract (clusters);

Here is the place where the algorithm is launched. It will return the array of clusters when the segmentation process will be over.

Remaining lines are responsible for the visualization of the colored cloud, where each cluster has its own color.

Compiling and running the program

Add the following lines to your CMakeLists.txt file:

cmake_minimum_required(VERSION 3.5 FATAL_ERROR)

project(region_growing_rgb_segmentation)

find_package(PCL 1.5 REQUIRED)

include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})

add_executable (region_growing_rgb_segmentation region_growing_rgb_segmentation.cpp)
target_link_libraries (region_growing_rgb_segmentation ${PCL_LIBRARIES})

After you have made the executable, you can run it. Simply do:

$ ./region_growing_rgb_segmentation

After the segmentation the cloud viewer window will be opened and you will see something similar to this image: