High-level class for template matching using the LINEMOD approach based on RGB and Depth data. More...

#include <pcl/recognition/linemod/line_rgbd.h>

Collaboration diagram for pcl::LineRGBD< PointXYZT, PointRGBT >:

Classes
struct	Detection
	A LineRGBD detection. More...

Public Member Functions
	LineRGBD ()
	Constructor. More...

virtual	~LineRGBD ()=default
	Destructor. More...

bool	loadTemplates (const std::string &file_name, std::size_t object_id=0)
	Loads templates from a LMT (LineMod Template) file. More...

bool	addTemplate (const SparseQuantizedMultiModTemplate &sqmmt, pcl::PointCloud< pcl::PointXYZRGBA > &cloud, std::size_t object_id=0)

void	setDetectionThreshold (float threshold)
	Sets the threshold for the detection responses. More...

void	setGradientMagnitudeThreshold (const float threshold)
	Sets the threshold on the magnitude of color gradients. More...

void	setIntersectionVolumeThreshold (const float threshold=1.0f)
	Sets the threshold for the decision whether two detections of the same template are merged or not. More...

void	setInputCloud (const typename pcl::PointCloud< PointXYZT >::ConstPtr &cloud)
	Sets the input cloud with xyz point coordinates. More...

void	setInputColors (const typename pcl::PointCloud< PointRGBT >::ConstPtr &cloud)
	Sets the input cloud with rgb values. More...

int	createAndAddTemplate (pcl::PointCloud< pcl::PointXYZRGBA > &cloud, const std::size_t object_id, const MaskMap &mask_xyz, const MaskMap &mask_rgb, const RegionXY &region)
	Creates a template from the specified data and adds it to the matching queue. More...

void	detect (std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > &detections)
	Applies the detection process and fills the supplied vector with the detection instances. More...

void	detectSemiScaleInvariant (std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > &detections, float min_scale=0.6944444f, float max_scale=1.44f, float scale_multiplier=1.2f)
	Applies the detection process in a semi-scale-invariant manner. More...

void	computeTransformedTemplatePoints (const std::size_t detection_id, pcl::PointCloud< pcl::PointXYZRGBA > &cloud)
	Computes and returns the point cloud of the specified detection. More...

std::vector< std::size_t >	findObjectPointIndices (const std::size_t detection_id)
	Finds the indices of the points in the input cloud which correspond to the specified detection. More...

Public Attributes
pcl::LINEMOD	linemod_
	LINEMOD instance. More...

pcl::ColorGradientModality< PointRGBT >	color_gradient_mod_
	Color gradient modality. More...

pcl::SurfaceNormalModality< PointXYZT >	surface_normal_mod_
	Surface normal modality. More...

pcl::PointCloud< PointXYZT >::ConstPtr	cloud_xyz_
	XYZ point cloud. More...

pcl::PointCloud< PointRGBT >::ConstPtr	cloud_rgb_
	RGB point cloud. More...

pcl::PointCloud< pcl::PointXYZRGBA >::CloudVectorType	template_point_clouds_
	Point clouds corresponding to the templates. More...

std::vector< pcl::BoundingBoxXYZ >	bounding_boxes_
	Bounding boxes corresponding to the templates. More...

std::vector< std::size_t >	object_ids_
	Object IDs corresponding to the templates. More...

std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection >	detections_
	Detections from last call of method detect (...). More...

Protected Member Functions
std::vector< std::size_t >	alignTemplatePoints (const std::size_t detection_id)
	Aligns the template points with the points found at the detection position of the specified detection. More...

void	refineDetectionsAlongDepth ()
	Refines the found detections along the depth. More...

void	applyProjectiveDepthICPOnDetections ()
	Applies projective ICP on detections to find their correct position in depth. More...

void	removeOverlappingDetections ()
	Checks for overlapping detections, removes them and keeps only the strongest one. More...

Static Protected Member Functions
static float	computeBoundingBoxIntersectionVolume (const BoundingBoxXYZ &box1, const BoundingBoxXYZ &box2)
	Computes the volume of the intersection between two bounding boxes. More...

Detailed Description

template<typename PointXYZT, typename PointRGBT = PointXYZT>
class pcl::LineRGBD< PointXYZT, PointRGBT >

High-level class for template matching using the LINEMOD approach based on RGB and Depth data.

Author: Stefan Holzer

Definition at line 72 of file line_rgbd.h.

Constructor & Destructor Documentation

◆ LineRGBD()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::LineRGBD< PointXYZT, PointRGBT >::LineRGBD ( )

inline

Constructor.

Definition at line 97 of file line_rgbd.h.

◆ ~LineRGBD()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

virtual pcl::LineRGBD< PointXYZT, PointRGBT >::~LineRGBD ( )

virtualdefault

Destructor.

Member Function Documentation

◆ addTemplate()

template<typename PointXYZT , typename PointRGBT >

bool pcl::LineRGBD< PointXYZT, PointRGBT >::addTemplate	(	const SparseQuantizedMultiModTemplate &	sqmmt,
		pcl::PointCloud< pcl::PointXYZRGBA > &	cloud,
		std::size_t	object_id = `0`
	)

Definition at line 314 of file line_rgbd.hpp.

References pcl::copyPointCloud(), pcl::BoundingBoxXYZ::depth, pcl::BoundingBoxXYZ::height, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

◆ alignTemplatePoints()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

std::vector<std::size_t> pcl::LineRGBD< PointXYZT, PointRGBT >::alignTemplatePoints ( const std::size_t detection_id )

inlineprotected

Aligns the template points with the points found at the detection position of the specified detection.

The detection ID refers to the last call of the method detect (...).

Parameters

[in] detection_id The ID of the detection (according to the last call of the method detect (...)).

Definition at line 247 of file line_rgbd.h.

◆ applyProjectiveDepthICPOnDetections()

template<typename PointXYZT , typename PointRGBT >

void pcl::LineRGBD< PointXYZT, PointRGBT >::applyProjectiveDepthICPOnDetections

protected

Applies projective ICP on detections to find their correct position in depth.

Definition at line 721 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, pcl::PointCloud< PointT >::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::PointCloud< PointT >::width, pcl::RegionXY::x, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

◆ computeBoundingBoxIntersectionVolume()

template<typename PointXYZT , typename PointRGBT >

float pcl::LineRGBD< PointXYZT, PointRGBT >::computeBoundingBoxIntersectionVolume	(	const BoundingBoxXYZ &	box1,
		const BoundingBoxXYZ &	box2
	)

staticprotected

Computes the volume of the intersection between two bounding boxes.

Parameters

[in]	box1	First bounding box.
[in]	box2	Second bounding box.

Definition at line 936 of file line_rgbd.hpp.

References pcl::BoundingBoxXYZ::depth, pcl::BoundingBoxXYZ::height, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

◆ computeTransformedTemplatePoints()

template<typename PointXYZT , typename PointRGBT >

void pcl::LineRGBD< PointXYZT, PointRGBT >::computeTransformedTemplatePoints	(	const std::size_t	detection_id,
		pcl::PointCloud< pcl::PointXYZRGBA > &	cloud
	)

Computes and returns the point cloud of the specified detection.

This is the template point cloud transformed to the detection coordinates. The detection ID refers to the last call of the method detect (...).

Parameters

[in]	detection_id	The ID of the detection (according to the last call of the method detect (...)).
[out]	cloud	The storage for the transformed points.

Definition at line 594 of file line_rgbd.hpp.

References pcl::PointCloud< PointT >::height, pcl::PointCloud< PointT >::resize(), pcl::PointCloud< PointT >::size(), pcl::PointCloud< PointT >::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

◆ createAndAddTemplate()

template<typename PointXYZT , typename PointRGBT >

int pcl::LineRGBD< PointXYZT, PointRGBT >::createAndAddTemplate	(	pcl::PointCloud< pcl::PointXYZRGBA > &	cloud,
		const std::size_t	object_id,
		const MaskMap &	mask_xyz,
		const MaskMap &	mask_rgb,
		const RegionXY &	region
	)

Creates a template from the specified data and adds it to the matching queue.

Parameters

	cloud
	object_id
[in]	mask_xyz	the mask that determine which parts of the xyz-modality are used for creating the template.
[in]	mask_rgb	the mask that determine which parts of the rgb-modality are used for creating the template.
[in]	region	the region which will be associated with the template (can be larger than the actual modality-maps).

Definition at line 221 of file line_rgbd.hpp.

References pcl::copyPointCloud(), pcl::BoundingBoxXYZ::depth, pcl::BoundingBoxXYZ::height, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

◆ detect()

template<typename PointXYZT , typename PointRGBT >

void pcl::LineRGBD< PointXYZT, PointRGBT >::detect ( std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > & detections )

Applies the detection process and fills the supplied vector with the detection instances.

Parameters

[out] detections The storage for the detection instances.

Definition at line 395 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::detection_id, pcl::RegionXY::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::object_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::SparseQuantizedMultiModTemplate::region, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::response, pcl::LINEMODDetection::score, pcl::LINEMODDetection::template_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::RegionXY::width, pcl::LINEMODDetection::x, pcl::BoundingBoxXYZ::x, pcl::RegionXY::x, pcl::LINEMODDetection::y, pcl::BoundingBoxXYZ::y, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

◆ detectSemiScaleInvariant()

template<typename PointXYZT , typename PointRGBT >

void pcl::LineRGBD< PointXYZT, PointRGBT >::detectSemiScaleInvariant	(	std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > &	detections,
		float	min_scale = `0.6944444f`,
		float	max_scale = `1.44f`,
		float	scale_multiplier = `1.2f`
	)

Applies the detection process in a semi-scale-invariant manner.

This is done by actually scaling the template to different sizes.

Definition at line 493 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::detection_id, pcl::RegionXY::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::object_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::SparseQuantizedMultiModTemplate::region, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::response, pcl::LINEMODDetection::scale, pcl::LINEMODDetection::score, pcl::LINEMODDetection::template_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::RegionXY::width, pcl::LINEMODDetection::x, pcl::BoundingBoxXYZ::x, pcl::RegionXY::x, pcl::LINEMODDetection::y, pcl::BoundingBoxXYZ::y, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

◆ findObjectPointIndices()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

std::vector<std::size_t> pcl::LineRGBD< PointXYZT, PointRGBT >::findObjectPointIndices ( const std::size_t detection_id )

inline

Finds the indices of the points in the input cloud which correspond to the specified detection.

The detection ID refers to the last call of the method detect (...).

Parameters

[in] detection_id The ID of the detection (according to the last call of the method detect (...)).

Definition at line 228 of file line_rgbd.h.

◆ loadTemplates()

template<typename PointXYZT , typename PointRGBT >

bool pcl::LineRGBD< PointXYZT, PointRGBT >::loadTemplates	(	const std::string &	file_name,
		std::size_t	object_id = `0`
	)

Loads templates from a LMT (LineMod Template) file.

Overrides old templates.

LineMod Template files are TAR files that store pairs of PCD datasets together with their LINEMOD signatures in SparseQuantizedMultiModTemplate format.

Parameters

[in]	file_name	The name of the file that stores the templates.
	object_id

Returns: true, if the operation was successful, false otherwise.

Definition at line 77 of file line_rgbd.hpp.

References pcl::BoundingBoxXYZ::depth, pcl::SparseQuantizedMultiModTemplate::deserialize(), pcl::io::TARHeader::file_name, pcl::io::TARHeader::getFileSize(), pcl::BoundingBoxXYZ::height, pcl::io::raw_close(), pcl::io::raw_lseek(), pcl::io::raw_open(), pcl::io::raw_read(), pcl::PCDReader::read(), pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

◆ refineDetectionsAlongDepth()

template<typename PointXYZT , typename PointRGBT >

void pcl::LineRGBD< PointXYZT, PointRGBT >::refineDetectionsAlongDepth

protected

Refines the found detections along the depth.

Definition at line 641 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, pcl::BoundingBoxXYZ::depth, pcl::RegionXY::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::RegionXY::width, pcl::RegionXY::x, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

◆ removeOverlappingDetections()

template<typename PointXYZT , typename PointRGBT >

void pcl::LineRGBD< PointXYZT, PointRGBT >::removeOverlappingDetections

protected

Checks for overlapping detections, removes them and keeps only the strongest one.

Definition at line 802 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, pcl::BoundingBoxXYZ::depth, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::detection_id, pcl::BoundingBoxXYZ::height, pcl::RegionXY::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::object_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::response, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::BoundingBoxXYZ::width, pcl::RegionXY::width, pcl::BoundingBoxXYZ::x, pcl::RegionXY::x, pcl::BoundingBoxXYZ::y, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

◆ setDetectionThreshold()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setDetectionThreshold ( float threshold )

inline

Sets the threshold for the detection responses.

Responses are between 0 and 1, where 1 is a best.

Parameters

[in] threshold The threshold used to decide where a template is detected.

Definition at line 130 of file line_rgbd.h.

◆ setGradientMagnitudeThreshold()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setGradientMagnitudeThreshold ( const float threshold )

inline

Sets the threshold on the magnitude of color gradients.

Color gradients with a magnitude below this threshold are not considered in the detection process.

Parameters

[in] threshold The threshold on the magnitude of color gradients.

Definition at line 140 of file line_rgbd.h.

◆ setInputCloud()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setInputCloud ( const typename pcl::PointCloud< PointXYZT >::ConstPtr & cloud )

inline

Sets the input cloud with xyz point coordinates.

The cloud has to be organized.

Parameters

[in] cloud The input cloud with xyz point coordinates.

Definition at line 162 of file line_rgbd.h.

◆ setInputColors()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setInputColors ( const typename pcl::PointCloud< PointRGBT >::ConstPtr & cloud )

inline

Sets the input cloud with rgb values.

The cloud has to be organized.

Parameters

[in] cloud The input cloud with rgb values.

Definition at line 174 of file line_rgbd.h.

◆ setIntersectionVolumeThreshold()

template<typename PointXYZT , typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setIntersectionVolumeThreshold ( const float threshold = 1.0f )

inline

Sets the threshold for the decision whether two detections of the same template are merged or not.

If ratio between the intersection of the bounding boxes of two detections and the original bounding boxes is larger than the specified threshold then they are merged. If detection A overlaps with detection B and B with C than A, B, and C are merged. Threshold has to be between 0 and 1.

Parameters

[in] threshold The threshold on the ratio between the intersection bounding box and the original bounding box.

Definition at line 153 of file line_rgbd.h.

Member Data Documentation

◆ bounding_boxes_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

std::vector<pcl::BoundingBoxXYZ> pcl::LineRGBD< PointXYZT, PointRGBT >::bounding_boxes_

Bounding boxes corresponding to the templates.

Definition at line 300 of file line_rgbd.h.

◆ cloud_rgb_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::PointCloud<PointRGBT>::ConstPtr pcl::LineRGBD< PointXYZT, PointRGBT >::cloud_rgb_

RGB point cloud.

Definition at line 295 of file line_rgbd.h.

◆ cloud_xyz_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::PointCloud<PointXYZT>::ConstPtr pcl::LineRGBD< PointXYZT, PointRGBT >::cloud_xyz_

XYZ point cloud.

Definition at line 293 of file line_rgbd.h.

◆ color_gradient_mod_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::ColorGradientModality<PointRGBT> pcl::LineRGBD< PointXYZT, PointRGBT >::color_gradient_mod_

Color gradient modality.

Definition at line 288 of file line_rgbd.h.

◆ detections_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

std::vector<typename pcl::LineRGBD<PointXYZT, PointRGBT>::Detection> pcl::LineRGBD< PointXYZT, PointRGBT >::detections_

Detections from last call of method detect (...).

Definition at line 305 of file line_rgbd.h.

◆ linemod_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::LINEMOD pcl::LineRGBD< PointXYZT, PointRGBT >::linemod_

LINEMOD instance.

Definition at line 286 of file line_rgbd.h.

◆ object_ids_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

std::vector<std::size_t> pcl::LineRGBD< PointXYZT, PointRGBT >::object_ids_

Object IDs corresponding to the templates.

Definition at line 302 of file line_rgbd.h.

◆ surface_normal_mod_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::SurfaceNormalModality<PointXYZT> pcl::LineRGBD< PointXYZT, PointRGBT >::surface_normal_mod_

Surface normal modality.

Definition at line 290 of file line_rgbd.h.

◆ template_point_clouds_

template<typename PointXYZT , typename PointRGBT = PointXYZT>

pcl::PointCloud<pcl::PointXYZRGBA>::CloudVectorType pcl::LineRGBD< PointXYZT, PointRGBT >::template_point_clouds_

Point clouds corresponding to the templates.

Definition at line 298 of file line_rgbd.h.

The documentation for this class was generated from the following files:

pcl/recognition/linemod/line_rgbd.h
pcl/recognition/impl/linemod/line_rgbd.hpp

Classes

Public Member Functions

Public Attributes

Protected Member Functions

Static Protected Member Functions

Detailed Description

template<typename PointXYZT, typename PointRGBT = PointXYZT> class pcl::LineRGBD< PointXYZT, PointRGBT >

Constructor & Destructor Documentation

◆ LineRGBD()

◆ ~LineRGBD()

Member Function Documentation

◆ addTemplate()

◆ alignTemplatePoints()

◆ applyProjectiveDepthICPOnDetections()

◆ computeBoundingBoxIntersectionVolume()

◆ computeTransformedTemplatePoints()

◆ createAndAddTemplate()

◆ detect()

◆ detectSemiScaleInvariant()

◆ findObjectPointIndices()

◆ loadTemplates()

◆ refineDetectionsAlongDepth()

◆ removeOverlappingDetections()

◆ setDetectionThreshold()

◆ setGradientMagnitudeThreshold()

◆ setInputCloud()

◆ setInputColors()

◆ setIntersectionVolumeThreshold()

Member Data Documentation

◆ bounding_boxes_

◆ cloud_rgb_

◆ cloud_xyz_

◆ color_gradient_mod_

◆ detections_

◆ linemod_

◆ object_ids_

◆ surface_normal_mod_

◆ template_point_clouds_

template<typename PointXYZT, typename PointRGBT = PointXYZT>
class pcl::LineRGBD< PointXYZT, PointRGBT >