[ Description | Data formats | Plane Segmentation Methods | Dataset ] The latest version is ... This page is under construction...
|
 |
Object detection, classification and manipulation are some of the capabilities required by autonomous robots. The main steps in object recognition and object classification are: segmentation, feature extraction, object representation and learning. To address the problem of learning object classification using multi-view range data, we used a relational approach (Farid et al., 2012a; Farid et al., 2012b; Farid et al., 2014a; Farid et al., 2014b). The first step of our object classification approach was to decompose a scene into shape primitives such as planes, followed by extracting a set of higher-level, relational features from the segmented regions. In this project, we compare our plane segmentation algorithm with state-of-the-art plane segmentation algorithms which are publicly available. We show that our segmentation outperforms and also produces better results especially for the robot action planning purposes.
A range image is similar to a 2D grey scale image. However, each pixel's value represents the distance from the sensor to the surface of an object in a scene from a particular viewpoint (Gächter, 2005; Gächter et al., 2006). This can be used to infer the shape of the object (Hegazy and Denzler, 2009).
The following pictures show a range image of a staircase with four steps, taken by a robot positioned in front of the staircase. In this grey scale image, the darker colour represents closer surfaces. For clarity, a colour-mapped version is also presented.
 |
 |
A range image can be transformed into a set of points in 3D space, producing a 3D point cloud (Rabbani Shah, 2006). The following pictures show the corresponding point cloud for the recently shown range images. Because using just one view, such as the front view is ambiguous, another view of the same point cloud is also shown. The point cloud has been segmented into planes (using the plane segmentation algorithm developed in this project) that are identified by unique colours. Some points in this point cloud are removed because they are far away, such as the points in the yellow region of the colour-mapped version, previously shown.
Front View  |
Top View  |
After this conversion, an ordered (or structured) point cloud is available. Due to the 2D nature of the range image, the neighbourhood of each point in the ordered point cloud can be easily and quickly found. This fact has a great impact on the preprocessing speed. However, it should not be considered a limitation. Many algorithms create a neighbourhood structure when the point cloud is unordered. Therefore, the input is not be limited by range images. Any point cloud representing a partial view of an object can be used similarly.
Several methods claim to provide good plane segmentation such as two algorithms provided by PCL (Rusu et al., 2011) using the RANSAC algorithm. We use these two state-of-the-art plane segmentation methods especially because they are publicly available:
We captured data during RoboCup Rescue competitions, as well as from rescue laboratories and other indoor locations. In this paper, we use a subset (45 images) of such data which we used for learning classes such as box (12 images), stairs (15 images) and pitch/roll ramp in a maze(18 images).
Since it is difficult to comprehend the range image, the corresponding colour (RGB) image of the scene will be shown in the rest of the project. For each class, different multi-view data are chosen. For example, the following figures show one view of some of the examples in this research which previously were used for training box, stairs and pitch/roll ramp classes respectively.
| Box class |  |
 |
 |
| Stairs class |  |
 |
 |
| Pitch/roll ramp class |  |
 |
 |
Before applying PCL plane segmentation algorithms, we need to prepare our data. Since our range images have 640 × 480 pixels, we sub-sample them to 160 × 120, while we converting them to point clouds.
| Stairs |  |
 |
 |
| Stairs |  |
 |
 |
| Stairs |  |
 |
 |
A set of scripts used for running plane segmentation methods:
A set of Cmake codes implemented to use plane segmentation methods:
Stefan Gächter. Results on range image segmentation for service robots. Technical report, Laboratoire de Systèmes Autonomes, Ecole Polytechnique Fédérale de Lausanne (EPFL), 2005. URL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.90.7553.
Stefan Gächter, Viet Nguyen, and Roland Siegwart. Results on range image segmentation for service robots. In Proceedings of IEEE International Conference on Computer Vision Systems (ICVS), pages 53–53, 2006. doi:10.1109/ICVS.2006.54.
Doaa Hegazy and Joachim Denzler. Generic 3D object recognition from time-of-flight images using boosted combined shape features. In Alpesh Ranchordas and Helder Araújo, editors, Proceedings of the Fourth International Conference on Computer Vision, Theory and Applications (VISAPP), volume 2, pages 321–326. INSTICC Press, 2009.
Tahir Rabbani Shah. Automatic reconstruction of industrial installations using point clouds and images. PhD thesis, Delft University of Technology, University of Engineering and Technology, Lahore, 2006. URL http://www.ncgeo.nl/phocadownload/62Rabbani.pdf.
Reza Farid and Claude Sammut, Plane-based object categorisation using relational learning: implementation details and extension of experiments, Technical Report UNSW-CSE-TR-201416, School of Computer Science and Engineering, The University of New South Wales. (PDF on [CSE-Repository] or [local])
Reza Farid and Claude Sammut, Plane-based object categorisation using relational learning, Machine Learning Journal, 94(1):3-23, 2014, doi: 10.1007/s10994-013-5352-9. ([Manuscript] Note: The final publication is available at link.springer.com)
Reza Farid and Claude Sammut, Plane-based object categorisation using relational learning. In Online Proceedings of the 22nd International Conference on Inductive Logic Programming (ILP2012), Dubrovnik, Croatia, 17-19 September 2012. ([PDF])
Reza Farid and Claude Sammut, A relational approach to plane-based object categorisation. Paper presented at Robotics Science and System (RSS) 2012 workshop on RGB-D: Advanced Reasoning with Depth Cameras, University of Sydney, 9-10 July 2012.(Paper[PDF], Poster[PDF])
R. B. Rusu and S. Cousins, “3D is here: Point Cloud Library (PCL),” in Proc. of ICRA2011, 2011, pp. 1–4.
Comments, suggestions or bugs can be sent to Reza Farid