Generalized framework for the parallel semantic segmentation of multiple objects and posterior manipulation

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedings – Annual report year: 2018Researchpeer-review

View graph of relations

The end-to-end approach presented in this paper deals with the recognition, detection, segmentation and grasping of objects, assuming no prior knowledge of the environment nor objects. The proposed pipeline is as follows: 1) Usage of a trained Convolutional Neural Net (CNN) that recognizes up to 80 different classes of objects in real time and generates bounding boxes around them. 2) An algorithm to derive in parallel the pointclouds of said regions of interest (ROI). 3) Eight different segmentation methods to remove background data and noise from the pointclouds and obtain a precise result of the semantically segmented objects. 4) Registration of the object's pointclouds over time to generate the best possible model. 5) Utilization of an algorithm to detect an array of grasping positions and orientations based mainly on the geometry of the object's model. 6) Implementation of the system on the humanoid robot MyBot, developed in the RIT Lab at KAIST. 7) An algorithm to find the bounding box of the object's model in 3D to then create a collision object and add it to the octomap. The collision checking between robot's hand and the object is removed to allow grasping using the MoveIt libraries. 8) Selection of the best grasping pose for a certain object, plus execution of the grasping movement. 9) Retrieval of the object and moving it to a desired final position.
Original languageEnglish
Title of host publicationProceedings of 2017 IEEE International Conference on Robotics and Biomimetics
Publication date2017
ISBN (Print)9781538637425
Publication statusPublished - 2017
Event2017 IEEE International Conference on Robotics and Biomimetics - The Parisian Macao, Macao, China
Duration: 5 Dec 20178 Dec 2017


Conference2017 IEEE International Conference on Robotics and Biomimetics
LocationThe Parisian Macao
CitationsWeb of Science® Times Cited: No match on DOI

    Research areas

  • Object detection, Convolutional neural nets, Semantic segmentation, Pointcloud processing, Grasping, Humanoid robot

ID: 146507043