Autonomous 3d model generation of unknown objects for dual-manipulator humanoid robots

Adrian Llopart, Ole Ravn, Nils Axel Andersen, Jong-Hwan Kim

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    1 Downloads (Pure)


    This paper proposes a novel approach for the autonomous 3D model generation of unknown objects. A humanoid robot (or any setup with two manipulators) holds the object to model in one hand, views it from different perspectives and registers the depth information using a RGB-D sensor. The occlusions due to limited movement of the manipulator and the gripper itself covering the object are avoided by switching the object from one hand to the other. This allows for additional viewpoints leading to the registration of more depth information of the object. The contributions of this paper are as follows: 1. A humanoid robot that manipulates objects and obtains depth information 2. Tracing the hand movements with the robots head to be able to see the object at every moment 3. Filtering the point clouds to remove parts of the robot from them 4. Utilizing the Normal Iterative Closest Point algorithm (depth points, surface normals and curvature information) to register point clouds over time. This method will be applied to those pointclouds that include the robots gripper for optimal convergence; the resultant transform is then applied to those point clouds that describe only the segmented object 5. Changing the object from one hand to another 6. Merging the resulting object’s partial point clouds from both the left and right hands 7. Generating a mesh of the object based on the triangulation of final points of the object’s surface. No prior knowledge of the objects is necessary. No human intervention nor external help (i.e visual markers, turntables..) is required either.
    Original languageEnglish
    Title of host publicationRobot Intelligence Technology and Applications 5
    Publication date2018
    ISBN (Print)9783319784519
    Publication statusPublished - 2018
    Event5th International Conference on Robot Intelligence Technology and Applications - KAIST, Daejeon, Korea, Republic of
    Duration: 13 Dec 201715 Dec 2017
    Conference number: RiTA 2017


    Conference5th International Conference on Robot Intelligence Technology and Applications
    NumberRiTA 2017
    Country/TerritoryKorea, Republic of
    SeriesAdvances in Intelligent Systems and Computing


    • 3D model creation
    • Humanoid robot
    • Point cloud processing


    Dive into the research topics of 'Autonomous 3d model generation of unknown objects for dual-manipulator humanoid robots'. Together they form a unique fingerprint.

    Cite this