Online Semantic Segmentation and Manipulation of Objects in Task Intelligence for Service Robots

Adrian Llopart, Ole Ravn, Nils Axel Andersen, Jong-Hwan Kim

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    Abstract

    Task Intelligence is the capacity of a robot to learn, reason and execute specific behaviours based on its environment. In this paper, the Task Intelligence problem formulated by the Robot Intelligence and Technology Laboratory at KAIST is researched further: specifically the proposed contribution is a brand new perceptual pipeline in which the recognition, detection, segmentation and grasping of objects is achieved assuming no prior knowledge of the environments arrangement nor the objects appearance. A Convolutional Neural Net (CNN) is used to detect, recognize and semantically label those objects that need to be interacted with. 3D point clouds, corresponding to the objects model, are extracted after several segmentation procedures and registered over time. Dimensional and positional information of the object is acquired. Additional grasping pose data is calculated. All of the collected knowledge is parsed so that the Task Intelligence system is able to deal with previously unknown objects in dynamic environments. This system is formed by an Episodic Memory (Deep-ART), an action sequence generator (FF-planner) and a trajectory warping module for pre-learnt behaviours. The proposed approach has been tested using the Webots simulator.
    Original languageEnglish
    Title of host publicationProceedings of 2018 15th International Conference on Control, Automation, Robotics and Vision
    PublisherIEEE
    Publication date2018
    Pages198-203
    ISBN (Print)9781538695821
    DOIs
    Publication statusPublished - 2018
    Event2018 15th International Conference on Control, Automation, Robotics and Vision - Marina Bay Sands Expo and Convention Centre, Singapore, Singapore
    Duration: 18 Nov 201821 Nov 2018
    https://icarcv.net/

    Conference

    Conference2018 15th International Conference on Control, Automation, Robotics and Vision
    LocationMarina Bay Sands Expo and Convention Centre
    Country/TerritorySingapore
    CitySingapore
    Period18/11/201821/11/2018
    Internet address

    Keywords

    • Task analysis
    • Three-dimensional displays
    • Encoding
    • Trajectory
    • Pipelines
    • End effectors
    • Task Intelligence
    • Semantic segmentation
    • CNN
    • Episodic Memory
    • Deep-ART
    • Humanoid robot

    Fingerprint

    Dive into the research topics of 'Online Semantic Segmentation and Manipulation of Objects in Task Intelligence for Service Robots'. Together they form a unique fingerprint.

    Cite this