Material-Based Segmentation of Objects

Jonathan Dyssel Stets, Rasmus Ahrenkiel Lyngby*, Jeppe Revall Frisvad, Anders Bjorholm Dahl

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

39 Downloads (Pure)

Abstract

We present a data-driven proof of concept method for image-based semantic segmentation of objects based on their materials. We target materials with complex radiometric appearances, such as reflective and refractive materials, as their detection is particularly challenging in many modern vision systems. Specifically, we select glass, chrome, plastic, and ceramics as these often appear in real-world settings. A large dataset of synthetic images is generated with the Blender 3D creation suite and the Cycles renderer. We use this data to fine-tune the pre-trained DeepLabv3+ semantic segmentation convolutional neural network. The network performs well on rendered test data and, although trained with rendered images only, the network generalizes so that the four selected materials can be segmented from real photos.
Original languageEnglish
Title of host publicationScandinavian Conference on Image Analysis
PublisherSpringer
Publication date2019
Pages152-163
ISBN (Print)978-3-030-20204-0
DOIs
Publication statusPublished - 2019
EventScandinavian Conference on Image Analysis - Norrköpings Visualisering Center, Norrköping, Sweden
Duration: 11 Jun 201913 Jun 2019
http://ssba.org.se/scia2019/

Conference

ConferenceScandinavian Conference on Image Analysis
LocationNorrköpings Visualisering Center
CountrySweden
CityNorrköping
Period11/06/201913/06/2019
Internet address
SeriesLecture Notes in Computer Science
Volume11482
ISSN0302-9743

Keywords

  • Semantic segmentation
  • Deep learning
  • Synthetic data

Cite this

Stets, J. D., Lyngby, R. A., Frisvad, J. R., & Dahl, A. B. (2019). Material-Based Segmentation of Objects. In Scandinavian Conference on Image Analysis (pp. 152-163). Springer. Lecture Notes in Computer Science, Vol.. 11482 https://doi.org/10.1007/978-3-030-20205-7_13
Stets, Jonathan Dyssel ; Lyngby, Rasmus Ahrenkiel ; Frisvad, Jeppe Revall ; Dahl, Anders Bjorholm. / Material-Based Segmentation of Objects. Scandinavian Conference on Image Analysis. Springer, 2019. pp. 152-163 (Lecture Notes in Computer Science, Vol. 11482).
@inproceedings{960e43a7f0764815bbb40c020b8f1df8,
title = "Material-Based Segmentation of Objects",
abstract = "We present a data-driven proof of concept method for image-based semantic segmentation of objects based on their materials. We target materials with complex radiometric appearances, such as reflective and refractive materials, as their detection is particularly challenging in many modern vision systems. Specifically, we select glass, chrome, plastic, and ceramics as these often appear in real-world settings. A large dataset of synthetic images is generated with the Blender 3D creation suite and the Cycles renderer. We use this data to fine-tune the pre-trained DeepLabv3+ semantic segmentation convolutional neural network. The network performs well on rendered test data and, although trained with rendered images only, the network generalizes so that the four selected materials can be segmented from real photos.",
keywords = "Semantic segmentation, Deep learning, Synthetic data",
author = "Stets, {Jonathan Dyssel} and Lyngby, {Rasmus Ahrenkiel} and Frisvad, {Jeppe Revall} and Dahl, {Anders Bjorholm}",
year = "2019",
doi = "10.1007/978-3-030-20205-7_13",
language = "English",
isbn = "978-3-030-20204-0",
pages = "152--163",
booktitle = "Scandinavian Conference on Image Analysis",
publisher = "Springer",

}

Stets, JD, Lyngby, RA, Frisvad, JR & Dahl, AB 2019, Material-Based Segmentation of Objects. in Scandinavian Conference on Image Analysis. Springer, Lecture Notes in Computer Science, vol. 11482, pp. 152-163, Scandinavian Conference on Image Analysis, Norrköping, Sweden, 11/06/2019. https://doi.org/10.1007/978-3-030-20205-7_13

Material-Based Segmentation of Objects. / Stets, Jonathan Dyssel; Lyngby, Rasmus Ahrenkiel; Frisvad, Jeppe Revall; Dahl, Anders Bjorholm.

Scandinavian Conference on Image Analysis. Springer, 2019. p. 152-163 (Lecture Notes in Computer Science, Vol. 11482).

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

TY - GEN

T1 - Material-Based Segmentation of Objects

AU - Stets, Jonathan Dyssel

AU - Lyngby, Rasmus Ahrenkiel

AU - Frisvad, Jeppe Revall

AU - Dahl, Anders Bjorholm

PY - 2019

Y1 - 2019

N2 - We present a data-driven proof of concept method for image-based semantic segmentation of objects based on their materials. We target materials with complex radiometric appearances, such as reflective and refractive materials, as their detection is particularly challenging in many modern vision systems. Specifically, we select glass, chrome, plastic, and ceramics as these often appear in real-world settings. A large dataset of synthetic images is generated with the Blender 3D creation suite and the Cycles renderer. We use this data to fine-tune the pre-trained DeepLabv3+ semantic segmentation convolutional neural network. The network performs well on rendered test data and, although trained with rendered images only, the network generalizes so that the four selected materials can be segmented from real photos.

AB - We present a data-driven proof of concept method for image-based semantic segmentation of objects based on their materials. We target materials with complex radiometric appearances, such as reflective and refractive materials, as their detection is particularly challenging in many modern vision systems. Specifically, we select glass, chrome, plastic, and ceramics as these often appear in real-world settings. A large dataset of synthetic images is generated with the Blender 3D creation suite and the Cycles renderer. We use this data to fine-tune the pre-trained DeepLabv3+ semantic segmentation convolutional neural network. The network performs well on rendered test data and, although trained with rendered images only, the network generalizes so that the four selected materials can be segmented from real photos.

KW - Semantic segmentation

KW - Deep learning

KW - Synthetic data

U2 - 10.1007/978-3-030-20205-7_13

DO - 10.1007/978-3-030-20205-7_13

M3 - Article in proceedings

SN - 978-3-030-20204-0

SP - 152

EP - 163

BT - Scandinavian Conference on Image Analysis

PB - Springer

ER -

Stets JD, Lyngby RA, Frisvad JR, Dahl AB. Material-Based Segmentation of Objects. In Scandinavian Conference on Image Analysis. Springer. 2019. p. 152-163. (Lecture Notes in Computer Science, Vol. 11482). https://doi.org/10.1007/978-3-030-20205-7_13