Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning

Patrick Schmidt; Dimitrios Katsatos; Dimitrios Alexiou; Ioannis Kostavelis; Dimitrios Giakoumis; Dimitrios Tzovaras; Lazaros Nalpantidis

doi:10.22260/ISARC2024/0116

Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning

Patrick Schmidt, Dimitrios Katsatos, Dimitrios Alexiou, Ioannis Kostavelis, Dimitrios Giakoumis, Dimitrios Tzovaras, Lazaros Nalpantidis

Department of Electrical and Photonics Engineering

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

83 Downloads (Orbit)

Abstract

The adoption of autonomous systems is a foreseeable necessity in the construction sector due to work hazards and labor shortages. This paper presents a semantic 3D understanding module that creates 3D models of construction sites with highlighted regions of interest for shotcrete application. The approach uses YOLOv8m-seg and SiamMask for robust semantic segmentation together with RTAB-Map and InfiniTAM for visual odometry and 3D reconstruction. Our method is the first step towards a novel, autonomous robot for shotcrete spraying and finishing. The effectiveness of our approach is shown on a mock-up construction site and provides evidence for the applicability of robotic construction

Original language	English
Title of host publication	Proceedings of the 41^st International Symposium on Automation and Robotics in Construction
Publisher	IEEE
Publication date	2024
Pages	896-903
ISBN (Print)	978-0-6458322-1-1
DOIs	https://doi.org/10.22260/ISARC2024/0116
Publication status	Published - 2024
Event	41^st International Symposium on Automation and Robotics in Construction - LILLIAD – Learning center innovation, Lille, France Duration: 3 Jun 2024 → 7 Jun 2024

Conference

Conference	41^st International Symposium on Automation and Robotics in Construction
Location	LILLIAD – Learning center innovation
Country/Territory	France
City	Lille
Period	03/06/2024 → 07/06/2024

Keywords

Construction Robotics
3D Reconstruction
Semantic Segmentation
Shotcrete Automation

Access to Document

10.22260/ISARC2024/0116

FulltextFinal published version, 2.25 MB

OpenUrl availability

Full text

Cite this

Schmidt, P., Katsatos, D., Alexiou, D., Kostavelis, I., Giakoumis, D., Tzovaras, D., & Nalpantidis, L. (2024). Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning. In Proceedings of the 41^st International Symposium on Automation and Robotics in Construction (pp. 896-903). IEEE. https://doi.org/10.22260/ISARC2024/0116

@inproceedings{b857a01852bd4a5094cac9d953e2e56d,

title = "Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning",

abstract = "The adoption of autonomous systems is a foreseeable necessity in the construction sector due to work hazards and labor shortages. This paper presents a semantic 3D understanding module that creates 3D models of construction sites with highlighted regions of interest for shotcrete application. The approach uses YOLOv8m-seg and SiamMask for robust semantic segmentation together with RTAB-Map and InfiniTAM for visual odometry and 3D reconstruction. Our method is the first step towards a novel, autonomous robot for shotcrete spraying and finishing. The effectiveness of our approach is shown on a mock-up construction site and provides evidence for the applicability of robotic construction",

keywords = "Construction Robotics, 3D Reconstruction, Semantic Segmentation, Shotcrete Automation",

author = "Patrick Schmidt and Dimitrios Katsatos and Dimitrios Alexiou and Ioannis Kostavelis and Dimitrios Giakoumis and Dimitrios Tzovaras and Lazaros Nalpantidis",

year = "2024",

doi = "10.22260/ISARC2024/0116",

language = "English",

isbn = "978-0-6458322-1-1",

pages = "896--903",

booktitle = "Proceedings of the 41st International Symposium on Automation and Robotics in Construction",

publisher = "IEEE",

address = "United States",

note = "41<sup>st</sup> International Symposium on Automation and Robotics in Construction, ISARC 2024 ; Conference date: 03-06-2024 Through 07-06-2024",

}

Schmidt, P, Katsatos, D, Alexiou, D, Kostavelis, I, Giakoumis, D, Tzovaras, D & Nalpantidis, L 2024, Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning. in Proceedings of the 41^st International Symposium on Automation and Robotics in Construction. IEEE, pp. 896-903, 41^st International Symposium on Automation and Robotics in Construction, Lille, France, 03/06/2024. https://doi.org/10.22260/ISARC2024/0116

Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning. / Schmidt, Patrick; Katsatos, Dimitrios; Alexiou, Dimitrios et al.
Proceedings of the 41^st International Symposium on Automation and Robotics in Construction. IEEE, 2024. p. 896-903.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

TY - GEN

T1 - Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning

AU - Schmidt, Patrick

AU - Katsatos, Dimitrios

AU - Alexiou, Dimitrios

AU - Kostavelis, Ioannis

AU - Giakoumis, Dimitrios

AU - Tzovaras, Dimitrios

AU - Nalpantidis, Lazaros

PY - 2024

Y1 - 2024

N2 - The adoption of autonomous systems is a foreseeable necessity in the construction sector due to work hazards and labor shortages. This paper presents a semantic 3D understanding module that creates 3D models of construction sites with highlighted regions of interest for shotcrete application. The approach uses YOLOv8m-seg and SiamMask for robust semantic segmentation together with RTAB-Map and InfiniTAM for visual odometry and 3D reconstruction. Our method is the first step towards a novel, autonomous robot for shotcrete spraying and finishing. The effectiveness of our approach is shown on a mock-up construction site and provides evidence for the applicability of robotic construction

AB - The adoption of autonomous systems is a foreseeable necessity in the construction sector due to work hazards and labor shortages. This paper presents a semantic 3D understanding module that creates 3D models of construction sites with highlighted regions of interest for shotcrete application. The approach uses YOLOv8m-seg and SiamMask for robust semantic segmentation together with RTAB-Map and InfiniTAM for visual odometry and 3D reconstruction. Our method is the first step towards a novel, autonomous robot for shotcrete spraying and finishing. The effectiveness of our approach is shown on a mock-up construction site and provides evidence for the applicability of robotic construction

KW - Construction Robotics

KW - 3D Reconstruction

KW - Semantic Segmentation

KW - Shotcrete Automation

U2 - 10.22260/ISARC2024/0116

DO - 10.22260/ISARC2024/0116

M3 - Article in proceedings

SN - 978-0-6458322-1-1

SP - 896

EP - 903

BT - Proceedings of the 41st International Symposium on Automation and Robotics in Construction

PB - IEEE

T2 - 41<sup>st</sup> International Symposium on Automation and Robotics in Construction

Y2 - 3 June 2024 through 7 June 2024

ER -

Towards autonomous shotcrete construction: semantic 3D reconstruction for concrete deposition using stereo vision and deep learning

Abstract

Conference

Keywords

Access to Document

OpenUrl availability

Fingerprint

Cite this