Robust Free Area Mapping for Autonomous Harbour Navigation

Research shows that a large parts of accidents at sea is caused by human error. Collisions and grounding can occur in part due to the error of the human navigator, be that due to fatigue, intoxication, inexperience or other causes. Many such accidents could be avoided by letting a automated navigator take partial or full control of the vessel at certain times, if such as system can be made in a robust enough way. Partial autonomy or decision support can be implemented via usage of e.g. a remotely controlled bridge, where the human navigator controls the vessel from land or his cabin, or by a strict supervision role, only intervening when the automated system prompts the navigator. Full autonomous vessels will be unmanned, and could be used in transportation roles to remote islands or crossings at night. Multiple sensors are already available for the navigator on most commercial vessels. These include object tracking and rudimentary path prediction via radar, Ship intent and positional data using AIS, and information available through sea charts, commonly plotted using an ECDIS system. On top of this, a human navigator also applies physical outlook through the vessel windows to assess risks in the surroundings of the vessel. This thesis investigates the possibility of a partial or full replacement of the physical outlook using electro optical sensors, in order to be able to disconnect the human navigator entirely and achieve autonomy. The research is part of the ShippingLab project, which aims to develop Denmark’s first autonomous commercial ferry. Object detection using visible sensors were investigated, and it was found that using a deep learning approach in the visible light and long wave infrared spectral ranges, a solution conforming to COLREGS could be achieved. The COLREGS set certain limits on the closest point of approach and time to closest point of approach for vessels in the vicinity, in terms of when an action to avoid a collision is necessary. The method was enhanced using ensembling of different spectral ranges, and the detection range was increased using image upscaling. Methods for tracking detected objects was researched, and it was found that an approach based on features present in the layers of the object detection network could be used for matching objects across frames. With an objects position within an image being known, it is possible to track these objects using a Kalman filtering approach. Distance metrics was also used to match objects. A free space detection algorithm was developed, via a novel image segmentation technique, based on grid sub partitioning and a classifier for these sub partitions. Reassembling these results in a rough segmentation of the free space in an image. Using projective geometry, it was shown that 2D image points could be backprojected into the 3D space by constraining the searchable area in 3D space to a plane. This enabled the transformation of the free space segmentation in to a reference frame common to the other sensors on a vessel, and allowed for the construction of a grid occupancy map based on this segmentation. The map is accurate enough to be used for navigational purposes, in conjunction with existing sensor information. Elements of the research was implemented in to the autonomy package of the ShippingLab project, and applied in a semi autonomous demo across the Limfjord in Aalborg, Denmark.