High-Resolution Sea Ice Maps with Convolutional Neural Networks

David Malmgren-Hansen, Allan Aasbjerg Nielsen, Matilde Brandt Kreiner, Roberto Saldo, Henning Skriver, Leif Toudal Pedersen, John Lavelle, Jørgen Buus-Hinkler

Research output: Contribution to conferenceConference abstract for conferenceResearchpeer-review

Abstract

Automatically generated high resolution sea ice maps have the potential to increase the use of satellite imagery in arctic applications. Applications include marine navigation, offshore operations, validation of ice models, and climate research. Especially for arctic marine navigation, frequent ice maps in high resolution are requested by most users, as documented by an internal project stakeholder survey. We present current results from our large-scale study of high resolution ice maps generation with Convolutional Neural Networks (CNNs). Our study is based on dual polarized (HH+HV) Extra Wide swath (EW) SAR data from the Copernicus Sentinel 1 satellite mission and we generate pixel-wise sea ice estimates in 40m x 40m resolution. The presentation will include a model validation against expert annotations of SAR images. In the near future we will expand our study to include AMSR2 Microwave Radiometer (MWR) data as input. The addition of MWR data can potentially solve the ambiguities in SAR data over open water, due to SAR backscatter variation at different wind conditions. Some CNN estimates are observed to confuse very homogeneous ice surfaces with similar backscatter open water scenarios, but results show a clear potential for this methodology. Our work is carried out under a Danish research project named Automated downstream Sea Ice Products (ASIP). The project goal is to automate generation of sea ice information from satellite images. ASIP is a collaboration between the Danish Meteorological Institute (DMI), the Technical University of Denmark and Harnvig Artic and Maritime. It sets out to automate, partially or fully, the extraction of arctic sea ice information from satellite imagery. Today, ice mapping is mainly done manually by ice-experts at national Ice Centers around the world. The project goal will enable analyzing larger quantities of satellite data, for better utilization of the available Sentinel-1 images and for providing ice maps to users more frequently. Recent literature shows an increased interest in algorithms for extraction of sea ice information, [2, 4]. As a part of the ASIP project a thorough analysis of the need for ice information was carried out among users by Harnvig Arctic and Maritime. This resulted in "ASIP Internal Stakeholder Survey Report", which substantiates the specic needs. One of the conclusions from this report is that 90% of use cases need simple ice/no-ice information for marine route planning purposes in high resolution (< 250m pr. pixel). Meeting this resolution requirement is unfortunately not possible with current MWR data alone, though its properties are otherwise good for ice concentration estimations. Hence, SAR data is the only source with regularly coverage as input data. A training data set was prepared by selecting all regional Ice Charts that were manually produced by the ice analysts in the DMI Ice Service based primarily on Sentinel-1 SAR imagery. SAR imagery has proven very suitable for Arctic sea ice monitoring, due to the radar sensors capability to see through clouds and in polar darkness. The training data set consists of a match-up of more than 900 ice charts and corresponding SAR imagery covering the period from the availability of the Sentinel-1A sensor data in November 2014 and Sentinel-1B data from September 2016, until December 2017. The individual ice charts cover dierent areas (depending on users and season) but the data set spans all of Greenland coast, apart from the northernmost regions, and with denser coverage in Southern Greenland. The CNN architecture we use in this work is inspired by models from image segmentation [1]. We train it on 400 000 sub-images of 250x250 pixels cut out from Sentinel-1 scenes corresponding to more than 25 billion pixel samples. Labels for each pixel are taken from ice concentration parameter in the ice-charts. Our initial approach has simply been to threshold ice concentrations, and consider <10% as open water and otherwise ice. This approach turns the problem into a pixelwise classication task, i.e. segmentation, and our CNN model produces a 250x250 segmentation mask for each input image. The way ice charts are produced combined with our threshold approach introduces an amount of mis-labelled pixels, but Deep Neural Networks (DNNs) are known to cope with "noisy" labels [3]. Current results can be seen in Figure 1, which shows the sub-images in a Sentinel-1B scene from June 2017 that matches with an DMI produced Ice Chart. It should be noted that labels are very coarse compared to the level of detail in a Sentinel-1 image. Despite this, the CNN still correctly classies the lead of water running into the ice in Figure 1 (marked with a redcircle).
Original languageEnglish
Publication date2019
Publication statusPublished - 2019
Event2019 ESA Living Planet Symposium
- Milano Congressi, Milano , Italy
Duration: 13 May 201917 May 2019

Conference

Conference2019 ESA Living Planet Symposium
LocationMilano Congressi
CountryItaly
CityMilano
Period13/05/201917/05/2019

Cite this

Malmgren-Hansen, D., Nielsen, A. A., Kreiner, M. B., Saldo, R., Skriver, H., Toudal Pedersen, L., ... Buus-Hinkler, J. (2019). High-Resolution Sea Ice Maps with Convolutional Neural Networks. Abstract from 2019 ESA Living Planet Symposium
, Milano , Italy.
Malmgren-Hansen, David ; Nielsen, Allan Aasbjerg ; Kreiner, Matilde Brandt ; Saldo, Roberto ; Skriver, Henning ; Toudal Pedersen, Leif ; Lavelle, John ; Buus-Hinkler, Jørgen. / High-Resolution Sea Ice Maps with Convolutional Neural Networks. Abstract from 2019 ESA Living Planet Symposium
, Milano , Italy.
@conference{a240bd9102f34eabbec39ae2bf5f0448,
title = "High-Resolution Sea Ice Maps with Convolutional Neural Networks",
abstract = "Automatically generated high resolution sea ice maps have the potential to increase the use of satellite imagery in arctic applications. Applications include marine navigation, offshore operations, validation of ice models, and climate research. Especially for arctic marine navigation, frequent ice maps in high resolution are requested by most users, as documented by an internal project stakeholder survey. We present current results from our large-scale study of high resolution ice maps generation with Convolutional Neural Networks (CNNs). Our study is based on dual polarized (HH+HV) Extra Wide swath (EW) SAR data from the Copernicus Sentinel 1 satellite mission and we generate pixel-wise sea ice estimates in 40m x 40m resolution. The presentation will include a model validation against expert annotations of SAR images. In the near future we will expand our study to include AMSR2 Microwave Radiometer (MWR) data as input. The addition of MWR data can potentially solve the ambiguities in SAR data over open water, due to SAR backscatter variation at different wind conditions. Some CNN estimates are observed to confuse very homogeneous ice surfaces with similar backscatter open water scenarios, but results show a clear potential for this methodology. Our work is carried out under a Danish research project named Automated downstream Sea Ice Products (ASIP). The project goal is to automate generation of sea ice information from satellite images. ASIP is a collaboration between the Danish Meteorological Institute (DMI), the Technical University of Denmark and Harnvig Artic and Maritime. It sets out to automate, partially or fully, the extraction of arctic sea ice information from satellite imagery. Today, ice mapping is mainly done manually by ice-experts at national Ice Centers around the world. The project goal will enable analyzing larger quantities of satellite data, for better utilization of the available Sentinel-1 images and for providing ice maps to users more frequently. Recent literature shows an increased interest in algorithms for extraction of sea ice information, [2, 4]. As a part of the ASIP project a thorough analysis of the need for ice information was carried out among users by Harnvig Arctic and Maritime. This resulted in {"}ASIP Internal Stakeholder Survey Report{"}, which substantiates the specic needs. One of the conclusions from this report is that 90{\%} of use cases need simple ice/no-ice information for marine route planning purposes in high resolution (< 250m pr. pixel). Meeting this resolution requirement is unfortunately not possible with current MWR data alone, though its properties are otherwise good for ice concentration estimations. Hence, SAR data is the only source with regularly coverage as input data. A training data set was prepared by selecting all regional Ice Charts that were manually produced by the ice analysts in the DMI Ice Service based primarily on Sentinel-1 SAR imagery. SAR imagery has proven very suitable for Arctic sea ice monitoring, due to the radar sensors capability to see through clouds and in polar darkness. The training data set consists of a match-up of more than 900 ice charts and corresponding SAR imagery covering the period from the availability of the Sentinel-1A sensor data in November 2014 and Sentinel-1B data from September 2016, until December 2017. The individual ice charts cover dierent areas (depending on users and season) but the data set spans all of Greenland coast, apart from the northernmost regions, and with denser coverage in Southern Greenland. The CNN architecture we use in this work is inspired by models from image segmentation [1]. We train it on 400 000 sub-images of 250x250 pixels cut out from Sentinel-1 scenes corresponding to more than 25 billion pixel samples. Labels for each pixel are taken from ice concentration parameter in the ice-charts. Our initial approach has simply been to threshold ice concentrations, and consider <10{\%} as open water and otherwise ice. This approach turns the problem into a pixelwise classication task, i.e. segmentation, and our CNN model produces a 250x250 segmentation mask for each input image. The way ice charts are produced combined with our threshold approach introduces an amount of mis-labelled pixels, but Deep Neural Networks (DNNs) are known to cope with {"}noisy{"} labels [3]. Current results can be seen in Figure 1, which shows the sub-images in a Sentinel-1B scene from June 2017 that matches with an DMI produced Ice Chart. It should be noted that labels are very coarse compared to the level of detail in a Sentinel-1 image. Despite this, the CNN still correctly classies the lead of water running into the ice in Figure 1 (marked with a redcircle).",
author = "David Malmgren-Hansen and Nielsen, {Allan Aasbjerg} and Kreiner, {Matilde Brandt} and Roberto Saldo and Henning Skriver and {Toudal Pedersen}, Leif and John Lavelle and J{\o}rgen Buus-Hinkler",
year = "2019",
language = "English",
note = "2019 ESA Living Planet Symposium<br/>, LPS 2019 ; Conference date: 13-05-2019 Through 17-05-2019",

}

Malmgren-Hansen, D, Nielsen, AA, Kreiner, MB, Saldo, R, Skriver, H, Toudal Pedersen, L, Lavelle, J & Buus-Hinkler, J 2019, 'High-Resolution Sea Ice Maps with Convolutional Neural Networks' 2019 ESA Living Planet Symposium
, Milano , Italy, 13/05/2019 - 17/05/2019, .

High-Resolution Sea Ice Maps with Convolutional Neural Networks. / Malmgren-Hansen, David; Nielsen, Allan Aasbjerg; Kreiner, Matilde Brandt; Saldo, Roberto; Skriver, Henning; Toudal Pedersen, Leif; Lavelle, John; Buus-Hinkler, Jørgen.

2019. Abstract from 2019 ESA Living Planet Symposium
, Milano , Italy.

Research output: Contribution to conferenceConference abstract for conferenceResearchpeer-review

TY - ABST

T1 - High-Resolution Sea Ice Maps with Convolutional Neural Networks

AU - Malmgren-Hansen, David

AU - Nielsen, Allan Aasbjerg

AU - Kreiner, Matilde Brandt

AU - Saldo, Roberto

AU - Skriver, Henning

AU - Toudal Pedersen, Leif

AU - Lavelle, John

AU - Buus-Hinkler, Jørgen

PY - 2019

Y1 - 2019

N2 - Automatically generated high resolution sea ice maps have the potential to increase the use of satellite imagery in arctic applications. Applications include marine navigation, offshore operations, validation of ice models, and climate research. Especially for arctic marine navigation, frequent ice maps in high resolution are requested by most users, as documented by an internal project stakeholder survey. We present current results from our large-scale study of high resolution ice maps generation with Convolutional Neural Networks (CNNs). Our study is based on dual polarized (HH+HV) Extra Wide swath (EW) SAR data from the Copernicus Sentinel 1 satellite mission and we generate pixel-wise sea ice estimates in 40m x 40m resolution. The presentation will include a model validation against expert annotations of SAR images. In the near future we will expand our study to include AMSR2 Microwave Radiometer (MWR) data as input. The addition of MWR data can potentially solve the ambiguities in SAR data over open water, due to SAR backscatter variation at different wind conditions. Some CNN estimates are observed to confuse very homogeneous ice surfaces with similar backscatter open water scenarios, but results show a clear potential for this methodology. Our work is carried out under a Danish research project named Automated downstream Sea Ice Products (ASIP). The project goal is to automate generation of sea ice information from satellite images. ASIP is a collaboration between the Danish Meteorological Institute (DMI), the Technical University of Denmark and Harnvig Artic and Maritime. It sets out to automate, partially or fully, the extraction of arctic sea ice information from satellite imagery. Today, ice mapping is mainly done manually by ice-experts at national Ice Centers around the world. The project goal will enable analyzing larger quantities of satellite data, for better utilization of the available Sentinel-1 images and for providing ice maps to users more frequently. Recent literature shows an increased interest in algorithms for extraction of sea ice information, [2, 4]. As a part of the ASIP project a thorough analysis of the need for ice information was carried out among users by Harnvig Arctic and Maritime. This resulted in "ASIP Internal Stakeholder Survey Report", which substantiates the specic needs. One of the conclusions from this report is that 90% of use cases need simple ice/no-ice information for marine route planning purposes in high resolution (< 250m pr. pixel). Meeting this resolution requirement is unfortunately not possible with current MWR data alone, though its properties are otherwise good for ice concentration estimations. Hence, SAR data is the only source with regularly coverage as input data. A training data set was prepared by selecting all regional Ice Charts that were manually produced by the ice analysts in the DMI Ice Service based primarily on Sentinel-1 SAR imagery. SAR imagery has proven very suitable for Arctic sea ice monitoring, due to the radar sensors capability to see through clouds and in polar darkness. The training data set consists of a match-up of more than 900 ice charts and corresponding SAR imagery covering the period from the availability of the Sentinel-1A sensor data in November 2014 and Sentinel-1B data from September 2016, until December 2017. The individual ice charts cover dierent areas (depending on users and season) but the data set spans all of Greenland coast, apart from the northernmost regions, and with denser coverage in Southern Greenland. The CNN architecture we use in this work is inspired by models from image segmentation [1]. We train it on 400 000 sub-images of 250x250 pixels cut out from Sentinel-1 scenes corresponding to more than 25 billion pixel samples. Labels for each pixel are taken from ice concentration parameter in the ice-charts. Our initial approach has simply been to threshold ice concentrations, and consider <10% as open water and otherwise ice. This approach turns the problem into a pixelwise classication task, i.e. segmentation, and our CNN model produces a 250x250 segmentation mask for each input image. The way ice charts are produced combined with our threshold approach introduces an amount of mis-labelled pixels, but Deep Neural Networks (DNNs) are known to cope with "noisy" labels [3]. Current results can be seen in Figure 1, which shows the sub-images in a Sentinel-1B scene from June 2017 that matches with an DMI produced Ice Chart. It should be noted that labels are very coarse compared to the level of detail in a Sentinel-1 image. Despite this, the CNN still correctly classies the lead of water running into the ice in Figure 1 (marked with a redcircle).

AB - Automatically generated high resolution sea ice maps have the potential to increase the use of satellite imagery in arctic applications. Applications include marine navigation, offshore operations, validation of ice models, and climate research. Especially for arctic marine navigation, frequent ice maps in high resolution are requested by most users, as documented by an internal project stakeholder survey. We present current results from our large-scale study of high resolution ice maps generation with Convolutional Neural Networks (CNNs). Our study is based on dual polarized (HH+HV) Extra Wide swath (EW) SAR data from the Copernicus Sentinel 1 satellite mission and we generate pixel-wise sea ice estimates in 40m x 40m resolution. The presentation will include a model validation against expert annotations of SAR images. In the near future we will expand our study to include AMSR2 Microwave Radiometer (MWR) data as input. The addition of MWR data can potentially solve the ambiguities in SAR data over open water, due to SAR backscatter variation at different wind conditions. Some CNN estimates are observed to confuse very homogeneous ice surfaces with similar backscatter open water scenarios, but results show a clear potential for this methodology. Our work is carried out under a Danish research project named Automated downstream Sea Ice Products (ASIP). The project goal is to automate generation of sea ice information from satellite images. ASIP is a collaboration between the Danish Meteorological Institute (DMI), the Technical University of Denmark and Harnvig Artic and Maritime. It sets out to automate, partially or fully, the extraction of arctic sea ice information from satellite imagery. Today, ice mapping is mainly done manually by ice-experts at national Ice Centers around the world. The project goal will enable analyzing larger quantities of satellite data, for better utilization of the available Sentinel-1 images and for providing ice maps to users more frequently. Recent literature shows an increased interest in algorithms for extraction of sea ice information, [2, 4]. As a part of the ASIP project a thorough analysis of the need for ice information was carried out among users by Harnvig Arctic and Maritime. This resulted in "ASIP Internal Stakeholder Survey Report", which substantiates the specic needs. One of the conclusions from this report is that 90% of use cases need simple ice/no-ice information for marine route planning purposes in high resolution (< 250m pr. pixel). Meeting this resolution requirement is unfortunately not possible with current MWR data alone, though its properties are otherwise good for ice concentration estimations. Hence, SAR data is the only source with regularly coverage as input data. A training data set was prepared by selecting all regional Ice Charts that were manually produced by the ice analysts in the DMI Ice Service based primarily on Sentinel-1 SAR imagery. SAR imagery has proven very suitable for Arctic sea ice monitoring, due to the radar sensors capability to see through clouds and in polar darkness. The training data set consists of a match-up of more than 900 ice charts and corresponding SAR imagery covering the period from the availability of the Sentinel-1A sensor data in November 2014 and Sentinel-1B data from September 2016, until December 2017. The individual ice charts cover dierent areas (depending on users and season) but the data set spans all of Greenland coast, apart from the northernmost regions, and with denser coverage in Southern Greenland. The CNN architecture we use in this work is inspired by models from image segmentation [1]. We train it on 400 000 sub-images of 250x250 pixels cut out from Sentinel-1 scenes corresponding to more than 25 billion pixel samples. Labels for each pixel are taken from ice concentration parameter in the ice-charts. Our initial approach has simply been to threshold ice concentrations, and consider <10% as open water and otherwise ice. This approach turns the problem into a pixelwise classication task, i.e. segmentation, and our CNN model produces a 250x250 segmentation mask for each input image. The way ice charts are produced combined with our threshold approach introduces an amount of mis-labelled pixels, but Deep Neural Networks (DNNs) are known to cope with "noisy" labels [3]. Current results can be seen in Figure 1, which shows the sub-images in a Sentinel-1B scene from June 2017 that matches with an DMI produced Ice Chart. It should be noted that labels are very coarse compared to the level of detail in a Sentinel-1 image. Despite this, the CNN still correctly classies the lead of water running into the ice in Figure 1 (marked with a redcircle).

M3 - Conference abstract for conference

ER -

Malmgren-Hansen D, Nielsen AA, Kreiner MB, Saldo R, Skriver H, Toudal Pedersen L et al. High-Resolution Sea Ice Maps with Convolutional Neural Networks. 2019. Abstract from 2019 ESA Living Planet Symposium
, Milano , Italy.