A Convolutional Neural Network Architecture for Sentinel-1 and AMSR2 Data Fusion

With a growing number of different satellite sensors, data fusion offers great potential in many applications. In this work, a Convolutional Neural Network (CNN) architecture is presented for fusing Sentinel-1 SAR imagery and the Advanced Microwave Scanning Radiometer 2 (AMSR2) data. The CNN is applied to prediction of Arctic sea ice for marine navigation and as input to sea ice forecast models. This generic model is specifically well suited for fusing data sources where the ground resolutions of the sensors differ with orders of magnitude, here 35 km x 62 km (for AMSR2, 6.9Ghz) compared to the 93 m x 87 m (for sentinel-1 IW mode). In this work, two optimization approaches are compared using the categorical cross-entropy error function in the specific application of CNN training on sea ice charts. In the first approach, concentrations are thresholded to be encoded in a standard binary fashion, and in the second, concentrations are used as the target probability directly. The second method leads to a significant improvement in R2 measured on the prediction of ice concentrations evaluated over the test set. The performance improves, both in terms of robustness to noise and alignment with mean concentrations from Ice Analysts in the validation data, and an R2 value of 0.89 is achieved over the independent test set. It can be concluded that CNNs are suitable for multi-sensor fusion even with sensors that differ in resolutions by large factors, such as in the case of Sentinel-1 SAR and AMSR2.