Data-driven Ultrasound Localization Microscopy using Deep Learning

Ultrasound localization microscopy (ULM) can surpass the spatial resolution limit of conventional ultrasound imaging by accumulating centroids of MBs injected into the bloodstream in one image frame. However, there is a trade-off between the resolution and data acquisition time. For accurate localization, low concentrations of diluted MBs are commonly used. That limits the number of detectable MBs, and the long data acquisition time is thus required. This Ph.D. project aims to localize high concentrations of MBs using data-driven deep learning methods.
Initially, localizing scatterers from radiofrequency (RF) channel data has been investigated since point spread functions (PSFs) of closely spaced scatterers overlap each other in beamformed ultrasound images. Convolutional neural networks (CNNs) were trained with simulated ultrasound data and non-overlapping Gaussian confidence maps for stable training. The performance was evaluated in the simulated test data and phantom measurements, showing that scatterers closer than the resolution limit of delay-and-sum (DAS) beamforming can be localized.
Next, a sub-pixel localization method using a CNN on the beamformed ultrasound images has been studied. Sub-pixel localization was achieved by fitting Gaussians in the extended non-overlapping Gaussian confidence maps. That allows utilizing computational resources efficiently as no additional upsampling is required. In a phantom experiment at a high MB concentration, the sub-pixel CNN localization method resolved a pair of channels spaced 22 μm away while centroid detection failed. Sub-pixel CNN localization was also tested on in vivo data and resulted in estimated MBs spaced closer than a wavelength.
Lastly, model-based neural networks for ULM have been investigated. The modelbased neural networks are designed based on mathematical foundations. Hence, compared with the model-agnostic data-driven methods, fewer learning parameters are required. The few learning parameters allow a short training time with a small number of data, good generalization, and fast inference speed. Deep unfolded ULM, which localizes the overlapping MBs using a model-based neural network, has shown comparable results to the fully data-driven methods on simulated and measured data. In addition, task-adaptive beamforming for MB localization has been investigated. By jointly optimizing a deep beamformer and localization network, ultrasound images tailored for MB localization were able to be obtained, and thus, the performance of deep unfolded ULM increased.