3D freehand ultrasound reconstruction using a piecewise smooth Markov random field

Highlights

• We developed a probabilistic model for 3D freehand ultrasound reconstruction.

• A piecewise-smooth Markov random field model is used to preserve object boundaries.

• Noise levels of obtained data are considered into the reconstruction process.

• An efficient optimization algorithm is introduced for the energy minimization.

• The algorithm and GPU computing speed up the optimization at least 46-times.

Abstract

In this paper, we introduce a novel three-dimensional (3D) reconstruction framework for ultrasound images using a piecewise smooth Markov random field (MRF) model from irregularly spaced B-scan images obtained by freehand scanning. Freehand 3D ultrasound imaging is a useful system for various clinical applications, including image-guided surgeries and interventions, as well as diagnoses, due to the variety of its scan ranges and relatively low cost. The reconstruction process performs a key role in this system because its sampling irregularities may cause undesired artifacts, and ultrasound images generally suffer from noise and distortions. However, traditional approaches are based on simple geometric interpolations, such as pixel-based or distance-weighted methods, which are sensitive to sampling density and speckle noise. These approaches generally have an additional limitation of smoothing objects boundaries. To reduce speckle noise and preserve boundaries, we devised a piecewise smooth (PS) MRF model and developed its optimization algorithm. In our framework, we can easily apply an individual noise level for each image pixel, which is specified by the characteristics of an ultrasound probe, and possibly, the lateral and axial positions of an image. As a result, the reconstructed volume has sharp object boundaries with reduced speckle noise and artifacts. Our PS-MRF model provides simple segmentation results within a reconstruction framework that is useful for various purposes, such as clear visualization. The corresponding optimization methods have also been developed, and we tested a virtual phantom and a physical phantom model. Experimental results show that our method outperforms existing methods in terms of interpolation and segmentation accuracy. With this method, all computations can be performed with practical time consumption and with an appropriate resolution, via parallel computing using graphic processing units.

Introduction

Ultrasound 3D imaging has received significant attention in many diagnostic areas, particularly obstetrics [1] and cardiology [2]. Not only 3D visualizations, such as in volume and surface rendering, but also 2D sectional images at various orientations, may provide helpful clues for diagnoses. In addition, many studies have demonstrated the applicability of 3D ultrasound imaging to image-guided surgery and interventions, e.g., neurosurgery [3], biopsy [4], and radiation therapy [5].

There are two types of probes that can generate 3D ultrasound data: dedicated 3D probes and conventional 2D probes with mechanical or freehand scanning. Dedicated 3D probes can scan the 3D range rapidly and can generate volumetric images quickly. However, systems with dedicated 3D probes are more expensive, and the large contact surface of the probe makes it difficult to obtain clean images of hidden structures under bones or gas. These probes are larger and heavier than 2D probes, and their scanning ranges are limited by their size.

A conventional probe generates 2D images initially, but 3D data can be obtained through sweeping the probe through the target volume while acquiring the position and orientation information from a position sensor. Therefore, the scanning ranges have a greater variety than those of dedicated 3D probes. Sweeping is conducted using a mechanical device or by hand. Mechanical sweeping generates regularly spaced B-scan images along a predefined path, while freehand sweeping generates irregularly spaced B-scan images along an arbitrary path. Mechanical devices can sweep or rotate the volume with a uniform speed, but it is impossible to scan larger volumes than the mechanics allows. This limitation cannot be overcome through simply increasing the size of the system [6]. Compared with other scanning methods, freehand ultrasound imaging has more freedom in terms of scanning range, and various normal 2D probes can be used directly. These advantages are useful for various applications. Fig. 1 shows a conceptual diagram of the freehand scanning system.

In freehand scanning, because B-scan images are captured at arbitrary locations and orientations, it is not guaranteed that the physical image pixel locations will match the voxel positions. Therefore, it is necessary to fill in missing voxel values in the volume of interest (VOI) from the scanned images. To obtain a better reconstruction result, noise and artifacts should be considered while interpolating the voxel values.

However, many previous studies have focused on the calibration of the position sensors to accurately convert the image coordinates to global coordinates [7]. Compared with the acquisition process, the reconstruction process has been considered simply in most works despite its importance for the quality of the final images of a volume.

There are numerous considerations in the reconstruction step to improve the volume quality. First, ultrasound images have various types of noise and artifacts, such as speckle noise, refraction, shadowing, reverberation, and so on. Most artifacts originate from the interaction between the ultrasound signal and inside materials, i.e. they represent different patterns according to the inner materials. High-level processing is usually required in order to reduce them. Meanwhile, speckle noise is typically distributed throughout B-scan images. Speckle noise also shows unique patterns according to the materials, but it is distributed all over images with high frequency compared to other artifacts (see Fig. 2(a)). This results in degradation of the image quality and makes it difficult for viewers to interpret these images and make diagnoses. Therefore, speckle noise reduction is an important issue in ultrasound image analysis, and it has been investigated in previous works as post-processing [8]. However, it is not practically efficient to filter every scanned image in case of this study.

Another important characteristic of ultrasound images that should be considered is that the spatial resolution is not uniform within an image due to the transducer and signal characteristics. Fig. 2(c) presents scan result of a phantom for quality assurance, and the image quality varies with the penetration depth as well as the lateral direction. Therefore, B-scan image pixel data cannot be considered to have the same confidence level over all pixels, and the data confidence should be considered according to pixel location.

However, most existing reconstruction methods involve simple averaging or interpolation, i.e. pixel nearest neighbor (PNN), voxel nearest neighbor (VNN), distance weighted (DW), and radial basis function (RBF) methods [10], [11]. These approaches interpolate the voxel intensities from the sampled data by simply taking the nearest neighbors (PNN and VNN), which are sensitive to the sampling density, or weighted averages (DW and RBF). To improve the quality, various median filters are introduced [12]. In addition, Wen et al. introduced a fast marching method to fill holes in order to overcome the limitations of the nearest-neighbor selection [13]. Instead of spatial interpolation of the voxels or pixels, another approach for the 3D reconstruction is to interpolate probe trajectories to create intermediate virtual scanning planes [14].

Another important approach uses a Bayesian framework to infer the voxel values in a grid [15], [16]. This approach assumes a three-dimensional parametric function that has basis functions centered at every voxel. Each basis function is determined by a corresponding coefficient, and all coefficients of the voxel grid are modeled as a 3D grid Markov random field (MRF) with the typical 6-connected neighborhood system [15]. In this approach, the observations are assumed to be Rayleigh distributed random variables to represent speckle noises.

However, one significant drawback of these methods is that they may not preserve object boundaries. Boundaries are easily smoothed out using these models because they assume uniform smoothness. To resolve this problem, we introduce a novel framework for reconstructing 3D ultrasound images from freehand scanning. The primary goals of this research are the following:

• Speckle noise reduction: Speckle noise results in images that are noisy in ways such that the interpolated values could be severely affected. Our first goal in this research is to reduce the influence of speckle noise.

• Boundary conservation: It can be helpful in many clinical applications if surface boundaries are clearly observable in ultrasound images. However, most state-of-the-art reconstruction algorithms smoothly interpolate the sampled data points.

• Noise level (data confidence) consideration: Within one B-scan image, the spatial resolution and image quality vary with the location. If several data are observed in close physical locations, e.g., the same voxel, their confidence level should be given significant consideration, rather than simply considering distances.

• Computation time efficiency: In many clinical applications, fast computation is required for practical use. In particular, the algorithm needs to be insensitive to the amount of data sampled from a wide scan range for the freehand system.

To achieve the above goals, we adopt a Bayesian framework to infer the voxel intensities from the observed data as in [15]. Through modeling the voxel grids as an MRF and applying an observation model that considers the noise distribution, we can alleviate the effects of speckle noise in the 3D reconstruction process. In addition, we use a piecewise smooth MRF for boundary conservation through considering the ultrasound imaging characteristics. MRF reconstruction models can be categorized into three types: smooth models, piecewise constant (PC) models, and piecewise smooth (PS) models [17]. The smooth models do not consider signal discontinuities, so object boundaries are easily blurred. The PC models can preserve the discontinuities, but they separate the gradual changes of signals into several different regions. One of the primary characteristics of ultrasound imaging is that the signal intensities for the same tissue may not be uniform due to attenuations or shadow effects. Therefore, the PC models also are not appropriate for ultrasound reconstruction. The PS models can represent regions with gradual changes and separate different regions, only with large jumps. For this reason, we propose a PS model that conserves the boundary information of different regions. Within our framework, the noise levels can be simply applied without changing or adding terms. In addition, the computation to get the optimal solution is accelerated by parallelization on graphic processing units (GPUs).

This paper is organized as follows. In Section 2, an overview of the Bayesian formulation of our problem is introduced, and several types of prior models and corresponding optimization methods are described. In Section 3, experiments with synthetic US images and real images are described. In Section 4, we discuss the advantages and limitations of the proposed method and suggest directions for future work.