University of Ljubljana, Faculty of Electrical Engineering, Slovenia. darko.skerl@fe.uni-lj.si
The accuracy and robustness of a registration method depend on a number of factors, such as imaging modality, image content and image degrading effects, the class of spatial transformation used for registration, similarity measure, optimization, and numerous implementation details. The complex interdependence of these factors makes the assessment of the influence of a particular factor on registration difficult, although it is often desirable to have some estimate of such influences prior to registration. The similarity measure used to create the cost function is one of the factors that most influences the quality of registration. Traditionally, limited information on the behavior of a similarity measure is obtained either by studying the quality of the final registration or by drawing plots of similarity measure values obtained by translating or rotating one image relative to the "gold standard." In this paper, we present a protocol for a more thorough, optimization-independent, and systematic statistical evaluation of similarity measures. This protocol estimates a similarity measure's capture range, the number, location and extent of local optima, and the accuracy and distinctiveness of the global optimum. To show that the proposed evaluation protocol is viable, we have conducted several experiments with nine similarity measures and real computed tomography and magnetic resonance (MR) images of a spine phantom, MR brain images, and MR and positron emission tomography brain images, for which "gold standard" registrations were available. We have also studied the impact of histogram bin size on the behavior of nine similarity measures. The proposed evaluation protocol is useful for selecting the best similarity measure and corresponding optimization method for a particular application, as well as for studying the influence of sampling, interpolation, histogram bin size, partial image overlap, and image degradation, such as noise, intensity inhomogeneity, and geometrical distortions on the behavior of a similarity measure.
Other papers by authors:
Comparative evaluation of similarity measures for the rigid registration of multi-modal head images.
Image registrations that are based on similarity measures simply adjust the parameters of an appropriate spatial transformation model until the similarity measure reaches an optimum. The numerous similarity measures that have been proposed in the past are differently sensitive to imaging modality, image content and differences in the image content, selection of the floating and target image, partial image overlap, etc. In this paper, we evaluate and compare 12 similarity measures for the rigid registration. To study the impact of different imaging modalities on the behavior of similarity measures, we have used 16 CT/MR and 6 PET/MR image pairs with known 'gold standard' registrations. The results for the PET/MR registration and for the registration of CT to both rectified and unrectified MR images indicate that mutual information, normalized mutual information and the entropy correlation coefficient are the most accurate similarity measures and have the smallest risk of being trapped in a local optimum. The results of an experiment on the impact of exchanging the floating and target image indicate that, especially in MR/PET registrations, the behavior of some similarity measures, such as mutual information, significantly depends on which image is the floating and which is the target.
Department of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia.
Purpose: A promising patient positioning technique is based on registering computed tomographic (CT) or magnetic resonance (MR) images to cone-beam CT images (CBCT). The extra radiation dose delivered to the patient can be substantially reduced by using fewer projections. This approach results in lower quality CBCT images. The purpose of this study is to evaluate a number of similarity measures (SMs) suitable for registration of CT or MR images to low-quality CBCTs. Methods and Materials: Using the recently proposed evaluation protocol, we evaluated nine SMs with respect to pretreatment imaging modalities, number of two-dimensional (2D) images used for reconstruction, and number of reconstruction iterations. The image database consisted of 100 X-ray and corresponding CT and MR images of two vertebral columns. Results: Using a higher number of 2D projections or reconstruction iterations results in higher accuracy and slightly lower robustness. The similarity measures that behaved the best also yielded the best registration results. The most appropriate similarity measure was the asymmetric multi-feature mutual information (AMMI). Conclusions: The evaluation protocol proved to be a valuable tool for selecting the best similarity measure for the reconstruction-based registration. The results indicate that accurate and robust CT/CBCT or even MR/CBCT registrations are possible if the AMMI similarity measure is used.
Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv. 2008 ;11 (Pt 1):942-50 18979836 (P,S,G,E,B)
University of Ljubljana, Faculty of Electrical Engineering, Slovenia. tomaz.vrtovec@fe.uni-lj.si
In the past, a number of methods were proposed for quantitative assessment of vertebral rotation from three-dimensional (3D) images. However, these methods were based on manual identification of distinctive anatomical landmarks, required manual determination of cross-sections from 3D images, and measured only axial vertebral rotation instead of the rotation in 3D. In this paper, we propose an automated method for quantitative assessment of vertebral rotation in 3D that is based on finding the planes of vertebral symmetry by matching image intensity gradients on both sides of each plane. The method was evaluated on 28 images of normal and pathological vertebrae, obtained by computed tomography (CT) and magnetic resonance (MR). For each vertebra, final angle displacements of 200 initial angle displacements, uniformly distributed within 30 degrees from manually obtained reference angles, were obtained. The results show that by the proposed method, vertebral rotation can be successfully estimated in 3D with an average accuracy of 1. degrees and precision of .5 degrees.
Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv. 2008 ;11 (Pt 1):762-70 18979815 (P,S,G,E,B)
Faculty of Electrical Engineering, University of Ljubljana, Slovenia. ziga.spiclin@fe.uni-lj.si
In this paper, a novel method for EEG to MRI registration is proposed. Initial registration is achieved by extracting and matching symmetry planes of MRI and EEG data, followed by iterative registration based on minimizing a cost function. Comparison of the intensity distributions of the whole MR image and MRI voxels around a head surface point yields global similarities, while the comparison of intensity distributions of MRI voxels around corresponding EEG points, which reflects the head's sagittal symmetry, yields local similarities. Therefore, when the EEG points are registered to the MR image, maximal global and local similarities should be obtained. The cost function, incorporating global and local similarities, was the sum of Kullback-Leibler divergences between corresponding intensity distributions. The proposed method was evaluated on clinical MRI data with simulated EEG data, yielding mean registration error of .48 +/- .33 mm, while with real EEG data an average root-mean-square point-to-surface error of 2.27 +/- .02 mm was obtained.
University of Ljubljana, Faculty of Electrical Engineering, Trzaska 25, SI-1000 Ljubljana, Slovenia.
The purpose of this study is to present a framework for quantitative analysis of spinal curvature in 3D. In order to study the properties of such complex 3D structures, we propose two descriptors that capture the characteristics of spinal curvature in 3D. The descriptors are the geometric curvature (GC) and curvature angle (CA), which are independent of the orientation and size of spine anatomy. We demonstrate the two descriptors that characterize the spinal curvature in 3D on 30 computed tomography (CT) images of normal spine and on a scoliotic spine. The descriptors are determined from 3D vertebral body lines, which are obtained by two different methods. The first method is based on the least-squares technique that approximates the manually identified vertebra centroids, while the second method searches for vertebra centroids in an automated optimization scheme, based on computer-assisted image analysis. Polynomial functions of the fourth and fifth degree were used for the description of normal and scoliotic spinal curvature in 3D, respectively. The mean distance to vertebra centroids was 1.1 mm (+/- .6 mm) for the first and 2.1 mm (+/-1.4 mm) for the second method. The distributions of GC and CA values were obtained along the 30 images of normal spine at each vertebral level and show that maximal thoracic kyphosis (TK), thoracolumbar junction (TJ) and maximal lumbar lordosis (LL) on average occur at T3/T4, T12/L1 and L4/L5, respectively. The main advantage of GC and CA is that the measurements are independent of the orientation and size of the spine, thus allowing objective intra- and inter-subject comparisons. The positions of maximal TK, TJ and maximal LL can be easily identified by observing the GC and CA distributions at different vertebral levels. The obtained courses of the GC and CA for the scoliotic spine were compared to the distributions of GC and CA for the normal spines. The significant difference in values indicates that the descriptors of GC and CA may be used to detect and quantify scoliotic spinal curvatures. The proposed framework may therefore improve the understanding of spine anatomy and aid in the clinical quantitative evaluation of spinal deformities.
Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv. 2007 ;10 (Pt 1):450-7 18051090 (P,S,G,E,B)
An important part of image-guided radiation therapy or surgery is registration of a three-dimensional (3D) preoperative image to two-dimensional (2D) images of the patient. It is expected that the accuracy and robustness of a 3D/2D image registration method do not depend solely on the registration method itself but also on the number and projections (views) of intraoperative images. In this study, we systematically investigate these factors by using registered image data, comprising of CT and X-ray images of a cadaveric lumbar spine phantom and the recently proposed 3D/2D registration method. The results indicate that the proportion of successful registrations (robustness) significantly increases when more X-ray images are used for registration.
Faculty of Electrical Engineering, University of Ljubljana, Trzaska 25, SI-1000 Ljubljana, Slovenia.
A novel method for automated curved planar reformation (CPR) of magnetic resonance (MR) images of the spine is presented. The CPR images, generated by a transformation from image-based to spine-based coordinate system, follow the structural shape of the spine and allow the whole course of the curved anatomy to be viewed in individual cross-sections. The three-dimensional (3D) spine curve and the axial vertebral rotation, which determine the transformation, are described by polynomial functions. The 3D spine curve passes through the centres of vertebral bodies, while the axial vertebral rotation determines the rotation of vertebrae around the axis of the spinal column. The optimal polynomial parameters are obtained by a robust refinement of the initial estimates of the centres of vertebral bodies and axial vertebral rotation. The optimization framework is based on the automatic image analysis of MR spine images that exploits some basic anatomical properties of the spine. The method was evaluated on 21 MR images from 12 patients and the results provided a good description of spine anatomy, with mean errors of 2.5 mm and 1.7 degrees for the position of the 3D spine curve and axial rotation of vertebrae, respectively. The generated CPR images are independent of the position of the patient in the scanner while comprising both anatomical and geometrical properties of the spine.
Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv. 2006 ;9 (Pt 2):135-43 17354765 (P,S,G,E,B)
Recommended:1 Cited:2
University of Ljubljana, Faculty of Electrical Engineering, Slovenia. tomaz.vrtovec@fe.uni-lj.si
We present a novel method for curved planar reformation (CPR) of spine images obtained by magnetic resonance (MR) imaging. CPR images, created via a transformation from image-based to spine-based coordinate system, follow the structural shape of the spine and allow the whole course of the curved structure to be viewed in a single image. The spine-based coordinate system is defined on the 3D spine curve and on the axial vertebral rotation, both described by polynomial models. The 3D spine curve passes through the centers of vertebral bodies, and the axial vertebral rotation determines the rotation of vertebral spinous processes around the spine. The optimal polynomial parameters are found in an optimization framework, based on image analysis. The method was evaluated on 19 MR images of the spine from 10 patients.
University of Ljubljana, Faculty of Electrical Engineering, Trzaska 25, 1000 Ljubljana, Slovenia.
Medical image acquisition devices provide a vast amount of anatomical and functional information, which facilitate and improve diagnosis and patient treatment, especially when supported by modern quantitative image analysis methods. However, modality specific image artifacts, such as the phenomena of intensity inhomogeneity in magnetic resonance images (MRI), are still prominent and can adversely affect quantitative image analysis. In this paper, numerous methods that have been developed to reduce or eliminate intensity inhomogeneities in MRI are reviewed. First, the methods are classified according to the inhomogeneity correction strategy. Next, different qualitative and quantitative evaluation approaches are reviewed. Third, 60 relevant publications are categorized according to several features and analyzed so as to reveal major trends, popularity, evaluation strategies and applications. Finally, key evaluation issues and future development of the inhomogeneity correction field, supported by the results of the analysis, are discussed.
University of Ljubljana, Faculty of Electrical, Engineering, Trzaska 25, SI-1000 Ljubljana, Slovenia; phone:+386-1-4768-327; fax:+386-1-4768-279; e-mail: tomaz.vrtovec@fe.uni-lj.si.
Traditional techniques for analyzing tortuous anatomical structures (e.g. arteries, colon, spine) in the coordinate system of the 3D image generally do not provide sufficient or qualitative enough diagnostic information, because planar cross-sections do not follow curved paths along the structures. To overcome this shortcoming, images in the coordinate system of the structure must be created. We propose a transformation from standard image-based to a novel spine-based coordinate system. The origin and axes of the proposed spine-based coordinate system are determined on the curve that represents the vertebral column, and the rotation of the vertebrae around the spine curve, both of which are described by polynomial models. The optimal polynomial parameters are obtained in an image analysis based optimization framework. The method has been evaluated on five CT spine images.
Latest similar papers:
Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada N2L 3G1.
A novel similarity measure for registering magnetic resonance (MR) and computed tomography (CT) images has been designed and built. MR-CT registration methods often rely on the statistical intensity relationship between the images. The proposed similarity measure instead depends on the statistical relationship between the complex phase order between the images. By utilizing the complex phase order likelihood (CPOL) as a similarity measure, structural relationships instead of intensity relationships are explicitly used. This approach can be advantageous for MR-CT registration, where the intensities of the CT imagery have highly complex and nonlinear relationships with the intensities of corresponding MR imagery but simpler linear structural relationships. This new similarity measure has been tested on real MR-CT 3D volumes and has been evaluated based on fiducial registration error to determine alignment accuracy. Quantitative results show that CPOL is capable of achieving comparable alignment accuracy when compared to normalized mutual information, while being more robust to imaging artifacts such as noise.
Yunkai Zhang,
James C H Chu,
Wenchien Hsi,
Atif J Khan,
Parthiv S Mehta,
Damian B Bernard,
Ross A Abrams
Department of Radiation Oncology, Rush University Medical Center, Chicago, IL.
We evaluated 4 volume-based automatic image registration algorithms from 2 commercially available treatment planning systems (Philips Syntegra and BrainScan). The algorithms based on cross correlation (CC), local correlation (LC), normalized mutual information (NMI), and BrainScan mutual information (BSMI) were evaluated with: (1) the synthetic computed tomography (CT) images,(2) the CT and magnetic resonance (MR) phantom images, and (3) the CT and MR head image pairs from 12 patients with brain tumors. For the synthetic images, the registration results were compared with known transformation parameters, and all algorithms achieved accuracy of submillimeter in translation and subdegree in rotation. For the phantom images, the registration results were compared with those provided by frame and marker-based manual registration. For the patient images, the results were compared with anatomical landmark-based manual registration to qualitatively determine how the results were close to a clinically acceptable registration. NMI and LC outperformed CC and BSMI, with the sense of being closer to a clinically acceptable result. As for the robustness, NMI and BSMI outperformed CC and LC. A guideline of image registration in our institution was given, and final visual assessment is necessary to guarantee reasonable results.
Ganesh Narayanasamy,
Gerald L LeCarpentier,
Marilyn Roubidoux,
J Brian Fowlkes,
Anne F Schott,
Paul L Carson
Department of Radiology, and Applied Physics Program, University of Michigan, Ann Arbor Michigan 48109, USA.
The purpose of this study was to evaluate the potential for use of image volume based registration (IVBaR) to aid in measurement of changes in the tumor during chemotherapy of breast cancer. Successful IVBaR could aid in the detection of such changes in response to neoadjuvant chemotherapy and potentially be useful for routine breast cancer screening and diagnosis. IVBaR was employed in a new method of automated estimation of tumor volume in studies following the radiologist identification of the tumor region in the prechemotherapy scan. The authors have also introduced a new semiautomated method for validation of registration based on Doppler ultrasound (U.S.) signals that are independent of the grayscale signals used for registration. This Institutional Review Board approved study was conducted on 10 patients undergoing chemotherapy and 14 patients with a suspicious/unknown mass scheduled to undergo biopsy. Reasonably reproducible mammographic positioning and nearly whole breast U.S. imaging were achieved. The image volume was registered offline with a mutual information cost function and global interpolation based on a thin-plate spline using MIAMI FUSE software developed at the University of Michigan. The success and accuracy of registration of the three dimensional (3D) U.S. image volume were measured by means of mean registration error (MRE). IVBaR was successful with MRE of 4.3 +/- 1.7 mm in 9 out of 10 reproducibility automated breast ultrasound (ABU) studies and in 12 out of 17 ABU image pairs collected before, during, or after 115 +/- 14 days of chemotherapy. Semiautomated tumor volume estimation was performed on registered image volumes giving 86 +/- 8% mean accuracy compared to the radiologist hand-segmented tumor volume on seven cases. Doppler studies yielded fractional volume of color pixels in the region surrounding the lesion and its change with changing breast compression. The Doppler study of patients with detectable blood flow included five patients with suspicious masses and three undergoing chemotherapy. Spatial alignment of the 3D blood vessel data from the Doppler studies provided independent measures for the validation of registration. In 15 Doppler image volume pairs scanned with differing breast compression, the mean centerline separation value was 1.5 +/- .6 mm, while MRE based on a few identifiable structural points common to the two grayscale image volumes was 1.1 +/- .6 mm. Another measure, the overlap ratio of blood vessels, was shown to increase from .32 to .59 (+84%) with IVBaR for pairs at various compression levels. These results show that successful registration of ABU scans may be accomplished for comparison and integration of information.
In this paper, an automatic histogram threshold approach based on a fuzziness measure is presented. This work is an improvement of an existing method. Using fuzzy logic concepts, the problems involved in finding the minimum of a criterion function are avoided. Similarity between gray levels is the key to find an optimal threshold. Two initial regions of gray levels, located at the boundaries of the histogram, are defined. Then, using an index of fuzziness, a similarity process is started to find the threshold point. A significant contrast between objects and background is assumed. Previous histogram equalization is used in small contrast images. No prior knowledge of the image is required.
Wolfgang Birkfellner,
Markus Stock,
Michael Figl,
Christelle Gendrin,
Johann Hummel,
Shuo Dong,
Joachim Kettenbach,
Dietmar Georg,
Helmar Bergmann
Center for Biomedical Engineering and Physics, Medical University Vienna, Waehringer Guertel 18-20 AKH 4L, A-1090 Vienna, Austria. wolfgang.birkfellner@meduniwien.ac.at
In this article, the authors evaluate a merit function for 2D/3D registration called stochastic rank correlation (SRC). SRC is characterized by the fact that differences in image intensity do not influence the registration result; it therefore combines the numerical advantages of cross correlation (CC)-type merit functions with the flexibility of mutual-information-type merit functions. The basic idea is that registration is achieved on a random subset of the image, which allows for an efficient computation of Spearman's rank correlation coefficient. This measure is, by nature, invariant to monotonic intensity transforms in the images under comparison, which renders it an ideal solution for intramodal images acquired at different energy levels as encountered in intrafractional kV imaging in image-guided radiotherapy. Initial evaluation was undertaken using a 2D/3D registration reference image dataset of a cadaver spine. Even with no radiometric calibration, SRC shows a significant improvement in robustness and stability compared to CC. Pattern intensity, another merit function that was evaluated for comparison, gave rather poor results due to its limited convergence range. The time required for SRC with 5% image content compares well to the other merit functions; increasing the image content does not significantly influence the algorithm accuracy. The authors conclude that SRC is a promising measure for 2D/3D registration in IGRT and image-guided therapy in general.
Brigham and Women's Hospital, Harvard Medical School, USA. mt@bwh.harvard.edu
We propose a novel Bayesian registration formulation in which image location is represented as a latent random variable. Location is marginalized to determine the maximum a priori (MAP) transform between images, which results in registration that is more robust than the alternatives of omitting locality (i.e. global registration) or jointly maximizing locality and transform (i.e. iconic registration). A mathematical link is established between the Bayesian registration formulation and the mutual information (MI) similarity measure. This leads to a novel technique for selecting informative image regions for registration, based on the MI of image intensity and spatial location. Experimental results demonstrate the effectiveness of the marginalization formulation and the MI-based region selection technique for ultrasound (US) to magnetic resonance (MR) registration in an image-guided neurosurgical application.
Lo Kwee-Seong Medical Image Analysis Laboratory, Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong. liaoshu@cse.ust.hk
Non-rigid image registration is a challenging task in medical image analysis. In recent years, there are two essential issues. First, intensity similarity is not necessarily equivalent to anatomical similarity when the anatomical correspondences between subject and template images are established. Second, the registration algorithm should be robust against monotonic gray-level transformation when aligning anatomical structures in the presence of bias fields. In this paper, a new feature based non-rigid registration method is proposed to deal with these two problems. The proposed method is based on a new type of image feature, called Uniform Spherical Structure Pattern (USSP). USSP encodes voxel-wise interaction information and geometric properties of anatomical structures. It is computationally efficient, rotation invariant and theoretically monotonic gray-level transformation invariant. The USSP feature is integrated with the Markov random field (MRF) discrete labeling framework to define energy function for registration in this paper. If the segmentation results are available, explicit anatomical correspondence can be established as an additional energy term. The energy function is optimized via the alpha-expansion algorithms. The proposed method is compared with three widely used non-rigid registration methods on both simulated and real databases obtained from BrainWeb and IBSR. Experimental results demonstrate that the proposed method achieves the highest registration accuracy among all the compared methods.
A new feature based non-rigid image registration method for magnetic resonance (MR) brain images is presented in this paper. Each image voxel is represented by a rotation invariant feature vector, which is computed by passing the input image volumes through a new bank of symmetric alpha stable (S$\alpha$S) filters. There are three main contributions presented in this paper. First, this work is motivated by the fact that the frequency spectrums of the brain MR images often exhibit non- Gaussian heavy tail behavior which cannot be satisfactorily modeled by the conventional Gabor filters. To this end, we propose the use of S$\alpha$S filters to model such behavior and show that the Gabor filter is a special case of the S$\alpha$S filter. Second, the maximum response orientation (MRO) selection criterion is designed to extract rotation invariant features for registration tasks. The MRO selection criterion also significantly reduces the number of dimensions of feature vectors and therefore lowers the computation time. Third, in case the segmentations of the input image volumes are available, the Fisher's separation criterion (FSC) is introduced such that the discriminating power of different feature types can be directly compared with each other before performing the registration process. Using FSC, weights can also be assigned automatically to different voxels in the brain MR images. The weight of each voxel determined by FSC reflects how distinctive and salient the voxel is. Using the most distinctive and salient voxels at the initial stage to drive the registration can reduce the risk of being trapped in the local optimum during image registration process. The larger the weight, the more important the voxel. With the extracted feature vectors and the associated weights, the proposed method registers the source and the target images in a hierarchical multi-resolution manner. The proposed method has been intensively evaluated on both simulated and real 3D datasets obtained from BrainWeb and IBSR respectively and compared with HAMMER, an extended version of HAMMER based on local histograms (LHF), FFD, Demons and the Gabor filter based registration method. It is shown that the proposed method achieves the highest registration accuracy among the five widely used image registration methods.
We introduce a new measure of image similarity called the Complex Wavelet Structural Similarity (CW-SSIM) index and show its applicability as a general purpose image similarity index. The key idea behind CW-SSIM is that certain image distortions lead to consistent phase changes in the local wavelet coefficients, and that a consistent phase shift of the coefficients does not change the structural content of the image. By conducting four case studies, we have demonstrated the superiority of the CW-SSIM index against other indices (e.g., Dice, Hausdorff distance) commonly used for assessing the similarity of a given pair of images. In addition, we show that the CW-SSIM index has a number of advantages. It is robust to small rotations and translations. It provides useful comparisons even without a pre-processing image registration step, which is essential for other indices. Moreover, it is computationally less expensive.
Uncertainties in image registration may be a significant source of errors in anatomy mapping as well as dose accumulation in radiotherapy. It is therefore essential to validate the accuracy of image registration. Here we propose a method to detect areas where mono modal b-spline registration performs well and to distinguish those from areas of the same image, where the registration is likely to be less accurate. It is a stochastic approach to automatically estimate the uncertainty of the resulting displacement vector field. The coefficients resulting from the b-spline registration are subject to moderate and randomly performed variations. A quantity is proposed to characterize the local sensitivity of the similarity measure to these variations. We demonstrate the statistical dependence between the local image registration error and this quantity by calculating their mutual information. We show the significance of the statistical dependence with an approach based on random re-distributions. The proposed method has the potential to divide an image into sub-regions which differ in the magnitude of their average registration error.
