An important part of image-guided radiation therapy or surgery is registration of a three-dimensional (3D) preoperative image to two-dimensional (2D) images of the patient. It is expected that the accuracy and robustness of a 3D/2D image registration method do not depend solely on the registration method itself but also on the number and projections (views) of intraoperative images. In this study, we systematically investigate these factors by using registered image data, comprising of CT and X-ray images of a cadaveric lumbar spine phantom and the recently proposed 3D/2D registration method. The results indicate that the proportion of successful registrations (robustness) significantly increases when more X-ray images are used for registration.

Purpose: A promising patient positioning technique is based on registering computed tomographic (CT) or magnetic resonance (MR) images to cone-beam CT images (CBCT). The extra radiation dose delivered to the patient can be substantially reduced by using fewer projections. This approach results in lower quality CBCT images. The purpose of this study is to evaluate a number of similarity measures (SMs) suitable for registration of CT or MR images to low-quality CBCTs. Methods and Materials: Using the recently proposed evaluation protocol, we evaluated nine SMs with respect to pretreatment imaging modalities, number of two-dimensional (2D) images used for reconstruction, and number of reconstruction iterations. The image database consisted of 100 X-ray and corresponding CT and MR images of two vertebral columns. Results: Using a higher number of 2D projections or reconstruction iterations results in higher accuracy and slightly lower robustness. The similarity measures that behaved the best also yielded the best registration results. The most appropriate similarity measure was the asymmetric multi-feature mutual information (AMMI). Conclusions: The evaluation protocol proved to be a valuable tool for selecting the best similarity measure for the reconstruction-based registration. The results indicate that accurate and robust CT/CBCT or even MR/CBCT registrations are possible if the AMMI similarity measure is used.

In image-guided therapy, high-quality preoperative images serve for planning and simulation, and intraoperatively as "background", onto which models of surgical instruments or radiation beams are projected. The link between a preoperative image and intraoperative physical space of the patient is established by image-to-patient registration. In this paper, we present a novel 3-D/2-D registration method. First, a 3-D image is reconstructed from a few 2-D X-ray images and next, the preoperative 3-D image is brought into the best possible spatial correspondence with the reconstructed image by optimizing a similarity measure (SM). Because the quality of the reconstructed image is generally low, we introduce a novel SM, which is able to cope with low image quality as well as with different imaging modalities. The novel 3-D/2-D registration method has been evaluated and compared to the gradient-based method (GBM) using standardized evaluation methodology and publicly available 3-D computed tomography (CT), 3-D rotational X-ray (3DRX), and magnetic resonance (MR) and 2-D X-ray images of two spine phantoms, for which gold standard registrations were known. For each of the 3DRX, CT, or MR images and each set of X-ray images, 1600 registrations were performed from starting positions, defined as the mean target registration error (mTRE), randomly generated and uniformly distributed in the interval of 0-20 mm around the gold standard. The capture range was defined as the distance from gold standard for which the final TRE was less than 2 mm in at least 95% of all cases. In terms of success rate, as the function of initial misalignment and capture range the proposed method outperformed the GBM. TREs of the novel method and the GBM were approximately the same. For the registration of 3DRX and CT images to X-ray images as few as 2-3 X-ray views were sufficient to obtain approximately 0.4 mm TREs, 7-9 mm capture range, and 80%-90% of successful registrations. To obtain similar results for MR to X-ray registrations, an image, reconstructed from at least 11 X-ray images was required. Reconstructions from more than 11 images had no effect on the registration results.

Evaluation and comparison of registration techniques for image-guided surgery is an important problem that has received little attention in the literature. In this paper we address the challenging problem of generating reliable "gold standard" data for use in evaluating the accuracy of 3D/2D registrations. We have devised a cadaveric lumbar spine phantom with fiducial markers and established highly accurate correspondences between 3D CT and MR images and 18 2D X-ray images. The expected target registration errors for target points on the pedicles are less than 0.26 mm for CT-to-X-ray registration and less than 0.42 mm for MR-to-X-ray registration. As such, the "gold standard" data, which has been made publicly available on the Internet (http://lit.fe.uni-lj.si/Downloads/downloads.asp), is useful for evaluation and comparison of 3D/2D image registration methods.

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91%(82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.

We propose a completely automated algorithm for the detection of the spinal centreline and the centres of vertebral bodies and intervertebral discs in images acquired by computed tomography (CT) and magnetic resonance (MR) imaging. The developed methods are based on the analysis of the geometry of spinal structures and the characteristics of CT and MR images and were evaluated on 29 CT and 13 MR images of lumbar spine. The overall mean distance between the obtained and the ground truth spinal centrelines and centres of vertebral bodies and intervertebral discs were 1.8 +/- 1.1 mm and 2.8 +/- 1.9 mm, respectively, and no considerable differences were detected among the results for CT, T(1)-weighted MR and T(2)-weighted MR images. The knowledge of the location of the spinal centreline and the centres of vertebral bodies and intervertebral discs is valuable for the analysis of the spine. The proposed method may therefore be used to initialize the techniques for labelling and segmentation of vertebrae.

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.

In the past, a number of methods were proposed for quantitative assessment of vertebral rotation from three-dimensional (3D) images. However, these methods were based on manual identification of distinctive anatomical landmarks, required manual determination of cross-sections from 3D images, and measured only axial vertebral rotation instead of the rotation in 3D. In this paper, we propose an automated method for quantitative assessment of vertebral rotation in 3D that is based on finding the planes of vertebral symmetry by matching image intensity gradients on both sides of each plane. The method was evaluated on 28 images of normal and pathological vertebrae, obtained by computed tomography (CT) and magnetic resonance (MR). For each vertebra, final angle displacements of 200 initial angle displacements, uniformly distributed within 30 degrees from manually obtained reference angles, were obtained. The results show that by the proposed method, vertebral rotation can be successfully estimated in 3D with an average accuracy of 1.0 degrees and precision of 0.5 degrees.

In this paper, a novel method for EEG to MRI registration is proposed. Initial registration is achieved by extracting and matching symmetry planes of MRI and EEG data, followed by iterative registration based on minimizing a cost function. Comparison of the intensity distributions of the whole MR image and MRI voxels around a head surface point yields global similarities, while the comparison of intensity distributions of MRI voxels around corresponding EEG points, which reflects the head's sagittal symmetry, yields local similarities. Therefore, when the EEG points are registered to the MR image, maximal global and local similarities should be obtained. The cost function, incorporating global and local similarities, was the sum of Kullback-Leibler divergences between corresponding intensity distributions. The proposed method was evaluated on clinical MRI data with simulated EEG data, yielding mean registration error of 0.48 +/- 0.33 mm, while with real EEG data an average root-mean-square point-to-surface error of 2.27 +/- 0.02 mm was obtained.

The purpose of this study is to present a framework for quantitative analysis of spinal curvature in 3D. In order to study the properties of such complex 3D structures, we propose two descriptors that capture the characteristics of spinal curvature in 3D. The descriptors are the geometric curvature (GC) and curvature angle (CA), which are independent of the orientation and size of spine anatomy. We demonstrate the two descriptors that characterize the spinal curvature in 3D on 30 computed tomography (CT) images of normal spine and on a scoliotic spine. The descriptors are determined from 3D vertebral body lines, which are obtained by two different methods. The first method is based on the least-squares technique that approximates the manually identified vertebra centroids, while the second method searches for vertebra centroids in an automated optimization scheme, based on computer-assisted image analysis. Polynomial functions of the fourth and fifth degree were used for the description of normal and scoliotic spinal curvature in 3D, respectively. The mean distance to vertebra centroids was 1.1 mm (+/-0.6 mm) for the first and 2.1 mm (+/-1.4 mm) for the second method. The distributions of GC and CA values were obtained along the 30 images of normal spine at each vertebral level and show that maximal thoracic kyphosis (TK), thoracolumbar junction (TJ) and maximal lumbar lordosis (LL) on average occur at T3/T4, T12/L1 and L4/L5, respectively. The main advantage of GC and CA is that the measurements are independent of the orientation and size of the spine, thus allowing objective intra- and inter-subject comparisons. The positions of maximal TK, TJ and maximal LL can be easily identified by observing the GC and CA distributions at different vertebral levels. The obtained courses of the GC and CA for the scoliotic spine were compared to the distributions of GC and CA for the normal spines. The significant difference in values indicates that the descriptors of GC and CA may be used to detect and quantify scoliotic spinal curvatures. The proposed framework may therefore improve the understanding of spine anatomy and aid in the clinical quantitative evaluation of spinal deformities.

ABSTRACT: BACKGROUND: Solving bioinformatics tasks often requires extensive computational power. Recent trends in processor architecture combine multiple cores into a single chip to improve overall performance. The Cell Broadband Engine (CBE), a heterogeneous multi-core processor, provides power-efficient and cost-effective high-performance computing. One application area is image analysis and visualisation, in particular registration of 2D cross-sections into 3D image datasets. Such techniques can be used to put different image modalities into spatial correspondence, for example, 2D images of histological cuts into morphological 3D frameworks. RESULTS: We evaluate the CBE-driven PlayStation 3 as a high performance, cost-effective computing platform by adapting a multimodal alignment procedure to several characteristic hardware properties. The optimisations are based on partitioning, vectorisation, branch reducing and loop unrolling techniques with special attention to 32-bit multiplies and limited local storage on the computing units. We show how a typical image analysis and visualisation problem, the multimodal registration of 2D cross-sections and 3D datasets, benefits from the multi-core based implementation of the alignment algorithm. We discuss several CBE-based optimisation methods and compare our results to standard solutions. More information and the source code are available from http://cbe.ipk-gatersleben.de. CONCLUSIONS: The results demonstrate that the CBE processor in a PlayStation 3 accelerates computational intensive multimodal registration, which is of great importance in biological/medical image processing. The PlayStation 3 as a low cost CBE-based platform offers an efficient option to conventional hardware to solve computational problems in image processing and bioinformatics.

PURPOSE: This paper is concerned with the reconstruction of vascular trees from few projections using discrete tomography. However, its computational cost is high and it lacks robustness when the data are inconsistent. We improve robustness by incorporating an intensity-based camera-correction method. The proposed approach is also capable of handling small motion artifacts by modeling them as repositionings of a virtual X-ray camera. We also present a parallel implementation which substantially reduces reconstruction time. METHODS: We propose a data-driven reduction of positional inconsistencies by minimizing the reconstruction residual to increase the robustness. Inspired by motion compen-sation algorithms in SPECT imaging, we combine an intensity-based 2D/3D-registration method with itera-tive reconstruction methods. Our objective is the robust vascular-tree reconstruction from positionally inconsistent data. The speed of the reconstruction is substantially increased by a volume-splitting scheme that allows parallel processing. RESULTS: Vascular trees in the liver can be accurately reconstructed from few positionally inconsistent projections using digitally reconstructed radiographs. We have tested the proposed method on synthetic projection data and on objects imaged with a new robotized C-arm. We measured a decrease in the average reconstruction residual of about 13% for real data compared to projection data without preprocessing. Over 4,600 reconstruction experiments were conducted to evaluate the speed-up obtained when employing the volume-splitting scheme. Reconstruction time decreased linearly with increased number of processor-cores, both for real and synthetic data. CONCLUSIONS: The proposed method reduces inconsistencies caused by positioning errors and small motion artifacts. No prior segmentation or detection of correspondences between projections is necessary, because all algorithms are intensity-based. As a result, the proposed method allows for robust, high-quality reconstructions, while reducing radiation dose substantially.

OBJECTIVE: Radio frequency ablation (RFA) can be used to treat liver cancer minimally invasively by depositing energy from the RF probe placed in the center of the tumor. The procedure relies on pre-operative imaging (typically MRI or CT) for the interventional planning and ultrasound (US) for intra-operative guidance during needle insertion. Visual presentation of co-registered pre- and intra-operative images would help to improve the navigation during the needle positioning phase. METHODS: In the present study, we compared six registration methods using different similarity metrics: two versions of the correlation ratio, bivariate correlation ratio, and conventional normalized mutual information and correlation coefficient. The accuracy, robustness and speed were assessed by computing rigid registrations between eight pairs of the MR and freehand 3D US datasets. RESULTS: The correlation ratio computed on the MR-gradient-norm and US images outperformed other similarity metrics in terms of robustness (40-82%) and demonstrated average accuracy (0.32 degrees , 0.69 mm) which is clinically acceptable for the RFA of liver cancer. CONCLUSIONS: We observed that the performance of all similarity metrics is largely dependent on the quality of the US images, sufficient field of view of the reconstructed 3D US and absence of motion artifacts.

PURPOSE: Robust and accurate automated co-registration of the coronary arteries in 3D CTA and 2D X-ray angiography during percutaneous coronary interventions (PCI), in order to present a fused visualization. METHODS: A novel vesselness-based similarity measure was developed, that avoids an explicit segmentation of the X-ray image. A stochastic optimizer searches the optimal registration using the similarity measure. RESULTS: Both simulated data and clinical data were used to investigate the accuracy and capture range of the proposed method. The experiments show that the proposed method outperforms the iterative closest point method in terms of accuracy (average residual error of 0.42 mm vs. 1.44 mm) and capture range (average 71.1 mm/20.3 degrees vs. 14.1 mm/5.2 degrees ). CONCLUSION: The proposed method has proven to be accurate and the capture range is ample for usage in PCI. Especially the absence of an explicit segmentation of the interventionally acquired X-ray images considerably aids the robustness of the method.

PURPOSE: This paper describes an approach for the three-dimensional (3D) shape and pose reconstruction of the human rib cage from few segmented two-dimensional (2D) projection images. Our work is aimed at supporting temporal subtraction techniques of subsequently acquired radiographs by establishing a method for the assessment of pose differences in sequences of chest radiographs of the same patient. METHODS: The reconstruction method is based on a 3D statistical shape model (SSM) of the rib cage, which is adapted to binary 2D projection images of an individual rib cage. To drive the adaptation we minimize a distance measure that quantifies the dissimilarities between 2D projections of the 3D SSM and the projection images of the individual rib cage. We propose different silhouette-based distance measures and evaluate their suitability for the adaptation of the SSM to the projection images. RESULTS: An evaluation was performed on 29 sets of biplanar binary images (posterior-anterior and lateral). Depending on the chosen distance measure, our experiments on the combined reconstruction of shape and pose of the rib cages yield reconstruction errors from 2.2 to 4.7mm average mean 3D surface distance. Given a geometry of an individual rib cage, the rotational errors for the pose reconstruction range from 0.1 degrees to 0.9 degrees . CONCLUSIONS: The results show that our method is suitable for the estimation of pose differences of the human rib cage in binary projection images. Thus, it is able to provide crucial 3D information for registration during the generation of 2D subtraction images.

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well handled. In 2D image-based registration, self-occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based nonrigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based,"direct" data term for registration. Self-occlusions are detected as shrinkage areas in the 2D warp. Experimental results on several challenging data sets show that our approach successfully registers images with self-occlusions while effectively detecting the self-occluded regions.

A novel similarity measure for registering magnetic resonance (MR) and computed tomography (CT) images has been designed and built. MR-CT registration methods often rely on the statistical intensity relationship between the images. The proposed similarity measure instead depends on the statistical relationship between the complex phase order between the images. By utilizing the complex phase order likelihood (CPOL) as a similarity measure, structural relationships instead of intensity relationships are explicitly used. This approach can be advantageous for MR-CT registration, where the intensities of the CT imagery have highly complex and nonlinear relationships with the intensities of corresponding MR imagery but simpler linear structural relationships. This new similarity measure has been tested on real MR-CT 3D volumes and has been evaluated based on fiducial registration error to determine alignment accuracy. Quantitative results show that CPOL is capable of achieving comparable alignment accuracy when compared to normalized mutual information, while being more robust to imaging artifacts such as noise.

The problem of reassembling image fragments arises in many scientific fields, such as forensics and archaeology. In the field of archaeology, the pictorial excavation findings are almost always in the form of painting fragments. The manual execution of this task is very difficult, as it requires great amount of time, skill and effort. Thus, the automation of such a work is very important and can lead to faster, more efficient, painting reassembly and to a significant reduction in the human effort involved. In this paper, an integrated method for automatic color based 2D image fragment reassembly is presented. The proposed 2D reassembly technique is divided into four steps. Initially, the image fragments which are probably spatially adjacent, are identified utilizing techniques employed in content based image retrieval systems. The second operation is to identify the matching contour segments for every retained couple of image fragments, via a dynamic programming technique. The next step is to identify the optimal transformation in order to align the matching contour segments. Many registration techniques have been evaluated to this end. Finally, the overall image is reassembled from its properly aligned fragments. This is achieved via a novel algorithm, which exploits the alignment angles found during the previous step. In each stage, the most robust algorithms having the best performance are investigated and their results are fed to the next step. We have experimented with the proposed method using digitally scanned images of actual torn pieces of paper image prints and we produced very satisfactory reassembly results.

Forensic terrestrial photogrammetry is one of the most valuable and low-cost resources of spatial data available today. Due to the ephemeral crime scene characteristics, these photographs can often capture information that is never to be seen again. This paper presents a novelty approach for the documentation, analysis, and visualization of crime scenes for which only a single perspective image is available. The photogrammetric process consists of a few well-known steps in close-range photogrammetry: features extraction, vanishing points computation, camera self-calibration, 3D metric reconstruction, dimensional analysis, and interactive visualization. Likewise, the method incorporates a quality control of the different steps accomplished sequentially. As a result, several cases of study are presented in the experimental results section in order to test their viability. The full approach can be applied easily through the free software, sv3DVision, which has been evaluated by a number of police officers, forensic scientists, and forensic educators satisfactorily.

In this article, the authors evaluate a merit function for 2D/3D registration called stochastic rank correlation (SRC). SRC is characterized by the fact that differences in image intensity do not influence the registration result; it therefore combines the numerical advantages of cross correlation (CC)-type merit functions with the flexibility of mutual-information-type merit functions. The basic idea is that registration is achieved on a random subset of the image, which allows for an efficient computation of Spearman's rank correlation coefficient. This measure is, by nature, invariant to monotonic intensity transforms in the images under comparison, which renders it an ideal solution for intramodal images acquired at different energy levels as encountered in intrafractional kV imaging in image-guided radiotherapy. Initial evaluation was undertaken using a 2D/3D registration reference image dataset of a cadaver spine. Even with no radiometric calibration, SRC shows a significant improvement in robustness and stability compared to CC. Pattern intensity, another merit function that was evaluated for comparison, gave rather poor results due to its limited convergence range. The time required for SRC with 5% image content compares well to the other merit functions; increasing the image content does not significantly influence the algorithm accuracy. The authors conclude that SRC is a promising measure for 2D/3D registration in IGRT and image-guided therapy in general.