Digital Image Processing Lab

Department of Radiology, University of Michigan


MIAMI Fuse: Mutual Information for Automatic Multimodality Image Fusion

An Application of MI-based Automated Registration:

Basic Neurosciences

Boklye Kim1, Jennifer L. Boes1, Kirk A. Frey 2, Charles R. Meyer1
1Department of Radiology, University of Michigan Medical Center
2Department of Internal Medicine, Division of Nuclear Medicine, University of Michigan Medical Center


This work is presented in full in Neuroimage, Volume 5, 1997. The full citation and abstract are on-line.


Figure 1

Rat brain - block face and autoradiography Figure 1. (37K color, 366x233 gif)

Figure 1 displays the reference video image (uncut specimen block face) (left) and the 2DG autoradiograph (right) deformed due to specimen sectioning with control points selected in both images. Blue spheres in the reference (left) and in the autoradiograph (right) indicate the homologous control points used for the starting vector in TPS registration, and the pink spheres shown in the autoradiograph image indicate the final locations of the points achieved after minimization of MI. The figure illustrates the definition of non-feature based control points by designating the points in an approximate elliptical pattern. The same sets of control points, defined once for a slice image pair, are used for all subsequent slices constituting the specimen block's volume.

Figure 2

Rat brain - affine vs warp Figure 2. (89K color, 374x442 gif)

Figure 2 displays the result of an automatic registration of an autoradiograph slice image in reference to the video block face image by affine (top) and thin-plate-spline transformation (bottom) using the MI cost function. The registered AR images resulting from the two methods - full affine (top right) and TPS warping (bottom right) - are presented by overlaying the edge-enhanced AR with the reference video for visual inspection of the mapping quality. Mismatches are observed as a result of full affine transform (top right), including the corpus callosum and edges around the hippocampus and auditory cortex in the ipsilateral hemisphere. Improved matching by the warping transform is observed in the figure at bottom right. The resulting MIs demonstrate the better registration: -1.065 (full affine) versus -1.150 (TPS warp). The final solutions of the affine and TPS warp are graphically depicted by grid lines in the figures in the left column.



The registrations were performed using our "MIAMI Fuse" (Mutual Information for Automatic Multimodality Image Fusion) software as described fully in Meyer, et al..