TurboReg

An ImageJ plugin for the automatic alignment of a source image or a stack to a target image.

Philippe Thévenaz, Biomedical Imaging Group, École polytechnique fédérale de Lausanne (EPFL)

Figure 1. Before (left) and after (right) having applied the alignment procedure to an eye fundus.

General Description

The purpose of this plugin is to register—in other words, to align or to match—two images, one of them being called the source image and the other the target image. Three alignment modes are available: manual, automatic, and batch. In all three cases, the user is given the opportunity to interactively specify some landmarks, which establishes the initial correspondence between the images. In the automatic and batch modes, the landmarks of the source image are then automatically refined to better match those of the target image. In the manual mode, this automatic refinement procedure is disengaged. Most of the time, the automatic algorithm requires no user input because it is robust enough that the default initial conditions are sufficiently accurate.

After completion of the registration process, the plugin uses the final position of the source and target landmarks to create a warped image that has the size of the target and that contains a distorted version of the source. The distortion is such that the landmarks of the source are mapped to those of the target. In the automatic and batch modes, the landmarks of the source have been refined to minimize the mean-square difference between the target and the warped image.

The plugin can also be called by a macro or by another plugin. In the latter case, the registration can proceed silently if desired, and the registration results can be retrieved for further processing.

I. Download

This distribution is dated July 7, 2011. It includes the complete set of source files, along with the precompiled classes and the application programming interface.

• ZIP distribution
• source file
• API

II. Related Work

This set of Java classes is based on the following paper:
P. Thévenaz, U.E. Ruttimann, M. Unser, "A Pyramid Approach to Subpixel Registration Based on Intensity," IEEE Transactions on Image Processing, vol. 7, no. 1, pp. 27-41, January 1998. Other relevant on-line publications are available at http://bigwww.epfl.ch/publications/.

It is written as a plugin for ImageJ. Please read the ImageJ documentation to learn how to install plugins.

III. Technical Explanations

Five types of distortion are considered:

1. Translation. Upon translation, a straight line is mapped to a straight line of identical orientation, with conservation of the distance between any pair of points. A single landmark in each image gives a complete description of a translation. The mapping is of the form x = u + Δu.
2. Rigid Body. Upon rigid-body transformation, the distance between any pair of points is conserved. A single landmark is necessary to describe the translational component of the rigid-body transformation, while the rotational component is given by an angle. The mapping is of the form x = { {cos θ, −sin θ}, {sin θ, cos θ} } ⋅ u + Δu.
3. Scaled rotation. Upon scaled rotation, a straight line is mapped to a straight line; moreover, the angle between any pair of lines is conserved (this is sometimes called a conformal mapping). A pair of landmarks in each image is needed to give a complete description of a scaled rotation. The mapping is of the form x = λ { {cos θ, −sin θ}, {sin θ, cos θ} } ⋅ u + Δu.
4. Affine. Upon affine transformation, a straight line is mapped to a straight line, with conservation of flat angles between lines (parallel or coincident lines remain parallel or coincident). In 2D, a simplex—three landmarks—in each image is needed to give a complete description of an affine transformation. The mapping is of the form x = { {a₁₁, a₁₂}, {a₂₁, a₂₂} } ⋅ u + Δu.
5. Bilinear. Upon bilinear transformation, a straight line is mapped to a conic section. In 2D, four landmarks in each image are needed to give a complete description of a bilinear transformation. The mapping is of the form x = { {a₁₁, a₁₂}, {a₂₁, a₂₂} } ⋅ u + b u₁ u₂ + Δu.

The position of the landmarks can be stored and retrieved. However, the size of the source and target images is checked upon landmark retrieval; a mismatch is disallowed.

The computations may favor either speed or accuracy. If speed is favored, then one of several consequences is that the warped image is computed from the source by nearest-neighbor interpolation. If accuracy is favored, then one of several consequences is that cubic-spline interpolation is performed instead.

The plugin can accommodate grayscale images and RBG stacks. Only one color plane of the RGB stack is taken into account for registration; the transformation applied to the two remaining color planes will be adjusted accordingly.

Manual and automatic modes: if a stack of grayscale images is selected, only the first slice is registered. The second slice, if any, acts as a boolean mask, with the following convention: the presence of a zero value in the second slice of either image indicates that the data [at the same location in the first slice of both images] should not be considered for automatic alignment; the presence of values that are jointly nonzero in the second slice of both images indicates that the data [at the same location in the first slice of both images] are relevant and should indeed participate in the mean-square computation. By convention, the data outside the initial image frames are considered as irrelevant, while every pixel within the initial image frames is considered relevant.

When the source image is an RGB stack, the output image that results from the registration process is returned as an RGB color image. Else, it is returned as a stack where the first slice contains the source image after the warping has been applied, and where the second slice contains the source mask after warping. A nearest-neighbor interpolation is used for the source mask. The output type is either RGB or float 32-bit. To know the exact domain where the mean-square criterion has been applied, the user must himself perform an "AND" operation between the mask (if any) of the target image on one hand, and the returned mask of the warped source image on the other hand: the returned mask does not provide the explicit result of this operation, but only one of its two operands. RGB stacks do not accomodate masks.

Batch mode: the target image plays the same role in the batch mode as in the manual or automatic modes. If the target image is a stack, its second slice still defines a registration mask; else, every pixel of the target image is considered relevant. The source image, however, is treated differently. It has to be a stack for the batch mode to be enabled, and every slice gets registered in turn to the same target image. Each slice is given the same default mask, with every pixel being considered relevant. The output image that results from the registration process is returned as a stack. Each slice contains a warped version of the corresponding slice of the source image. As in the case of manual and automatic modes, the output type is float 32-bit. The batch mode is not available for RGB stacks.

In some situations, a single target is desirable because the source images differ essentially by their geometry, less so by their content. TurboReg is appropriate in those situations. But there also exist cases where the content of the several source images gradually evolves from frame to frame; this might happen for example with a time series, or with a 3D stack of slices that are acquired with only a loose control of the slice-to-slice alignment. A companion plugin has been written to help in this different settings; it is named StackReg and is available here.

IV. User Manual

Moving a point

Figure 2. Tool to move points.

To select a point, click anywhere in the source or target image. The point closest to where you clicked will be highlighted. To move points, drag within the image. The point closest to where you clicked will be moved to the position where you release the mouse. Alternatively, you can use the keyboard arrow keys to get a finer control over the position of the landmarks. Hitting the tab key will change the selection of the current point. Note: it is not possible to move a point outside of the image frame.

For the rigid-body transformation, there are two types of landmark. The green cross determines the translation. The other two landmarks are used to determine the orientation only, as indicated by the blue and brown lines.

Image magnification/minification

Figure 3. Zooming tool.

To magnify an image, select the magnification tool and click in the image. To minify an image, use the same tool with the <Control> keyboard modifier.

Dialog box

Figure 4. Dialog box.

This dialog box will appear if at least two grayscale images or RGB stacks are available at the time when TurboReg is launched. If that is not the case, a message will inform you of this requirement.

The target image that is currently selected determines the reference to which the source image will be aligned/matched/registered. If you decide to select as target the current source, your new selection will be honored, but the source image will be changed as well to reflect another arbitrary choice (it is not possible to register an image to itself).

The source image that is currently selected determines which image will be aligned/matched/registered to the target. If you decide to select as source the current target, your new selection will be honored, but the target image will be changed as well to reflect another arbitrary choice (it is not possible to register an image to itself).

When selecting another target or source, or when TurboReg is launched, you may notice some activity in the progress bar of ImageJ. This activity notifies you that some background preprocessing is going on. You can safely ignore it and proceed (you don't have to wait until this preprocessing is complete if you want to perform additional operations). The same applies when switching the distortion type from Bilinear, or to Bilinear.

There are five different types of distortion: Translation, Rigid Body, Scaled Rotation, Affine, and Bilinear; those correspond to 1, (1 + 2), 2, 3, and 4 landmarks, respectively. The landmarks are reset to their default position when changing the distortion type.

You can save the current configuration of landmarks in a text file by clicking the button [Save Now...]. The format of the file is rigid; it is nevertheless possible to edit its numerical fields if desired. This may allow you to overcome the limitation that forbids one to interactively move a point outside of the image frame.

You can restore a configuration of landmarks from a text file by clicking the button [Load...]. The configuration must have been previously created by TurboReg. A check on the current size of the target and source images is performed to ensure that the stored configuration makes sense with respect to the current images.

The automatic alignment mode is used to nudge the initial source landmarks, which results in refined landmarks. Those can be saved by checking the box [Save on Exit].

You can choose one of two different tradeoffs between registration speed and accuracy. When speed is favored, the output image suffers from low-quality interpolation (nearest-neighbor). Moreover, in the automatic registration mode, the accuracy is further reduced because the refinement of landmarks is made coarser. When accuracy is favored instead, the output image benefits from high-quality interpolation (cubic spline). Moreover, in the automatic registration mode, more effort is spent on refining the landmarks. Please note that it is not possible to perform automatic registration when speed is favored and when at least one of the dimensions of the target or source images is too small; in that case, manual registration is the only mode allowed (with the quality set to 'Fast'). Else, set the quality to 'Accurate'.

The [Credits...] button gives you basic information and provides a link to the present page.

Figure 5. Credits.

V. Macro

Some additional flexibility is available when calling TurboReg from ImageJ's macro language, which may be suitable in certain specialized situations. Most importantly, the landmark positions are directly accessible to further processing. Also, it is possible to specify cropping zones in the source and target images, which is useful when there is a reduced overlap between images, a situation that often arises when stitching together a mosaic of images.

Although TurboReg is scriptable, it is not recordable. Here is an example of a call to TurboReg from a macro:
run("TurboReg ", "-align -file path/source.tif 40 80 639 479 -file path/target.tif 0 0 639 479 -translation 320 240 331.7 210 -showOutput");
A fully developed and commented example is available here.

The full syntax follows one of the three cases below:

-help
1) Prints to the 'Results' window of ImageJ the full syntax of TurboReg, and quit.

-align <ALIGNMENT_OPTIONS>
2) Register a source image to a target image. The details of the command are given in <ALIGNMENT_OPTIONS>. This corresponds most closely to the automatic mode of the interactive version of TurboReg. The quality is always set to 'Accurate'.

-transform <TRANSFORMATION_OPTIONS>
3) Given a set of landmarks, transform a source image. The details of the command are given in <TRANSFORMATION_OPTIONS>. This corresponds most closely to the manual mode of the interactive version of TurboReg. The quality is always set to 'Accurate'.

VI. Plugin-to-Plugin

To call TurboReg from another plugin, we suggest the following Java statement:
    Object myTurboRegObject = IJ.runPlugIn("TurboReg_", myTurboRegOptions);
where myTurboRegOptions is a String object that follows exactly the same syntax as described in the Macro case. Methods of the returned object myTurboRegObject can then be accessed thanks to a reflection mechanism. This approach facilitates code maintenance, especially the independence of all plugins; moreover, it limits the risk of having duplicate class names inside the hierarchy of folders spanned by ImageJ.

There are three public methods of interest inside the class TurboReg_. Those are getSourcePoints, getTargetPoints, and getTransformedImage. Typical calls look like:
    Method method = myTurboRegObject.getClass().getMethod("getSourcePoints", null);
    double[][] mySourcePoints = (double[][])method.invoke(myTurboRegObject, null);
    method = myTurboRegObject.getClass().getMethod("getTargetPoints", null);
    double[][] myTargetPoints = (double[][])method.invoke(myTurboRegObject, null);
    method = myTurboRegObject.getClass().getMethod("getTransformedImage", null);
    ImagePlus myTransformedImage = (ImagePlus)method.invoke(myTurboRegObject, null);
where the necessary encapsulating blocks (e.g., try {}catch(){} ) have not been shown, for simplicity. The full description of these three methods and of their parameters is available from the API of TurboReg. A fully developed example of a plugin that takes advantage of the interoperability of TurboReg is StackReg.

VII. Application Programming Interface

A technical Javadoc documentation is available.

VIII. Known Bugs

On Mac OS, MRJ versions prior to 2.2.5 won't let you operate the plugin. To update MRJ, launch "Software Update" from the control panels and follow the instructions.

The dialog box is modeless, which means you have access to all functions of ImageJ. While some actions are safe (e.g., histogram computation), others may unfortunately lead to difficulties, such as:

• Launching TurboReg while another instance of TurboReg is already running results in disaster.
• Closing an image while the dialog of TurboReg is active results in an incoherent list of target and source images. Selecting the closed image results in disaster.
• Cropping or resizing the target or source image while the dialog of TurboReg is active may lead to unexpected behavior.
• Processing (say, smoothing) the target or source image while the dialog of TurboReg is active may sometimes lead to the registration of the unprocessed image. A workaround is to swap target for source, and back.
• Switching focus to a third image (neither source, nor target) doesn't restore the proper toolbar. Unexpected behavior may result when trying to toggle between tools.
• ...
• Many more: by now, I hope you get the idea of what is allowed and what is not. In case of doubt, act as if the dialog box would be modal instead of modeless. [Technical note: a modeless dialog box is necessary to allow for interactive access to the landmarks.]

IX. Conditions of Use

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

You undertake to include a citation or acknowledgment whenever you present or publish results that are based on this program. EPFL makes no warranties of any kind on this software and shall in no event be liable for damages of any kind in connection with the use and exploitation of this technology.