EPFL
 Biomedical Imaging GroupSTI
EPFL
  Download Algorithms
English only   BIG > Download Algorithms >TurboReg

 CONTENTS
 Home Page
 News & Events
 Members
 Publications
 Tutorials and Reviews
 Research
 Demos
 Download Algorithms
 TurboReg
 Red bullet Related Work
 Red bullet Technical Explanations
 Red bullet User Manual
 Red bullet Macro
 Red bullet Plugin-to-Plugin
 Red bullet API
 Red bullet Conditions of Use

 DOWNLOAD
 UNIX distribution
 Macintosh distribution
 PC distribution

TurboReg

An ImageJ plugin for the automatic alignment of a source image or a stack to a target image.

Philippe Thévenaz, Biomedical Imaging Group, Swiss Federal Institute of Technology Lausanne




UnregisteredRegistered

Figure 1. Before (left) and after (right) having applied the alignment procedure to an eye fundus.

General Description

The purpose of this plugin is to register—in other words, to align or to match—two images, one of them being called the source image and the other the target image. Three alignment modes are available: manual, automatic, and batch. In all three cases, the user is given the opportunity to interactively specify some landmarks, which establishes the initial correspondence between the images. In the automatic and batch modes, the landmarks of the source image are then automatically refined to better match those of the target image. In the manual mode, this automatic refinement procedure is disengaged. Most of the time, the automatic algorithm requires no user input because it is robust enough that the default initial conditions are sufficiently accurate.

After completion of the registration process, the plugin uses the final position of the source and target landmarks to create a warped image that has the size of the target and that contains a distorted version of the source. The distortion is such that the landmarks of the source are mapped to those of the target. In the automatic and batch modes, the landmarks of the source have been refined to minimize the mean-square difference between the target and the warped image.

The plugin can also be called by a macro or by another plugin. In the latter case, the registration can proceed silently if desired, and the registration results can be retrieved for further processing.


I. Download

This distribution is dated July 7, 2011. It includes the complete set of source files, along with the precompiled classes and the application programming interface.


II. Related Work

This set of Java classes is based on the following paper:
P. Thévenaz, U.E. Ruttimann, M. Unser, "A Pyramid Approach to Subpixel Registration Based on Intensity," IEEE Transactions on Image Processing, vol. 7, no. 1, pp. 27-41, January 1998. Other relevant on-line publications are available at http://bigwww.epfl.ch/publications/.

It is written as a plugin for ImageJ. Please read the ImageJ documentation to learn how to install plugins.


III. Technical Explanations

Five types of distortion are considered:

  1. Translation. Upon translation, a straight line is mapped to a straight line of identical orientation, with conservation of the distance between any pair of points. A single landmark in each image gives a complete description of a translation. The mapping is of the form x = u + Δu.
  2. Rigid Body. Upon rigid-body transformation, the distance between any pair of points is conserved. A single landmark is necessary to describe the translational component of the rigid-body transformation, while the rotational component is given by an angle. The mapping is of the form x = { {cos θ, −sin θ}, {sin θ, cos θ} } ⋅ u + Δu.
  3. Scaled rotation. Upon scaled rotation, a straight line is mapped to a straight line; moreover, the angle between any pair of lines is conserved (this is sometimes called a conformal mapping). A pair of landmarks in each image is needed to give a complete description of a scaled rotation. The mapping is of the form x = λ { {cos θ, −sin θ}, {sin θ, cos θ} } ⋅ u + Δu.
  4. Affine. Upon affine transformation, a straight line is mapped to a straight line, with conservation of flat angles between lines (parallel or coincident lines remain parallel or coincident). In 2D, a simplex—three landmarks—in each image is needed to give a complete description of an affine transformation. The mapping is of the form x = { {a11, a12}, {a21, a22} } ⋅ u + Δu.
  5. Bilinear. Upon bilinear transformation, a straight line is mapped to a conic section. In 2D, four landmarks in each image are needed to give a complete description of a bilinear transformation. The mapping is of the form x = { {a11, a12}, {a21, a22} } ⋅ u + b u1 u2 + Δu.

The position of the landmarks can be stored and retrieved. However, the size of the source and target images is checked upon landmark retrieval; a mismatch is disallowed.

The computations may favor either speed or accuracy. If speed is favored, then one of several consequences is that the warped image is computed from the source by nearest-neighbor interpolation. If accuracy is favored, then one of several consequences is that cubic-spline interpolation is performed instead.

The plugin can accommodate grayscale images and RBG stacks. Only one color plane of the RGB stack is taken into account for registration; the transformation applied to the two remaining color planes will be adjusted accordingly.

Manual and automatic modes: if a stack of grayscale images is selected, only the first slice is registered. The second slice, if any, acts as a boolean mask, with the following convention: the presence of a zero value in the second slice of either image indicates that the data [at the same location in the first slice of both images] should not be considered for automatic alignment; the presence of values that are jointly nonzero in the second slice of both images indicates that the data [at the same location in the first slice of both images] are relevant and should indeed participate in the mean-square computation. By convention, the data outside the initial image frames are considered as irrelevant, while every pixel within the initial image frames is considered relevant.

When the source image is an RGB stack, the output image that results from the registration process is returned as an RGB color image. Else, it is returned as a stack where the first slice contains the source image after the warping has been applied, and where the second slice contains the source mask after warping. A nearest-neighbor interpolation is used for the source mask. The output type is either RGB or float 32-bit. To know the exact domain where the mean-square criterion has been applied, the user must himself perform an "AND" operation between the mask (if any) of the target image on one hand, and the returned mask of the warped source image on the other hand: the returned mask does not provide the explicit result of this operation, but only one of its two operands. RGB stacks do not accomodate masks.

Batch mode: the target image plays the same role in the batch mode as in the manual or automatic modes. If the target image is a stack, its second slice still defines a registration mask; else, every pixel of the target image is considered relevant. The source image, however, is treated differently. It has to be a stack for the batch mode to be enabled, and every slice gets registered in turn to the same target image. Each slice is given the same default mask, with every pixel being considered relevant. The output image that results from the registration process is returned as a stack. Each slice contains a warped version of the corresponding slice of the source image. As in the case of manual and automatic modes, the output type is float 32-bit. The batch mode is not available for RGB stacks.

In some situations, a single target is desirable because the source images differ essentially by their geometry, less so by their content. TurboReg is appropriate in those situations. But there also exist cases where the content of the several source images gradually evolves from frame to frame; this might happen for example with a time series, or with a 3D stack of slices that are acquired with only a loose control of the slice-to-slice alignment. A companion plugin has been written to help in this different settings; it is named StackReg and is available here.


IV. User Manual

Moving a point

Tool to move points is selected

Figure 2. Tool to move points.

To select a point, click anywhere in the source or target image. The point closest to where you clicked will be highlighted. To move points, drag within the image. The point closest to where you clicked will be moved to the position where you release the mouse. Alternatively, you can use the keyboard arrow keys to get a finer control over the position of the landmarks. Hitting the tab key will change the selection of the current point. Note: it is not possible to move a point outside of the image frame.

For the rigid-body transformation, there are two types of landmark. The green cross determines the translation. The other two landmarks are used to determine the orientation only, as indicated by the blue and brown lines.

Image magnification/minification

Zooming tool

Figure 3. Zooming tool.

To magnify an image, select the magnification tool and click in the image. To minify an image, use the same tool with the <Control> keyboard modifier.

Dialog box

Dialog box

Figure 4. Dialog box.

This dialog box will appear if at least two grayscale images or RGB stacks are available at the time when TurboReg is launched. If that is not the case, a message will inform you of this requirement.

The target image that is currently selected determines the reference to which the source image will be aligned/matched/registered. If you decide to select as target the current source, your new selection will be honored, but the source image will be changed as well to reflect another arbitrary choice (it is not possible to register an image to itself).

The source image that is currently selected determines which image will be aligned/matched/registered to the target. If you decide to select as source the current target, your new selection will be honored, but the target image will be changed as well to reflect another arbitrary choice (it is not possible to register an image to itself).

When selecting another target or source, or when TurboReg is launched, you may notice some activity in the progress bar of ImageJ. This activity notifies you that some background preprocessing is going on. You can safely ignore it and proceed (you don't have to wait until this preprocessing is complete if you want to perform additional operations). The same applies when switching the distortion type from Bilinear, or to Bilinear.

There are five different types of distortion: Translation, Rigid Body, Scaled Rotation, Affine, and Bilinear; those correspond to 1, (1 + 2), 2, 3, and 4 landmarks, respectively. The landmarks are reset to their default position when changing the distortion type.

You can save the current configuration of landmarks in a text file by clicking the button [Save Now...]. The format of the file is rigid; it is nevertheless possible to edit its numerical fields if desired. This may allow you to overcome the limitation that forbids one to interactively move a point outside of the image frame.

You can restore a configuration of landmarks from a text file by clicking the button [Load...]. The configuration must have been previously created by TurboReg. A check on the current size of the target and source images is performed to ensure that the stored configuration makes sense with respect to the current images.

The automatic alignment mode is used to nudge the initial source landmarks, which results in refined landmarks. Those can be saved by checking the box [Save on Exit].

You can choose one of two different trade-offs between registration speed and accuracy. When speed is favored, the output image suffers from low-quality interpolation (nearest-neighbor). Moreover, in the automatic registration mode, the accuracy is further reduced because the refinement of landmarks is made coarser. When accuracy is favored instead, the output image benefits from high-quality interpolation (cubic spline). Moreover, in the automatic registration mode, more effort is spent on refining the landmarks. Please note that it is not possible to perform automatic registration when speed is favored and when at least one of the dimensions of the target or source images is too small; in that case, manual registration is the only mode allowed (with the quality set to 'Fast'). Else, set the quality to 'Accurate'.

The [Credits...] button gives you basic information and provides a link to the present page.

Credits

Figure 5. Credits.


V. Macro

Some additional flexibility is available when calling TurboReg from ImageJ's macro language, which may be suitable in certain specialized situations. Most importantly, the landmark positions are directly accessible to further processing. Also, it is possible to specify cropping zones in the source and target images, which is useful when there is a reduced overlap between images, a situation that often arises when stitching together a mosaic of images.

Although TurboReg is scriptable, it is not recordable. Here is an example of a call to TurboReg from a macro:
run("TurboReg ", "-align -file path/source.tif 40 80 639 479 -file path/target.tif 0 0 639 479 -translation 320 240 331.7 210 -showOutput");
A fully developed and commented example is available here.

The full syntax follows one of the three cases below:

-help
1) Prints to the 'Results' window of ImageJ the full syntax of TurboReg, and quit.

-align <ALIGNMENT_OPTIONS>
2) Register a source image to a target image. The details of the command are given in <ALIGNMENT_OPTIONS>. This corresponds most closely to the automatic mode of the interactive version of TurboReg. The quality is always set to 'Accurate'.

-transform <TRANSFORMATION_OPTIONS>
3) Given a set of landmarks, transform a source image. The details of the command are given in <TRANSFORMATION_OPTIONS>. This corresponds most closely to the manual mode of the interactive version of TurboReg. The quality is always set to 'Accurate'.

<ALIGNMENT_OPTIONS> Consists of the following six mandatory fields, in that order:
    <SOURCE> 2.1) Reference to the source. Masks are dealt with like in the interactive version of TurboReg. However, HSB-stacks and RGB-stacks are not recognized.
    <SOURCE_CROPPING> 2.2) How to crop the source.
    <TARGET> 2.3) Reference to the target. Masks are dealt with like in the interactive version of TurboReg. However, HSB-stacks and RGB-stacks are not recognized.
    <TARGET_CROPPING> 2.4) How to crop the target.
    <TRANSFORMATION> 2.5) Which transformation to consider.
    <VERBOSE>
2.6) What to report.
<SOURCE> There are two mutually exclusive ways to access the source:
    -file <NAME> 2.1.1) Disk-based access.
    -window <NAME>
2.1.2) ImageJ-based access.
<NAME> String with optional double quotes. Escape characters (e.g., '%20') are not interpreted, with the exception of the backslash character '\' which is changed into a forward slash '/', to cope with Windows-style path separator.
<SOURCE_CROPPING> Used to reduce the processing load, when convenient. Consists of the following four mandatory fields, in that order:
    sourceCropLeft 2.2.1) Leftmost pixel to consider. Often 0.
    sourceCropTop 2.2.2) Topmost pixel to consider. Often 0.
    sourceCropRight 2.2.3) Rightmost pixel to consider. It is often given by one less than the width of the image.
    sourceCropBottom 2.2.4) Bottom-most pixel to consider. It is often given by one less than the height of the image.
<TARGET> There are two mutually exclusive ways to access the target:
    -file <NAME> 2.3.1) Disk-based access.
    -window <NAME>
2.3.2) ImageJ-based access.
<TARGET_CROPPING> Used to reduce the processing load, when convenient. Consists of the following four mandatory fields, in that order:
    targetCropLeft 2.4.1) Leftmost pixel to consider. Often 0.
    targetCropTop 2.4.2) Topmost pixel to consider. Often 0.
    targetCropRight 2.4.3) Rightmost pixel to consider. It is often given by one less than the width of the image.
    targetCropBottom 2.4.4) Bottom-most pixel to consider. It is often given by one less than the height of the image.
<TRANSFORMATION> There are five mutually exclusive transformations available:
    -translation <LANDMARKS_1> 2.5.1) Translates the source image. Requires one landmark in each image.
    -rigidBody <LANDMARKS_3> 2.5.2) Translates and rotates the source image. Requires three landmarks in each image; the first landmark gives the overall translation and the two remaining ones determine the rotation angle.
    -scaledRotation <LANDMARKS_2> 2.5.3) Translates, rotates, and enlarges or reduces the source image by an isotropic factor. Requires two landmarks in each image.
    -affine <LANDMARKS_3> 2.5.4) Translates, rotates, shears, skews, and enlarges or reduces the source image by an anisotropic factor. Requires three landmarks in each image.
    -bilinear <LANDMARKS_4>
2.5.5) Performs a bilinear transformation on the source image. Requires four landmarks in each image.
<LANDMARKS_1> Gives the initial configuration of landmarks that will be refined for a translation:
    sourcePointX1 2.5.1.1) Horizontal component of the first landmark in the source image.
    sourcePointY1 2.5.1.2) Vertical component of the first landmark in the source image.
    targetPointX1 2.5.1.3) Horizontal component of the first landmark in the target image.
    targetPointY1 2.5.1.4) Vertical component of the first landmark in the target image.
<LANDMARKS_3> Gives the initial configuration of landmarks that will be refined for a rigid-body or an affine transformation:
    sourcePointX1 2.5.2.1) Horizontal component of the first landmark in the source image.
    sourcePointY1 2.5.2.2) Vertical component of the first landmark in the source image.
    targetPointX1 2.5.2.3) Horizontal component of the first landmark in the target image.
    targetPointY1 2.5.2.4) Vertical component of the first landmark in the target image.
    sourcePointX2 2.5.2.5) Horizontal component of the second landmark in the source image.
    sourcePointY2 2.5.2.6) Vertical component of the second landmark in the source image.
    targetPointX2 2.5.2.7) Horizontal component of the second landmark in the target image.
    targetPointY2 2.5.2.8) Vertical component of the second landmark in the target image.
    sourcePointX3 2.5.2.9) Horizontal component of the third landmark in the source image.
    sourcePointY3 2.5.2.10) Vertical component of the third landmark in the source image.
    targetPointX3 2.5.2.11) Horizontal component of the third landmark in the target image.
    targetPointY3 2.5.2.12) Vertical component of the third landmark in the target image.
<LANDMARKS_2> Gives the initial configuration of landmarks that will be refined for a scaled rotation:
    sourcePointX1 2.5.3.1) Horizontal component of the first landmark in the source image.
    sourcePointY1 2.5.3.2) Vertical component of the first landmark in the source image.
    targetPointX1 2.5.3.3) Horizontal component of the first landmark in the target image.
    targetPointY1 2.5.3.4) Vertical component of the first landmark in the target image.
    sourcePointX2 2.5.3.5) Horizontal component of the second landmark in the source image.
    sourcePointY2 2.5.3.6) Vertical component of the second landmark in the source image.
    targetPointX2 2.5.3.7) Horizontal component of the second landmark in the target image.
    targetPointY2 2.5.3.8) Vertical component of the second landmark in the target image.
<LANDMARKS_4> Gives the initial configuration of landmarks that will be refined for a bilinear transformation:
    sourcePointX1 2.5.5.1) Horizontal component of the first landmark in the source image.
    sourcePointY1 2.5.5.2) Vertical component of the first landmark in the source image.
    targetPointX1 2.5.5.3) Horizontal component of the first landmark in the target image.
    targetPointY1 2.5.5.4) Vertical component of the first landmark in the target image.
    sourcePointX2 2.5.5.5) Horizontal component of the second landmark in the source image.
    sourcePointY2 2.5.5.6) Vertical component of the second landmark in the source image.
    targetPointX2 2.5.5.7) Horizontal component of the second landmark in the target image.
    targetPointY2 2.5.5.8) Vertical component of the second landmark in the target image.
    sourcePointX3 2.5.5.9) Horizontal component of the third landmark in the source image.
    sourcePointY3 2.5.5.10) Vertical component of the third landmark in the source image.
    targetPointX3 2.5.5.11) Horizontal component of the third landmark in the target image.
    targetPointY3 2.5.5.12) Vertical component of the third landmark in the target image.
    sourcePointX3 2.5.5.13) Horizontal component of the fourth landmark in the source image.
    sourcePointY3 2.5.5.14) Vertical component of the fourth landmark in the source image.
    targetPointX3 2.5.5.15) Horizontal component of the fourth landmark in the target image.
    targetPointY3 2.5.5.16) Vertical component of the fourth landmark in the target image.
<VERBOSE> There are two mutually exclusive ways to report the resuts:
    -hideOutput 2.6.1) Performs a silent registration. Nothing can be retrieved from a inside a macro, neither the registered image, nor the refined landmarks. Useful only in the plugin-to-plugin case.
    -showOutput
2.6.2) Displays the output image in a new window called 'Output'. In addition, put the coordinates of the refined landmarks in the 'Results' table of ImageJ. The columns are labeled sourceX, sourceY, targetX, and targetY, in that order. There is one pair or source/target landmarks per line. The number of lines depends on the transformation.
<TRANSFORMATION_OPTIONS> Consists of the following four mandatory fields, in that order:
    <SOURCE> 3.1) Reference to the source. Masks are dealt with like in the interactive version of TurboReg. However, HSB-stacks and RGB-stacks are not recognized.
    <OUTPUT_SIZE> 3.2) Size of the source after transformation.
    <TRANSFORMATION> 3.3) Which transformation to consider.
    <VERBOSE>
3.4) What to report.
<OUTPUT_SIZE> Used to impose the size of the output image. Consists of the following two mandatory fields, in that order:
    outputWidth 3.2.1) Width of the output image. The target landmarks are given relative to the coordinate system of this output image.
    outputHeight 3.2.2) Height of the output image. The target landmarks are given relative to the coordinate system of this output image.

VI. Plugin-to-Plugin

To call TurboReg from another plugin, we suggest the following Java statement:
    Object myTurboRegObject = IJ.runPlugIn("TurboReg_", myTurboRegOptions);
where myTurboRegOptions is a String object that follows exactly the same syntax as described in the Macro case. Methods of the returned object myTurboRegObject can then be accessed thanks to a reflection mechanism. This approach facilitates code maintenance, especially the independence of all plugins; moreover, it limits the risk of having duplicate class names inside the hierarchy of folders spanned by ImageJ.

There are three public methods of interest inside the class TurboReg_. Those are getSourcePoints, getTargetPoints, and getTransformedImage. Typical calls look like:
    Method method = myTurboRegObject.getClass().getMethod("getSourcePoints", null);
    double[][] mySourcePoints = (double[][])method.invoke(myTurboRegObject, null);
    method = myTurboRegObject.getClass().getMethod("getTargetPoints", null);
    double[][] myTargetPoints = (double[][])method.invoke(myTurboRegObject, null);
    method = myTurboRegObject.getClass().getMethod("getTransformedImage", null);
    ImagePlus myTransformedImage = (ImagePlus)method.invoke(myTurboRegObject, null);

where the necessary encapsulating blocks (e.g., try {}catch(){} ) have not been shown, for simplicity. The full description of these three methods and of their parameters is available from the API of TurboReg. A fully developed example of a plugin that takes advantage of the interoperability of TurboReg is StackReg.


VII. Application Programming Interface

A technical Javadoc documentation is available.


VIII. Known Bugs

On Mac OS, MRJ versions prior to 2.2.5 won't let you operate the plugin. To update MRJ, launch "Software Update" from the control panels and follow the instructions.

The dialog box is modeless, which means you have access to all functions of ImageJ. While some actions are safe (e.g., histogram computation), others may unfortunately lead to difficulties, such as:

  • Launching TurboReg while another instance of TurboReg is already running results in disaster.
  • Closing an image while the dialog of TurboReg is active results in an incoherent list of target and source images. Selecting the closed image results in disaster.
  • Cropping or resizing the target or source image while the dialog of TurboReg is active may lead to unexpected behavior.
  • Processing (say, smoothing) the target or source image while the dialog of TurboReg is active may sometimes lead to the registration of the unprocessed image. A workaround is to swap target for source, and back.
  • Switching focus to a third image (neither source, nor target) doesn't restore the proper toolbar. Unexpected behavior may result when trying to toggle between tools.
  • ...
  • Many more: by now, I hope you get the idea of what is allowed and what is not. In case of doubt, act as if the dialog box would be modal instead of modeless. [Technical note: a modeless dialog box is necessary to allow for interactive access to the landmarks.]

IX. Conditions of Use

© 2011 Philippe Thévenaz

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

You undertake to include a citation or acknowledgment whenever you present or publish results that are based on this program. EPFL makes no warranties of any kind on this software and shall in no event be liable for damages of any kind in connection with the use and exploitation of this technology.