 |
 |
 |
 |
 |
|
|
|
|
|
Deconvolution is one of the most common image-reconstruction tasks that arise in 3D fluorescence microscopy. The aim of this challenge is to benchmark existing deconvolution algorithms and to stimulate the community to look for novel, global and practical approaches to this problem.
The challenge will be divided into two stages: a training phase and a competition (testing) phase. It will primarily be based on realistic-looking synthetic data sets representing various sub-cellular structures. In addition it will rely on a number of common and advanced performance metrics to objectively assess the quality of the results.
The challenge is open to three groups of participants: academic algorithm designers, developers of open-source (or free) software and providers of commercial solutions. Note that users of deconvolution software are also allowed to enter the tournament; their group assignment will be determined by the type of software they use.
After a training stage, the participants will enter a qualification stage whose goal is to select three finalists in each group. The selection will be based on a weighted average of different performance metrics. However, there will be no global ranking of the participants.
A half-day special session dedicated to the challenge will take place at the beginning of the ISBI conference. All the participants are encouraged to attend this special session. The final stage of the tournament will take place during that special session: each finalist will be invited to demonstrate his method and skills during a live deconvolution experiment. In addition, we will discuss the results and the lessons learned from the event.
The challenge will be based on a set of computer-generated 3D image stacks that simulate optically-sectioned digital micrographs of biological samples. This will allow us to define an unambiguous ground truth for all deconvolution tasks that will be assigned to the participants. However, the ground truth will NOT be accessible to the participants, as in a real deconvolution problem.
The images reproduce several distinct sub-cellular structures that are typically observed in mitotic cells (point sources, filaments, membranes and dense volumetric features). To account for the use of different fluorescent markers, each class corresponds to a specific color channel. For simplicity, we assume that there is no cross-talk between these color channels.
Our simulation engine is further based on a faithful physical description of digital microscopy, combining an advanced PSF model with a noise generator that accounts for shot noise, electronic noise and quantization.
The idea of the challenge is to make the participants follow a reasonably complete deconvolution workflow, with several hurdles that are representative of real situations. These hurdles may include algorithm selection, PSF-inaccuracy compensation, noise-model calibration and regularization-parameter adjustment. We will ensure a realistic setting by adequately choosing the information that is disclosed to the participants, while limiting the total number of unknowns so as to keep the task accessible.
Ultimately the goal for the participants is to generate the best-possible estimate of the ground-truth data, by inverting the convolution operation without amplifying noise in an overwhelming manner. To allow for a direct comparison, the estimate will have to follow the same sampling conventions as the ground truth, especially regarding the voxel sizes and the origin of the coordinate system.
The quality of the deconvolved data will be judged using a collection of standard performance metrics (e.g., MSE, SSIM, NMISE, etc.), as well as more advanced structure-oriented and post-processing-based metrics.
The best deconvolution results will be selected based on an aggregate measure computed as a weighted average of the aforementioned metrics.
The training stage of the 2nd edition of the challenge will begin soon. Follow this link for early registration.
July 15, 2013