Single-Molecule Localization Microscopy  •  Software Benchmarking

The Challenge 2013 is turned to an online permanent challenge

The Grand Challenge Localization Microscopy was organized as Workshop in the conference IEEE International Symposium on Biomedical Imaging. Nearly 30 software have been run, mostly by the authors of the software, thus constituting the first world-wide effort in benchmarking the localization software in a comprehensive review. The results are now published.

The 2013 ISBI challenge is now re-open as a online permanent challenge and a new SMLM challenge will be held in 2016.

banner

Results

Comparative results

Publication of the results in the Nature Methods article

D. Sage, H. Kirshner, T. Pengo, N. Stuurman, J. Min, S. Manley & M. Unser, Quantitative evaluation of software packages for single-molecule localization microscopy, Nature Methods 12, 717–724 (2015).

Read full PDF
Results of a software

The detailled results of the evaluation for a specific software has been communicated to the the participants for every submitted localization files corresponding to a dataset. It is included the full statistical analysis, some rendering images, some comparative rendering image (tested software localization vs ground-truth), and some cross-section profiles.

Example of comparitive close-ups

Rendering images by sum up of 2D Gaussian functions. The composite color image cobimes red channel (Ground-truth localizations) and the green channel (tested software localizations).

Scale bar: 4400 nm
Rendering pixelsize: 44 nm/pixel
Scale bar: 400 nm
Rendering pixelsize: 4 nm/pixel
Scale bar: 100 nm
Rendering pixelsize: 1 nm/pixel

Example of cross-section

Rendering images by sum up of 2D Gaussian functions. The composite color image cobimes red channel (Ground-truth localizations) and the green channel (tested software localizations).

Detections in function of the local SNR

Detections in function of the closest distance

Java tool for evaluation

CompareLocalization: Java application

A Java application to compare two sets of localization is given at CompareLocalization.jar. The positions of the molecules have to be stored in a column-separated text file, as CSV file. Information to parse this file has to provided in a XML description file or provided using the graphical user interface. A tool to create the XML file available

Examples

This application can run on any machine with at least Java 1.5 installed.

© 2017 Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 31 Mar 2017