Benchmarking of SMLM Software
Comprehensive review of the single-molecule localization microscopy (SMLM, PALM, STORM) software packages.
NATURE METHODS | ANALYSIS
D. Sage, H. Kirshner, T. Pengo, N. Stuurman, J. Min, S. Manley & M. Unser, Quantitative evaluation of software packages for single-molecule localization microscopy, Nature Methods (2015). doi:10.1038/nmeth.3442
Abstract The quality of super-resolution images obtained by single-molecule localization microscopy (SMLM) depends largely on the software used to detect and accurately localize point sources. In this work, we focus on the computational aspects of super-resolution microscopy and present a comprehensive evaluation of localization software packages. Our philosophy is to evaluate each package as a whole, thus maintaining the integrity of the software. We prepared synthetic data that represent three-dimensional structures modeled after biological components, taking excitation parameters, noise sources, point-spread functions and pixelation into account. We then asked developers to run their software on our data; most responded favorably, allowing us to present a broad picture of the methods available. We evaluated their results using quantitative and user-interpretable criteria: detection rate, accuracy, quality of image reconstruction, resolution, software usability and computational resources. These metrics reflect the various tradeoffs of SMLM software packages and help users to choose the software that fits their needs.
From April 10 to May 10 a second round of the challenge is running. All the DH have be submitted
with the new normalized datasets.
28 Aug: Special Session
» Live TV YouTube Channel
A large panel of software have submitted localization files to the challenge in all modalities.
The competition Challenge 2016 will run from 15 June to 22 July 2016. The results will be presented SMLMS 2016 symposium in Lausanne Switzerland, 28-30th August.
The first dataset is released. The same sample is simulated in 4 modalities: 2D, 3D-astigmatism, 3D-biplane, and 3D-double-helix.
Seamus Holden and Daniel Sage
Nature Photonics 10, 2016, PDF.
A new challenge will be held in August 2016. Send an email to be kept informed.
Advanced online publication of the comparative results on the Nature Methods website.
Update of the comprehensive list of SMLM software packages. 45 for localization software and 6 for deconvolution-type reconstruction software.
These reference datasets are designed to be largely used by the developpers to validate theirs software and by the users to check a software. They can be freely used if the sources and references are properly cited.
The ISBI Challenge is turned to a permanent online challenge.
Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 31 Mar 2017