Single-Molecule Localization Microscopy  •  Software Benchmarking

Challenge 2D • Theoretical PSF

The Grand Challenge Localization Microscopy was presented as a Workshop in IEEE conference ISBI. Nearly 30 software have been run, mostly by the authors of the software, thus constituting the first world-wide effort in benchmarking the localization software in a comprehensive review.

The results are in the paper: Sage et al. Quantitative evaluation of software packages for SMLM, Nature Methods 12, 2015.


Goals of the challenge

The goal of this challenge is to benchmark currently available localization software. In particular, comparing their performance in terms of detection rate, localization accuracy, and image rendering. Accessibility, usability for the end-user, and computational time will be evaluated, too. The benchmark data consists of simulated images of various biological structure such as tubulins. The simulated data accounts for fluorophores activation and excitation, image formation, and known perturbations models. As the ground truth exact positions of the fluorophores are known, the benchmarking metrics will use use objective measures.

The challenge is open to all individuals or teams, academic or corporate, existing or newly developed localization microscopy software. Participation is free.

To announce your participation, send email indicating the name of the software that you want to represent or to test and the name of a contact person. This pre-registration gives access to the ground-truth of the training datasets.

IEEE ISBI Workshop

April, 8, 2013, San Francisco
Presentation Speaker
IntroductionDaniel Sage
PSF models Hagai Kirshner
Photon quantification Nico Stuurman
Datasets Daniel Sage
Rendering Thomas Pengo
File format Nico Stuurman
Assessment Daniel Sage
B-recsHerveé Raoult
FacePALM Eelco Hoogendoorn
FALCON Junhong Min
Fast-ML-HD Kyungsang Kim
FPGA Manfred Kirchgessner
GPUgaussMLE Keith Lidke
Insight3Ryan McGorty
MicroManager LM Nico Stuurman
QuickPALM Christophe Zimmer
RadialSymmetry Raghuveer Parthasarathy
simpleSTORM Luca Fiaschi
SOSpluginIhor Smal
ThunderSTORMPavel Krizek
Wavelet FluoroBrancroft Trevor Ashley
Additional software (a-livePALM, DOASTORM, PeakFit, RapidSTORM)Thomas Pengo
Results - Real datasets Junhong Min
Results - Localization Thomas Pengo
Round-table, discussions Everybody

Committee • Challenge 2013

Daniel Sage

Biomedical Imaging Group (BIG)
Ecole Polytechnique Fédérale de Lausanne (EPFL)

Hagai Kirshner

Biomedical Imaging Group (BIG)
Ecole Polytechnique Fédérale de Lausanne (EPFL)

Thomas Pengo

Center for Genomic Regulation
Barcelona, Spain

Nico Stuurmann

Vale Lab
University of California, San Francisco (UCSF)

Junhong Min

Bio Imaging Signal Processing
Korea Advanced Institute of Sci. and Tech. (KAIST)

Suliana Manley

Laboratory of Experimental Biophysics (LEB)
Ecole Polytechnique Fédérale de Lausanne (EPFL)

Sponsors • Challenge 2013

Hamamatsu, Camera Products

Mad City Labs


Euro‐BioImaging is a large‐scale pan‐European research infrastructure project on the European Strategy Forum on Research Infrastructures (ESFRI) Roadmap.

Open Bio Image Alliance

© 2018 Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 30 Nov 2018