Single-Molecule Localization Microscopy  •  Software Benchmarking

Grand Challenge on 2D/3D SMLM software

The challenge 3D has been presented at Single Molecule Localization Microscopy Symposium (SMLMS' 2016). The challenge relies on hyper-realistic datasets, using a 4-states photophysic model and experimental PSF. It covers three 3D localization techniques: astigmatism, biplane, and double-helix.

The results are presented on the bioRxiv preprint server: Super-resolution fight club: A broad assessment of 2D & 3D single-molecule localization microscopy software Jul 5, 2018, doi.

banner
Structure Signal/Noise
Ratio
Density Name Release (last update) Access
Data
Access
Ground-thruth
2D 3D
Astigmatism
3D
Double-Helix
3D
Biplane
Calibration Z-stack
Beads Very high 6 beads Beads 25.10.2016 Open Open Yes Yes Yes Yes
Training Datasets
Microtubules High Low MT0.N1.LD 07.06.2016 Open Open Yes Yes Yes Yes
Microtubules High High MT0.N1.HD 07.06.2016 Open Open Yes Yes Yes Yes
Microtubules Low Low MT0.N2.LD 07.06.2016 Open Open Yes Yes Yes Yes
Microtubules Low High MT0.N2.HD 07.06.2016 Open Open Yes Yes Yes Yes
Contest Datasets
Microtubules High Low MT1.N1.LD 25.10.2016 Open No - Yes Yes Yes
Microtubules High High MT2.N1.HD 11.03.2017 Open No - Yes Yes Yes
Microtubules Low Low MT3.N2.LD 25.10.2016 Open No Yes Yes Yes Yes
Microtubules Low High MT4.N2.HD 25.10.2016 Open No Yes Yes Yes Yes
Endoplasmic reticulum Medium Low ER1.N3.LD 07.06.2018 Open No Yes - - -
Endoplasmic reticulum Medium High ER2.N3.HD 07.06.2016 Open No Yes - - -
Real Datasets
Microtubules High Low Tubulin-A647-3D 04.10.2018 Open Unknown - Yes - -
Nuclear pore complex High Low NPC-A647-3D 04.10.2018 Open Unknown - Yes - -

Calibration • Z-stack of beads

2D • AS • DH • BP

Published June 7, 2016.

banner

Experimental conditions
  • z-stack of 6 beads in 2D, 3D-Astigmatism, 3D-Double-Helix, 3D-Biplane
  • Z-step = 10nm, Z-range = -750nm to 750nm, Z-focal plane at 0nm (151 slices)

Download page

Ground-truth of the position of the beads

Preliminary Results

Realistic Simulations • 3D • Experimental PSF

The experimental datasets consist of sequences of frames from real stained biologicial samples. A Z-stack of beads is provided for the axial calibration. The same 3D simulated structure is imaged in different experimental conditions and for different modality: 2D, 3D astigmatism (AS), 3D biplane (BP), and 3D double-helix (DH). As a testing phase, we have also the quad-plane (QP) datasets for high-density conditions. The training datasets are provided with ground-truth localizations.

picture1
Microtubules • Contest datasets

Published June 7, 2016 as training dataset for the SMLM challenge 2016

Three crossing microtubules in a field of view of 6.4 x 6.4 x 1.5 μm

Two experimental imaging conditions:

N1: Typical photon counts and background levels for Alexa647 labelled STORM sample

N2: Photoswitchable fluorescent protein labelled sample such as mEos2 or Dendra2

MT1.N1.LD
Low density Molecule density: 0.2 High SNR (N1) 19'996 frames 2D • AS • DH • BP
MT2.N1.HD
High density Molecule density: 2 High SNR (N1) 3'125 frames 2D • AS • DH • BP • QP
MT3.N2.LD
Low density Molecule density: 0.2 Low SNR (N2) 20'000 frames 2D • AS • DH • BP
MT4.N2.HD
High density Low SNR (N2) 2D • AS • DH • BP • QP

picture1
Endoplasmic Reticulum • Contest datasets

Published June 7, 2016 as training dataset for the SMLM challenge 2016

Simulation of cellular organelle (endoplasmic reticulum/mitochondria) inspired structure in the field of view of 6.4 x 6.4 x 0.7 μm

One experimental imaging conditions:

N3: Typical photon counts and level of background for site specific dye-labelled live cell STORM sample such as ER Tracker

ER1.N3.LD
Low density Molecule density: 0.2 Very low SNR (N3) 19'620 frames 2D
ER2.N3.HD
High density Molecule density: 5 Very low SNR (N3) '020 frames 2D

picture1
Microtubules • Training datasets

Published June 7, 2016 as training dataset for the SMLM challenge 2016

Three crossing microtubules in a field of view of 6.4 x 6.4 x 1.5 μm

Two experimental imaging conditions:

N1: Typical photon counts and background levels for Alexa647 labelled STORM sample

N2: Photoswitchable fluorescent protein labelled sample such as mEos2 or Dendra2

Ground-truth
localizations
MT0.N1.LD
Low density Molecule density: 0.2 High SNR (N1) 2DASBPDH
MT0.N1.HD
High density Molecule density: 2 High SNR (N1) 2DASBPDH
MT0.N2.LD
Low density Molecule density: 0.2 Low SNR (N2) 2DASBPDH
MT0.N2.HD
High density Molecule density: 2 Low SNR (N2) 2DASBPDH

Real Experiments • 3D Astigmatism

The experimental datasets consist of sequences of frames from real stained biologicial samples. A Z-stack of beads is provided for the axial calibration.

Tubulin-A647-3D

3D astigmatic image of microtubules imaged using dSTORM with a cylindrical lens (112'683 frames).
Microtubules in U-2 OS cells, labeled with anti-alpha tubulin primary and Alexa Fluor 647-coupled secondary antibodies.
Calibration data for the tubulin dataset: fluorescent beads adsorbed to a glass coverslide in water are stepped in Z at high SNR.

Courtesy of Jonas Ries (EMBL)

Reference: Li et al., Nature Methods 15, 2018. (raw data of Figure S6)

NPC-A647-3D

3D astigmatic dSTORM image of nuclear pore complex.
Nup107-SNAP-BG-AF647 in U-2 OS cells, imaged using dSTORM with a cylindrical lens (150'000 frames).
Calibration data for the dataset: fluorescent beads adsorbed to a glass coverslide in water are stepped in Z at high SNR.

Courtesy of Jonas Ries (EMBL)

Reference: Li et al., Nature Methods 15, 2018. (raw data of Figure S9)

© 2018 Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 11 Nov 2018