Single-Molecule Localization Microscopy  •  Software Benchmarking

3D SMLM Software Challenge

The 3D SMLM challenge is an ongoing competition to assess SMLM software on both simulated and real reference datasets using objective assessment metrics.

The 3D challenge remains continuously open to new submissions from both 2D and 3D SMLM software.

The first round of the challenge was presented in the Special Session at SMLMS 2016.

banner

Results on 3D Real Datasets

In this section, we present the localization results on real 3D datasets (astigmatism). We provide two real datasets: Tubulin and Nuclear Pore Complex with the courtesy to Jonas Ries (EMBL) [Reference: Li et al., Nature Methods 15, 2018]. The localizations results of the software have been standardized (wobble-corrected if needed it). In addition, the drifted correction was applied. In order to facilitate the comparison, all the results are display using the same rendering method and the same colormap (color coded in Z for the XY plane).

The selected software are: 3D-DAOSTORM, Cspline, MIAtool, QuickPALM, RapidSTORM, SMAP-2018, ThunderSTORM, and WaveTracer.

Tubulin-A647-3D

Microtubules

NPC-A647-3D

Nuclear pore complex

Tubulin-A647-3D

View dataset information
Download raw data


Convention reconstruction
Integrating all the frames

3D astigmatic image of microtubules imaged using dSTORM

Courtesy of Jonas Ries (EMBL)

Reference: Li et al., Nature Methods 15 2018. (raw data of Figure S6)

3D-DAOSTORM

Temporal grouping: Yes

Wobble correction: Yes (Check)

Drift correction: Yes


Cspline

Temporal grouping: Yes

Wobble correction: Not required

Drift correction: Yes


MIAtool

Temporal grouping: Yes

Wobble correction: Yes (Check)

Drift correction: Yes


QuickPALM

Temporal grouping: No

Wobble correction: No

Drift correction: Yes


RapidSTORM

Temporal grouping: No

Wobble correction: Yes (Check)

Drift correction: Yes


SMAP-2018

Temporal grouping: Yes

Wobble correction: Not required

Drift correction: Yes


ThunderSTORM

Temporal grouping: Yes

Wobble correction: Yes (Check)

Drift correction: Yes


WaveTracer

Temporal grouping: No

Wobble correction: Not required

Drift correction: Yes


NPC-A647-3D

View dataset information
Download raw data


Convention reconstruction
Integrating all the frames

3D astigmatic dSTORM image of nuclear pore complex

Courtesy of Jonas Ries (EMBL)

Reference: Li et al., Nature Methods 15 2018. (raw data of Figure S9)

3D-DAOSTORM

Temporal grouping: Yes

Wobble correction: Yes (Check)

Drift correction: Yes


Cspline

Temporal grouping: Yes

Wobble correction: Not required

Drift correction: Yes


MIAtool

Temporal grouping: Yes

Wobble correction: Yes (Check)

Drift correction: Yes


QuickPALM

Temporal grouping: No

Wobble correction: No

Drift correction: Yes


RapidSTORM

Temporal grouping: No

Wobble correction: Yes (Check)

Drift correction: Yes


SMAP-2018

Temporal grouping: Yes

Wobble correction: Not required

Drift correction: Yes


ThunderSTORM

Temporal grouping: Yes

Wobble correction: No

Drift correction: Yes


WaveTracer

Temporal grouping: No

Wobble correction: Not required

Drift correction: Yes


© 2018 Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 30 Nov 2018