Single-Molecule Localization Microscopy  •  Software Benchmarking

Second edition of the challenge focussed on 3D

The challenge has been presented at Single Molecule Localization Microscopy Symposium (SMLMS) at Lausanne, Switzerland, August 28-30, 2016. This edition will be focussed on 3D localization techniques, astigmatism, biplane, or double-helix.

The results are presented on the bioRxiv preprint server: Super-resolution fight club: A broad assessment of 2D & 3D single-molecule localization microscopy software Jul 5, 2018, doi.

banner

Leaderboard

Results per Software

Side-by-Side Graphs

Summary

Modality AS

Dataset Participants Jaccard
Average
RMSE Lateral
Average
RMSE Axial
Average
Winner Jaccard
Winner
RMSE Lateral
Winner
RMSE Axial
Winner
○ MT1.N1.LD 18 73.7 38.3 66.3 SMAP-2018 84.5 26.3 29.1
● MT2.N1.HD 15 52.7 66.3 109.3 SMolPhot 66.3 50.5 66.3
△ MT3.N2.LD 17 69.3 44.9 84.0 SMolPhot 85.8 39.2 60.3
▲ MT4.N2.HD 15 43.3 77.5 128.4 SMolPhot 58.0 76.4 100.9

Modality BP

Dataset Participants Jaccard
Average
RMSE Lateral
Average
RMSE Axial
Average
Winner Jaccard
Winner
RMSE Lateral
Winner
RMSE Axial
Winner
○ MT1.N1.LD 8 75.1 27.0 90.5 SMAP-2018 87.1 12.3 21.7
● MT2.N1.HD 8 54.1 55.3 116.1 SMAP-2018 60.7 43.1 48.9
△ MT3.N2.LD 6 66.8 41.9 84.2 SMAP-2018 83.1 28.5 46.8
▲ MT4.N2.HD 7 41.9 69.3 134.7 SMAP-2018 60.5 54.6 72.3

Modality DH

Dataset Participants Jaccard
Average
RMSE Lateral
Average
RMSE Axial
Average
Winner Jaccard
Winner
RMSE Lateral
Winner
RMSE Axial
Winner
○ MT1.N1.LD 9 72.8 36.6 34.0 SMAP-2018 77.1 27.0 20.9
● MT2.N1.HD 8 29.1 82.9 77.9 CSpline 45.6 67.4 69.2
△ MT3.N2.LD 9 49.7 53.5 55.2 SMAP-2018 65.6 45.3 42.1
▲ MT4.N2.HD 6 25.8 96.6 103.1 SMAP-2018 30.0 90.6 93.7

Modality 2D

Dataset Participants Jaccard
Average
RMSE Lateral
Average
RMSE Axial
Average
Winner Jaccard
Winner
RMSE Lateral
Winner
RMSE Axial
Winner
○ ER1.N3.LD 32 72.3 35.9 - ADCG 90.7 31.0 -
● ER2.N3.HD 28 56.7 52.1 - ADCG 75.0 45.6 -
△ MT3.N2.LD 32 73.4 36.1 - SMolPhot 90.1 29.0 -
▲ MT4.N2.HD 29 55.2 52.3 - ADCG 69.9 51.0 -

© 2017 Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 31 Mar 2017