Single-Molecule Localization Microscopy  •  Software Benchmarking

3D SMLM Software Challenge

The 3D SMLM challenge is an ongoing competition to assess SMLM software on both simulated and real reference datasets using objective assessment metrics.

The 3D challenge remains continuously open to new submissions from both 2D and 3D SMLM software.

The first round of the challenge was presented in the Special Session at SMLMS 2016.

banner

Summary

With the widespread uptake of 2D and 3D single molecule localization microscopy, a large set of different data analysis packages have been developed to generate super-resolution images. To guide researchers on the optimal analytical software for their experiments, we have designed, in a large community effort, a competition to extensively characterise and rank these options. We generated realistic simulated datasets for popular imaging modalities - 2D, astigmatic 3D, biplane 3D, and double helix 3D - and initially evaluated 36 participant packages against these data. This provides the first broad assessment of 3D single molecule localization microscopy software, provides a holistic view of how the latest 2D and 3D single molecule localization software perform in realistic conditions, and ultimately provides insight into the current limits of the field.

The first results are presented on the bioRxiv preprint server: Super-resolution fight club: A broad assessment of 2D & 3D single-molecule localization microscopy software.

The competition website is continuously updated with contributions from new software, so keep an eye on the competition results page to stay up to date with the latest developments.

Participation

The challenge is open to all individuals or teams, academic or corporate, existing or newly developed localization microscopy software. Participation is free.

The challenge is turn on as a permanent challenge.

How to register? To announce your participation, send an email to indicating the name of the software that you want to represent and the name of a contact person.

How to participate?

  • Pre-register by send an email with the name of the software and a contact name
  • Choose one or several modalities: 2D, AS (3D-astigmatism), DH (3D-double-helix) or BP (3D-biplane)
  • Download the datasets and the z-stack of beads for your calibration.
  • Submit your documents

Documents to submit

  • 3 questionnaires, general information, usability, and computational time
  • 1 wobble correction process: localization on the z-stack of beads, wooble correction file, or localization already corrected by the software
  • 4 localization files of a given category/modality
    • File format: the results of the localization are given in a delimiter-separated values text file, typically in a coma-separated values (CSV) or tab-separated values (TSV/TAB). The unreadable lines are discarded (e.g. NaN).
    • At least the following columns: X, Y, Z, frame, number of photons. The organizers reserve the right to permute the columns.
    • To better fit the same origin, the organizers reserve the right to apply the following offset: -50nm on X (1/2 pixel), -50nm on Y (1/2 pixel), -750 nm (focal plane at 750nm), +1 frame (index of first frame is 1).

© 2018 Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 30 Nov 2018