Second edition of the challenge focussed on 3DThe challenge has been presented at Single Molecule Localization Microscopy Symposium (SMLMS) at Lausanne, Switzerland, August 28-30, 2016. This edition will be focussed on 3D localization techniques, astigmatism, biplane, or double-helix. |
3. Image Formation Model (PSF)
4. Noise Model
7. Tool: Baseline Performances
8. Tool: Comparison Localization
The synthetic datasets were designed to be as similar as possible to images derived from cellular structures in real experimental conditions To achieve the high degree of realism, we defined mathematical models for biological structure that try to imitate microtubules and endoplasmic reticulum/mitochondria. These structure have a tubular shape in the 3D space. Typically, microtubules are defined with their central axis elongating in a 3D space having an average outer diameter of 25 nm with an inner, hollow tube of 15 nm diameter.
The underlying sample structure is formalized in a continuous space which allows rendering of digital images at any scale, from very high resolution (up to 1 nm/pixel) to low resolution (camera resolution: 100 nm). The continuous-domain 3D curve is represented by means of a polynomial spline. The sample is imaged in a limited field of view, i.e., less than 6.4 × 6.4 μm2, and the center lines of the microtubules have limited variation along the z (vertical) axis, i.e., less than 1.5 μm. The fluorescent markers are uniformly distributed over the structure according to the required density. The photon emission rate of each fluorophore is controlled by a photo-activation model (see below).
The exact locations of all fluorophores are therefore stored at high precision, as floating point numbers expressed in nanometers. This ground-truth file is useful for conducting objective evaluations without human bias.
Reference: Sage et al. Quantitative evaluation of software packages for single-molecule localization microscopy, Nature Methods 2015.
Flux of photons
The flux of photons is given by the relation: F = Φ . P . σ / e in photons/seconds
Φ is the quantum yield og the dye
P is power of the laser (not spatially uniform) in W/cm^{2}
e = h . c / λ is the energy of 1 photon is power of the laser (it is not spatially uniform)
σ = 1000 . ln(10) . ε / N_{A} is the absorption cross section in cm^{2}
ε is the molar extinction coefficient (EC) or absorptivity in cm^{2}/mol
4-states photophysics model
Given a list of source locations from the structure simulator, fluorophore blinking was modelled by a 4 state process:
The OFF to ON transition is not Poisson distributed it is uniform random, to reflect that in normal experimental conditions constant imaging density is maintained by tuning the photoactivation rate during the experiment. All other transitions are Poisson distributed. All switching is calculated at sub-frame resolution and then total fluorophore on-time was integrated over each frame.
The actual mean lifetime in On state is 1/(1/Ton + 1/Tbleach) due to two decay paths. Switching rates were chosen as "fluorescent-protein-like". For the training datasets: Ton = 3; Tdark = 2.5; Tbl=1.5; (units of frames)
Fractional fluorophore on-times per frame (between 0 and 1) were then multiplied by the mean photon emission. At the end of this process a list of XY positions, on-frames and (noise-free) intensities for all activated fluorophores was obtained.
Code is here: https://github.com/SMLM-Challenge/Challenge2016/tree/master/challenge_simulator/photophysics
In order to achieve as realistic a simulation as possible, model point spread functions were derived from experimentally measured PSFs. For each modality, images of fluorescent beads were recorded the conditions below. Signal to noise of recorded PSFs was maximised in all cases by maximizing exposure time and averaging over several frames to increase dynamic range.
Sample (all modalities)
PSF imaging conditions | Optics | Camera | Pixelsize | Z-stack | |
2D | 2D PSF | Nikon NA 1.49 TIRF oil objective on Nikon N-STORM commercial microscope | Andor iXon EMCCD |
Pixel size at sample plane: 43 nm (1.5x microscope and 2.5x SIM magnifiers in place) | Z-stack step size: 10 nm, 3 μm Z-range |
AS | Astigmatic PSF | Nikon NA 1.49 TIRF oil objective | EMCCD 160μm pixel size |
Pixel size at sample plane: 100 nm | Z-stack step size: 10 nm, 3 μm Z-range |
DH | Double-Helix PSF | Nikon NA 1.49 TIRF oil objective | EMCCD |
Pixel size at sample plane: 100 nm | Z-stack step size: 20 nm, 3 μm Z-range |
BP | Biplane PSF | The biplane model PSF was constructed from the 2D PSF data – see below. |
Model PSF construction
This yielded 3 high SNR model PSFs (2D, astigmatism and double-helix) with a voxel size of 10x10x10 nm^{3}. A central Z-range of 1.5 μm was selected. The biplane PSF was constructed by duplicating the 2D PSF and offsetting it by -250 nm and 250 nm for each Z-plane. For the DH-PSF, the transmission of the combined phase mask/ 4f system was measured as 96 %, which was approximated as 100 % brightness relative to the 2D-SPF and AS-PSF. The ground truth XY=0 was defined as the image centre of mass of the in-focus frame of the model PSF, and Z=0 was defined as the in-focus frame. Accounts for shifts in the fitted XY centre of the model PSF by localization software due to systematic offsets and Z-dependent variation of the model PSF centre of mass are dealt the wobble correction.
PSF generation code is here: https://github.com/SMLM-Challenge/Challenge2016/tree/master/challenge_simulator/psf_generation
// Pseudo-code noise model for LM challenge 2016. // This assumes all input light is fluorescence // (background, signal) and thus follows poisson statistics // The camera is an EMCCD, specifically the // Photometrics Evolve Delta 512 for each pixel n_photIn is the input electrons // n_ie, apply noise model // poisson noise including shot noise and spurious // charge plus binomial quantum efficiency conversion // is just a Poisson distribution as per Hirsch. n_ie = poisson(QE*n_photIn + c); // emccd model, shape param k=n_ie, // scale param theta=EMgain // after Basden et al Mon Not R Astron Soc 2003 n_oe = gammadistribution(n_ie,EMgain); n_oe = n_oe + gaussian(read_noise); // analog to digital converter ADU_out = int(n_oe/e_per_adu)+ baseline; //restrict to 16 bit ADU_out = min(ADU_out,65535) end
A constant mean autofluorescent background was added to the noise-free simulated images, and these images were then fed through the noise model representing Poisson distributed fluorescence emission recorded on a high quantum efficiency back-illuminated EMCCD.
Reference: model is inspired by Hirsch et al., PLoS One 2015.
Quantum Efficiency | QE = 0.9 | Evolve quantum efficiency @700 nm) |
Readout noise | read_noise = 74.4 | Manufacturer measured rms electrons for Evolve |
Spurious charge | c = 0.0002 | Manufacturer quoted spurious charge (CIC only, dark counts negligible) for Evolve |
EM Gain Register | EMgain = 300 | Manufacturer quoted spurious charge (CIC only, dark counts negligible) for Evolve |
Baseline | baseline = 100 | Typical value |
Electron per analog-to-digital units | e_per_adu = 45 | ADC conversion factor, arbitary value similar to typical |
Total system gain | gain = 6 | 0.9 * 300 / 45 = 6 |
Note: Quantization noise not an issue as noted by lots of astronomers and e.g. [1] Hirsch et al, PLoS One 2015.
Java code of the simulator is available on Github https://github.com/SMLM-Challenge/Challenge2016/tree/master/simulator-java
Comparison of 3D localization to ground truth for experimentally derived model PSFs requires correction of of depth-dependent lateral distortion, here called wobble. This is due to arbitrary systematic offset since definition of PSF centre is arbitrary. It was mentioned in several sources but discussed most fully here: Carlini et al., Correction of a Depth-Dependent Lateral Distortion in 3D Super-Resolution Imaging, PLOS One, 2015.
In order to correct for this, competitors should either:
If the wobble/ offset stuff is confusing – please contact us. In any case for submitted data, there is the option for us to automatically calculate the correction based on the uploaded bead localizations, without the need to worry.
Matlab code and example are available on Github https://github.com/SMLM-Challenge/Challenge2016
Objectives: This tool is a minimalist SMLM software that performs localizations of bright emitters on the 4 modalities of the challenge 2016: 2D, 3D-Astigmatism, 3D-Double-Helix, and 3D-Biplane. This SMLM_BaselineLocalization software is only designed to establish the performance baseline for the SMLM challenge.
It is fast and very inaccurate! It has intentionally limited lines of code and relies only on two threshold parameters.
3D Calibration: there is a very simple calibration tool that has to run on a z-stack of beads to find the linear relation between the axial position Z and the shape of the bead.
Distribution: The open-source software SMLM_BaselineLocalization is released as a Java ImageJ plugin.
Objectives: This is a piece of software to compute the rate of detection and the accuracy in 3D based on 2 sets of localization (x, y, z).
Wobble correction: 3D localization requires correction of depth-dependent lateral distortion, called wobble, see above.
Assessment: CompareLocalization3D performs 8 assessments:
The open-source software CompareLocalization3D is released as a stand-alone Java application.
© 2017
Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Last update: 31 Mar 2017