Fluorescence microscopy is an indispensable tool for studying the spatiotemporal dynamics of cell and tissue development in life sciences, enabling high-resolution imaging of biological processes within cells. However, its imaging quality is limited by the microscope optics and the achievable fluorophore density, bleaching, and photo-toxicity¹.

Fluorescence microscopy-based image restoration has received widespread attention in the life sciences for its ability to break through the physical limits of fluorescence microscopy imaging, achieving better imaging speed, spatial resolution and depth. Benefiting from deep learning technology, fluorescence microscopy-based image restoration^1,2,3,4 has led to significant progress, better revealing the complex interactions and dynamics inside cells and biomolecules, and expanding the scope of observation of biological phenomena. However, most current task-specific methods have limited generalizability to different fluorescence microscopy-based image restoration problems.

Here, we seek to improve generalizability and explore the potential of applying a pre-trained foundation model to fluorescence microscopy-based image restoration. We provide a unified foundation model (UniFMIR) to address different restoration problems, including super-resolution, 3D denoising, isotropic reconstruction, image projection and volumetric reconstruction (Figure 1), and show that UniFMIR offers higher image restoration precision, better generalization, and increased versatility.

To demonstrate the generalization ability of our pretrained UniFMIR approach, we validated its image restoration performance on unseen data from DeepBacs⁴ for bacterial image analysis purposes, including two denoising datasets (E. coli_H-NS-mScarlet-I, E. coli_MreB) and two superresolution datasets (E. coli, S. aureus). We fine-tuned UniFMIR on the new bacterial microscopy images and then conducted denoising and superresolution experiments. In addition to the denoising and superresolution models developed on DeepBacs, we also compared our results with those of a baseline model trained from scratch, which had the same structure as UniFMIR, and the pretrained UniFMIR model without fine-tuning. Our UniFMIR method restored clear prokaryote structures to enhance the low-phototoxicity live-cell microscopy data and predict accurate mappings of biological target shapes, obtaining higher PSNR/SSIM values (Figure 2).

UniFMIR provides a universal solution for fluorescence microscopy-based image restoration. Pre-trained UniFMIR can be applied to different tasks, imaging modalities and biological structures through simple fine-tuning, demonstrating the great role of foundation model methods in biological imaging research. We believe that the image restoration capabilities of UniFMIR can be continuously enhanced by further expanding the amount and diversity of training data.

1. Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nature methods 15, 1090-1097 (2018).
2. Qiao, C. et al.Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nature methods, 1–9 (2021).
3. Wang, Z. et al. Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning. Nature Methods 18, 551–556 (2021).
4. Spahn, C. et al. Deepbacs for multi-task bacterial image analysis using open-source deep learning approaches. Commun Biol 5, (2022).