To the Editor — Cutting-edge innovations in biological microscopy are increasingly blurring the line between data acquisition and data analysis. Computational microscopy and machine-learning-based methods take this paradigm to an extreme, often producing raw measurements that are not human-interpretable without postprocessing1,2,3. Furthermore, new data-adaptive imaging methods rely on data processing during acquisition to actively control various hardware settings of the microscope.

Testing new ideas and applying them for biological discovery is often impeded by a lack of control software that is capable of meeting demands for speed and performance, integrating new and diverse types of hardware, providing the flexibility to adapt in real time to the data being captured, and providing user-friendly programming interfaces. As a result, researchers often end up developing custom software that works only with specific instruments, using closed-source and/or proprietary programming languages. In addition to slowing development, this lack of a consistent software ecosystem creates barriers to reproduction and replication of new techniques. These include the increased likelihood of bugs in code that is not reused and tested in different contexts, the burden on new users of understanding new code bases, and the extra work of disentangling high-level code that is not properly abstracted from the underlying hardware. In many fields the Python programming language — in particular, the NumPy and SciPy ecosystem4,5 — has emerged as a common framework for reproducing research results that rely on complex, multilayer scientific workflows in a portable, scriptable form6 that is accessible to researchers with various levels of programming experience.

The central challenge of such a consolidated approach in microscopy is the diversity of hardware used in different types of microscopes, ranging from custom-built components on optical tables to turnkey commercial systems. µManager7,8 is an essential tool for controlling microscopes, thanks to an extensive library of ‘device adapters’ for controlling different types of hardware, from cameras to complete microscopes, with a single programming interface. Community contributions of device adapters, plug-ins and scripts provide hundreds of developer-years of microscopy automation.

Despite the power of these libraries, which are written in C++ and Java, they are often difficult to integrate with the latest developments in computer vision and scientific computing, which most readily interface with NumPy. This not only increases the difficulty of developing techniques that rely on both customized data capture and data analysis, but also hinders the dissemination and adoption of these techniques by fragmenting them across multiple tools and programming languages and making them harder to understand and test in new contexts.

Many groups are developing microscopy control software that is written mostly or entirely in Python, including, Python Microscope9, Python Microscopy Environment (http://www.python-microscopy.org/) and Tormenta10 (a complete list is at https://doi.org/10.5281/zenodo.4433237). However, these tools are currently limited to specific types of microscopy (for example, super-resolution) and/or support a relatively small number of hardware devices. Thus, despite the potential benefits of such a design, they face an enormous task to match µManager’s flexibility and device support.

To address these needs, we present here Pycro-Manager. Pycro-Manager is built on a translation layer that converts Java objects, functions and data into language-agnostic messages that are reconstituted as Python objects or functions and as NumPy arrays (Fig. 1a and Supplementary Information). As a result, the existing capabilities of µManager can be called as if they had been written in Python, without forcing users to learn Java, forcing Java developers to learn Python, or abandoning of the relative strengths of either language (Supplementary Information).

Fig. 1: Pycro-Manager.
figure 1

a, Software architecture overview. Grey: the existing parts of µManager provide generic microscope control abstracted from specific hardware, a graphical user interface (GUI), a Java plug-in interface, and an acquisition engine that automates various aspects of data collection. Orange: Pycro-Manager enables access to these components through Python over a network-compatible transport layer, as well as a concise, high-level programming interface for acquiring data. Purple: these provide integration of data acquisition with Python libraries for hardware control, data visualization, scientific computing and so forth. b, Pycro-Manager’s high-level programming interface. The data acquisition process in Pycro-Manager starts with a source of acquisition events (blue), from either programming or a GUI. These events are passed to the acquisition engine (green), which optimizes them to take advantage of hardware triggering where available, sends instructions to hardware and acquires images. The resulting images are then saved and displayed in the GUI (magenta). The three main abstractions of the Pycro-Manager high-level programming interface (acquisition events, acquisition hooks, and image processors) enable fine-grained control and customization of this process. c, Code examples. Code snippets for implementing acquisition events (blue), acquisition hooks (green) and image processors (magenta). (“Python” and the Python logos are trademarks or registered trademarks of the Python Software Foundation, used with permission from the Foundation. C++ logo from http://isocpp.org/about).

In addition, Pycro-Manager provides a new library of high-level programmatic building blocks (in Python) for customizing data acquisition. These are designed to facilitate creation of complex data acquisitions with concise, readable code while maintaining the flexibility required for customization and the ability to handle multi-terabyte datasets acquired at speeds over 1 GB s–1 (Supplementary Information).

The high-level application programming interface (API) contains three main abstractions: acquisition events, acquisition hooks and image processors (Fig. 1b,c). These building blocks can be used individually or combined for more complex applications. Acquisition events enable customized instructions for how to adjust hardware when acquiring images and how to index those images along arbitrary axes (for example, time, z, etc.) for storage and display. For common microscopy workflows like time-lapses and z-stacks, events can be automatically generated by high-level functions (Fig. 1c). Alternatively, they can be created manually from nested Python lists and dictionaries to allow a greater degree of customization. Acquisition hooks enable the execution of arbitrary Python code concurrently at various stages of the data acquisition process. Image processors give access to the image data as soon as it is acquired, for modification or for use in data-driven feedback loops on the acquisition process.

The combination of Jupyter notebooks (https://jupyter.org/) — interactive documents that consolidate text, code, equations and results — and the Pycro-Manager high-level API provides a means to describe the full workflow of a research project, from data acquisition to analysis to results, thereby facilitating understanding, dissemination and reproduction of new microscopy technologies. For techniques that rely heavily on computation, these notebooks can be as valuable as (or even more valuable than) their corresponding research papers.

In the Supplementary Information and online documentation (https://pycro-manager.readthedocs.io/en/latest/#), we provide tutorials describing the basic features of Pycro-Manager, as well as Jupyter notebooks outlining sample applications in microscope alignment, light sheet microscopy, integrated sample processing, computational microscopy and machine learning, and the control of microscopes over networks.

The source code and documentation for Pycro-Manager can be found in the Supplementary Information, and future updates will be posted at https://github.com/micro-manager/pycro-manager and https://pycro-manager.readthedocs.io/en/latest/#.

Reporting summary

Further information on experimental design is available in the Nature Research Reporting Summary linked to this paper.