To understand how the brain supports behavior, researchers need to measure neural activity and behavior at the same time, and with enough precision to connect one to the other. This can sound simpler than it is. In many settings, the main difficulty is not obtaining the recordings themselves, but making sure that signals from different devices are aligned on the same timeline. When neural recordings, motion capture, video, muscle activity, and other physiological signals are out of sync, interpreting their relationship becomes substantially more difficult.

This problem was the starting point for our paper, Thalamus: A real-time system for synchronized, closed-loop multimodal behavioral and electrophysiological data capture. We developed Thalamus to address the practical challenge of synchronized multimodal data capture in clinical and experimental environments. Thalamus is an open-source platform designed to integrate heterogeneous data streams in real time, support closed-loop experiments, and work with the types of sensors and hardware already used in neurosurgical and clinical research. By combining flexibility, low-latency performance, and compatibility with existing workflows, the system was designed to make multimodal brain–behavior research more practical in real-world settings.

One of the key ideas that shaped the system emerged during development: synchronization is often misunderstood. It is tempting to think that different data streams must be perfectly matched and aligned during the experiment itself. In practice, forcing streams into alignment in real time can actually reduce data quality. Any processing—such as binning, resampling, or filtering—can discard information. Instead, Thalamus prioritizes recording accurate timestamps alongside raw data. This preserves the maximum amount of information and allows synchronization to be refined later, as analysis methods improve. In other words, rather than “locking in” one interpretation of timing, the system keeps the underlying data flexible.

The need for this type of system is especially clear in the operating room and related clinical settings. These environments place strict constraints on time, space, and workflow. Experimental systems must be set up quickly, function reliably, and integrate with existing equipment without creating additional complexity. At the same time, researchers increasingly want to study behavior in more detail and relate it directly to brain activity. This requires synchronized acquisition across multiple data streams, not only for later analysis, but also for real-time computation and feedback.

Existing tools address parts of this problem, but often do so asymmetrically. Some platforms focus on neural data alone. Others support synchronization, but only for a limited set of devices or with substantial hardware customization. Some are suitable for offline analysis but not for low-latency closed-loop use. Others are difficult to adapt to the practical demands of clinical research. Our aim with Thalamus was to address these needs within a single system by emphasizing seven design requirements: practicality, reliability, flexibility, low latency with closed-loop capability, signal transparency, open-source accessibility, and technical as well as real-world validation.

Thalamus is organized as a modular node-based pipeline. Each node receives, processes, and outputs data, and nodes can be combined to support a wide range of acquisition and analysis tasks. This structure makes the system adaptable to different experimental paradigms and device combinations. It also allows users to build new pipelines without redesigning the overall architecture. In practice, this means that neural signals, behavioral data, visual inputs, and other streams can be incorporated into the same synchronized framework.

A related design principle was that abstractions in the system should reflect the real-world behavior of the data. In software engineering, it is common to simplify systems through abstraction, but oversimplification can introduce hidden limitations. For example, treating all data streams as identical messages might make implementation easier, but it can obscure important differences and reduce performance. In Thalamus, external communication and storage use standardized message formats, but within the system, data is handled in a way that preserves efficiency and detail. For instance, image data can be passed directly in memory between processing steps, avoiding unnecessary copying or conversion. This approach reduces latency and enables the system to keep up with high-throughput data streams.

A key architectural decision in Thalamus was to separate the software into two layers. High-level application logic and user interaction are handled in Python, while time-sensitive acquisition and control functions are handled in C++. This tiered structure allowed us to combine flexibility and ease of development with the performance needed for low-latency operation. Communication between these layers is managed through gRPC, which also supports synchronization of application state and data streaming. This design made it possible to support real-time visualization, closed-loop computation, and extensibility across devices and programming environments.

An important aspect of the project was to ensure that the system was not only flexible in principle, but also validated in practice. We therefore evaluated Thalamus through a series of bench and clinical tests. These included synchronization benchmarks, video processing under load, real-time analysis, and closed-loop latency measurements. Across these tests, the system demonstrated millisecond-range precision. In the closed-loop bench test, the best median roundtrip latency was 0.536 ms, with an internal latency of 0.036 ms. These results show that a general-purpose multimodal platform can still meet the timing requirements needed for demanding closed-loop applications.

Reliability was equally important. In clinical research, data collection opportunities are often brief and cannot be repeated. A software failure during that period can result in permanent loss of valuable information. For that reason, Thalamus was designed with fail-safe features and recovery in mind. In the validation reported in the paper, no crashes occurred during bench or clinical testing. Even when a crash was intentionally induced, 99.999% of data packets were recovered. This level of robustness is essential for the kinds of settings in which the system is intended to operate.

Behind the scenes, achieving this level of reliability required attention not just to the code itself, but to how the system is built and deployed. A recurring challenge in research software is that systems become tied to a single machine or configuration, making them difficult to maintain or reproduce. Considerable effort went into ensuring that Thalamus could be built from source across a wide range of environments, from laboratory workstations to hospital laptops. This makes it possible to reproduce bugs, test updates, and adapt the system without disrupting ongoing experiments. In practice, this kind of portability can be as important as performance.

We also considered openness and long-term usability to be central to the project. Many multimodal research systems are developed locally for a specific study and are difficult for others to reuse or extend. By making Thalamus open source and modular, we aimed to provide a platform that other researchers can adapt to their own devices, paradigms, and computational needs. This is particularly important in a field where hardware and analysis methods continue to evolve rapidly. A flexible and transparent framework can reduce duplicated effort and support more reproducible development across groups.

More broadly, this work reflects a practical point that often receives less attention than the final scientific findings: high-quality neuroscience depends on high-quality infrastructure. When data streams are not synchronized, it becomes difficult to interpret the relationship between brain activity and behavior with confidence. When systems are too rigid or unreliable, promising experimental opportunities may be lost. By addressing synchronization, integration, and real-time processing together—and by preserving the fidelity of the underlying data—Thalamus was developed to help make multimodal and closed-loop neuroscience more feasible in clinical settings.