Speaker
Description
Large-scale scientific imaging datasets -ranging from terabytes to petabytes- are increasingly central to neuroscience and other scientific fields. These datasets require heterogeneous tools for analysis and visualization, which impose conflicting requirements on file formats, metadata schemas, and storage access patterns. Converting between formats or duplicating data is a common workaround, but this introduces inefficiencies, storage overhead, and potential errors in large-volume workflows.
We present tiamat, the Tiled Image Access and Manipulation Toolkit, a flexible and extensible Python framework that facilitates reading, transforming, and exposing large image datasets through a configurable pipeline of readers, transformers, and interfaces.
Tiamat supports on-the-fly transformations such as normalization, axis reordering, and colormapping, while streaming data to diverse endpoints—including Napari, Neuroglancer, OpenSeadragon, Python/Numpy scripts, and FUSE—without requiring intermediate file conversion or duplication. We demonstrate its use within the EBRAINS platform, where tiamat delivers 1µm-resolution histological brain images from the BigBrain dataset directly from high-performance GPFS storage to web-based viewers and analysis clients.
Tiamat decouples data storage from visualization and analysis workflows, enabling modular, reusable, and domain-agnostic image processing pipelines.
Its plugin-based design and compatibility with multiple tools offer a scalable solution for managing large scientific image datasets. Tiamat is implemented in Python, released under the Apache 2.0 license, and deployed via docker. The source code is available here.
