Speaker
Description
Robust, validated, and widely accessible tools are necessary for consistent scientific advancement and discovery. In neuroimaging, scientific advancement has outpaced the availability of well-implemented, trustworthy tools, forcing researchers to generate in-house software. Unfortunately, such in-house software rarely follows best practices of software engineering, documentation, testing, or validation, relying on lab- or individual-specific approaches instead.
Compounding the challenges at hand, descriptions of implementation, evaluation and/or benchmarking for neuroimaging software are typically sparse, reducing transparency for users and creating a risk of inappropriate usage. While the scale, complexity, and analytic acumen required for neuroimaging studies have increased exponentially over the last decade, there has not been a corresponding emphasis on software infrastructure to support such advances, and this has perpetuated a crisis of reproducibility through unaccounted heterogeneity across tools and algorithms.
We present the NMIND Consortium, a collaborative focused on standardizing scientific software development best practices, facilitating tool evaluation, and minimizing redundancy in the field.
NMIND has adopted three guiding principles:
-
Alignment, which refers to development and adoption of standards for critical components of data processing, analysis pipelines, and associated software. These include, but are not limited to: 1) coding & infrastructure standards, 2) testing & benchmarking standards, and 3) documentation standards.
-
Testing, which refers to accessible mechanisms for evaluating the compliance of tools with the NMIND standards. Tool contributors will be able to interact with these mechanisms through Web-based public interfaces and programmatic toolkits.
-
Engagement, which refers to the widespread promotion and adoption of the NMIND collaborative standards and testing in the field, through the efforts of field researchers, educators, and resource generators.
Towards the above-mentioned objectives, there are several ways in which members of the community can currently participate in NMIND.
For tool evaluation, we invite use of the NMIND Coding Standards Checklist (available at https://nmind.org). Integration of the checklist into software development and release workflows can assist with publication, and improve documentation, infrastructure and testing surrounding tools. Importantly, we welcome feedback, and expect these checklists will evolve as living standards within the community.
You can join us monthly for our regular one-day Hackathons, oriented around establishing a community and a common time to work on improving NMIND toolkits and to interact with other community efforts (e.g., BIDS, ReproNim, HBCD). Outcomes of group-based tool reviews, a regular focus of these events, are shared under the Proceedings section of the NMIND website. Software coding skills are not required for participation.
The past decade of the neuroimaging community has witnessed landmark advances in data collection, processing and analysis. Though, arguably, the greatest advance has been the emergence of an open science culture, with open data, tools and knowledge bases serving as incubators and accelerators for collaboration. Looking forward, the NMIND collaborative model will be essential for the field to take the next major step in its evolution towards a reproducible science capable of delivering critically needed scientific and clinical deliverables.
Please see our full paper in Nature Human Behavior: https://rdcu.be/dfGtn.
