The design of a scientific data management system based on DOMAS at CSNS-II (preliminary stage)

Abstract

At the second stage of China Spallation Neutron Source (CSNS-II), it is predicted that 2 PB raw experimental data will be produced annually from twenty instruments. Scientific computing puts forward higher requirements for data sharing, utilization, retrieval, analysis efficiency, and security. However, the existing data management system (DMS) based on ICAT has several limitations including poor scalability of metadata database, imperfect data-management lifecycle and inflexible API. To ensure the accuracy, usability, scalability and efficiency of CSNS-II experimental data, a new scientific data management system is therefore designed based on the DOMAS framework developed by the Computing Center of IHEP. The data acquisition, transmission, storage and service systems are re-designed and tailored specifically for CSNS-II. Upon its completion, the new DMS will overcome the existing challenges and offer functions such as online display, search functionality and rapid download capabilities for metadata, raw data and analyzed data; flexible and user-friendly authorization; and data lifecycle management. Ultimately, the implementation of the new Data Management System (DMS) is expected to enhance the efficiency of experimental data analysis, propelling CSNS-II to achieve international advanced standards. Furthermore, it aims to reinforce self-reliance and technological strength in the field of science and technology at a high level in China. The development and deployment of the new DMS begin at the end of 2023.

Keywords

Data management metadata DOMAS CSNS

Get full access to this article

View all access options for this article.

References

Y.M.

Abiven and

Gagey, Synchrotron SOLEIL, Saint Aubin, France, Energy [GeV] 2, 2.75.

Borland and

Blednykh, The upgrade of the advanced photon source. Brookhaven National Lab. (BNL), Upton, NY (United States), 2018.

Flannery,

Matthews,

Griffin et al., ICAT: Integrating data infrastructure for facilities based science, in: 2009 Fifth IEEE International Conference on e-Science, IEEE, 2009, pp. 201–207.

Garoby,

Vergara,

Danared et al., The European spallation source design, Physica Scripta 93(1) (2017), 014001.

Hu,

Lei,

Wang et al., DOMAS: A data management software framework for advanced light sources, Journal of Synchrotron Radiation 31(2) (2024).

Jiao,

Xu,

X.H.

Cui et al., The HEPS project, Journal of synchrotron radiation 25(6) (2018), 1611–1618. doi:10.1107/S1600577518012110.

Könnecke,

F.A.

Akeroyd,

H.J.

Bernstein et al., The NeXus data format, Journal of applied crystallography 48(1) (2015), 301–305. doi:10.1107/S1600576714027575.

SciCat project-data acquisition, management and publication (available at: https://scicatproject.github.io/.

Sjöblom,

Padmanabhan,

Klingberg et al., Stable operation of the MAX IV Laboratory Synchrotron Facility, 2019.

10.

Wang,

Yu,

Liu et al., Deploying the big data science center at the Shanghai synchrotron radiation facility: The first superfacility platform in China, Machine Learning: Science and Technology 2(3) (2021), 035003.