A growing disparity between supercomputer computation speeds and I/O rates means that it is rapidly becoming infeasible to analyze supercomputer application output only after that output has been written to a file system. Instead, data-generating applications must run concurrently with data reduction and/or analysis operations, with which they exchange information via high-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis and reduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODAR motif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools that assist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings.
AbramsonDSosicRGiddyJ, et al. (1995) Nimrod: a tool for performing parametrised simulations using distributed workstations. In: 4th IEEE international symposium on high performance distributed computing, Washington, DC, USA, 2–4 August 1995, pp. 112–121. IEEE.
2.
AinsworthMTuglukOWhitneyB, et al. (2020) Multilevel techniques for compression and reduction of scientific data---The unstructured case. SIAM Journal on Scientific Computing42(2): A1402–A1427.
3.
AinsworthMTuglukOWhitneyB, et al. (2019) Multilevel techniques for compression and reduction of scientific data---The multivariate case. SIAM Journal on Scientific Computing41(2): A1278–A1303.
4.
AlexanderFAlmgrenABellJ, et al. (2020a) Exascale applications: skin in the game. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences378(2166): 20190056.
5.
AlexanderFJAngJBilbreyJA, et al. (2020b) Co-design center for exascale machine learning technologies (ExaLearn). International Journal of High-Performance Computing Applicationsin press.
6.
AsanovicKBodikRCatanzaroBC, et al. (2006) The Landscape of parallel Computing Research: A View from Berkeley. Berkeley: University of California, Technical Report UCB/EECS-2006-183.
7.
AyachitUBauerADuqueEP, et al. (2016) Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures. In: SC’16: proceedings of the international conference for high performance computing, networking, storage and analysis, Salt Lake City, UT, 13–18 November 2016, pp. 921–932. IEEE.
8.
AymarRBarabaschiPShimomuraY (2002) The ITER design. Plasma Physics and Controlled Fusion44(5): 519–565. DOI: 10.1088/0741-3335/44/5/304.
9.
BalaprakashPDongarraJGamblinT, et al. (2018) Autotuning in high-performance computing applications. Proceedings of the IEEE106(11): 2068–2083.
10.
BarrettRFBorkarSDosanjhSS, et al. (2013) On the role of co-design in high performance computing. In: (eds) Transition of HPC towards Exascale Computing. IOS Press, 141.
11.
BauerACAbbasiHAhrensJ, et al. (2016) In situ methods, infrastructures, and applications on high performance computing platforms. Computer Graphics Forum35: 577–597.
12.
BeazleyDMLomdahlPS (1996) Lightweight computational steering of very large scale molecular dynamics simulations. In: SC’96 computer graphics forum, Pittsburgh, PA, 1 January 1996, p. 50. IEEE.
13.
BraceALeeHMaH, et al. (2021) Achieving 100X faster simulations of complex biological phenomena by coupling ML to HPC ensembles. arXiv preprint arXiv:2104.04797.
14.
CarrieroNGelernterD (1989) Linda in context. Communications of the ACM32(4): 444–458.
15.
ChildsHAhernSDAhrensJ, et al. (2020) A terminology for in situ visualization and analysis systems. International Journal of High Performance Computing Applications34.
16.
ChoiJYChangCSDominskiJ, et al. (2018) Coupling exascale multiphysics applications: Methods and lessons learned. In: 14th international conference on e-Science, Amsterdam, The Netherlands, 29 October–1 November 2018, pp. 442–452. IEEE.
17.
ChoiJYLoganJMehtaK, et al. (2019) A co-design study of fusion whole device modeling using code coupling. In: IEEE/ACM 5th international workshop on data analysis and reduction for big scientific data, Denver, CO, 17 November 2019, pp. 35–41. IEEE.
18.
ChoiJYLoganJWolfM, et al. (2017) TGE: machine learning based task graph embedding for large-scale topology mapping. In: International conference on cluster computing, Honolulu, HI, 5–8 September 2017, pp. 587–591. IEEE.
19.
CurtisBKrasnerHIscoeN (1988) A field study of the software design process for large systems. Communications of the ACM31(11): 1268–1287.
20.
De MichellGGuptaRK (1997) Hardware/software co-design. Proceedings of the IEEE85(3): 349–365.
21.
DiSCappelloF (2016) Fast error-bounded lossy HPC data compression with SZ. In: IEEE international parallel and distributed processing symposium, Chicago, IL, 23–27 May 2016, pp. 730–739. IEEE.
22.
DiffenderferJFoxALHittingerJA, et al. (2019) Error analysis of ZFP compression for floating-point data. SIAM Journal on Scientific Computing41(3): A1867–A1898.
23.
DocanCParasharMKlaskyS (2012) DataSpaces: an interaction and coordination framework for coupled simulation workflows. Cluster Computing15(2): 163–181.
24.
DominskiJKuSChangC-S, et al. (2018) A tight-coupling scheme sharing minimum information across a spatial interface between gyrokinetic turbulence codes. Physics of Plasmas25(7): 072308. DOI: 10.1063/1.5044707.
25.
DuplyakinDBrownJRicciR (2016) Active learning in performance analysis. In: IEEE conference on cluster computing (CLUSTER), Taipei, Taiwan, 12–16 September 2016, pp. 182–191. IEEE.
26.
EfraimidisPSSpirakisPG (2006) Weighted random sampling with a reservoir. Information Processing Letters97(5): 181–185.
27.
FosterI (1994) Designing and Building Parallel Programs. Addison-Wesley.
28.
FosterIAinsworthMAllenB, et al. (2017) Computing just what you need: online data analysis and reduction at extreme scales. In: European conference on parallel processing, pp. 3–19.
29.
FosterIInsleyJvon LaszewskiG, et al. (1999) Distance visualization: data exploration on the grid. Computer32(12): 36–43.
30.
FosterIKesselmanC (2006) Scaling system-level science: scientific exploration and IT implications. Computer39(11): 31–39.
31.
GodoyWFPodhorszkiNWangR, et al. (2020) ADIOS 2: the adaptable input output system. A framework for high-performance data management. SoftwareX12: 100561.
32.
GrayJShenoyP (2000) Rules of thumb in data engineering. In: 16th international conference on data engineering, San Diego, CA, 29 February–3 March 2000, pp. 3–10. IEEE.
33.
GuoHLenzDXuJ, et al. (2021) FTK: a simplicial spacetime meshing framework for robust and scalable feature tracking. IEEE Transactions on Visualization & Computer Graphics.
HaSJeongWMatyasfalviG, et al. (2020) Chimbuko: a workflow-level scalable performance trace analysis tool. arXiv preprint arXiv:2008.13742
36.
HeWWangJGuoH, et al. (2019) InSituNet: deep image synthesis for parameter space exploration of ensemble simulations. IEEE Transactions on Visualization & Computer Graphics26(1): 23–33.
37.
HoeflerTGroppWKramerW, et al. (2011) Performance modeling for systematic performance tuning. In: SC’11: proceedings of 2011 international conference for high performance computing, networking, storage and analysis, Seattle, WA, 12–18 November 2011, pp. 1–12. IEEE.
38.
InsleyJAPapkaMEDongS, et al. (2007) Runtime visualization of the human arterial tree. IEEE Transactions on Visualization and Computer Graphics13(4): 810–821.
39.
JohnsonCParkerSGHansenC, et al. (1999) Interactive simulation and visualization. Computer32(12): 59–65.
40.
KlaskySAbbasiHLoganJ, et al. (2011) In situ data processing for extreme-scale computing. Proceedings of SciDAC: 1–16.
41.
LarsenMAhrensJAyachitU, et al. (2017) The alpine in situ infrastructure: ascending from the ashes of strawman. In: Workshop on in situ infrastructures on enabling extreme-scale analysis and visualization, pp. 42–46.
42.
LeeHTurilliMJhaS, et al. (2019) DeepDriveMD: deep-learning driven adaptive molecular simulations for protein folding. In: 3rd workshop on deep learning on supercomputers, Denver, CO, 17 November 2019, pp. 12–19. IEEE.
43.
LiangXDiSTaoD, et al. (2018) Error-controlled lossy compression optimized for high compression ratios of scientific datasets. In: International conference on big data, Seattle, WA, 10–13 December 2018, pp. 438–447. IEEE.
44.
MaKLWangCYuH, et al. (2007) In-situ processing and visualization for ultrascale simulations. Journal of Physics: Conference Series78: 012043.
45.
MalakarPMunsonTKnightC, et al. (2018) Topology-aware space-shared co-analysis of large-scale molecular dynamics simulations. In: SC’18: international conference for high performance computing, networking, storage and analysis, Dallas, TX, 11–16 November 2018, pp. 305–319. IEEE.
46.
MalakarPVishwanathVKnightC, et al. (2016) Optimal execution of co-analysis for large-scale molecular dynamics simulations. In: SC’16: proceedings of the international conference for high performance computing, networking, storage and analysis, Salt Lake City, UT, 13–18 November 2016, pp. 702–715. IEEE.
47.
MalakarPVishwanathVMunsonT, et al. (2015) Optimal scheduling of in-situ analysis for large-scale scientific simulations. In: SC’15: proceedings of the international conference for high performance computing, networking, storage and analysis, pp. 1–11. IEEE.
48.
MehtaKAllenBWolfM, et al. (2019) A codesign framework for online data analysis and reduction. In: IEEE/ACM workflows in support of large-scale science, Denver, CO, 17 November 2019, pp. 11–20. IEEE.
49.
OusterhoutJ (2018) A Philosophy of Software Design. Yaknyam Press.
50.
OzikJCollierNWozniakJM, et al. (2016) From desktop to large-scale model exploration with Swift/T. In: Winter simulation conference, Washington, DC, USA, 11–14 December 2016, pp. 206–220. IEEE.
51.
ParnasDL (1972) On the criteria to be used in decomposing systems into modules. In: (eds) Pioneers and Their Contributions to Software Engineering. Springer, 479–498.
52.
PouchardLHuckKMatyasfalviG, et al. (2018) Prescriptive provenance for streaming analysis of workflows at scale. In: New York scientific data summit, NY, USA, 6–8 August 2018, pp. 1–6. IEEE.
53.
RatanaworabhanPKeJBurtscherM (2006) Fast lossless compression of scientific floating-point data. In: Data compression conference, Snowbird, UT, 28–30 March 2006, pp. 133–142. IEEE.
54.
SchmidtDChenW-CMathesonMA, et al. (2017) Programming with big data in R: scaling analytics from one to thousands of nodes. Big Data Research8: 1–11.
55.
ShendeSSMalonyAD (2006) The TAU parallel performance system. The International Journal of High Performance Computing Applications20(2): 287–311.
56.
ShuTGuoYWozniakJ, et al. (2020) In-situ workflow auto-tuning via combining performance models of component applications. arXiv preprint arXiv:2008.06991
SubhlokJStichnothJMO’HallaronDR, et al. (1993) Exploiting task and data parallelism on a multicomputer. In: ACM SIGPLAN notices 4th symposium on principles and practice of parallel programming, 28, pp. 13–22. ACM.
59.
SuchytaEKlaskySPodhoeszkiN, et al. (2021) The exascale framework for high fidelity coupled simulations (EFFIS): enabling whole device modeling in fusion science. International Journal of High Performance Computing Applications.
60.
TaoDDiSChenZ, et al. (2017) Significantly improving lossy compression for scientific data sets based on multidimensional prediction and error-controlled quantization. In: IEEE international parallel and distributed processing symposium, Orlando, FL, 29 May–2 June 2017, pp. 1129–1139. IEEE.
61.
TaoDDiSGuoH, et al. (2019) Z-Checker: a framework for assessing lossy compression of scientific data. The International Journal of High Performance Computing Applications33(2): 285–303.
ValievMBylaskaEJGovindN, et al. (2010) NWChem: a comprehensive and scalable open-source solution for large scale molecular simulations. Computer Physics Communications181(9): 1477–1489.
64.
WozniakJMArmstrongTGWildeM, et al. (2013) Swift/T: scalable data flow programming for distributed-memory task-parallel applications. In: 13th international conference on cluster, cloud and grid computing, Delft, The Netherlands, 13–16 May 2013, pp. 95–102. IEEE.
65.
WozniakJMDavisPEShuT, et al. (2018a) Scaling deep learning for cancer with advanced workflow storage integration. In: IEEE/ACM machine learning in HPC environments, Dallas, TX, 12 November 2018, pp. 114–123. IEEE.
66.
WozniakJMDorierMRossR, et al. (2019) MPI jobs within MPI jobs: a practical way of enabling task-level fault-tolerance in HPC workflows. Future Generation Computer Systems101: 576–589.
67.
WozniakJMJainRBalaprakashP, et al. (2018b) CANDLE/supervisor: a workflow framework for machine learning applied to cancer research. BMC Bioinformatics19(18): 491.
68.
XieCXuWMuellerK (2019) A visual analytics framework for the detection of anomalous call stack trees in high performance computing applications. IEEE Transactions on Visualization and Computer Graphics25(1): 215–224.
69.
YakushinIMehtaKChenJ, et al. (2020) Feature-preserving lossy compression for in situ data analysis. In: 49th international conference on parallel processing: workshops, pp. 1–9. ACM.
70.
YooSHuangHKasiviswanathanSP (2016) Streaming spectral clustering. In: 32nd international conference on data engineering, Helsinki, Finland, 16–20 March 2016, pp. 637–648. IEEE.
71.
ZhangXHuangHMuellerK, et al. (2018) Streaming classical multidimensional scaling. In: New York scientific data summit, NY, USA, 6–8 August 2018, pp. 1–2. IEEE.
72.
ZhaoKDiSLiangX, et al. (2020) Significantly improving lossy compression for HPC datasets with second-order prediction and parameter optimization. In: 29th international symposium on high-performance parallel and distributed computing, pp. 89–100. IEEE.
