Classical molecular dynamics (MD) calculations represent a significant part of the utilization time of high-performance computing systems. As usual, the efficiency of such calculations is based on an interplay of software and hardware that are nowadays moving to hybrid GPU-based technologies. Several well-developed open-source MD codes focused on GPUs differ both in their data management capabilities and in performance. In this work, we analyze the performance of LAMMPS, GROMACS and OpenMM MD packages with different GPU backends on Nvidia Volta and AMD Vega20 GPUs. We consider the efficiency of solving two identical MD models (generic for material science and biomolecular studies) using different software and hardware combinations. We describe our experience in porting the CUDA backend of LAMMPS to ROCm HIP that shows considerable benefits for AMD GPUs comparatively to the OpenCL backend.
AbrahamMJMurtolaTSchulzR, et al. (2015) GROMACS: high performance molecular simulations through multi-level parallelism from laptops to supercomputers. SoftwareX1–2: 19–25.
2.
AndersonJALorenzCDTravessetA (2008) General purpose molecular dynamics simulations fully implemented on graphics processing units. Journal of Computational Physics227(10): 5342–5359.
3.
AntropovA (2020) Diffusion of nanobubbles in fcc aluminum. JETP Letters112(5): 310–315.
4.
AntropovAStegailovV (2020) Nanobubbles diffusion in bcc uranium: theory and atomistic modelling. Journal of Nuclear Materials533: 152110.
5.
BaloghGDRegulyIZMudaligeGR (2018) Comparison of parallelisation approaches, languages, and compilers for unstructured mesh algorithms on GPUs. In: JarvisSWrightSHammondS (eds) High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation. Cham: Springer International Publishing, pp. 22–43.
6.
BastrakovSDonchenkoRGonoskovA, et al. (2012) Particle-in-cell plasma simulation on heterogeneous cluster systems. Journal of Computational Science3(6): 474–479.
7.
BerendsenHvan der SpoelDvan DrunenR (1995) GROMACS: a message-passing parallel molecular dynamics implementation. Computer Physics Communications91(1): 43–56.
8.
BrownWMKohlmeyerAPlimptonSJ, et al. (2012) Implementing molecular dynamics on hybrid high performance computers—particle-particle particle-mesh. Computer Physics Communications183(3): 449–459.
9.
BrownWMWangPPlimptonSJ, et al. (2011) Implementing molecular dynamics on hybrid high performance computers—short range forces. Computer Physics Communications182(4): 898–911.
10.
BrownWMYamadaM (2013) Implementing molecular dynamics on hybrid high performance computers—three-body potentials. Computer Physics Communications184(12): 2785–2793.
11.
Carter EdwardsHTrottCRSunderlandD (2014). Kokkos: enabling manycore performance portability through polymorphic memory access patterns. Journal of Parallel and Distributed Computing74(12): 3202–3216.
12.
ChandarDDBoppanaBKumarV (2018). A comparative study of different overset grid solvers between OpenFOAM, StarCCM+ and Ansys-Fluent. In 2018 AIAA Aerospace Sciences Meeting, Kissimmee, FL, 8–12 January 2018.
13.
CorsettiF (2014) Performance analysis of electronic structure codes on HPC systems: a case study of SIESTA. PLoS One9(4): 1–8.
DoddaLSCabeza de VacaITirado-RivesJ, et al. (2017) LigParGen web server: an automatic OPLS-AA parameter generator for organic ligands. Nucleic Acids Research45(W1): W331–W336.
17.
DongWKangLQuanZ, et al. (2016) Implementing molecular dynamics simulation on Sunway TaihuLight system. In 2016 IEEE 18th international conference on high performance computing and communications; IEEE 14th international conference on smart city; IEEE 2nd international conference on data science and systems (HPCC/SmartCity/DSS), Sydney, NSW, 12–14 December 2016, pp. 443–450.
18.
DongWLiKKangL, et al. (2018) Implementing molecular dynamics simulation on the Sunway TaihuLight system with heterogeneous many-core processors. Concurrency and Computation: Practice and Experience30(16): e4468.
19.
DuanXGaoPZhangT, et al. (2018) Redesigning LAMMPS for peta-scale and hundred-billion-atom simulation on Sunway TaihuLight. In SC18: International conference for high performance computing, networking, storage and analysis, 11-16 Novmber 2018, Dallas, TX, USA, pp. 148–159.
20.
EastmanPFriedrichsMSChoderaJD, et al. (2013) OpenMM 4: a reusable, extensible, hardware independent library for high performance molecular simulation. Journal of Chemical Theory and Computation9(1): 461–469.
21.
EastmanPSwailsJChoderaJD, et al. (2017) OpenMM 7: rapid development of high performance algorithms for molecular dynamics. PLOS Computational Biology13: 1–17.
22.
GlaserJNguyenTDAndersonJA, et al. (2015) Strong scaling of general-purpose molecular dynamics simulations on GPUs. Computer Physics Communications192: 97–107.
23.
GolovynskyiASirenkoVLazarievT, et al. (2019). High performance computing system design for ANSYS CFD and mechanical codes. In 2019 IEEE 15th international conference on the experience of designing and application of cad systems (CADSM), Polyana, Ukraine, 26 February–2 March 2019, pp. 14–18.
24.
GongJMarkidisSLaureE, et al. (2016) Nekbone performance on GPUs with OpenACC and CUDA Fortran implementations. The Journal of Supercomputing72(11): 4160–4180.
25.
HalverRHombergWSutmannG (2018) Function portability of molecular dynamics on heterogeneous parallel architectures with OpenCL. The Journal of Supercomputing74(4): 1522–1533.
26.
HalverRMeinkeJHSutmannG (2020) Kokkos implementation of an Ewald Coulomb solver and analysis of performance portability. Journal of Parallel and Distributed Computing138: 48–54.
27.
HansonWA (2020) The CORAL supercomputer systems. IBM Journal of Research and Development64(3/4): 1:1–1:10.
28.
HuangJMacKerellADJr (2013) CHARMM36 all-atom additive protein force field: validation based on comparison to NMR data. Journal of Computational Chemistry34(25): 2135–2145.
29.
HumphreyWDalkeASchultenK (1996) VMD—visual molecular dynamics. Journal of Molecular Graphics14: 33–38.
30.
KondratyukNOrekhovM (2020) Shear viscosity of n-pentane from diffusivity based molecular dynamics methods. Journal of Physics: Conference Series1556: 012048.
KondratyukNDNormanGEStegailovVV (2016). Self-consistent molecular dynamics calculation of diffusion in higher n-alkanes. The Journal of Chemical Physics145(20): 204504.
33.
KondratyukNDPisarevVV (2019) Calculation of viscosities of branched alkanes from 0.1 to 1000 MPa by molecular dynamics methods using COMPASS force field. Fluid Phase Equilibria498: 151–159.
34.
KondratyukNDPisarevVVEwenJP (2020). Probing the high-pressure viscosity of hydrocarbon mixtures using molecular dynamics simulations. The Journal of Chemical Physics153(15): 154502.
35.
KutznerCPállSFechnerM, et al. (2015) Best bang for your buck: GPU nodes for GROMACS biomolecular simulations. Journal of Computational Chemistry, 36(26): 1990–2008.
36.
KutznerCPállSFechnerM, et al. (2019) More bang for your buck: improved use of GPU nodes for GROMACS 2018. Journal of Computational Chemistry40(27): 2418–2431.
37.
KuznetsovEKondratyukNLogunovM, et al. (2020) Performance and portability of state-of-art molecular dynamics software on modern GPUs. In: WyrzykowskiRDeelmanEDongarraJKarczewskiK (eds) Parallel Processing and Applied Mathematics. Cham: Springer International Publishing, pp. 324–334.
38.
KuznetsovEStegailovV (2019) Porting CUDA-based molecular dynamics algorithms to AMD ROCm platform using hip framework: performance analysis. In: VoevodinVSobolevS (eds) Supercomputing. Cham: Springer International Publishing, pp. 121–130.
39.
LeeJChengXSwailsJM, et al. (2016) CHARMM-GUI input generator for NAMD, GROMACS, AMBER, OpenMM, and CHARMM/OpenMM simulations using the CHARMM36 additive force field. Journal of Chemical Theory and Computation12(1): 405–413.
40.
LópezANichollsWSticklandMT, et al. (2015) CFD study of jet impingement test erosion using Ansys Fluent and OpenFOAM. Computer Physics Communications197: 88–95.
41.
MeiXChuX (2017) Dissecting GPU memory hierarchy through microbenchmarking. IEEE Transactions on Parallel and Distributed Systems28(1): 72–86.
42.
MerzPTShirtsMR (2018) Testing for physical validity in molecular simulations. PLoS One13(9): 1–22.
43.
MesnardOBarbaLA (2017) Reproducible and replicable computational fluid dynamics: it’s harder than you think. Computing in Science & Engineering19(4): 44–55.
44.
NikolskiiVStegailovV (2018) Domain-decomposition parallelization for molecular dynamics algorithm with short-ranged potentials on epiphany architecture. Lobachevskii Journal of Mathematics39(9): 1228–1238.
45.
NikolskiiVVecherVStegailovV (2016) Performance of MD-algorithms on hybrid systems-on-chip Nvidia Tegra K1 & X1. In: VoevodinVSobolevS (eds) Supercomputing. Cham: Springer International Publishing, pp. 199–211.
PisarevVKondratyukN (2019) Prediction of viscosity-density dependence of liquid methane+n-butane+n-pentane mixtures using the molecular dynamics method and empirical correlations. Fluid Phase Equilibria501: 112273.
51.
PlimptonS (1995) Fast parallel algorithms for short-range molecular dynamics. Journal of Computational Physics117(1): 1–19.
52.
RegulyIZMudaligeGR (2020) Productivity, performance, and portability for computational fluid dynamics applications. Computers Fluids199: 104425.
RojekKWyrzykowskiR (2017) Performance modeling of 3D MPDATA simulations on GPU cluster. The Journal of Supercomputing73(2): 664–675.
55.
RojekKWyrzykowskiRKuczynskiL (2016) Systematic adaptation of stencil-based 3D MPDATA to GPU architectures. Concurrency and Computation: Practice and Experience29(9): e3970.
56.
RomeroJPhillipsERuetschG, et al. (2018) A performance study of Quantum Espresso’s PWscf code on multi-core and GPU systems. In: JarvisSWrightSHammondS (eds) High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation. Cham: Springer International Publishing, pp. 67–87.
57.
RusuVHHortaVAHortaBA, et al. (2014) MDWiZ: a platform for the automated translation of molecular dynamics simulations. Journal of Molecular Graphics and Modelling48: 80–86.
58.
ShawDEGrossmanJPBankJA, et al. (2014) Anton 2: raising the bar for performance and programmability in a special-purpose molecular dynamics supercomputer. In SC ‘14: Proceedings of the international conference for high performance computing, networking, storage and analysis, New Orleans, LA, USA, 16–21 November 2014, pp. 41–53.
59.
ShirtsMRKleinCSwailsJM, et al. (2017) Lessons learned from comparing molecular dynamics engines on the SAMPL5 dataset. Journal of Computer-Aided Molecular Design31(1): 147–161.
60.
SmirnovGSStegailovVV (2016) Efficiency of classical molecular dynamics algorithms on supercomputing hardware. Matematicheskoe Modelirovanie28(5): 95–108.
61.
SmirnovGSStegailovVV (2019). Formation free energies of point defects and thermal expansion of bcc U and Mo. Journal of Physics: Condensed Matter31(23): 235704.
62.
StegailovVDlinnovaEIsmagilovT, et al. (2019a) Angara interconnect makes GPU-based Desmos supercomputer an efficient tool for molecular dynamics calculations. The International Journal of High Performance Computing Applications33(3): 507–521.
63.
StegailovVSmirnovGVecherV (2019b) VASP hits the memory wall: processors efficiency comparison. Concurrency and Computation: Practice and Experience31(19): e5136.
64.
StegailovVVOrekhovNDSmirnovGS (2015) HPC hardware efficiency for quantum and classical molecular dynamics. In: International conference on parallel computing technologies, Petrozavodsk, Russia, 31 August–4 September 2015, pp. 469–473.
65.
SunYMukherjeeSBaruahT, et al. (2018) Evaluating performance tradeoffs on the Radeon Open Compute platform. In: 2018 IEEE international symposium on performance analysis of systems and software (ISPASS), Belfast, UK, 2–4 April 2018, pp. 209–218.
66.
SurminIBastrakovSEfimenkoE, et al. (2016) Particle-in-cell laser-plasma simulation on Xeon Phi coprocessors. Computer Physics Communications202: 204–210.
67.
TchipevNSecklerSHeinenM, et al. (2019) TweTriS: twenty trillion-atom simulation. The International Journal of High Performance Computing Applications33(5): 838–854.
68.
TeijeiroCHammerschmidtTDrautzR, et al. (2019) Optimized parallel simulations of analytic bond-order potentials on hybrid shared/distributed memory with MPI and OpenMP. The International Journal of High Performance Computing Applications33(2): 227–241.
69.
TsaiYMCojeanTRibizelT, et al. (2020). Preparing Ginkgo for AMD GPUs—a testimonial on porting CUDA code to HIP. arXiv preprint arXiv:2006.14290.
70.
VermaasJVHardyDJStoneJE, et al. (2016) TopoGromacs: automated topology conversion from CHARMM to GROMACS within VMD. Journal of Chemical Information and Modeling56(6): 1112–1116.
71.
YangCKurthTWilliamsS (2020). Hierarchical roofline analysis for GPUs: accelerating performance optimization for the NERSC-9 perlmutter system. Concurrency and Computation: Practice and Experience32(20): e5547.
72.
YuYAnHChenJ, et al. (2017) Pipelining computation and optimization strategies for scaling GROMACS on the Sunway many-core processor. In: IbrahimSChooK-KRYanZPedryczW (eds) Algorithms and Architectures for Parallel Processing. Cham: Springer International Publishing, pp. 18–32.
73.
ZubairMWarnerJWagnerD (2019) Optimization of a solver for computational materials and structures problems on NVIDIA Volta and AMD instinct GPUs. In 2019 IEEE/ACM 10th workshop on latest advances in scalable algorithms for large-scale systems (ScalA), Denver, CO, USA, 18 November 2019, pp. 9–16.
