With the acquisition and widespread use of more resources that rely on accelerator/wide vector–based computing, there has been a strong demand for science and engineering applications to take advantage of these latest assets. This, however, has been extremely challenging due to the diversity of systems to support their extreme concurrency, complex memory hierarchies, costly data movement, and heterogeneous node architectures. To address these challenges, we design a programming model and describe its ease of use in the development of a new MAGMA Templates library that delivers high-performance scalable linear algebra portable on current and emerging architectures. MAGMA Templates derives its performance and portability by (1) building on existing state-of-the-art linear algebra libraries, like MAGMA, SLATE, Trilinos, and vendor-optimized math libraries, and (2) providing access (seamlessly to the users) to the latest algorithms and architecture-specific optimizations through a single, easy-to-use C++-based API.
AbalenkovsMAbdelfattahADongarraJ, et al. (2015) Parallel programming models for dense linear algebra on heterogeneous systems. Supercomputing Frontiers and Innovations2(4): 67–86. DOI:10.14529/jsfi1504.
2.
AbdelfattahAAnztHBouteillerA, et al. (2017) Roadmap for the development of a linear algebra library for exascale computing: SLATE: software for linear algebra targeting Exascale. SLATE Working Notes 1, ICL-UT-17-02.
3.
AbduljabbarMAl FarhanMAl-HarthiN, et al. (2018) Extreme scale FMM-accelerated boundary integral equation solver for wave scattering. SIAM Journal on Scientific Computing41(3): C245–C268.
4.
AbduljabbarMAl FarhanMYokotaR, et al. (2017) Performance evaluation of computation and communication kernels of the Fast Multipole Method on Intel Manycore architecture. In: 23rd international conference on parallel and distributed computing Euro-Par 2017: Parallel Processing, LNCS, Vol. 10417. Santiago de Compostela, Spain: Springer, pp. 553–564.
5.
AgulloEDemmelJDongarraJ, et al. (2009) Numerical linear algebra on emerging architectures: the PLASMA and MAGMA projects. Journal of Physics: Conference Series180: 012037.
6.
Al FarhanMA (2019) Unstructured computations on emerging architectures. DOI:10.25781/KAUST. Available at: http://hdl.handle.net/10754/644902 (accessed 2 September 2019).
7.
Al FarhanMAKeyesDE (2018) Optimizations of unstructured aerodynamics computations for many-core architectures. IEEE Transactions on Parallel and Distributed Systems29(10): 2317–2332.
8.
Al FarhanMAKaushikDKKeyesDE (2016) Unstructured computational aerodynamics on many integrated core architecture. Parallel Computing59: 97–118. Theory and Practice of Irregular Applications.
AnztHTomovSDongarraJ (2014) Implementing a Sparse Matrix Vector Product for the Sell-C/Sell-C-σ Formats on NVIDIA GPUs. Technical Report UT-EECS-14-727.
11.
DavisTAHuY (2011) The university of florida sparse matrix collection. ACM Transactions on Mathematical Software38(1): 1:1–1:25.
12.
EdwardsHCTrottCRSunderlandD (2014) Kokkos: enabling manycore performance portability through polymorphic memory access patterns. Journal of Parallel and Distributed Computing74(12): 3202–3216. Available at: http://www.sciencedirect.com/science/article/pii/S0743731514001257 (accessed 2 September 2019). Domain-Specific Languages and High-Level Frameworks for High-Performance Computing.
GatesMKurzakJChararaA, et al. (2019) Slate: Design of a modern distributed and accelerated linear algebra library. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ‘19. New York, NY, USA: Association for Computing Machinery. ISBN 9781450362290. DOI:10.1145/3295500.3356223.
15.
GatesMLuszczekPAbdelfattahA, et al. (2017) C++ API for BLAS and LAPACK. Technical Report 2, ICL-UT-17-03. Revision 02-21-2018.
16.
GuoDGroppWOlsonLN (2016) A hybrid format for better performance of sparse matrix-vector multiplication on a GPU. The International Journal of High Performance Computing Applications30(1): 103–120.
17.
HaidarATomovSDongarraJ, et al. (2018) Harnessing gpu tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC ‘18. Piscataway, NJ, USA: IEEE Press, pp. 47:1–47:11.
18.
HaidarAWuPTomovS, et al. (2017) Investigating half precision arithmetic to accelerate dense linear system solvers. In: SC16 ScalA17: 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems. ACM, Denver, CO: ACM.
19.
HerouxMABartlettRAHowleVE, et al. (2005) An overview of the trilinos project. ACM Transactions on Mathematical Software31(3): 397–423.
20.
KurzakJGatesMChararaA, et al. (2019a) Least squares solvers for distributed-memory machines with gpu accelerators. In: Proceedings of the ACM International Conference on Supercomputing, ICS ‘19. New York, NY, USA: ACM. ISBN 978-1-4503-6079 -1, pp. 117–126. DOI:10.1145/3330345.3330356.
21.
KurzakJTsaiYMGatesM, et al. (2019b) Massively parallel automated software tuning. In: Proceedings of the 48th International Conference on Parallel Processing, ICPP 2019. New York, NY, USA: Association for Computing Machinery. ISBN 9781450362955. DOI:10.1145/3337821.3337908.
22.
KurzakJWuPGatesM, et al. (2017) Designing SLATE: software for linear algebra targeting Exascale. SLATE Working Notes 3, ICL-UT-17-06.
23.
LiRSaadY (2013) GPU-accelerated preconditioned iterative linear solvers. The Journal of Supercomputing63(2): 443–466.
24.
LiYDongarraJTomovS (2009) A note on auto-tuning GEMM for GPUs. In: Proceedings of the 2009 International Conference on Computational Science, ICCS’09. Baton Roube, LA: Springer.
25.
LiuWVinterB (2015) CSR5: an efficient storage format for cross-platform sparse matrix-vector multiplication. In: Proceedings of the 29th ACM on International Conference on Supercomputing, ICS ‘15. New York, NY, USA: ACM. ISBN 978-1-4503-3559 -1, pp. 339–350.
26.
MedinaDSSt-CyrAWarburtonT (2014) OCCA: a unified approach to multi-threading languages. arXiv preprint arXiv:1403.0968.
TomovSDongarraJBaboulinM (2010) Towards dense linear algebra for hybrid gpu accelerated manycore systems. Parallel Computing System Application36(5-6): 232–240.
29.
WhaleyRCPetitetADongarraJJ (2001) Automated empirical optimizations of software and the ATLAS project. Parallel Computing27(1): 3–35. New Trends in High Performance Computing.
