Finding the forest in the trees: Enabling performance optimization on heterogeneous architectures through data science analysis of ensemble performance data

Abstract

In this work, we develop novel data science methodologies for ensemble performance data that have the potential to uncover orders of magnitude of performance that is unknowingly being left on the table. Building on years of successful performance tool design and tool integration into million-line codes at Lawrence Livermore National Laboratory (Caliper (Boehme et al. 2016), Hatchet (Bhatele et al. 2019; Brink et al. 2020))—successes highlighted as key deliverables in meeting LLNL’s L1 and L2 milestones (Rieben and Weiss 2020)—we design a data science methodology for integrating multi-dimensional, multi-scale, multi-architecture, and multi-tool performance data, and provide data analytics and interactive visualization capabilities for further analysis and exploration of the data. Our work provides developers with a comprehensive multi-dimensional performance landscape, enabling enhanced capabilities for pinpointing performance bottlenecks on emerging hardware platforms composed of heterogeneous elements.

Keywords

HPC performance analysis exploratory data analysis heterogeneous computing simulations

Get full access to this article

View all access options for this article.

References

Aaziz

Cook

, et al. (2018) A methodology for characterizing the correspondence between real and proxy applications. In: 2018 IEEE International Conference on Cluster Computing (CLUSTER). Belfast, UK: IEEE. pp. 190–200. DOI:10.1109/CLUSTER.2018.00037.

Adhianto

Banerjee

Fagan

, et al. (2010) Hpctoolkit: tools for performance analysis of optimized parallel programs. Concurrency and Computation: Practice and Experience 22(6): 685–701.

Bhatele

Brink

Gamblin

(2019) Hatchet: Pruning the Overgrowth in Parallel Profiles. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’19. New York, NY, USA: Association for Computing Machinery. ISBN 9781450362290. URL. DOI: 10.1145/3295500.3356219

Boehme

Gamblin

Beckingsale

, et al. (2016) Caliper: performance introspection for hpc software stacks. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’16. Salt Lake, UT, USA: IEEE Press. ISBN 9781467388153.

Brink

Lumsden

Scully-Allison

, et al. (2020) Usability and performance improvements in hatchet. In: 2020 IEEE/ACM International Workshop on HPC User Support Tools (HUST) and Workshop on Programming and Performance Visualization Tools (ProTools). GA, USA: IEEE, pp. 49–58. DOI: 10.1109/HUSTProtools51951.2020.00013.

Browne

Dongarra

Garner

, et al. (2000) A portable programming interface for performance evaluation on modern processors. The International Journal of High Performance Computing Applications 14(3): 189–204. URL. DOI: 10.1177/109434200001400303

Dongarra

Malony

Moore

, et al. (2004) Scalable performance analysis infrastructure for terascale systems. Future Generation Computer Systems - FGCS.

Hornung

Keasler

(2014) The RAJA Portability Layer: Overview and Status. https://www.osti.gov/biblio/1169830 URL. DOI: 10.2172/1169830

Islam

Ayala

Jensen

, et al. (2019) Toward a programmable analysis and visualization framework for interactive performance analytics. In: 2019 IEEE/ACM International Workshop on Programming and Performance Visualization Tools (ProTools). Denver, CO, USA: IEEE. pp. 70–77. DOI:10.1109/ProTools49597.2019.00015

10.

Islam

Thiagarajan

Bhatele

, et al. (2016) A machine learning framework for performance coverage analysis of proxy applications. In: SC ’16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Salt Lake, UT, USA: IEEE. pp. 538–549. DOI: 10.1109/SC.2016.45

11.

Karavanic

Miller

(1999) Improving online performance diagnosis by the use of historical performance data. In: SC ’99: Proceedings of the 1999 ACM/IEEE Conference on Supercomputing. OR, USA: IEEE. pp. 42–42. DOI: 10.1109/SC.1999.10029

12.

Knüpfer

Rössel

, et al (2012) Score-p: a joint performance measurement run-time infrastructure for periscope,scalasca, tau, and vampir. In: Brunst

Müller

Nagel

, et al. (eds), Tools for High Performance Computing 2011. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 79–91. ISBN 978-3-642-31476-6.

13.

Mellor-Crummey

(2003) HPCToolkit: multi-platform tools for profile-based performance analysis. In: 5th International Workshop on Automatic Performance Analysis (APART).

14.

Rieben

Weiss

(2020) The multiphysics on advanced platforms project. In: ATDM FY20 L1 Report. Technical report.

15.

Shende

Malony

(2006) The tau parallel performance system. The International Journal of High Performance Computing Applications 20(2): 287–311. URL. DOI: 10.1177/1094342006064482

16.

The Open SpeedShop Team (2006) Open SpeedShop for Linux. URL http://www.openspeedshop.org.

17.

Tukey

, et al. (1977) Exploratory data analysis. Reading, MA: volume 2.

18.

Yang

Gayatri

Kurth

, et al. (2018) An empirical roofline methodology for quantitatively assessing performance portability. In: 2018 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC). Dallas, TX, USA: IEEE. pp. 14–23. DOI:10.1109/P3HPC.2018.00005