The process of empirical autotuning results in the generation of many code variants which are tested, found to be suboptimal, and discarded. By retaining annotated performance profiles of each variant tested over the course of many autotuning runs of the same code across different hardware environments and different input datasets, we can apply machine learning algorithms to generate classifiers for runtime selection of code variants from a library, generate specialized variants, and potentially speed the process of autotuning by starting the search from a point predicted to be close to optimal. In this paper, we show how the TAU Performance System suite of tools can be applied to autotuning to enable reuse of performance data generated through autotuning.
BellRMalonyAShendeS (2003) Paraprof: A portable, extensible, and scalable tool for parallel performance profile analysis. In: Euro-Par 2003 Parallel Processing, pp. 17–26.
2.
BrowneSDongarraJGarnerNHoGMucciP (2000) A portable programming interface for performance evaluation on modern processors. International Journal of High Performance Computing Application14(3): 189–204. DOI: 10.1177/109434200001400303.
3.
CavazosJFursinGAgakovFBonillaEO’BoyleMFPTemamO (2007) Rapidly selecting good compiler optimizations using performance counters. In: Proceedings of the International Symposium on Code Generation and Optimization (CGO ‘07). Washington, DC: IEEE Computer Society, pp. 185–197. DOI: 10.1109/CGO.2007.32.
4.
ChenCChameJHallM (2008) CHiLL: A framework for composing high-level loop transformations. Technical report, University of Utah.
5.
FrigoSteven MJohnsonG (2005) The design and implementation of FFTW3. Proceedings of the IEEE93(2): 216–231.
6.
GanapathiADattaKFoxAPattersonD (2009) A case for machine learning to optimize multicore performance. In: Proceedings of the First USENIX conference on Hot topics in parallelism (HotPar ‘09). Berkeley, CA, USA: USENIX Association.
7.
HallMFrankEHolmesGPfahringerBReutemannPWittenIH (2009) The WEKA data mining software: an update. SIGKDD Explorations Newsletter11(1): 10–18. DOI: 10.1145/1656274.1656278.
8.
HartonoANorrisBSadayappanP (2009) Annotation-based empirical performance tuning using Orio. In: Proceedings of the 23rd IEEE International Parallel & Distributed Processing Symposium, Rome, Italy. Also available as Preprint ANL/MCS-P1556-1008, http://www.mcs.anl.gov/uploads/cels/papers/P1556.pdf.
9.
HollingsworthJTiwariA (2010) End-to-End Auto-Tuning with Active Harmony, chapter 10. Boca Raton, FL: CRC Press, pp. 217–238. DOI: 10.1201/b10509-11.
10.
HuckKAMalonyAD (2005) PerfExplorer: A performance data mining framework for large-scale parallel computing. In: Proceedings of the 2005 ACM/IEEE conference on Supercomputing (SC ‘05). Washington, DC, USA: IEEE Computer Society, pp. 41–53. DOI: 10.1109/SC.2005.55.
11.
HuckKAMalonyADBellRMorrisA (2005) Design and implementation of a parallel performance data management framework. In: Proceedings of the International Conference on Parallel Computing, pp. 473–482.
12.
LiaoCQuinlanDJVuducRPanasT (2010) Effective source-to-source outlining to support whole program empirical optimization. In: Proceedings of the 22nd international conference on Languages and Compilers for Parallel Computing (LCPC ‘09). Berlin: Springer-Verlag, pp. 308–322. DOI: 10.1007/978-3-642-13374-9_21.
13.
LindlanKACunyJMalonyAD. (2000) A tool framework for static and dynamic analysis of object-oriented software with templates. In: Proceedings of the 2000 ACM/IEEE conference on Supercomputing (Supercomputing ‘00). Washington, DC, USA: IEEE Computer Society (CDROM).
14.
MalonyADBiersdorffSSpearWMayanglambamS (2010) An experimental approach to performance measurement of heterogeneous parallel applications using cuda. In: Proceedings of the 24th ACM International Conference on Supercomputing (ICS ‘10). New York: ACM Press, pp. 127–136. DOI: 10.1145/1810085.1810105.
15.
MonsifrotABodinFQuiniouR (2002) A machine learning approach to automatic production of compiler heuristics. In: Proceedings of the 10th International Conference on Artificial Intelligence: Methodology, Systems, and Applications (AIMSA ‘02). London: Springer-Verlag, pp. 41–50.
16.
MurthyGRavishankarMBaskaranMSadayappanP (2010) Optimal loop unrolling for GPGPU programs. In: 2010 IEEE International Symposium on Parallel Distributed Processing (IPDPS), pp. 1–11. DOI: 10.1109/IPDPS.2010.5470423.
17.
PellegriniSWangJFahringerTMoritschH (2009) Optimizing MPI runtime parameter settings by using machine learning. In: Proceedings of the 16th European PVM/MPI Users’ Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface. Berlin: Springer-Verlag, pp. 196–206. DOI: 10.1007/978-3-642-03770-2_26.
18.
QuinlanDJ (2000) ROSE: Compiler support for object-oriented frameworks. Parallel Processing Letters10(2/3): 215–226.
19.
QuinlanJR (1993) C4.5: Programs for Machine Learning. San Francisco, CA: Morgan Kaufmann Publishers Inc.
20.
RudyGKhanMMHallMChenCChameJ (2011) A programming language interface to describe transformations and code generation. In: Proceedings of the 23rd international conference on Languages and compilers for parallel computing (LCPC ‘10). Berlin: Springer-Verlag, pp. 136–150.
21.
ShendeSSMalonyAD (2006) The TAU parallel performance system. International Journal of High Performance Computing Application20(2): 287–311. DOI: 10.1177/1094342006064482.
22.
ShinJHallMWChameJChenCFischerPFHovlandPD (2010) Speeding up nek5000 with autotuning and specialization. In: Proceedings of the 24th ACM International Conference on Supercomputing (ICS ‘10). New York: ACM Press, pp. 253–262. DOI: 10.1145/1810085.1810120.
23.
StephensonMAmarasingheS (2005) Predicting unroll factors using supervised classification. In: Proceedings of the International Symposium on Code Generation and Optimization (CGO ‘05). Washington, DC: IEEE Computer Society, pp. 123–134. DOI: 10.1109/CGO.2005.29.
24.
TiwariA (2011) Tuning Parallel Applications in Parallel. Ph.D. thesis, University of Maryland, College Park.
25.
TiwariAHollingsworthJK (2011) Online adaptive code generation and tuning. In: Proceedings of the 2011 IEEE International Parallel and Distributed Processing Symposium (IPDPS ‘11). Washington, DC: IEEE Computer Society, pp. 879–892. DOI: 10.1109/IPDPS.2011.86.
26.
TiwariAHollingsworthJKChenC. (2011) Auto-tuning full applications: A case study. International Journal of High Performance Computing Application25(3): 286–294. DOI: 10.1177/1094342011414744.
27.
WhaleyCPetitetADongarraJJ (2000) Automated empirical optimization of software and the ATLAS project. Parallel Computing27: 2001.
28.
WhaleyRCDongarraJJ (1998) Automatically tuned linear algebra software. In: Proceedings of the 1998 ACM/IEEE conference on Supercomputing (Supercomputing ‘98). Washington, DC: IEEE Computer Society, pp. 1–27(CDROM).
