This work covers the PHAST Library’s employment, a hardware-agnostic programming library, to a real-world application like the Caffe framework. The original implementation of Caffe consists of two different versions of the source code: one to run on CPU platforms and another one to run on the GPU side. With PHAST, we aim to develop a single-source code implementation capable of running efficiently on CPU and GPU. In this paper, we start by carrying out a basic Caffe implementation performance analysis using PHAST. Then, we detail possible performance upgrades. We find that the overall performance is dominated by few ‘heavy’ layers. In refining the inefficient parts of this version, we find two different approaches: improvements to the Caffe source code and improvements to the PHAST Library itself, which ultimately translates into improved performance in the PHAST version of Caffe. We demonstrate that our PHAST implementation achieves performance portability on CPUs and GPUs. With a single source, the PHAST version of Caffe provides the same or even better performance than the original version of Caffe built from two different codebases. For the MNIST database, the PHAST implementation takes an equivalent amount of time as native code in CPU and GPU. Furthermore, PHAST achieves a speedup of 51% and a 49% with the CIFAR-10 database against native code in CPU and GPU, respectively. These results provide a new horizon for software development in the upcoming heterogeneous computing era.
AdveSBodikR (2019) I-USHER: Interfaces to Unlock the Specialized Hardware Revolution. Information Science and Technology (ISAT), p. 27. URLhttp://rsim.cs.illinois.edu/Talks/I-USHER.pdf.
AmaralVNorbertoBGoulãoM, et al. (2020) Programming languages for data-Intensive HPC applications: a systematic mapping study. Parallel Computing91: 102584. DOI: 10.1016/j.parco.2019.102584.
5.
BlackfordLSPetitetAPozoR, et al. (2002) An updated set of basic linear algebra subprograms (BLAS). ACM Transactions on Mathematical Software28(2): 135–151. DOI: 10.1145/567806.567807.
6.
ChenPWahibMTakizawaS, et al. (2019) A versatile software systolic execution model for gpu memory-bound kernels. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 19. New York, NY, USA: Association for Computing Machinery, p. 81. DOI: 10.1145/3295500.3356162.
ConstantinescuDNavarroACorberaF, et al. (2020) Efficiency and productivity for decision making on low-power heterogeneous CPU+GPU SoCs. The Journal of Supercomputing77(1): 44–65. DOI: 10.1007/s11227-020-03257-3.
DukhanM (2019) The indirect convolution algorithm. In: Efficient Deep Learning for Compute Vision (ECV) Workshop, p. 10. URLhttps://arxiv.org/abs/1907.02129
11.
EdwardsHCTrottCR (2013) Kokkos: enabling performance portability across manycore architectures. In: 2013 Extreme Scaling Workshop (Xsw 2013), Boulder, CO, USA, 15–16 August 2013, pp. 18–24.
12.
GeorganasEAvanchaSBanerjeeK, et al (2018) Anatomy of high-performance deep learning convolutions on simd architectures. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC ’18, Dallas Texas, 11–16 November 2018. IEEE Press. DOI: 10.1109/SC.2018.00069.
13.
Gómez-HernándezEJMartínezPAPeccerilloB, et al. (2020) Using PHAST to port Caffe library: First experiences and lessons learned. In: 13th International Workshop on Programmability and Architectures for Heterogeneous Multicores, p. 11. URLhttps://arxiv.org/abs/2005.13076.
14.
GoliMNarasimhanKReyesR, et al. (2020) Towards cross-platform performance portability of DNN models using SYCL. In: 2020 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC), GA, USA, 13 November 2020, pp. 25–35. DOI: 10.1109/P3HPC51967.2020.00008.
JiaYShelhamerEDonahueJ, et al. (2014) Caffe: convolutional architecture for fast feature embedding. URLhttps://arxiv.org/abs/1408.5093
20.
JouppiNPYoungCPatilN, et al. (2017) In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, ISCA ’17. New York, NY, USA: Association for Computing Machinery, pp. 1–12. DOI: 10.1145/3079856.3080246.
KingmaDPBaJ (2015) Adam: a method for stochastic optimization. In: BengioYLeCunY (eds), Conference Track Proceedings. 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, p. 15. URLhttp://arxiv.org/abs/1412.6980
MooreGE (2006) Cramming more components onto integrated circuits, Reprinted from Electronics, volume 38, number 8, April 19, 1965, pp.114 ff. IEEE Solid-State Circuits Society Newsletter11(3): 33–35. DOI: 10.1109/N-SSC.2006.4785860.
27.
NeelyJ (2016) Doe Centers of Excellence Performance Portability Meeting. Technical Report LLNL-TR-700962. Livermore, CA (United States): Lawrence Livermore National Lab.(LLNL).
PaszkeAGrossSMassaF, et al. (2019) PyTorch: An Imperative Style, High-Performance Deep Learning Library. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates, Inc., pp. 8024–8035.
30.
PeccerilloBBartoliniS (2019a) PHAST - a portable high-level modern C++ programming library for GPUs and multi-cores. IEEE Transactions on Parallel and Distributed Systems30(1): 174–189. DOI: 10.1109/TPDS.2018.2855182.
31.
PeccerilloBBartoliniS (2019b) Task-DAG support in single-source PHAST library: enabling flexible assignment of tasks to CPUs and GPUs in Heterogeneous Architectures. In: Proceedings of the 10th International Workshop on Programming Models and Applications for Multicores and Manycores, Washington, DC, USA, 17 February 2019. PMAM@PPoPP 2019, pp. 91–100. DOI: 10.1145/3303084.3309496.
32.
PeccerilloBBartoliniS (2020) Flexible task‐DAG management in PHAST library: Data‐parallel tasks and orchestration support for heterogeneous systems. Concurrency and Computation: Practice and Experience34: e5842. DOI: 10.1002/cpe.5842.
33.
PeccerilloBMartínezPA (2021) PHAST AI v1.0. DOI: 10.5281/zenodo.5557540.
PennycookSJSewallJDJacobsenDW, et al. (2021) Navigating performance, portability, and productivity. Computing in Science & Engineering23(5): 28–38. DOI: 10.1109/MCSE.2021.3097276.
36.
ReutherAMichaleasPJonesM, et al. (2020) Survey of machine learning accelerators. In: 2020 IEEE High Performance Extreme Computing Conference (HPEC), Waltham, MA, 22–24 September 2020. DOI: 10.1109/hpec43674.2020.9286149.
37.
RotemNFixJAbdulrasoolS, et al. (2019) Glow: Graph lowering compiler techniques for neural networks. arXiv. URLhttps://arxiv.org/abs/1805.00907.
38.
SzeVChenYYangT, et al. (2017) Efficient processing of deep neural networks: a tutorial and survey. Proceedings of the IEEE105(12): 2295–2329. DOI: 10.1109/JPROC.2017.2761740.
39.
YamadaYMomoseS (2018) Vector engine processor of nec’s brand-new supercomputer sx-aurora tsubasa. Proceedings of A Symposium on High Performance Chips30: 19–21. Hot Chips.
40.
ZhangJFranchettiFLowTM (2018) High performance zero-memory overhead direct convolutions. In: DyJKrauseA (eds), Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, Stockholmsmässan, Stockholm Sweden. PMLR, volume 80, pp. 5776–5785. URLhttp://proceedings.mlr.press/v80/zhang18d.html.
