Deep neural networks (DNNs) have demonstrated effectiveness in many domains including object recognition, speech recognition, natural language processing, and health care. Typically, the computations involved in DNN training and inferencing are time consuming and require efficient implementations. Existing frameworks such as TensorFlow, Theano, Torch, Cognitive Tool Kit (CNTK), and Caffe enable Graphics Processing Unit (GPUs) as the status quo devices for DNN execution, leaving Central Processing Unit (CPUs) behind. Moreover, existing frameworks forgo or limit cross layer optimization opportunities that have the potential to improve performance by significantly reducing data movement through the memory hierarchy. In this article, we describe an alternative approach called SWIRL, a compiler that provides high-performance CPU implementations for DNNs. SWIRL is built on top of the existing domain-specific language (DSL) for DNNs called LATTE. SWIRL separates DNN specification and its schedule using predefined transformation recipes for tensors and layers commonly found in DNN layers. These recipes synergize with DSL constructs to generate high-quality fused, vectorized, and parallelized code for CPUs. On an Intel Xeon Platinum 8180M CPU, SWIRL achieves performance comparable with Tensorflow integrated with MKL-DNN; on average 1.00× of Tensorflow inference and 0.99× of Tensorflow training. It also outperforms the original LATTE compiler on average by 1.22× and 1.30× on inference and training, respectively.
AbadiMAgarwalABarhamP, et al. (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Available at:http://tensorflow.org/ (accessed 6 January 2019).
AlkarSThomasJShanbhagA, et al. (2017) Weld: a common runtime for high performance data analytics. In: 8th Biennial Conference on Innovative Data Systems Research (CIDR), CIDR ’17, CA, USA, 8–11 January 2017.
4.
BergstraJBreuleuxOBastienF, et al. (2010) Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), Austin, TX, 10 June, Oral Presentation.
5.
CatanzaroBKamilSLeeY, et al. (2009) SEJITS: Getting productivity and performance with selective embedded JIT specialization. Programming Models for Emerging Architectures1(1): 1–9.
6.
ChafiHSujeethAKBrownKJ, et al. (2011) A domain-specific approach to heterogeneous parallelism. ACM SIGPLAN Notices46(8): 35–46.
7.
ChenTMoreauTJiangZ, et al. (2018) Tvm: end-to-end optimization stack for deep learning. arXiv preprint arXiv:1802.04799.
8.
ChetlurSWoolleyCVandermerschP, et al. (2014) cuDNN: efficient primitives for deep learning. CoRRabs/1410.0759. Available at:http://arxiv.org/abs/1410.0759.
CollobertRKavukcuogluKFarabetC (2011) Torch7: a MATLAB-like environment for machine learning. In: BigLearn, NIPS Workshop, EPFL-CONF-192376, 2011.
11.
DonadioSBrodmanJRoederT, et al. (2005) A language for the compact representation of multiple program versions. In: Workshop on Languages and Compilers for Parallel Computing (LCPC), Hawthorne, NY, 20–22 October 2005, pp. 136–151. Berlin, Heidelberg: Springer.
HallMWChameJChenC, et al. (2009) Loop transformation recipes for code generation and auto-tuning. In: Proceedings of the 22nd International Workshop on Languages and Compilers for Parallel Computing, Newark, DE, 08–10 October 2009, pp. 50–64. Berlin, Heidelberg: Springer.
16.
HartonoANorrisBSadayappanP (2009) Annotation-based empirical performance tuning using Orio. In: IPDPS, Washington, DC, USA, 23–29 May 2009.
17.
HezavehYDLevasseurLPMarshallPJ (2017) Fast automated analysis of strong gravitational lenses with convolutional neural networks. Nature548: 555–557.
JiaYShelhamerEDonahueJ, et al. (2014) Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093.
20.
JinLWangZGuR, et al. (2014) Training large scale deep neural networks on the intel xeon phi many-core coprocessor. In 2014 IEEE International Parallel & Distributed Processing Symposium Workshops, 2014, pp. 1622–1630. IEEE.
21.
KrizhevskyASutskeverIHintonGE (2012) Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems1: 1097–1105.
22.
KurthTZhangJSatishN, et al. (2017) Deep learning at 15pf: supervised and semi-supervised classification for scientific data. In: Proceedings of the international conference for high performance computing, networking, storage and analysis, SC ’17, New York, NY, USA, pp. 7:1–7:11. ACM. ISBN 978-1-4503-5114-0, DOI:10.1145/3126908.3126916. Available at:http://doi.acm.org/10.1145/3126908.3126916.
LavinAGrayS (2015) Fast algorithms for convolutional neural networks. CoRRabs/1509.09308. Available at:http://arxiv.org/abs/1509.09308.
25.
LiuYRacahEPrabhat, et al. (2016) Application of deep convolutional neural networks for detecting extreme weather in climate datasets. CoRRabs/1605.01156. Available at:http://arxiv.org/abs/1605.01156.
26.
MathieuMHenaffMLeCunY (2013) Fast training of convolutional networks through ffts. CoRRabs/1312.5851. Available at:http://arxiv.org/abs/1312.5851.
MilovaESveshnikovaSGankevichI (2016) Speedup of deep neural network learning on the mic-architecture. In: 2016 international conference on high performance computing simulation (HPCS). pp. 989–992. DOI: 10.1109/HPCSim.2016.7568443.
Ragan-KelleyJBarnesCAdamsA, et al. (2013) Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. ACM SIGPLAN Notices48(6): 519–530.
32.
RussakovskyODengJSuHKrauseJ, et al. (2015) ImageNet large scale visual recognition challenge. International Journal of Computer Vision (IJCV)115(3): 211–252.
33.
SermanetPEigenDZhangX, et al. (2013) OverFeat: integrated recognition, localization and detection using convolutional networks. CoRRabs/1312.6229. Available at:http://arxiv.org/abs/1312.6229.
34.
Shashank KairaCYangXDe AndradeV, et al. (2018) Automated correlative segmentation of large transmission x-ray microscopy (txm) tomograms using deep learning. Materials Characterization142: 203–210.
35.
SilverDHuangAMaddisonCJ, et al. (2016) Mastering the game of Go with deep neural networks and tree search. Nature529(7587): 484–489.
36.
SimonyanKZissermanA (2014) Very deep convolutional networks for large-scale image recognition. CoRRabs/1409.1556.
37.
SzegedyCLiuWJiaY, et al. (2014) Going deeper with convolutions. CoRR abs/1409.4842. Available at:http://arxiv.org/abs/1409.4842.
38.
TeixeiraTSFXAncourtCPaduaD, et al. (2019) Locus: a system and a language for program optimization. In: Proceedings of the 2019 IEEE/ACM international symposium on code generation and optimization, CGO 2019, Piscataway, NJ, USA, pp. 217–228. Available at:http://dl.acm.org/citation.cfm?id=3314872.3314898.
39.
TruongLBarikRTotoniE, et al. (2016) Latte: a language, compiler, and runtime for elegant and efficient deep neural networks. In: Proceedings of the 37th ACM SIGPLAN conference on programming language design and implementation, PLDI ‘16, New York, NY, USA, pp. 209–223. ACM. ISBN 978-1-4503-4261-2, DOI:10.1145/2908080.2908105. Available at:http://doi.acm.org/10.1145/2908080.2908105.
ZlateskiALeeKSeungHS (2015) ZNN - a fast and scalable algorithm for training 3d convolutional networks on multi-core and many-core shared memory machines. CoRRabs/1510.06706. Available at:http://arxiv.org/abs/1510.06706.
