利用现代架构最大化CPU上的深度学习性能.pdf

1、Accelerating Deep Learning Workloads on Modern CPUsBibek BhattaraiIntelAdvanced matrix Extensions(AMX)q Why should I care?q What does it solve?q How does it work?q How do I leverage it?What DL workloads usually run in CPUs?q Smaller Transformer models:BERT,RoBERTa,DistilBERT,MobileBERT,TinyBERT,etc.

2、q Lightweight CNNs:Lightweight CNN architectures such as MobileNet and SqueezeNet,EfficientNet q Recurrent Neural Networks(RNNs):RNN-based models like LSTM or GRU q Recommendation modelsq Tree based models:XGBoost,Random Forest,etc.Mostly Inference!LLMs/SLMs on CPUUses Intel Advanced Matrix Extensio

3、ns(AMX)!GEMMs dominate the Runtimeq Transformers(e.g.BERT,GPT models):70%of runtime 1q Convolutional Neural Network(CNNs):50-60%of total runtime 2q Recurrent Neural Networks(RNNs):40-60%of runtime 11 https:/bpb-us- Why GEMM is at the heart of deep learning Pete Wardens blogNave MatMulM=N=K=1024Runti

4、me=730msrowsxcolsA is MxKB is KxNC is M x Nfor(int m=0;m M;m+)for(int n=0;n N;n+)float sum=0.0f;for(int k=0;k K;k+)sum+=Am*M+k*Bk*K+n;Cm*M+n=sum;Data are stored in row-major format!Reorder LoopsrowsxcolsA is MxKB is KxNC is M x NM=N=K=1024Runtime=130msfor(int m=0;m M;m+)for(int k=0;k K;k+)for(int n=

5、0;n N;n+)Cm*M+n+=Am*M+k*Bk*K+n;Tiled MatmulA *B =CrowsxcolsA is MxKB is KxNC is M x NSize tiles to fit on local CacheM=N=K=1024T=128Runtime=89msfor(int m=0;m M;m+=T)for(int n=0;n N;n+=T)for(int k=0;k K;k+=T)const int minMt=std:min(m+T,M);const int minNt=std:min(n+T,N);const int minKt=std:min(k+T,K);

6、for(int mt=m;mt minMt;+mt)for(int kt=k;kt minKt;+kt)for(int nt=n;nt minNt;+nt)Cmt*M+nt+=Amt*M+kt*Bkt*K+nt;Data locality is very Important!q Implementing efficient matmul is much more complicatedq Tiling is not cache size-agnostic q Multi-tiered caches complicates things q Vector processing units add