用于高性能计算和人工智能的近 HBM 处理.pdf

1、JongryoolKimProcessing Near Memory for HPC and AI with HBM:AIA(Accelerate Indirect memory Access)Memory System Research,SK hynixProcessing Near Memory for HPC and AI with HBM:AIA(Accelerate Indirect memory Access)Memory System Research,SK hynixJongryoolKimThe Problem:Indirect(Irregular)Memory Access

2、HPC Physics SimulationsSparse Linear Algebra(SpMV,SpMM,SpGEMM,)Graph applications(BFS,PageRank,SSSP,.)AI(LLMs,GNNs.)The Performance Penalty:Why Indirect Memory Access HurtsThe cycle of InefficiencyBig Cache Miss RatioMemory Bandwidth is WastedExecution Units StallThe Challenge:Unlocking Performance

3、in AI and HPCHPC ApplicationUME is a C+20 proxy based on a large computational physics applicationChallenge of Sparsity for HPC applicationSignificant(52%)time is spent memory operations due to unstructured mesh operationsApproximates memory layout and indexing of an unstructured meshExtracts an imp

4、ortant computational kernel7 computation loops with indirect memory accesses(1 or 2 levels of indirections)Want to move only the data we need for computationInstruction Count PercentageLoad 6,775,030,84918%Branching6,063,697,70716%Integer Add 5,334,155,68214%Array Indexing4,855,537,53213%Conditional

5、 3,299,248,2749%Store 2,599,966,4277%Type cast 1,959,938,0435%Sign extension 1,541,094,4044%Point to CornersConnectivity ArrayCorner to ZoneConnectivity ArrayIndirect(Irregular)Memory AccessCPUHBMAccelerated Indirect memory Access(AIA)-HBMIrregular memory access in parallel using multiple processing

6、 units and DRAM channels efficiently Only the resulting data is delivered reduces data movement between processor and memoryAIA-HBM(with Los Alamos National Lab.)Indirect Memory ReadIndirect Memory Read with AIAAIA-HBM PrototypePerformance Benefic for UME w/AIAUME HPC Application throughput(million