采用 CXL3.1 分层内存和 MRDIMM 解决方案的下一代 AI_HPC 服务器性能.pdf

1、David ZhuoTechnology Solution ArchitectSr.Director of Technical MarketingNextNext-Gen Gen AI App Server Performance AI App Server Performance with with CXL3.1 Tier Memory CXL3.1 Tier Memory and and MRDIMMMRDIMMGen2Gen2 solutionsolutionNextNext-Gen Gen AI App AI App Server Performance Server Performa

2、nce with with CXL3.1 Tier Memory CXL3.1 Tier Memory and MRDIMMand MRDIMMGen2Gen2 solutionsolutionOCP SPECIAL FOCUS:ARTIFICIAL INTELLIGENCE(AI)David ZhuoTechnology Solution ArchitectSr.Director of Technical MarketingDatacenter Use Case by workload memory demand applicationsstorage demand applications

3、 Memory-Intensive ApplicationsDLRM serverLLM and HPC serversIMDB serverIMDB serverBackgroundFig-1 Memory-Intensive ApplicationsDLRM serverLLM and HPC serversIMDB serverIMDB serverBackgroundFig-1.AAI App Server Memory Intensive App Server Background and Motivation1.The HBM availability,supply stabili

4、ty,and high TCOThose factors are slowing down the AI server development and grow.In this new era,we should explore how to use fewer HBM.Instead,increase host memory capacity and performance.This will keep scalability,supply availability and lower TCO.*Host Memory(CPU-manage memory CPU near memory,fa

5、r memory,pooled memory)JBOM The challenges to resolve the memory capacity and performance of Next-Gen Memory-Intensive App ServerBackground and Motivation2.Memory subsystem scalability and lower the TCO requirementThe diversity of Memory-Intensive App Server architectures depends on their use cases

6、and workloads(Fig-1).How to avoid the under-provisioning,over-provisioning of the memory subsystem and stranded memory are the challenges to Next-Gen Memory-Intensive App Server architectures today.Fig-1.AThe challenges to resolve the memory capacity and performance of Next-Gen Memory-Intensive App