陈争胜-在边缘DC部署大模型：实践和加速 陈争胜.pdf

1、Deploying Large Model in edge DC:practice and accelerationJack ChenYsemi Computing遇 贤 微 电 子致力于研发高性能计算和数据中心的CPUYSEMI Computing Introduction7/17/20232 YSEMI Computing founded in 2020YSEMI Computing founded in 2020 100+Employees across Shenzhen/Shanghai/Xian100+Employees across Shenzhen/Shanghai/Xian P

2、roviding silicon,platform and system for cloud computing Providing silicon,platform and system for cloud computing data centersdata centers First generation product is 160Cores Armv9 Datacenter First generation product is 160Cores Armv9 Datacenter CPU,3.2GHz and estimatedCPU,3.2GHz and estimated 620

3、+SPECint2017620+SPECint2017 Members of LF Members of LF EdageEdage,OpenEulerOpenEuler,ODCC etc.,ODCC etc.Why Large Language Mode on edge is important?7/17/20233Data PrivacyPervasive AILatencyChatGPT-like LLM user cases in edge DC7/17/20234code explanationTime Complexity Calculationprogram code trans

4、lationFix code bugsparagraph productionstory creationsummary descriptionText CategorizationPerson switchCategoryFAQreview generationText Sentiment AnalysisAdvanced Sentiment Scoringinterview questions and answersText to Emojilanguage chatbotEngineeringCommunicationContent generationMarketingChalleng

5、es in LLM deployment on edge7/17/20235ChallengesDemands are hugeThe recent rise of ChatGPT-like Large Language Model(LLM)has promoted the vigorous development of AI on the application side,which also puts forward unprecedented demands on edge devices.Computing Force is limitedGPT-3 has 175 billion p

6、arameters while GPT-4 has more parameters.Big model size and large training computing force will limit the usage in edge and terminals.smaller model size moderate computing force lower but usable accuracyIP PhoneTablet PCPC ClientMobile ClientThin clientTV screenCollaborationSoftwareComputing force