1-陈齐翔-基于 Ray 的分布式 AI Agent 框架.pdf

1、演讲嘉宾：陈齐翔-蚂蚁集团Who are weAnt GroupAlipayMission:“Make it easy to do business anywhere.”Ray Team2nd largest team contributing 26%+to Ray Core code)over 1.5 million CPU cores onlineOperating Ray China CommunityHistory of Ray in Ant1Background2Motivation3Design&Impl.1A typical AI AgentLLM-based AgentSour

2、ce:“A Survey on Large Language Model based Autonomous Agents”(https:/arxiv.org/abs/2308.11432)Autonomous agent frameworkTypically requires:Profile:Personality,Misson Memory:Knowledge,Experience Planning:Split intricate task to simpler sub-tasks Action:Function calling A naive RAG agent 1.Observe a T

3、ask2.Think what to do3.Utilize Tools to take actions and get the result4.Put into Memory5.Repeat 24 until task is completedReference:“ReAct:Synergizing Reasoning and Acting in Language Models”(https:/arxiv.org/abs/2210.03629)NetworkComputationDisk I/OMemoryToolsReActactionfeedbackAgentToolsVectorDBO

4、bserveThinkActKnowledgeQueryRelevant DocsQueryRealtime NewsLearn from this doc:xxxThought:Should parse and load to DBAction:index_docDeploy the agenthttps:/naive-Agent crafting platform1.No idea why my App/Pod is dead2.No metrics to spying workload3.No traffic control for heavy workload4.Low GPU uti

5、lization on hybrid workload5.Bind to the platform provided agent library6.Need to be productionized!User Complains2 MotivationAgents are very creative1.Constantly emerging innovative ideas2.Quick validation with the lowest cost3.*Easily convert proved ideas and deployed to prod.env.4.GPU+CPU+Service

6、 Calling in one user task!5.Diverse software stack across scenariosLangchainRayProsRich&friendly librariesComplete toolchain from UI to algorithmConsHuge works to do to productionizeProsOne-stop for AI workloadEasily scale from local to distributedHeterogeneous resource schedulingNot binding computa