LLM部署导航：技巧、窍门和方法.pdf

1、ConfidentialNavigating LLM Deployment:Tips,Tricks,and Techniques 2.0Meryem Arik,Co-founder/CEO TitanMLConfidentialAKA:How to deploy LLMs if you dont work at(If you are from one of these orgs,you are still welcome!)ConfidentialWhat you will get out of this sessionLearning best practices for self-host

2、ed AI deployments in corporate and enterprise environments1Understanding the difference between your deployments and deployments at AI Labs 23Evaluating when self-hosting is right for youConfidentialBut first Hi!Meryem Arik,CEO TitanML,Forbes 30U30About TitanML:Building infrastructure for efficient,

3、scalable LLM deploymentSpecializing in on-premise&VPC AI deploymentsOur Expertise:Deep experience in self-hosting inference infrastructure!Building AI apps/infra within your org?Lets chat!ConfidentialWhat you will get out of this sessionUnderstanding the difference between your deployments and deplo

4、yments at AI Labs Learning best practices for self-hosted AI deployments in corporate and enterprise environments123Evaluating when self-hosting is right for youConfidentialFirstly What is self-hosting?API HostedSelf-HostedEg,OpenAI,Anthropic,etcTotal control over data and modelData&QueriesResponses

5、Application MicroserviceData&QueriesResponsesApplication MicroserviceYour environment(VPC/On-Prem)3rd party compute environment(eg OpenAI)ConfidentialWhy would you ever want to self-host?7Decreased CostImproved PerformancePrivacy&SecurityAnd its great if you find this kind of stuff cool!Confidential

6、Why would you ever want to self-host?91.Deploy at scale2.Use smaller specialized models when performance matches1.Running embedding/reranking AI workloads2.Operating in a specialized domain3.Have clearly defined task requirements1.Legal restrictions on third-party data sharing2.Region-specific deplo