使用 DBRX、SPARK 和 LANCEDB 构建生产规模、完全私有的 OSS RAG 流水线.pdf

1、CEO/Cofounder,LanceDBChang SheJune 12,2024Data&AI SummitProduction-scale,Private RAG Pipelines with LanceDBChang SheData tools(2 decades)PandasBig Data/RecSysLanceDB:The Database for Multimodal AICEO/Co-founder LanceDBchanghiskhanAgendaWhy RAG?Productionizing RAG in the EnterpriseDBRX+LanceDB+SparkT

2、ext-only-MultimodalRAGExtend model knowledgeReduce hallucinationWorks in conjunction with fine-tuningR Retrieval A Augmented G GenerationRAGChatSearchWhats next for RAG?Enterprise decision supportR Retrieval A Augmented G GenerationProductionScaleRetrieval qualityData privacyVendor lock-inPitfalls f

3、or RAG in ProductionScaleNum tablesNum of vectorsQueries per second(QPS)Update frequencyScale sneaks up on you in productionUnnecessary copy/slow/$Unnecessary copy/slow/$QualityDifferent retrieval modesHybrid+rerankingFine-tuning embeddingsComposability+customizabilityMore than just vector searchPri

4、vacyModels will commoditizeData is enterprise valueData should stay on-premiseRAG context is valuableFinancials leaving premises!Financials leaving premises!Searches leaving premisesUnnecessary copiesSearches leaving premisesUnnecessary copiesLock-inStorage should be openWalled-garden-data lakeRAG c

5、omponents should be easily swappableLong-term flexibility is importantFinancials leaving premises!Financials leaving premises!Searches leaving premisesComplex+Slow+$Searches leaving premisesComplex+Slow+$Everything stays onEverything stays on-prempremspark.readspark.read.format(“lance”).format(“lanc

6、e”)df.writedf.write.save(“table.lance”).save(“table.lance”)parallel udfparallel udfEffortless scaleBillions of vectors 10K+QPSHigh recall+low latency10 x more efficient than alternativesEnterprise readyBYOCSOC2Data Lake integrationMultimodalData:image,audio,vi