已知未知：LLM 中不确定性的贝叶斯多路径框架.pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.I N V 5 0 6Known Unknowns:Bayesian Multi-Path Framework for Uncertainty in LLMsJae Oh WooSenior Applied ScientistBaishali ChaudhuryApplied Scientist IIAWS GenAI Inno

2、vation Center 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Agenda Introduction Background Our Approach Experimental Results Key Takeaways 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Introd

3、uction 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.IntroductionEnterprise LLM Uncertainty ChallengesOverconfident Wrong AnswersWasted Compute on Easy QuestionsUnreliable Mission-Critical OutputsInconsistent Expert ConsultationFramework ObjectivesBayesian Multi-Path Uncertainty

4、 Quantification using Bradley-Terry+PageRank models Automated Response Triggering based on uncertainty thresholds(RAG/Tools/Foundation)OVERCONFIDENT 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Background 2025,A

5、mazon Web Services,Inc.or its affiliates.All rights reserved.LLM UncertaintySpaceKnownknownClear confidence 2+2=4Known-unknownAdmits uncertainty The integral has multiple valid formsUnknown-knownForgets or overlooksModel fails to recall known formulaUnknown-unknownHallucinates confidently,Fabricated

6、 citation,unseen theorem 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.QuantifyingUncertaintyConsistency-based”If I try again,will I say the same thing?”Self-evaluation“Do I believe my own answer is correct?”Logit based“How confident are my tokens mathematically?”Graph/Tensor UQ