使用 LaunchDarkly 的 AI 配置在生产环境中进行实时幻觉检测（由 LaunchDarkly 赞助）.pdf

1、Hallucinations matterGPT-4 accuracy dropped from 97.6%to 2.4%in 3 months-with NO code changes(Stanford/UC Berkeley,2023)sama on GPT-4o updates leading to sycophantic behavior(2025)Model behaviors drift&change spontaneouslyPredictable-same input=same outputTestable-unit tests guarantee behaviorVersio

2、ned-git commits,rollbacks,audit trailsDeterministic deployment strategies fail for probabilistic systems.Opaque-cant trace why a decision was madeDynamic-Cant unit test creativity or reasoningEvolving-behavior changes without code changesThe Drift CycleDevs own the outcome but not the fix burnout,re

3、work,and attritionStressful dev experienceNo production control means bugs and outages hit users harderInnovation velocity slows when time is spent on fixing rollbacksReduced velocityIncreased riskWEEK 0Deploy optimizedWEEK 1-3Silent drift beginsWEEK 6Emergency debuggingWEEK 4-5Customers complainWEE

4、K 7Rebuild&redeployEvery 6-8 weeks,the cycle repeatsDeployment gets code to production,but teams have no control at runtime.The model layerProvider-controlled territory.Same endpoint,same version,different behavior.The user layerQueries evolve organically.Month 1:Do I have dental?Month 6:Pasting 2,0

5、00-word medical histories.The knowledge layerAI failures are traceable:if youre watchingYour domain expertise is,constantly decaying.Policies update monthly,regulations change quarterly.By the time you notice somethings wrong,customers have already been affected.Add per-node accountability and treat

6、 observability as a lagging indicator.The patterns are detectable but only if youre watching all three layers.Inject evaluators to supervise agents in-step so you catch issues as they happen,not after customers complain.MaintenanceMaintenanceThe cost of manual AI managementNew Features/R&DNew Featur