追踪不可追踪之物：LLM 和代理的全栈可观测性（由 groundcover 赞助）.pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Tracing the Untraceable:Full-Stack Observability for LLMs and AgentsMonitoring modern AI systems in productionShahar AzulayCo-Founder&CEO at groundcoverAIM 212-SWhy LLM Observability MattersMulti-Turn AgentsRetrieval PipelinesTool-A

2、ugmented FlowsYou have to observeToken UsageIf you want to have$left01Response LatencyYour own APIs depend on it02Error RatesCan impact user experience03LLM ResponsesDetect hallucinations and inconsistency04The speed in which you stack changesNew tool every month?The ChallengesThe data is sensitive

3、and privateWho can upload what to an LLM?groundcoverflips the model.More frictionless,secured and granularBYOC:All inside your cloud.Zero instrumentation with eBPFHighly granular data visibilityZero Instrumentation,Zero FrictionThe eBPF AdvantageNo Code ChangesDeploy without modifying applications o

4、r adding SDKs to your stack.Kernel-Level VisibilityCapture full API request and response payloads at the network layer.Real-Time InsightsMonitor LLM calls,token usage,and reasoning paths as they happen,correlated to all logs,metrics and other traces.Your Cloud,Your Control The perfect balance betwee

5、n operational ease and security.Customer cloudLogsTracesEventsCustom MetricsPrivate Endpointgroundcover managed accountK8s ClusterManaged Data Planegroundcover cloudControl PlaneUIInfrastructure managementObservability dataAuthenticationProtect LLM pipelines fromdata exposureAll data captured by gro

6、undcover stays within your environment.Theres no third-party storage,no outbound traffic,and no riskImprove performance and manage spendinstant access to token usage,latency,throughput,and error rates across every LLM interaction.Monitor LLM responses with ful