GitHub Copilot 如何每天处理 4 亿次代码完成请求.pdf

1、QCon San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024How GitHub Copilot serves 400 million code completion requests a dayQCon San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024How

2、GitHub Copilot serves 400 million code completion requests a San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024GitHubQCon San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024Code compl

3、etion,chat,refactoring,in your IDEAvailable as an extension in many popular IDEs;VSCode,Visual Studio,IntelliJ,Neovim,Xcode GitHub CopilotDemoQCon San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024The goal is interactive code completion in y

4、our IDEQCon San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024Minimise latency before and between requests Amortise connection setup costs Avoid network latency where possible Stream responses immediatelyBuilding a cloud hosted autocompletio

5、n serviceTCP 3-way handshake TLS handshake is 57 legs Some of these steps can be pipelined Per leg latency highly correlated with distance Connection setup is expensive QCon San Francisco/How GitHub Copilot serves 400 million code completion requests a daydavecheney/November 18,2024Copilot alpha hos

6、ted directly on OpenAI.Users would source their own OpenAI token and plug it into the extension.LLM providers dont want to be in the business of end user management;would prefer a per service API key.How can we make this API key available to all Copilot IDE extensions?A little bit of historyQCon San