CSET：2024AI安全的关键概念：机器学习中可靠的不确定性量化方法分析报告（英文版）（13页）.pdf

1、Issue BriefJune 2024Key Concepts in AI SafetyReliable Uncertainty Quantification in Machine LearningAuthorsTim G.J.RudnerHelen Toner Center for Security and Emerging Technology|1 This paper is the fifth installment in a series on“AI safety,”an area of machine learning research that aims to identify

2、causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably.Other papers in the series describe three categories of AI safety issuesproblems of robustness,assurance,and specification.This paper introduces the idea of uncertainty quanti

3、fication,i.e.,training machine learning systems that“know what they dont know.”Introduction The last decade of progress in machine learning research has given rise to systems that are surprisingly capable but also notoriously unreliable.The chatbot ChatGPT,developed by OpenAI,provides a good illustr

4、ation of this tension.Users interacting with the system after its release in November 2022 quickly found that while it could adeptly find bugs in programming code and author Seinfeld scenes,it could also be confounded by simple tasks.For example,one dialogue showed the bot claiming that the fastest

5、marine mammal was the peregrine falcon,then changing its mind to the sailfish,then back to the falcondespite the obvious fact that neither of these choices is a mammal.This kind of uneven performance is characteristic of deep learning systemsthe type of AI systems that have seen most progress in rec

6、ent yearsand presents a significant challenge to their deployment in real-world contexts.An intuitive way to handle this problem is to build machine learning systems that“know what they dont know”that is,systems that can recognize and account for situations where they are more likely to make mistake