使用 Delta Lake UniForm 为开放数据湖提供可移植性.pdf

1、DELIVERINGDELIVERINGPORTABILITYPORTABILITYTO OPEN DATA LAKESTO OPEN DATA LAKESWITH WITH DELTA LAKE UNIFORMDELTA LAKE UNIFORMTomohiro TanakaTomohiro TanakaSenior Cloud Support Engineer,Amazon Web ServicesSenior Cloud Support Engineer,Amazon Web Services2024 Jun.112024 Jun.11 2024,Amazon Web Services,

2、Inc.or its affiliates.All rights reserved.Tomohiro TanakaTomohiro Tanaka2Senior Cloud Support Engineer,AWS Support EngineeringWorking for customers to build data lake with Open Table FormatsContributions to Apache IcebergCo-author of Serverless ETL and Analytics with AWS Gluetomtonguein/ttomtan 2024

3、,Amazon Web Services,Inc.or its affiliates.All rights reserved.3Rise of Open Table FormatDelta Lake Universal Format(UniForm)UniForm Under-the-hoodDemo:Extend Delta Lake capabilities to other OTF clientsAgendaAgenda 2024,Amazon Web Services,Inc.or its affiliates.All rights reserved.Rise of Open Tabl

4、e FormatRise of Open Table Format 2024,Amazon Web Services,Inc.or its affiliates.All rights reserved.What are Open Table Formats(OTFs)?What are Open Table Formats(OTFs)?An abstraction layer providing enhancement and new capabilities for traditional data lakesACID transactionNew features such as Time

5、-travel,UPSERT,Compaction etc.5Linux FoundationLinux FoundationDelta LakeDelta LakeApache IcebergApache IcebergApache HudiApache Hudi 2024,Amazon Web Services,Inc.or its affiliates.All rights reserved.Challenges of Open Table FormatChallenges of Open Table FormatAs scaling your OTF analytic environm

6、ents along with business growth,Difficulty in managing multiple OTF environments across organizations and teamsLimited access to some OTFs depending on your client or service specification6 2024,Amazon Web Services,Inc.or its affiliates.All rights reserved.Delta Lake Universal Format Delta Lake Univ