使用 DatabricksIQ 将数据智能引入 Delta Lake.pdf

1、This information is provided to outline Databricks general product direction and is for informational purposes only.Customers who purchase Databricks services should make their purchase decisions relying solely upon services,features,and functions that are currently available.Unreleased features or

2、functionality described in forward-looking statements are subject to change at Databricks discretion and may not be delivered as planned or at allProduct safe harbor statement2024 Databricks Inc.All rights reservedINTRODUCING DATA INTRODUCING DATA INTELLIGENCE TO DELTA INTELLIGENCE TO DELTA LAKE WIT

3、H LAKE WITH DatabricksIQDatabricksIQSirui Sun-Sr.Staff Product Manager Terry Kim-Sr.Staff Software EngineerSirui SunSr.Staff Product ManagerProduct Lead,Delta Lake Previously Google,MicrosoftBased in SeattleTalk to me aboutAll things DeltaAll things storageTerry KimSr.Staff Software EngineerTechnica

4、l Lead,Delta Lake Previously Microsoft,YahooBased in SeattleTalk to me aboutAll things DeltaAll things storageHow to make a Delta table go fast-decision:what columns do I pick?-decision:is the cardinality correct?CREATECREATE TABLETABLE tbl1 PARTITIONPARTITION BYBY date ASAS How to make a Delta tabl

5、e go fast-decision:what columns do I pick?-decision:is the cardinality correct?CREATECREATE TABLETABLE tbl1 PARTITIONPARTITION BYBY date ASAS-decision:what is the best target file size?ALTER TABLEALTER TABLE tbl1 SET TBLPROPERTIESSET TBLPROPERTIES(delta.targetFileSize=104857600)-decision:do I enable

6、 auto-compact and optimize writes?ALTERALTER TABLETABLE tbl1 SETSET TBLPROPERTIESTBLPROPERTIES(“delta.autoOptimize.autoCompact”=“true”,“delta.autoOptimize.optimizeWrite”=“true”);#1 File size optimizationHow to make a Delta table go fast-decision:what columns do I pick?-decision:is the cardinality co