Lakehouse Federation: Architecture & Production Tests

What Is Lakehouse Federation?什么是湖仓联邦？

This focused article is part of the federated queries and data virtualization guide; use the pillar guide to compare related concepts, methods, and implementation decisions across the full topic.

本文是联邦查询与数据虚拟化指南内容集群中的专题文章；如需比较完整主题下的相关概念、方法与实施决策，请返回基石指南。

Lakehouse federation is a query-access pattern that lets lakehouse users discover and read selected data in external systems through governed connections and catalogs, without first copying every source table into lakehouse storage. The lakehouse query engine may push filters or aggregations to a remote system, receive bounded results, and combine them with local lakehouse data.

湖仓联邦是一种查询访问模式：湖仓用户通过受治理的连接和目录发现并读取外部系统中的选定数据，而不必先把每张源表复制到湖仓存储。湖仓查询引擎可以把过滤或聚合下推到远程系统，接收有边界的结果，再与本地湖仓数据组合。

The exact phrase is strongly associated with Databricks, whose documentation names its query-federation capability “Lakehouse Federation.” In that implementation, a Unity Catalog connection records access to an external system and a foreign catalog mirrors external metadata for read-oriented queries. Other vendors use terms such as federated catalogs, external tables, query federation or data federation for related patterns. Their connectors, security model, write behavior, caching and limits are not interchangeable.

该英文短语与Databricks关系密切，其文档把查询联邦能力命名为“Lakehouse Federation”。在该实现中，Unity Catalog连接记录对外部系统的访问，Foreign Catalog则镜像外部元数据以支持面向读取的查询。其他厂商会使用联邦目录、外部表、查询联邦或数据联邦等术语描述相关模式，但其连接器、安全模型、写入行为、缓存和限制并不相同。

Primary purpose: answer an approved cross-system question quickly while data remains under its source owner. Federation is an access choice, not proof that the data is integrated, semantically consistent, cheap to query or suitable for every production workload.

主要目的：在数据仍由源系统所有者管理时，快速回答一个获批的跨系统问题。联邦是一种访问选择，并不证明数据已经集成、语义一致、查询成本低廉或适合所有生产负载。

How Lakehouse Federation Architecture Works湖仓联邦架构如何工作

A governed connection represents the source.受治理连接代表外部来源。
It records the source type, endpoint, authentication method and accountable owner. Credentials should resolve from an approved secret system rather than being embedded in notebooks or SQL.
连接记录来源类型、端点、认证方式与问责所有者。凭据应从获批的Secret系统解析，而不是嵌入Notebook或SQL。
A foreign or federated catalog exposes metadata.Foreign或Federated Catalog公开元数据。
External databases, schemas and tables appear in a lakehouse namespace. The catalog is a governed description and access surface; it does not necessarily copy source rows.
外部数据库、Schema和表出现在湖仓命名空间中。该目录是受治理的描述与访问界面，并不必然复制源数据行。
The planner divides local and remote work.规划器划分本地与远程工作。
Connector capabilities, SQL dialect, types and statistics influence which filters, projections, aggregates or joins can execute remotely and which remain in lakehouse compute.
连接器能力、SQL方言、数据类型与统计信息会影响哪些过滤、投影、聚合或连接可在远程执行，哪些留在湖仓计算侧。
The source enforces its part of the query.源系统执行其查询部分。
Remote compute reads only what its credentials and local policies allow. Good pushdown reduces transfer; unsafe or unsupported pushdown must fail clearly rather than change meaning.
远程计算只读取其凭据与本地策略允许的内容。良好下推可减少传输；不安全或不受支持的下推必须明确失败，而不能改变语义。
The lakehouse combines bounded results.湖仓组合有边界的结果。
Returned rows may be joined with lakehouse tables, filtered again, presented to consumers or used to assess whether a recurring workload should be materialized.
返回行可以与湖仓表连接、再次过滤、提供给消费者，或用于判断持续负载是否应被物化。
Evidence follows the entire path.证据覆盖整条路径。
A production design links user identity, catalog object, policy decision, query text or hash, remote request, source load, result version, errors and reviewer evidence.
生产设计应把用户身份、目录对象、策略决策、查询文本或哈希、远程请求、源负载、结果版本、错误与复核证据连接起来。

The current Databricks query-federation documentation describes connections and foreign catalogs as securable Unity Catalog objects and documents read-oriented behavior and connector-specific limits. AWS documents a comparable lakehouse-centered pattern using federated connections, federated catalogs, Athena and Lake Formation. These similarities support a generic architecture model, not a claim that the products behave identically.

Databricks当前查询联邦文档把连接与Foreign Catalog描述为Unity Catalog中的可安全控制对象，并记录面向读取的行为与连接器专属限制。AWS则记录了使用联邦连接、联邦目录、Athena和Lake Formation的类似湖仓中心模式。这些相似点支持通用架构模型，但不代表产品行为完全相同。

Lakehouse Federation vs Ingestion, CDC and Data Sharing湖仓联邦与摄取、CDC和数据共享的区别

Choose by workload behavior, not by the desire to avoid all copies依据负载行为选型，而不是追求绝对零复制
Pattern模式	Data path数据路径	Strong fit适用情形	Primary trade-off主要取舍
Lakehouse/query federation湖仓/查询联邦	Runs selected work on the remote system and returns results at query time在查询时于远程系统执行选定工作并返回结果	Ad hoc, POC, current lookup, incremental migration即席、POC、当前查找、渐进迁移	Inherits network, source load, connector and remote availability继承网络、源负载、连接器与远程可用性
Batch or query-based ingestion批量或查询式摄取	Copies scheduled selections into governed lakehouse storage按计划把选定数据复制到受治理湖仓存储	Repeatable history, transformation, workload isolation可重复历史、转换、负载隔离	Freshness lag, duplicate storage and pipeline ownership新鲜度延迟、重复存储与管道所有权
CDC or managed replicationCDC或托管复制	Continuously applies captured source changes to a target持续把捕获的来源变化应用到目标	Low-lag repeated analytics and source isolation低延迟重复分析与源系统隔离	Change semantics, recovery, deletes and schema evolution变化语义、恢复、删除与Schema演进
Data sharing数据共享	A provider publishes selected datasets through a sharing contract or protocol提供方通过共享契约或协议发布选定数据集	Controlled distribution across organizations or platforms跨组织或平台的受控分发	Provider-defined assets, protocol behavior and recipient lifecycle提供方定义的资产、协议行为与接收方生命周期
Materialized view or cache物化视图或缓存	Stores the result of a recurring remote or cross-source computation存储持续远程或跨源计算的结果	Repeated expensive queries with acceptable refresh lag可接受刷新延迟的重复高成本查询	Refresh correctness, staleness and invalidation刷新正确性、陈旧与失效处理

“Lakehouse Federation vs query federation” is usually a terminology question rather than a technical choice: Databricks uses Lakehouse Federation for its query-federation platform. “Lakehouse Federation vs Delta Sharing” and “vs Lakeflow Connect” are real architecture choices because they change ownership, data movement, freshness, source load and failure behavior. A hybrid is normal: federate to explore, ingest the stable high-volume subset, and use sharing for provider-published datasets.

“Lakehouse Federation与query federation的区别”通常是术语问题，而非技术选型：Databricks用Lakehouse Federation指其查询联邦平台。“与Delta Sharing比较”和“与Lakeflow Connect比较”则是真实架构选择，因为它们会改变所有权、数据移动、新鲜度、源负载与失败行为。混合方式很常见：先联邦探索，再摄取稳定的高容量子集，并用共享机制分发由提供方发布的数据集。

How to Implement Lakehouse Federation Safely如何安全实施湖仓联邦

Select one bounded workload.选择一个有边界负载。
Name the decision, consumers, source tables, lakehouse tables, frequency, freshness and measurable pass conditions.
明确决策、消费者、源表、湖仓表、频率、新鲜度与可衡量通过条件。
Confirm connector and platform compatibility.确认连接器与平台兼容。
Read current vendor documentation for the exact cloud, runtime, source version, authentication method, identifiers, types, limits and quotas.
针对准确的云、Runtime、源版本、认证方式、标识符、类型、限制与配额阅读当前厂商文档。
Approve identity, policy and network paths.批准身份、策略与网络路径。
Create the least-privilege source role, store secrets, approve purpose, configure routing and test data plus token endpoints.
创建最小权限源角色、保存Secret、批准用途、配置路由，并测试数据端点与令牌端点。
Create the governed connection and catalog.创建受治理连接与目录。
Assign owners, grants, classifications, descriptions, support contacts and lifecycle dates. Expose only the required database and schemas.
分配所有者、授权、分类、描述、支持联系人与生命周期日期，只暴露所需数据库和Schema。
Validate metadata and semantics.验证元数据与语义。
Check object discovery, case handling, names, types, precision, time, nulls, keys, grain and metadata refresh behavior before joining data.
在连接数据前，检查对象发现、大小写处理、名称、类型、精度、时间、Null、键、粒度与元数据刷新行为。
Run positive, negative and source-load tests.运行正向、反向与源负载测试。
Compare golden results, prove denied access, inspect pushdown, inject failures, rotate credentials and measure both lakehouse and source resources.
比较黄金结果、证明拒绝访问、检查下推、注入故障、轮换凭据，并同时衡量湖仓与源资源。
Choose the durable operating mode.选择持久运营方式。
Keep the workload federated only if latency, correctness, source load, availability, cost and support remain acceptable. Otherwise ingest, replicate, share or materialize the approved subset.
仅当延迟、正确性、源负载、可用性、成本与支持仍可接受时才保持联邦；否则摄取、复制、共享或物化获批子集。
Operationalize change and exit.运营化变更与退出。
Monitor schema and credential changes, rehearse source and connector incidents, review grants, and prove revocation removes discovery and query access.
监控Schema与凭据变化，演练来源和连接器事件，复核授权，并证明撤销会移除发现与查询访问。

Lakehouse Federation Production Test Matrix湖仓联邦生产测试矩阵

Test correctness, security, performance, failure and lifecycle together同时测试正确性、安全、性能、失败与生命周期
Test测试	Pass condition通过条件	Evidence证据
Metadata fidelity元数据保真	Approved objects appear with correct names, types, precision and refresh behavior获批对象以正确名称、类型、精度与刷新行为出现	Catalog snapshot compared with source metadata目录快照与源元数据比较
Result correctness结果正确性	Golden queries reconcile for rows, aggregates, nulls, time and decimals黄金查询在行、聚合、Null、时间与小数方面核对一致	Source query, federated query, versions and difference report源查询、联邦查询、版本与差异报告
Authorization授权	Approved personas succeed; prohibited catalog, table, row and column paths fail获批身份成功；禁止的目录、表、行与列路径失败	Grant state, source role, decisions and denial logs授权状态、源角色、决策与拒绝日志
Pushdown and source load下推与源负载	Remote work matches plan and stays within scan, CPU, I/O, connection and timeout budgets远程工作符合计划，并处于扫描、CPU、I/O、连接与超时预算内	Lakehouse plan, remote SQL and source telemetry湖仓计划、远程SQL与源遥测
Failure and cancellation失败与取消	Network, source, connector and token failures are visible, bounded and recoverable网络、来源、连接器与令牌失败可见、有界且可恢复	Injected timeline, retry, cancel, alert and recovery records注入时间线、重试、取消、告警与恢复记录
Change and revocation变更与撤销	Schema change fails safely or remains compatible; removed access disappears within targetSchema变化安全失败或保持兼容；撤销访问在目标时间内消失	Metadata refresh, compatibility, cache, secret and grant evidence元数据刷新、兼容、缓存、Secret与授权证据

Where InfiniSynapse Fits in a Lakehouse Federation EvaluationInfiniSynapse在湖仓联邦评估中的位置

InfiniSynapse's public website describes direct connections to supported databases and multi-source joint analysis. That can help an approved team evaluate a bounded question across connected sources or compare representative results during an architecture assessment. The public product page does not establish InfiniSynapse as a lakehouse, Lakehouse Federation service, Unity Catalog, foreign catalog, CDC pipeline, data-sharing protocol, query optimizer, lineage system, source policy engine or federation control plane.

InfiniSynapse官网描述了对受支持数据库的直接连接与多源联合分析。这可以帮助获批团队评估跨已连接来源的有边界问题，或在架构评估中比较代表性结果。公开产品页并未证明InfiniSynapse是湖仓、Lakehouse Federation服务、Unity Catalog、Foreign Catalog、CDC管道、数据共享协议、查询优化器、血缘系统、源策略引擎或联邦控制面。

Keep owner approval, credentials, least-privilege source roles, network policy, row and column permissions, masking, source protection, audit, retention and lifecycle control in the responsible systems. Before analysis, prepare approved sources, supported connection details, permitted schemas, keys and grain, definitions, expected filters, observation times and a known reconciliation sample.

所有者批准、凭据、最小权限源角色、网络策略、行列权限、脱敏、源系统保护、审计、留存与生命周期控制应保留在负责系统中。分析前，请准备获批来源、受支持的连接信息、允许的Schema、键与粒度、定义、预期过滤、观察时间与已知核对样本。

Evaluate an approved connected-source question评估一个获批的已连接来源问题

When those controls and inputs are ready, use the InfiniSynapse Web App to evaluate connected-source analysis and reconcile representative results. It does not create, govern or operate the surrounding lakehouse federation.

当这些控制与输入准备就绪后，可使用InfiniSynapse Web App评估已连接来源分析并核对代表性结果。它不会创建、治理或运营外围湖仓联邦。

Evaluate connected-source analysis评估已连接来源分析

Lakehouse Federation FAQ湖仓联邦常见问题

What is Lakehouse Federation in Databricks?

Databricks中的Lakehouse Federation是什么？

Lakehouse Federation is Databricks' query-federation capability for accessing external systems through Unity Catalog. A connection represents the endpoint and credentials, while a foreign catalog mirrors external metadata for governed, primarily read-oriented queries. Connector behavior and requirements vary by source, cloud and runtime.

Lakehouse Federation是Databricks通过Unity Catalog访问外部系统的查询联邦能力。连接表示端点与凭据，Foreign Catalog镜像外部元数据，以支持受治理、主要面向读取的查询。连接器行为与要求会因来源、云和Runtime而不同。

What is the primary purpose of lakehouse federation?

湖仓联邦的主要目的是什么？

Its primary purpose is to answer approved questions against external data in place without first building a complete ingestion pipeline or copying every source table. It is useful for ad hoc reporting, proof of concept, current lookups and incremental migration when source load and network dependency are acceptable.

其主要目的是原地针对外部数据回答获批问题，而不必先构建完整摄取管道或复制每张源表。当源负载与网络依赖可接受时，它适合即席报告、概念验证、当前查找与渐进迁移。

Is Lakehouse Federation the same as query federation?

Lakehouse Federation与query federation相同吗？

In Databricks terminology, Lakehouse Federation is its query-federation platform, so they are not competing architectures there. More generally, query federation describes distributed query behavior, while lakehouse federation emphasizes that a lakehouse catalog, governance surface and compute environment are the access point.

在Databricks术语中，Lakehouse Federation就是其查询联邦平台，因此二者不是竞争架构。更一般地说，query federation描述分布式查询行为，而lakehouse federation强调以湖仓目录、治理界面与计算环境作为访问入口。

How is Lakehouse Federation different from Delta Sharing?

Lakehouse Federation与Delta Sharing有何不同？

Lakehouse Federation sends selected query work to an external system and returns results at query time. Delta Sharing is a data-sharing protocol through which a provider publishes selected datasets to recipients. The patterns differ in provider control, data path, supported operations, freshness and failure behavior.

Lakehouse Federation在查询时向外部系统发送选定工作并返回结果；Delta Sharing是一种数据共享协议，由提供方把选定数据集发布给接收方。两种模式在提供方控制、数据路径、支持操作、新鲜度与失败行为方面不同。

Is Lakehouse Federation read-only?

Lakehouse Federation是只读的吗？

Databricks currently documents foreign-table access through Lakehouse Federation as read-only, except for a specific catalog-federation case involving a legacy Hive metastore. Do not generalize that rule to every product, and never infer write guarantees from successful reads; verify the exact platform and connector documentation.

Databricks当前文档说明，通过Lakehouse Federation访问Foreign Table通常只读，但涉及旧版Hive Metastore的特定目录联邦场景例外。不能把该规则推广到所有产品，也不能从读取成功推断写入保证；必须验证准确的平台和连接器文档。

When should lakehouse federation be replaced with ingestion or CDC?

何时应把湖仓联邦替换为摄取或CDC？

Choose ingestion, CDC or materialization when a workload becomes frequent, high-volume, latency-sensitive, transformation-heavy, historically repeatable, unavailable-source tolerant or too disruptive to the operational source. Define thresholds for transfer, source load, latency, availability, correctness and cost before production rollout.

当负载变得高频、高容量、延迟敏感、转换密集、要求可重复历史、必须容忍来源不可用，或对运营来源干扰过大时，应选择摄取、CDC或物化。生产上线前应定义传输、源负载、延迟、可用性、正确性与成本阈值。

How should lakehouse federation be tested?

应如何测试湖仓联邦？

Test metadata and type fidelity, golden-result reconciliation, permissions, masking, pushdown, remote SQL, transfer, source load, concurrency, timeout and cancellation, network and token failures, schema changes, credential rotation, metadata refresh and revocation. Correlate lakehouse evidence with source-side telemetry.

应测试元数据与类型保真、黄金结果核对、权限、脱敏、下推、远程SQL、传输、源负载、并发、超时与取消、网络和令牌失败、Schema变化、凭据轮换、元数据刷新与撤销，并把湖仓证据与源端遥测关联。

Official and Primary Sources官方与第一方来源

The vendor documents establish current product behavior and examples; they do not define one universal lakehouse-federation standard. Verify the applicable cloud, edition, runtime, connector, source version, authentication, network, permissions, identifiers, limits, quotas, license and support terms for your environment.

这些厂商文档用于说明当前产品行为与示例，并不定义一个通用湖仓联邦标准。请针对自身环境验证适用的云、版本、Runtime、连接器、源版本、认证、网络、权限、标识符、限制、配额、许可与支持条款。