Federated Queries & Data Virtualization: Complete Guide联邦查询与数据虚拟化完整指南

Federated Queries and Data Virtualization: The Short Answer快速回答：联邦查询与数据虚拟化是什么

Federation reduces the need to copy every dataset before it can be explored, but it does not eliminate data movement: query results, join inputs, metadata, and temporary data still cross boundaries. It also does not replace warehouses, lakehouses, streaming systems, or durable integration pipelines. Use it when freshness, source autonomy, rapid access, or data-residency constraints outweigh the cost of remote execution. Materialize or replicate workloads that are repetitive, historically intensive, latency-sensitive, or too expensive for source systems.

联邦能够减少“必须先复制全部数据才能分析”的需求，但不会让数据移动完全消失：查询结果、连接输入、元数据和临时数据仍会跨越边界。它也不会取代数据仓库、湖仓、流系统或持久集成管道。当新鲜度、源自治、快速接入或数据驻留约束比远程执行成本更重要时，联邦更合适；对于重复频繁、依赖长期历史、延迟要求严格或会给源系统带来过高成本的负载，则应考虑物化或复制。

The architectural goal is not “zero-copy at any cost.” It is the least unnecessary movement consistent with correctness, performance, security, and operability. That principle keeps federated access useful without turning production systems into an uncontrolled analytics cluster.

架构目标不是“不惜代价实现零复制”，而是在正确性、性能、安全性和可运维性成立的前提下，减少不必要的数据移动。这一原则可以保留联邦访问的价值，同时避免把生产系统变成失控的分析集群。

1. What Is a Federated Data Query?1. 什么是联邦数据查询？

A federated data query addresses multiple autonomous data systems through one request and returns a logically unified result. The sources can differ in engine, schema, location, ownership, and access protocol: for example, an on-premises PostgreSQL order database, a cloud warehouse containing campaign facts, an object store with Iceberg tables, and a service API with customer status. The consumer writes against a common catalog or virtual model rather than manually exporting each source and joining files on a laptop.

联邦数据查询通过一次请求访问多个自治数据系统，并返回逻辑统一的结果。来源可以在引擎、模式、位置、所有权和访问协议上都不同，例如本地PostgreSQL订单数据库、保存营销事实的云仓库、包含Iceberg表的对象存储，以及提供客户状态的服务API。使用者面向公共目录或虚拟模型编写查询，而不必分别导出每个来源，再在个人电脑上手工合并文件。

“Autonomous” matters. A participating source keeps control of its storage, workload management, availability, schema changes, and local policies. The federation layer cannot assume that every source supports identical SQL, transactions, types, collation, functions, statistics, or concurrency. Connectors translate capabilities and expose limitations. The global engine must preserve semantics when it rewrites work; if a source compares strings differently, supports a different timestamp precision, or cannot execute a function consistently, an unsafe pushdown can change the answer.

“自治”非常关键。参与来源仍然控制自己的存储、负载管理、可用性、模式变化和本地策略。联邦层不能假设每个来源都支持相同的SQL、事务、类型、排序规则、函数、统计信息或并发。连接器负责转换能力并暴露限制。全局引擎在重写工作时必须保留语义；如果源端字符串比较方式不同、时间戳精度不同，或函数行为不一致，不安全的下推就可能改变答案。

Federated querying differs from conventional movement-based data integration. ETL, ELT, CDC, and replication create or maintain a copy in a target designed for repeated use. Federation resolves a request at query time against current source data and may cache or stage only selected intermediate results. Integration gives the target stronger control over performance, history, and isolation; federation gives faster access and preserves source location. A hybrid system often uses both.

联邦查询不同于传统的移动式数据集成。ETL、ELT、CDC与复制会在目标中创建或维护副本，以便重复使用；联邦则在查询时针对当前源数据解析请求，只可能缓存或暂存选定的中间结果。集成让目标更能控制性能、历史和隔离，联邦则更快提供访问并保留数据位置。混合系统往往同时使用两者。

Appropriate early use cases include cross-system discovery, migration assessment, incident investigation, long-tail reporting, governance-led access to sensitive sources, and proofs of concept. Poor first use cases include unrestricted self-service joins against a critical transaction database, recurring scans of entire remote tables, strict subsecond dashboards across a high-latency network, or legally required historical reconstruction when the source retains only current state.

适合作为早期用例的场景包括跨系统发现、迁移评估、事故调查、长尾报表、对敏感来源的治理式访问，以及概念验证。不适合作为首批用例的场景包括：对关键交易数据库开放无限制自助连接、反复全表扫描远程来源、在高延迟网络上要求严格亚秒级仪表板，或当源只保留当前状态时仍要求依法重建历史。

2. Why Are Federated Queries Needed?2. 为什么需要联邦查询？

Enterprise data rarely lives in one place. Acquisitions, regional autonomy, SaaS adoption, separate cloud accounts, regulated environments, and specialized engines create a durable distribution of data. Centralization programs can take years, and the estate changes while they run. Federation provides a controlled way to answer some cross-source questions during that reality instead of making every question wait for a new ingestion pipeline.

企业数据很少只存在于一个地方。并购、区域自治、SaaS采用、独立云账户、受监管环境和专业引擎会造成长期的数据分布。集中化项目可能持续多年，而在项目推进过程中数据体系仍会变化。联邦提供一种受控方式，让部分跨源问题无需等待新的摄取管道就能得到回答。

Cost is another driver, but the saving is conditional. Avoiding a full copy can reduce storage, pipeline development, reconciliation, and duplicate governance work. It can also increase remote compute, network transfer, connector infrastructure, query-engine capacity, source licensing, and on-call complexity. A useful cost comparison measures the workload—not only bytes stored. Ten exploratory queries may be cheaper through federation; a dashboard that executes the same remote join every minute may be cheaper and safer after materialization.

成本也是驱动因素，但节省具有条件。避免完整复制可以减少存储、管道开发、核对和重复治理工作，却也可能增加远程计算、网络传输、连接器基础设施、查询引擎容量、源端许可证和运维复杂度。有意义的成本比较必须衡量真实负载，而不只是存储字节。十次探索查询可能更适合联邦；每分钟执行相同远程连接的仪表板，则可能在物化后更便宜、更安全。

Freshness is a strong benefit when source systems are authoritative and change faster than pipelines can economically copy them. Query-time access can observe a current account state, inventory position, or operational event without waiting for a batch. Yet “real time” is not guaranteed. End-to-end latency includes planning, authentication, source queues, remote execution, transfer, cross-source joins, and result rendering. Teams should define a measurable freshness and response-time objective rather than labeling the architecture real time.

当源系统具有权威性，且变化速度快于管道经济复制能力时，新鲜度优势尤其明显。查询时访问可以查看当前账户状态、库存位置或业务事件，而无需等待批处理。但“实时”并非天然保证。端到端延迟包括规划、身份验证、源端排队、远程执行、传输、跨源连接与结果呈现。团队应定义可衡量的新鲜度和响应时间目标，而不是简单给架构贴上实时标签。

Governance and compliance can also favor in-place access. Policies may prohibit broad replication, require regional processing, or demand that local source controls remain authoritative. Federation can centralize policy evaluation and audit while minimizing persistent copies. It does not automatically make an architecture compliant: intermediate results, caches, spill files, logs, query text, and exported results may still contain sensitive data and require classification, encryption, retention, and deletion controls.

治理与合规也可能倾向原位访问。政策可能禁止广泛复制、要求区域内处理，或要求本地源控制继续保持权威。联邦可以在减少持久副本的同时集中策略评估与审计。但它不会自动让架构合规：中间结果、缓存、溢出文件、日志、查询文本和导出结果仍可能包含敏感数据，需要分类、加密、保留和删除控制。

The final driver is speed of learning. Architects can expose a narrow, read-only virtual view, validate demand and semantics, then decide whether a durable product deserves ingestion. This reverses an expensive sequence in which teams build pipelines before consumers prove value. The virtual view can remain for long-tail access, become the source of a controlled materialization job, or be retired when a curated product replaces it.

最后一个驱动因素是学习速度。架构师可以先公开范围狭窄、只读的虚拟视图，验证需求与语义，再决定某个持久数据产品是否值得摄取。这改变了“先建昂贵管道、后验证消费者价值”的顺序。虚拟视图可以继续用于长尾访问，也可以成为受控物化任务的来源，或在精选数据产品替代后退出。

3. Core Concepts: Data Virtualization vs. Data Federation3. 核心概念：数据虚拟化 vs 数据联邦

data virtualization and data federation overlap, but they are not perfectly interchangeable. Data federation is the technical capability to query and combine autonomous sources. Data virtualization is the managed abstraction layer around that capability: virtual schemas, semantic definitions, reusable views, security policies, discovery, lineage, APIs, caching, and service management. A federated engine can exist without a rich virtualization product; a virtualization platform usually relies on federation for live multi-source access.

数据虚拟化与数据联邦彼此重叠，但并不完全等价。数据联邦是查询并组合自治来源的技术能力；数据虚拟化则是在该能力周围建立的受管理抽象层，包括虚拟模式、语义定义、可复用视图、安全策略、发现、血缘、API、缓存和服务管理。一个联邦引擎可以没有完整的数据虚拟化产品；数据虚拟化平台通常依赖联邦来实现实时多源访问。

The difference is useful because architecture discussions otherwise confuse an execution engine with an operating model. An engine may execute SQL across catalogs but leave naming, business semantics, access requests, ownership, and lifecycle to other systems. A virtualization layer may publish customer, order, or risk data services that hide physical source names and remain stable while connectors or storage change. That stability is valuable, but it creates an interface that must be governed like any other product.

区分两者很重要，否则架构讨论会把执行引擎与运营模式混为一谈。引擎可以跨目录执行SQL，却把命名、业务语义、访问申请、所有权和生命周期交给其他系统。虚拟化层则可以发布客户、订单或风险数据服务，隐藏物理源名称，并在连接器或存储变化时保持接口稳定。这种稳定性很有价值，但也意味着该接口必须像其他产品一样治理。

Advantages shared by both approaches include faster access to distributed data, fewer unnecessary durable copies, source autonomy, a consistent interface, and centralized policy opportunities. Limitations include dependency on source availability, variable connector capabilities, cross-network latency, incomplete statistics, semantic mismatch, and the possibility that one poorly designed query harms several systems. Virtualization adds discoverability and reuse but can become a centralized bottleneck if every view, policy, or change waits on one platform team.

两种方式共享的优势包括更快访问分布式数据、减少不必要的持久副本、保留源自治、提供一致接口，以及集中策略的机会。局限包括依赖源可用性、连接器能力不一致、跨网络延迟、统计信息不完整、语义不匹配，以及一个糟糕查询可能同时伤害多个系统。虚拟化增强发现与复用，但如果每个视图、策略或变更都要等待一个平台团队，也会形成新的集中瓶颈。

Choose a federation-first implementation when expert users need flexible cross-source SQL and can work with technical catalogs. Add virtualization capabilities when many consumers need stable, discoverable, policy-controlled interfaces or when physical change must be hidden. Do not add an abstraction layer merely to rename tables; each layer should own a clear contract, lifecycle, and service expectation.

当专业用户需要灵活跨源SQL并能理解技术目录时，可优先采用联邦引擎；当大量使用者需要稳定、可发现、受策略控制的接口，或必须隐藏物理变化时，再增加虚拟化能力。不要只为改表名而增加抽象层；每一层都应拥有清晰契约、生命周期与服务预期。

4. Federation Architecture Explained4. 联邦架构详解

A production federation architecture separates concerns so that query logic, source connectivity, metadata, policy, execution, and operations can evolve without becoming one opaque appliance. The classic federated database system idea remains useful: multiple component databases retain autonomy while a federation layer presents a coherent interface. Modern implementations extend that pattern to warehouses, object stores, lakehouse tables, APIs, streams, search indexes, and SaaS systems.

生产级联邦架构需要分离关注点，使查询逻辑、源连接、元数据、策略、执行与运维能够独立演进，而不是形成一个不透明设备。经典联邦数据库系统思想仍然适用：多个组成数据库保留自治，联邦层提供一致接口。现代实现进一步把这一模式扩展到数据仓库、对象存储、湖仓表、API、流、搜索索引和SaaS系统。

At the top is the consumer and query interface: SQL endpoints, BI tools, notebooks, APIs, or applications. Authentication establishes identity and session context. A semantic or virtual schema translates consumer-facing names into logical objects. The catalog supplies tables, columns, types, constraints, owners, classifications, statistics, freshness, and source mappings. Without trustworthy metadata, the optimizer guesses and users cannot understand what a virtual object means.

最上层是使用者与查询接口：SQL端点、BI工具、Notebook、API或应用。身份验证建立身份与会话上下文；语义或虚拟模式把面向使用者的名称转换为逻辑对象；目录提供表、列、类型、约束、责任人、分类、统计、新鲜度和源映射。没有可信元数据，优化器只能猜测，使用者也无法理解虚拟对象的含义。

The global query layer parses and validates requests, resolves objects, checks privileges, creates a logical plan, applies rewrites, estimates alternatives, selects an execution plan, coordinates distributed tasks, and assembles results. It may contain a cost-based optimizer, rule engine, scheduler, memory manager, exchange service, spill storage, result cache, and workload manager. Some engines run coordinator and worker roles separately; others use managed serverless execution. The logical responsibility is the same even when deployment differs.

全局查询层负责解析与验证请求、解析对象、检查权限、创建逻辑计划、应用重写、估算备选方案、选择执行计划、协调分布式任务并汇总结果。它可能包含基于成本的优化器、规则引擎、调度器、内存管理器、交换服务、溢出存储、结果缓存与负载管理器。有些引擎分离协调者和工作节点，有些采用托管无服务器执行；即使部署不同，逻辑责任仍然一致。

The connector layer is the contract between global planning and local systems. A connector discovers metadata, maps types and functions, creates source-specific queries or API calls, authenticates, opens splits or partitions, reads pages, reports statistics, and declares which predicates, projections, aggregations, limits, or joins can be pushed down safely. Connector quality is not measured by basic connectivity alone. Correctness under type conversion, null semantics, collation, time zones, schema changes, throttling, and retries matters more.

连接器层是全局规划与本地系统之间的契约。连接器发现元数据、映射类型与函数、创建源特定查询或API调用、完成身份验证、打开分片或分区、读取数据页、报告统计，并声明哪些谓词、投影、聚合、限制或连接能够安全下推。连接器质量不能只用“是否连通”衡量；在类型转换、空值语义、排序规则、时区、模式变化、限流与重试下保持正确性更重要。

At the bottom are autonomous sources. Their workload managers, indexes, partitions, materialized views, permissions, maintenance windows, and failure modes remain authoritative. A source guardrail can cap concurrency, duration, scanned bytes, memory, or returned rows. The federation must respect local rate limits and avoid turning an interactive source into an accidental batch engine. Network paths, private endpoints, DNS, certificates, firewalls, proxies, and egress policies are part of this layer, not an afterthought.

底层是自治数据源。它们自己的负载管理器、索引、分区、物化视图、权限、维护窗口和故障模式仍然具有权威性。源端护栏可以限制并发、时长、扫描字节、内存或返回行数。联邦必须尊重本地限流，避免把交互式源变成意外的批处理引擎。网络路径、私有端点、DNS、证书、防火墙、代理和传出策略都属于这一层，而不是事后补充。

Cross-cutting services include secrets, encryption keys, audit logs, lineage, observability, configuration delivery, incident response, and chargeback. High availability requires more than multiple coordinators: credentials, catalog services, DNS, network paths, spill storage, and source endpoints must be considered. Because a federated answer can depend on several systems at once, the weakest dependency often determines the visible service level.

横向服务包括密钥、加密密钥、审计日志、血缘、可观测性、配置交付、事故响应与成本归属。高可用不仅意味着多个协调器；凭据、目录服务、DNS、网络路径、溢出存储与源端点都必须纳入考虑。由于联邦答案可能同时依赖多个系统，最弱的依赖往往决定最终可见服务等级。

5. How Does a Federated Query Work?5. 联邦查询如何工作？

A federated query is not simply broadcast to every source. A correct engine narrows the request through a sequence of resolution, planning, capability negotiation, distributed execution, and validation. Understanding this lifecycle helps engineers diagnose whether a slow query is caused by user SQL, global planning, a connector, the network, a source, or result assembly.

联邦查询并不是简单广播到所有来源。正确的引擎会依次完成解析、规划、能力协商、分布式执行与验证，从而缩小请求范围。理解这个生命周期有助于工程师判断慢查询究竟来自用户SQL、全局规划、连接器、网络、数据源还是结果汇总。

1. Parse and validate. Convert SQL or another request into a syntax tree, resolve names, check types, expand views, and reject invalid or unauthorized references.
2. Build a logical plan. Represent scans, filters, projections, joins, aggregations, sorts, windows, and limits independently of physical execution.
3. Rewrite and simplify. Prune unused columns, fold constants, infer predicates, decorrelate subqueries where possible, and normalize expressions without changing semantics.
4. Discover capabilities and statistics. Ask connectors what operations they can perform and obtain row counts, null fractions, ranges, distinct values, partitions, indexes, and cost signals when available.
5. Choose a physical plan. Select join order and distribution, decide what to push down, assign stages and exchanges, reserve resources, and establish retry boundaries.
6. Generate source work. Translate fragments into source SQL, storage scans, API calls, or native operations with bound parameters and identity context.
7. Execute in parallel. Sources and workers produce pages or streams; exchanges repartition, broadcast, merge, or gather data for remaining operators.
8. Combine and return. Complete cross-source joins and aggregations, apply final ordering or limits, enforce output policies, and stream or store the result.
9. Record evidence. Emit query text or fingerprints, plans, lineage, policy decisions, timing, source load, transferred bytes, retries, errors, and result status.

1. 解析与验证。把SQL或其他请求转换为语法树，解析名称、检查类型、展开视图，并拒绝无效或未授权引用。
2. 建立逻辑计划。用独立于物理执行的方式表示扫描、过滤、投影、连接、聚合、排序、窗口与限制。
3. 重写与简化。裁剪未使用列、折叠常量、推导谓词、在可能时去相关子查询，并在不改变语义的情况下规范表达式。
4. 发现能力与统计。询问连接器能执行哪些操作，并在可用时获取行数、空值比例、范围、唯一值、分区、索引与成本信号。
5. 选择物理计划。选择连接顺序与分布，决定下推内容，分配阶段与交换，预留资源并建立重试边界。
6. 生成源端工作。把片段转换为源SQL、存储扫描、API调用或本地操作，并携带参数和身份上下文。
7. 并行执行。来源和工作节点生成数据页或数据流，交换阶段对数据进行重分区、广播、合并或汇聚。
8. 组合并返回。完成跨源连接与聚合，应用最终排序或限制，执行输出策略，再流式返回或保存结果。
9. 记录证据。输出查询文本或指纹、计划、血缘、策略决策、耗时、源负载、传输字节、重试、错误与结果状态。

predicate pushdown is the best-known optimization. A condition such as WHERE region = 'APAC' is sent to a source so irrelevant rows never cross the network. Projection pushdown requests only needed columns. Aggregation pushdown asks a source to calculate grouped results. Limit, Top-N, function, dereference, and sometimes join pushdown reduce central work further. Trino’s official pushdown documentation emphasizes that support is connector- and source-specific and can be verified in an execution plan.

谓词下推是最知名的优化。像WHERE region = 'APAC'这样的条件会发送到源端，使无关行不跨越网络；投影下推只请求必要列；聚合下推要求源端计算分组结果；限制、Top-N、函数、嵌套字段以及有时的连接下推还能进一步减少中央工作。Trino的官方下推文档强调，下推支持取决于连接器与数据源，并可通过执行计划验证。

Pushdown must preserve meaning. A source may use case-insensitive string comparisons while the global engine is case-sensitive; timestamps, decimal rounding, null ordering, regular expressions, and time-zone functions can also differ. If equivalence is uncertain, a connector should retain the operation in the global engine or explicitly reject it. Moving less data is not an optimization if it returns a different answer.

下推必须保留含义。源端可能使用不区分大小写的字符串比较，而全局引擎区分大小写；时间戳、小数舍入、空值排序、正则表达式与时区函数也可能不同。如果无法确认等价性，连接器应把操作保留在全局引擎，或明确拒绝。若答案发生变化，减少数据移动就不能称为优化。

Failures require stage-aware behavior. A read-only source fragment may be safe to retry, but an API with metered calls or non-repeatable snapshots may not be. If sources change during a long query, the result may not represent one globally consistent instant because independent systems rarely share a transaction boundary. The service should document consistency semantics, snapshot support, timeout policy, cancellation propagation, partial-result behavior, and whether retries can observe newer data.

失败处理必须理解执行阶段。只读源片段可能可以安全重试，但计量调用或不可重复快照的API未必适合。如果来源在长查询期间发生变化，结果可能并不代表同一个全局一致时点，因为独立系统很少共享事务边界。服务必须说明一致性语义、快照支持、超时策略、取消传播、部分结果行为，以及重试是否可能看到更新数据。

6. Core Query Optimization Techniques6. 查询优化的核心技术

Federated query optimization is harder than optimization inside one database because the planner has partial and changing knowledge. Remote sources expose different statistics, capabilities, costs, and queue states. Network transfer may dominate CPU. A fast local scan can become slow after millions of rows cross a region boundary. An apparently expensive source aggregation may be the cheapest global choice if it reduces transfer by three orders of magnitude.

联邦查询优化比单一数据库内部优化更困难，因为规划器掌握的信息不完整且持续变化。远程来源暴露的统计、能力、成本和队列状态各不相同；网络传输可能比CPU更重要；本地扫描很快，但几百万行跨区域传输后可能变慢；看似昂贵的源端聚合，如果能把传输量降低几个数量级，反而可能是全局最便宜的选择。

A cost-based optimizer compares candidate plans using estimated row counts and the expected CPU, memory, input/output, and network cost of operators. It needs table and column statistics such as size, row count, null fraction, distinct values, ranges, and histograms. Trino’s cost-based optimization documentation notes that connectors must supply table statistics for cost-based strategies. When statistics are missing, stale, or mapped incorrectly, estimates become unknown and plan quality degrades.

基于成本的优化器使用估计行数，以及算子的CPU、内存、输入输出和网络成本来比较候选计划。它需要表与列统计，例如大小、行数、空值比例、唯一值、范围与直方图。Trino的基于成本优化官方文档指出，连接器必须提供表统计，成本策略才能工作。当统计缺失、过期或映射错误时，估算会变成未知，计划质量也会下降。

Join order is usually decisive. Joining two selective dimensions before a large fact table can reduce work; joining two large remote tables centrally can overwhelm network and memory. A planner may broadcast a small relation to workers, repartition both inputs on a key, colocate joins within one source, or stage a temporary result near another source. Each choice has correctness, security, and operational consequences. Temporary staging may violate a residency rule even if it improves speed.

连接顺序通常具有决定性。先连接两个选择性高的维表，再处理大事实表可以减少工作；在中央连接两个大型远程表则可能压垮网络与内存。规划器可以把小关系广播到工作节点，按键重分区两侧输入，在同一源内完成连接，或在另一个源附近暂存临时结果。每种选择都有正确性、安全与运维影响；即使临时暂存能提升速度，也可能违反驻留规则。

Rules complement cost estimates. Predicate and projection pushdown, partition pruning, constant folding, dynamic filtering, common-subexpression reuse, limit pushdown, and early aggregation often reduce data before expensive exchanges. Dynamic filtering can collect keys from a small join side and use them to prune the large side during execution. Adaptive planning may revise decisions when observed row counts differ from estimates. These techniques help, but connector coverage and source semantics determine whether they apply.

规则可以补充成本估算。谓词与投影下推、分区裁剪、常量折叠、动态过滤、公共子表达式复用、限制下推与提前聚合，往往能在昂贵交换前减少数据。动态过滤可以从连接的小侧收集键，并在执行时用这些键裁剪大侧。自适应规划还可以在实测行数偏离估计时修订决策。这些技术都有帮助，但最终是否适用取决于连接器覆盖与源端语义。

Optimize from evidence. Capture a query fingerprint, logical and physical plans, per-stage input and output, connector-generated requests, source execution plans, time to first row, total duration, bytes transferred, peak memory, spill, and consumer wait. Compare repeated runs only after accounting for caches, source load, statistics refresh, and parameter values. A query that becomes fast only because a remote cache is warm may still violate a cold-start service level.

优化必须依据证据。需要捕获查询指纹、逻辑与物理计划、各阶段输入输出、连接器生成的请求、源端执行计划、首行时间、总耗时、传输字节、峰值内存、溢出与使用者等待。比较重复运行时必须考虑缓存、源负载、统计刷新和参数值。一个只有远程缓存预热后才变快的查询，仍可能不满足冷启动服务等级。

Set guardrails as well as tuning goals: maximum remote scan, maximum transferred bytes, concurrency per source, statement timeout, result-size cap, memory limit, and cancellation deadline. Route recurring expensive queries to curated views, caches, aggregates, or materialized products. The best optimizer cannot turn every unconstrained cross-source join into an appropriate interactive workload.

除了调优目标，还要设置护栏：最大远程扫描、最大传输字节、每个源的并发、语句超时、结果大小上限、内存限制与取消期限。把重复昂贵查询引导到精选视图、缓存、聚合或物化产品。即使最好的优化器，也无法把所有无约束跨源连接都变成合适的交互负载。

7. Distributed Databases and Distributed SQL7. 分布式数据库与分布式SQL

A distributed database stores and manages one logical database across multiple nodes or locations under a coordinated system. distributed sql commonly presents SQL semantics over that distributed storage and execution environment. Federation, by contrast, coordinates independently managed systems that may have different engines, owners, transactions, and service levels. Both distribute work, but the boundary of control is different.

分布式数据库在协调系统下，把一个逻辑数据库的存储与管理分布到多个节点或位置。分布式SQL通常在这种分布式存储与执行环境之上提供SQL语义。联邦则协调由不同团队独立管理的系统，这些系统可能具有不同引擎、所有者、事务与服务等级。两者都会分布工作，但控制边界不同。

A distributed database can make global guarantees because it owns placement, replication, membership, consensus or transaction protocols, and often statistics. A federation layer usually cannot impose a global transaction, coordinated snapshot, index, or storage layout on its sources. It must negotiate capabilities and tolerate partial information. A query joining current values from three autonomous systems may be internally correct yet not represent one consistent business instant.

分布式数据库因为控制数据放置、复制、成员关系、共识或事务协议，通常也控制统计信息，因此可以提供全局保证。联邦层一般无法在源端强制统一事务、协调快照、索引或存储布局，只能协商能力并容忍信息不完整。连接三个自治系统当前值的查询，内部计算可能正确，却未必代表同一个一致业务时点。

MPP architecture is an execution pattern rather than a synonym for either category. A federation engine may use massively parallel workers to scan and combine remote results. A warehouse or distributed SQL database may also use MPP over data it owns. The design question is where data is managed and which system controls optimization and consistency, not whether several CPUs execute in parallel.

MPP是一种执行模式，并不等同于任何一个类别。联邦引擎可以使用大规模并行工作节点扫描并组合远程结果；数据仓库或分布式SQL数据库也可以在自己管理的数据上采用MPP。真正的设计问题是数据在哪里管理、哪个系统控制优化与一致性，而不是是否有多个CPU并行执行。

Use distributed storage when the workload needs predictable repeated performance, global transactions, controlled placement, or one operational database at scale. Use federation when sources must remain autonomous or when consolidating them would be slow, prohibited, or disproportionate to the question. A hybrid can federate several distributed platforms, but architects should be explicit about which layer owns each guarantee.

当负载需要可预测的重复性能、全局事务、受控放置或大规模单一业务数据库时，应使用分布式存储；当来源必须保持自治，或集中化速度太慢、被禁止、或与问题价值不成比例时，可使用联邦。混合架构可以联邦多个分布式平台，但架构师必须明确每项保证由哪一层负责。

8. Federated Data Governance and Security8. 联邦数据治理与安全

federated access expands the trust boundary. A user authenticates to the federation, the federation authenticates to sources, policies may be evaluated globally and locally, and results may contain combinations that are more sensitive than either input alone. federated data governance must therefore cover identity, metadata, policy, lineage, audit, privacy, retention, and operational ownership across every hop.

联邦访问会扩大信任边界。用户向联邦层验证身份，联邦层再向来源验证；策略可能在全局和本地分别评估；最终结果组合后的敏感性可能高于任何单一输入。因此，联邦数据治理必须覆盖每一跳的身份、元数据、策略、血缘、审计、隐私、保留与运维责任。

Choose an identity model deliberately. Pass-through identity lets a source evaluate the end user but requires compatible identity, delegation, and auditing across systems. Service identity simplifies connections but shifts detailed authorization to the federation and can make source logs show one shared account. Hybrid models use constrained service credentials plus user context or policy tags. Whichever model is chosen, test revocation, group changes, dormant accounts, credential rotation, and break-glass access.

应有意识地选择身份模型。透传身份让源端评估最终用户，但要求跨系统兼容身份、委派与审计；服务身份简化连接，却把细粒度授权转移到联邦层，并可能让源日志只显示一个共享账户；混合模型则使用受限服务凭据，并携带用户上下文或策略标签。无论选择哪种模型，都要测试权限撤销、组变化、休眠账户、凭据轮换与紧急访问。

Apply least privilege at source, federation, virtual-object, row, column, and result levels as required. Masking and row filters must be enforced where they cannot be bypassed by alternative routes. If a filter is pushed down, confirm the connector preserves policy semantics. If it is evaluated centrally, confirm restricted raw values are allowed to leave the source. Policies also need deterministic conflict handling when a global rule and a stricter local rule disagree; the safer effective rule should win.

应根据需要在源、联邦、虚拟对象、行、列与结果层执行最小权限。脱敏与行过滤必须在无法被其他路径绕过的位置执行。若过滤被下推，要确认连接器保留策略语义；若在中央评估，要确认受限原始值可以离开源端。当全局规则与更严格的本地规则冲突时，还要有确定性的处理方式，最终应采用更安全的有效规则。

Lineage should connect the consumer request to virtual views, policy decisions, source objects, pushed fragments, transformations, and materialized or exported results. Dynamic SQL and connector-generated statements make lineage harder, so capture stable object identifiers and query fingerprints. Audit records should answer who requested what, under which purpose and role, which sources were touched, what policy allowed or denied access, how much data returned, and whether the result was exported.

血缘应把使用者请求与虚拟视图、策略决策、源对象、下推片段、转换以及物化或导出结果连接起来。动态SQL和连接器生成语句会增加血缘难度，因此要捕获稳定对象标识和查询指纹。审计记录应能回答：谁以什么目的和角色请求了什么、访问了哪些来源、哪项策略允许或拒绝访问、返回多少数据、结果是否被导出。

Privacy risk includes composition. A non-sensitive customer table and a non-sensitive location table may produce a sensitive joined view. Use classification propagation, purpose limitation, query controls, result-size limits, aggregation thresholds, and review for re-identification risk where appropriate. Protect temporary artifacts: exchanged pages, caches, spill files, preview samples, logs, and error messages. Encrypt data in transit and at rest, isolate tenant or workload boundaries, and ensure deletion or expiration covers federation-managed copies.

隐私风险还来自组合。一个非敏感客户表与一个非敏感位置表连接后，可能产生敏感视图。应使用分类传播、用途限制、查询控制、结果大小限制、聚合阈值，并在适用时评审重新识别风险。还要保护临时资产：交换数据页、缓存、溢出文件、预览样例、日志与错误消息。数据在传输和静态时都应加密，租户或负载边界应隔离，删除或到期规则必须覆盖联邦管理的副本。

Operational governance assigns owners and service levels. A virtual dataset needs a business owner, source steward, platform owner, policy owner, and support route. Define who approves semantic change, connector upgrades, new consumers, heavy queries, and emergency shutdown. Local source teams need visibility into federated demand and the ability to protect production without silently breaking consumers. Change notices and contracts reduce surprises on both sides.

运营治理需要分配责任与服务等级。虚拟数据集需要业务责任人、源端管理员、平台责任人、策略责任人与支持路径。要明确谁批准语义变化、连接器升级、新使用者、重查询与紧急停机。源端团队需要看到联邦需求，并能保护生产系统，同时不能静默破坏使用者。变更通知与契约可以减少双方意外。

9. Lakehouse Federation9. 湖仓联邦（Lakehouse Federation）

lakehouse federation connects a lakehouse query and governance environment to external databases or catalogs so users can discover and query remote data without first ingesting everything. It is useful during migration, exploratory analysis, on-demand reporting, and cases where remote compute or data residency should remain authoritative. The lakehouse can provide a consistent catalog and access interface while queries execute partly in the external system and partly in the lakehouse engine.

湖仓联邦把湖仓查询与治理环境连接到外部数据库或目录，使用户无需先摄取全部数据就能发现并查询远程内容。它适用于迁移期间、探索分析、按需报表，以及远程计算或数据驻留应保持权威的场景。湖仓可以提供一致的目录与访问接口，而查询则部分在外部系统执行，部分在湖仓引擎执行。

Open table formats such as Apache Iceberg and Delta Lake address storage metadata, transactions, schema evolution, partitioning, and snapshot behavior for data in object storage. They do not by themselves federate arbitrary operational databases or APIs. A query engine still needs catalogs, connectors, credentials, planning, and governance to combine open tables with other sources. Conversely, a federation can query proprietary sources without converting them to an open table format.

Apache Iceberg与Delta Lake等开放表格式处理对象存储中数据的存储元数据、事务、模式演进、分区与快照行为，但它们本身不会联邦任意业务数据库或API。查询引擎仍需要目录、连接器、凭据、规划与治理，才能把开放表与其他来源组合。反过来，联邦也可以在不把专有来源转换成开放表格式的情况下进行查询。

Databricks describes query federation in Lakehouse Federation as translating SQL into statements pushed to foreign databases and using both local and remote compute. Its documentation positions the approach for on-demand reporting, proofs of concept, exploratory pipeline work, and incremental migration, while documenting read-only behavior and connector-specific limitations. The exact constraints are product-specific, but the architectural lesson is general: remote access must be evaluated by use case and connector.

Databricks在Lakehouse Federation查询联邦文档中说明，SQL会被转换为下推到外部数据库的语句，并同时使用本地与远程计算。其文档把该方式定位于按需报表、概念验证、探索性管道工作和增量迁移，同时记录只读行为与连接器特定限制。具体约束属于产品实现，但通用架构启示是：远程访问必须按使用场景和连接器实测。

A practical decision asks whether to query an open table directly, federate a database, create an external table, ingest with CDC, or materialize a curated lakehouse table. Consider change frequency, expected scans, history, source impact, semantics, concurrency, consistency, governance, and cost. Federation can accelerate access; curated lakehouse data often serves repeated, high-volume, history-dependent workloads better. Use evidence from the pilot to move workloads between the two patterns.

实际决策要在直接查询开放表、联邦数据库、创建外部表、使用CDC摄取或物化精选湖仓表之间选择。需要考虑变化频率、预期扫描、历史、源端影响、语义、并发、一致性、治理与成本。联邦可以加速接入；精选湖仓数据通常更适合重复、高容量且依赖历史的负载。应依据试点证据在两种模式之间迁移负载。

10. Federated Analytics10. 联邦分析（Federated Analytics）

federated analytics applies federated access to analytical questions across organizational, technological, or geographic boundaries. A BI analyst might combine current orders from an operational database with targets in a warehouse. A risk team might join a governed customer view with regional events that cannot be broadly copied. A data scientist might inspect features across a lakehouse and a model registry before deciding what to materialize for training.

联邦分析把联邦访问应用于跨组织、技术或地域边界的分析问题。BI分析师可以把业务数据库中的当前订单与仓库目标结合；风险团队可以把受治理客户视图与不允许广泛复制的区域事件连接；数据科学家可以先检查湖仓和模型注册表中的特征，再决定哪些数据需要物化用于训练。

Interactive BI needs predictable response time, stable semantics, high concurrency, and source protection. Federation is often strongest for selective drill-through, current-state lookup, and low-frequency long-tail questions. A dashboard that refreshes many tiles for hundreds of users can multiply remote queries dramatically. Semantic-model caching, aggregate tables, result caching, concurrency limits, and scheduled materialization may be required even when the logical interface remains virtual.

交互式BI需要可预测响应时间、稳定语义、高并发与源端保护。联邦通常最适合选择性下钻、当前状态查找和低频长尾问题。一个为数百名用户刷新多个组件的仪表板，可能把远程查询放大许多倍。即使逻辑接口仍然是虚拟的，也可能需要语义模型缓存、聚合表、结果缓存、并发限制与定时物化。

Data science introduces large scans, iterative experimentation, and reproducibility requirements. Training should not repeatedly scan critical operational systems, and a live source may change between runs. Use federation for discovery, feature feasibility, narrow sampling, and source comparison. Once a workload is accepted, create versioned snapshots or governed feature datasets where reproducibility, scale, and isolation require them. Record source versions, query text, time windows, and mapping decisions.

数据科学会带来大规模扫描、迭代实验与可复现要求。训练不应反复扫描关键业务系统，实时来源也可能在两次运行之间变化。可使用联邦进行发现、特征可行性评估、窄范围抽样与来源比较；负载获得认可后，如果复现、规模与隔离需要，则应创建有版本快照或受治理特征数据集，并记录源版本、查询文本、时间窗口与映射决策。

A hypothetical example: an operations team wants to understand late shipments. The virtual view joins current fulfillment status in an operational database, carrier events from an API, and order value in a warehouse. The pilot limits analysis to one region and one week, pushes date and region filters to each source, maps status codes, and aggregates carrier events before joining. It measures source CPU, transferred bytes, p95 response time, and reconciliation against small approved extracts. If analysts repeat the query daily at larger scale, the team materializes a curated shipment fact while preserving the virtual view for current exception lookup.

假设一个运营团队要分析延迟发货。虚拟视图连接业务数据库中的当前履约状态、承运商API事件以及仓库中的订单价值。试点限制在一个区域和一周范围，把日期与区域过滤下推到每个源，映射状态代码，并在连接前聚合承运商事件。团队测量源CPU、传输字节、p95响应时间，并与小范围批准导出进行核对。如果分析师每天重复执行且规模扩大，团队就物化精选发货事实表，同时保留虚拟视图用于当前异常查找。

Define success in consumer and system terms: question answered correctly, time to first useful result, p50/p95 response, concurrency, freshness, source load, transferred bytes, failed and canceled queries, semantic defects, policy violations, support effort, and cost per workload. A fast query that produces inconsistent measures or overwhelms a source is not successful federated analytics.

成功标准必须同时覆盖使用者与系统：问题是否正确回答、获得首个有用结果的时间、p50/p95响应、并发、新鲜度、源负载、传输字节、失败与取消查询、语义缺陷、策略违规、支持投入与每个负载成本。一个查询即使很快，如果产生不一致指标或压垮源系统，也不能算成功的联邦分析。

11. Tools and Platform Selection11. 工具与平台选型

Evaluate data virtualization tools, a data virtualization platform, or data virtualization software against a defined workload rather than a generic feature list. Some offerings emphasize a distributed SQL engine and connectors; others add semantic modeling, data services, catalogs, lineage, caching, low-code design, or centralized policy. Open-source engines can offer transparency and portability, while commercial or managed platforms can reduce packaging and support work. Neither sourcing model is automatically lower cost or lower risk.

评估数据虚拟化工具、数据虚拟化平台或数据虚拟化软件时，应以明确工作负载为依据，而不是看通用功能列表。有些产品强调分布式SQL引擎与连接器，另一些还提供语义建模、数据服务、目录、血缘、缓存、低代码设计或集中策略。开源引擎可以提高透明度与可移植性，商业或托管平台可以减少打包与支持工作，但任何来源模式都不会天然拥有更低成本或更低风险。

Connector count is not connector quality. Test the hardest source version, private network path, authentication mode, type edge cases, schema changes, predicate and aggregation pushdown, cancellation, throttling, retries, and large-result behavior. AWS documents that Athena Federated Query uses data-source connectors, manages parallelism, and pushes filter predicates, while its supported sources and behavior depend on connector type. The general lesson is to validate exact operations, not accept a connector logo as proof.

连接器数量不代表连接器质量。应测试最困难的源版本、私网路径、身份方式、类型边界、模式变化、谓词与聚合下推、取消、限流、重试和大结果行为。AWS文档说明Athena Federated Query使用数据源连接器、管理并行度并下推过滤谓词，同时支持来源与行为取决于连接器类型。通用启示是：必须验证准确操作，不能把连接器图标当成证据。

Run a proof of concept with production-like identities, realistic data volume and skew, concurrent users, expected source load, and failure injection. Include a selective query, a cross-source join, an aggregation, a schema change, a revoked credential, a source timeout, a canceled query, and a result that exceeds limits. Measure correctness, p50 and p95 duration, bytes scanned and transferred, source CPU, memory and spill, policy decisions, recovery, operator effort, and cost. Reject any platform that cannot make these behaviors visible.

概念验证应使用接近生产的身份、真实数据量与倾斜、并发用户、预期源负载和故障注入。至少包含选择性查询、跨源连接、聚合、模式变化、撤销凭据、源超时、取消查询以及超过限制的结果。测量正确性、p50与p95耗时、扫描与传输字节、源CPU、内存与溢出、策略决策、恢复、运维投入和成本。无法让这些行为可见的平台不应通过评估。

For the broader procurement method, use the companion data integration tools selection guide. This guide adds federation-specific gates: semantic equivalence under pushdown, source workload protection, incomplete statistics, multi-source consistency, intermediate-result security, and the ability to route unsuitable recurring workloads to materialization.

更通用的采购方法可参考配套的数据集成工具选型指南。本指南增加联邦特有门槛：下推时的语义等价、源负载保护、不完整统计、多源一致性、中间结果安全，以及把不适合的重复负载转向物化的能力。

12. Implementation Best Practices12. 实施最佳实践

Begin with one high-value, bounded question. Name the consumer, decision, sources, owners, grain, freshness, response-time target, concurrency, security classification, acceptable source load, and fallback. Prefer a read-only use case with selective filters and a small final result. Avoid starting with an enterprise-wide virtual layer or a dashboard that scans every system. A narrow end-to-end slice exposes identity, metadata, networking, semantics, optimization, and operations without making the blast radius unmanageable.

从一个高价值、边界清晰的问题开始。明确使用者、决策、来源、责任人、粒度、新鲜度、响应时间目标、并发、安全分类、可接受源负载与回退方案。优先选择只读、过滤选择性高且最终结果较小的用例。不要从企业级全域虚拟层或扫描所有系统的仪表板开始。一条窄而完整的路径可以暴露身份、元数据、网络、语义、优化与运维问题，同时保持可控影响面。

1. Establish ownership and guardrails. Obtain source approval; define identities, quotas, timeouts, scan and result caps, maintenance behavior, and emergency disablement.
2. Profile sources and networks. Measure schemas, types, keys, statistics, data skew, indexes, partitions, source capacity, round-trip latency, throughput, and egress.
3. Build the catalog and contracts. Record business definitions, mappings, owners, classifications, freshness, consistency, deprecation, and source-change expectations.
4. Implement the simplest virtual view. Select only necessary columns and rows, keep transformations explicit, and avoid stacking opaque views.
5. Verify pushdown and plans. Inspect global and source plans; confirm semantic equivalence and measure what crosses each boundary.
6. Test correctness and failure. Reconcile approved samples, test nulls and time zones, inject source and network failures, cancel queries, and verify no unsafe partial result.
7. Load-test safely. Increase concurrency and data scale under source-team supervision; record saturation, queues, tail latency, and cancellation delay.
8. Instrument before release. Publish dashboards, alerts, lineage, audit, cost, ownership, runbooks, and consumer-facing service expectations.
9. Pilot and review. Limit users and workload, collect evidence, materialize recurring heavy paths, and approve expansion only after operational review.

1. 建立责任与护栏。取得源端批准，定义身份、配额、超时、扫描与结果上限、维护行为和紧急停用。
2. 剖析来源与网络。测量模式、类型、键、统计、倾斜、索引、分区、源容量、往返延迟、吞吐与传出。
3. 建立目录与契约。记录业务定义、映射、责任人、分类、新鲜度、一致性、弃用与源变化预期。
4. 实现最简单虚拟视图。只选择必要列与行，保持转换显式，避免叠加不透明视图。
5. 验证下推与计划。检查全局和源端计划，确认语义等价，并测量每个边界的传输内容。
6. 测试正确性与失败。核对批准样例，测试空值与时区，注入源和网络故障，取消查询，并确认不会返回不安全的部分结果。
7. 安全进行负载测试。在源团队监督下提高并发与数据规模，记录饱和、队列、尾延迟和取消延迟。
8. 发布前完成监控。提供面板、告警、血缘、审计、成本、责任、运行手册与使用者服务预期。
9. 试点并复审。限制用户与负载，收集证据，把重复重负载物化，只有通过运维复审后才扩大。

Treat virtual schemas and views as versioned products. Use code review, tests, promotion between environments, dependency analysis, change notices, and deprecation windows. Avoid exposing physical source names directly when consumers need stability, but do not hide lineage. A contract should state grain, keys, semantics, freshness, consistency, known limitations, owner, and support route. Changing a join or definition is a product change even if SQL still compiles.

把虚拟模式与视图当成有版本的数据产品。使用代码评审、测试、环境晋级、依赖分析、变更通知与弃用窗口。当使用者需要稳定性时，不要直接暴露物理源名称，但也不能隐藏血缘。契约应说明粒度、键、语义、新鲜度、一致性、已知限制、责任人与支持路径。即使SQL仍能编译，连接或定义变化也属于产品变化。

Observe both engine and business behavior. Infrastructure metrics include coordinator and worker capacity, queues, CPU, memory, spill, exchanges, connector pools, network, retries, source time, and cache. Data metrics include row counts, null and uniqueness trends, mapping failures, freshness, reconciliation, policy denials, and result-size distributions. Consumer metrics include adoption, abandoned queries, time to answer, repeated expensive patterns, support requests, and exports. Together they reveal which virtual workloads should be optimized, restricted, materialized, or retired.

同时观测引擎与业务行为。基础设施指标包括协调器与工作节点容量、队列、CPU、内存、溢出、交换、连接池、网络、重试、源耗时与缓存；数据指标包括行数、空值与唯一性趋势、映射失败、新鲜度、核对、策略拒绝和结果大小分布；使用者指标包括采用、放弃查询、获得答案时间、重复昂贵模式、支持请求与导出。它们共同揭示哪些虚拟负载应优化、限制、物化或退出。

Define an explicit materialization threshold. Examples include repeated scans above a remote-byte budget, p95 latency beyond the service objective, source CPU above an agreed limit, high-value history not retained by the source, or a recurring join that dominates cost. The threshold is a policy, not an admission of failure. A healthy federation routes each workload to the least risky execution pattern as evidence changes.

定义明确的物化阈值。例如：重复扫描超过远程字节预算、p95延迟超出服务目标、源CPU超过约定限制、源不保留高价值历史，或某个重复连接占据主要成本。这个阈值是一项策略，而不是承认失败。健康的联邦会随着证据变化，把每个负载路由到风险最低的执行模式。

Capacity planning should model the multiplication created by consumers. One BI page can issue several queries; scheduled refresh, retries, parameter exploration, and concurrent viewers can turn one design query into hundreds of remote statements. Forecast peak sessions, statements per interaction, remote fragments per statement, average and tail rows returned, and the source concurrency each connector opens. Test admission control, queue priority, per-tenant limits, and backpressure before broad access. Reserve capacity for production traffic at every source, and make the federation shed analytical load predictably when that reserve is threatened. A graceful rejection with an actionable message is safer than allowing queues and timeouts to cascade through unrelated systems.

容量规划必须考虑使用者带来的放大效应。一个BI页面可能发出多条查询；定时刷新、重试、参数探索和并发访问会把一条设计查询放大为数百条远程语句。应预测峰值会话、每次交互的语句数、每条语句的远程片段数、平均与尾部返回行数，以及每个连接器打开的源端并发。广泛开放前，要测试准入控制、队列优先级、租户限制与背压。每个来源都应为生产流量保留容量，并在容量受到威胁时让联邦以可预测方式削减分析负载。带有可操作说明的受控拒绝，比让队列和超时跨无关系统级联更安全。

Incident response needs a federation-specific failure taxonomy. Distinguish authentication and authorization failures, catalog or schema drift, unsupported pushdown, remote syntax and type errors, source overload, connector pool exhaustion, network interruption, worker memory pressure, spill failure, cancellation delay, and inconsistent or incomplete results. Route alerts to the team able to act, include the query fingerprint and affected sources, and suppress duplicate symptoms without hiding scope. Run exercises in which one source slows, returns a changed type, or becomes unavailable while other sources remain healthy. Verify that cancellation reaches remote work, temporary data expires, consumers receive an accurate status, and recovery does not unleash a retry storm. Afterward, update connector tests, source contracts, guardrails, and runbooks—not only the immediate query.

事故响应需要联邦专用的故障分类。应区分身份验证与授权失败、目录或模式漂移、不支持的下推、远程语法与类型错误、源过载、连接池耗尽、网络中断、工作节点内存压力、溢出失败、取消延迟，以及不一致或不完整结果。告警要发送给能够采取行动的团队，并包含查询指纹和受影响来源；可以抑制重复症状，但不能隐藏影响范围。应演练某个来源变慢、返回变化类型或不可用，而其他来源仍健康的情况；验证取消是否到达远程工作、临时数据是否过期、使用者是否收到准确状态，以及恢复是否会触发重试风暴。复盘后要更新连接器测试、源契约、护栏与运行手册，而不只是修改当前查询。

13. Federated Query vs. Data Mesh13. 联邦查询 vs 数据网格（Data Mesh）

Data mesh is an organizational and architectural approach built around domain ownership, data as a product, self-service platform capability, and federated computational governance. It does not prescribe one storage engine or require query federation. Domains may publish materialized tables, streams, APIs, models, or virtual views. Federation is one technical capability that can make distributed domain products discoverable and queryable through a common interface.

数据网格是一种组织与架构方法，围绕领域所有权、数据即产品、自助平台能力与联邦计算治理展开。它不指定某一种存储引擎，也不要求必须使用查询联邦。领域可以发布物化表、流、API、模型或虚拟视图；联邦则是一种技术能力，可让分布式领域产品通过公共接口被发现和查询。

The terms also use “federated” differently. Query federation distributes execution across data sources. Federated governance distributes decision authority across domains while standardizing policies that require interoperability. The data mesh principles described by Zhamak Dehghani frame federated computational governance around local domain context plus globally interoperable standards. A centralized query engine can support that model, but it can also undermine it if one platform team owns every semantic decision and delivery queue.

两者对“联邦”的使用也不同。查询联邦把执行分布到数据源；联邦治理则把决策权分布到各领域，同时标准化互操作所需政策。Zhamak Dehghani提出的数据网格原则把联邦计算治理描述为本地领域上下文与全局互操作标准的结合。集中查询引擎可以支持这一模式，但如果一个平台团队控制所有语义决策与交付队列，也可能破坏它。

Use federation in a data mesh when domain products remain distributed, consumers need cross-domain discovery or analysis, and the platform can enforce shared identity, metadata, policy, lineage, and workload controls without erasing domain accountability. Do not let virtual views compensate for missing ownership, undocumented definitions, weak quality, or unstable interfaces. Federation connects products; it does not create product discipline by itself.

当领域产品保持分布、使用者需要跨领域发现或分析，并且平台能够执行共享身份、元数据、策略、血缘与负载控制，同时不抹去领域责任时，联邦适合用于数据网格。不要让虚拟视图掩盖所有权缺失、定义未记录、质量薄弱或接口不稳定。联邦可以连接产品，但不会自行建立产品纪律。

14. Summary and Next Steps14. 总结与下一步

Federated queries combine data from autonomous sources through one logical request. Data virtualization adds the managed layer of virtual schemas, semantics, policy, discovery, lineage, reuse, and service operations. The architecture is valuable when data should remain in place, freshness and source autonomy matter, or building a new copy is disproportionate. It is unsuitable when repeated heavy scans, durable history, strict isolation, predictable low latency, or source availability require materialization.

联邦查询通过一次逻辑请求组合自治来源的数据；数据虚拟化进一步增加虚拟模式、语义、策略、发现、血缘、复用与服务运营层。当数据应保留原处、新鲜度与源自治很重要，或为问题建立新副本成本不成比例时，该架构具有价值。当负载需要重复重扫描、持久历史、严格隔离、可预测低延迟或不依赖源可用性时，则应采用物化。

A dependable implementation starts with a bounded use case, explicit owners and contracts, accurate metadata, tested connectors, safe identity, source guardrails, evidence-based optimization, complete observability, and a materialization policy. Validate global and source plans; measure transferred data and source impact; test failure and cancellation; and protect intermediate results. Expand only when operational evidence supports it.

可靠实施应从边界清晰的用例、明确责任与契约、准确元数据、经过测试的连接器、安全身份、源端护栏、证据驱动优化、完整可观测性与物化策略开始。需要验证全局和源端计划，测量传输数据与源影响，测试失败与取消，并保护中间结果。只有运维证据充分时才扩大范围。

The most durable design treats federation as a routing choice inside a broader data architecture. Some questions should run live at the source, some should use a cache or aggregate, and some should be served from replicated, historical data products. Keep the consumer contract stable where possible, but let measured workload behavior determine the physical path. This avoids both extremes: copying every dataset by default and forcing every workload through remote query execution.

最持久的设计会把联邦视为更广泛数据架构中的一种路由选择。有些问题应在源端实时运行，有些应使用缓存或聚合，还有一些应由复制并保留历史的数据产品提供。尽可能保持使用者契约稳定，但让实测负载行为决定物理路径，从而避免两个极端：默认复制每个数据集，或强迫每种负载都通过远程查询执行。

data virtualization data federation federated search distributed database cost-based optimizer query optimization database virtualization database federation federated database federated database system federated query federated analytics federated access federated data governance federation architecture lakehouse federation distributed SQL predicate pushdown data virtualization software data virtualization tools data virtualization platform

数据虚拟化 数据联邦 联邦搜索 分布式数据库 基于成本的优化器 查询优化 数据库虚拟化 数据库联邦 联邦数据库 联邦数据库系统 联邦查询 联邦分析 联邦访问 联邦数据治理 联邦架构 湖仓联邦 分布式SQL 谓词下推 数据虚拟化软件 数据虚拟化工具 数据虚拟化平台

Frequently Asked Questions常见问题

Sources and Evidence Notes资料来源与证据说明

Primary technical references used for this guide include the Trino pushdown documentation, Trino cost-based optimization documentation, Amazon Athena Federated Query documentation, and Databricks Lakehouse Federation documentation. The data mesh discussion references Data Mesh Principles and Logical Architecture. Product limits change, so implementation decisions must be verified against current official documentation and observed plans.

本指南的一手技术资料包括Trino下推文档、Trino基于成本优化文档、Amazon Athena联邦查询文档与Databricks Lakehouse Federation文档。数据网格部分引用数据网格原则与逻辑架构。产品限制会变化，因此实施决策必须依据当前官方文档与实测计划重新验证。