On this page本页目录
What Is Schema Mapping?什么是Schema映射?
Place this specific workflow in context with the complete data integration guide, which connects the definitions, alternatives, validation steps, and related implementation guides.
可通过完整的数据集成指南理解本专题在整体流程中的位置;该指南串联了定义、替代方案、验证步骤与相关实施文章。
Schema mapping is an explicit, directional specification of how structures in a source schema correspond to structures in a target schema. It covers objects or tables, fields or paths, types, keys, relationships, cardinality, nullability, constraints, transformations, and exception behavior so valid source instances can be queried or converted into valid target instances.
Schema映射是一份明确且有方向的规范,用于说明源Schema中的结构如何对应到目标Schema。它涵盖对象或表、字段或路径、类型、键、关系、基数、可空性、约束、转换和异常行为,使有效源实例能够被查询或转换成有效目标实例。
A database schema mapping may be a reviewable crosswalk, a declarative mapping file, a set of source-to-target dependencies, or executable rules in a migration or integration product. The representation differs by system, but the engineering questions stay consistent: what is included, how structures correspond, what must be transformed, what information can be lost, and how the result will be verified.
数据库Schema映射可以表现为可评审的对照表、声明式映射文件、源到目标依赖集合,或迁移与集成产品中的可执行规则。不同系统的表示方式不同,但工程问题一致:包含什么、结构如何对应、哪些内容必须转换、哪些信息可能丢失,以及如何验证结果。
The term is sometimes used loosely for field mapping or schema matching. This guide uses the stricter data-integration meaning: reviewed correspondences plus the rules needed to make the target structure valid. It does not mean an entity-relationship diagram, search-engine structured-data markup, or merely choosing an ORM class-to-table annotation.
该词有时被宽泛地用于字段映射或Schema匹配。本指南采用更严格的数据集成含义:经过评审的对应关系,加上让目标结构有效所需的规则。它不等同于实体关系图、搜索引擎结构化数据标记,也不只是选择ORM的类到表注解。
What a Source-to-Target Schema Mapping Must Record源到目标Schema映射必须记录什么
| Area区域 | Record应记录内容 | Why it matters重要原因 |
|---|---|---|
| Identity身份 | Mapping ID, direction, source and target version, status, owner映射ID、方向、源目标版本、状态、负责人 | Prevents an approved rule from being applied to the wrong release防止把已批准规则应用到错误版本 |
| Structural path结构路径 | Source object/path and target object/path, including nesting源对象/路径与目标对象/路径,包括嵌套 | Makes split, merge, flatten, and expansion explicit明确拆分、合并、扁平化和展开 |
| Contract契约 | Type, format, length, precision, enum, nullability, default类型、格式、长度、精度、枚举、可空性、默认值 | Exposes incompatibility before runtime在运行前暴露不兼容 |
| Keys and relations键与关系 | Primary, natural, surrogate and foreign keys; cardinality and ordering主键、自然键、代理键、外键、基数与顺序 | Preserves identity and referential integrity保留身份与引用完整性 |
| Rule规则 | Expression, lookup, filter, aggregation, precedence, reject condition表达式、查找、过滤、聚合、优先级、拒绝条件 | Turns correspondence into repeatable behavior把对应关系转成可重复行为 |
| Proof证据 | Fixtures, expected outputs, assertions, reconciliation and exceptions样例、预期输出、断言、对账与异常 | Makes approval testable rather than subjective让批准可测试而非主观判断 |
A diagram is useful for orientation, but it is not the complete deliverable. The reviewable artifact should identify every material target requirement, including intentionally unmapped elements. “No source,” “not in scope,” “derived,” and “rejected” are different decisions and should not be represented by the same blank cell.
图表有助于理解,但不是完整交付物。可评审产物应标识每个重要目标要求,包括有意不映射的元素。“无来源”“不在范围”“派生”和“拒绝”是不同决定,不应都用同一个空白单元格表示。
How to Do Schema Mapping Step by Step如何逐步完成Schema映射
- Fix scope and versions. Name the exact source and target schema releases, direction, objects, data range, deployment mode, owners, and acceptance gates.固定范围与版本。明确源目标Schema版本、方向、对象、数据区间、部署方式、负责人和验收门槛。
- Profile structure and examples. Extract metadata, constraints, representative instances, edge cases, frequency, and value patterns without treating inference as a declared contract.剖析结构与样例。提取元数据、约束、代表性实例、边界情况、频率和值模式,但不要把推断当成已声明契约。
- Match candidate elements. Propose correspondences using names, definitions, types, paths, keys, examples, and domain evidence. Flag ambiguity instead of forcing a match.匹配候选元素。依据名称、定义、类型、路径、键、样例和领域证据提出对应关系;存在歧义时应标记,而不是强行匹配。
- Design target-valid rules. Specify direction, cardinality, transformations, join and split logic, defaults, constraints, lossiness, and exception destinations.设计满足目标契约的规则。规定方向、基数、转换、连接与拆分逻辑、默认值、约束、信息损失和异常去向。
- Review meaning and privacy. Confirm units, identifiers, consent, sensitive fields, retention, masking, and authorization with domain and security owners.评审含义与隐私。与业务域和安全负责人确认单位、标识符、授权、敏感字段、保留期、脱敏与访问权限。
- Implement against a versioned artifact. Translate the approved specification into the actual mapping language or code; keep generated and hand-authored logic traceable to mapping IDs.依据版本化产物实施。把已批准规范转为实际映射语言或代码,并让生成和手写逻辑都能追溯到映射ID。
- Validate in layers. Run schema, fixture, constraint, key, reconciliation, exception, security, and regression tests before controlled release.分层验证。在受控发布前运行Schema、样例、约束、键、对账、异常、安全和回归测试。
- Monitor drift and govern change. Detect either-side changes, assess impact, version the map, rerun tests, and require approval before incompatible behavior reaches production.监测漂移并治理变更。检测任一端变化、评估影响、版本化映射、重跑测试,并在不兼容行为进入生产前重新批准。
Schema Mapping Example: Nested Orders to Relational TablesSchema映射示例:嵌套订单转关系表
Hypothetical example: the names and rules below illustrate design decisions; they do not describe an InfiniSynapse customer, benchmark, or deployed integration.
假设示例:以下名称和规则仅用于说明设计决定,不代表InfiniSynapse客户、基准测试或已部署集成。
Assume a source API returns one order object with an order identifier, creation timestamp, an embedded customer object, and an items array. The target uses three relational tables: ORDER, CUSTOMER, and ORDER_LINE. The map must preserve order identity, customer reuse, one-to-many item cardinality, line order, money precision, and invalid-item exceptions.
假设源API返回一个订单对象,其中包含订单标识、创建时间、嵌套客户对象和商品数组。目标使用ORDER、CUSTOMER和ORDER_LINE三张关系表。映射必须保留订单身份、客户复用、一对多商品基数、行顺序、金额精度以及无效商品异常。
| Source path源路径 | Target目标 | Rule and test规则与测试 |
|---|---|---|
| order.id | ORDER.order_id | Required string; preserve exactly; reject blank or duplicate IDs.必填字符串;原样保留;空白或重复ID进入拒绝队列。 |
| order.created_at | ORDER.created_utc | Parse declared timestamp format, convert to UTC, retain source offset in audit evidence.按声明格式解析时间,转为UTC,并在审计证据中保留源偏移。 |
| order.customer | CUSTOMER | Upsert by approved customer key; do not infer identity from name alone.按批准客户键更新插入;不得仅凭姓名推断身份。 |
| order.items[*] | ORDER_LINE | Emit one row per element with order_id foreign key and deterministic line_number.每个数组元素生成一行,带order_id外键和确定性line_number。 |
| items[*].unit_price | ORDER_LINE.unit_price | Parse decimal with declared currency; reject unsupported currency or excess precision.按声明币种解析十进制;不支持币种或精度超限时拒绝。 |
| missing items | ORDER and ORDER_LINE | Distinguish absent, empty, and malformed arrays; acceptance policy decides whether an empty order is valid.区分缺失、空数组和畸形数组;由验收政策决定空订单是否有效。 |
The example is intentionally structural. Business rules still decide whether guest customers are allowed, whether zero-line orders are meaningful, and how corrections are replayed. A mapping that passes type checks but silently creates duplicate customers or loses item order is not valid.
该示例刻意聚焦结构。访客客户是否允许、零行订单是否有意义、修正如何重放,仍由业务规则决定。只通过类型检查,却静默创建重复客户或丢失商品顺序的映射,并不有效。
Schema Mapping vs Matching, Data Mapping, and MigrationSchema映射与匹配、数据映射及迁移的区别
| Practice工作 | Primary question主要问题 | Typical output典型产物 |
|---|---|---|
| Schema matchingSchema匹配 | Which elements may correspond?哪些元素可能对应? | Candidate matches, evidence, confidence, unresolved ambiguity候选匹配、证据、置信度与未解歧义 |
| Schema mappingSchema映射 | How do valid structures translate directionally?有效结构如何有方向地翻译? | Structural correspondences, constraints, transformations and tests结构对应、约束、转换和测试 |
| Data mapping数据映射 | Which source data elements populate which targets?哪些源数据元素填充哪些目标? | Field and value rules; often broader implementation detail字段和值规则;通常是更广的实施细节 |
| Semantic mapping语义映射 | How do concepts align in meaning and context?概念在含义与上下文上如何对齐? | Typed conceptual relationships with evidence and scope带证据和范围的类型化概念关系 |
| Data modeling数据建模 | How should one domain be structured?一个领域应该如何组织结构? | Conceptual, logical, or physical model and constraints概念、逻辑或物理模型及约束 |
| Schema migrationSchema迁移 | How is a schema and its data changed over time?Schema及其数据如何随时间变更? | Versioned DDL or migration steps, deployment and rollback版本化DDL或迁移步骤、部署与回滚 |
The practices cooperate. Matching can propose correspondences; semantic review confirms meaning; schema mapping specifies structural translation; data mapping fills in value-level rules; migration or integration code executes the approved design. Keeping the stages traceable makes disagreement visible and prevents an automated suggestion from becoming an unreviewed production rule.
这些工作可以协作:匹配提出候选对应,语义评审确认含义,Schema映射规定结构翻译,数据映射补充值级规则,迁移或集成代码执行已批准设计。保持阶段可追溯能让分歧显现,并防止自动建议直接变成未经评审的生产规则。
How to Validate Schema Mapping Results如何验证Schema映射结果
- Contract validation: parse both schemas; check required objects, paths, types, formats, keys, uniqueness, foreign keys, cardinality, nullability, enums, lengths, and precision.契约验证:解析两端Schema;检查必需对象、路径、类型、格式、键、唯一性、外键、基数、可空性、枚举、长度和精度。
- Fixture validation: test normal, boundary, missing, empty, null, duplicate, malformed, out-of-order, unsupported-code, and high-cardinality inputs with exact expected outputs.样例验证:用精确预期输出测试正常、边界、缺失、空、null、重复、畸形、乱序、不支持代码和高基数输入。
- Reconciliation: compare source and target counts at the declared grain, distinct keys, orphan rates, null distributions, rejected records, aggregates, and traceable samples.对账:按声明粒度比较源目标数量、不同键、孤儿率、空值分布、拒绝记录、汇总和可追溯样本。
- Operational proof: verify deterministic reruns, idempotency where required, exception routing, security controls, rollback evidence, observability, and expected query behavior.运行证明:验证确定性重跑、必要时的幂等性、异常路由、安全控制、回滚证据、可观测性和预期查询行为。
- Regression: pin known fixtures and failures to the mapping version; rerun them after either schema, rule, lookup, or runtime changes.回归:把已知样例和失败绑定到映射版本;任一Schema、规则、查找表或运行环境变化后重新执行。
Counts alone are weak evidence. A total can match even when keys are swapped, relationships are orphaned, or values are truncated. Combine structural assertions, record-level traceability, business reconciliation, and exception review.
仅比较数量属于弱证据。即使键被交换、关系变成孤儿或值被截断,总数仍可能相同。因此必须结合结构断言、记录级追溯、业务对账和异常审查。
Handle Schema Drift and Evolution Deliberately有意识地处理Schema漂移与演进
Schema drift is any observed structural change that can affect the mapping: a field appears or disappears, a type widens or narrows, an enum gains a value, nesting changes, a required property becomes optional, or a key and relationship change. Some changes are backward compatible for one consumer but breaking for another. Compatibility therefore belongs to a specific mapping direction, consumer contract, and deployment mode.
Schema漂移是任何可能影响映射的结构变化:字段新增或消失、类型变宽或变窄、枚举增加取值、嵌套改变、必填属性变为可选,或键与关系变化。某些变化对一个消费者向后兼容,却可能破坏另一个消费者。因此兼容性必须绑定到具体映射方向、消费者契约和部署方式。
Store source version, target version, mapping version, transformation dependency versions, fixtures, approvals, and effective dates together. Compare proposed schemas before deployment, classify impact, require review for breaking or lossy changes, and run the complete regression suite. Unknown fields should follow an explicit policy—preserve, quarantine, ignore with telemetry, or reject—not disappear silently.
应把源版本、目标版本、映射版本、转换依赖版本、样例、批准记录和生效日期一起保存。部署前比较候选Schema,分类影响,对破坏性或有损变化重新评审,并运行完整回归套件。未知字段必须遵循明确政策:保留、隔离、带遥测忽略或拒绝,而不能静默消失。
Use InfiniSynapse to Review Approved Mapping Evidence使用InfiniSynapse评审已批准的映射证据
After defining the mapping question and obtaining approved access, InfiniSynapse can help teams analyze connected databases, files, and documents with natural-language questions. Prepare versioned source and target definitions, a draft mapping table, privacy-safe fixtures, exception samples, validation results, and focused review questions such as “which required targets lack an approved source?” or “which sampled records violate the declared one-to-many relationship?”
在明确映射问题并获得批准访问后,InfiniSynapse可帮助团队用自然语言分析已连接的数据库、文件和文档。请准备版本化的源目标定义、映射草案、隐私安全样例、异常样本、验证结果和聚焦问题,例如“哪些必填目标没有已批准来源?”或“哪些样本记录违反声明的一对多关系?”
Use read-only access where possible and exclude sensitive values unless explicitly authorized. InfiniSynapse should not be described as automatically discovering authoritative schemas, generating or approving mappings, converting DDL, migrating production data, enforcing constraints, deploying transformations, or managing schema versions. Those remain responsibilities of the source system, target platform, integration runtime, engineering, security, and governance teams.
应尽可能使用只读访问;除非明确获准,否则排除敏感值。不得把InfiniSynapse描述为会自动发现权威Schema、生成或批准映射、转换DDL、迁移生产数据、强制约束、部署转换或管理Schema版本。这些仍属于源系统、目标平台、集成运行环境、工程、安全与治理团队的责任。
Bring versioned schemas, a draft source-to-target map, privacy-safe fixtures, expected results, exception samples, and specific validation questions. Use InfiniSynapse to explore approved connected evidence; keep implementation, approval, and deployment in governed systems.
请准备版本化Schema、源到目标映射草案、隐私安全样例、预期结果、异常样本和具体验证问题。使用InfiniSynapse探索已批准的连接证据;实施、批准与部署仍应留在受治理系统中。
Analyze approved connected sources分析批准的已连接来源Schema Mapping FAQSchema映射常见问题
What is schema mapping?
什么是Schema映射?
Schema mapping is an explicit specification of how structures in a source schema correspond to structures in a target schema, including objects or tables, fields or paths, data types, keys, relationships, cardinality, constraints, transformations, and exception behavior.
Schema映射是一份明确规范,说明源Schema中的结构如何对应到目标Schema,包括对象或表、字段或路径、数据类型、键、关系、基数、约束、转换以及异常行为。
What is the difference between schema matching and schema mapping?
Schema匹配与Schema映射有什么区别?
Schema matching discovers or proposes which schema elements correspond. Schema mapping turns reviewed correspondences into directional rules that explain how valid source instances are translated, queried, or materialized in the target structure.
Schema匹配用于发现或提出哪些Schema元素可能对应;Schema映射则把经过评审的对应关系转成有方向的规则,说明有效源实例如何被翻译、查询或实体化为目标结构。
What should a schema mapping document include?
Schema映射文档应该包含什么?
Record versioned source and target schemas, object and field paths, types and formats, keys, relationship cardinality, nullability, defaults, transformations, filters, lossiness, assumptions, ownership, validation fixtures, expected exceptions, and approval status.
应记录版本化的源与目标Schema、对象和字段路径、类型与格式、键、关系基数、可空性、默认值、转换、过滤、信息损失、假设、负责人、验证样例、预期异常和批准状态。
How do you validate a schema mapping?
如何验证Schema映射?
Validate schema syntax, required paths, type and format conversions, keys, referential integrity, cardinality, null and default behavior, representative and adversarial fixtures, rejected records, aggregate reconciliation, reversibility where required, and regression results after either schema changes.
应验证Schema语法、必需路径、类型和格式转换、键、引用完整性、基数、空值和默认值行为、代表性与刁钻测试样例、拒绝记录、汇总对账、必要时的可逆性,以及任一Schema变化后的回归结果。
Authoritative Sources and Next Steps权威来源与下一步
Schema mapping has no single universal execution format. Use the specification and official documentation for the mechanism you actually deploy. W3C R2RML is a Recommendation for customized relational-to-RDF mappings; MongoDB documents explicit relational-to-document schema decisions; AWS DMS documents selection and transformation rules for migration tasks; and JSON Schema defines structural validation vocabulary. These are concrete mechanisms and examples, not interchangeable standards for every mapping project.
Schema映射没有单一通用执行格式。应使用实际部署机制对应的规范和官方文档。W3C R2RML是定制关系数据到RDF映射的推荐标准;MongoDB记录了关系结构到文档结构的明确决定;AWS DMS记录迁移任务的选择与转换规则;JSON Schema则定义结构验证词汇。这些是具体机制与示例,不是每个映射项目都可互换的标准。
- W3C R2RML mapping language RecommendationW3C R2RML映射语言推荐标准
- MongoDB relational-to-document schema mapping documentationMongoDB关系到文档Schema映射文档
- AWS DMS table selection and transformation rulesAWS DMS表选择与转换规则
- JSON Schema Core specificationJSON Schema核心规范
- InfiniSynapse guide to identifying data sourcesInfiniSynapse数据源识别指南
- InfiniSynapse guide to data quality checksInfiniSynapse数据质量检查指南
- InfiniSynapse guide to data lineageInfiniSynapse数据血缘指南
