grok-bot:u9YedcQf8xZY-wfQ_BBc0

Damwright

湖仓架构师:把数据需求按你的分层方式(如 medallion)依次做成概念、逻辑和物理模型

简介

湖仓架构师:把数据需求按你的分层方式(如 medallion)依次做成概念、逻辑和物理模型

包含内容

  • skill allocate-medallion-responsibilities Assign datasets and transformations to Bronze, Silver, and Gold and select a justified modeling approach for each. Use when planning a medallion subject area or resolving where validation, canonical entities, history, dimensions, facts, and aggregates belong.
  • skill assess-platform-and-tool-selection Run a defensible platform or tool selection: requirements, a weighted scoring matrix, open table format choice (Delta, Iceberg, Hudi), build vs buy, a proof-of-concept plan with pass/fail criteria, lock-in and exit cost, and a TCO structure. Use when choosing a lakehouse platform, catalog, table format, ingestion/orchestration/quality/BI tool, or when someone proposes replacing one.
  • skill build-conceptual-data-model Build a business-facing conceptual model with clear entity definitions and relationship cardinalities. Use when defining subject-area boundaries, discussing how business concepts relate, or producing an ERD before logical table design.
  • skill design-bronze-record-contract Design the record shape and replay contract for a Bronze dataset. Use when landing JSON, events, snapshots, or CDC into Databricks and deciding payload preservation, delivery identity, provenance, drift, and deletion semantics.
  • skill design-data-quality-and-observability Design data quality rules and monitoring for lakehouse products: quality dimensions, expectation tiers (warn, quarantine, fail), freshness, volume, schema and distribution monitors, SLOs, incident severity and runbooks. Use when building or hardening a pipeline, writing a data contract, after a data incident, or when consumers ask "can I trust this table".
  • skill design-databricks-physical-model Translate a logical lakehouse model into Databricks Delta tables, views, types, constraints, and workload-appropriate layouts. Use for physical ERDs, draft DDL, surrogate implementations, Unity Catalog placement, and enforcement decisions.
  • skill design-dimensions-and-history Design a dimension and choose history behavior per attribute, including mixed Type 1 and Type 2 changes. Use for conformed dimensions, surrogate keys, current versus historical reporting, late changes, and dimension-version resolution in Databricks.
  • skill design-engineering-patterns Design and publish reusable lakehouse patterns and standards for developers and data engineers to follow, such as naming conventions, layer templates, ingestion, SCD, merge, quality-check, and folder/repo patterns. Use when the team needs a standard, when the same problem is solved inconsistently, or when onboarding engineers.
  • skill design-gold-facts-and-measures Design Gold fact tables with explicit grain, dimensions, measures, and aggregation rules. Use when modeling analytical events, periodic snapshots, accumulating snapshots, factless relationships, KPI inputs, or aggregates.
  • skill design-logical-entities-and-keys Design implementation-independent entities, attributes, domains, and identifiers. Use when resolving logical grain, candidate keys, normalization, reference codes, supertypes, or identity namespaces before Databricks physical design.
  • skill design-semantic-layer-and-metrics Design a governed semantic layer so every tool computes metrics the same way: metric definition specs (formula, grain, filters, owner), certified vs exploratory metrics, metric views or semantic models, BI integration, and change control. Use when two reports disagree on a number, when a domain publishes KPIs, when adopting a semantic/metrics tool, or when AI/BI assistants need trusted definitions.
  • skill design-silver-canonical-model Model validated Silver entities, events, and nested child collections from Bronze records. Use when choosing canonical grain, typed mappings, deduplication, update ordering, schema-drift handling, or current-state versus history structures.
  • skill design-streaming-and-latency-architecture Decide between batch, micro-batch, and continuous streaming based on how fresh data really needs to be, then design the stream correctly: watermarks, exactly-once and idempotency, late data, state, cost trade-offs, and failure recovery. Use when someone asks for "real-time", when designing CDC or event ingestion, or when a streaming job is costly or unreliable.
  • skill document-model-contracts-and-lineage Produce an implementation-ready model dictionary, data contracts, source-to-target mappings, and requirement lineage. Use when handing a model to engineers, documenting a medallion product, or closing metadata gaps in an existing design.
  • skill explain-tradeoffs-to-business Use when a lakehouse design choice or trade-off needs business sign-off, funding, or explaining to leaders: turns it into a jargon-free one-page decision brief with simple diagrams showing reasoning, cost, time, risk, and impact.
  • skill lakehouse-architect-getting-started Use as the first conversation with a new owner of the lakehouse architect bot.
  • skill maintain-model-and-assess-change Assess a proposed model change and keep conceptual, logical, physical, and deployed designs aligned. Use for schema evolution, drift, changed business definitions, new source versions, dimension-history changes, or downstream impact analysis.
  • skill model-bridges-and-hierarchies Model many-to-many, ternary, recursive, and changing relationships without double-counting. Use for customer-to-institution offerings, agent assignments, multivalued contact reasons, organization hierarchies, or fact allocation through bridges.
  • skill model-calendar-and-local-time Design event timestamps, calendar and time dimensions, and institution-local reporting dates. Use when facts need UTC and local views, multiple time zones, fiscal calendars, daylight-saving handling, or time-zone assignment history.
  • skill plan-migration-from-legacy-warehouse Plan and run a migration from a legacy warehouse or ETL stack (Teradata, Oracle, SQL Server, Netezza, Hadoop, older Snowflake/Redshift) to the lakehouse: inventory, complexity scoring, wave planning, code translation, parallel run, reconciliation, cutover and rollback, and decommission. Use when scoping, estimating, or executing a migration, or when a wave is ready to cut over.
  • skill profile-source-and-reverse-model Profile source data and reconstruct an evidence-backed as-is model from schemas, DDL, JSON, SQL, and sample records. Use before modeling an unfamiliar source or when existing Databricks tables and documentation disagree.
  • skill publish-lakehouse-docs-to-repo Publish lakehouse design artifacts (ADRs, data contracts, model dictionaries, source-to-target mappings, naming and pattern standards, diagrams) to a GitHub repo on a new branch with a pull request. Use when a design is ready to hand to data engineers.
  • skill review-data-model-quality Review a conceptual, logical, or physical model against requirements, data evidence, and Databricks constraints. Use for design reviews, modeling PRs, grain/key validation, and identifying defects before engineering handoff.
  • skill scope-modeling-requirements Turn a lakehouse modeling request into bounded, testable data requirements. Use when starting a subject area, clarifying reporting questions, or identifying missing grain, history, identity, and ownership requirements before designing tables.
  • skill select-modeling-scheme Choose the right modeling scheme for a lakehouse product: relational/3NF, dimensional, Data Vault, Anchor, UML, ORM/FCO-IM, or NoSQL/document. Use when starting a subject area, when allocate-medallion-responsibilities needs a scheme decision, or when someone proposes a specialized method.
  • memory Memory entries · 3 项
  • plugin Plugin references · 1 项

依赖

附件

此记录没有单独发布的附件。