Back to the index · Previous: Deployment: Dev to QA to Prod · Reference: Git integration
This chapter is a practical test, run on 2026-11-01. One business model was promoted from Dev to QA to Production with Git, and another with the built-in Deployment screen. Every step is done by a person: a developer commits, a reviewer merges the pull request, a developer updates the next environment. Nothing runs by itself.
What was promoted#
| Path | Business model | What it needs (found by DataSimple) |
|---|---|---|
| Git | Retail sales (star schema gold.retail_fact_sales with its dimensions, and the OBT gold.retail_obt_sales) |
8 objects: notebooks RTL 01, RTL 02, RTL 03 (Silver), RTL 04 (Gold star schema and two OBTs), RTL 06 (data quality); pipelines RETAIL Daily Lakehouse Build and Schedule: retail_daily_lakehouse_build; metric RETAIL Net sales |
| Deployment | Manufacturing production (star schema gold.mfg_fact_production with date, product, plant and line dimensions) |
6 objects: notebooks MFG 01, MFG 02 (Silver), MFG 04 (Gold star schema and OBT); metrics MFG Net revenue, mfg_net_unit_price, mfg_scrap_rate_pct |
Not promoted: the other 81 notebooks in Dev and everything else not needed by these two models.
Manufacturing production has no pipeline in its list: the MFG Daily Lakehouse Build pipeline also builds sensor readings (MFG 03), which this star schema does not use. DataSimple names it under Not included instead of taking the extra notebook.
The environments#
| Environment | Branch in datasimple/ds_git_deploy |
Starting point |
|---|---|---|
| Dev | Dev |
the full workspace (89 notebooks, 17 pipelines …) |
| QA | QA |
empty workspace |
| Prod | Prod |
empty workspace |
In this test three tenants of one installation stood in for the three environments. In real use each environment is its own installation; the steps are the same.
Each environment was connected to its branch once, under Administration → Git integration (see the administration chapter of the developer handbook). QA and Prod were also registered as Deployment targets (Deployment → Connect the next environment).
Path 1: Git#
Step 1. Dev: commit one model#
- Deployment → Git panel. It shows Dev is connected to branch Dev.
- What to commit → One business model and its lineage → Retail sales.
- The panel lists the 8 objects above, from Bronze to Gold (28 tables), and Only these go to Git; nothing else on the branch is added or removed.
- Check: all 8 are new.
- Message Retail sales star schema and its lineage (Bronze to Gold) → Commit this model.
Result: one commit on Dev with 13 files:
datasimple/notebooks/RTL 01 · Silver current state (SCD 1).notebook/notebook.yaml, content.py
datasimple/notebooks/RTL 02 · Silver customers and products (SCD 1 + SCD 2).notebook/…
datasimple/notebooks/RTL 03 · Silver web events (JSON flattened).notebook/…
datasimple/notebooks/RTL 04 · Gold star schema and two OBTs.notebook/…
datasimple/notebooks/RTL 06 · Data quality checks.notebook/…
datasimple/pipelines/RETAIL Daily Lakehouse Build.pipeline.yaml
datasimple/pipelines/Schedule_ retail_daily_lakehouse_build.pipeline.yaml
datasimple/semantics/RETAIL Net sales.metric.yaml
Check again: Git already has this model as it is here.
Step 2. GitHub: pull request Dev → QA#
- In GitHub: Pull requests → New pull request, base
QA← compareDev. - Review the 13 files (notebook code is plain text in
content.py, so the diff reads like code). - Create pull request, then Merge pull request.
Step 3. QA: update from Git#
- Sign in to QA → Deployment → Git panel (QA is connected to branch QA).
- Check: 8 objects in Git, not here.
- Update from Git.
Result: 8 created, 0 failed.
| Check in QA | Result |
|---|---|
| Notebooks | the 5 RTL notebooks, nothing else |
| Pipelines | both retail pipelines, switched off (a new pipeline always arrives off in a target) |
| Pipeline steps | each of the 5 steps points at QA's own copy of its notebook |
| Metric | RETAIL Net sales |
| Git panel | This workspace and the branch are the same. |
| Deployment history | Updated from Git, by the QA developer, with Roll back |
Step 4. GitHub: pull request QA → Prod, then Prod: update from Git#
Same as steps 2 and 3, with base Prod ← compare QA, then Update from Git in Prod.
Result: the same 8 objects created in Prod, 0 failed; Prod and its branch are the same.
Path 2: Deployment screen#
Step 1. Dev → QA#
- Sign in to Dev → Deployment → card 3, target QA → Compare.
- Tick the 6 Manufacturing production objects (New) → Deploy 6.
- The check before deploying lists lake tables QA does not have yet (
bronze.mfg_*,silver.mfg_*,gold.mfg_*). That is information: Deployment moves definitions; the tables are built when the pipeline or notebooks run in QA. Leave Create on target off unless you want empty tables created now.
Result: 6 created, 0 failed.
Step 2. QA → Prod#
Sign in to QA → Deployment → target Prod → Compare → tick the same 6 → Deploy 6.
Result: 6 created, 0 failed.
Both paths at a glance#
| Git | Deployment | |
|---|---|---|
| Who starts each hop | a developer (Commit, Update from Git) and a reviewer (pull request merge) | a developer (Deploy) |
| Choosing what moves | What to commit → one business model and its lineage, or every change | tick objects after Compare; a pipeline pulls in its notebooks |
| Review | pull request in GitHub | approval request, when the target environment requires one |
| History | Git history, and Deployment history in each environment | Deployment history in both environments |
| Roll back | Deployment history → Roll back in the environment | the same |
| Data, passwords, Data Source addresses | never moved | never moved |
The two can be mixed. After this test QA's Git panel lists the three Manufacturing production notebooks and three metrics as new: they came by Deployment, not from the branch. Commit them from QA only if the QA branch should carry them; otherwise leave them.
Found and fixed during this test#
| What | Fix |
|---|---|
| A promoted pipeline lost the link to its notebooks (ids compared as text against numbers) | ids compared as numbers; every notebook step keeps its notebook |
| Deployment refused a promotion because Data Sources the selected notebooks do not use had no match in the target | only Data Sources the selected objects refer to are checked |
| A storage target's access key id and bucket name were written to its Git file | storage targets never write the key id, bucket, account or path |
| Commit all would have sent all 89 notebooks to QA | What to commit → one business model and its lineage |
Repeat this test#
- In Dev, pick a model in What to commit, check the list, Commit this model.
- Pull request Dev → QA in GitHub; review; merge.
- QA: Check, Update from Git; look at Notebooks, Pipelines (off), Deployment history.
- Pull request QA → Prod; merge; Prod: Update from Git.
- For the Deployment path: Dev → Compare → tick → Deploy; then from QA to Prod.
Next: 13. Reference