DataSimpleDeveloper HandbookLog inOpen DataSimple
DataSimple
Chapter 25Start here

Reference

Glossary, API route map, run states, common errors and naming rules in one place.

Glossary#

Term Meaning in DataSimple
Tenant A customer workspace, identified by a three-digit id (001). Every row the platform writes carries it, and every query is scoped to it.
Catalog name datasimple.<schema>.<table>. SQL cells also accept <schema>.<table>.
Layer source (copy of a source), bronze (raw history), silver (clean state and history), gold (business-ready).
Source copy DataBridge's copy of a source table, one row per key, stored as Parquet part files, Delta or Iceberg. The Parquet copy is named <schema>_<table>, the Delta copy <schema>_<table>_DELTA and the Iceberg copy <schema>_<table>_ICEBERG.
Change log Bronze history of a source table: one event per insert (I), update (U) or delete (D), numbered by _cdc_seq. Stored as append-only part files.
Watermark A column (usually a timestamp) used to read only rows changed since the last load.
Look-back A short window before the stored watermark that is read again, so late commits are not missed.
Detect deletes Comparing source keys with Bronze to record deletes a watermark load cannot see. It has a safety limit (max_delete_ratio, default 20 %).
SCD Type 1 / Type 2 Current state only, or one row per version with __start_at, __end_at, __is_current.
Declarative table A table defined with @ds.table (or @ds.dimension, @ds.fact, @ds.obt) and built by ds.run() in dependency order.
Quarantine Rows that fail a quality rule, written to a separate table instead of the target.
Spark session The Spark engine instance that runs a user's cells. Depending on platform setup, a user's notebooks share one session or each has its own, up to a per-user limit. Notebooks run by Dataflow get their own session.
SQL guard The check every statement goes through in the Spark session: tenant stamping, protected columns, refusal of unsafe statements, and the tenant boundary (only your tenant's tables and lake paths can be read or written).
Policy snapshot The grants, row filters and column masks sent with a cell so the Spark session can enforce them.
Object log The history of table definitions (CREATE, ALTER, DROP, RENAME).
Run log One entry per cell, pipeline step or DataBridge load, with status, time and steps.
Lineage edge "Table A feeds table B", recorded by cells, pipelines, DataBridge, dbt and declarative tables.
Share / recipient A set of tables offered through the Delta Sharing protocol, and an outside party holding a bearer token for it.
Environment A complete DataSimple instance (development, QA, production) that work is promoted between.
Deploy key The stable identity of an object across environments, used to remap ids on promotion.

API route map#

All routes are relative to https://<your-datasimple-host>. Call them with a personal API token (Authorization: Bearer <PERSONAL_TOKEN>). Your role must include the matching screen. A personal token cannot create or revoke tokens, sign in or out, change or reset passwords, manage users, roles or role templates, or create sharing recipients or dbt client keys. Do those in the application while signed in.

Prefix Area Chapter
/api/auth/me Who you are (works with a personal token) 19
/api/auth/tokens Personal API tokens 18
/api/notebooks Notebooks, folders, schedules, external connections, saved joins, run log 02, 03, 10
/api/notebooks/spark Run cells (/run, /sql), stop, status, pre-warm, Trash, join builder 02, 23
/api/notebooks/unity-catalog, /api/2.1/unity-catalog Catalog, governance, Unity Catalog-compatible REST 07
/api/notebooks/governance Governance actions 07
/api/notebooks/semantic Glossary terms, metrics, business meaning 13
/api/notebooks/dq Quality rules, results, alerts, sweeps 12
/api/notebooks/sharing Shares, recipients, access log. A share is managed by its creator, anyone with MANAGE on it, or an administrator 14
/delta-sharing Delta Sharing protocol (bearer token, no session) 14
/api/pipelines (also /api/notebooks/pipelines) Pipelines, runs, stop, re-run failed 08
/api/workflows Workflows, runs, schedule as pipeline 09
/api/notebooks/airflow Dataflow (DAGs, runs, tasks, logs) 11
/api/notebooks/dbt, /api/notebooks/dbt/clients, /dbt-sql dbt projects, clients, SQL gateway for dbt 22
/api/databridge DataBridge loads, jobs, schedules, file ingest 04
/api/bronze-replication Bronze loads screen 05
/api/cloud-connections Lake destinations 03
/api/notebooks/query-history Query history 15
/api/usage Usage and cost 15
/api/deploy Environments, inventory, compare, promote, accept, history, roll back 16
/api/sarah, /api/agents, /api/everything Assistants 17
/api/admin Users, roles, role templates, tab access, tenants, activity and error logs 18
/api/kinetic Kinetic Pen (turn pointers and words into code, saved recordings, save as a business formula) 27
/api/notebooks/git Git integration for notebooks 02
/api/activity History activity (History tab) 15
/api/file-alerts New-file alerts 04
/api/notebooks/flink Streaming jobs (Pipelines tab) 08
/help/handbook This Developer Handbook (older help links forward here) —

Run states#

Object States
Notebook cell ok, error (run log)
Notebook schedule run running, success, partial (a cell failed), error
Pipeline run running, success, partial, error, cancelled; step results carry kept_from_run after a re-run
Workflow run running, ok, error, cancelled; boxes ok, error, skipped
DataBridge job queued, running, done, failed, cancelled; only a file copy can also end partial (some files copied, some failed)
Delete check ok, in_sync, refused (with a reason such as above_threshold), skipped (no key, or no Bronze yet), error (nothing recorded)
File ingest run ok, no_new_files, empty (new files had no readable rows), busy (another run holds the lock), cancelled, error
Quality alert open, acknowledged, resolved

Common errors#

Message or code Where Meaning and what to do
401 Login required Any /api route No session, or it was closed because the same user signed in elsewhere (one session per user). Sign in again.
403 with a tab name Any /api route Your role lacks that tab. Ask an administrator.
403 TOKEN_NOT_ALLOWED Personal token A token cannot create or revoke tokens, sign in or out, change passwords, manage users, roles or role templates, or create sharing recipients or dbt client keys. Sign in to the application to do this.
401 TOKEN_INVALID Personal token Revoked or expired token. Create a new one.
409 Pipeline "…" is already running (run #N) Pipelines A run is in progress; the answer names it. Wait for it to finish, or stop it.
409 This schedule is already running Schedules (Run now) Wait for the current run to finish, then run it again. A clock-triggered run that comes while one is running is skipped.
409 Stop or wait for the running workflow first. Workflows The workflow is running. Wait or stop it.
All Spark sessions are busy; try again in a minute. Notebook cells Every Spark session is busy and the cap is reached.
All your Spark sessions are busy (limit N per user) Notebook cells All of your own sessions are running. Try again in a minute, or stop a running cell.
Spark session restarted Notebook cells Your session was closed to free capacity for others, to make room for another of your notebooks, or after it sat idle. Variables are gone; re-run setup cells.
REQUIRES_SINGLE_PART_NAMESPACE Python spark.sql("… datasimple.gold.t") Known limit: in Python, use gold.t or read_table("datasimple.gold.t"). SQL cells accept the full name.
TABLE_OR_VIEW_NOT_FOUND Cells Check the name; a table loaded by DataBridge may need one SQL cell to attach it before Python spark.sql can see it.
Table or view … was not found in your catalog. Cells The table is not in your tenant's catalog. Check the name.
The lake path … is not in your catalog. Cells The path belongs to another tenant or is outside your catalog. Use a table name from your catalog.
The table name … cannot be used: choose another name. Cells The name is already taken outside your tenant. Pick another name.
Direct file-path queries bypass Unity Catalog governance SQL on parquet.\s3a://…`` Query the table by its catalog name.
… are missing from the source, above the 20% limit; no deletes recorded DataBridge delete check Check the source. If the deletes are real, raise max_delete_ratio for that table.
Another file-ingest run is already loading … File ingest Another run holds the table lock. Nothing was written; run again when it finishes.
You submitted this rule, so you cannot approve it Data quality approval Four-eyes rule. Another person must approve.
a metric must read, not write Semantics Metric SQL must be a SELECT or an aggregate expression.
RESOURCE_DOES_NOT_EXIST / UNAUTHENTICATED / PERMISSION_DENIED Delta Sharing Share not granted, token revoked or expired, or table has filters or masks.

Naming and data rules#

Rule Detail
Schema and table names Letters, digits and underscore; start with a letter or underscore
DataBridge source copies datasimple.source.<schema>_<table> for the Parquet copy (for example public_orders), <schema>_<table>_DELTA for the Delta copy and <schema>_<table>_ICEBERG for the Iceberg copy
Audit columns Bronze and change logs carry _op_type, _cdc_seq, _load_ts, _batch_id, _row_hash and related columns. See Bronze.
Tenant column _tenantid, stamped on insert
SCD Type 2 columns __start_at, __end_at, __is_current
Star schema Dimensions carry a surrogate key (<name>_key) and an Unknown member with key -1
VACUUM Retention of at least 168 hours
Dropped tables Go to Trash; restore with the Trash panel or UNDROP TABLE