Back to the index · Previous: Spaces
Step by step with screenshots: Table Audit — a practical guide reads a Gold, a Silver and a Bronze table, asks Sarah, opens the Audit tab, adds the table's versions, filters, exports and verifies, with the real numbers and costs of one run. Test it by hand: manual test cases.
How it works#
Every lake table (bronze, silver, gold) has one Audit history: who read it, who wrote it and how many rows changed, every change to its definition, every grant, row filter and column mask with the value before and after, and every lineage link that was added or taken away. It is all on one page, newest first.
What is recorded:
| Kind | What you see | Where it comes from |
|---|---|---|
| Reads | who, from where (SQL query, notebook, Sarah, agent, Everything, API token, ML model, share, Dataflow), the run, rows returned | every SQL and Python run that reads the table |
| Writes | operation, rows inserted / updated / deleted, the table's row count after, table version, who, run | every write: SQL, notebook helpers, Dataflow, DataBridge loads, dbt, ML scoring output |
| Definition | CREATE, ALTER, RENAME, DROP, REFRESH, who and how | the definition log (also on Definition & history) |
| Access | grants, revokes, row filters, column masks, with before and after | governance changes |
| Lineage | "built from" or "feeds" another table, added or removed | the lineage graph |
| Versions | the table's own version log, each version matched to the app user who wrote it | read on request |
Nothing is added to your tables. The audit lives in the app's own records, so the tables your engineers design stay as they are.
Who sees what#
- The Audit tab shows only to people with SELECT on the table.
- Reads (who looked at the table) show only to the table's owner and administrators. Everyone else with SELECT sees writes, definition, access and lineage.
- You only ever see your own tenant's history.
Open a table's audit#
- Left rail → Lakehouse → right-click a table, for example
gold.dbt_retail_daily_store_sales→ Details & audit. - Choose the Audit tab.
- Narrow the list:
- press a kind (Reads, Writes, Definition, Access, Lineage, Versions) to show only those; press more than one to combine them; press none to see all
- pick a person under Everyone
- set a from and to date
- Press an Access line to see the value before and after the change.
Table versions#
Press Table versions to add the table's own versions to the list. Each version shows its operation and rows, and the app user who wrote it when the app recorded that write (matched by version number, or by a write within two minutes). Versions written before the audit started show not recorded. This reads the table's version log with one short Spark job, so it runs only when you press it.
Export#
Press CSV to download what the list shows, with the filters you set. Table versions are not in the file; they are read from the table itself only when you press Table versions. The file opens in any spreadsheet. Text that starts with =, +, - or @ is kept as text, so a cell can never run as a formula.
Verify#
Press Verify to check that no audit record was changed or removed. Every record carries a fingerprint of itself and of the record before it, one chain per tenant. If a record in the middle was edited or deleted, the chain no longer adds up and Verify names the first record that does not match.
- Audit records intact means every chain adds up.
- Records from before the audit started are counted as older rows from before the chain; they are kept but carry no fingerprint.
Audit records cannot be edited or deleted from the app. The only way they leave is the log archive: old records are copied to the archive first and only then removed, and Verify starts the chain again from the oldest record kept.
Good to know#
- Recording never slows a query. Reads and writes are noted after the answer has gone back.
- A query that reads several tables records one read per table; the rows returned are the whole query's.
- A model's predict call scores rows the caller sends, so no table rows are read; it shows as a model use on the table the model was trained on.
- A read through a share shows the recipient's name.
- A read through a view is recorded on the table the view reads (Sarah's Gold views too).
- In Python,
read_table("…")and SQL insidespark.sql("…")are recorded;spark.table("…")is not yet. - A query that fails records no read.
- Access changes made before the audit started are listed without before / after.
- In the CSV, times are UTC and the
originvaluesql_editormeans a SQL query run outside a notebook cell.
Cost#
Opening the Audit tab, filtering, exporting and Verify read the app's own database only; they cost nothing on the cluster. Table versions runs one short Spark job on the cluster that is already running (a few seconds); it starts nothing new.
Next: Back to the index