Concepts

Data shape & metadata

The metadata columns every row carries and how to read the append-only history.

Every table Fabric writes carries standard metadata columns alongside the source data. These are what let you read the history correctly.

ColumnMeaning
__opr snapshot, c insert, u update, d delete
__tableSource table name
__source_ts_epochEvent timestamp (milliseconds since epoch)
__dbSource database name

Append-only history

Fabric writes append-only: every change is a new row, so the table is a full history of the source. Fabric does not upsert on the write path — consumers merge using the identifier fields and the tombstone columns.

  • Snapshot rows (__op = r) are the initial load of existing data.
  • Deletes (__op = d) are tombstone rows, so the historical row stays but is marked deleted.

Reading the data

To get the current state of a row, dedupe by the identifier fields and take the latest by __source_ts_epoch; drop the rows marked deleted. Every Iceberg engine — Trino, Spark, DuckDB, and friends — can read it with a plain query.