The ClickHouse sink lands the full append-only CDC history on a plain MergeTree table — one row per change event. Fabric never upserts and MergeTree never merges duplicate keys away, so this is the sink where nothing collapses rows. Current state is a read-side concern.
Prerequisites
- A reachable ClickHouse server, self-managed or Cloud. It must support
insert_deduplication_tokenon plainMergeTree,async_insert, and the column types Fabric maps to. - Network access on the port matching the chosen protocol: 8123 (HTTP) or 8443 (HTTP + TLS), 9000 (native) or 9440 (native + TLS).
- A user with
CREATE DATABASE,CREATE TABLE,ALTER TABLE, andINSERT, plusSELECTonsystem.columns:
CREATE USER fabric IDENTIFIED BY '<password>';
GRANT CREATE DATABASE, CREATE TABLE, ALTER TABLE, INSERT ON *.* TO fabric;
GRANT SELECT ON system.columns TO fabric;A single node is expected: clustered and distributed writes are not implemented.
Connection fields
| Field | Required | Default | Notes |
|---|---|---|---|
| Name | Yes | — | Destination label. |
| Host | Yes | — | |
| Port | Yes | 8123 (HTTP) / 9000 (native) | Follows the protocol until you edit it. |
| Protocol | Yes | http | http or native. |
| Username | Yes | default | |
| Password | No | empty | Blank on create means an empty password (the stock default user); blank on edit keeps the stored password. |
| Database | Yes | default | The database the client initially connects to — not where syncs land. |
| Secure | No | off | Use TLS (HTTPS for the HTTP interface). |
| Cluster name | No | — | Reserved for a future clustered writer; leave blank. |
How tables land
A sync's namespace becomes a ClickHouse database, created on first write, and the table name sits directly inside it: <namespace>.<table>. There is no schema level in between.
- Tables are plain
MergeTree, ordered by the identifier columns and__source_ts_ms. The ordering is a locality index, not a uniqueness constraint. - Every
INSERTcarries a deterministic deduplication token derived from the batch, so a replayed block after a lost acknowledgement no-ops instead of duplicating history. - Schema evolution is additive: a new column lands as a
Nullablecolumn, and existing columns are never retyped. - Small blocks are written asynchronously; larger ones go synchronous. The commit gate is tuned to avoid ClickHouse's "too many parts" failure.
Reading current state
Because the table holds the raw history, resolve current state on read: group by the identifier columns and take the latest version with argMax, dropping rows where __deleted is true:
SELECT id, argMax(email, __source_ts_ms) AS email
FROM raw.customers
WHERE __deleted = 'false'
GROUP BY id;An AggregatingMergeTree materialized view on top is the performant alternative for large tables. Fabric ships the history, not the current-state view.
Gotchas
Watch out
- The Database field is not where syncs land — the namespace database is created for you.
- Frequent small inserts can outrun background merges and hit "too many parts". The tuned commit gate, asynchronous small blocks, and default partitioning are the mitigations — do not lower them.
- On a table created by an older writer, Fabric re-applies the deduplication settings; a user without
ALTERsilently loses retry dedup. - Test connection proves reachability and auth only. Verify
CREATE DATABASE/CREATE TABLE/ALTER/INSERTwith the first sync.