Postgres or MySQL → Iceberg on S3

Postgres or MySQL into Apache Iceberg on S3 — Fabric's default destination, and the combination where your data stays in your own bucket, readable by Trino, Spark, DuckDB, or anything else that speaks Iceberg. This is the path most teams start with.

Source-side setup is the usual PostgreSQL or MySQL work, with the grants on Least-privilege logins. The destination is Iceberg. What it costs is on Pricing.

Before you start

• An S3 bucket (or MinIO, Cloudflare R2, Azure, GCS) for the table data and metadata, and a prefix to keep it under.
• A catalog — the destination wizard walks you through it: JDBC (a Postgres holding the catalog tables), REST (Nessie), Glue, or Hive Metastore. Whichever you pick, its credentials go in the same form; the Iceberg page lists the fields per catalog and storage type.
• Storage credentials with read and write on the bucket prefix — Fabric both writes the Parquet files and maintains the table (compaction, snapshot expiry, orphan cleanup).

Network access and TLS

Fabric's runners connect out to your Postgres or MySQL database and your destination, so nothing needs to be opened inbound. Outbound, the database firewall must allow the runner's egress address: TODO(E6-egress) — the fixed address is not published yet; it lands here and on the security page when it is. Until then, a connection that times out rather than being refused is usually the firewall.

On the hosted service a new source starts at Require TLS: the server must offer TLS, the certificate is not verified, and disable/prefer are refused. Choose Verify full when you want the certificate checked against a CA you paste into the form — managed services publish theirs (RDS, Cloud SQL, Azure). Two real messages and their fixes:

• TLS to db.example.com failed: the server does not offer TLS and this source requires it (require). Turn TLS on at the database, or change the mode to disable or prefer (self-hosted installs only — the hosted service refuses those two).
• TLS to db.example.com failed: the server's certificate did not validate against the CA this source carries (verify-full). The CA is missing, wrong, or the wrong one for this server. Paste the issuing CA, or drop back to require to skip verification.

A certificate that is valid but does not carry the host name you dialled gets its own sentence, and points at verify-ca (the CA check without the name check) — the full mode list, and what each one asserts, is on Sources.

The five steps

1. Add the destination. Destinations → Add destination → pick the catalog kind, then the storage kind, then fill the connection. The wizard's carousel is ordered for this path: catalog first, storage second.
2. Add the source. Syncs → New sync → the PostgreSQL or MySQL card → connection fields → Create source & continue. A PostgreSQL source also asks for a replication slot name, and a MySQL source for a Server ID — any name/number will do, and the wizard will not continue without one.
3. Pick the tables. Select the tables, set the identifier columns per row (default id), and choose Target namespace — the Iceberg namespace the tables are created in inside the catalog.
4. Create the sync. Sink step → your Iceberg destination → Test connection (the step will not unlock without it) → Review & save → Save sync. (Leaving the picker on the default uses the install's seeded destination, if it has one.)
5. Start replication and watch rows arrive. The sync's detail page → Start → the badge turns snapshotting then running. The stats row reports rows and lag; the Tables tab reports per-table rows, snapshots, and the last commit.

See the rows arrive

-- Trino, Spark, DuckDB: point the catalog at the same warehouse
SELECT count(*), max(__source_ts_ms) FROM iceberg.raw.orders;

The catalog name and warehouse come from the destination you created. Snapshot rows, inserts, updates, and tombstones all land as rows — current state is a read-side query over __source_ts_ms and __deleted, described in Data shape & metadata.

When it does not work

• Writes fail with a 403 from the object store. The storage credentials lack write on the prefix, or the bucket's region is not the one configured. Fix the credentials in the destination and restart the runner.
• The catalog rejects the commit. The JDBC catalog's Postgres is unreachable from the runner, or its user lacks CREATE on the catalog schema. The sync's logs carry the catalog's own error.
• Tables grow and snapshots pile up. Iceberg maintenance is what keeps that flat: Fabric applies a maintenance policy per table — see Maintenance. Tables created before the policy existed are the ones to check first.