Fabric

Guide

Postgres or MySQL → Snowflake

The same source setup with key-pair auth, an internal stage, and COPY INTO landing in your own account.

Postgres or MySQL in, Snowflake out: the full append-only CDC history lands in your own Snowflake account through an internal stage and COPY INTO. This is the combination for teams whose analysts already live in Snowflake.

Source-side setup is the same as any other path — PostgreSQL or MySQL, with the grants on Least-privilege logins. The destination is Snowflake. What it costs is on Pricing.

Before you start

  • The account identifier — <org>-<account_locator>, the same string the Snowflake URL carries.
  • A user and a role that can create objects in one database: CREATE SCHEMA on the database, CREATE STAGE and CREATE TABLE on the schema, and INSERT/SELECT on its tables. Fabric creates the schemas, stages, and tables it needs — never the database itself. The Snowflake page has the GRANT block.
  • Key-pair auth, preferably. Generate a PKCS#8 private key, register the public half on the Snowflake user, and paste the private key (plus its passphrase, if it has one) into the destination form. A password is the fallback.

Network access and TLS

Fabric's runners connect out to your Postgres or MySQL database and your destination, so nothing needs to be opened inbound. Outbound, the database firewall must allow the runner's egress address: TODO(E6-egress) — the fixed address is not published yet; it lands here and on the security page when it is. Until then, a connection that times out rather than being refused is usually the firewall.

On the hosted service a new source starts at Require TLS: the server must offer TLS, the certificate is not verified, and disable/prefer are refused. Choose Verify full when you want the certificate checked against a CA you paste into the form — managed services publish theirs (RDS, Cloud SQL, Azure). Two real messages and their fixes:

  • TLS to db.example.com failed: the server does not offer TLS and this source requires it (require). Turn TLS on at the database, or change the mode to disable or prefer (self-hosted installs only — the hosted service refuses those two).
  • TLS to db.example.com failed: the server's certificate did not validate against the CA this source carries (verify-full). The CA is missing, wrong, or the wrong one for this server. Paste the issuing CA, or drop back to require to skip verification.

A certificate that is valid but does not carry the host name you dialled gets its own sentence, and points at verify-ca (the CA check without the name check) — the full mode list, and what each one asserts, is on Sources.

The five steps

  1. Add the destination. Destinations → Add destination → Snowflake: name, account, username, warehouse, database, role (optional), and the private key or password.
  2. Add the source. Syncs → New sync → the PostgreSQL or MySQL card → connection fields → Create source & continue. A PostgreSQL source also asks for a replication slot name, and a MySQL source for a Server ID — any name/number will do, and the wizard will not continue without one.
  3. Pick the tables. Select the tables and set the identifier columns per row. Target namespace becomes the Snowflake schema inside the destination's database; the table name is the third level of the namespace.
  4. Create the sync. Sink step → your Snowflake destination → Test connection (the step will not unlock without it) → Review & save → Save sync.
  5. Start replication and watch rows arrive. The sync's detail page → Start → the badge turns snapshotting then running, and rows and lag are reported on the stats row.

See the rows arrive

SELECT count(*), max(__source_ts_ms) FROM ANALYTICS.RAW.ORDERS;

ANALYTICS is the destination's database, RAW the namespace/schema you chose, ORDERS the source table. The history is append-only, so current state is a read-side query over __source_ts_ms and __deleted — see Data shape & metadata.

When it does not work

  • "404 Not Found" on the connection test. Snowflake builds the host from the account field — <org>-<account_locator>, or the full <org>-<account>.<region> — so a typo or a missing region surfaces as 404 Not Found: post …snowflakecomputing.com:443/session/v1/ login-request rather than an auth error. Check the Account field against the locator on the Snowflake URL you log in with.
  • Authentication fails at the first commit. A private key that does not match the public key registered on the Snowflake user, or a passphrase pasted with the wrong key, surfaces as Snowflake's own JWT error in the sync's logs. Re-paste the key including its -----BEGIN …----- lines.
  • "Insufficient privileges" on the first write. The role cannot create the schema or stage. Run the GRANT block from the Snowflake page against the destination's database and schema, then restart the runner.
  • Nothing lands and the pipeline looks healthy. Check that the sync's destination is the Snowflake row you think it is — the sync page names it, and the Sink step's picker is where it was chosen. A sync writes to exactly one destination.