Supported systems

Sources

Source	Formats	Read by
Postgres	Tables, by include patterns	Binary COPY into Arrow
Postgres changes (postgres_cdc)	Inserts, updates and deletes of tables, by include patterns	A snapshot, then a logical replication slot (pgoutput), in whole transactions. See Postgres changes.
Files in S3	Parquet, CSV, JSON, NDJSON; gzip or zstd for text formats	The destination itself where it can load them exactly, otherwise SQLLoad’s reader
Files in a local folder	As for S3	As for S3
Python	Dicts, Arrow data (pyarrow, polars, pandas), JSON and NDJSON bytes	The sqlload package, in process
HTTP APIs	JSON, through sl.http in a Python source	Rust
Kafka	Topics, through the separate exporter	Rust

Destinations

Destination	Write strategies	How data arrives
Snowflake	append, merge, delete_insert, replace	Parquet staged on an external, user, table or named stage, then COPY; or Snowpipe Streaming, or INSERT, by load_method. Files under an external stage load with COPY directly.
Databricks	append, merge, delete_insert, replace	Parquet staged in a Unity Catalog volume, then COPY INTO; one statement per write. Files of a files source pass through SQLLoad.
DuckDB	append, merge, delete_insert, replace	Arrow appends; Parquet files read by DuckDB with read_parquet.
Parquet files in S3	append, delete_insert, replace	Parquet written by SQLLoad, zstd-compressed.
Parquet files in a local folder	As for S3	As for S3

Source and destination combinations

From \ Into	Snowflake	Databricks	DuckDB	Parquet files
Postgres	Yes	Yes	Yes	Yes
Postgres changes (postgres_cdc)	Yes	No	Yes	Yes, as a changelog
Files	Yes, read by Snowflake from an external stage	Yes, staged by SQLLoad in the volume	Yes; Parquet read by DuckDB	No
Python	Yes	Yes	Yes	Yes

A files source into a files target is refused at plan. merge isn’t available for files sources or files targets.

These are the systems in the current release. Planned ones are on the roadmap.

Testing

System	How it’s tested
Postgres	Real Postgres in containers. postgres_cdc on Postgres 15, 16 and 17, plus live loads into DuckDB and Snowflake.
DuckDB	Real, in process.
S3	A real S3-compatible store in containers; benchmarks against AWS S3.
Local files	Real files in temporary folders.
Kafka	Redpanda in containers.
Snowflake	A local fake for protocol and failure tests, plus live tests on a real Snowflake account.
Databricks	A local fake workspace (OAuth, statements, the volume) for protocol and failure tests, plus live tests of every pathway on a real SQL warehouse. See Databricks.

Platforms

The sqlload and sqlload-cli packages are released for Linux (x86_64 and aarch64, glibc 2.28 or newer) and macOS 11 or newer (arm64). The sqlload package needs CPython 3.10 or newer. See Install.