CLI Reference
plan, apply and load
Preview a project's changes, create tables, and load data.
sqlload plan [SELECTOR ...] [OPTIONS]sqlload apply [SELECTOR ...] [OPTIONS]sqlload load [SELECTOR ...] [OPTIONS]| Command | Does |
|---|---|
plan |
Prints what apply and load would do: tables to create or alter, each column’s type mapping, write strategy, route, batches and cursor. Changes nothing. |
apply |
Creates or changes destination tables (and Snowflake stages SQLLoad owns). Never loads data. |
load |
Loads the selected tables, creating any that don’t exist yet. |
apply and load print the plan first. See Projects for how the
three fit together.
Options
Section titled “Options”All three take the same options.
| Option | Meaning |
|---|---|
SELECTOR ..., -s, --select <SELECTOR>... |
Sources or tables to select: a source name, a destination table name, or a source’s schema.table. All when none is given. |
--exclude <SELECTOR>... |
Sources or tables to leave out. Wins over --select. |
--start-cursor <VALUE> |
Read from this cursor value instead of the destination’s maximum. Typed from each table’s cursor: an integer, an RFC 3339 timestamp or a date. |
--end-cursor <VALUE> |
Read only cursor values below this one. |
--reload |
Rebuild the selected tables from the whole source, with the current columns and types. Asks first, unless --yes. Refuses postgres_cdc tables, which --resnapshot copies again; see Reloading. |
--resnapshot |
postgres_cdc sources: drop and recreate the replication slot, then copy every table again. Asks first, unless --yes. See Postgres changes. |
--follow |
load of one postgres_cdc source only: keep streaming after catching up, writing a batch every batch_interval, until --max-duration passes or Ctrl-C or SIGTERM. See Keep streaming. |
--max-duration <DURATION> |
With --follow: stop after this long, such as 10m, once the changes read so far are written. |
--yes |
Rebuild with --reload or --resnapshot without asking. Needed when not on a terminal. |
--allow-type-change-rebuild |
Allow table rebuilds that type_change_rebuild = "require_confirmation" would block. |
--concurrency <N> |
The most tables loading at once. Default: the project’s concurrency, else 4. |
--json |
plan only: print the plan as one JSON document. |
Reading a plan
Section titled “Reading a plan”$ sqlload planPlan ready 2 selected, 2 objects to create, 2 to load
Objects (2) sqlload apply├── raw.app__customers create table│ ├── customer_id int -> BIGINT│ ├── email text -> VARCHAR│ ├── country text -> VARCHAR│ ├── created_at timestamp_tz -> TIMESTAMP WITH TIME ZONE│ └── + _sqlload_load_id, _loaded_at└── raw.app__orders create table ├── order_id int -> BIGINT ├── customer_id int -> BIGINT ├── status text -> VARCHAR ├── total decimal(12,2) -> DECIMAL(12,2) ├── updated_at timestamp_tz -> TIMESTAMP WITH TIME ZONE └── + _sqlload_load_id, _loaded_at
Loads (2) sqlload load├── app__customers merge (unique_key: customer_id)│ ├── source postgres app public.customers│ ├── route read as Arrow -> DuckDB takes Arrow│ └── batch about every 1 GB└── app__orders merge (unique_key: order_id) ├── source postgres app public.orders ├── route read as Arrow -> DuckDB takes Arrow ├── batch about every 1 GB, in updated_at order (an index leads with it, NULLs first), between distinct values └── cursor updated_at first run, everything
✓ Plan complete 2 selected, 2 objects to create, 2 to load| Section | Lists |
|---|---|
Objects |
Tables apply creates or alters: each column as its portable type -> the warehouse type, added and widened columns, excluded and masked columns. |
Loads |
Each table load loads, with its write strategy, and lines for source, files, schema, route, batch, cursor, state and note. |
Skipped |
Tables left out on purpose, such as by skip_tables_with_columns or new_tables = "skip", with the reason. |
Errors |
Tables that can’t load as configured, with the reason and the fix. |
Warnings |
Tables that load, but slower or with less than you might expect. |
A table under Errors is left out; the others still run, and the command exits 1.
Loading
Section titled “Loading”$ sqlload load…loaded raw.app__customers 50 rows 0.0sloaded raw.app__orders 1,000 rows 0.0s
✓ Completed 2 loaded, 1,050 rows, 0.1sEach table prints created, altered, loaded or failed. A failed table doesn’t stop the
others; the summary counts them and the command exits 1.
--reload rebuilds tables from scratch. On a terminal it shows the plan and asks; in scripts, add
--yes:
$ sqlload load --reload --yes app__customers…loaded raw.app__customers 50 rows 0.0s
✓ Completed 1 loaded, 50 rows, 0.1s