Skip to content

About

Troubleshooting

Common errors from plan and load, what they mean, and the fix for each.

Most problems show up in sqlload plan, under Errors, with the fix. The table below lists the common ones; each message is copied from SQLLoad.

Message Fix
error: environment variable PGPASSWORD is not set Set the variable a ${ENV:NAME} in sqlload.toml refers to.
no description; add description = "..." under [sources.app.tables."public.orders"] Every table SQLLoad writes needs a description. Add one where the message says.
load_method is required: how should data get into Snowflake? Add a load_method to the Snowflake connection. See Snowflake.
n_free: numeric without a precision has no fixed scale; pin a type such as "decimal(38,10)" or "text" Pin the column in columns, such as columns = { amount = "decimal(38,18)" }.
columns "total" is not a column of the source table (names are case-sensitive); did you mean "email"? Fix the column name, or move the entry to the right table.
total has type numeric(12,2), which cannot be read as int Pin a type the source column can be read as, usually the same or wider.
write_strategy = "merge" needs a unique_key (the table has no primary key) Set unique_key, or use delete_insert with a cursor_column.
write_strategy = "replace" reloads the whole table, so it takes no cursor_column Remove cursor_column, or choose merge, delete_insert or append.
lookback re-inserts the window on every run with append; use delete_insert (no key needed) or merge Change the strategy, or drop lookback.
merge needs a table format to update rows in place; a files target takes append, replace or delete_insert Use one of those strategies on a files target.
... was rewritten in place since it was loaded ...; load it again with --reload, or use delete_insert on a partition cursor A files source file changed after it loaded under append. Reload, or partition the source.
--reload rebuilds ... confirm it in a terminal, or add --yes Add --yes when running without a terminal.
tags: kind = "postgres_cdc" streams text[] values as the source wrote them ...; pin it with columns = { tags = "text" }, or exclude it Pin array and json columns to text. See Postgres changes.
slot sqlload_app is gone ... or slot sqlload_app lost WAL it had not confirmed ... Run sqlload load --resnapshot. See The slot and WAL.
column tier of app__customers was added with a default ... or ... now loads column region, which the table already had at its snapshot ... A schema change the stream can’t carry. Run sqlload load --resnapshot. See Schema changes.
publication app_changes does not publish "public"."refunds"; add them with ALTER PUBLICATION ... A table joined the source. Add it to the named publication as the tables’ owner. See Adding tables.
rows captured before slot sqlload_app was recreated are still in app__refunds ... A table rejoined the source. Run sqlload load --resnapshot, or drop its destination table and run sqlload apply. See Adding tables.
a files target takes kind = "postgres_cdc" only with cdc_mode = "changelog" ... Set cdc_mode = "changelog". See Into Parquet files.
a postgres_cdc table is copied again with --resnapshot, not --reload Use sqlload load --resnapshot for the captured tables, and --exclude the postgres_cdc source to reload the others. See Reloading.
cannot connect to Postgres at ... under [sources.NAME] That source’s server can’t be reached. Commands that don’t select it still run; see Selecting tables.
load --follow follows one postgres_cdc source and nothing else; ... is not captured, so load them without --follow Select a postgres_cdc source, or load the other tables without --follow.
source app does not follow while app__orders is blocked: its tables share one slot ... load --follow follows a whole postgres_cdc source. Select the source, not some of its tables. See Keep streaming.
[sources.app] batch_interval must be at least 1s ... Set batch_interval to 1s or more.
Postgres query failed: permission denied for database postgres (SQLSTATE 42501) The user can’t create the publication. Create it as the tables’ owner and name it with publication. See Requirements.
Python source shop uses connection wh, which target dev loads into Give the Python source its own connection; table functions never get warehouse credentials.
Python table orders is declared twice (...); table names are unique across the project Rename one of the functions, or pass name= to the decorator.
skipped rows outside the cursor window; use ctx.cursor_value to read only new data A warning: the load is correct, but the function reads more than it needs. Filter on ctx.cursor_value.
  • It doesn’t match include, or matches exclude.
  • It has a column matching skip_tables_with_columns; plan lists it under Skipped.
  • It appeared after the source’s first load, and new_tables = "skip". Name it exactly in include.

plan shows each table’s route. Warnings name routes that are much slower than the usual one, such as copy_to_internal_stage, hmac-masked files, or load_method = { type = "insert" }. For files into Snowflake, an external stage over the files’ bucket lets Snowflake read them in place. See Load methods.

A cursor table without an ordered read commits once for the whole read. For a large first load, set batch_size (such as batch_size = "1d") with cursor_start, so it commits window by window. See Batches.

sqlload load --reload --yes TABLE rebuilds it from the whole source. Deleting the destination table works too: SQLLoad keeps no other state, so the next run loads everything again. A postgres_cdc table starts over with sqlload load --resnapshot; see Re-snapshot.

--log-format json writes structured logs to standard error. sqlload plan --json prints the plan as one JSON document.