Go to file
Paul Trowbridge 72750cb57f Add destination-table preflight to the run path
The engine built staging as `LIKE dest` and merged into it without ever
checking the destination existed — a missing dest (e.g. a file-based deploy
that never went through the web wizard) failed deep in the staging step with
a raw `relation ... does not exist`.

Add `engine/dest.reconcile_dest()`, called as step 3b of run_module (after the
dry-run return, before staging):
  - missing + not create -> DestError telling the user to pass --create-dest
  - missing + create     -> CREATE SCHEMA + CREATE TABLE from the module's
                            column map
  - exists               -> fail loudly if any mapped column is absent (drift);
                            no auto-ALTER, since the positional load can't
                            tolerate a mid-table add

Expose via `pipekit run --create-dest`. Default (scheduled groups, API) is
unchanged: validate-and-fail, never silently provision.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 12:35:02 -04:00
.archive/pre-rewrite Initial commit: Pipekit rewrite. 2026-04-22 00:38:26 -04:00
bin bin/pipekit auto-detects venv; stop rewriting it in deploy.sh. 2026-04-23 00:34:32 -04:00
config activate the ffsbglr1 2026-07-22 15:16:07 -04:00
pipekit Add destination-table preflight to the run path 2026-07-24 12:35:02 -04:00
systemd Add scheduling, harden deploy, and update docs 2026-06-03 21:18:13 -04:00
.gitignore drivers: make staging DDL dialect-aware; stop tracking pipekit.db 2026-06-10 23:33:36 -04:00
CLAUDE.md jrunner: use bulk copy (-b) for Postgres dests too (COPY) 2026-06-18 23:14:15 -04:00
config.yaml web: add session-cookie login for web UI 2026-06-04 13:21:50 -04:00
deploy.sh deploy/cli: fix /etc/pipekit permissions so non-root group members can write secrets 2026-06-04 13:35:06 -04:00
reconcile.py feat: cross-DB reconciliation tool 2026-07-22 15:21:17 -04:00
requirements.txt web: add session-cookie login for web UI 2026-06-04 13:21:50 -04:00
SPEC.md feat: version-control pipeline definitions via export/apply 2026-07-22 10:56:55 -04:00