The merged interface work changed observable behaviour while still
reporting "jrunner version 1.2", so two distinguishable binaries shared a
version string. 1.3 covers:
--passfile credentials read from an alias file at an arbitrary path
--strict exit 1 on failure (default still exits 0 for /opt/sync)
@summary machine-readable status/rows/ms line on stderr
execute() statements returning no result set are successes, and a
procedure's result set is found past any update counts
-f json implemented; previously advertised but fell through to CSV
This is also what a minimum-version check can key on — pipekit still
carries fallbacks for a pre-1.3 jar and needs a way to know it can drop
them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/opt/sync uses CALL statements as migration sources — rlarp.QUOTE_REBUILD,
SB_UD_R2, SB_GJ_R1 — and a stored procedure may report update counts
before it opens its cursor. The previous commit concluded "no result set"
on the first false from execute(), which would have silently migrated
nothing for such a procedure.
Now the canonical JDBC walk: step through update counts via
getMoreResults() until a result set appears, and only then treat the
statement as producing none. The last update count is retained so plain
DML still reports rows affected.
This is strictly more capable than the executeQuery() it replaced, which
threw outright in this situation. Verified against the Postgres analogue
(UPDATE followed by SELECT in one batch): the SELECT's rows come through,
pure DML still reports its count, plain SELECT is unaffected. The DB2
procedures themselves were not invoked — they have production side effects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
json and table were advertised in --help but both fell through to
outputCSV, so -f json silently produced CSV. table is now dropped from
the help text rather than advertised and unimplemented; json is real.
Two things CSV cannot express, both of which callers need:
* NULL vs empty string. outputCSV writes an empty field for both, so
the merge, reconcile and the wizard all see them as identical. JSON
emits null and "" distinctly.
* Column types. The wizard's SQL-entry mode defaults every dest column
to text purely because CSV carries no type metadata; the header now
reports name, type, precision and scale (verified against DB2 for i:
CHAR precision=5, DECIMAL precision=7 scale=2).
Values are JSON strings, never JSON numbers — a DECIMAL rendered as a
number would pass through a float and lose exactness. Output is written
incrementally so a large result set streams as CSV does while remaining
one valid document. Zero-row results still report their columns, which is
what query-column introspection relies on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two coupled changes: the summary is what a caller should read instead of
parsing prose, and the statement fix is what makes a non-zero exit code
usable at all.
Summary: emit one "@summary {json}" line carrying status, row count and
elapsed ms. Deliberately on stderr — migration-mode stdout carries the
progress output pipekit streams to its live log, and query-mode stdout
must stay pure CSV, so structured data on stdout would corrupt one or the
other. Previously the row count reached the caller only because a \r
progress tick and a trailing " rows written" happened to render on the
same line, which pipekit then matched with three different regexes.
Statements: query mode used executeQuery(), which raises when a statement
returns no result set — Postgres "No results were returned by the query",
SQL Server "The statement did not return a result set". DDL, INSERT,
DELETE and TRUNCATE all hit this despite succeeding, so pipekit carries a
_BENIGN_EXCEPTION_SUBSTRINGS allowlist to tell a working TRUNCATE from a
real failure. That made --strict unusable: it would have failed every
merge, staging DDL and hook. Now execute() + getUpdateCount() treats them
as the successes they are and reports rows affected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every error path did printStackTrace() then System.exit(0), making a
failed run indistinguishable from a successful one to any caller checking
$?. pipekit works around this by grepping stdout for stack-trace text
(_detect_silent_failure); 112 /opt/sync scripts run under `set -e` and
cannot detect a jrunner failure at all.
All 14 error exits now route through die(), which honours --strict.
Opt-in rather than default: flipping it unconditionally would change the
behaviour of those 112 scripts at once. --strict is pre-scanned from argv
so it applies to failures during argument parsing too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
-scp/-dcp put passwords on argv, where any local user can read them with
ps. This was not theoretical: a running migration was observed exposing
both an AS/400 and a Postgres password in plaintext.
-sc/-dc aliases already avoided that, but only read ~/.jrunnerpass, and
the pipekit service account is created with --no-create-home, so it had
no way to use them. --passfile points at an arbitrary path.
Also warn when the passfile is world-readable, and report the resolved
path in errors instead of a hardcoded ~/.jrunnerpass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Extends -b to Postgres destinations: stream the source ResultSet into PG with
COPY <dt> FROM STDIN (FORMAT csv) via the JDBC CopyManager, instead of batched
INSERTs. COPY is text-based so the server parses each field into the column
type — no per-type quoting needed. Every non-null value is CSV-quoted (so
empty string stays distinct from NULL, which is an empty unquoted field);
rows are flushed in 1000-row buffers with a 10k-row progress counter.
Validated DB2->PG: numeric precision (123.4567), jsonb, unicode, embedded
quotes, NULL vs empty-string all correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bulk path showed no progress during a load (only the final count). Emit
an in-place counter (\r + rows) every 10k rows from the BulkSource adapter,
which the caller pulls one row at a time, so it streams live. Prefix the
final count print with \r so it starts a fresh line instead of concatenating
onto the last tick (which produced a garbage row count like 3000035000).
Verified: ticks emit at 10k/20k/30k, final row_count parses correctly, and
pipekit's progress-collapse renders it as a single updating line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bulk path printed no count, so the trailing " rows written" line had no
number and callers parsing stdout got nothing. Count rows in the BulkSource
adapter (one per getRowData) and print it, matching the INSERT path's
"<n> rows written" so the count is captured.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in -b flag (migration mode, SQL Server dest only) that streams the
source ResultSet straight into SQL Server over the TDS bulk-load protocol
instead of 250-row INSERT...VALUES round trips. A BulkSource adapter
(ISQLServerBulkData) maps PG source types to JDBC types we control: string-ish
types (text/varchar/char/bpchar/json/jsonb/uuid/numeric) go through NVARCHAR via
getString so SQL Server converts losslessly — notably numeric, since PG reports
unconstrained numeric as scale 0 which made a typed DECIMAL path round
(123.45 -> 123). Default stays the INSERT path, so nothing regresses.
Validated against live PG->SQL Server: int4/text/jsonb/numeric/date plus nulls,
unicode, quotes, and numeric precision (123.45, 0.123456) all correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The migration INSERT builder's switch quoted varchar/text/char/clob/date/time
but let everything else fall to a default that emits rs.getString() unquoted
(correct for numerics, broken for strings). A pg->SQL Server pull of a jsonb
column failed with "Incorrect syntax near 'volume_bucket'" — the JSON text's
embedded double-quotes were read as a SQL identifier. Quote json/jsonb, plus
bpchar (PG char(n)) and uuid, like varchar.
Note: the default case still emits unquoted; other unhandled string types
(e.g. bool->'t'/'f') would need similar handling or a quote-by-default flip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
setFetchSize(10000) is a no-op on the PostgreSQL JDBC driver while autoCommit
is true — the driver loads the entire ResultSet into memory, OOM/GC-thrashing
on large source tables (a pg->SQL Server pull pinned the box: 4GB heap, swap
full, 0 rows written). PG only uses a server-side cursor when autoCommit is
false AND fetchSize > 0.
Set the source connection to manual commit ONLY in migration mode: the
migration source is read-only so never committing is harmless. Query mode is
excluded on purpose — callers (pipekit's run_dest_sql) run committed DDL/DML
through query mode, and autoCommit=false would roll those back on close.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds DATETIME, DATETIME2, SMALLDATETIME, and DATETIMEOFFSET cases to
the TIMESTAMP branch so SQL Server datetime columns are handled correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements .pgpass-style credential file for jrunner. Named aliases can
be used with -sc and -dc flags instead of spelling out -scu/-scn/-scp
for each invocation. Explicit flags still take priority over the file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove all diagnostic output in query mode - no front matter, timestamps, or metadata. Query results go directly to stdout for seamless piping to visidata/pspg/less.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Query mode auto-activates when destination flags are omitted, outputting CSV/TSV to stdout for interactive data exploration of DB2 iSeries queries.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Readme changes:
- Document that deployment directory must exist first
- Show mkdir -p commands before deploy
- Explain atomic deployment behavior (extracts to /tmp first)
Version bump to 1.0:
- Major refactoring: renamed app to jrunner
- Simplified deployment script
- Updated documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changes:
- Rename app/ directory to jrunner/ (preserves git history)
- Update settings.gradle to reference jrunner module
- Update readme.md with new paths (jrunner/build/, /opt/jrunner)
- Update CLAUDE.md documentation with new file paths
Build outputs now named jrunner.zip, jrunner.jar, bin/jrunner instead
of generic "app" names. This makes the project structure clearer and
aligns module name with project name.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
