/opt/sync uses CALL statements as migration sources — rlarp.QUOTE_REBUILD,
SB_UD_R2, SB_GJ_R1 — and a stored procedure may report update counts
before it opens its cursor. The previous commit concluded "no result set"
on the first false from execute(), which would have silently migrated
nothing for such a procedure.
Now the canonical JDBC walk: step through update counts via
getMoreResults() until a result set appears, and only then treat the
statement as producing none. The last update count is retained so plain
DML still reports rows affected.
This is strictly more capable than the executeQuery() it replaced, which
threw outright in this situation. Verified against the Postgres analogue
(UPDATE followed by SELECT in one batch): the SELECT's rows come through,
pure DML still reports its count, plain SELECT is unaffected. The DB2
procedures themselves were not invoked — they have production side effects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
json and table were advertised in --help but both fell through to
outputCSV, so -f json silently produced CSV. table is now dropped from
the help text rather than advertised and unimplemented; json is real.
Two things CSV cannot express, both of which callers need:
* NULL vs empty string. outputCSV writes an empty field for both, so
the merge, reconcile and the wizard all see them as identical. JSON
emits null and "" distinctly.
* Column types. The wizard's SQL-entry mode defaults every dest column
to text purely because CSV carries no type metadata; the header now
reports name, type, precision and scale (verified against DB2 for i:
CHAR precision=5, DECIMAL precision=7 scale=2).
Values are JSON strings, never JSON numbers — a DECIMAL rendered as a
number would pass through a float and lose exactness. Output is written
incrementally so a large result set streams as CSV does while remaining
one valid document. Zero-row results still report their columns, which is
what query-column introspection relies on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two coupled changes: the summary is what a caller should read instead of
parsing prose, and the statement fix is what makes a non-zero exit code
usable at all.
Summary: emit one "@summary {json}" line carrying status, row count and
elapsed ms. Deliberately on stderr — migration-mode stdout carries the
progress output pipekit streams to its live log, and query-mode stdout
must stay pure CSV, so structured data on stdout would corrupt one or the
other. Previously the row count reached the caller only because a \r
progress tick and a trailing " rows written" happened to render on the
same line, which pipekit then matched with three different regexes.
Statements: query mode used executeQuery(), which raises when a statement
returns no result set — Postgres "No results were returned by the query",
SQL Server "The statement did not return a result set". DDL, INSERT,
DELETE and TRUNCATE all hit this despite succeeding, so pipekit carries a
_BENIGN_EXCEPTION_SUBSTRINGS allowlist to tell a working TRUNCATE from a
real failure. That made --strict unusable: it would have failed every
merge, staging DDL and hook. Now execute() + getUpdateCount() treats them
as the successes they are and reports rows affected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every error path did printStackTrace() then System.exit(0), making a
failed run indistinguishable from a successful one to any caller checking
$?. pipekit works around this by grepping stdout for stack-trace text
(_detect_silent_failure); 112 /opt/sync scripts run under `set -e` and
cannot detect a jrunner failure at all.
All 14 error exits now route through die(), which honours --strict.
Opt-in rather than default: flipping it unconditionally would change the
behaviour of those 112 scripts at once. --strict is pre-scanned from argv
so it applies to failures during argument parsing too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
setup.sh hardcoded openjdk-17 and only ever checked `java -version`, so it
passed its own Java check on a box whose JAVA_HOME pointed at a JDK that had
been removed -- then died at `./gradlew --version` with the real cause
(JAVA_HOME is set to an invalid directory) swallowed by a /dev/null redirect.
- latest_installable_jdk() asks apt/dnf/pacman for the highest openjdk-N-jdk
within [MIN_JAVA, MAX_JAVA]. MAX_JAVA=21 is the ceiling the gradle-8.5
wrapper can run on; raise it alongside gradle-wrapper.properties.
- install path uses that version instead of a hardcoded 17, and an installed
but older JDK now prompts for an upgrade rather than passing silently.
- fix_java_home() repoints JAVA_HOME at the JDK behind the java on PATH when
the inherited one has no bin/java, and prints the permanent fix.
- version parser tolerates "Picked up _JAVA_OPTIONS" preamble and 1.8.0_x.
- missing git is fatal instead of being recorded and ignored; gradle's own
error output survives a wrapper failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011thcefjGXbPwcgLmRqFWbb
-scp/-dcp put passwords on argv, where any local user can read them with
ps. This was not theoretical: a running migration was observed exposing
both an AS/400 and a Postgres password in plaintext.
-sc/-dc aliases already avoided that, but only read ~/.jrunnerpass, and
the pipekit service account is created with --no-create-home, so it had
no way to use them. --passfile points at an arbitrary path.
Also warn when the passfile is world-readable, and report the resolved
path in errors instead of a hardcoded ~/.jrunnerpass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Update readme + CLAUDE: -b is no longer SQL-Server-only. Describe the Postgres
COPY FROM STDIN path (CopyManager, text-based, CSV-quoted, empty vs NULL) next
to the existing SQL Server SQLServerBulkCopy path; DB2 still falls back to INSERT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends -b to Postgres destinations: stream the source ResultSet into PG with
COPY <dt> FROM STDIN (FORMAT csv) via the JDBC CopyManager, instead of batched
INSERTs. COPY is text-based so the server parses each field into the column
type — no per-type quoting needed. Every non-null value is CSV-quoted (so
empty string stays distinct from NULL, which is an empty unquoted field);
rows are flushed in 1000-row buffers with a 10k-row progress counter.
Validated DB2->PG: numeric precision (123.4567), jsonb, unicode, embedded
quotes, NULL vs empty-string all correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the -b flag to the readme/CLAUDE flag lists and describe the bulk copy
migration sub-mode: SQLServerBulkCopy via the BulkSource adapter (SQL Server
dest only), why numeric/string-ish types route through NVARCHAR, the ~111min
-> ~4min win, and the 10k-row live progress counter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bulk path showed no progress during a load (only the final count). Emit
an in-place counter (\r + rows) every 10k rows from the BulkSource adapter,
which the caller pulls one row at a time, so it streams live. Prefix the
final count print with \r so it starts a fresh line instead of concatenating
onto the last tick (which produced a garbage row count like 3000035000).
Verified: ticks emit at 10k/20k/30k, final row_count parses correctly, and
pipekit's progress-collapse renders it as a single updating line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bulk path printed no count, so the trailing " rows written" line had no
number and callers parsing stdout got nothing. Count rows in the BulkSource
adapter (one per getRowData) and print it, matching the INSERT path's
"<n> rows written" so the count is captured.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an opt-in -b flag (migration mode, SQL Server dest only) that streams the
source ResultSet straight into SQL Server over the TDS bulk-load protocol
instead of 250-row INSERT...VALUES round trips. A BulkSource adapter
(ISQLServerBulkData) maps PG source types to JDBC types we control: string-ish
types (text/varchar/char/bpchar/json/jsonb/uuid/numeric) go through NVARCHAR via
getString so SQL Server converts losslessly — notably numeric, since PG reports
unconstrained numeric as scale 0 which made a typed DECIMAL path round
(123.45 -> 123). Default stays the INSERT path, so nothing regresses.
Validated against live PG->SQL Server: int4/text/jsonb/numeric/date plus nulls,
unicode, quotes, and numeric precision (123.45, 0.123456) all correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reflect the two behavior changes: (1) migration mode sets the source
connection to autoCommit=false so PostgreSQL's setFetchSize actually streams
(it's ignored otherwise) — and why query mode is excluded; (2) json/jsonb/
bpchar/uuid are now quoted, plus document the default-emits-unquoted gotcha
for future type additions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The migration INSERT builder's switch quoted varchar/text/char/clob/date/time
but let everything else fall to a default that emits rs.getString() unquoted
(correct for numerics, broken for strings). A pg->SQL Server pull of a jsonb
column failed with "Incorrect syntax near 'volume_bucket'" — the JSON text's
embedded double-quotes were read as a SQL identifier. Quote json/jsonb, plus
bpchar (PG char(n)) and uuid, like varchar.
Note: the default case still emits unquoted; other unhandled string types
(e.g. bool->'t'/'f') would need similar handling or a quote-by-default flip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
setFetchSize(10000) is a no-op on the PostgreSQL JDBC driver while autoCommit
is true — the driver loads the entire ResultSet into memory, OOM/GC-thrashing
on large source tables (a pg->SQL Server pull pinned the box: 4GB heap, swap
full, 0 rows written). PG only uses a server-side cursor when autoCommit is
false AND fetchSize > 0.
Set the source connection to manual commit ONLY in migration mode: the
migration source is read-only so never committing is harmless. Query mode is
excluded on purpose — callers (pipekit's run_dest_sql) run committed DDL/DML
through query mode, and autoCommit=false would roll those back on close.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds DATETIME, DATETIME2, SMALLDATETIME, and DATETIMEOFFSET cases to
the TIMESTAMP branch so SQL Server datetime columns are handled correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements .pgpass-style credential file for jrunner. Named aliases can
be used with -sc and -dc flags instead of spelling out -scu/-scn/-scp
for each invocation. Explicit flags still take priority over the file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Option 2 now runs installDist and symlinks /usr/local/bin/jrunner and
/usr/local/bin/jrq to the build output, avoiding the rm -rf of the
project directory when the repo lives at the deploy path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
deploy.sh now generates a commented ~/.jrqrc template on first install
so users know what to fill in. Skips creation if the file already exists
to avoid overwriting existing credentials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds jrq, a convenience wrapper that loads connection details from
~/.jrqrc so users can run query mode without typing credentials each
time. deploy.sh now installs jrq alongside jrunner in all deploy modes,
with a /usr/local/bin symlink for global installs. jrq resolves the
jrunner binary relative to its own location to avoid hardcoded paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clarify that both query and migration modes use streaming with no array
storage. Query mode streams directly to stdout, while migration mode
streams into a SQL string buffer (250 rows). The 10k fetch size is a
JDBC driver hint for network efficiency, not application-level storage.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Create setup.sh that checks for and installs dependencies:
- Detects Java 11+ or offers to install via package manager
- Verifies Gradle wrapper presence
- Checks for unzip (needed for deployment)
- Runs test build to verify everything works
- Provides clear next steps after successful setup
Update readme with Quick Start section featuring setup script as the recommended approach for new systems.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Use read -e to enable readline support, providing tab completion and line editing when entering custom deployment paths. Also expand tilde to home directory.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove all diagnostic output in query mode - no front matter, timestamps, or metadata. Query results go directly to stdout for seamless piping to visidata/pspg/less.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Query mode auto-activates when destination flags are omitted, outputting CSV/TSV to stdout for interactive data exploration of DB2 iSeries queries.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add chown to set deployed files to current user instead of leaving
them owned by root. This matches the original copy_to_apt.sh behavior.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Readme changes:
- Document that deployment directory must exist first
- Show mkdir -p commands before deploy
- Explain atomic deployment behavior (extracts to /tmp first)
Version bump to 1.0:
- Major refactoring: renamed app to jrunner
- Simplified deployment script
- Updated documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Completely rewrote deploy script to be simpler and safer:
- Requires directory to exist (no automatic creation)
- Builds and extracts to /tmp/ FIRST
- Only clears target directory after build/extract succeeds
- If build fails, existing deployment stays untouched
Usage:
sudo mkdir -p /opt/jr_test
./deploy.sh /opt/jr_test
This atomic approach prevents broken deployments from failed builds.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Previous version failed when /opt/jrunner existed from a prior run,
because mv cannot move over an existing directory. Now the script:
1. Removes target directory if it exists
2. Removes intermediate jrunner/ directory if it exists and differs
from target (prevents collision)
3. Extracts cleanly
4. Renames to target name
This fixes the issue where ./deploy.sh /opt/jr_test would fail if
/opt/jrunner already existed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Prevent bin/ and lib/ in project root from showing up in git if the
deploy script accidentally extracts to the wrong location.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The zip file contains 'jrunner/' as top-level directory, so unzipping
always created /opt/jrunner regardless of custom path. Now the script
renames the directory after unzipping if a custom name was specified.
Example:
./deploy.sh /opt/jr_test
Now correctly creates /opt/jr_test (not /opt/jrunner).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Prevent dangerous sudo rm -rf operations:
- Check that DEPLOY_DIR is not empty
- Block deployment to critical system directories (/, /usr, /etc, etc.)
- Only remove directory if it already exists
- Show message before removal for clarity
This prevents catastrophic mistakes like accidentally deploying to /
or deleting important system directories.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add mkdir -p to create parent directory if it doesn't exist. This allows
deploying to any path without requiring manual directory creation.
Example:
./deploy.sh /home/user/testing/jrunner
Now works even if /home/user/testing doesn't exist yet.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add clear documentation showing:
- deploy.sh with no arguments uses default /opt/jrunner location
- deploy.sh with argument deploys to custom location for testing
- Explain symlink behavior (only for default location)
- Show usage examples for both deployment types
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Allow specifying custom deployment directory as argument, defaulting to
/opt/jrunner if not provided. Symlink to /usr/local/bin only created
for default location to avoid overwriting production.
Usage:
./deploy.sh # deploys to /opt/jrunner (default)
./deploy.sh /opt/jrunner-test # test deployment
This allows testing new builds without affecting production deployment.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Renamed copy_to_apt.sh to deploy.sh (clearer name) and updated to:
- Use new jrunner/ paths instead of app/
- Add build step so script handles full build+deploy
- Create symlink to /usr/local/bin for system-wide access
- Remove unused JR environment variable export
- Add set -e for error handling
- Add progress messages
Usage: ./deploy.sh
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changes:
- Rename app/ directory to jrunner/ (preserves git history)
- Update settings.gradle to reference jrunner module
- Update readme.md with new paths (jrunner/build/, /opt/jrunner)
- Update CLAUDE.md documentation with new file paths
Build outputs now named jrunner.zip, jrunner.jar, bin/jrunner instead
of generic "app" names. This makes the project structure clearer and
aligns module name with project name.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add run.yml to gitignore as it's a personal config file for local testing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Gradle 7.5.1 only supports up to Java 18. Upgraded to 8.5 to support
Java 20 and fix "Unsupported class file major version 64" build error.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Documents build commands, architecture, and implementation details to help
Claude Code instances understand the codebase structure and data flow.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
