java data mover using jdbc
Go to file
Paul Trowbridge 8d7797b769 Walk past update counts to find a procedure's result set
/opt/sync uses CALL statements as migration sources — rlarp.QUOTE_REBUILD,
SB_UD_R2, SB_GJ_R1 — and a stored procedure may report update counts
before it opens its cursor. The previous commit concluded "no result set"
on the first false from execute(), which would have silently migrated
nothing for such a procedure.

Now the canonical JDBC walk: step through update counts via
getMoreResults() until a result set appears, and only then treat the
statement as producing none. The last update count is retained so plain
DML still reports rows affected.

This is strictly more capable than the executeQuery() it replaced, which
threw outright in this situation. Verified against the Postgres analogue
(UPDATE followed by SELECT in one batch): the SELECT's rows come through,
pure DML still reports its count, plain SELECT is unaffected. The DB2
procedures themselves were not invoked — they have production side effects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 13:00:21 -04:00
gradle/wrapper upgrade gradle wrapper to 8.5 for java 20 compatibility 2026-01-06 21:42:36 -05:00
jrunner Walk past update counts to find a procedure's result set 2026-08-11 13:00:21 -04:00
.gitattributes initial 2022-10-20 11:12:19 -04:00
.gitignore gitignore accidentally extracted distribution files 2026-01-06 22:20:22 -05:00
CLAUDE.md docs: -b now covers Postgres (COPY) as well as SQL Server 2026-06-18 23:16:00 -04:00
deploy.sh Fix global install to use installDist instead of wiping project dir 2026-03-02 22:01:51 -05:00
gradlew initial 2022-10-20 11:12:19 -04:00
gradlew.bat initial 2022-10-20 11:12:19 -04:00
jrq Add jrq query wrapper script and deploy integration 2026-02-26 06:40:44 -05:00
LICENSE add MIT license 2026-01-06 22:44:47 -05:00
readme.md docs: -b now covers Postgres (COPY) as well as SQL Server 2026-06-18 23:16:00 -04:00
settings.gradle rename app module to jrunner for consistency 2026-01-06 21:53:08 -05:00
setup.sh Add automated setup script for new installations 2026-01-16 11:01:26 -05:00

Quick Start

The easiest way to get started on a new system:

git clone https://gitea.hptrow.me/pt/jrunner.git
cd jrunner
./setup.sh

The setup script will:

  • Check for Java 11+ (offers to install if missing)
  • Verify Gradle wrapper is present
  • Run a test build to ensure everything works
  • Show you next steps

Manual Installation

If you prefer to install dependencies manually:

Install Java JDK

Option 1: Package manager (recommended)

# Ubuntu/Debian
sudo apt update && sudo apt install openjdk-17-jdk

# Fedora/RHEL
sudo dnf install java-17-openjdk-devel

# Arch
sudo pacman -S jdk-openjdk

Option 2: Manual download Download from https://www.oracle.com/java/technologies/downloads/

wget https://download.oracle.com/java/19/latest/jdk-19_linux-x64_bin.tar.gz
tar -xvf jdk-19_linux-x64_bin.tar.gz
sudo mv jdk-19.0.1 /opt/
export JAVA_HOME=/opt/jdk-19.0.1
export PATH=$PATH:$JAVA_HOME/bin

Test: java --version

Install Gradle (optional)

Gradle wrapper (gradlew) is included in the repo, so manual Gradle installation is not required.

build

./gradlew build

deploy

Run the interactive deploy script:

./deploy.sh

The script will prompt you to choose:

  1. Local install - Fast, no sudo required, installs to ./jrunner/build/install/jrunner
  2. Global install - Installs to /opt/jrunner with symlink at /usr/local/bin/jrunner
  3. Custom directory - Prompts for path with tab-completion support

The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails.

manual deployment

./gradlew build
sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner

Or for local testing:

./gradlew installDist
# Binary at: ./jrunner/build/install/jrunner/bin/jrunner

usage

Query Mode (new in v1.1)

Query mode outputs results to stdout for piping to visidata, pspg, or less. It activates automatically when destination flags are omitted.

Basic query to CSV:

jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql

Pipe to visidata:

jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql | visidata -f csv

TSV format:

jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql -f tsv

SQL Server example:

jrunner -scu "jdbc:sqlserver://hostname:1433;databaseName=mydb" -scn user -scp pass -sq query.sql

PostgreSQL example:

jrunner -scu "jdbc:postgresql://hostname:5432/dbname" -scn user -scp pass -sq query.sql

jrq - Query Mode Wrapper

The jrq wrapper script simplifies query mode usage by storing connection details in a config file, eliminating the need to type credentials repeatedly.

Setup:

Create ~/.jrqrc with your database connection details:

JR_URL="jdbc:as400://s7830956"
JR_USER="myusername"
JR_PASS="mypassword"

Usage:

# Output to stdout
jrq query.sql

# Pipe to visidata
jrq query.sql | vd

# Use TSV format
jrq query.sql tsv

# Pipe TSV to less with column display
jrq query.sql tsv | less -S

Configuration options in ~/.jrqrc:

  • JR_URL - JDBC connection URL (required)
  • JR_USER - Database username (required)
  • JR_PASS - Database password (required)
  • JR_BIN - Path to jrunner binary (default: /opt/jrunner/jrunner/build/install/jrunner/bin/jrunner)
  • JR_FORMAT - Default output format: csv or tsv (default: csv)

Note: You can also set these as environment variables instead of using a config file:

export JR_URL="jdbc:as400://hostname"
export JR_USER="username"
export JR_PASS="password"
jrq query.sql | vd

Migration Mode

Full migration mode with both source and destination:

jrunner -scu jdbc:postgresql://source:5432/sourcedb \
        -scn sourceuser \
        -scp sourcepass \
        -dcu jdbc:postgresql://dest:5432/destdb \
        -dcn destuser \
        -dcp destpass \
        -sq query.sql \
        -dt public.target_table

Command-line flags

Source connection:

  • -scu - source JDBC URL
  • -scn - source username
  • -scp - source password
  • -sq - path to source SQL query file

Destination connection (migration mode only):

  • -dcu - destination JDBC URL
  • -dcn - destination username
  • -dcp - destination password
  • -dt - fully qualified destination table name

Options:

  • -t - trim text fields (default: true)
  • -c - clear target table before insert (default: true)
  • -b - bulk load into the destination instead of batched INSERTs — far faster on large/wide tables. SQL Server: TDS bulk-load via SQLServerBulkCopy. Postgres: COPY FROM STDIN. Other dests (e.g. DB2) fall back to INSERT. (migration mode only)
  • -f - output format: csv, tsv (query mode only, default: csv)