java data mover using jdbc
Go to file
Paul Trowbridge c0f6e3a6e6 Document streaming architecture and memory usage
Clarify that both query and migration modes use streaming with no array
storage. Query mode streams directly to stdout, while migration mode
streams into a SQL string buffer (250 rows). The 10k fetch size is a
JDBC driver hint for network efficiency, not application-level storage.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-25 14:28:26 -05:00
gradle/wrapper upgrade gradle wrapper to 8.5 for java 20 compatibility 2026-01-06 21:42:36 -05:00
jrunner Update readme, CLAUDE.md, and bump version to 1.1 2026-01-16 10:58:37 -05:00
.gitattributes initial 2022-10-20 11:12:19 -04:00
.gitignore gitignore accidentally extracted distribution files 2026-01-06 22:20:22 -05:00
CLAUDE.md Document streaming architecture and memory usage 2026-01-25 14:28:26 -05:00
deploy.sh Enable tab completion for custom directory input 2026-01-16 10:54:41 -05:00
gradlew initial 2022-10-20 11:12:19 -04:00
gradlew.bat initial 2022-10-20 11:12:19 -04:00
LICENSE add MIT license 2026-01-06 22:44:47 -05:00
readme.md Add automated setup script for new installations 2026-01-16 11:01:26 -05:00
settings.gradle rename app module to jrunner for consistency 2026-01-06 21:53:08 -05:00
setup.sh Add automated setup script for new installations 2026-01-16 11:01:26 -05:00

Quick Start

The easiest way to get started on a new system:

git clone https://gitea.hptrow.me/pt/jrunner.git
cd jrunner
./setup.sh

The setup script will:

  • Check for Java 11+ (offers to install if missing)
  • Verify Gradle wrapper is present
  • Run a test build to ensure everything works
  • Show you next steps

Manual Installation

If you prefer to install dependencies manually:

Install Java JDK

Option 1: Package manager (recommended)

# Ubuntu/Debian
sudo apt update && sudo apt install openjdk-17-jdk

# Fedora/RHEL
sudo dnf install java-17-openjdk-devel

# Arch
sudo pacman -S jdk-openjdk

Option 2: Manual download Download from https://www.oracle.com/java/technologies/downloads/

wget https://download.oracle.com/java/19/latest/jdk-19_linux-x64_bin.tar.gz
tar -xvf jdk-19_linux-x64_bin.tar.gz
sudo mv jdk-19.0.1 /opt/
export JAVA_HOME=/opt/jdk-19.0.1
export PATH=$PATH:$JAVA_HOME/bin

Test: java --version

Install Gradle (optional)

Gradle wrapper (gradlew) is included in the repo, so manual Gradle installation is not required.

build

./gradlew build

deploy

Run the interactive deploy script:

./deploy.sh

The script will prompt you to choose:

  1. Local install - Fast, no sudo required, installs to ./jrunner/build/install/jrunner
  2. Global install - Installs to /opt/jrunner with symlink at /usr/local/bin/jrunner
  3. Custom directory - Prompts for path with tab-completion support

The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails.

manual deployment

./gradlew build
sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner

Or for local testing:

./gradlew installDist
# Binary at: ./jrunner/build/install/jrunner/bin/jrunner

usage

Query Mode (new in v1.1)

Query mode outputs results to stdout for piping to visidata, pspg, or less. It activates automatically when destination flags are omitted.

Basic query to CSV:

jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql

Pipe to visidata:

jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql | visidata -f csv

TSV format:

jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql -f tsv

SQL Server example:

jrunner -scu "jdbc:sqlserver://hostname:1433;databaseName=mydb" -scn user -scp pass -sq query.sql

PostgreSQL example:

jrunner -scu "jdbc:postgresql://hostname:5432/dbname" -scn user -scp pass -sq query.sql

Migration Mode

Full migration mode with both source and destination:

jrunner -scu jdbc:postgresql://source:5432/sourcedb \
        -scn sourceuser \
        -scp sourcepass \
        -dcu jdbc:postgresql://dest:5432/destdb \
        -dcn destuser \
        -dcp destpass \
        -sq query.sql \
        -dt public.target_table

Command-line flags

Source connection:

  • -scu - source JDBC URL
  • -scn - source username
  • -scp - source password
  • -sq - path to source SQL query file

Destination connection (migration mode only):

  • -dcu - destination JDBC URL
  • -dcn - destination username
  • -dcp - destination password
  • -dt - fully qualified destination table name

Options:

  • -t - trim text fields (default: true)
  • -c - clear target table before insert (default: true)
  • -f - output format: csv, tsv (query mode only, default: csv)