pt/jrunner
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 5 Wiki Activity

Compare commits

base: pt:master
Branches Tags
pt:master
pt:rename-app-to-jrunner
pt:testread
pt:v1.0
pt:v0.41
pt:v0.40
pt:v0.38
pt:v0.36
pt:v0.35
pt:v0.34
pt:v0.33
pt:v0.32
pt:v0.31
pt:v0.30
pt:v0.29
pt:v0.28
..
compare: pt:testread
Branches Tags
pt:master
pt:rename-app-to-jrunner
pt:testread
pt:v1.0
pt:v0.41
pt:v0.40
pt:v0.38
pt:v0.36
pt:v0.35
pt:v0.34
pt:v0.33
pt:v0.32
pt:v0.31
pt:v0.30
pt:v0.29
pt:v0.28

No commits in common. "master" and "testread" have entirely different histories.
master ... testread

10 changed files with 18 additions and 304 deletions
Show Stats Expand all files Collapse all files

7
.gitignore vendored
View File

@ -5,10 +5,3 @@
build
*.swp
# Ignore local configuration files
run.yml
# Ignore accidentally extracted distribution files in project root
/bin/
/lib/

90
CLAUDE.md
View File

@ -1,90 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
jrunner is a Java CLI tool for migrating data between databases. It reads data from a source database using SQL queries and writes it to a destination table, batching inserts for performance. The tool supports multiple database types via JDBC drivers including PostgreSQL, IBM AS/400, and Microsoft SQL Server.
## Build and Test Commands
Build the project:
```bash
gradle build
# or use wrapper
./gradlew build
```
Run tests:
```bash
gradle test
# or use wrapper
./gradlew test
```
Build distribution package:
```bash
gradle build
# Creates jrunner/build/distributions/jrunner.zip
```
Deploy to /opt (as documented in readme.md):
```bash
sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
```
## Architecture
### Single-File Design
The entire application logic resides in `jrunner/src/main/java/jrunner/jrunner.java`. This is a monolithic command-line tool with no abstraction layers or separate modules.
### Data Flow
1. Parse command-line arguments (-scu, -scn, -scp for source; -dcu, -dcn, -dcp for destination)
2. Read SQL query from file specified by -sq flag
3. Connect to source and destination databases via JDBC
4. Execute source query and fetch results (fetch size: 10,000 rows)
5. Build batched INSERT statements (250 rows per batch)
6. Execute batches against destination table specified by -dt flag
7. Optionally clear target table before insert if -c flag is set
### Type Handling
The tool includes explicit handling for different SQL data types in a switch statement (lines 229-312). Supported types include VARCHAR, TEXT, CHAR, CLOB, DATE, TIME, TIMESTAMP, and BIGINT. String types get quote escaping and optional trimming.
### Database Drivers
JDBC drivers are configured in `jrunner/build.gradle`:
- PostgreSQL: org.postgresql:postgresql:42.5.0
- IBM AS/400 (JT400): net.sf.jt400:jt400:11.0
- Microsoft SQL Server: com.microsoft.sqlserver:mssql-jdbc:9.2.0.jre8
- SQL Server Integrated Auth: com.microsoft.sqlserver:mssql-jdbc_auth:9.2.0.x64
The AS/400 driver requires explicit Class.forName() registration (line 144).
## Configuration
The project uses a YAML configuration format (run.yml) to specify database connections, SQL script paths, and runtime options. However, the main application currently uses command-line arguments instead of parsing this YAML file.
Command-line flags:
- `-scu` - source JDBC URL
- `-scn` - source username
- `-scp` - source password
- `-dcu` - destination JDBC URL
- `-dcn` - destination username
- `-dcp` - destination password
- `-sq` - path to source SQL query file
- `-dt` - fully qualified destination table name
- `-t` - trim text fields (default: true)
- `-c` - clear target table before insert (default: true)
## Key Implementation Details
### Batch Size
INSERT statements are batched at 250 rows (hardcoded at line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
### Error Handling
SQLException handling prints stack trace and exits immediately with System.exit(0). There is no transaction rollback or partial failure recovery.
### Performance Considerations
- Result set fetch size is set to 10,000 rows (line 173)
- Progress counter prints with carriage return for real-time updates
- Timestamps captured at start (line 174) and end (line 368) for duration tracking

21
LICENSE
View File

@ -1,21 +0,0 @@
MIT License
Copyright (c) 2026 Paul Trowbridge
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

2
jrunner/build.gradle → app/build.gradle
View File

@ -26,8 +26,6 @@ dependencies {
//jdbc drivers
implementation 'org.postgresql:postgresql:42.5.0'
implementation 'net.sf.jt400:jt400:11.0'
implementation 'com.microsoft.sqlserver:mssql-jdbc:9.2.0.jre8'
implementation 'com.microsoft.sqlserver:mssql-jdbc_auth:9.2.0.x64'
}
application {

79
jrunner/src/main/java/jrunner/jrunner.java → app/src/main/java/jrunner/jrunner.java
View File

@ -4,7 +4,6 @@ import java.util.*;
import java.nio.file.Files;
import java.nio.file.Path ;
import java.nio.file.Paths;
import java.time.*;
public class jrunner {
//static final String QUERY = "SELECT * from rlarp.osm LIMIT 100";
@ -20,7 +19,6 @@ public class jrunner {
String sq = "";
String dt = "";
Boolean trim = true;
Boolean clear = true;
Integer r = 0;
Integer t = 0;
String sql = "";
@ -36,10 +34,8 @@ public class jrunner {
String[] getv = null;
Integer cols = null;
String[] dtn = null;
Timestamp tsStart = null;
Timestamp tsEnd = null;
msg = "jrunner version 1.0";
msg = "jrunner version 0.35";
msg = msg + nl + "-scu source jdbc url";
msg = msg + nl + "-scn source username";
msg = msg + nl + "-scp source passowrd";
@ -49,7 +45,6 @@ public class jrunner {
msg = msg + nl + "-sq path to source query";
msg = msg + nl + "-dt fully qualified name of destination table";
msg = msg + nl + "-t trim text";
msg = msg + nl + "-c clear target table";
msg = msg + nl + "--help info";
//---------------------------------------parse args into variables-------------------------------------------------
@ -65,7 +60,6 @@ public class jrunner {
scn = args[i+1];
break;
//source password
//import java.time.*;
case "-scp":
scp = args[i+1];
break;
@ -101,9 +95,6 @@ public class jrunner {
case "-t":
trim = true;
break;
case "-c":
clear = true;
break;
case "-v":
System.out.println(msg);
return;
@ -170,9 +161,6 @@ public class jrunner {
//----------------------------------------open resultset------------------------------------------------------------
try {
stmt = scon.createStatement();
stmt.setFetchSize(10000);
tsStart = Timestamp.from(Instant.now());
System.out.println(tsStart);
rs = stmt.executeQuery(sq);
//while (rs.next()) {
// System.out.println(rs.getString("x"));
@ -203,67 +191,39 @@ public class jrunner {
e.printStackTrace();
System.exit(0);
}
//-------------------------clear the target table if requeted----------------------------------------------------
if (clear) {
System.out.println("------------clear target table----------------------------");
sql = "DELETE FROM " + dt;
try {
stmtd = dcon.createStatement();
System.out.println(" " + sql);
stmtd.executeUpdate(sql);
} catch (SQLException e) {
e.printStackTrace();
System.out.println(sql);
System.exit(0);
}
}
System.out.println("------------row count-------------------------------------");
//-------------------------------build & execute sql-------------------------------------------------------------
try {
sql = "";
while (rs.next()) {
r++;
t++;
nr = "";
for (int i = 1; i <= cols; i++){
switch (dtn[i].toUpperCase()){
switch (dtn[i]){
case "VARCHAR":
nc = rs.getString(i);
nc = rs.getString(i).replace("'","''");
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
nc = nc.replaceAll("'","''");
if (trim) { nc = nc.trim();}
nc = "'" + nc + "'";
break;
case "TEXT":
nc = rs.getString(i);
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
nc = nc.replaceAll("'","''");
if (trim) { nc = nc.trim();}
nc = "'" + nc + "'";
break;
case "CHAR":
nc = rs.getString(i);
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
nc = nc.replaceAll("'","''");
if (trim) { nc = nc.trim();}
nc = "'" + nc + "'";
break;
case "CLOB":
nc = rs.getString(i);
nc = rs.getString(i).replace("'","''");
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
if (trim) { nc = nc.trim();}
nc = "'" + nc + "'";
break;
case "CHAR":
nc = rs.getString(i).replace("'","''");
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
nc = nc.replaceAll("'","''");
if (trim) { nc = nc.trim();}
nc = "'" + nc + "'";
break;
@ -279,21 +239,19 @@ public class jrunner {
}
break;
case "TIME":
nc = rs.getString(i);
nc = rs.getString(i).replace("'","''");
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
nc.replaceAll("'","''");
nc = "'" + nc + "'";
break;
case "TIMESTAMP":
nc = rs.getString(i);
nc = rs.getString(i).replace("'","''");
if (rs.wasNull() || nc == null) {
nc = "NULL";
break;
}
nc.replaceAll("'","''");
nc = "'" + nc + "'";
break;
case "BIGINT":
@ -365,11 +323,6 @@ public class jrunner {
e.printStackTrace();
}
System.out.println(" rows written");
tsEnd = Timestamp.from(Instant.now());
System.out.println(tsStart);
System.out.println(tsEnd);
//long time = Duration.between(tsStart, tsEnd).toMillis();
//System.out.println("time elapsed: " + time);
System.out.println();
System.out.println("");
}
}

0
jrunner/src/test/java/jrunner/AppTest.java → app/src/test/java/jrunner/AppTest.java
View File

42
deploy.sh
View File

@ -1,42 +0,0 @@
#!/bin/bash
set -e
DEPLOY_DIR="${1:-/opt/jrunner}"
# Prevent deleting critical system directories
case "${DEPLOY_DIR}" in
/|/bin|/boot|/dev|/etc|/lib|/lib64|/proc|/root|/run|/sbin|/sys|/usr|/var)
echo "Error: Cannot deploy to system directory: ${DEPLOY_DIR}"
exit 1
;;
esac
if [ ! -d "${DEPLOY_DIR}" ]; then
echo "Error: Directory does not exist: ${DEPLOY_DIR}"
echo "Create it first: sudo mkdir -p ${DEPLOY_DIR}"
exit 1
fi
echo "Building jrunner..."
./gradlew build
echo "Extracting to temporary location..."
sudo rm -rf /tmp/jrunner
sudo unzip -q jrunner/build/distributions/jrunner.zip -d /tmp/
echo "Deploying to ${DEPLOY_DIR}..."
sudo rm -rf "${DEPLOY_DIR}"/*
sudo mv /tmp/jrunner/* "${DEPLOY_DIR}"/
sudo rm -rf /tmp/jrunner
echo "Fixing ownership..."
sudo chown -R $USER:$USER "${DEPLOY_DIR}"
# Only create symlink for /opt/jrunner
if [ "${DEPLOY_DIR}" = "/opt/jrunner" ]; then
echo "Creating symlink..."
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
fi
echo "✅ Deployed to ${DEPLOY_DIR}"
echo "Run '${DEPLOY_DIR}/bin/jrunner --help' to test"

2
gradle/wrapper/gradle-wrapper.properties vendored
View File

@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

77
readme.md
View File

@ -1,77 +0,0 @@
## install java jdk.
find downloads page and get latest tarball.
https://www.oracle.com/java/technologies/downloads/
```
wget https://download.oracle.com/java/19/latest/jdk-19_linux-x64_bin.tar.gz
tar -xvf downloaded_file
```
move the extracted folder to /opt
put the extracted location in your path variable
```
export JAVA_HOME=/opt/jdk-19.0.1
export PATH=$PATH:$JAVA_HOME/bin
```
`java --version` to test
## install gradle (optional)
Gradle wrapper (`gradlew`) is included in the repo, so manual Gradle installation is not required.
If you prefer to install Gradle system-wide:
```
wget https://services.gradle.org/distributions/gradle-8.5-bin.zip
unzip -d /opt/gradle gradle-8.5-bin.zip
export PATH=$PATH:/opt/gradle/gradle-8.5/bin
gradle -v
```
## clone this repo
```
git clone https://gitea.hptrow.me/pt/jrunner.git
cd jrunner
```
## build
```
./gradlew build
```
## deploy
### using the deploy script (recommended)
First, create the deployment directory:
```
sudo mkdir -p /opt/jrunner
```
Then deploy:
```
# Deploy to /opt/jrunner (default, creates system-wide symlink)
./deploy.sh
# Deploy to custom location (for testing, no symlink)
sudo mkdir -p /opt/jrunner-test
./deploy.sh /opt/jrunner-test
```
The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails. When deploying to `/opt/jrunner`, it creates a symlink at `/usr/local/bin/jrunner`.
### manual deployment
```
./gradlew build
sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
```
## usage
After deployment to default location:
```
jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
```
After deployment to custom location:
```
/opt/jrunner-test/bin/jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
```

2
settings.gradle
View File

@ -8,4 +8,4 @@
*/
rootProject.name = 'jrunner'
include('jrunner')
include('app')