Compare commits

...

5 Commits

Author SHA1 Message Date
ba24b874fc Add automated setup script for new installations
Create setup.sh that checks for and installs dependencies:
- Detects Java 11+ or offers to install via package manager
- Verifies Gradle wrapper presence
- Checks for unzip (needed for deployment)
- Runs test build to verify everything works
- Provides clear next steps after successful setup

Update readme with Quick Start section featuring setup script as the recommended approach for new systems.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-16 11:01:26 -05:00
424d7d4ebb Update readme, CLAUDE.md, and bump version to 1.1
- Document query mode feature with examples
- Update deploy script documentation
- Add dual mode operation explanation to CLAUDE.md
- Document CSV/TSV output formats
- Update version from 1.0 to 1.1

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-16 10:58:37 -05:00
56fecff550 Enable tab completion for custom directory input
Use read -e to enable readline support, providing tab completion and line editing when entering custom deployment paths. Also expand tilde to home directory.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-16 10:54:41 -05:00
1ca507d1dd Make deploy script interactive with local/global/custom options
- Option 1: Local install (uses gradlew installDist)
- Option 2: Global install to /opt/jrunner with symlink
- Option 3: Custom directory (mandatory user input)
- Removed default directory, now requires explicit choice

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-16 10:51:58 -05:00
1717c7ee2c Make query mode silent for clean piping to pagers
Remove all diagnostic output in query mode - no front matter, timestamps, or metadata. Query results go directly to stdout for seamless piping to visidata/pspg/less.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-15 14:05:21 -05:00
5 changed files with 429 additions and 99 deletions

View File

@ -28,10 +28,16 @@ gradle build
# Creates jrunner/build/distributions/jrunner.zip
```
Deploy to /opt (as documented in readme.md):
Local install for testing (recommended):
```bash
sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
./gradlew installDist
# Creates executable at jrunner/build/install/jrunner/bin/jrunner
```
Deploy using interactive script:
```bash
./deploy.sh
# Choose: 1) Local install, 2) Global install to /opt, 3) Custom directory
```
## Architecture
@ -39,7 +45,33 @@ sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
### Single-File Design
The entire application logic resides in `jrunner/src/main/java/jrunner/jrunner.java`. This is a monolithic command-line tool with no abstraction layers or separate modules.
### Dual Mode Operation (v1.1+)
The tool operates in two modes:
**Query Mode** (new in v1.1):
- Activates automatically when destination flags are not provided
- Outputs query results to stdout in CSV or TSV format
- Silent operation - no diagnostic output, just clean data
- Designed for piping to visidata, pspg, less, or other data tools
- Format controlled by -f flag (csv or tsv)
**Migration Mode** (original functionality):
- Activates when destination flags are provided
- Reads from source, writes to destination with batched INSERTs
- Shows progress counters and timing information
### Data Flow
**Query Mode:**
1. Parse command-line arguments (-scu, -scn, -scp for source)
2. Read SQL query from file specified by -sq flag
3. Connect to source database via JDBC
4. Execute source query and fetch results (fetch size: 10,000 rows)
5. Output results to stdout in CSV or TSV format
6. Close connection and exit
**Migration Mode:**
1. Parse command-line arguments (-scu, -scn, -scp for source; -dcu, -dcn, -dcp for destination)
2. Read SQL query from file specified by -sq flag
3. Connect to source and destination databases via JDBC
@ -68,23 +100,36 @@ Command-line flags:
- `-scu` - source JDBC URL
- `-scn` - source username
- `-scp` - source password
- `-dcu` - destination JDBC URL
- `-dcn` - destination username
- `-dcp` - destination password
- `-dcu` - destination JDBC URL (migration mode only)
- `-dcn` - destination username (migration mode only)
- `-dcp` - destination password (migration mode only)
- `-sq` - path to source SQL query file
- `-dt` - fully qualified destination table name
- `-dt` - fully qualified destination table name (migration mode only)
- `-t` - trim text fields (default: true)
- `-c` - clear target table before insert (default: true)
- `-c` - clear target table before insert (default: true, migration mode only)
- `-f` - output format: csv, tsv (query mode only, default: csv)
## Key Implementation Details
### Batch Size
INSERT statements are batched at 250 rows (hardcoded at line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
### Mode Detection
Query mode is automatically detected at runtime (line 131) by checking if all destination flags (dcu, dcn, dcp, dt) are empty. This allows seamless switching between query and migration modes without explicit mode flags.
### Query Mode Output (v1.1+)
Query mode uses dedicated output methods:
- `outputQueryResults()` - Dispatches to format-specific methods
- `outputCSV()` - RFC 4180 compliant CSV with proper quote escaping
- `outputTSV()` - Tab-separated with tabs/newlines replaced by spaces
- All output goes to stdout; no diagnostic messages in query mode
- Helper methods: `escapeCSV()` and `escapeTSV()` for proper formatting
### Batch Size (Migration Mode)
INSERT statements are batched at 250 rows (hardcoded around line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
### Error Handling
SQLException handling prints stack trace and exits immediately with System.exit(0). There is no transaction rollback or partial failure recovery.
### Performance Considerations
- Result set fetch size is set to 10,000 rows (line 173)
- Progress counter prints with carriage return for real-time updates
- Timestamps captured at start (line 174) and end (line 368) for duration tracking
- Result set fetch size is set to 10,000 rows (line 190)
- Progress counter prints with carriage return for real-time updates (migration mode only)
- Timestamps captured at start and end for duration tracking (migration mode only)
- Query mode has no progress output to keep stdout clean for piping

View File

@ -1,7 +1,40 @@
#!/bin/bash
set -e
DEPLOY_DIR="${1:-/opt/jrunner}"
echo "jrunner deployment script"
echo "========================="
echo ""
echo "Select deployment option:"
echo " 1) Local install (./jrunner/build/install)"
echo " 2) Global install (/opt/jrunner)"
echo " 3) Custom directory"
echo ""
read -p "Enter choice [1-3]: " choice
case $choice in
1)
DEPLOY_MODE="local"
DEPLOY_DIR="./jrunner/build/install/jrunner"
;;
2)
DEPLOY_MODE="global"
DEPLOY_DIR="/opt/jrunner"
;;
3)
DEPLOY_MODE="custom"
read -e -p "Enter deployment directory (required): " DEPLOY_DIR
if [ -z "$DEPLOY_DIR" ]; then
echo "Error: Directory path is required"
exit 1
fi
# Expand tilde to home directory
DEPLOY_DIR="${DEPLOY_DIR/#\~/$HOME}"
;;
*)
echo "Error: Invalid choice"
exit 1
;;
esac
# Prevent deleting critical system directories
case "${DEPLOY_DIR}" in
@ -11,32 +44,46 @@ case "${DEPLOY_DIR}" in
;;
esac
if [ ! -d "${DEPLOY_DIR}" ]; then
echo "Error: Directory does not exist: ${DEPLOY_DIR}"
echo "Create it first: sudo mkdir -p ${DEPLOY_DIR}"
exit 1
fi
echo ""
echo "Building jrunner..."
./gradlew build
echo "Extracting to temporary location..."
sudo rm -rf /tmp/jrunner
sudo unzip -q jrunner/build/distributions/jrunner.zip -d /tmp/
if [ "$DEPLOY_MODE" = "local" ]; then
echo "Installing locally with gradle..."
./gradlew installDist
echo ""
echo "✅ Installed locally at: ${DEPLOY_DIR}"
echo "Run './jrunner/build/install/jrunner/bin/jrunner --help' to test"
else
# Global or custom deployment
if [ ! -d "${DEPLOY_DIR}" ]; then
echo "Creating directory: ${DEPLOY_DIR}"
sudo mkdir -p "${DEPLOY_DIR}"
fi
echo "Deploying to ${DEPLOY_DIR}..."
sudo rm -rf "${DEPLOY_DIR}"/*
sudo mv /tmp/jrunner/* "${DEPLOY_DIR}"/
sudo rm -rf /tmp/jrunner
echo "Extracting to temporary location..."
sudo rm -rf /tmp/jrunner-deploy
sudo unzip -q jrunner/build/distributions/jrunner.zip -d /tmp/jrunner-deploy
echo "Fixing ownership..."
sudo chown -R $USER:$USER "${DEPLOY_DIR}"
echo "Deploying to ${DEPLOY_DIR}..."
sudo rm -rf "${DEPLOY_DIR}"/*
sudo mv /tmp/jrunner-deploy/jrunner/* "${DEPLOY_DIR}"/
sudo rm -rf /tmp/jrunner-deploy
# Only create symlink for /opt/jrunner
if [ "${DEPLOY_DIR}" = "/opt/jrunner" ]; then
echo "Creating symlink..."
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
echo "Fixing ownership..."
sudo chown -R $USER:$USER "${DEPLOY_DIR}"
# Create symlink for global install
if [ "$DEPLOY_MODE" = "global" ]; then
echo "Creating symlink at /usr/local/bin/jrunner..."
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
fi
echo ""
echo "✅ Deployed to ${DEPLOY_DIR}"
echo "Run '${DEPLOY_DIR}/bin/jrunner --help' to test"
if [ "$DEPLOY_MODE" = "global" ]; then
echo "Or simply: jrunner --help"
fi
fi
echo "✅ Deployed to ${DEPLOY_DIR}"
echo "Run '${DEPLOY_DIR}/bin/jrunner --help' to test"

View File

@ -41,7 +41,7 @@ public class jrunner {
Timestamp tsStart = null;
Timestamp tsEnd = null;
msg = "jrunner version 1.0";
msg = "jrunner version 1.1";
msg = msg + nl + "-scu source jdbc url";
msg = msg + nl + "-scn source username";
msg = msg + nl + "-scp source passowrd";
@ -136,12 +136,7 @@ public class jrunner {
// Detect query mode when destination flags are not provided
queryMode = dcu.isEmpty() && dcn.isEmpty() && dcp.isEmpty() && dt.isEmpty();
if (queryMode) {
System.err.println("------------query mode------------------------------------");
System.err.println("source db uri: " + scu);
System.err.println("source db username: " + scn);
System.err.println("output format: " + outputFormat);
} else {
if (!queryMode) {
System.out.println("------------db info---------------------------------------");
System.out.println("source db uri: " + scu);
System.out.println("source db username: " + scn);
@ -165,9 +160,7 @@ public class jrunner {
//-------------------------------------------establish connections-------------------------------------------------
//source database
if (queryMode) {
System.err.println("------------open database connection---------------------");
} else {
if (!queryMode) {
System.out.println("------------open database connections---------------------");
}
try {
@ -176,8 +169,10 @@ public class jrunner {
System.out.println("issue connecting to source:");
e.printStackTrace();
System.exit(0);
}
System.out.println(" ✅ source database");
}
if (!queryMode) {
System.out.println(" ✅ source database");
}
//destination database
if (!queryMode) {
try {
@ -194,7 +189,9 @@ public class jrunner {
stmt = scon.createStatement();
stmt.setFetchSize(10000);
tsStart = Timestamp.from(Instant.now());
System.out.println(tsStart);
if (!queryMode) {
System.out.println(tsStart);
}
rs = stmt.executeQuery(sq);
//while (rs.next()) {
// System.out.println(rs.getString("x"));
@ -208,8 +205,10 @@ public class jrunner {
//---------------------------------------build meta---------------------------------------------------------------
try {
cols = rs.getMetaData().getColumnCount();
System.out.println("------------source metadata-------------------------------");
System.out.println("number of cols: " + cols);
if (!queryMode) {
System.out.println("------------source metadata-------------------------------");
System.out.println("number of cols: " + cols);
}
getv = new String[cols + 1];
dtn = new String[cols + 1];
} catch (SQLException e) {
@ -219,7 +218,9 @@ public class jrunner {
try {
for (int i = 1; i <= cols; i++){
dtn[i] = rs.getMetaData().getColumnTypeName(i);
System.out.println(" * " + rs.getMetaData().getColumnName(i) + ": " + dtn[i]);
if (!queryMode) {
System.out.println(" * " + rs.getMetaData().getColumnName(i) + ": " + dtn[i]);
}
}
} catch (SQLException e) {
e.printStackTrace();
@ -418,15 +419,8 @@ public class jrunner {
outputTSV(rs, cols);
break;
case "table":
System.err.println("Table format not yet implemented, defaulting to CSV");
outputCSV(rs, cols);
break;
case "json":
System.err.println("JSON format not yet implemented, defaulting to CSV");
outputCSV(rs, cols);
break;
default:
System.err.println("Unknown format: " + format + ", defaulting to CSV");
outputCSV(rs, cols);
break;
}

150
readme.md
View File

@ -1,35 +1,52 @@
## install java jdk.
find downloads page and get latest tarball.
https://www.oracle.com/java/technologies/downloads/
## Quick Start
The easiest way to get started on a new system:
```bash
git clone https://gitea.hptrow.me/pt/jrunner.git
cd jrunner
./setup.sh
```
The setup script will:
- Check for Java 11+ (offers to install if missing)
- Verify Gradle wrapper is present
- Run a test build to ensure everything works
- Show you next steps
## Manual Installation
If you prefer to install dependencies manually:
### Install Java JDK
**Option 1: Package manager (recommended)**
```bash
# Ubuntu/Debian
sudo apt update && sudo apt install openjdk-17-jdk
# Fedora/RHEL
sudo dnf install java-17-openjdk-devel
# Arch
sudo pacman -S jdk-openjdk
```
**Option 2: Manual download**
Download from https://www.oracle.com/java/technologies/downloads/
```bash
wget https://download.oracle.com/java/19/latest/jdk-19_linux-x64_bin.tar.gz
tar -xvf downloaded_file
```
move the extracted folder to /opt
put the extracted location in your path variable
```
tar -xvf jdk-19_linux-x64_bin.tar.gz
sudo mv jdk-19.0.1 /opt/
export JAVA_HOME=/opt/jdk-19.0.1
export PATH=$PATH:$JAVA_HOME/bin
```
`java --version` to test
Test: `java --version`
## install gradle (optional)
### Install Gradle (optional)
Gradle wrapper (`gradlew`) is included in the repo, so manual Gradle installation is not required.
If you prefer to install Gradle system-wide:
```
wget https://services.gradle.org/distributions/gradle-8.5-bin.zip
unzip -d /opt/gradle gradle-8.5-bin.zip
export PATH=$PATH:/opt/gradle/gradle-8.5/bin
gradle -v
```
## clone this repo
```
git clone https://gitea.hptrow.me/pt/jrunner.git
cd jrunner
```
## build
```
@ -40,22 +57,17 @@ cd jrunner
### using the deploy script (recommended)
First, create the deployment directory:
Run the interactive deploy script:
```
sudo mkdir -p /opt/jrunner
```
Then deploy:
```
# Deploy to /opt/jrunner (default, creates system-wide symlink)
./deploy.sh
# Deploy to custom location (for testing, no symlink)
sudo mkdir -p /opt/jrunner-test
./deploy.sh /opt/jrunner-test
```
The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails. When deploying to `/opt/jrunner`, it creates a symlink at `/usr/local/bin/jrunner`.
The script will prompt you to choose:
1. **Local install** - Fast, no sudo required, installs to `./jrunner/build/install/jrunner`
2. **Global install** - Installs to `/opt/jrunner` with symlink at `/usr/local/bin/jrunner`
3. **Custom directory** - Prompts for path with tab-completion support
The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails.
### manual deployment
```
@ -64,14 +76,72 @@ sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
```
Or for local testing:
```
./gradlew installDist
# Binary at: ./jrunner/build/install/jrunner/bin/jrunner
```
## usage
After deployment to default location:
```
jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
### Query Mode (new in v1.1)
Query mode outputs results to stdout for piping to visidata, pspg, or less. It activates automatically when destination flags are omitted.
**Basic query to CSV:**
```bash
jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql
```
After deployment to custom location:
**Pipe to visidata:**
```bash
jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql | visidata -f csv
```
/opt/jrunner-test/bin/jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
**TSV format:**
```bash
jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql -f tsv
```
**SQL Server example:**
```bash
jrunner -scu "jdbc:sqlserver://hostname:1433;databaseName=mydb" -scn user -scp pass -sq query.sql
```
**PostgreSQL example:**
```bash
jrunner -scu "jdbc:postgresql://hostname:5432/dbname" -scn user -scp pass -sq query.sql
```
### Migration Mode
Full migration mode with both source and destination:
```bash
jrunner -scu jdbc:postgresql://source:5432/sourcedb \
-scn sourceuser \
-scp sourcepass \
-dcu jdbc:postgresql://dest:5432/destdb \
-dcn destuser \
-dcp destpass \
-sq query.sql \
-dt public.target_table
```
### Command-line flags
**Source connection:**
- `-scu` - source JDBC URL
- `-scn` - source username
- `-scp` - source password
- `-sq` - path to source SQL query file
**Destination connection (migration mode only):**
- `-dcu` - destination JDBC URL
- `-dcn` - destination username
- `-dcp` - destination password
- `-dt` - fully qualified destination table name
**Options:**
- `-t` - trim text fields (default: true)
- `-c` - clear target table before insert (default: true)
- `-f` - output format: csv, tsv (query mode only, default: csv)

174
setup.sh Executable file
View File

@ -0,0 +1,174 @@
#!/bin/bash
set -e
echo "jrunner Development Environment Setup"
echo "====================================="
echo ""
# Color codes for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Check if running on Linux
if [[ "$OSTYPE" != "linux-gnu"* ]]; then
echo -e "${YELLOW}Warning: This script is designed for Linux. You may need to manually install dependencies.${NC}"
echo ""
fi
# Function to check if command exists
command_exists() {
command -v "$1" >/dev/null 2>&1
}
# Function to get Java version
get_java_version() {
if command_exists java; then
java -version 2>&1 | head -n 1 | awk -F '"' '{print $2}' | awk -F '.' '{print $1}'
else
echo "0"
fi
}
echo "Checking dependencies..."
echo ""
# Check Java
JAVA_VERSION=$(get_java_version)
if [ "$JAVA_VERSION" -ge 11 ]; then
echo -e "${GREEN}${NC} Java $JAVA_VERSION detected"
java -version 2>&1 | head -n 1
JAVA_OK=true
else
echo -e "${RED}${NC} Java 11+ not found"
JAVA_OK=false
fi
# Check Git
if command_exists git; then
echo -e "${GREEN}${NC} Git detected"
DEPENDENCIES_OK=true
else
echo -e "${RED}${NC} Git not found"
DEPENDENCIES_OK=false
fi
# Check if gradlew exists
if [ -f "./gradlew" ]; then
echo -e "${GREEN}${NC} Gradle wrapper found (no need to install Gradle)"
GRADLE_OK=true
else
echo -e "${YELLOW}!${NC} Gradle wrapper not found in current directory"
GRADLE_OK=false
fi
echo ""
# If Java is missing, offer installation help
if [ "$JAVA_OK" = false ]; then
echo -e "${YELLOW}Java 11+ is required to build and run jrunner${NC}"
echo ""
echo "Installation options:"
echo ""
echo "1. Install OpenJDK via package manager (recommended):"
echo " Ubuntu/Debian: sudo apt update && sudo apt install openjdk-17-jdk"
echo " Fedora/RHEL: sudo dnf install java-17-openjdk-devel"
echo " Arch: sudo pacman -S jdk-openjdk"
echo ""
echo "2. Download Oracle JDK manually:"
echo " Visit: https://www.oracle.com/java/technologies/downloads/"
echo " Download JDK 17+ tarball and extract to /opt"
echo " Add to PATH: export JAVA_HOME=/opt/jdk-17 && export PATH=\$PATH:\$JAVA_HOME/bin"
echo ""
read -p "Would you like to install OpenJDK via package manager? (requires sudo) [y/N]: " install_java
if [[ "$install_java" =~ ^[Yy]$ ]]; then
if command_exists apt; then
echo "Installing OpenJDK 17 via apt..."
sudo apt update
sudo apt install -y openjdk-17-jdk
elif command_exists dnf; then
echo "Installing OpenJDK 17 via dnf..."
sudo dnf install -y java-17-openjdk-devel
elif command_exists pacman; then
echo "Installing OpenJDK via pacman..."
sudo pacman -S --noconfirm jdk-openjdk
else
echo -e "${RED}Could not detect package manager. Please install Java manually.${NC}"
exit 1
fi
# Verify installation
JAVA_VERSION=$(get_java_version)
if [ "$JAVA_VERSION" -ge 11 ]; then
echo -e "${GREEN}✓ Java installed successfully${NC}"
JAVA_OK=true
else
echo -e "${RED}✗ Java installation failed${NC}"
exit 1
fi
else
echo ""
echo "Please install Java manually and re-run this script."
exit 1
fi
fi
# Check for unzip (needed for deployment)
if ! command_exists unzip; then
echo -e "${YELLOW}Note: 'unzip' is recommended for deployment${NC}"
read -p "Install unzip? [y/N]: " install_unzip
if [[ "$install_unzip" =~ ^[Yy]$ ]]; then
if command_exists apt; then
sudo apt install -y unzip
elif command_exists dnf; then
sudo dnf install -y unzip
elif command_exists pacman; then
sudo pacman -S --noconfirm unzip
fi
fi
fi
echo ""
echo -e "${GREEN}All required dependencies are installed!${NC}"
echo ""
# Verify we're in the right directory
if [ ! -f "./gradlew" ]; then
echo -e "${RED}Error: gradlew not found. Are you in the jrunner project directory?${NC}"
exit 1
fi
# Make gradlew executable
chmod +x ./gradlew
echo "Testing build system..."
if ./gradlew --version > /dev/null 2>&1; then
echo -e "${GREEN}${NC} Gradle wrapper working"
else
echo -e "${RED}${NC} Gradle wrapper failed"
exit 1
fi
echo ""
echo "Running test build..."
if ./gradlew build; then
echo ""
echo -e "${GREEN}✓ Build successful!${NC}"
echo ""
echo "Next steps:"
echo " 1. Install locally for testing:"
echo " ./gradlew installDist"
echo " ./jrunner/build/install/jrunner/bin/jrunner --help"
echo ""
echo " 2. Or run the interactive deploy script:"
echo " ./deploy.sh"
echo ""
echo " 3. See readme.md for usage examples"
else
echo -e "${RED}✗ Build failed${NC}"
echo "Check the error messages above and try again."
exit 1
fi