Update readme, CLAUDE.md, and bump version to 1.1
- Document query mode feature with examples - Update deploy script documentation - Add dual mode operation explanation to CLAUDE.md - Document CSV/TSV output formats - Update version from 1.0 to 1.1 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
56fecff550
commit
424d7d4ebb
71
CLAUDE.md
71
CLAUDE.md
@ -28,10 +28,16 @@ gradle build
|
||||
# Creates jrunner/build/distributions/jrunner.zip
|
||||
```
|
||||
|
||||
Deploy to /opt (as documented in readme.md):
|
||||
Local install for testing (recommended):
|
||||
```bash
|
||||
sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
|
||||
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
|
||||
./gradlew installDist
|
||||
# Creates executable at jrunner/build/install/jrunner/bin/jrunner
|
||||
```
|
||||
|
||||
Deploy using interactive script:
|
||||
```bash
|
||||
./deploy.sh
|
||||
# Choose: 1) Local install, 2) Global install to /opt, 3) Custom directory
|
||||
```
|
||||
|
||||
## Architecture
|
||||
@ -39,7 +45,33 @@ sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
|
||||
### Single-File Design
|
||||
The entire application logic resides in `jrunner/src/main/java/jrunner/jrunner.java`. This is a monolithic command-line tool with no abstraction layers or separate modules.
|
||||
|
||||
### Dual Mode Operation (v1.1+)
|
||||
|
||||
The tool operates in two modes:
|
||||
|
||||
**Query Mode** (new in v1.1):
|
||||
- Activates automatically when destination flags are not provided
|
||||
- Outputs query results to stdout in CSV or TSV format
|
||||
- Silent operation - no diagnostic output, just clean data
|
||||
- Designed for piping to visidata, pspg, less, or other data tools
|
||||
- Format controlled by -f flag (csv or tsv)
|
||||
|
||||
**Migration Mode** (original functionality):
|
||||
- Activates when destination flags are provided
|
||||
- Reads from source, writes to destination with batched INSERTs
|
||||
- Shows progress counters and timing information
|
||||
|
||||
### Data Flow
|
||||
|
||||
**Query Mode:**
|
||||
1. Parse command-line arguments (-scu, -scn, -scp for source)
|
||||
2. Read SQL query from file specified by -sq flag
|
||||
3. Connect to source database via JDBC
|
||||
4. Execute source query and fetch results (fetch size: 10,000 rows)
|
||||
5. Output results to stdout in CSV or TSV format
|
||||
6. Close connection and exit
|
||||
|
||||
**Migration Mode:**
|
||||
1. Parse command-line arguments (-scu, -scn, -scp for source; -dcu, -dcn, -dcp for destination)
|
||||
2. Read SQL query from file specified by -sq flag
|
||||
3. Connect to source and destination databases via JDBC
|
||||
@ -68,23 +100,36 @@ Command-line flags:
|
||||
- `-scu` - source JDBC URL
|
||||
- `-scn` - source username
|
||||
- `-scp` - source password
|
||||
- `-dcu` - destination JDBC URL
|
||||
- `-dcn` - destination username
|
||||
- `-dcp` - destination password
|
||||
- `-dcu` - destination JDBC URL (migration mode only)
|
||||
- `-dcn` - destination username (migration mode only)
|
||||
- `-dcp` - destination password (migration mode only)
|
||||
- `-sq` - path to source SQL query file
|
||||
- `-dt` - fully qualified destination table name
|
||||
- `-dt` - fully qualified destination table name (migration mode only)
|
||||
- `-t` - trim text fields (default: true)
|
||||
- `-c` - clear target table before insert (default: true)
|
||||
- `-c` - clear target table before insert (default: true, migration mode only)
|
||||
- `-f` - output format: csv, tsv (query mode only, default: csv)
|
||||
|
||||
## Key Implementation Details
|
||||
|
||||
### Batch Size
|
||||
INSERT statements are batched at 250 rows (hardcoded at line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
|
||||
### Mode Detection
|
||||
Query mode is automatically detected at runtime (line 131) by checking if all destination flags (dcu, dcn, dcp, dt) are empty. This allows seamless switching between query and migration modes without explicit mode flags.
|
||||
|
||||
### Query Mode Output (v1.1+)
|
||||
Query mode uses dedicated output methods:
|
||||
- `outputQueryResults()` - Dispatches to format-specific methods
|
||||
- `outputCSV()` - RFC 4180 compliant CSV with proper quote escaping
|
||||
- `outputTSV()` - Tab-separated with tabs/newlines replaced by spaces
|
||||
- All output goes to stdout; no diagnostic messages in query mode
|
||||
- Helper methods: `escapeCSV()` and `escapeTSV()` for proper formatting
|
||||
|
||||
### Batch Size (Migration Mode)
|
||||
INSERT statements are batched at 250 rows (hardcoded around line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
|
||||
|
||||
### Error Handling
|
||||
SQLException handling prints stack trace and exits immediately with System.exit(0). There is no transaction rollback or partial failure recovery.
|
||||
|
||||
### Performance Considerations
|
||||
- Result set fetch size is set to 10,000 rows (line 173)
|
||||
- Progress counter prints with carriage return for real-time updates
|
||||
- Timestamps captured at start (line 174) and end (line 368) for duration tracking
|
||||
- Result set fetch size is set to 10,000 rows (line 190)
|
||||
- Progress counter prints with carriage return for real-time updates (migration mode only)
|
||||
- Timestamps captured at start and end for duration tracking (migration mode only)
|
||||
- Query mode has no progress output to keep stdout clean for piping
|
||||
|
||||
@ -41,7 +41,7 @@ public class jrunner {
|
||||
Timestamp tsStart = null;
|
||||
Timestamp tsEnd = null;
|
||||
|
||||
msg = "jrunner version 1.0";
|
||||
msg = "jrunner version 1.1";
|
||||
msg = msg + nl + "-scu source jdbc url";
|
||||
msg = msg + nl + "-scn source username";
|
||||
msg = msg + nl + "-scp source passowrd";
|
||||
|
||||
87
readme.md
87
readme.md
@ -40,22 +40,17 @@ cd jrunner
|
||||
|
||||
### using the deploy script (recommended)
|
||||
|
||||
First, create the deployment directory:
|
||||
Run the interactive deploy script:
|
||||
```
|
||||
sudo mkdir -p /opt/jrunner
|
||||
```
|
||||
|
||||
Then deploy:
|
||||
```
|
||||
# Deploy to /opt/jrunner (default, creates system-wide symlink)
|
||||
./deploy.sh
|
||||
|
||||
# Deploy to custom location (for testing, no symlink)
|
||||
sudo mkdir -p /opt/jrunner-test
|
||||
./deploy.sh /opt/jrunner-test
|
||||
```
|
||||
|
||||
The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails. When deploying to `/opt/jrunner`, it creates a symlink at `/usr/local/bin/jrunner`.
|
||||
The script will prompt you to choose:
|
||||
1. **Local install** - Fast, no sudo required, installs to `./jrunner/build/install/jrunner`
|
||||
2. **Global install** - Installs to `/opt/jrunner` with symlink at `/usr/local/bin/jrunner`
|
||||
3. **Custom directory** - Prompts for path with tab-completion support
|
||||
|
||||
The script builds, extracts to a temporary location, and only updates the target directory after the build succeeds. This ensures your existing deployment stays intact if the build fails.
|
||||
|
||||
### manual deployment
|
||||
```
|
||||
@ -64,14 +59,72 @@ sudo unzip jrunner/build/distributions/jrunner.zip -d /opt/
|
||||
sudo ln -sf /opt/jrunner/bin/jrunner /usr/local/bin/jrunner
|
||||
```
|
||||
|
||||
Or for local testing:
|
||||
```
|
||||
./gradlew installDist
|
||||
# Binary at: ./jrunner/build/install/jrunner/bin/jrunner
|
||||
```
|
||||
|
||||
## usage
|
||||
|
||||
After deployment to default location:
|
||||
```
|
||||
jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
|
||||
### Query Mode (new in v1.1)
|
||||
|
||||
Query mode outputs results to stdout for piping to visidata, pspg, or less. It activates automatically when destination flags are omitted.
|
||||
|
||||
**Basic query to CSV:**
|
||||
```bash
|
||||
jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql
|
||||
```
|
||||
|
||||
After deployment to custom location:
|
||||
**Pipe to visidata:**
|
||||
```bash
|
||||
jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql | visidata -f csv
|
||||
```
|
||||
/opt/jrunner-test/bin/jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
|
||||
|
||||
**TSV format:**
|
||||
```bash
|
||||
jrunner -scu "jdbc:as400://hostname" -scn user -scp pass -sq query.sql -f tsv
|
||||
```
|
||||
|
||||
**SQL Server example:**
|
||||
```bash
|
||||
jrunner -scu "jdbc:sqlserver://hostname:1433;databaseName=mydb" -scn user -scp pass -sq query.sql
|
||||
```
|
||||
|
||||
**PostgreSQL example:**
|
||||
```bash
|
||||
jrunner -scu "jdbc:postgresql://hostname:5432/dbname" -scn user -scp pass -sq query.sql
|
||||
```
|
||||
|
||||
### Migration Mode
|
||||
|
||||
Full migration mode with both source and destination:
|
||||
```bash
|
||||
jrunner -scu jdbc:postgresql://source:5432/sourcedb \
|
||||
-scn sourceuser \
|
||||
-scp sourcepass \
|
||||
-dcu jdbc:postgresql://dest:5432/destdb \
|
||||
-dcn destuser \
|
||||
-dcp destpass \
|
||||
-sq query.sql \
|
||||
-dt public.target_table
|
||||
```
|
||||
|
||||
### Command-line flags
|
||||
|
||||
**Source connection:**
|
||||
- `-scu` - source JDBC URL
|
||||
- `-scn` - source username
|
||||
- `-scp` - source password
|
||||
- `-sq` - path to source SQL query file
|
||||
|
||||
**Destination connection (migration mode only):**
|
||||
- `-dcu` - destination JDBC URL
|
||||
- `-dcn` - destination username
|
||||
- `-dcp` - destination password
|
||||
- `-dt` - fully qualified destination table name
|
||||
|
||||
**Options:**
|
||||
- `-t` - trim text fields (default: true)
|
||||
- `-c` - clear target table before insert (default: true)
|
||||
- `-f` - output format: csv, tsv (query mode only, default: csv)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user