Compare commits
No commits in common. "809f2a8949fabaa14bf35bc9b8fb472d03ae71a3" and "b3a9151effcffd0204ea2d791878bb0b4c3dfd98" have entirely different histories.
809f2a8949
...
b3a9151eff
3
.gitignore
vendored
3
.gitignore
vendored
@ -5,6 +5,3 @@
|
|||||||
build
|
build
|
||||||
|
|
||||||
*.swp
|
*.swp
|
||||||
|
|
||||||
# Ignore local configuration files
|
|
||||||
run.yml
|
|
||||||
|
|||||||
93
CLAUDE.md
93
CLAUDE.md
@ -1,93 +0,0 @@
|
|||||||
# CLAUDE.md
|
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
||||||
|
|
||||||
## Project Overview
|
|
||||||
|
|
||||||
jrunner is a Java CLI tool for migrating data between databases. It reads data from a source database using SQL queries and writes it to a destination table, batching inserts for performance. The tool supports multiple database types via JDBC drivers including PostgreSQL, IBM AS/400, and Microsoft SQL Server.
|
|
||||||
|
|
||||||
## Build and Test Commands
|
|
||||||
|
|
||||||
Build the project:
|
|
||||||
```bash
|
|
||||||
gradle build
|
|
||||||
# or use wrapper
|
|
||||||
./gradlew build
|
|
||||||
```
|
|
||||||
|
|
||||||
Run tests:
|
|
||||||
```bash
|
|
||||||
gradle test
|
|
||||||
# or use wrapper
|
|
||||||
./gradlew test
|
|
||||||
```
|
|
||||||
|
|
||||||
Build distribution package:
|
|
||||||
```bash
|
|
||||||
gradle build
|
|
||||||
# Creates app/build/distributions/app.zip
|
|
||||||
```
|
|
||||||
|
|
||||||
Deploy to /opt (as documented in readme.md):
|
|
||||||
```bash
|
|
||||||
sudo cp app/build/distributions/app.zip /opt/
|
|
||||||
sudo rm -rf /opt/app/
|
|
||||||
sudo unzip /opt/app.zip -d /opt/
|
|
||||||
sudo rm /opt/app.zip
|
|
||||||
sudo chown ptrowbridge:ptrowbridge -R /opt/app/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Single-File Design
|
|
||||||
The entire application logic resides in `app/src/main/java/jrunner/jrunner.java`. This is a monolithic command-line tool with no abstraction layers or separate modules.
|
|
||||||
|
|
||||||
### Data Flow
|
|
||||||
1. Parse command-line arguments (-scu, -scn, -scp for source; -dcu, -dcn, -dcp for destination)
|
|
||||||
2. Read SQL query from file specified by -sq flag
|
|
||||||
3. Connect to source and destination databases via JDBC
|
|
||||||
4. Execute source query and fetch results (fetch size: 10,000 rows)
|
|
||||||
5. Build batched INSERT statements (250 rows per batch)
|
|
||||||
6. Execute batches against destination table specified by -dt flag
|
|
||||||
7. Optionally clear target table before insert if -c flag is set
|
|
||||||
|
|
||||||
### Type Handling
|
|
||||||
The tool includes explicit handling for different SQL data types in a switch statement (lines 229-312). Supported types include VARCHAR, TEXT, CHAR, CLOB, DATE, TIME, TIMESTAMP, and BIGINT. String types get quote escaping and optional trimming.
|
|
||||||
|
|
||||||
### Database Drivers
|
|
||||||
JDBC drivers are configured in `app/build.gradle`:
|
|
||||||
- PostgreSQL: org.postgresql:postgresql:42.5.0
|
|
||||||
- IBM AS/400 (JT400): net.sf.jt400:jt400:11.0
|
|
||||||
- Microsoft SQL Server: com.microsoft.sqlserver:mssql-jdbc:9.2.0.jre8
|
|
||||||
- SQL Server Integrated Auth: com.microsoft.sqlserver:mssql-jdbc_auth:9.2.0.x64
|
|
||||||
|
|
||||||
The AS/400 driver requires explicit Class.forName() registration (line 144).
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
The project uses a YAML configuration format (run.yml) to specify database connections, SQL script paths, and runtime options. However, the main application currently uses command-line arguments instead of parsing this YAML file.
|
|
||||||
|
|
||||||
Command-line flags:
|
|
||||||
- `-scu` - source JDBC URL
|
|
||||||
- `-scn` - source username
|
|
||||||
- `-scp` - source password
|
|
||||||
- `-dcu` - destination JDBC URL
|
|
||||||
- `-dcn` - destination username
|
|
||||||
- `-dcp` - destination password
|
|
||||||
- `-sq` - path to source SQL query file
|
|
||||||
- `-dt` - fully qualified destination table name
|
|
||||||
- `-t` - trim text fields (default: true)
|
|
||||||
- `-c` - clear target table before insert (default: true)
|
|
||||||
|
|
||||||
## Key Implementation Details
|
|
||||||
|
|
||||||
### Batch Size
|
|
||||||
INSERT statements are batched at 250 rows (hardcoded at line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
|
|
||||||
|
|
||||||
### Error Handling
|
|
||||||
SQLException handling prints stack trace and exits immediately with System.exit(0). There is no transaction rollback or partial failure recovery.
|
|
||||||
|
|
||||||
### Performance Considerations
|
|
||||||
- Result set fetch size is set to 10,000 rows (line 173)
|
|
||||||
- Progress counter prints with carriage return for real-time updates
|
|
||||||
- Timestamps captured at start (line 174) and end (line 368) for duration tracking
|
|
||||||
2
gradle/wrapper/gradle-wrapper.properties
vendored
2
gradle/wrapper/gradle-wrapper.properties
vendored
@ -1,5 +1,5 @@
|
|||||||
distributionBase=GRADLE_USER_HOME
|
distributionBase=GRADLE_USER_HOME
|
||||||
distributionPath=wrapper/dists
|
distributionPath=wrapper/dists
|
||||||
distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip
|
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-bin.zip
|
||||||
zipStoreBase=GRADLE_USER_HOME
|
zipStoreBase=GRADLE_USER_HOME
|
||||||
zipStorePath=wrapper/dists
|
zipStorePath=wrapper/dists
|
||||||
|
|||||||
50
readme.md
50
readme.md
@ -15,41 +15,31 @@ export PATH=$PATH:$JAVA_HOME/bin
|
|||||||
|
|
||||||
`java --version` to test
|
`java --version` to test
|
||||||
|
|
||||||
## install gradle (optional)
|
## install gradle
|
||||||
Gradle wrapper (`gradlew`) is included in the repo, so manual Gradle installation is not required.
|
https://docs.gradle.org/current/userguide/installation.html
|
||||||
If you prefer to install Gradle system-wide:
|
go to gradle.org and find the latest bin.zip file (burried in the `direct link` hyperlink)
|
||||||
```
|
```
|
||||||
wget https://services.gradle.org/distributions/gradle-8.5-bin.zip
|
wget https://services.gradle.org/distributions/gradle-7.6-bin.zip
|
||||||
unzip -d /opt/gradle gradle-8.5-bin.zip
|
unzip -d /opt/gradle gradle-7.6-bin.zip
|
||||||
export PATH=$PATH:/opt/gradle/gradle-8.5/bin
|
```
|
||||||
gradle -v
|
point to the gradle install
|
||||||
|
```
|
||||||
|
export PATH=$PATH:/opt/gradle/gradle-7.6/bin
|
||||||
|
gradle -v` to validate
|
||||||
```
|
```
|
||||||
|
|
||||||
## clone this repo
|
## clone this repo
|
||||||
```
|
clone
|
||||||
git clone https://gitea.hptrow.me/pt/jrunner.git
|
`git clone https://gitea.hptrow.me/pt/jrunner.git`
|
||||||
cd jrunner
|
|
||||||
```
|
|
||||||
|
|
||||||
## build
|
build
|
||||||
```
|
gradle build`
|
||||||
./gradlew build
|
|
||||||
```
|
|
||||||
|
|
||||||
## deploy system-wide
|
copy to opt for use
|
||||||
```
|
```
|
||||||
sudo unzip app/build/distributions/app.zip -d /opt/
|
sudo cp app/build/distributions/app.zip /opt/
|
||||||
sudo ln -sf /opt/app/bin/app /usr/local/bin/jrunner
|
sudo rm -rf /opt/app/
|
||||||
```
|
sudo unzip /opt/app.zip -d /opt/
|
||||||
|
sudo rm /opt/app.zip
|
||||||
Now you can run from anywhere:
|
sudo chown ptrowbridge:ptrowbridge -R /opt/app/
|
||||||
```
|
|
||||||
jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
|
|
||||||
```
|
|
||||||
|
|
||||||
To update after rebuilding:
|
|
||||||
```
|
|
||||||
./gradlew build
|
|
||||||
sudo rm -rf /opt/app
|
|
||||||
sudo unzip app/build/distributions/app.zip -d /opt/
|
|
||||||
```
|
```
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user