Compare commits
4 Commits
b3a9151eff
...
809f2a8949
| Author | SHA1 | Date | |
|---|---|---|---|
| 809f2a8949 | |||
| f47eaf4bac | |||
| 1392e0a6d8 | |||
| b0ee4c77d9 |
3
.gitignore
vendored
3
.gitignore
vendored
@ -5,3 +5,6 @@
|
|||||||
build
|
build
|
||||||
|
|
||||||
*.swp
|
*.swp
|
||||||
|
|
||||||
|
# Ignore local configuration files
|
||||||
|
run.yml
|
||||||
|
|||||||
93
CLAUDE.md
Normal file
93
CLAUDE.md
Normal file
@ -0,0 +1,93 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
jrunner is a Java CLI tool for migrating data between databases. It reads data from a source database using SQL queries and writes it to a destination table, batching inserts for performance. The tool supports multiple database types via JDBC drivers including PostgreSQL, IBM AS/400, and Microsoft SQL Server.
|
||||||
|
|
||||||
|
## Build and Test Commands
|
||||||
|
|
||||||
|
Build the project:
|
||||||
|
```bash
|
||||||
|
gradle build
|
||||||
|
# or use wrapper
|
||||||
|
./gradlew build
|
||||||
|
```
|
||||||
|
|
||||||
|
Run tests:
|
||||||
|
```bash
|
||||||
|
gradle test
|
||||||
|
# or use wrapper
|
||||||
|
./gradlew test
|
||||||
|
```
|
||||||
|
|
||||||
|
Build distribution package:
|
||||||
|
```bash
|
||||||
|
gradle build
|
||||||
|
# Creates app/build/distributions/app.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
Deploy to /opt (as documented in readme.md):
|
||||||
|
```bash
|
||||||
|
sudo cp app/build/distributions/app.zip /opt/
|
||||||
|
sudo rm -rf /opt/app/
|
||||||
|
sudo unzip /opt/app.zip -d /opt/
|
||||||
|
sudo rm /opt/app.zip
|
||||||
|
sudo chown ptrowbridge:ptrowbridge -R /opt/app/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Single-File Design
|
||||||
|
The entire application logic resides in `app/src/main/java/jrunner/jrunner.java`. This is a monolithic command-line tool with no abstraction layers or separate modules.
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
1. Parse command-line arguments (-scu, -scn, -scp for source; -dcu, -dcn, -dcp for destination)
|
||||||
|
2. Read SQL query from file specified by -sq flag
|
||||||
|
3. Connect to source and destination databases via JDBC
|
||||||
|
4. Execute source query and fetch results (fetch size: 10,000 rows)
|
||||||
|
5. Build batched INSERT statements (250 rows per batch)
|
||||||
|
6. Execute batches against destination table specified by -dt flag
|
||||||
|
7. Optionally clear target table before insert if -c flag is set
|
||||||
|
|
||||||
|
### Type Handling
|
||||||
|
The tool includes explicit handling for different SQL data types in a switch statement (lines 229-312). Supported types include VARCHAR, TEXT, CHAR, CLOB, DATE, TIME, TIMESTAMP, and BIGINT. String types get quote escaping and optional trimming.
|
||||||
|
|
||||||
|
### Database Drivers
|
||||||
|
JDBC drivers are configured in `app/build.gradle`:
|
||||||
|
- PostgreSQL: org.postgresql:postgresql:42.5.0
|
||||||
|
- IBM AS/400 (JT400): net.sf.jt400:jt400:11.0
|
||||||
|
- Microsoft SQL Server: com.microsoft.sqlserver:mssql-jdbc:9.2.0.jre8
|
||||||
|
- SQL Server Integrated Auth: com.microsoft.sqlserver:mssql-jdbc_auth:9.2.0.x64
|
||||||
|
|
||||||
|
The AS/400 driver requires explicit Class.forName() registration (line 144).
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
The project uses a YAML configuration format (run.yml) to specify database connections, SQL script paths, and runtime options. However, the main application currently uses command-line arguments instead of parsing this YAML file.
|
||||||
|
|
||||||
|
Command-line flags:
|
||||||
|
- `-scu` - source JDBC URL
|
||||||
|
- `-scn` - source username
|
||||||
|
- `-scp` - source password
|
||||||
|
- `-dcu` - destination JDBC URL
|
||||||
|
- `-dcn` - destination username
|
||||||
|
- `-dcp` - destination password
|
||||||
|
- `-sq` - path to source SQL query file
|
||||||
|
- `-dt` - fully qualified destination table name
|
||||||
|
- `-t` - trim text fields (default: true)
|
||||||
|
- `-c` - clear target table before insert (default: true)
|
||||||
|
|
||||||
|
## Key Implementation Details
|
||||||
|
|
||||||
|
### Batch Size
|
||||||
|
INSERT statements are batched at 250 rows (hardcoded at line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
SQLException handling prints stack trace and exits immediately with System.exit(0). There is no transaction rollback or partial failure recovery.
|
||||||
|
|
||||||
|
### Performance Considerations
|
||||||
|
- Result set fetch size is set to 10,000 rows (line 173)
|
||||||
|
- Progress counter prints with carriage return for real-time updates
|
||||||
|
- Timestamps captured at start (line 174) and end (line 368) for duration tracking
|
||||||
2
gradle/wrapper/gradle-wrapper.properties
vendored
2
gradle/wrapper/gradle-wrapper.properties
vendored
@ -1,5 +1,5 @@
|
|||||||
distributionBase=GRADLE_USER_HOME
|
distributionBase=GRADLE_USER_HOME
|
||||||
distributionPath=wrapper/dists
|
distributionPath=wrapper/dists
|
||||||
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-bin.zip
|
distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip
|
||||||
zipStoreBase=GRADLE_USER_HOME
|
zipStoreBase=GRADLE_USER_HOME
|
||||||
zipStorePath=wrapper/dists
|
zipStorePath=wrapper/dists
|
||||||
|
|||||||
54
readme.md
54
readme.md
@ -15,31 +15,41 @@ export PATH=$PATH:$JAVA_HOME/bin
|
|||||||
|
|
||||||
`java --version` to test
|
`java --version` to test
|
||||||
|
|
||||||
## install gradle
|
## install gradle (optional)
|
||||||
https://docs.gradle.org/current/userguide/installation.html
|
Gradle wrapper (`gradlew`) is included in the repo, so manual Gradle installation is not required.
|
||||||
go to gradle.org and find the latest bin.zip file (burried in the `direct link` hyperlink)
|
If you prefer to install Gradle system-wide:
|
||||||
```
|
```
|
||||||
wget https://services.gradle.org/distributions/gradle-7.6-bin.zip
|
wget https://services.gradle.org/distributions/gradle-8.5-bin.zip
|
||||||
unzip -d /opt/gradle gradle-7.6-bin.zip
|
unzip -d /opt/gradle gradle-8.5-bin.zip
|
||||||
```
|
export PATH=$PATH:/opt/gradle/gradle-8.5/bin
|
||||||
point to the gradle install
|
gradle -v
|
||||||
```
|
|
||||||
export PATH=$PATH:/opt/gradle/gradle-7.6/bin
|
|
||||||
gradle -v` to validate
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## clone this repo
|
## clone this repo
|
||||||
clone
|
|
||||||
`git clone https://gitea.hptrow.me/pt/jrunner.git`
|
|
||||||
|
|
||||||
build
|
|
||||||
gradle build`
|
|
||||||
|
|
||||||
copy to opt for use
|
|
||||||
```
|
```
|
||||||
sudo cp app/build/distributions/app.zip /opt/
|
git clone https://gitea.hptrow.me/pt/jrunner.git
|
||||||
sudo rm -rf /opt/app/
|
cd jrunner
|
||||||
sudo unzip /opt/app.zip -d /opt/
|
```
|
||||||
sudo rm /opt/app.zip
|
|
||||||
sudo chown ptrowbridge:ptrowbridge -R /opt/app/
|
## build
|
||||||
|
```
|
||||||
|
./gradlew build
|
||||||
|
```
|
||||||
|
|
||||||
|
## deploy system-wide
|
||||||
|
```
|
||||||
|
sudo unzip app/build/distributions/app.zip -d /opt/
|
||||||
|
sudo ln -sf /opt/app/bin/app /usr/local/bin/jrunner
|
||||||
|
```
|
||||||
|
|
||||||
|
Now you can run from anywhere:
|
||||||
|
```
|
||||||
|
jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
|
||||||
|
```
|
||||||
|
|
||||||
|
To update after rebuilding:
|
||||||
|
```
|
||||||
|
./gradlew build
|
||||||
|
sudo rm -rf /opt/app
|
||||||
|
sudo unzip app/build/distributions/app.zip -d /opt/
|
||||||
```
|
```
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user