Compare commits
4 Commits
b3a9151eff
...
809f2a8949
| Author | SHA1 | Date | |
|---|---|---|---|
| 809f2a8949 | |||
| f47eaf4bac | |||
| 1392e0a6d8 | |||
| b0ee4c77d9 |
3
.gitignore
vendored
3
.gitignore
vendored
@ -5,3 +5,6 @@
|
||||
build
|
||||
|
||||
*.swp
|
||||
|
||||
# Ignore local configuration files
|
||||
run.yml
|
||||
|
||||
93
CLAUDE.md
Normal file
93
CLAUDE.md
Normal file
@ -0,0 +1,93 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
jrunner is a Java CLI tool for migrating data between databases. It reads data from a source database using SQL queries and writes it to a destination table, batching inserts for performance. The tool supports multiple database types via JDBC drivers including PostgreSQL, IBM AS/400, and Microsoft SQL Server.
|
||||
|
||||
## Build and Test Commands
|
||||
|
||||
Build the project:
|
||||
```bash
|
||||
gradle build
|
||||
# or use wrapper
|
||||
./gradlew build
|
||||
```
|
||||
|
||||
Run tests:
|
||||
```bash
|
||||
gradle test
|
||||
# or use wrapper
|
||||
./gradlew test
|
||||
```
|
||||
|
||||
Build distribution package:
|
||||
```bash
|
||||
gradle build
|
||||
# Creates app/build/distributions/app.zip
|
||||
```
|
||||
|
||||
Deploy to /opt (as documented in readme.md):
|
||||
```bash
|
||||
sudo cp app/build/distributions/app.zip /opt/
|
||||
sudo rm -rf /opt/app/
|
||||
sudo unzip /opt/app.zip -d /opt/
|
||||
sudo rm /opt/app.zip
|
||||
sudo chown ptrowbridge:ptrowbridge -R /opt/app/
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Single-File Design
|
||||
The entire application logic resides in `app/src/main/java/jrunner/jrunner.java`. This is a monolithic command-line tool with no abstraction layers or separate modules.
|
||||
|
||||
### Data Flow
|
||||
1. Parse command-line arguments (-scu, -scn, -scp for source; -dcu, -dcn, -dcp for destination)
|
||||
2. Read SQL query from file specified by -sq flag
|
||||
3. Connect to source and destination databases via JDBC
|
||||
4. Execute source query and fetch results (fetch size: 10,000 rows)
|
||||
5. Build batched INSERT statements (250 rows per batch)
|
||||
6. Execute batches against destination table specified by -dt flag
|
||||
7. Optionally clear target table before insert if -c flag is set
|
||||
|
||||
### Type Handling
|
||||
The tool includes explicit handling for different SQL data types in a switch statement (lines 229-312). Supported types include VARCHAR, TEXT, CHAR, CLOB, DATE, TIME, TIMESTAMP, and BIGINT. String types get quote escaping and optional trimming.
|
||||
|
||||
### Database Drivers
|
||||
JDBC drivers are configured in `app/build.gradle`:
|
||||
- PostgreSQL: org.postgresql:postgresql:42.5.0
|
||||
- IBM AS/400 (JT400): net.sf.jt400:jt400:11.0
|
||||
- Microsoft SQL Server: com.microsoft.sqlserver:mssql-jdbc:9.2.0.jre8
|
||||
- SQL Server Integrated Auth: com.microsoft.sqlserver:mssql-jdbc_auth:9.2.0.x64
|
||||
|
||||
The AS/400 driver requires explicit Class.forName() registration (line 144).
|
||||
|
||||
## Configuration
|
||||
|
||||
The project uses a YAML configuration format (run.yml) to specify database connections, SQL script paths, and runtime options. However, the main application currently uses command-line arguments instead of parsing this YAML file.
|
||||
|
||||
Command-line flags:
|
||||
- `-scu` - source JDBC URL
|
||||
- `-scn` - source username
|
||||
- `-scp` - source password
|
||||
- `-dcu` - destination JDBC URL
|
||||
- `-dcn` - destination username
|
||||
- `-dcp` - destination password
|
||||
- `-sq` - path to source SQL query file
|
||||
- `-dt` - fully qualified destination table name
|
||||
- `-t` - trim text fields (default: true)
|
||||
- `-c` - clear target table before insert (default: true)
|
||||
|
||||
## Key Implementation Details
|
||||
|
||||
### Batch Size
|
||||
INSERT statements are batched at 250 rows (hardcoded at line 324). When the batch threshold is reached, sql is prepended with "INSERT INTO {table} VALUES" and executed.
|
||||
|
||||
### Error Handling
|
||||
SQLException handling prints stack trace and exits immediately with System.exit(0). There is no transaction rollback or partial failure recovery.
|
||||
|
||||
### Performance Considerations
|
||||
- Result set fetch size is set to 10,000 rows (line 173)
|
||||
- Progress counter prints with carriage return for real-time updates
|
||||
- Timestamps captured at start (line 174) and end (line 368) for duration tracking
|
||||
2
gradle/wrapper/gradle-wrapper.properties
vendored
2
gradle/wrapper/gradle-wrapper.properties
vendored
@ -1,5 +1,5 @@
|
||||
distributionBase=GRADLE_USER_HOME
|
||||
distributionPath=wrapper/dists
|
||||
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-bin.zip
|
||||
distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip
|
||||
zipStoreBase=GRADLE_USER_HOME
|
||||
zipStorePath=wrapper/dists
|
||||
|
||||
54
readme.md
54
readme.md
@ -15,31 +15,41 @@ export PATH=$PATH:$JAVA_HOME/bin
|
||||
|
||||
`java --version` to test
|
||||
|
||||
## install gradle
|
||||
https://docs.gradle.org/current/userguide/installation.html
|
||||
go to gradle.org and find the latest bin.zip file (burried in the `direct link` hyperlink)
|
||||
## install gradle (optional)
|
||||
Gradle wrapper (`gradlew`) is included in the repo, so manual Gradle installation is not required.
|
||||
If you prefer to install Gradle system-wide:
|
||||
```
|
||||
wget https://services.gradle.org/distributions/gradle-7.6-bin.zip
|
||||
unzip -d /opt/gradle gradle-7.6-bin.zip
|
||||
```
|
||||
point to the gradle install
|
||||
```
|
||||
export PATH=$PATH:/opt/gradle/gradle-7.6/bin
|
||||
gradle -v` to validate
|
||||
wget https://services.gradle.org/distributions/gradle-8.5-bin.zip
|
||||
unzip -d /opt/gradle gradle-8.5-bin.zip
|
||||
export PATH=$PATH:/opt/gradle/gradle-8.5/bin
|
||||
gradle -v
|
||||
```
|
||||
|
||||
## clone this repo
|
||||
clone
|
||||
`git clone https://gitea.hptrow.me/pt/jrunner.git`
|
||||
|
||||
build
|
||||
gradle build`
|
||||
|
||||
copy to opt for use
|
||||
```
|
||||
sudo cp app/build/distributions/app.zip /opt/
|
||||
sudo rm -rf /opt/app/
|
||||
sudo unzip /opt/app.zip -d /opt/
|
||||
sudo rm /opt/app.zip
|
||||
sudo chown ptrowbridge:ptrowbridge -R /opt/app/
|
||||
git clone https://gitea.hptrow.me/pt/jrunner.git
|
||||
cd jrunner
|
||||
```
|
||||
|
||||
## build
|
||||
```
|
||||
./gradlew build
|
||||
```
|
||||
|
||||
## deploy system-wide
|
||||
```
|
||||
sudo unzip app/build/distributions/app.zip -d /opt/
|
||||
sudo ln -sf /opt/app/bin/app /usr/local/bin/jrunner
|
||||
```
|
||||
|
||||
Now you can run from anywhere:
|
||||
```
|
||||
jrunner -scu jdbc:postgresql://... -scn user -scp pass ...
|
||||
```
|
||||
|
||||
To update after rebuilding:
|
||||
```
|
||||
./gradlew build
|
||||
sudo rm -rf /opt/app
|
||||
sudo unzip app/build/distributions/app.zip -d /opt/
|
||||
```
|
||||
|
||||
Loading…
Reference in New Issue
Block a user