pf_app
Paul Trowbridge
1807014323
pf_gkey is the Perspective table index -- an opaque handle read only by table.update() and table.remove(). It was the raw concat_ws of every grain column, which on the 24-column grain of fc_osm_skinny_29 averaged 233 characters. Being unique per row by construction, it also defeated Arrow's dictionary encoding, so it alone accounted for 65.6 MB of a 109 MB payload -- more than the other 31 columns combined, all of which do dictionary cleanly. md5 of the same string keeps the determinism undo depends on (routes/log.js recomputes the key through this same grainOf) and keeps collisions out of reach at 128 bits, while fixing the width at 32 characters. Measured on the live 285,685-row aggregate: payload 109.0 MB -> 54.2 MB, all 285,685 keys still distinct. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| lib | ||
| public | ||
| routes | ||
| setup_sql | ||
| ui | ||
| .env.example | ||
| .gitignore | ||
| CLAUDE.md | ||
| install.sh | ||
| package-lock.json | ||
| package.json | ||
| PERSPECTIVE.md | ||
| pf_perspective_options.md | ||
| pf_spec.md | ||
| pf.sh | ||
| server.js | ||