Thirteen modules across four functional layers — ingest, transform, orchestrate, output. Click any node to open its manifest. Hover to isolate its dependency edges. Drag to pan. Scroll to zoom.

Three end-to-end paths through the system. Each arrow is a function call carrying a typed value.

Five subcommands. Two working today; three planned. Python API surface mirrors the CLI for embedding in notebooks.

The two CSVs that flow between stages. Column-level types and descriptions.

Where everything lives.

SGE submission scripts and the gotchas they exist to protect against.

# laptop — push the feature branch
$ git push origin feature/v0.2-psf

# cluster — pull, install, submit
$ cd ~/palfitology && git pull
$ pip install -e . --no-deps && hash -r

$ cd ~/PALFITology_OLD
$ ./scripts/submit_all.sh 1638518.csv
  array job     -> 806464 (8 tasks)
  reconcile job -> held on 806464

$ qstat -u $USER
  806464 .... running on 8 nodes

Rules of the road. Every convention exists because a previous attempt was bitten by its absence.

Hard limits — numerical, data, performance, cluster, and process.

Why the codebase looks the way it does.

v0.1 to v0.6 (cluster validation in progress).