Repository source: docs/runbooks/fastmod_audit_backfill.md
Fastmod Historical Audit and Backfill
Use this runbook only as a deployment operator. Historical scoring is opt-in; historical evaluations never create automatic meta tags. See root meta tagging for the separate new-publication-only policy.
Before auditing, use Tools → Fastmod to confirm at least one enabled rubric.
The audit and backfill target the current enabled set and template revision;
changing that set creates no automatic backfill. Old pending compositions are
excluded with rubric_changed_before_score; audit again to explicitly request
current-composition work.
Run
./v3 fast-score audit --include-existing. Confirm it identifies the expected read-model path, selected model, candidate states, estimate, and read-only confirmation.For a non-default model, set both private per-million-token price values or pass both pricing options, then rerun the audit.
Start small:
./v3 fast-score backfill --include-existing --confirm --max-posts=1 --max-cost-usd=0.01.Run
./v3 task-queue run --limit=1 --score-limit=1 --work-limit=250, then inspect./v3 fast-score status. Confirm the recent batch has a private state and a reserved estimate no larger than its maximum cost.--score-limitis the provider-call cap;--work-limitis a separate cap for all examined rows, including no-cost local exclusions.Inspect the matching
fast_post_scoreexchange privately. Confirm its post/content hash and provider model match the batch's target; do not copy its payload into public data.Create a deliberately invalid provider configuration only in a disposable environment. Confirm retries follow the usual cap, then confirm a retry is not started if its reservation would exceed the batch maximum cost.
Confirm a newly published post still enters ordinary work and is not counted as historical backfill.
In a disposable private database, age a completed batch past one year and run
./v3 fast-score prune --before=.... Confirm its batch, provenance, work, score, and matching private exchange records are removed.
Do not use direct SQLite writes to alter batch limits, work state, or reserved cost. Create a new bounded batch instead.