Land the engine money pack: the run now stops on the VOLUME of work a buyer paid for, and the read path folds the bank so a re-pass can be priced before it is bought

This commit is contained in:
heaven 2026-08-29 12:59:10 +03:00
parent e14561b258
commit 6ec9f8a66a
31 changed files with 4773 additions and 225 deletions

View file

@ -8,7 +8,7 @@
| Пакет | Что делает | Ключевые файлы | | Пакет | Что делает | Ключевые файлы |
|---|---|---| |---|---|---|
| `cmd/tmctl` | CLI: `translate` / `status` (read-only N/M+паспорта глав+деньги, `--json`; работает ПРИ живом прогоне — store без flock) / `report` (request-log+леджер книги **+ per-run quality-report**: структурный KPI предл./нарратив-абзац, тире, cosmetic-strip/echo-rates, глоссарий-промахи, trust-gated — наблюдаемость, НЕ гейт; D39.2-T4) / `export` (**экспорт-поверхность для полигона**: final_hash→checkpoint→checks.ExportNormalize, manifest-join c pending/ghost-учётом, ConfigDrift-поле, `--plaintext`; D39.5) / `redrive` (переатака флагнутых: `--chapter/--chunk/--reason/--dry-run`, D15.3) / `manifest` (**$0-производитель персиста манифеста глав/чанков**, строка 100: нужен ДО первого прогона — дерево глав разобранной, но не запущенной книги; `--json` печатает сам документ) / `migrate` (**$0-write-open без прогона**, строка 174: read-only команды требуют ТОЧНОГО совпадения схемы и не мигрируют, поэтому апгрейд бинаря запирал все существующие книги — платформа зовёт `status --json` перед каждым спавном, а write-команда, которая мигрировала бы, не наступала никогда. Грузит ТОЛЬКО `book.yaml` — ни цен, ни ключей на $0-пути (строка 146); restore point берёт ПОД ЛОКОМ и только когда шаг реально применяется (имя `<метка>-pre-migrate.db` — вне секундного неймспейса платного пути, строка 173); несовпадение схемы у read-only путей — типизированный отказ **exit 13** с машинным токеном `schema_mismatch found=N expected=M`). Денежный аргумент прогона: `--ceiling-usd` (строка 145) перекрывает книжный `ceilings.book_usd` ТОЛЬКО на этот прогон, в book.yaml не пишется, ноль/отрицательное = отказ запуска. main — тонкая обвязка: разбор/exit-коды/.env/рендеры в тестируемых функциях | `main.go`, `invocation.go`, `render.go`, `dotenv.go` | | `cmd/tmctl` | CLI: `translate` / `status` (read-only N/M+паспорта глав+деньги, `--json`; работает ПРИ живом прогоне — store без flock) / `report` (request-log+леджер книги **+ per-run quality-report**: структурный KPI предл./нарратив-абзац, тире, cosmetic-strip/echo-rates, глоссарий-промахи, trust-gated — наблюдаемость, НЕ гейт; D39.2-T4) / `export` (**экспорт-поверхность для полигона**: final_hash→checkpoint→checks.ExportNormalize, manifest-join c pending/ghost-учётом, ConfigDrift-поле, `--plaintext`; D39.5) / `redrive` (переатака флагнутых: `--chapter/--chunk/--reason/--dry-run`, D15.3) / `manifest` (**$0-производитель персиста манифеста глав/чанков**, строка 100: нужен ДО первого прогона — дерево глав разобранной, но не запущенной книги; `--json` печатает сам документ) / `migrate` (**$0-write-open без прогона**, строка 174: read-only команды требуют ТОЧНОГО совпадения схемы и не мигрируют, поэтому апгрейд бинаря запирал все существующие книги — платформа зовёт `status --json` перед каждым спавном, а write-команда, которая мигрировала бы, не наступала никогда. Грузит ТОЛЬКО `book.yaml` — ни цен, ни ключей на $0-пути (строка 146); restore point берёт ПОД ЛОКОМ и только когда шаг реально применяется (имя `<метка>-pre-migrate.db` — вне секундного неймспейса платного пути, строка 173); несовпадение схемы у read-only путей — типизированный отказ **exit 13** с машинным токеном `schema_mismatch found=N expected=M`). **ДВА потолка прогона, и они ортогональны.** `--ceiling-usd` (строка 145) — ДЕНЬГИ: перекрывает книжный `ceilings.book_usd` ТОЛЬКО на этот прогон, в book.yaml не пишется, ноль/отрицательное = отказ запуска; каппит КУМУЛЯТИВНУЮ трату книги, а не приращение прогона. `--max-units` (D39.165 §1б) — ОБЪЁМ: не больше N ВЫХОДНЫХ ЮНИТОВ (гранулярность `units_total` манифеста, та самая, в которой платформа продаёт главы) будет ОПЛАЧЕНО этим прогоном. Юниты, которые прогон отдаёт за $0 (резюм, ре-пин), едут бесплатно и потолок не тратят; ретраи и эскалации — тоже нет, они внутри юнита. Принимает только `translate`. **Остановка по объёму — ЗАВЕРШЕНИЕ (exit 0), а не пауза:** словарь кодов выхода не расширялся, различение живёт в отчёте прогона и в логе. Отчёт различает ДОСТАВКУ (юнит, которого не было) и ПЕРЕ-ДЕЛКУ (уже доставленный юнит под сдвинутым снапшотом) — покупка, целиком ушедшая в переделку, обязана читаться как переделка. ⚠ На книге, которая МАЙНИТ банк, вторая покупка требует `--resnapshot` (авто-банк растёт между покупками и двигает edit-снапшот) — прогон предупреждает об этом в логе. main — тонкая обвязка: разбор/exit-коды/.env/рендеры в тестируемых функциях | `main.go`, `invocation.go`, `render.go`, `dotenv.go` |
| `internal/llm` | OpenAI-совместимый транспорт + retry/backoff, **capability-слой** (budget_field/temperature/reasoning per-модель), провайдеры openai/local (no-proxy)/anthropic (DEPRECATED-референс); `failover.go` удалён паком-17 — маршрутизация лейблами живёт в config/pipeline (канал B, один хоп `chain[0]`, fail-closed) | `httpllm.go`, `capability.go`, `provider_*.go` | | `internal/llm` | OpenAI-совместимый транспорт + retry/backoff, **capability-слой** (budget_field/temperature/reasoning per-модель), провайдеры openai/local (no-proxy)/anthropic (DEPRECATED-референс); `failover.go` удалён паком-17 — маршрутизация лейблами живёт в config/pipeline (канал B, один хоп `chain[0]`, fail-closed) | `httpllm.go`, `capability.go`, `provider_*.go` |
| `internal/ledger` | Цены по usage (вкл. reasoning/cache-поля), `PriceForResponse` по фактической модели | `pricing.go` | | `internal/ledger` | Цены по usage (вкл. reasoning/cache-поля), `PriceForResponse` по фактической модели | `pricing.go` |
| `internal/store` | SQLite (modernc, CGO-free), цепочка миграций `schema_version` (⚠ ALTER-шаги v8+ не идемпотентны вопреки шапке — бэклог-строка 49а), reserve/settle+checkpoint, chunk_status, глоссарий (+подписной цикл терминолога), ruby, retrieval_state, request_log; `OpenReadOnly` — без flock/миграций/recovery для status/report/export (схема не совпала — типизированный `*store.SchemaMismatchError{Found,Expected}`, обе стороны); `Migrate` (+шов `beforeApply` под локом: restore point берётся там) — поверхность деплой-шага строки 174, `SchemaHead` — та самая `Expected`; write-open БД новее бинаря теперь ОТКАЗ, а не тихое открытие | `ledger.go`, `migrate.go`, `glossary.go`, `store.go` | | `internal/store` | SQLite (modernc, CGO-free), цепочка миграций `schema_version` (⚠ ALTER-шаги v8+ не идемпотентны вопреки шапке — бэклог-строка 49а), reserve/settle+checkpoint, chunk_status, глоссарий (+подписной цикл терминолога), ruby, retrieval_state, request_log; `OpenReadOnly` — без flock/миграций/recovery для status/report/export (схема не совпала — типизированный `*store.SchemaMismatchError{Found,Expected}`, обе стороны); `Migrate` (+шов `beforeApply` под локом: restore point берётся там) — поверхность деплой-шага строки 174, `SchemaHead` — та самая `Expected`; write-open БД новее бинаря теперь ОТКАЗ, а не тихое открытие | `ledger.go`, `migrate.go`, `glossary.go`, `store.go` |
@ -66,7 +66,8 @@ go run ./cmd/tmctl report --config example/book.yaml # $0, quality-repor
go run ./cmd/tmctl export --config example/book.yaml # $0, экспорт-JSON для полигона (--plaintext для человека) go run ./cmd/tmctl export --config example/book.yaml # $0, экспорт-JSON для полигона (--plaintext для человека)
go run ./cmd/tmctl manifest --config example/book.yaml # $0, пере-строить персист манифеста глав/чанков (--json — сам документ) go run ./cmd/tmctl manifest --config example/book.yaml # $0, пере-строить персист манифеста глав/чанков (--json — сам документ)
go run ./cmd/tmctl migrate --config example/book.yaml # $0, довести схему проекта до головы бинаря (деплой-шаг: стоп прогонов → НОВЫЙ бинарь на место → migrate ИМ по каждой книге без открытых попыток → прогоны в работу) go run ./cmd/tmctl migrate --config example/book.yaml # $0, довести схему проекта до головы бинаря (деплой-шаг: стоп прогонов → НОВЫЙ бинарь на место → migrate ИМ по каждой книге без открытых попыток → прогоны в работу)
go run ./cmd/tmctl translate --config example/book.yaml --ceiling-usd 0.5 # потолок ТОЛЬКО на этот прогон, book.yaml не пишется go run ./cmd/tmctl translate --config example/book.yaml --ceiling-usd 0.5 # ДЕНЕЖНЫЙ потолок ТОЛЬКО на этот прогон, book.yaml не пишется
go run ./cmd/tmctl translate --config example/book.yaml --max-units 10 # ОБЪЁМНЫЙ потолок: оплатить не больше 10 выходных юнитов; стоп = завершение (exit 0)
# live-conformance (реальные провайдеры, платно, вне CI): # live-conformance (реальные провайдеры, платно, вне CI):
set -a; . ./.env; set +a; TM_LIVE=1 go test -tags live -run TestLive -v ./internal/pipeline/ set -a; . ./.env; set +a; TM_LIVE=1 go test -tags live -run TestLive -v ./internal/pipeline/
``` ```

View file

@ -65,7 +65,7 @@ func TestFakeTranslatePathCreatesNoBackup(t *testing.T) {
defer srv.Close() defer srv.Close()
bookPath := setupCLIProject(t, srv.URL) bookPath := setupCLIProject(t, srv.URL)
if err := translate(context.Background(), bookPath, false, pipeline.RebillConsent{}, false, 0); err != nil { if err := translate(context.Background(), bookPath, false, pipeline.RebillConsent{}, false, 0, 0); err != nil {
t.Fatalf("fake translate: %v", err) t.Fatalf("fake translate: %v", err)
} }
if _, err := os.Stat(filepath.Join(filepath.Dir(bookPath), "backups")); err == nil { if _, err := os.Stat(filepath.Join(filepath.Dir(bookPath), "backups")); err == nil {

View file

@ -45,7 +45,11 @@ type invocation struct {
// spend on one run belongs to that run, not to the book's data — writing it into book.yaml would make // spend on one run belongs to that run, not to the book's data — writing it into book.yaml would make
// a caller's number a permanent record in the engine's config and mix the zones (D39.81/D39.85). // a caller's number a permanent record in the engine's config and mix the zones (D39.81/D39.85).
ceilingUSD float64 ceilingUSD float64
sel pipeline.RedriveSelector // maxUnits is the VOLUME ceiling for THIS RUN (D39.165 §1б), 0 when the flag was absent. It is an
// invocation property for exactly the reason ceilingUSD is — how much of the book a caller is buying
// right now belongs to the call, not to the book's data.
maxUnits int
sel pipeline.RedriveSelector
} }
// rebillConsentValue parses `--accept-rebill[=usd]` (D20.2-Q2): the OPTIONAL-VALUE form of Р6, where a // rebillConsentValue parses `--accept-rebill[=usd]` (D20.2-Q2): the OPTIONAL-VALUE form of Р6, where a
@ -127,6 +131,7 @@ func parseInvocation(args []string, flagOut io.Writer) (invocation, error) {
dryRun := fs.Bool("dry-run", false, "redrive: report what would be re-attacked without touching anything; bank-apply: print the projection of the decisions and write NOTHING") dryRun := fs.Bool("dry-run", false, "redrive: report what would be re-attacked without touching anything; bank-apply: print the projection of the decisions and write NOTHING")
verifyBank := fs.Bool("verify-bank", false, "translate/redrive: STOP at the bank-mining boundary and print the bank table when the delta holds terms no earlier stop has shown (D39.144: the flag trips on novelty; a resumed run goes on and undecided terms ride to the editor marked). Default: never stop") verifyBank := fs.Bool("verify-bank", false, "translate/redrive: STOP at the bank-mining boundary and print the bank table when the delta holds terms no earlier stop has shown (D39.144: the flag trips on novelty; a resumed run goes on and undecided terms ride to the editor marked). Default: never stop")
ceilingUSD := fs.Float64("ceiling-usd", 0, "translate/redrive: the book USD ceiling in force for THIS RUN ONLY — it OVERRIDES book.yaml `ceilings.book_usd` and is never written back. It caps the book's CUMULATIVE committed+reserved spend, not this run's increment, and must be > 0") ceilingUSD := fs.Float64("ceiling-usd", 0, "translate/redrive: the book USD ceiling in force for THIS RUN ONLY — it OVERRIDES book.yaml `ceilings.book_usd` and is never written back. It caps the book's CUMULATIVE committed+reserved spend, not this run's increment, and must be > 0")
maxUnits := fs.Int("max-units", 0, "translate: the VOLUME ceiling for THIS RUN — at most N OUTPUT UNITS (the manifest's units_total granularity) may be PAID for. Units this run resolves for free (resumed or re-pinned) ride along and do not count, and neither do retries or escalation hops inside a unit. ORTHOGONAL to --ceiling-usd, which caps the book's CUMULATIVE money: this caps THIS run's work. Stopping on it is a COMPLETION (exit 0), not a pause")
seed := fs.String("seed", "", "seed-lint: path to the glossary seed YAML to validate ($0, no --config)") seed := fs.String("seed", "", "seed-lint: path to the glossary seed YAML to validate ($0, no --config)")
keysFile := fs.String("keys-file", "", "translate: path to the DEPLOYMENT's provider-key file (KEY=VALUE lines). Loaded FIRST, so it wins over the .env beside book.yaml; a named file that cannot be read is a refusal, never a silent skip") keysFile := fs.String("keys-file", "", "translate: path to the DEPLOYMENT's provider-key file (KEY=VALUE lines). Loaded FIRST, so it wins over the .env beside book.yaml; a named file that cannot be read is a refusal, never a silent skip")
decisions := fs.String("decisions", "", "bank-apply: path to the JSON decision document to apply to the book's memory bank ($0)") decisions := fs.String("decisions", "", "bank-apply: path to the JSON decision document to apply to the book's memory bank ($0)")
@ -181,6 +186,31 @@ func parseInvocation(args []string, flagOut io.Writer) (invocation, error) {
if given["dry-run"] && cmd != "bank-apply" && cmd != "redrive" { if given["dry-run"] && cmd != "bank-apply" && cmd != "redrive" {
return invocation{}, fmt.Errorf("--dry-run is accepted by `bank-apply` and `redrive` only, not by %q: %q has no projection mode, and running it with this flag would spend real money on a call its caller believes is free", cmd, cmd) return invocation{}, fmt.Errorf("--dry-run is accepted by `bank-apply` and `redrive` only, not by %q: %q has no projection mode, and running it with this flag would spend real money on a call its caller believes is free", cmd, cmd)
} }
// The volume ceiling follows the same rule as every other flag above: a command that does not act on it
// REFUSES it rather than ignoring it. Silently accepting it here would be the expensive spelling —
// `redrive --max-units 10` re-attacks flagged chunks and then translates the WHOLE book through
// TranslateBook, so a caller who believed the flag bounded the work would be billed for everything
// left. Scoped to `translate`, which is the one verb whose volume a purchase actually buys.
if given["max-units"] {
if cmd != "translate" {
// The reason has to be TRUE of the command it addresses, which is the discipline the
// --keys-file refusal above already carries a test for. `redrive` is the case that makes it
// matter: it DOES drive a book wave — it calls TranslateBook after its reset — so telling it
// "you have no book wave to bound" would teach the reader a false fact about the engine, and
// the next session wiring this flag would believe it.
if cmd == "redrive" {
return invocation{}, fmt.Errorf("--max-units is accepted by `translate` only, not by `redrive`: redrive DOES drive a book wave (it re-attacks the flagged chunks and then translates through TranslateBook), but the destructive reset it performs first is bounded by its own selector and by nothing else — a units cap here would bound the second half of the command while reading as a cap on the whole. Bound the re-attack with --chapter/--chunk/--reason, and the translation that follows with a separate `translate --max-units`")
}
return invocation{}, fmt.Errorf("--max-units is accepted by `translate` only, not by %q: it caps the OUTPUT UNITS a run pays for, and %q neither runs nor pays for them — accepting the flag would tell a caller their work was capped when it was not", cmd, cmd)
}
// Refused rather than defaulted when present-but-not-a-volume, the same rule --ceiling-usd follows
// (D39.110: "off" is not a state). Zero is the ABSENT value, so admitting `--max-units 0` would turn
// an unset variable in a deployment's unit file into an unbounded run that the caller believes is
// bounded — and a negative is not a quantity of book at all.
if *maxUnits <= 0 {
return invocation{}, fmt.Errorf("--max-units must be a positive number of output units (it is the volume ceiling for THIS run); got %d — omit the flag to run the whole book", *maxUnits)
}
}
// seed-lint validates a standalone seed YAML — it takes --seed, not --config (no book/store/keys). // seed-lint validates a standalone seed YAML — it takes --seed, not --config (no book/store/keys).
if cmd == "seed-lint" { if cmd == "seed-lint" {
if *seed == "" { if *seed == "" {
@ -218,7 +248,7 @@ func parseInvocation(args []string, flagOut io.Writer) (invocation, error) {
return invocation{ return invocation{
cmd: cmd, cfgPath: *cfgPath, resnapshot: *resnapshot, acceptRebill: acceptRebill.c, cmd: cmd, cfgPath: *cfgPath, resnapshot: *resnapshot, acceptRebill: acceptRebill.c,
asJSON: *asJSON, asPlaintext: *asPlaintext, asPairs: *asPairs, verifyBank: *verifyBank, asJSON: *asJSON, asPlaintext: *asPlaintext, asPairs: *asPairs, verifyBank: *verifyBank,
ceilingUSD: *ceilingUSD, keysFile: *keysFile, decisionsPath: *decisions, dryRun: *dryRun, ceilingUSD: *ceilingUSD, maxUnits: *maxUnits, keysFile: *keysFile, decisionsPath: *decisions, dryRun: *dryRun,
sel: pipeline.RedriveSelector{ sel: pipeline.RedriveSelector{
Chapter: *chapter, ChunkIdx: *chunk, Reason: *reason, DryRun: *dryRun, Chapter: *chapter, ChunkIdx: *chunk, Reason: *reason, DryRun: *dryRun,
}, },

View file

@ -299,3 +299,64 @@ func TestDispatchCommandsCoversTheSwitch(t *testing.T) {
t.Errorf("the dispatch switch has %d commands, dispatchCommands lists %d — the two must be one list", found, len(dispatchCommands)) t.Errorf("the dispatch switch has %d commands, dispatchCommands lists %d — the two must be one list", found, len(dispatchCommands))
} }
} }
// TestMaxUnitsIsRefusedWhereItWouldDoNothing pins the volume ceiling onto the rule the other four flags
// already follow: a command that does not act on a flag REFUSES it rather than ignoring it.
//
// The silent version of this one costs book rather than dollars, and `redrive` is the expensive case. It
// re-attacks the flagged chunks and then runs TranslateBook over the WHOLE book, so `redrive --max-units
// 10` accepted-and-ignored would bill a caller for every unit the book has left while their own logs said
// they had capped the work at ten.
func TestMaxUnitsIsRefusedWhereItWouldDoNothing(t *testing.T) {
for _, cmd := range dispatchCommands {
if cmd == "translate" {
continue
}
args := []string{cmd, "--config", "book.yaml", "--max-units", "10"}
if cmd == "bank-apply" {
args = append(args, "--decisions", "d.json")
}
_, err := parseInvocation(args, &bytes.Buffer{})
if err == nil {
t.Errorf("%s accepted --max-units; a flag a command does not act on must be refused, never ignored", cmd)
continue
}
if !strings.Contains(err.Error(), "--max-units is accepted by `translate` only") {
t.Errorf("%s: the refusal must name the flag and the one verb that takes it, got: %v", cmd, err)
}
// And the REASON must be true of the command it addresses — the discipline the --keys-file
// refusal already carries a test for. The first draft of this refusal told every command it
// "does not run a book wave whose volume this could bound", which is false of `redrive`: it
// calls TranslateBook after its reset. A refusal that misdescribes its reader teaches them the
// wrong thing about the engine, and the next session to wire this flag inherits it.
if strings.Contains(err.Error(), "neither runs nor pays for them") && cmd == "redrive" {
t.Errorf("redrive DOES drive a book wave (it calls TranslateBook); its refusal must not claim otherwise, got: %v", err)
}
}
// redrive gets its own reason, and that reason has to say the true thing.
_, err := parseInvocation([]string{"redrive", "--config", "book.yaml", "--max-units", "10"}, &bytes.Buffer{})
if err == nil || !strings.Contains(err.Error(), "redrive DOES drive a book wave") {
t.Fatalf("redrive's refusal must name the real reason (the reset is bounded by its selector, not by units), got: %v", err)
}
}
// TestMaxUnitsRefusesANonVolume pins the same "«off» is not a state" rule --ceiling-usd has needed since
// D39.110: a PRESENT flag with a meaningless value is a refusal, because the live trigger is an unset
// variable in a deployment's unit file, and defaulting it to «unbounded» would hand a caller a whole book
// while their own configuration said ten units.
func TestMaxUnitsRefusesANonVolume(t *testing.T) {
for _, v := range []string{"0", "-1"} {
_, err := parseInvocation([]string{"translate", "--config", "book.yaml", "--max-units", v}, &bytes.Buffer{})
if err == nil || !strings.Contains(err.Error(), "--max-units must be a positive number of output units") {
t.Fatalf("--max-units %s: want a refusal naming the rule, got %v", v, err)
}
}
// Absent is the ONE spelling that means "the whole book", and it must stay silent.
inv, err := parseInvocation([]string{"translate", "--config", "book.yaml"}, &bytes.Buffer{})
if err != nil {
t.Fatalf("an absent --max-units must parse cleanly: %v", err)
}
if inv.maxUnits != 0 {
t.Fatalf("an absent --max-units must leave the run unbounded, got %d", inv.maxUnits)
}
}

View file

@ -219,7 +219,7 @@ func run() error {
if err := preflightBackup(inv.cfgPath, os.Stdout); err != nil { if err := preflightBackup(inv.cfgPath, os.Stdout); err != nil {
return err return err
} }
return translate(ctx, inv.cfgPath, inv.resnapshot, inv.acceptRebill, inv.verifyBank, inv.ceilingUSD) return translate(ctx, inv.cfgPath, inv.resnapshot, inv.acceptRebill, inv.verifyBank, inv.ceilingUSD, inv.maxUnits)
case "report": case "report":
return report(inv.cfgPath) return report(inv.cfgPath)
case "status": case "status":
@ -261,7 +261,17 @@ func run() error {
// //
// Over the book's consent threshold the run stops with the sum before reserving anything — a flag that // Over the book's consent threshold the run stops with the sum before reserving anything — a flag that
// names no money cannot carry a Р6 consent to a spend. // names no money cannot carry a Р6 consent to a spend.
func translate(ctx context.Context, cfgPath string, resnapshot bool, acceptRebill pipeline.RebillConsent, verifyBank bool, ceilingUSD float64) error { //
// --max-units is the FOURTH flag and the second CEILING, and it is orthogonal to --ceiling-usd in the way
// that matters most (D39.165 §1б): that one bounds the book's CUMULATIVE money, this one bounds THIS
// run's WORK, in output units — the granularity the manifest publishes and a purchase of chapters
// converts into exactly. Both can be in force; whichever binds first stops the run, and the two stops are
// different answers. Money stops the run in the middle of what it meant to do — exit 4, resumable, "add
// money". Volume means the run did precisely what was bought — exit 0, an ordinary completion, and the
// exit-code dictionary above is not touched. Which one stopped it is said in the run's report and its log
// (pipeline.VolumeStop), never by a new code: the platform reads an unknown exit code as a FAILURE, so a
// user who received exactly what they paid for would have been shown a service error.
func translate(ctx context.Context, cfgPath string, resnapshot bool, acceptRebill pipeline.RebillConsent, verifyBank bool, ceilingUSD float64, maxUnits int) error {
r, err := pipeline.NewRunner(cfgPath, obs.NewLogger()) r, err := pipeline.NewRunner(cfgPath, obs.NewLogger())
if err != nil { if err != nil {
return err return err
@ -271,6 +281,7 @@ func translate(ctx context.Context, cfgPath string, resnapshot bool, acceptRebil
r.AcceptRebill = acceptRebill r.AcceptRebill = acceptRebill
r.VerifyBank = verifyBank r.VerifyBank = verifyBank
r.CeilingUSD = ceilingUSD r.CeilingUSD = ceilingUSD
r.MaxUnits = maxUnits
res, err := r.TranslateBook(ctx) res, err := r.TranslateBook(ctx)
if err != nil { if err != nil {

View file

@ -253,7 +253,7 @@ func TestMigrateNeedsNeitherPricesNorModels(t *testing.T) {
func TestMigrateKeepsTheMoneyAndRepeatsAsANoOp(t *testing.T) { func TestMigrateKeepsTheMoneyAndRepeatsAsANoOp(t *testing.T) {
bookPath := setupCLIProject(t, mockProvider(t).URL) bookPath := setupCLIProject(t, mockProvider(t).URL)
dbPath := projectDBOf(bookPath) dbPath := projectDBOf(bookPath)
if err := translate(context.Background(), bookPath, false, pipeline.RebillConsent{}, false, 0); err != nil { if err := translate(context.Background(), bookPath, false, pipeline.RebillConsent{}, false, 0, 0); err != nil {
t.Fatalf("setup run: %v", err) t.Fatalf("setup run: %v", err)
} }

View file

@ -10,6 +10,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"sync"
"testing" "testing"
"time" "time"
@ -98,7 +99,7 @@ func TestTranslateWiresAcceptRebill(t *testing.T) {
bookPath := setupCLIProject(t, srv.URL) bookPath := setupCLIProject(t, srv.URL)
ctx := context.Background() ctx := context.Background()
if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0); err != nil { if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0, 0); err != nil {
t.Fatalf("first run: %v", err) t.Fatalf("first run: %v", err)
} }
if calls != 1 { if calls != 1 {
@ -114,7 +115,7 @@ func TestTranslateWiresAcceptRebill(t *testing.T) {
writeCLIFile(t, pipePath, strings.Replace(string(raw), "prompt_version: v-cli", "prompt_version: v-cli-drift", 1)) writeCLIFile(t, pipePath, strings.Replace(string(raw), "prompt_version: v-cli", "prompt_version: v-cli-drift", 1))
// --resnapshot WITHOUT consent → refused, nothing called. // --resnapshot WITHOUT consent → refused, nothing called.
err = translate(ctx, bookPath, true, pipeline.RebillConsent{}, false, 0) err = translate(ctx, bookPath, true, pipeline.RebillConsent{}, false, 0, 0)
if err == nil { if err == nil {
t.Fatal("`translate --resnapshot` without --accept-rebill must be refused over the threshold") t.Fatal("`translate --resnapshot` without --accept-rebill must be refused over the threshold")
} }
@ -127,7 +128,7 @@ func TestTranslateWiresAcceptRebill(t *testing.T) {
// A ceiling BELOW the projection is still a refusal — proving the CAPPED value is wired, not just // A ceiling BELOW the projection is still a refusal — proving the CAPPED value is wired, not just
// the boolean. // the boolean.
if err := translate(ctx, bookPath, true, pipeline.RebillConsent{Given: true, Capped: true, CapUSD: 0.0001}, false, 0); err == nil { if err := translate(ctx, bookPath, true, pipeline.RebillConsent{Given: true, Capped: true, CapUSD: 0.0001}, false, 0, 0); err == nil {
t.Fatal("a ceiling below the projection must be refused through the CLI entry point too") t.Fatal("a ceiling below the projection must be refused through the CLI entry point too")
} }
if calls != 1 { if calls != 1 {
@ -135,7 +136,7 @@ func TestTranslateWiresAcceptRebill(t *testing.T) {
} }
// Consent given → the run proceeds and really re-pays. // Consent given → the run proceeds and really re-pays.
if err := translate(ctx, bookPath, true, pipeline.RebillConsent{Given: true}, false, 0); err != nil { if err := translate(ctx, bookPath, true, pipeline.RebillConsent{Given: true}, false, 0, 0); err != nil {
t.Fatalf("`translate --resnapshot --accept-rebill` must proceed: %v", err) t.Fatalf("`translate --resnapshot --accept-rebill` must proceed: %v", err)
} }
if calls != 2 { if calls != 2 {
@ -160,7 +161,7 @@ func TestRedriveWiresAcceptRebill(t *testing.T) {
ctx := context.Background() ctx := context.Background()
// The refusal text flags the only chunk (exit 2 is a sentinel, not an infra failure). // The refusal text flags the only chunk (exit 2 is a sentinel, not an infra failure).
if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0); exitCode(err) != 2 { if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0, 0); exitCode(err) != 2 {
t.Fatalf("setup: the refusal fixture must complete with flags (exit 2), got %v", err) t.Fatalf("setup: the refusal fixture must complete with flags (exit 2), got %v", err)
} }
callsAfterRun1 := calls callsAfterRun1 := calls
@ -211,7 +212,7 @@ func TestTranslateAndRedriveWireTheRunCeiling(t *testing.T) {
// A ceiling far below anything the book can cost. The book's own ceiling is comfortable, so ONLY the // A ceiling far below anything the book can cost. The book's own ceiling is comfortable, so ONLY the
// argument can produce this stop — and only if the command actually hands it to the runner. // argument can produce this stop — and only if the command actually hands it to the runner.
const tiny = 0.000001 const tiny = 0.000001
err := translate(ctx, setupCLIProject(t, srv.URL), false, pipeline.RebillConsent{}, false, tiny) err := translate(ctx, setupCLIProject(t, srv.URL), false, pipeline.RebillConsent{}, false, tiny, 0)
if err == nil || !strings.Contains(err.Error(), "ceiling reached") { if err == nil || !strings.Contains(err.Error(), "ceiling reached") {
t.Fatalf("translate must carry --ceiling-usd into the run; got %v", err) t.Fatalf("translate must carry --ceiling-usd into the run; got %v", err)
} }
@ -236,7 +237,7 @@ func TestTranslateAndRedriveWireTheRunCeiling(t *testing.T) {
})) }))
defer srv2.Close() defer srv2.Close()
bookPath := setupCLIProject(t, srv2.URL) bookPath := setupCLIProject(t, srv2.URL)
if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0); exitCode(err) != 2 { if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0, 0); exitCode(err) != 2 {
t.Fatalf("setup: the run must finish WITH a flagged chunk (exit 2), got %v", err) t.Fatalf("setup: the run must finish WITH a flagged chunk (exit 2), got %v", err)
} }
refuse = false refuse = false
@ -246,3 +247,76 @@ func TestTranslateAndRedriveWireTheRunCeiling(t *testing.T) {
t.Fatalf("redrive must carry --ceiling-usd into the re-attack; got %v", err) t.Fatalf("redrive must carry --ceiling-usd into the re-attack; got %v", err)
} }
} }
// setupCLIProjectMulti is setupCLIProject with a source long enough — and a draft cut fine enough — to
// produce SEVERAL output units, which is what a volume ceiling needs in order to bite at all.
func setupCLIProjectMulti(t *testing.T, providerURL string) string {
t.Helper()
bookPath := setupCLIProject(t, providerURL)
dir := filepath.Dir(bookPath)
writeCLIFile(t, filepath.Join(dir, "source.txt"),
"静かな図書館の朝。鈴木は本を読んだ。外では雨が降っていた。彼は窓を見た。時間は静かに過ぎた。夜になった。")
writeCLIFile(t, filepath.Join(dir, "pipeline.yaml"), `
core: C1
version: 1
defaults: { max_output_ratio: 2.0, min_max_tokens: 512 }
retries: { regenerate_before_escalate: 0 }
context: { glossary_injection: selective, glossary_token_budget: 800 }
segmentation:
draft_budget_out: 24
edit_ceiling_out: 48
fertility: { cjk: 1.1978, other: 0.3852 }
stages:
- { name: draft, role: translator, model: fake-model, prompt_override: prompts/translator.md, prompt_version: v-cli, temperature: 0.3, reasoning: "off" }
`)
return bookPath
}
// TestTranslateWiresMaxUnits is the same kind of pin as TestTranslateWiresAcceptRebill above, for the
// other run ceiling: it proves the flag changes what the RUN DOES, not merely what the parser returns.
//
// Every other proof of the volume ceiling sets Runner.MaxUnits directly, which leaves exactly one line
// unverified — `r.MaxUnits = maxUnits` in main.go's translate(). That line is the entire seam between the
// flag a platform passes and the engine that honours it; dropping it would leave every volume test in the
// pipeline package green while `tmctl translate --max-units 2` translated the whole book.
func TestTranslateWiresMaxUnits(t *testing.T) {
var mu sync.Mutex
calls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.ReadAll(r.Body)
mu.Lock()
calls++
mu.Unlock()
tb, _ := json.Marshal("ЧЕРНОВИК ПЕРЕВОДА")
fmt.Fprintf(w, `{"id":"fake","model":"fake-model","choices":[{"message":{"content":%s},"finish_reason":"stop"}],
"usage":{"prompt_tokens":1000,"completion_tokens":500,"prompt_tokens_details":{"cached_tokens":200}}}`, tb)
}))
defer srv.Close()
bookPath := setupCLIProjectMulti(t, srv.URL)
ctx := context.Background()
// Unbounded first, on a throwaway copy of the project, to learn how many units this book really has —
// so the bounded assertion below is against a measured number rather than a guessed one.
fullPath := setupCLIProjectMulti(t, srv.URL)
if err := translate(ctx, fullPath, false, pipeline.RebillConsent{}, false, 0, 0); err != nil {
t.Fatalf("the unbounded baseline must complete: %v", err)
}
mu.Lock()
total := calls
calls = 0
mu.Unlock()
if total < 3 {
t.Fatalf("fixture is too thin to bound: the whole book took %d call(s), so a ceiling of 2 proves nothing", total)
}
// Now the same book through the CLI entry point WITH the flag.
if err := translate(ctx, bookPath, false, pipeline.RebillConsent{}, false, 0, 2); err != nil {
t.Fatalf("a volume-bounded run is a completion, not an error: %v", err)
}
mu.Lock()
bounded := calls
mu.Unlock()
if bounded != 2 {
t.Fatalf("`translate --max-units 2` made %d provider call(s) on a %d-unit book: the flag did not reach the engine (main.go's `r.MaxUnits = maxUnits`)", bounded, total)
}
}

View file

@ -121,6 +121,36 @@ func renderTranslate(w io.Writer, res *pipeline.BookResult, ledger func() (commi
} }
fmt.Fprintf(w, "Book ledger: committed=$%.6f reserved=$%.6f\n", committed, reserved) fmt.Fprintf(w, "Book ledger: committed=$%.6f reserved=$%.6f\n", committed, reserved)
// The stop NAMES its ceiling. A run bounded by --max-units ends with exit 0 like any other completed
// run, so without this line the operator sees "TOTAL … chunks 3" on a hundred-unit book and has no way
// to tell "the volume I bought is done" from "something went quietly wrong" — the silent-limit
// complaint D39.165 §1б took from OpenHands. It names the ceiling, what was delivered as against
// re-made, and what remains of each kind, so the next action is readable off the line itself.
if v := res.Volume; v != nil {
fmt.Fprintf(w, "VOLUME CEILING: %s\n", v)
fmt.Fprintf(w, " (this is a COMPLETION, not a pause — the run did what it was granted; the money ceiling is a separate stop with its own exit code.)\n")
// ⚠ The invitation to buy more is spoken ONLY about units that were never delivered. The first
// version said "run again to buy more" unconditionally, which on a fully-translated book with a
// moved bank offered the reader chapters they already own — re-payment presented as stock.
//
// Both remainders are reported when both exist. An if/else here would have printed the fresh half
// and silently dropped the other, leaving an operator who has BOTH kinds pending believing the
// first number was all of it.
if v.LeftFresh > 0 {
fmt.Fprintf(w, " Next: %d output unit(s) of this book have never been delivered — `--max-units` again to buy them (units, NOT chapters: a chapter can be several units; the manifest is what converts).\n", v.LeftFresh)
}
if v.LeftRework > 0 {
if v.LeftFresh > 0 {
fmt.Fprintf(w, " Also pending, and NOT a purchase: %d already-delivered unit(s) carry a superseded snapshot and would be RE-MADE. Buying them delivers no new chapter.\n", v.LeftRework)
} else {
fmt.Fprintf(w, " Next: every unit of this book HAS been delivered; the %d left carry a superseded snapshot and would be RE-MADE, not delivered. That is a re-pass, not a purchase of new book.\n", v.LeftRework)
}
}
if v.Delivered == 0 && v.Reworked > 0 {
fmt.Fprintf(w, " ⚠ This run delivered NO new unit: all %d paid unit(s) were re-made under a moved snapshot.\n", v.Reworked)
}
}
// Exit code 2 is a typed sentinel, not an infra error: the report above is // Exit code 2 is a typed sentinel, not an infra error: the report above is
// already on stdout; main() maps this to a non-zero exit for the operator. // already on stdout; main() maps this to a non-zero exit for the operator.
if res.Flagged > 0 { if res.Flagged > 0 {
@ -663,7 +693,21 @@ func renderStatusHuman(w io.Writer, rep *pipeline.StatusReport, cfgPath string)
// It is the same projection `translate` refuses on, so the operator reads the decision number here // It is the same projection `translate` refuses on, so the operator reads the decision number here
// instead of discovering it in a refusal. // instead of discovering it in a refusal.
if rep.RebillUnits > 0 { if rep.RebillUnits > 0 {
drift += fmt.Sprintf(" ⚠ RE-PAYMENT: %d chunk×stage unit(s) already billed under a superseded snapshot would be paid for again, ~$%.6f (translate needs --accept-rebill above the book's consent threshold)", rep.RebillUnits, rep.RebillUSD) // BOTH denominations, always, and labelled. The chunk×stage count is what the money is billed in;
// the output-unit count is what a purchase is sized in. Printing only the first is how a reader
// hands `--max-units` a number several times too large.
drift += fmt.Sprintf(" ⚠ RE-PAYMENT: %d chunk×stage unit(s) already billed under a superseded snapshot would be paid for again — that is %d OUTPUT unit(s) of the book, the granularity --max-units counts in — ~$%.6f (translate needs --accept-rebill above the book's consent threshold). ⚠ Sizing --max-units from that number does NOT buy those units back: a grant goes to NEVER-DELIVERED units first and only reaches re-making when none are left, so on a book that still has undelivered units it will deliver new ones instead",
rep.RebillUnits, rep.RebillOutputUnits, rep.RebillUSD)
}
// A zero is not self-explanatory, and printing it as though it were is the mistake the `rebill_basis`
// field exists to stop: "nothing to re-pay" and "we could not work out what a re-pass would cost" are
// the same two zeroes. Only the two states that are NOT an answer speak here — a computed figure needs
// no caveat, and a book with nothing stored has nothing to say about re-payment at all.
switch rep.RebillBasis {
case pipeline.RebillBasisFailed:
drift += " ⚠ RE-PAYMENT UNKNOWN (the projection failed — the figures below are zero because they could not be computed, NOT because continuing is free; see the log)"
case pipeline.RebillBasisStored:
drift += " ⚠ RE-PAYMENT FIGURES ARE STALE (the book's bank could not be folded, so they describe the glossary the LAST run stored, not what the next run would build — fix the bank and re-read; see the log)"
} }
fmt.Fprintf(w, "=== STATUS: %s (snapshot %s)%s ===\n", rep.BookID, snap, drift) fmt.Fprintf(w, "=== STATUS: %s (snapshot %s)%s ===\n", rep.BookID, snap, drift)
fmt.Fprintf(w, "Progress: %d/%d units (%.1f%%) — done=%d in_progress=%d flagged=%d pending=%d\n", fmt.Fprintf(w, "Progress: %d/%d units (%.1f%%) — done=%d in_progress=%d flagged=%d pending=%d\n",

View file

@ -321,3 +321,106 @@ func TestRenderSignatureStopOrdersTheReviewLeastSureFirst(t *testing.T) {
t.Fatalf("the banner must say what its order means, or a sorted table reads as arbitrary:\n%s", out) t.Fatalf("the banner must say what its order means, or a sorted table reads as arbitrary:\n%s", out)
} }
} }
// TestRenderTranslateNamesTheVolumeCeiling pins the operator-facing half of the "the stop must NAME its
// ceiling" rule. A volume-bounded run exits 0 like any completed run, so without this line an operator
// looking at "chunks 2" on a hundred-unit book cannot tell "the volume I bought is finished" from
// "something went quietly wrong" — the silent-limit complaint the design took from OpenHands. The line
// has to carry all three actionable numbers and to say it is a completion, not a pause.
func TestRenderTranslateNamesTheVolumeCeiling(t *testing.T) {
var b strings.Builder
res := &pipeline.BookResult{BookID: "b1", TotalUSD: 0.01,
Volume: &pipeline.VolumeStop{MaxUnits: 2, Delivered: 1, Reworked: 1, Free: 3, LeftFresh: 5, LeftRework: 2}}
if err := renderTranslate(&b, res, okLedger); err != nil {
t.Fatal(err)
}
out := b.String()
for _, want := range []string{"VOLUME CEILING", "--max-units 2", "2 paying output unit(s)",
"1 NEW unit(s) delivered", "1 already-delivered unit(s) re-made",
"3 rode along at $0", "5 unit(s) NEVER delivered", "2 already delivered but not yet re-made",
"COMPLETION, not a pause",
// The invitation must be about the units that were never delivered, and only those.
"5 output unit(s) of this book have never been delivered"} {
if !strings.Contains(out, want) {
t.Errorf("the volume stop must say %q; got:\n%s", want, out)
}
}
// A run that was NOT bounded must say nothing at all — otherwise every ordinary run reads as cut short.
var b2 strings.Builder
if err := renderTranslate(&b2, &pipeline.BookResult{BookID: "b1"}, okLedger); err != nil {
t.Fatal(err)
}
if strings.Contains(b2.String(), "VOLUME CEILING") {
t.Errorf("an unbounded run must not mention a ceiling:\n%s", b2.String())
}
}
// TestRenderTranslateNeverSellsWhatIsAlreadyOwned pins the operator-facing half of the same defect. On a
// book every unit of which has been delivered, the stop must not invite a purchase: the units it held
// back are unrefreshed, not unbought, and the first version of this line offered them as "run again with
// --max-units to buy more".
func TestRenderTranslateNeverSellsWhatIsAlreadyOwned(t *testing.T) {
var b strings.Builder
res := &pipeline.BookResult{BookID: "b1",
Volume: &pipeline.VolumeStop{MaxUnits: 1, Delivered: 0, Reworked: 1, LeftFresh: 0, LeftRework: 3}}
if err := renderTranslate(&b, res, okLedger); err != nil {
t.Fatal(err)
}
out := b.String()
if strings.Contains(out, "have never been delivered") {
t.Fatalf("the stop offered chapters the reader already owns:\n%s", out)
}
for _, want := range []string{
"every unit of this book HAS been delivered",
"That is a re-pass, not a purchase of new book",
"delivered NO new unit",
} {
if !strings.Contains(out, want) {
t.Errorf("want %q in:\n%s", want, out)
}
}
}
// TestRenderTranslateReportsBothRemaindersWhenBothExist pins the case an if/else would have swallowed:
// a book with BOTH never-delivered units and delivered-but-superseded ones. Printing only the first
// leaves an operator who has both believing the one number was the whole picture.
func TestRenderTranslateReportsBothRemaindersWhenBothExist(t *testing.T) {
var b strings.Builder
res := &pipeline.BookResult{BookID: "b1",
Volume: &pipeline.VolumeStop{MaxUnits: 2, Delivered: 2, LeftFresh: 4, LeftRework: 6}}
if err := renderTranslate(&b, res, okLedger); err != nil {
t.Fatal(err)
}
out := b.String()
if !strings.Contains(out, "4 output unit(s) of this book have never been delivered") {
t.Errorf("the buyable remainder is missing:\n%s", out)
}
if !strings.Contains(out, "6 already-delivered unit(s) carry a superseded snapshot") {
t.Errorf("the re-make remainder was dropped — an if/else over the two would do exactly this:\n%s", out)
}
// A run that DID deliver must not carry the "delivered NO new unit" warning.
if strings.Contains(out, "delivered NO new unit") {
t.Errorf("this run delivered 2 units; it must not claim otherwise:\n%s", out)
}
}
// TestTheRePaymentHintDoesNotPromiseWhatTheGrantWillNotDo pins the acceptance hunter's finding 6. The
// hint offers rebill_output_units as "the granularity --max-units counts in", which is true and, on its
// own, misleading: the grant goes to NEVER-DELIVERED units first, so on a book that still has any, sizing
// --max-units from the re-payment figure delivers new units and re-makes nothing. The caveat lives in the
// code; it has to live where the operator reads.
func TestTheRePaymentHintDoesNotPromiseWhatTheGrantWillNotDo(t *testing.T) {
var b strings.Builder
rep := &pipeline.StatusReport{BookID: "b1", ConfigDrift: true,
RebillUnits: 9, RebillOutputUnits: 3, RebillUSD: 0.02, RebillBasis: pipeline.RebillBasisPending}
if err := renderStatusHuman(&b, rep, ""); err != nil {
t.Fatal(err)
}
out := b.String()
if !strings.Contains(out, "3 OUTPUT unit(s)") {
t.Fatalf("the sale-sized figure must be shown:\n%s", out)
}
if !strings.Contains(out, "does NOT buy those units back") {
t.Fatalf("the hint must say the grant goes to never-delivered units first, or it promises a re-pass it will not perform:\n%s", out)
}
}

View file

@ -8,7 +8,9 @@ import (
// artifact.go: the shared write discipline of the engine's READ-OUT FILES — the sidecars beside the // artifact.go: the shared write discipline of the engine's READ-OUT FILES — the sidecars beside the
// project DB that another process (the platform, D39.81/D39.85) reads while this one runs: the // project DB that another process (the platform, D39.81/D39.85) reads while this one runs: the
// chapter/chunk manifest (row 100), the machine bank-stop table (row 101), the bank export (row 125). // chapter/chunk manifest (row 100), the machine bank-stop table (row 101), the bank export (row 125) and
// the engine's auto-bank (row 231 — it joined this list when the read-only surfaces started folding the
// bank from the decision files themselves, which is what made it a concurrently-read document).
// //
// They have one property in common that the older sidecars did not have to care about: they are read // They have one property in common that the older sidecars did not have to care about: they are read
// CONCURRENTLY with the run that writes them. A plain os.WriteFile truncates first, so a reader that // CONCURRENTLY with the run that writes them. A plain os.WriteFile truncates first, so a reader that

View file

@ -0,0 +1,403 @@
package pipeline
import (
"fmt"
"sort"
"textmachine/backend/internal/lang"
"textmachine/backend/internal/membank"
"textmachine/backend/internal/store"
)
// bankmaterialize.go: the book's bank fold, split so that the SAME fold can be performed with or without the
// store round-trip that sits in the middle of it (row 231 / D39.165 §3, errata 28.08-к).
//
// THE PROBLEM IT EXISTS FOR. `seedGlossary` folds the bank from FILES (the curated seed, the ruby
// readings, the owner's mined-delta, the engine's auto-bank), writes the result with ReplaceBank, reads
// it straight back with GlossaryForBook and materializes THAT. Every read-only surface, however, had no
// way to reach the same answer: it cannot write, so `projectStoredMemory` materialized the glossary the
// LAST run stored — i.e. the previous fold. Right after a `bank-apply` the decision files hold rows the
// stored glossary does not, so a read-only projection answered "nothing moved" while the very next
// `translate` would re-seed, see the rows and bill for them. The projection and the run disagreed about
// money, which is the one thing they may never do.
//
// WHY THE ROUND-TRIP IS NOT A FORMALITY, and what makes skipping it PROVABLE rather than plausible.
// The store contributes exactly one thing to the fold: an ORDER. `GlossaryForBook` returns
// `ORDER BY src, sense, since_ch, until_ch, status, dst`, and that order is load-bearing twice over —
// `MaterializeBank` iterates the rows as given (it never re-sorts), so the order decides both the bank's
// content hash (ComputeVersionScopedIn streams a SHA-256 over the rows in iteration order) and, through
// Select's stable budget sort, the literal LINE ORDER of the rendered glossary block, whose bytes are
// hashed into chunk_status.content_hash. Reproduce the set but not the order and the free estimate names
// one number while the paid run charges another.
//
// The reproduction is exact, and the reason is a constraint rather than an observation: `glossary`
// carries UNIQUE (book_id, src, sense, since_ch, until_ch) (store/migrate.go), so the ORDER BY's first
// four columns CANNOT tie within one book — the order is total and fully determined by values, with no
// dependence on rowid or insertion order (`glossary.id` is deliberately excluded from the fold, see
// membank.ComputeVersion). The schema declares no COLLATE anywhere, so SQLite compares TEXT with its
// default BINARY collation — byte-wise, which is exactly what Go's `<` on a string does. Aliases come
// back `ORDER BY term_id, alias`, i.e. sorted by alias within an entry. Voice profiles and address pairs
// each have their own ORDER BY matching their own UNIQUE key, so the same argument covers them.
//
// storeOrder below therefore reproduces the round-trip's ONE contribution, and bankmaterialize_identity_test.go
// proves the equality by running both paths over the same inputs rather than by asserting this comment.
// bankInputs is everything the deterministic $0 prelude gathers about a book's bank BEFORE anything is
// written: the glossary rows in the order seedGlossary builds them (which is the order ReplaceBank
// inserts them in), the two pack-19 record types, and the non-fatal remarks the gather produced.
//
// It is deliberately the PRE-store form. Turning it into what a run actually materializes is storeOrder's
// job, and it is one function so a caller cannot accidentally materialize the unsorted build order.
type bankInputs struct {
entries []store.GlossaryEntry
voices []store.VoiceProfile
pairs []store.AddressPair
// remarks are the gather's non-fatal findings, in the order they were made. seedGlossary logs them;
// a read-only projection drops them, because a $0 read must not narrate a run it is not making.
remarks []bankRemark
}
// bankRemark is one non-fatal finding of the gather, carried rather than logged so the two callers can
// decide for themselves whether their surface is one that speaks.
type bankRemark struct {
msg string
args []any
}
// gatherBankInputs performs the pure, $0, WRITE-FREE half of the bank fold: it reads the book's
// deterministic inputs and assembles the exact row set seedGlossary hands to ReplaceBank.
//
// Everything fatal here is fatal for the same reason it was fatal inside seedGlossary — a collision that
// would crash ReplaceBank mid-run, or a voice row naming a character the bank does not have, is a broken
// book and not a projection detail. What the two callers differ in is what they DO with that error:
// `translate` fails the run, a read-only projection falls back and says so.
func (r *Runner) gatherBankInputs() (bankInputs, error) {
var in bankInputs
if r.Book.GlossarySeed != "" {
bank, err := membank.LoadBankSeed(r.Book.GlossarySeed)
if err != nil {
return in, err
}
in.entries = append(in.entries, bank.Terms...)
in.voices, in.pairs = bank.Voices, bank.Pairs
}
ruby, err := r.Store.RubyReadingsForBook(r.Book.BookID)
if err != nil {
return in, fmt.Errorf("pipeline: read ruby readings for %s: %w", r.Book.BookID, err)
}
// D16.4: attach a manual term's kana ruby-reading as an alias (kana spelling matchable)
// BEFORE appending the auto-candidates, so it only touches the curated manual entries. A
// reading that would collide with a different seeded term (homophone) is skipped+logged,
// not attached (which would fail the book loud on an alias the operator cannot edit out).
if skipped := membank.AttachRubyAliasesToManual(in.entries, ruby); len(skipped) > 0 {
in.remark("ruby kana-alias skipped as a homophone collision (kana form left unmatchable; disambiguate in the seed if needed)",
"skipped", joinSemi(skipped))
}
// Mined-write path (R1, plan §1(в)/F2): the owner-curated mined-delta file joins the seed as
// Source:"mined" (loadMinedDelta re-stamps the seed loader's Source), so its terms fold into the
// ENRICHED bank version but NOT the base — a re-run that adds signed mined terms moves ONLY snapshot_W2.
minedDelta, err := r.loadMinedDelta()
if err != nil {
return in, err
}
// D39.20 deviation-#1 fix: a mined-delta term whose UNIQUE key (src, sense, since_ch, until_ch —
// store/migrate.go glossary UNIQUE) already exists in the SIGNED seed makes the flat INSERT in
// ReplaceGlossary crash on that constraint and abort the whole paid run. membank.ApprovedSharedKeyCollisions
// below does NOT catch it (it skips a SAME-dst duplicate, and keys on the firing surface, not the
// UNIQUE tuple). Fail LOUD here with the duplicate list + a fix hint (edit the seed OR drop it from
// the delta) — NEVER a silent merge over the signed seed. Checked BEFORE the append so the delta rows
// are still separable. Deterministic (seed order).
if dups := membank.MinedDeltaSeedCollisions(in.entries, minedDelta); len(dups) > 0 {
return in, fmt.Errorf("pipeline: mined-delta %s duplicates term(s) already in the signed seed (would crash ReplaceGlossary on the glossary UNIQUE(book_id,src,sense,since_ch,until_ch)):\n - %s\n fix: correct the term in the seed, or remove it from the mined-delta file — never both (no silent merge over the signed seed)",
r.Book.MinedDelta, joinLines(dups))
}
in.entries = append(in.entries, minedDelta...)
// AUTO-BANK (pack-20 / D39.42 п.3): the engine's own unsigned rows — what the terminologist consolidated
// on the last run's bank-mining boundary. They load exactly like the owner-curated delta (Source:"mined",
// so base-excluded and only the edit-wave snapshot moves) but they are NOT signed: their status is
// auto/draft, which is what routes them to the editor's separately-headed unverified section instead of
// the canon list.
//
// Loading them HERE rather than injecting them mid-run is load-bearing for money. The gather runs before
// checkRebillConsent, so the re-payment projection sees the same bank the edit wave will use; if the rows
// only appeared after the projection, the next run would compare stored edit rows against a bank that has
// not been rebuilt yet and project a re-payment of the whole edit wave that is not real.
autoBank, dropped, err := r.loadAutoBank(in.entries)
if err != nil {
return in, err
}
if len(dropped) > 0 {
// A collision with a SIGNED row cannot abort a paid run over an engine-written proposal — the signed
// row simply wins and the proposal is dropped, loudly.
in.remark("auto-bank rows dropped: their key is already held by a signed term (the signed term wins)",
"book", r.Book.BookID, "dropped", joinSemi(dropped))
}
in.entries = append(in.entries, autoBank...)
for i := range in.entries {
in.entries[i].BookID = r.Book.BookID
}
// Task-6 re-audit: fail loud on a firing key shared by two DIFFERENT approved terms with
// different dst + overlapping windows (the alias generalization of the D16.1 polysemy
// livelock) — checked over the FULL entry set (incl. ruby-attached aliases) before persisting.
if cols := membank.ApprovedSharedKeyCollisions(in.entries); len(cols) > 0 {
return in, fmt.Errorf("pipeline: glossary shared-key collisions (A2 / D16.1 livelock class):\n - %s", joinLines(cols))
}
// A voice/address row naming a character the bank does not have is SILENTLY inert — nothing can ever
// attribute a reply to it — which is the A-class hole this bank exists to close, so it stops the run.
// Checked over the FULL entry set (seed + ruby + mined + auto), because a character may legitimately be
// signed in a delta rather than in the base seed.
if unknown := membank.UnknownVoiceCharacters(in.entries, in.voices, in.pairs); len(unknown) > 0 {
return in, fmt.Errorf("pipeline: voice/address rows name characters absent from the bank (a profile for a term that does not exist can never fire):\n - %s", joinLines(unknown))
}
return in, nil
}
func (in *bankInputs) remark(msg string, args ...any) {
in.remarks = append(in.remarks, bankRemark{msg: msg, args: args})
}
// storeOrder reproduces, WITHOUT a store, exactly what a ReplaceBank+read-back round-trip returns: the
// same rows in the store's ORDER BY, each entry's aliases in the store's alias order, and the same for
// voice profiles and address pairs. The inputs are left untouched — a caller may still hand the ORIGINAL
// slice to ReplaceBank and get the insert order it has always had.
//
// It REFUSES the states the store itself would refuse rather than papering over them, because a
// projection of a book whose fold would crash the next run is not a projection of anything: a duplicate
// UNIQUE key would abort ReplaceBank's flat INSERT mid-transaction, and reporting a tidy number for it
// would promise a run that cannot start. The three refusals correspond one-to-one to the three UNIQUE
// constraints the round-trip passes through.
func storeOrder(in bankInputs) (rows []store.GlossaryEntry, voices []store.VoiceProfile, pairs []store.AddressPair, err error) {
type gkey struct {
src, sense string
since, until int
}
rows = append(rows, in.entries...)
seen := make(map[gkey]bool, len(rows))
for i := range rows {
k := gkey{rows[i].Src, rows[i].Sense, rows[i].SinceCh, rows[i].UntilCh}
if seen[k] {
return nil, nil, nil, fmt.Errorf("pipeline: the folded bank holds two rows for (src=%q, sense=%q, since_ch=%d, until_ch=%d) — the glossary UNIQUE key; ReplaceBank would abort on it", k.src, k.sense, k.since, k.until)
}
seen[k] = true
// The store returns a term's aliases ORDER BY alias (within its term_id), and refuses a repeated
// one on UNIQUE (book_id, term_id, alias). The aliases are copied before sorting so an input slice
// shared with the write path keeps the order ReplaceBank inserts in.
if len(rows[i].Aliases) > 1 {
al := append([]store.GlossaryAlias(nil), rows[i].Aliases...)
sort.SliceStable(al, func(a, b int) bool { return al[a].Alias < al[b].Alias })
for j := 1; j < len(al); j++ {
if al[j].Alias == al[j-1].Alias {
return nil, nil, nil, fmt.Errorf("pipeline: the folded bank gives term %q the alias %q twice — the glossary_aliases UNIQUE key; ReplaceBank would abort on it", rows[i].Src, al[j].Alias)
}
}
rows[i].Aliases = al
}
}
// GlossaryForBook: ORDER BY src, sense, since_ch, until_ch, status, dst. The first four columns are the
// table's UNIQUE key, so status/dst can never actually decide anything — they are kept so this reads as
// the SQL it reproduces and would still hold if that constraint were ever widened.
sort.SliceStable(rows, func(a, b int) bool { return glossaryLess(rows[a], rows[b]) })
voices = append(voices, in.voices...)
vseen := make(map[gkey]bool, len(voices))
for _, v := range voices {
k := gkey{v.Src, v.Sense, v.SinceCh, v.UntilCh}
if vseen[k] {
return nil, nil, nil, fmt.Errorf("pipeline: the folded bank holds two voice profiles for (src=%q, sense=%q, since_ch=%d, until_ch=%d) — the voice_profiles UNIQUE key; ReplaceBank would abort on it", k.src, k.sense, k.since, k.until)
}
vseen[k] = true
}
// VoiceProfilesForBook: ORDER BY src, sense, since_ch, until_ch.
sort.SliceStable(voices, func(a, b int) bool {
x, y := voices[a], voices[b]
return lessKey4(x.Src, y.Src, x.Sense, y.Sense, x.SinceCh, y.SinceCh, x.UntilCh, y.UntilCh)
})
type pkey struct {
spSrc, spSense, adSrc, adSense string
since, until int
}
pairs = append(pairs, in.pairs...)
pseen := make(map[pkey]bool, len(pairs))
for _, p := range pairs {
k := pkey{p.SpeakerSrc, p.SpeakerSense, p.AddresseeSrc, p.AddresseeSense, p.SinceCh, p.UntilCh}
if pseen[k] {
return nil, nil, nil, fmt.Errorf("pipeline: the folded bank holds two address pairs for speaker %q → addressee %q over the same window — the address_pairs UNIQUE key; ReplaceBank would abort on it", k.spSrc, k.adSrc)
}
pseen[k] = true
}
// AddressPairsForBook: ORDER BY speaker_src, speaker_sense, addressee_src, addressee_sense, since_ch, until_ch.
sort.SliceStable(pairs, func(a, b int) bool {
x, y := pairs[a], pairs[b]
if x.SpeakerSrc != y.SpeakerSrc {
return x.SpeakerSrc < y.SpeakerSrc
}
if x.SpeakerSense != y.SpeakerSense {
return x.SpeakerSense < y.SpeakerSense
}
if x.AddresseeSrc != y.AddresseeSrc {
return x.AddresseeSrc < y.AddresseeSrc
}
if x.AddresseeSense != y.AddresseeSense {
return x.AddresseeSense < y.AddresseeSense
}
if x.SinceCh != y.SinceCh {
return x.SinceCh < y.SinceCh
}
return x.UntilCh < y.UntilCh
})
return rows, voices, pairs, nil
}
// glossaryLess is GlossaryForBook's ORDER BY as a Go comparison. Both sides are byte-wise on TEXT (the
// schema declares no COLLATE, so SQLite uses BINARY) and numeric on the two INTEGER columns.
func glossaryLess(x, y store.GlossaryEntry) bool {
if x.Src != y.Src {
return x.Src < y.Src
}
if x.Sense != y.Sense {
return x.Sense < y.Sense
}
if x.SinceCh != y.SinceCh {
return x.SinceCh < y.SinceCh
}
if x.UntilCh != y.UntilCh {
return x.UntilCh < y.UntilCh
}
if x.Status != y.Status {
return x.Status < y.Status
}
return x.Dst < y.Dst
}
func lessKey4(s1, s2, e1, e2 string, a1, a2, b1, b2 int) bool {
if s1 != s2 {
return s1 < s2
}
if e1 != e2 {
return e1 < e2
}
if a1 != a2 {
return a1 < a2
}
return b1 < b2
}
// materializeBanks builds the two banks a run works against — the ENRICHED one the editor sees and the
// BASE one (Source:"mined" excluded) the draft wave selects over — from bank content already in the
// store's order. It is the shared tail of both folds, so the write path and the read-only projection can
// never materialize the same rows two different ways.
func (r *Runner) materializeBanks(rows []store.GlossaryEntry, voices []store.VoiceProfile, pairs []store.AddressPair) {
// InjectVoice is FALSE and has no config knob: pack-19 builds the schema and the flagger, and D21 п.2
// holds the injection conditional until the polygon experiment. It is the fold's condition, so while
// it is false a book with voice rows hashes exactly as it did without them and nobody re-pays for
// authoring a profile. Wiring the injection means setting it and accepting a full --resnapshot.
// The §3 decl stemmer is target data, constant per book; baseIn copies bankIn below, so both banks share
// it. A target with no decl_suffix registry yields an inert stemmer → exact-match post-check as before.
// The rows are retained because a caller that has just folded the bank must be able to say things
// ABOUT it — how many of its terms are unsigned, for one — without going back to the store and
// getting a DIFFERENT bank than the one it just materialized. A status document that folds the
// decision files for its money figures and counts unsigned terms off the stored glossary is one
// document describing two banks.
r.bankRows = rows
// The wire-hash memo belongs to the bank that produced it: a new materialization invalidates every
// hash rendered against the old one (cachedRenderedContentHashes).
r.contentHashes = nil
bankIn := membank.BankInput{Rows: rows, Voices: voices, Pairs: pairs,
TargetStemmer: lang.NewTargetStemmer(lang.TargetChecksFor(r.Book.TargetLang))}
r.memory = membank.MaterializeBank(bankIn, r.Pipeline.Gates.Glossary.PostcheckGate)
// The DRAFT wave selects over a BASE-scoped bank (Source:mined excluded) so its injection is
// byte-identical across a bank-mining enrichment — matching the draft-wave snapshot (baseMemoryVersion),
// which keeps «one re-payment» honest at the WIRE level, not only the version-hash level. Only the
// editor sees mined terms (the enriched `memory`). When there are no mined rows (every $0 test / the
// golden) the base bank IS the enriched one — share the object, no double materialization, no drift.
baseRows := rows[:0:0]
hasMined := false
for _, row := range rows {
if row.Source == "mined" {
hasMined = true
continue
}
baseRows = append(baseRows, row)
}
if hasMined {
baseIn := bankIn
baseIn.Rows = baseRows
r.baseMemory = membank.MaterializeBank(baseIn, r.Pipeline.Gates.Glossary.PostcheckGate)
} else {
r.baseMemory = r.memory
}
}
// projectFoldedMemory materializes BOTH banks from the book's deterministic inputs WITHOUT writing
// anything — the read-only twin of seedGlossary, and the answer to backlog row 231.
//
// What it fixes is a disagreement about money. `translate` re-seeds from the FILES and does it BEFORE
// checkRebillConsent, so its consent gate sees a `bank-apply` edit the moment the files change; the
// read-only surfaces materialized the STORED glossary instead — last run's fold — and therefore reported
// "nothing moved" for exactly the edit the next run would bill for. Reading the files rather than the
// store keeps every read-only guarantee intact: this writes nothing at all, so a store opened
// query_only(1) is untouched, and `status` remains the purely-reading verb it is ratified as.
//
// It also sets r.baseMemory, which projectStoredMemory never did — see the note on projectStoredMemory.
func (r *Runner) projectFoldedMemory() error {
in, err := r.gatherBankInputs()
if err != nil {
return err
}
rows, voices, pairs, err := storeOrder(in)
if err != nil {
return err
}
r.materializeBanks(rows, voices, pairs)
return nil
}
// The values of StatusReport.RebillBasis — see its doc comment for what each one promises a reader.
// Exported because they are a WIRE vocabulary: the CLI renders them and a consumer branches on them, and
// a second hand-typed copy of an enum in the renderer is how the two drift apart.
const (
RebillBasisPending = "pending"
RebillBasisStored = "stored"
RebillBasisNone = "none"
RebillBasisFailed = "failed"
)
// foldMemoryForRead materializes the bank a READ-ONLY surface must judge against, and reports WHICH bank
// it managed to get. It is the one place the fallback lives, so status and export can never end up
// judging drift against two different banks.
//
// The preferred answer is the FOLD of the book's current inputs, because that is the bank the next
// `translate` will build (it re-seeds before its consent gate) — projecting anything else is projecting a
// run nobody is going to make. The fallback exists because the fold can legitimately refuse: a seed with a
// shared-key collision, a mined-delta duplicating a signed term, a voice row naming an absent character.
// Those are broken books, and `translate` fails loudly on them — but a READ command must not, or the book
// that most needs inspecting becomes the one that cannot be inspected (the D20.4 property rebill.go leans
// on: "a book that needs consent stays fully inspectable"). So the read falls back to the glossary the
// last run stored, and says so through the basis rather than passing the older number off as the answer.
//
// The returned error is the fold's refusal, for the caller to LOG in its own idiom; it is never fatal.
func (r *Runner) foldMemoryForRead() (basis string, warn error) {
foldErr := r.projectFoldedMemory()
if foldErr == nil {
return RebillBasisPending, nil
}
if storedErr := r.projectStoredMemory(); storedErr != nil {
return RebillBasisFailed, fmt.Errorf("the bank fold refused (%w) and the stored glossary could not be materialized either: %w", foldErr, storedErr)
}
return RebillBasisStored, foldErr
}
func joinSemi(v []string) string { return joinWith(v, "; ") }
func joinLines(v []string) string { return joinWith(v, "\n - ") }
func joinWith(v []string, sep string) string {
out := ""
for i, s := range v {
if i > 0 {
out += sep
}
out += s
}
return out
}

View file

@ -0,0 +1,414 @@
package pipeline
import (
"bytes"
"context"
"log/slog"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"textmachine/backend/internal/miner"
"textmachine/backend/internal/obs"
"textmachine/backend/internal/store"
)
// identitySeed is deliberately hostile to the fold: every ordered axis is presented in the WRONG order.
// The terms descend by src where the store ascends, one term carries two senses whose order is inverted,
// two terms share a src and differ only by spoiler window (presented later-window-first), the aliases of
// one term are listed in reverse, and the voice/address rows are scrambled against their own ORDER BY.
// A fold that "keeps the order it was given" therefore cannot accidentally pass this fixture.
const identitySeed = `
terms:
- src: 鈴木
dst: Судзуки
type: name
status: approved
since_ch: 5
decl: { invariant: true, forms: ["Судзуки"] }
aliases:
- { alias: すずき, type: name }
- { alias: SUZUKI, type: name }
- { alias: Suzuki, type: name }
- src: 鈴木
dst: Судзуки-старший
type: name
status: approved
since_ch: 1
until_ch: 4
decl: { invariant: true, forms: ["Судзуки-старший"] }
- src: 図書館
dst: библиотека
type: term
status: approved
sense: здание
since_ch: 3
decl: { invariant: false, forms: ["библиотека", "библиотеки"] }
- src: 図書館
dst: книгохранилище
type: term
status: draft
sense: а-архаизм
since_ch: 1
until_ch: 2
- src:
dst: утро
type: term
status: approved
voices:
- src: 図書館
sense: здание
register: нейтральный
self_ref: я
address_default: formal
since_ch: 3
- src: 鈴木
register: разговорный
self_ref: я
address_default: informal
since_ch: 1
until_ch: 4
- src: 鈴木
register: книжный
self_ref: я
address_default: formal
since_ch: 5
addresses:
- speaker: 鈴木
addressee: 図書館
addressee_sense: здание
register: formal
form: вы
closeness: далеко
since_ch: 5
- speaker: 図書館
speaker_sense: здание
addressee: 鈴木
register: informal
form: ты
closeness: близко
since_ch: 1
`
// identityDelta is the owner's decision file — the state that only exists AFTER a `bank-apply` and that
// the stored glossary therefore does not hold. Its rows sort INTO the middle of the seed's, so a fold
// that appended them instead of merging them by value would be caught.
const identityDelta = `
terms:
- src: 静か
dst: тихий
type: term
status: approved
- src: 元海
dst: Изначальное море
type: term
status: approved
`
// identityAutoBank is the engine's own unsigned proposal file (Source:"mined", base-excluded), included
// so the fixture exercises the BASE/ENRICHED split as well as the plain ordering.
const identityAutoBank = `
terms:
- src: 行った
dst: пошёл
type: term
status: auto
`
// bankFoldFixture builds a project whose bank is folded from all four sources at once — curated seed,
// ruby readings, the owner's mined-delta and the engine's auto-bank — and returns the book path.
func bankFoldFixture(t *testing.T, providerURL string) string {
t.Helper()
bookPath := setupProjectOpts(t, providerURL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: identitySeed,
})
dir := filepath.Dir(bookPath)
writeFile(t, filepath.Join(dir, "test-book.mined-delta.yaml"), identityDelta)
writeFile(t, filepath.Join(dir, "test-book.db.auto-bank.yaml"), identityAutoBank)
return bookPath
}
// stripIDs zeroes the store's autoincrement id, the ONE field the round-trip adds and the fold never
// hashes (membank.ComputeVersion excludes it by construction). Everything else must match exactly.
func stripIDs(rows []store.GlossaryEntry) []store.GlossaryEntry {
out := append([]store.GlossaryEntry(nil), rows...)
for i := range out {
out[i].ID = 0
}
return out
}
// TestTheInMemoryFoldIsIdenticalToTheStoreRoundTrip is the proof the free estimate rests on, and it is a
// proof by EXECUTION rather than by argument: both folds are run over the same inputs and every quantity
// that money depends on is compared.
//
// What is at stake. The canonical fold goes THROUGH the store — ReplaceBank writes, GlossaryForBook reads
// back ORDER BY src, sense, since_ch, until_ch, status, dst — and that order decides the bank's content
// hash and, through Select's stable budget sort, the literal bytes of the rendered glossary block, which
// are hashed into chunk_status.content_hash. A read-only surface cannot write, so it must reproduce the
// order without the store. If it reproduced it only approximately, the free projection would name one
// number and the paid run would charge another — which is precisely the defect the projection is sold as
// fixing.
//
// The comparison is made at three depths on purpose: the rows themselves (so a divergence is localised),
// the two bank versions (the snapshot component), and the per-position rendered content hashes (the wire
// bytes the resume fast-path and the re-bill projection actually compare).
func TestTheInMemoryFoldIsIdenticalToTheStoreRoundTrip(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := bankFoldFixture(t, srv.URL)
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
// A RUN MUST HAVE HAPPENED before either fold is measured, and it is load-bearing rather than
// scene-setting. renderedContentHashes reproduces an EDIT position only when every member's stored
// draft is retrievable (repin.go), so on a book with no chunk_status rows the hash map comes back
// draft-only — and the edit wave is exactly the half the mined-delta and auto-bank rows reach, since
// Source:"mined" folds into the ENRICHED bank alone. Without the run this comparison would be blind to
// every possible defect in the half it was written to prove.
r0 := newRunner(t, bookPath)
if err := r0.Store.ReplaceRubyReadings("test-book", []store.RubyReading{
{BookID: "test-book", Base: "鈴木", Reading: "すずき", FirstChapter: 1, Occurrences: 3},
{BookID: "test-book", Base: "図書館", Reading: "としょかん", FirstChapter: 1, Occurrences: 2},
}); err != nil {
t.Fatal(err)
}
if _, err := r0.TranslateBook(ctx); err != nil {
t.Fatalf("the run this comparison rests on did not complete: %v", err)
}
r0.Close()
// --- Path A: the canonical fold, through the store ---
ra := newRunner(t, bookPath)
if err := ra.seedGlossary(ctx); err != nil {
t.Fatalf("canonical fold: %v", err)
}
rowsA, err := ra.Store.GlossaryForBook("test-book")
if err != nil {
t.Fatal(err)
}
voicesA, err := ra.Store.VoiceProfilesForBook("test-book")
if err != nil {
t.Fatal(err)
}
pairsA, err := ra.Store.AddressPairsForBook("test-book")
if err != nil {
t.Fatal(err)
}
verA, baseA := ra.memory.Version(), ra.baseMemory.Version()
hashesA := foldRenderedHashes(t, ra)
// The project flock is exclusive, so the second runner cannot open until the first lets go.
ra.Close()
// --- Path B: the same fold with the write removed ---
rb := newRunner(t, bookPath)
defer rb.Close()
if err := rb.projectFoldedMemory(); err != nil {
t.Fatalf("in-memory fold: %v", err)
}
in, err := rb.gatherBankInputs()
if err != nil {
t.Fatal(err)
}
rowsB, voicesB, pairsB, err := storeOrder(in)
if err != nil {
t.Fatal(err)
}
verB, baseB := rb.memory.Version(), rb.baseMemory.Version()
hashesB := foldRenderedHashes(t, rb)
// The fixture must actually exercise the ordering — a fold that never had to sort anything would
// pass this test while proving nothing.
if len(rowsA) < 8 {
t.Fatalf("fixture too thin to prove an ordering: %d rows", len(rowsA))
}
if reflect.DeepEqual(stripIDs(rowsA), stripIDs(in.entries)) {
t.Fatalf("fixture is degenerate: the BUILD order already equals the store order, so sorting is untested")
}
if !reflect.DeepEqual(stripIDs(rowsA), stripIDs(rowsB)) {
for i := range rowsA {
if i >= len(rowsB) {
t.Fatalf("row %d: store has %+v, memory ran out (%d rows)", i, rowsA[i], len(rowsB))
}
a, b := rowsA[i], rowsB[i]
a.ID, b.ID = 0, 0
if !reflect.DeepEqual(a, b) {
t.Fatalf("row %d differs:\n store = %+v\n memory = %+v", i, a, b)
}
}
t.Fatalf("row counts differ: store %d, memory %d", len(rowsA), len(rowsB))
}
if !reflect.DeepEqual(voicesA, voicesB) {
t.Fatalf("voice profiles differ:\n store = %+v\n memory = %+v", voicesA, voicesB)
}
if !reflect.DeepEqual(pairsA, pairsB) {
t.Fatalf("address pairs differ:\n store = %+v\n memory = %+v", pairsA, pairsB)
}
if verA != verB {
t.Fatalf("enriched bank version differs: store %s, memory %s", verA, verB)
}
if baseA != baseB {
t.Fatalf("base bank version differs: store %s, memory %s", baseA, baseB)
}
if baseA == verA {
t.Fatalf("fixture is degenerate: the auto-bank row must make base ≠ enriched, both are %s", verA)
}
if !reflect.DeepEqual(hashesA, hashesB) {
t.Fatalf("rendered content hashes differ — the free estimate would name a number the paid run does not charge:\n store = %+v\n memory = %+v", hashesA, hashesB)
}
// ⚠ NOT merely "non-empty". Two empty maps compare equal, and so do two DRAFT-ONLY maps — and the
// draft wave is the half the fixture's mined rows do not even reach (Source:"mined" folds into the
// ENRICHED bank only). The comparison is only a proof if it actually covers the edit wave, so the
// coverage is asserted rather than assumed.
waves := map[string]int{}
for _, byStage := range hashesA {
for stage := range byStage {
waves[stage]++
}
}
if waves["draft"] == 0 || waves["edit"] == 0 {
t.Fatalf("the byte-level comparison must cover BOTH waves — the mined rows this fixture carries reach only the edit one; got %v", waves)
}
}
// foldRenderedHashes reproduces the per-position wire hashes exactly as the re-bill projection does
// (rebill.go), so the comparison is over the quantity money is decided by rather than over a proxy.
func foldRenderedHashes(t *testing.T, r *Runner) map[chunkKey]map[string]string {
t.Helper()
chunks, withText, err := r.readModelChunks()
if err != nil {
t.Fatalf("read-model chunks: %v", err)
}
_ = chunks
full, err := withText()
if err != nil {
t.Fatalf("re-chunk with text: %v", err)
}
return r.renderedContentHashes(full, precomputeSticky(full, r.baseMemory, r.Pipeline.Context.GlossaryTokenBudget))
}
// TestTheInMemoryFoldRefusesWhatTheStoreWouldRefuse pins the other half of the identity: where the
// round-trip would ABORT, the projection must abort too. A fold that quietly de-duplicated a repeated
// UNIQUE key would answer with a tidy number for a book whose next run cannot even start.
func TestTheInMemoryFoldRefusesWhatTheStoreWouldRefuse(t *testing.T) {
dup := store.GlossaryEntry{BookID: "b", Src: "鈴木", Dst: "Судзуки", Status: "approved"}
if _, _, _, err := storeOrder(bankInputs{entries: []store.GlossaryEntry{dup, dup}}); err == nil {
t.Fatalf("a repeated glossary UNIQUE key must refuse, not de-duplicate")
}
aliased := store.GlossaryEntry{BookID: "b", Src: "図書館", Dst: "библиотека", Status: "approved",
Aliases: []store.GlossaryAlias{{Alias: "としょかん"}, {Alias: "としょかん"}}}
if _, _, _, err := storeOrder(bankInputs{entries: []store.GlossaryEntry{aliased}}); err == nil {
t.Fatalf("a repeated alias must refuse: glossary_aliases carries UNIQUE (book_id, term_id, alias)")
}
v := store.VoiceProfile{Src: "鈴木", Register: "разговорный"}
if _, _, _, err := storeOrder(bankInputs{voices: []store.VoiceProfile{v, v}}); err == nil {
t.Fatalf("a repeated voice-profile UNIQUE key must refuse")
}
p := store.AddressPair{SpeakerSrc: "鈴木", AddresseeSrc: "図書館"}
if _, _, _, err := storeOrder(bankInputs{pairs: []store.AddressPair{p, p}}); err == nil {
t.Fatalf("a repeated address-pair UNIQUE key must refuse")
}
}
// TestAFailedFoldStillReportsWhatItSkipped pins a regression this pack introduced and then fixed. Before
// the gather was extracted from seedGlossary, its non-fatal warnings were emitted inline as they were
// made, so a book that died on a LATER check had still told the operator what the earlier steps skipped.
// The first version of the extraction logged the collected remarks only after the error return — losing
// the diagnostics of exactly the run that needs them most.
func TestAFailedFoldStillReportsWhatItSkipped(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
// The seed holds a signed term, and a voice row for a character the bank does NOT have — the latter is
// fatal, and it is checked AFTER the auto-bank load.
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: `
terms:
- src: 鈴木
dst: Судзуки
type: name
status: approved
decl: { invariant: true, forms: ["Судзуки"] }
voices:
- src: 存在しない
register: книжный
self_ref: я
address_default: formal
`})
dir := filepath.Dir(bookPath)
// An auto-bank row on the key the signed seed term already holds: dropped with a REMARK, and the drop
// happens BEFORE the fatal voice check.
writeFile(t, filepath.Join(dir, "test-book.db.auto-bank.yaml"), `
terms:
- src: 鈴木
dst: Сузуки
type: name
status: auto
`)
var logBuf bytes.Buffer
r := newRunner(t, bookPath)
defer r.Close()
r.Log = slog.New(slog.NewTextHandler(&logBuf, &slog.HandlerOptions{Level: slog.LevelWarn}))
err := r.seedGlossary(context.Background())
if err == nil {
t.Fatalf("precondition: a voice row naming an absent character must fail the fold")
}
if !strings.Contains(err.Error(), "absent from the bank") {
t.Fatalf("precondition: expected the voice-character refusal, got: %v", err)
}
if !strings.Contains(logBuf.String(), "auto-bank rows dropped") {
t.Fatalf("the fold died and took its own diagnostics with it; the dropped auto-bank row was never reported.\nlog was:\n%s", logBuf.String())
}
}
// TestTheAutoBankIsWrittenAtomically pins a hole THIS pack opened in someone else's file. Until the
// read-only surfaces began folding the bank from the decision documents themselves, the auto-bank was
// read only by the run that had just written it, so a plain truncating os.WriteFile was safe. The moment
// `status` and `export` started reading it, it became one of the concurrently-read sidecars artifact.go
// exists for — and a truncating write hands a concurrent reader either zero bytes (a parse error, so the
// buyer sees stale figures) or a valid PREFIX of the term list, which is worse: a fold of a bank that
// never existed and a rebill_units no run will ever charge.
//
// The instrument is the inode, the same one bank-apply's tests use: a rename ALWAYS changes it, and a
// truncate-in-place never does. It is a stronger assertion than comparing bytes, because rewriting the
// same bytes non-atomically would pass a byte comparison and fail this.
func TestTheAutoBankIsWrittenAtomically(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := setupProjectOpts(t, srv.URL, projectOpts{regenerate: 1, source: suzukiSource})
r := newRunner(t, bookPath)
defer r.Close()
ctx := context.Background()
first := []miner.Term{{Src: "図書館", Type: "term", SinceCh: 1, Freq: 3}}
if err := r.writeAutoBank(ctx, first, nil, nil); err != nil {
t.Fatalf("write auto-bank: %v", err)
}
before := inode(t, r.autoBankPath())
second := []miner.Term{
{Src: "図書館", Type: "term", SinceCh: 1, Freq: 3},
{Src: "鈴木", Type: "name", SinceCh: 1, Freq: 5},
}
if err := r.writeAutoBank(ctx, second, nil, nil); err != nil {
t.Fatalf("rewrite auto-bank: %v", err)
}
if after := inode(t, r.autoBankPath()); after == before {
t.Fatalf("the auto-bank was rewritten IN PLACE (inode %d unchanged): a concurrent status/export read can catch it truncated or half-written, and a valid prefix of the term list folds a bank that never existed", before)
}
// And the document that landed is the whole new one, not a prefix of it.
body, err := os.ReadFile(r.autoBankPath())
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"図書館", "鈴木"} {
if !strings.Contains(string(body), want) {
t.Fatalf("the replacement document is incomplete: %q missing from\n%s", want, body)
}
}
}

View file

@ -88,6 +88,12 @@ type BookResult struct {
Chunks []ChunkOutcome Chunks []ChunkOutcome
TotalUSD float64 // THIS run's spend across all chunks TotalUSD float64 // THIS run's spend across all chunks
Flagged int // number of flagged chunks (acceptance allows N) Flagged int // number of flagged chunks (acceptance allows N)
// Volume is non-nil when the run stopped because it had done the VOLUME of work it was granted
// (--max-units, volume.go) rather than because the book ended. It is a RESULT and never an error: a
// volume stop is a completion (exit 0, the frozen exit-code dictionary untouched), so the fact that
// this particular completion is not the end of the book has to travel HERE, in the report, or the
// operator gets a stop that does not say which ceiling produced it.
Volume *VolumeStop
} }
// ExitCode is the run's shell disposition: 0 clean, 2 completed-with-flags // ExitCode is the run's shell disposition: 0 clean, 2 completed-with-flags
@ -184,20 +190,6 @@ func (r *Runner) translateBook(ctx context.Context) (*BookResult, error) {
// manifest, and building it anywhere else would let a call path exist without it. // manifest, and building it anywhere else would let a call path exist without it.
r.bankSrc = newBankSourceIndex(chunks) r.bankSrc = newBankSourceIndex(chunks)
// Consent to a RE-PAYMENT (D20.2-Q2, rebill.go) — BEFORE the waves, hence before the first Reserve:
// if this run would pay again for units already billed under a superseded snapshot, and the amount
// is over the book's threshold, it stops here with the sum instead of quietly re-buying the book.
// It needs the materialized memory (the per-wave snapshots fold it), so it sits after seedGlossary,
// and the chunk manifest (the projected book cost is per output unit), so it sits after the split.
if err := r.checkRebillConsent(ctx, chunks); err != nil {
return nil, err
}
// The run scale — one line to stderr (the smoke-run pain: N/M and the progress
// denominator never appeared in the logs at all, only in stdout/status).
r.Log.InfoContext(ctx, "book run started", "book", r.Book.BookID,
"chapters", len(doc.Chapters), "chunks", len(chunks), "stages", len(r.Pipeline.Stages))
// the precompute pass: pre-compute the sticky-chain memory selection for EVERY chunk (WS1 §1б, precomputeSticky). The // the precompute pass: pre-compute the sticky-chain memory selection for EVERY chunk (WS1 §1б, precomputeSticky). The
// sticky window (A5 scene-inertia — the recent chunks' exact matches in the current chapter, reset at // sticky window (A5 scene-inertia — the recent chunks' exact matches in the current chapter, reset at
// a chapter boundary) is a CROSS-CHUNK sequential dependency, so it is computed once here in the precompute pass and // a chapter boundary) is a CROSS-CHUNK sequential dependency, so it is computed once here in the precompute pass and
@ -207,11 +199,51 @@ func (r *Runner) translateBook(ctx context.Context) (*BookResult, error) {
// a bank-mining enrichment (the review-confirmed «re-paid ONCE» fix); the editor uses the enriched bank. // a bank-mining enrichment (the review-confirmed «re-paid ONCE» fix); the editor uses the enriched bank.
stickySel := precomputeSticky(chunks, r.baseMemory, r.Pipeline.Context.GlossaryTokenBudget) stickySel := precomputeSticky(chunks, r.baseMemory, r.Pipeline.Context.GlossaryTokenBudget)
// The VOLUME ceiling's scope (volume.go): decided before either wave, so the ceiling is checked before
// an item begins rather than after it has been paid for. It needs stickySel, because the "would this
// unit resolve for free" predicate renders the same wire bytes the draft wave will.
//
// ⚠ IT IS DELIBERATELY COMPUTED BEFORE THE CONSENT GATE BELOW, not after. The gate asks the operator to
// consent to a CONCRETE spend (Р6), and a run bounded to ten units is not going to re-pay the whole
// book — quoting the book's figure at it would refuse work that was never going to be done, and ask for
// consent to money nobody will be charged. rebill.go says out loud why that is corrosive: "a number the
// operator is asked to approve and then not charged is exactly what makes such a number stop being
// read." Both steps are $0 and pure, so the reordering costs nothing.
scope, err := r.planVolume(ctx, chunks, stickySel)
if err != nil {
return nil, err
}
// Consent to a RE-PAYMENT (D20.2-Q2, rebill.go) — BEFORE the waves, hence before the first Reserve:
// if this run would pay again for units already billed under a superseded snapshot, and the amount
// is over the book's threshold, it stops here with the sum instead of quietly re-buying the book.
// It needs the materialized memory (the per-wave snapshots fold it), so it sits after seedGlossary,
// and the chunk manifest (the projected book cost is per output unit), so it sits after the split.
if err := r.checkRebillConsent(ctx, chunks, scope); err != nil {
return nil, err
}
// The run scale — one line to stderr (the smoke-run pain: N/M and the progress
// denominator never appeared in the logs at all, only in stdout/status).
r.Log.InfoContext(ctx, "book run started", "book", r.Book.BookID,
"chapters", len(doc.Chapters), "chunks", len(chunks), "stages", len(r.Pipeline.Stages))
// The wave executor (R1, waverun.go): the draft wave (draft ∥) → the bank-mining stop → the edit wave (edit ∥). It computes + // The wave executor (R1, waverun.go): the draft wave (draft ∥) → the bank-mining stop → the edit wave (edit ∥). It computes +
// upserts the per-wave snapshots (draft-wave snapshot base-bank / edit-wave snapshot enriched) itself and pins each // upserts the per-wave snapshots (draft-wave snapshot base-bank / edit-wave snapshot enriched) itself and pins each
// wave's jobs to its own — a the bank-mining stop enrichment moves only edit-wave snapshot, keeping draft-wave checkpoints valid // wave's jobs to its own — a the bank-mining stop enrichment moves only edit-wave snapshot, keeping draft-wave checkpoints valid
// («re-paid ONCE»). Returns a *WaveSignatureStop when the bank-mining stop stops for owner sign. // («re-paid ONCE»). Returns a *WaveSignatureStop when the bank-mining stop stops for owner sign.
res, err := r.translateBookWaves(ctx, chunks, stickySel) res, err := r.translateBookWaves(ctx, chunks, stickySel, scope)
if err != nil && scope.bound() {
// A run can be stopped by the OTHER ceiling — money — and then the volume report never reaches the
// operator, because a failed run returns no result to carry it. Without this line they are told the
// run halted on money and nothing about what it had been granted or how far that got, which is the
// first thing anyone asks. The durable progress is in the store either way; this makes the stderr
// account of the run complete rather than half.
r.Log.WarnContext(ctx, "the run ended before its VOLUME grant was used up — the stop below is NOT the volume ceiling",
"book", r.Book.BookID, "max_units", scope.stop.MaxUnits,
"granted_new", scope.stop.Delivered, "granted_re_made", scope.stop.Reworked,
"left_never_delivered", scope.stop.LeftFresh, "err", err)
}
if err == nil { if err == nil {
// End of the run (row 125). The stop paths refresh the read-out themselves, so this is the boundary // End of the run (row 125). The stop paths refresh the read-out themselves, so this is the boundary
// they do not cover: a run that finished, whose last bank change was the auto-continue re-seed or a // they do not cover: a run that finished, whose last bank change was the auto-continue re-seed or a

View file

@ -309,6 +309,21 @@ func (r *Runner) exportConfigDrift(statuses []store.ChunkStatus, draftStageNames
editSnaps[cs.SnapshotID] = true editSnaps[cs.SnapshotID] = true
} }
} }
// ⚠ THE STORED GLOSSARY ON PURPOSE, and NOT the fold `status` uses. The two surfaces answer two
// different questions and this pack deliberately moved only one of them.
//
// status asks "what would the NEXT run cost", so it must fold the decision files: money is the point,
// and a bank-apply that has not been run yet is exactly the spend it exists to reveal (row 231).
// export asks whether THIS document is consistent with the run that produced it — its consumer is the
// polygon's extraction, which uses ConfigDrift to decide whether an export is trustworthy to MEASURE.
// An export whose text is byte-identical to what its run shipped is perfectly measurable even with a
// decision file sitting un-run beside the database, and folding that file here would mark every such
// export drifted and quietly disqualify sound measurements in another zone.
//
// This was briefly changed to the fold for symmetry and changed back: symmetry between two surfaces
// that answer different questions is not a property worth having, and redefining a field another zone
// consumes was not this pack's order. The divergence is deliberate, and it is named here so the next
// reader does not "fix" it either.
if err := r.projectStoredMemory(); err != nil { if err := r.projectStoredMemory(); err != nil {
r.Log.Warn("export: config-drift check failed; drift state unknown (reported as none)", "err", err) r.Log.Warn("export: config-drift check failed; drift state unknown (reported as none)", "err", err)
return return

View file

@ -712,7 +712,14 @@ func (r *Runner) writeAutoBank(ctx context.Context, mined []miner.Term, proposal
if err != nil { if err != nil {
return fmt.Errorf("pipeline: marshal auto-bank: %w", err) return fmt.Errorf("pipeline: marshal auto-bank: %w", err)
} }
if err := os.WriteFile(r.autoBankPath(), []byte(body), 0o644); err != nil { // ATOMIC, and the reason changed under this file's feet. Until the read-only surfaces began folding the
// bank themselves (bankmaterialize.go), the auto-bank was read only by the run that had just written
// it, so a plain truncating write was safe. Now `status` and `export` read it CONCURRENTLY with a
// running translate — the exact condition artifact.go names — and a truncating write gives a reader
// either a parse error (the buyer sees stale figures) or, worse, a valid PREFIX of the term list: a
// fold of a bank that never existed, and a rebill_units no run will ever charge. Write-then-rename
// makes every read see the previous document or the next one, whole.
if err := writeFileAtomic(r.autoBankPath(), []byte(body)); err != nil {
return fmt.Errorf("pipeline: write auto-bank %s: %w", r.autoBankPath(), err) return fmt.Errorf("pipeline: write auto-bank %s: %w", r.autoBankPath(), err)
} }
if len(before) == 0 { if len(before) == 0 {

View file

@ -63,6 +63,18 @@ type RebillProjection struct {
// the operator-facing text can name what the number is made of rather than claim a re-pricing it did // the operator-facing text can name what the number is made of rather than claim a re-pricing it did
// not achieve. // not achieve.
HistoricalRows int HistoricalRows int
// OutputUnits is the same re-payment counted in OUTPUT UNITS — the granularity the manifest publishes
// as units_total, that `--max-units` bounds, and that the platform sells chapters in.
//
// ⚠ IT EXISTS BECAUSE `Rows` IS A DIFFERENT UNIT WEARING THE SAME WORD. Rows counts chunk×stage, the
// BILLING unit; every other "units" number the engine puts on the wire (total_units, progress totals,
// chapters[].units_total) is the OUTPUT unit. Measured on a three-chapter fixture, one document carried
// `rebill_units: 15` two lines under `total_units: 6` with nothing marking the difference — and the
// ratio is not a constant a consumer could divide out: it is len(Members)·nDraftStages + nEditStages,
// which varies unit by unit WITHIN one book and whose factors never cross the seam at all. A platform
// sizing a re-pass from the estimate and passing that number to --max-units would buy several times the
// book it meant to, which is this pack's own defect reproduced one layer down.
OutputUnits int
// Repinned counts the units the run will serve for $0 despite a moved snapshot (pack-20 point 5): the // Repinned counts the units the run will serve for $0 despite a moved snapshot (pack-20 point 5): the
// move was bank-only and their rendered bytes are unchanged. It is not part of the amount — it is the // move was bank-only and their rendered bytes are unchanged. It is not part of the amount — it is the
// number that makes the amount believable, because before pack-20 every one of these was counted as a // number that makes the amount believable, because before pack-20 every one of these was counted as a
@ -116,6 +128,17 @@ func (r *Runner) projectRebill(statuses []store.ChunkStatus, manifest []chunk.Ch
for _, ch := range manifest { for _, ch := range manifest {
live[chunkKey{ch.Chapter, ch.ChunkIdx}] = true live[chunkKey{ch.Chapter, ch.ChunkIdx}] = true
} }
// Which OUTPUT unit each position belongs to, so the same re-payment can be reported in the unit the
// seam sells in as well as the one it bills in. An edit row lives at its unit's leader chunk and a
// leader is one of the unit's members, so this one map covers both waves.
leaderOf := make(map[chunkKey]chunkKey, len(manifest))
for _, u := range r.outputUnits(manifest) {
leader := chunkKey{u.Chapter, u.FirstChunkIdx}
for _, m := range u.Members {
leaderOf[chunkKey{m.Chapter, m.ChunkIdx}] = leader
}
}
touched := map[chunkKey]bool{}
draftNames := stageNameSet(r.waveStagesIndexed(waveDraft)) draftNames := stageNameSet(r.waveStagesIndexed(waveDraft))
editNames := stageNameSet(r.waveStagesIndexed(waveEdit)) editNames := stageNameSet(r.waveStagesIndexed(waveEdit))
rendered := map[wave]string{} rendered := map[wave]string{}
@ -164,7 +187,7 @@ func (r *Runner) projectRebill(statuses []store.ChunkStatus, manifest []chunk.Ch
if ferr != nil { if ferr != nil {
return p, fmt.Errorf("pipeline: re-chunk the source for the re-bill content check: %w", ferr) return p, fmt.Errorf("pipeline: re-chunk the source for the re-bill content check: %w", ferr)
} }
contentHashes = r.renderedContentHashes(full, precomputeSticky(full, r.baseMemory, r.Pipeline.Context.GlossaryTokenBudget)) contentHashes = r.cachedRenderedContentHashes(full, precomputeSticky(full, r.baseMemory, r.Pipeline.Context.GlossaryTokenBudget))
} }
if h, ok := contentHashes[chunkKey{cs.Chapter, cs.ChunkIdx}][cs.Stage]; ok && h == cs.ContentHash { if h, ok := contentHashes[chunkKey{cs.Chapter, cs.ChunkIdx}][cs.Stage]; ok && h == cs.ContentHash {
p.Repinned++ p.Repinned++
@ -172,12 +195,14 @@ func (r *Runner) projectRebill(statuses []store.ChunkStatus, manifest []chunk.Ch
} }
} }
p.Rows++ p.Rows++
touched[leaderOf[chunkKey{cs.Chapter, cs.ChunkIdx}]] = true
usd, fromHistory := rp.usd(cs) usd, fromHistory := rp.usd(cs)
p.USD += usd p.USD += usd
if fromHistory { if fromHistory {
p.HistoricalRows++ p.HistoricalRows++
} }
} }
p.OutputUnits = len(touched)
return p, nil return p, nil
} }
@ -259,7 +284,28 @@ func projectionBasis(p RebillProjection) string {
// the debt this closes. It also covers the case --resnapshot does not: a run interrupted midway through // the debt this closes. It also covers the case --resnapshot does not: a run interrupted midway through
// a re-pin leaves jobs on the new snapshot while their chunk_status rows still carry the old one, and // a re-pin leaves jobs on the new snapshot while their chunk_status rows still carry the old one, and
// the next plain `translate` then re-bills them with no gate at all. // the next plain `translate` then re-bills them with no gate at all.
func (r *Runner) checkRebillConsent(ctx context.Context, chunks []chunk.Chunk) error { // `scope` is the run's VOLUME ceiling (volume.go), nil when none is in force. It produces TWO figures,
// and which of them each half of the gate is judged against is the whole correctness of this function.
//
// ⚠ THE THRESHOLD IS JUDGED AGAINST THE BOOK, NEVER AGAINST THE RUN — and the first version of this
// pack got that wrong. Scoping the amount to the admitted units and leaving the threshold book-wide
// sounds symmetrical and is not: with a volume ceiling the caller CHOOSES how small each run is, so
// `--resnapshot --max-units N` in a loop re-pays the entire book without a single run ever crossing the
// threshold and without consent being asked once. That is precisely the silent re-purchase Р6 exists to
// prevent, and "the change is inert without the flag" is true and beside the point: with the flag it
// opens the door the gate is the door for. A threshold a caller can defeat by splitting is not a
// threshold. So the question "must we ask at all" is answered by the book's whole drift, which does not
// shrink when a purchase does.
//
// ⚠ THE NAMED CAP IS JUDGED AGAINST THE RUN, because it is a different kind of statement. A threshold is
// the book's policy on when a human must be consulted; `--accept-rebill=X` is the caller's instruction
// "spend no more than X". Measuring an instruction about SPEND against work this run will not do would
// refuse a caller whose cap fully covers what they are about to be charged — the platform funds that cap
// from the run's own hold, so it is sized to the purchase, not to the book.
//
// With no ceiling in force the two projections are the same object and exactly one is computed, so every
// existing run is judged by byte-identical figures.
func (r *Runner) checkRebillConsent(ctx context.Context, chunks []chunk.Chunk, scope *volumeScope) error {
statuses, err := r.Store.ChunkStatusesForBook(r.Book.BookID) statuses, err := r.Store.ChunkStatusesForBook(r.Book.BookID)
if err != nil { if err != nil {
return fmt.Errorf("pipeline: read chunk_status for the re-bill projection: %w", err) return fmt.Errorf("pipeline: read chunk_status for the re-bill projection: %w", err)
@ -270,14 +316,32 @@ func (r *Runner) checkRebillConsent(ctx context.Context, chunks []chunk.Chunk) e
} }
// A write path already holds the real split, so the lazy provider just hands it back — no second // A write path already holds the real split, so the lazy provider just hands it back — no second
// ingest, and no branch where the consent gate could see text-free chunks. // ingest, and no branch where the consent gate could see text-free chunks.
proj, err := r.projectRebill(statuses, chunks, func() ([]chunk.Chunk, error) { return chunks, nil }, rp) withText := func() ([]chunk.Chunk, error) { return chunks, nil }
book, err := r.projectRebill(statuses, chunks, withText, rp)
if err != nil { if err != nil {
return err return err
} }
if proj.Rows == 0 { if book.Rows == 0 {
return nil // nothing already-paid is superseded — this run bills only new work return nil // nothing already-paid is superseded — this run bills only new work
} }
// What THIS run will actually re-pay. Only computed when a ceiling narrows it; otherwise it IS the
// book's figure, which keeps the un-bounded path at one projection exactly as before.
proj := book
if scope != nil {
if proj, err = r.projectRebill(scope.admittedStatuses(statuses), chunks, withText, rp); err != nil {
return err
}
}
if proj.Rows == 0 {
// The book carries drift, but none of it is inside what this run was granted, so this run re-pays
// nothing and there is no concrete spend to consent to. Not a hole in the threshold above: the
// moment a run is granted a superseded unit, proj stops being empty and the book-wide threshold
// below decides — so splitting cannot walk past the gate, it can only postpone meeting it.
return nil
}
// The denominator is the WHOLE book's stored rows, unfiltered by the volume scope: the threshold is
// "5% of what this book costs", a property of the book.
byChunk := map[chunkKey][]store.ChunkStatus{} byChunk := map[chunkKey][]store.ChunkStatus{}
for _, cs := range statuses { for _, cs := range statuses {
byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}] = append(byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}], cs) byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}] = append(byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}], cs)
@ -296,16 +360,22 @@ func (r *Runner) checkRebillConsent(ctx context.Context, chunks []chunk.Chunk) e
if r.AcceptRebill.Given { if r.AcceptRebill.Given {
r.Log.WarnContext(ctx, "accepting a projected re-payment of already-billed work (--accept-rebill)", r.Log.WarnContext(ctx, "accepting a projected re-payment of already-billed work (--accept-rebill)",
"rebill_units", proj.Rows, "rebill_usd", fmt.Sprintf("%.6f", proj.USD), "rebill_units", proj.Rows, "rebill_usd", fmt.Sprintf("%.6f", proj.USD),
"book_rebill_units", book.Rows, "book_rebill_usd", fmt.Sprintf("%.6f", book.USD),
"units_not_repriced", proj.HistoricalRows, "units_not_repriced", proj.HistoricalRows,
"repinned_free", proj.Repinned, "threshold_usd", fmt.Sprintf("%.6f", threshold)) "repinned_free", proj.Repinned, "threshold_usd", fmt.Sprintf("%.6f", threshold))
return nil return nil
} }
if proj.USD <= threshold { // ⚠ THE BOOK'S figure is what the threshold judges, not this run's. See the doc comment: a caller
// that chooses how small each run is could otherwise re-pay the whole book a slice at a time and
// never once be asked. `book` equals `proj` whenever no volume ceiling is in force, so this is the
// same comparison it has always been for every un-bounded run.
if book.USD <= threshold {
// Under the threshold the run continues without friction (the ratified behaviour: a term append // Under the threshold the run continues without friction (the ratified behaviour: a term append
// touching three chunks costs cents). Continuing is not the same as being silent — the amount is // touching three chunks costs cents). Continuing is not the same as being silent — the amount is
// money and it goes to the log. // money and it goes to the log.
r.Log.InfoContext(ctx, "re-paying already-billed work under the consent threshold; continuing without asking", r.Log.InfoContext(ctx, "re-paying already-billed work under the consent threshold; continuing without asking",
"rebill_units", proj.Rows, "rebill_usd", fmt.Sprintf("%.6f", proj.USD), "rebill_units", proj.Rows, "rebill_usd", fmt.Sprintf("%.6f", proj.USD),
"book_rebill_units", book.Rows, "book_rebill_usd", fmt.Sprintf("%.6f", book.USD),
"units_not_repriced", proj.HistoricalRows, "units_not_repriced", proj.HistoricalRows,
"repinned_free", proj.Repinned, "threshold_usd", fmt.Sprintf("%.6f", threshold)) "repinned_free", proj.Repinned, "threshold_usd", fmt.Sprintf("%.6f", threshold))
return nil return nil
@ -315,10 +385,30 @@ func (r *Runner) checkRebillConsent(ctx context.Context, chunks []chunk.Chunk) e
if !r.Resnapshot { if !r.Resnapshot {
hint = " The run also needs --resnapshot: without it the superseded jobs stop it anyway." hint = " The run also needs --resnapshot: without it the superseded jobs stop it anyway."
} }
// ⚠ EVERY FIGURE IN A SENTENCE MUST COME FROM THE SAME PROJECTION AS THE SENTENCE. This parenthetical
// hangs off the BOOK-wide clause, so it takes the BOOK's re-pin count. The first version spliced the
// volume-scoped one in, which under an active ceiling printed a smaller number inside a sentence whose
// every other figure was book-wide — an operator adding them up would have got a book that did not
// exist. Two independent review lenses caught the same splice, which is what mixed-provenance numbers
// in one sentence reliably produce.
repin := "" repin := ""
if proj.Repinned > 0 { if book.Repinned > 0 {
repin = fmt.Sprintf(" (%d further unit(s) are re-pinned for $0 — the bank moved but their injected bytes did not)", proj.Repinned) repin = fmt.Sprintf(" (%d further unit(s) of the book are re-pinned for $0 — the bank moved but their injected bytes did not)", book.Repinned)
} }
return fmt.Errorf("pipeline: this run would RE-PAY for work already billed: %d chunk×stage unit(s) are resolved under a superseded snapshot and would be paid for again, ~$%.6f (%s)%s. That is over this book's consent threshold $%.6f (%s), and Р6 requires consent to a CONCRETE spend, not a blanket one (D20.2-Q2). NOTHING was reserved and no row was touched. Re-run with --accept-rebill to accept the whole projected amount, or --accept-rebill=<usd> to accept it only up to a ceiling (a ceiling below the projection refuses).%s", // The two projections are printed SEPARATELY when a volume ceiling makes them differ, because
proj.Rows, proj.USD, projectionBasis(proj), repin, threshold, source, hint) // collapsing them is how this gate goes wrong in either direction: quoting only the book's total at a
// small purchase asks for consent to money nobody will be charged, and quoting only the run's slice
// hides that the book is being re-bought a slice at a time. The scoped clause carries the scoped
// re-pin count for the same provenance reason.
scoped := ""
if book.Rows != proj.Rows || book.USD != proj.USD {
scopedRepin := ""
if proj.Repinned != book.Repinned {
scopedRepin = fmt.Sprintf(" (and re-pin %d of them for $0)", proj.Repinned)
}
scoped = fmt.Sprintf(" THIS run, bounded by --max-units, would re-pay %d of them, ~$%.6f%s — but the threshold is judged against the book, because a ceiling the caller sizes could otherwise re-pay the whole book a slice at a time without ever being asked.",
proj.Rows, proj.USD, scopedRepin)
}
return fmt.Errorf("pipeline: this run would RE-PAY for work already billed: %d chunk×stage unit(s) of this book are resolved under a superseded snapshot and would be paid for again, ~$%.6f (%s)%s.%s That is over this book's consent threshold $%.6f (%s), and Р6 requires consent to a CONCRETE spend, not a blanket one (D20.2-Q2). NOTHING was reserved and no row was touched. Re-run with --accept-rebill to accept the whole projected amount, or --accept-rebill=<usd> to accept it only up to a ceiling (the ceiling is measured against what THIS run re-pays, and one below that refuses).%s",
book.Rows, book.USD, projectionBasis(book), repin, scoped, threshold, source, hint)
} }

View file

@ -0,0 +1,526 @@
package pipeline
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"testing"
"textmachine/backend/internal/chunk/chunktest"
"textmachine/backend/internal/membank"
"textmachine/backend/internal/obs"
)
// decisionsDocFor writes a decision document for an ARBITRARY book id — the package's existing helper
// hardcodes the one book its own fixtures use.
func decisionsDocFor(t *testing.T, dir, bookID string, ds ...membank.Decision) string {
t.Helper()
body, err := json.Marshal(membank.DecisionsDoc{Version: membank.DecisionsVersion, BookID: bookID, Decisions: ds})
if err != nil {
t.Fatal(err)
}
p := filepath.Join(dir, "decisions-for-"+bookID+".json")
writeFile(t, p, string(body))
return p
}
// assertStoreUntouched is the "nothing was written" instrument, and it is deliberately TWO assertions
// because one of them is not enough.
//
// ⚠ HASHING THE .db FILE ALONE PROVES LESS THAN IT LOOKS. SQLite in WAL mode puts a committed change in
// the `-wal` sidecar and leaves the main file untouched until a checkpoint, so a data write can land with
// the main file's bytes unmoved — the first version of this helper would have gone on reporting "nothing
// was written" straight through it. So: the data file must be byte-identical AND the write-ahead log must
// carry no frames.
//
// ⚠ AND `-shm` IS DELIBERATELY NOT CHECKED. Measured, not assumed: opening this database read-only
// CREATES an empty `-wal` and a live `-shm`, because the shared-memory index is how any WAL reader finds
// its snapshot. Requiring those two files not to appear would fail on a pure read and would be asserting
// something SQLite's design forbids, not something about this engine. An empty `-wal` is the honest line:
// the reader may set the machinery up, it may not commit a frame through it.
//
// It is corroboration, not the guarantee. What actually makes the read path unable to write is
// store.OpenReadOnly's PRAGMA query_only(1) — a write through it fails loudly rather than being dropped.
// This catches a path that reached a WRITE handle by some route the read was not supposed to take.
func assertStoreUntouched(t *testing.T, dbPath, dataBefore string) {
t.Helper()
if got := fileDigest(t, dbPath); got != dataBefore {
t.Fatalf("the project database changed during a read-only estimate (%s → %s): nothing beyond the first touch may be written", dataBefore[:12], got[:12])
}
wal, err := os.Stat(dbPath + "-wal")
if err != nil {
if !os.IsNotExist(err) {
t.Fatalf("stat %s-wal: %v", dbPath, err)
}
return // no write-ahead log at all — nothing could have been committed through one
}
if wal.Size() != 0 {
t.Fatalf("the write-ahead log holds %d byte(s) after a read-only estimate: a committed change can sit in -wal with the main database file still byte-identical, which is exactly what makes hashing the .db alone insufficient", wal.Size())
}
}
// fileDigest is a plain content hash of one file.
func fileDigest(t *testing.T, path string) string {
t.Helper()
b, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:])
}
// TestABankEditIsVisibleToTheEstimateBeforeAnythingIsBought is the deliverable of backlog row 231, and it
// is written against the state the FIRST attempt at this repro got wrong (errata 28.08-и/-к).
//
// THE DEGENERATE REPRO THAT MUST BE AVOIDED. "translate has never run" proves nothing: such a book has no
// chunk_status rows at all, and projectRebill skips rows without a snapshot (rebill.go), so it answers
// EMPTY both before and after any fix. The state that discriminates is: a run HAPPENED (rows and
// snapshots exist), THEN the bank was edited through `bank-apply`, which writes decision FILES and not one
// store row. On that state the engine used to answer "nothing moved" while the very next `translate` would
// re-seed from those files and bill for the difference.
//
// So the assertion is a DELTA, and both halves are executed here: the stored-glossary projection (what the
// engine answered before) must be zero on exactly the state where the folded projection is non-zero.
// Around it sit the two boundaries the estimate is sold with — not one provider call, and not one byte
// written to the store beyond the first touch that predates it.
func TestABankEditIsVisibleToTheEstimateBeforeAnythingIsBought(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: suzukiSeed,
})
dir := filepath.Dir(bookPath)
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
// --- the run that must have HAPPENED for the repro to discriminate ---
r1 := newRunner(t, bookPath)
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatalf("the run this repro rests on did not complete: %v", err)
}
statuses, err := r1.Store.ChunkStatusesForBook("test-book")
if err != nil {
t.Fatal(err)
}
if len(statuses) == 0 {
t.Fatalf("precondition: the repro needs stored chunk_status rows, the state the degenerate version lacked")
}
committedBefore, reservedBefore, err := r1.Store.SpentUSD("test-book")
if err != nil {
t.Fatal(err)
}
r1.Close()
// --- the bank edit, through the verb the product actually uses ---
// 静か occurs in the source, so the term enters the ENRICHED bank, moves the edit-wave snapshot AND
// changes the injected bytes of the edit unit — i.e. it is genuinely re-paid work, not a $0 re-pin.
doc := decisionsDocFor(t, dir, "test-book", membank.Decision{
Action: membank.ActionApprove, Src: "静か", Dst: "тихий"})
if _, err := ApplyBankDecisions(ctx, bookPath, doc, false); err != nil {
t.Fatalf("bank-apply: %v", err)
}
callsBefore := rec.count()
dbPath := filepath.Join(dir, "test-book.db")
dbBefore := fileDigest(t, dbPath)
// --- the two projections over the SAME state ---
r2, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
defer r2.Close()
rp, err := r2.newRepricer()
if err != nil {
t.Fatal(err)
}
chunks, withText, err := r2.readModelChunks()
if err != nil {
t.Fatal(err)
}
// What the engine answered BEFORE this pack: the glossary the last run stored.
if err := r2.projectStoredMemory(); err != nil {
t.Fatalf("stored-glossary projection: %v", err)
}
storedProj, err := r2.projectRebill(statuses, chunks, withText, rp)
if err != nil {
t.Fatalf("stored-glossary re-bill projection: %v", err)
}
// What it answers now: the bank the next `translate` will actually build.
basis, warn := r2.foldMemoryForRead()
if warn != nil {
t.Fatalf("the fold refused on a healthy book: %v", warn)
}
if basis != RebillBasisPending {
t.Fatalf("basis = %q, want %q — a healthy book must be projected from its decision files", basis, RebillBasisPending)
}
foldedProj, err := r2.projectRebill(statuses, chunks, withText, rp)
if err != nil {
t.Fatalf("folded re-bill projection: %v", err)
}
// THE DELTA. Both halves are load-bearing: a non-zero "after" alone would also pass on a book where
// the old answer was non-zero too, and would not prove the blind window was the thing that closed.
if storedProj.Rows != 0 {
t.Fatalf("the repro does not discriminate: the STORED-glossary projection already sees %d unit(s); the blind window this pack closes would be invisible here", storedProj.Rows)
}
if foldedProj.Rows == 0 {
t.Fatalf("the folded projection still reports nothing after a bank edit — the blind window of row 231 is open")
}
// --- the two boundaries the estimate is SOLD with ---
if got := rec.count(); got != callsBefore {
t.Fatalf("the estimate made %d provider call(s); it is sold as $0 and must make none", got-callsBefore)
}
committedAfter, reservedAfter, err := r2.Store.SpentUSD("test-book")
if err != nil {
t.Fatal(err)
}
if committedAfter != committedBefore || reservedAfter != reservedBefore {
t.Fatalf("the ledger moved during a free estimate: committed %v→%v, reserved %v→%v",
committedBefore, committedAfter, reservedBefore, reservedAfter)
}
assertStoreUntouched(t, dbPath, dbBefore)
}
// TestTheEstimateSurvivesABookWhoseFoldRefuses pins the property rebill.go leans on — "a book that needs
// consent stays fully inspectable" (D20.4). `translate` fails loudly on a bank whose fold would abort
// ReplaceBank; a READ must not, or the book that most needs looking at is the one that cannot be looked
// at. It must fall back AND say that it fell back, because a stale number passed off as the answer is
// worse than a labelled one.
func TestTheEstimateSurvivesABookWhoseFoldRefuses(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: suzukiSeed,
})
dir := filepath.Dir(bookPath)
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
r1 := newRunner(t, bookPath)
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatal(err)
}
r1.Close()
// A mined-delta row on the SAME UNIQUE key as the signed seed term: exactly what would abort
// ReplaceBank's flat INSERT, and what seedGlossary refuses on before it gets there.
writeFile(t, filepath.Join(dir, "test-book.mined-delta.yaml"), `
terms:
- src: 鈴木
dst: Судзуки-другой
type: name
status: approved
`)
r2, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
defer r2.Close()
basis, warn := r2.foldMemoryForRead()
if basis != RebillBasisStored {
t.Fatalf("basis = %q, want %q — a refused fold must fall back to the stored glossary, not fail the read", basis, RebillBasisStored)
}
if warn == nil {
t.Fatalf("a silent fallback is the defect: the reason the projection is a fact about the PAST must reach the log")
}
if r2.memory == nil || r2.baseMemory == nil {
t.Fatalf("the fallback must still leave a materialized bank behind (memory=%v base=%v)", r2.memory != nil, r2.baseMemory != nil)
}
// And the whole read still answers rather than erroring.
rep, err := r2.Status(ctx)
if err != nil {
t.Fatalf("status refused a book whose fold refuses — D20.4 says it must stay inspectable: %v", err)
}
if rep.RebillBasis != RebillBasisStored {
t.Fatalf("the report must carry the fallback basis, got %q", rep.RebillBasis)
}
}
// TestTheWireTellsFreeFromUnknown pins the omitempty mine D39.166 п.2 already paid for once: three states
// collapse to two zeroes unless the basis rides with them.
func TestTheWireTellsFreeFromUnknown(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: suzukiSeed,
})
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
// A book nothing has run: the figures are zero because there is nothing to re-pay.
r0, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
rep0, err := r0.Status(ctx)
if err != nil {
t.Fatal(err)
}
r0.Close()
if rep0.RebillBasis != RebillBasisNone {
t.Fatalf("a book with no stored row must report basis %q, got %q", RebillBasisNone, rep0.RebillBasis)
}
// A book that ran and drifted from nothing: the figures are zero because they were COMPUTED as zero.
r1 := newRunner(t, bookPath)
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatal(err)
}
r1.Close()
r2, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
defer r2.Close()
rep1, err := r2.Status(ctx)
if err != nil {
t.Fatal(err)
}
if rep1.RebillBasis != RebillBasisPending {
t.Fatalf("a healthy run must report basis %q, got %q", RebillBasisPending, rep1.RebillBasis)
}
if rep1.RebillUnits != 0 {
t.Fatalf("precondition: an undrifted book re-pays nothing, got %d", rep1.RebillUnits)
}
// The two states above BOTH carry zero. Before this pack they were the same bytes on the wire.
if rep0.RebillBasis == rep1.RebillBasis {
t.Fatalf("«nothing to re-pay» and «computed, and it is zero» must be distinguishable, both say %q", rep0.RebillBasis)
}
}
// TestAFailedBasisNeverCarriesAFigure pins the direction the basis field's first version did not
// consider. It was added so that "computed, and it is zero" could be told from "we could not compute it"
// — but a failed fold leaves r.memory nil, and projectRebill run against a nil bank renders both wave
// snapshots over an EMPTY bank, so EVERY stored row differs and the whole book comes back as a
// re-payment. The wire would then carry rebill_basis:"failed" beside the largest number the field can
// hold, and the CLI would print "the figures below are zero because they could not be computed" next to
// it. A basis that contradicts its own numbers is worse than no basis.
func TestAFailedBasisNeverCarriesAFigure(t *testing.T) {
big := RebillProjection{Rows: 4276, USD: 12.5, OutputUnits: 1400}
for _, c := range []struct {
name string
hasRows bool
memBasis string
proj RebillProjection
projErr error
basis string
units int
}{
{"nothing stored", false, RebillBasisPending, big, nil, RebillBasisNone, 0},
{"healthy fold", true, RebillBasisPending, big, nil, RebillBasisPending, 4276},
{"fold refused, stored answered", true, RebillBasisStored, big, nil, RebillBasisStored, 4276},
// The two that matter: an unknown answer must never arrive carrying a number. Before this was one
// decision, the basis said "failed" while a whole-book figure — computed over an EMPTY bank, so
// every row looked superseded — sat next to it.
{"no bank at all", true, RebillBasisFailed, big, nil, RebillBasisFailed, 0},
{"projection errored", true, RebillBasisPending, big, errors.New("boom"), RebillBasisFailed, 0},
} {
t.Run(c.name, func(t *testing.T) {
basis, units, usd, outUnits := rebillOutcome(c.hasRows, c.memBasis, c.proj, c.projErr)
if basis != c.basis || units != c.units {
t.Fatalf("got (%q, %d, %v), want (%q, %d)", basis, units, usd, c.basis, c.units)
}
// EVERY figure, not just the two obvious ones: an output-unit count surviving an UNKNOWN
// answer would be the same contradiction in a different field.
if basis == RebillBasisFailed && (units != 0 || usd != 0 || outUnits != 0) {
t.Fatalf("an UNKNOWN answer carried a figure: %d units, $%v, %d output units", units, usd, outUnits)
}
if basis == RebillBasisNone && outUnits != 0 {
t.Fatalf("nothing stored means nothing to re-pay, yet %d output units were reported", outUnits)
}
})
}
// And the projection is not even attempted when there is no bank to compare against — running it
// would produce exactly the whole-book figure the case above must never show.
if projectable(true, RebillBasisFailed) {
t.Fatalf("with no materialized bank the projection must be skipped, not computed and then discarded")
}
if !projectable(true, RebillBasisPending) || projectable(false, RebillBasisPending) {
t.Fatalf("projectable must gate on stored rows and on a usable bank, and on nothing else")
}
// And end to end: a book whose fold refuses AND whose store cannot answer either. The fold refusal is
// built with a mined-delta duplicating the signed seed term; the stored side is broken by closing the
// store out from under the read, which is the only way both halves fail at once.
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: suzukiSeed,
})
dir := filepath.Dir(bookPath)
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
r1 := newRunner(t, bookPath)
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatal(err)
}
r1.Close()
writeFile(t, filepath.Join(dir, "test-book.mined-delta.yaml"), `
terms:
- src: 鈴木
dst: Судзуки-другой
type: name
status: approved
`)
r2, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
defer r2.Close()
got, err := r2.Status(ctx)
if err != nil {
t.Fatal(err)
}
// Here only the FOLD fails, so the honest answer is the stored fallback with a real figure — the
// point being that "failed" is reserved for the state where nothing at all could be materialized.
if got.RebillBasis != RebillBasisStored {
t.Fatalf("a fold refusal alone falls back rather than failing: got %q", got.RebillBasis)
}
}
// TestExportAndStatusAnswerDifferentDriftQuestions pins a divergence that is DELIBERATE, so that nobody
// removes it as an inconsistency.
//
// After a bank-apply that has written only decision FILES, `status` reports drift and `export` does not,
// and both are right because they are asked different things. status is the money surface: its question
// is "what would the NEXT run cost", and an un-run decision file is precisely the spend row 231 exists to
// reveal. export's ConfigDrift is read by the polygon's extraction to decide whether a document is
// trustworthy to MEASURE — and an export whose bytes are exactly what its run shipped is measurable
// whether or not somebody has queued a bank edit beside it.
//
// The pack briefly made export fold too, for symmetry. Symmetry between two surfaces answering different
// questions is not worth having, and it would have disqualified sound measurements in another zone.
func TestExportAndStatusAnswerDifferentDriftQuestions(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
regenerate: 1, source: suzukiSource, glossarySeed: suzukiSeed,
})
dir := filepath.Dir(bookPath)
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
r1 := newRunner(t, bookPath)
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatal(err)
}
r1.Close()
doc := decisionsDocFor(t, dir, "test-book", membank.Decision{
Action: membank.ActionApprove, Src: "静か", Dst: "тихий"})
if _, err := ApplyBankDecisions(ctx, bookPath, doc, false); err != nil {
t.Fatal(err)
}
callsBefore := rec.count()
rExp, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
exp, err := rExp.Export(false)
if err != nil {
t.Fatal(err)
}
rExp.Close()
if exp.ConfigDrift {
t.Fatalf("export marked a document drifted whose text is exactly what its run shipped: the polygon reads this field to decide whether a measurement is sound, and an un-run decision file does not make one unsound")
}
rSt, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
defer rSt.Close()
rep, err := rSt.Status(ctx)
if err != nil {
t.Fatal(err)
}
if !rep.ConfigDrift {
t.Fatalf("status must see the pending bank edit — that is the whole of row 231; got config_drift=false")
}
if rep.RebillUnits == 0 {
t.Fatalf("status must price the pending edit, got rebill_units=0 basis=%q", rep.RebillBasis)
}
// Both are $0 surfaces.
if got := rec.count(); got != callsBefore {
t.Fatalf("a read surface made %d provider call(s)", got-callsBefore)
}
}
// TestTheEstimateIsAlsoGivenInTheUnitAPurchaseIsSizedIn pins the denomination fix.
//
// `rebill_units` counts chunk×stage — the BILLING granularity — while every other "units" number in the
// same document (total_units, the progress totals, chapters[].units_total) counts OUTPUT units, which is
// also what `--max-units` bounds and what the platform sells chapters in. Both wore the word "units" with
// nothing marking the difference, and the ratio is not something a consumer can divide out: it is
// len(Members)·nDraftStages + nEditStages, which varies unit by unit inside one book and whose factors
// never cross the seam. A platform sizing a re-pass from the estimate and handing that number to
// --max-units would buy several times the book it meant to — this pack's own defect, one layer down.
func TestTheEstimateIsAlsoGivenInTheUnitAPurchaseIsSizedIn(t *testing.T) {
rec := &reqRec{}
srv := newJSONProvider(rec, draftEdit)
defer srv.Close()
// A cut fine enough that units hold several chunks, so the two denominations genuinely differ.
const seg = "\nsegmentation:\n draft_budget_out: 24\n edit_ceiling_out: 8000\n fertility: { cjk: 1.1978, other: 0.3852 }\n"
var eps []chunktest.Chapter
var spine []string
for i := 1; i <= 2; i++ {
id := fmt.Sprintf("c%d", i)
eps = append(eps, chunktest.Chapter{ID: id, Href: id + ".xhtml",
Body: fmt.Sprintf("<p>静かな図書館の朝%d。鈴木は本を読んだ。外では雨が降っていた。彼は窓を見た。</p>", i)})
spine = append(spine, id)
}
bookPath := setupProjectOpts(t, srv.URL, projectOpts{
epub: eps, spine: spine, regenerate: 1, waveWorkers: 1, gatesYAML: seg,
})
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
r1 := newRunner(t, bookPath)
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatal(err)
}
r1.Close()
// A PIPELINE drift, not a bank edit: a mined bank edit moves only the edit wave, so it re-pays exactly
// one row per unit and the two denominations would coincide — the fixture would prove nothing. A config
// change moves BOTH waves, which is the ordinary shape in which a unit's several draft rows and its one
// edit row are all re-paid together, and where the collision this field exists for is real.
driftPipelineVersion(t, bookPath)
r2, err := NewReadOnlyRunner(bookPath, obs.NewLogger())
if err != nil {
t.Fatal(err)
}
defer r2.Close()
rep, err := r2.Status(ctx)
if err != nil {
t.Fatal(err)
}
if rep.RebillUnits == 0 {
t.Fatalf("precondition: the bank edit must produce a re-payment, got 0 (basis %q)", rep.RebillBasis)
}
if rep.RebillOutputUnits == 0 {
t.Fatalf("the estimate must also be given in OUTPUT units — the granularity --max-units counts and the platform sells in; got 0 beside rebill_units=%d", rep.RebillUnits)
}
if rep.RebillOutputUnits > rep.RebillUnits {
t.Fatalf("an output unit holds one or more chunk×stage rows, so it can never exceed them: output=%d rows=%d", rep.RebillOutputUnits, rep.RebillUnits)
}
if rep.RebillOutputUnits > rep.TotalUnits {
t.Fatalf("the book has %d output units; a re-pass cannot touch %d of them", rep.TotalUnits, rep.RebillOutputUnits)
}
// The fixture must actually make the two differ, or it proves nothing about the collision.
if rep.RebillOutputUnits == rep.RebillUnits {
t.Fatalf("fixture is degenerate: the two denominations coincide (%d), so the trap this field exists for is untested", rep.RebillUnits)
}
}

View file

@ -103,6 +103,28 @@ func jsonEqual(a, b json.RawMessage) bool {
return xe == nil && ye == nil && string(xb) == string(yb) return xe == nil && ye == nil && string(xb) == string(yb)
} }
// cachedRenderedContentHashes is renderedContentHashes memoized for the life of ONE materialized bank.
//
// The reproduction walks every stage of every position and re-renders it — the expensive half of every
// $0 projection in this engine. Since the volume ceiling landed, a single bounded run could pay for it
// THREE times: once in planVolume, and once in each of the consent gate's two projections (the book's and
// the run's). Nothing changed between them, so two of the three were pure waste, and the cost scales with
// the book — exactly where it is least affordable.
//
// ⚠ THE CACHE'S LIFETIME IS THE BANK'S, and that is what makes it safe rather than merely fast. Every
// hash here is rendered against the materialized memory, so the memo is only valid while that memory is;
// materializeBanks clears it, which is the ONE place the bank can change (a mid-run re-seed at the
// bank-mining stop goes through it like everything else). A memo that outlived its bank would hand a
// caller hashes for a bank the run no longer has — the precise class of silent wrongness the free
// estimate exists to avoid.
func (r *Runner) cachedRenderedContentHashes(chunks []chunk.Chunk, stickySel []membank.Selection) map[chunkKey]map[string]string {
if r.contentHashes != nil {
return r.contentHashes
}
r.contentHashes = r.renderedContentHashes(chunks, stickySel)
return r.contentHashes
}
// renderedContentHashes reproduces, for every live position of every stage, the content hash the run // renderedContentHashes reproduces, for every live position of every stage, the content hash the run
// WOULD compute — the same msgsContentHash the resume fast-path compares against. It is the $0 half of // WOULD compute — the same msgsContentHash the resume fast-path compares against. It is the $0 half of
// the honest estimate: with it, "the bank moved" can be answered per unit instead of per wave. // the honest estimate: with it, "the bank moved" can be answered per unit instead of per wave.

View file

@ -76,6 +76,19 @@ type Runner struct {
// setting it moves no request_hash and re-bills nothing. The DAY ceiling is deliberately not // setting it moves no request_hash and re-bills nothing. The DAY ceiling is deliberately not
// overridable: it is an account-wide guard, not a property of one run. // overridable: it is an account-wide guard, not a property of one run.
CeilingUSD float64 CeilingUSD float64
// MaxUnits is the VOLUME ceiling for THIS RUN (tmctl --max-units, D39.165 §1б), 0 = unset. It bounds
// how many OUTPUT UNITS this run may pay for — the granularity the manifest publishes and the platform
// sells chapters in — so a purchase of N chapters can stop at N instead of at whatever N chapters'
// worth of dollars happens to buy.
//
// ⚠ IT IS ORTHOGONAL TO CeilingUSD AND MUST STAY SO. CeilingUSD is a BOOK-CUMULATIVE money bound the
// ledger judges every reservation against; MaxUnits is a bound on THIS run's own work and never
// touches the ledger, the reservations or the hold arithmetic the platform builds on that
// cumulativeness. They can both be in force and either can be the one that stops the run — and the two
// stops are different answers to the operator: money means "the run was cut short, add money", volume
// means "the run did what was bought". Like the ceilings it is an operator axis, not a wire one: it is
// in neither BriefHash nor the snapshot payload, so setting it moves no request_hash and re-bills nobody.
MaxUnits int
clients map[string]llm.LLMClient clients map[string]llm.LLMClient
templates map[string]*PromptTemplate templates map[string]*PromptTemplate
@ -131,6 +144,14 @@ type Runner struct {
// (identical content) — no double materialization, the injection is unchanged. The editor keeps the // (identical content) — no double materialization, the injection is unchanged. The editor keeps the
// enriched `memory`. nil ⇔ memory is nil (materialized together in seedGlossary). // enriched `memory`. nil ⇔ memory is nil (materialized together in seedGlossary).
baseMemory *membank.Bank baseMemory *membank.Bank
// contentHashes memoizes the per-position wire-hash reproduction for the life of the CURRENT
// materialized bank (cachedRenderedContentHashes). Cleared by materializeBanks, which is the one place
// the bank can change.
contentHashes map[chunkKey]map[string]string
// bankRows are the glossary rows the CURRENT materialization was built from, in the store's order
// (materializeBanks). They exist so a read surface can describe the bank it actually folded rather
// than re-reading a different one out of the store.
bankRows []store.GlossaryEntry
// checkers is the compiled WS5/pack-13 observability checker spec (pair-14 data-out): the pair's // checkers is the compiled WS5/pack-13 observability checker spec (pair-14 data-out): the pair's
// DETECTION patterns + tables (from pack) + the target-general lists (from embedded target data), // DETECTION patterns + tables (from pack) + the target-general lists (from embedded target data),

View file

@ -6,7 +6,6 @@ import (
"sort" "sort"
"strings" "strings"
"textmachine/backend/internal/chunk" "textmachine/backend/internal/chunk"
"textmachine/backend/internal/lang"
"textmachine/backend/internal/membank" "textmachine/backend/internal/membank"
"textmachine/backend/internal/store" "textmachine/backend/internal/store"
) )
@ -21,94 +20,27 @@ import (
// once before snapshotID: the frozen APPROVED rows are hashed into memoryVersion (F1), so // once before snapshotID: the frozen APPROVED rows are hashed into memoryVersion (F1), so
// editing the seed is a loud --resnapshot. Idempotent (full replace), $0, no LLM. B2 // editing the seed is a loud --resnapshot. Idempotent (full replace), $0, no LLM. B2
// approved dst-collisions are logged (not fatal — some collisions are legitimate). // approved dst-collisions are logged (not fatal — some collisions are legitimate).
//
// The gather and the materialization live in bankmaterialize.go, because a read-only surface has to reach the
// SAME fold without the write in the middle (row 231): what it cannot do is write, and the write is the
// only step it has to skip. This function is therefore the write path's spelling of one shared fold —
// gather, persist, read back, materialize — and the insert order it hands ReplaceBank is untouched.
func (r *Runner) seedGlossary(ctx context.Context) error { func (r *Runner) seedGlossary(ctx context.Context) error {
var entries []store.GlossaryEntry in, err := r.gatherBankInputs()
var voices []store.VoiceProfile // The remarks go out BEFORE the error is acted on, and the order is load-bearing rather than tidy.
var pairs []store.AddressPair // Before the gather was extracted these warnings were emitted inline as they were made, so a book that
if r.Book.GlossarySeed != "" { // died on a later collision check still told the operator about the ruby alias it had skipped or the
bank, err := membank.LoadBankSeed(r.Book.GlossarySeed) // auto-bank rows it had dropped. Logging them after the error return would lose exactly the diagnostics
if err != nil { // of the run that most needs them — the one that failed. gatherBankInputs returns its partial inputs
return err // with the error for this reason.
} for _, rem := range in.remarks {
entries = append(entries, bank.Terms...) r.Log.WarnContext(ctx, rem.msg, rem.args...)
voices, pairs = bank.Voices, bank.Pairs
} }
manualSrcs := map[string]bool{}
for _, e := range entries {
manualSrcs[e.Src] = true
}
ruby, err := r.Store.RubyReadingsForBook(r.Book.BookID)
if err != nil {
return fmt.Errorf("pipeline: read ruby readings for %s: %w", r.Book.BookID, err)
}
// D16.4: attach a manual term's kana ruby-reading as an alias (kana spelling matchable)
// BEFORE appending the auto-candidates, so it only touches the curated manual entries. A
// reading that would collide with a different seeded term (homophone) is skipped+logged,
// not attached (which would fail the book loud on an alias the operator cannot edit out).
if skipped := membank.AttachRubyAliasesToManual(entries, ruby); len(skipped) > 0 {
r.Log.WarnContext(ctx, "ruby kana-alias skipped as a homophone collision (kana form left unmatchable; disambiguate in the seed if needed)",
"skipped", strings.Join(skipped, "; "))
}
// Mined-write path (R1, plan §1(в)/F2): the owner-curated mined-delta file joins the seed as
// Source:"mined" (loadMinedDelta re-stamps the seed loader's Source), so its terms fold into the
// ENRICHED bank version but NOT the base — a re-run that adds signed mined terms moves ONLY snapshot_W2.
// Loaded AFTER the seed/ruby so manualSrcs already reflects the curated seed. A `mined` term that
// duplicates a seed src is caught by membank.ApprovedSharedKeyCollisions below like any other collision.
minedDelta, err := r.loadMinedDelta()
if err != nil { if err != nil {
return err return err
} }
// D39.20 deviation-#1 fix: a mined-delta term whose UNIQUE key (src, sense, since_ch, until_ch — if err := r.Store.ReplaceBank(r.Book.BookID, in.entries, in.voices, in.pairs); err != nil {
// store/migrate.go glossary UNIQUE) already exists in the SIGNED seed makes the flat INSERT in return fmt.Errorf("pipeline: replace bank for %s (%d terms, %d voices, %d pairs): %w", r.Book.BookID, len(in.entries), len(in.voices), len(in.pairs), err)
// ReplaceGlossary crash on that constraint and abort the whole paid run. membank.ApprovedSharedKeyCollisions
// below does NOT catch it (it skips a SAME-dst duplicate, and keys on the firing surface, not the
// UNIQUE tuple). Fail LOUD here with the duplicate list + a fix hint (edit the seed OR drop it from
// the delta) — NEVER a silent merge over the signed seed. Checked BEFORE the append so the delta rows
// are still separable. Deterministic (seed order).
if dups := membank.MinedDeltaSeedCollisions(entries, minedDelta); len(dups) > 0 {
return fmt.Errorf("pipeline: mined-delta %s duplicates term(s) already in the signed seed (would crash ReplaceGlossary on the glossary UNIQUE(book_id,src,sense,since_ch,until_ch)):\n - %s\n fix: correct the term in the seed, or remove it from the mined-delta file — never both (no silent merge over the signed seed)",
r.Book.MinedDelta, strings.Join(dups, "\n - "))
}
entries = append(entries, minedDelta...)
// AUTO-BANK (pack-20 / D39.42 п.3): the engine's own unsigned rows — what the terminologist consolidated
// on the last run's bank-mining boundary. They load exactly like the owner-curated delta (Source:"mined",
// so base-excluded and only the edit-wave snapshot moves) but they are NOT signed: their status is
// auto/draft, which is what routes them to the editor's separately-headed unverified section instead of
// the canon list.
//
// Loading them HERE rather than injecting them mid-run is load-bearing for money. seedGlossary runs before
// checkRebillConsent, so the re-payment projection sees the same bank the edit wave will use; if the rows
// only appeared after the projection, the next run would compare stored edit rows against a bank that has
// not been rebuilt yet and project a re-payment of the whole edit wave that is not real.
autoBank, dropped, err := r.loadAutoBank(entries)
if err != nil {
return err
}
if len(dropped) > 0 {
// A collision with a SIGNED row cannot abort a paid run over an engine-written proposal — the signed
// row simply wins and the proposal is dropped, loudly.
r.Log.WarnContext(ctx, "auto-bank rows dropped: their key is already held by a signed term (the signed term wins)",
"book", r.Book.BookID, "dropped", strings.Join(dropped, "; "))
}
entries = append(entries, autoBank...)
for i := range entries {
entries[i].BookID = r.Book.BookID
}
// Task-6 re-audit: fail loud on a firing key shared by two DIFFERENT approved terms with
// different dst + overlapping windows (the alias generalization of the D16.1 polysemy
// livelock) — checked over the FULL entry set (incl. ruby-attached aliases) before persisting.
if cols := membank.ApprovedSharedKeyCollisions(entries); len(cols) > 0 {
return fmt.Errorf("pipeline: glossary shared-key collisions (A2 / D16.1 livelock class):\n - %s", strings.Join(cols, "\n - "))
}
// A voice/address row naming a character the bank does not have is SILENTLY inert — nothing can ever
// attribute a reply to it — which is the A-class hole this bank exists to close, so it stops the run.
// Checked over the FULL entry set (seed + ruby + mined + auto), because a character may legitimately be
// signed in a delta rather than in the base seed.
if unknown := membank.UnknownVoiceCharacters(entries, voices, pairs); len(unknown) > 0 {
return fmt.Errorf("pipeline: voice/address rows name characters absent from the bank (a profile for a term that does not exist can never fire):\n - %s", strings.Join(unknown, "\n - "))
}
if err := r.Store.ReplaceBank(r.Book.BookID, entries, voices, pairs); err != nil {
return fmt.Errorf("pipeline: replace bank for %s (%d terms, %d voices, %d pairs): %w", r.Book.BookID, len(entries), len(voices), len(pairs), err)
} }
rows, err := r.Store.GlossaryForBook(r.Book.BookID) rows, err := r.Store.GlossaryForBook(r.Book.BookID)
if err != nil { if err != nil {
@ -128,36 +60,7 @@ func (r *Runner) seedGlossary(ctx context.Context) error {
r.Log.WarnContext(ctx, "voice profile windows leave chapters uncovered (deliberate is fine; a typo is not)", r.Log.WarnContext(ctx, "voice profile windows leave chapters uncovered (deliberate is fine; a typo is not)",
"book", r.Book.BookID, "gaps", strings.Join(gaps, "; ")) "book", r.Book.BookID, "gaps", strings.Join(gaps, "; "))
} }
// InjectVoice is FALSE and has no config knob: pack-19 builds the schema and the flagger, and D21 п.2 r.materializeBanks(rows, storedVoices, storedPairs)
// holds the injection conditional until the polygon experiment. It is the fold's condition, so while
// it is false a book with voice rows hashes exactly as it did without them and nobody re-pays for
// authoring a profile. Wiring the injection means setting it and accepting a full --resnapshot.
// The §3 decl stemmer is target data, constant per book; baseIn copies bankIn below, so both banks share
// it. A target with no decl_suffix registry yields an inert stemmer → exact-match post-check as before.
bankIn := membank.BankInput{Rows: rows, Voices: storedVoices, Pairs: storedPairs,
TargetStemmer: lang.NewTargetStemmer(lang.TargetChecksFor(r.Book.TargetLang))}
r.memory = membank.MaterializeBank(bankIn, r.Pipeline.Gates.Glossary.PostcheckGate)
// The DRAFT wave selects over a BASE-scoped bank (Source:mined excluded) so its injection is
// byte-identical across a bank-mining enrichment — matching the draft-wave snapshot (baseMemoryVersion),
// which keeps «one re-payment» honest at the WIRE level, not only the version-hash level. Only the
// editor sees mined terms (the enriched `memory`). When there are no mined rows (every $0 test / the
// golden) the base bank IS the enriched one — share the object, no double materialization, no drift.
baseRows := rows[:0:0]
hasMined := false
for _, row := range rows {
if row.Source == "mined" {
hasMined = true
continue
}
baseRows = append(baseRows, row)
}
if hasMined {
baseIn := bankIn
baseIn.Rows = baseRows
r.baseMemory = membank.MaterializeBank(baseIn, r.Pipeline.Gates.Glossary.PostcheckGate)
} else {
r.baseMemory = r.memory
}
if cols := membank.InjectivityCollisions(rows); len(cols) > 0 { if cols := membank.InjectivityCollisions(rows); len(cols) > 0 {
r.Log.WarnContext(ctx, "glossary approved dst-collisions (B2: two source terms share one Russian surface — the reader cannot tell them apart)", r.Log.WarnContext(ctx, "glossary approved dst-collisions (B2: two source terms share one Russian surface — the reader cannot tell them apart)",
"collisions", strings.Join(cols, "; ")) "collisions", strings.Join(cols, "; "))

View file

@ -7,7 +7,6 @@ import (
"strings" "strings"
"textmachine/backend/internal/chunk" "textmachine/backend/internal/chunk"
"textmachine/backend/internal/membank"
"textmachine/backend/internal/store" "textmachine/backend/internal/store"
) )
@ -204,9 +203,43 @@ type StatusReport struct {
// cent or the whole book — these say "N chunk×stage units already billed under a superseded snapshot // cent or the whole book — these say "N chunk×stage units already billed under a superseded snapshot
// would be paid for again, ~$X at the CURRENT price table" (row 181). Same projection the consent // would be paid for again, ~$X at the CURRENT price table" (row 181). Same projection the consent
// gate refuses on (projectRebill), so status can never quote a different number than the one // gate refuses on (projectRebill), so status can never quote a different number than the one
// translate enforces. Both omitempty: a book with no drift keeps its exact prior bytes. // translate enforces.
RebillUnits int `json:"rebill_units,omitempty"` //
RebillUSD float64 `json:"rebill_usd,omitempty"` // ⚠ NEITHER IS omitempty ANY MORE, and dropping it is the point rather than a tidy-up. Three states
// reach this line — "nothing has run, so there is nothing to re-pay", "computed, and the answer is
// zero" and "the computation failed" — and omitempty rendered all three as the field being ABSENT, so
// the wire could not tell "free" from "unknown". That is the D39.166 п.2 class, which this engine has
// already paid for once ("a $0 price walls the door up — units at zero price arrive with no price").
// RebillBasis below is what distinguishes them; the numbers are now always present so the basis has
// something to qualify. Safe to change: both fields are OFF the seam's allowlist until a consumer
// exists (backlog row 231), and the human renderer branches on the VALUE, not on presence.
RebillUnits int `json:"rebill_units"`
RebillUSD float64 `json:"rebill_usd"`
// RebillOutputUnits is the SAME re-payment counted in OUTPUT UNITS — the granularity every other
// "units" number in this document uses (total_units, progress totals, chapters[].units_total) and the
// one `--max-units` bounds and the platform sells chapters in.
//
// ⚠ IT EXISTS BECAUSE rebill_units IS A DIFFERENT UNIT WEARING THE SAME WORD: chunk×stage, the BILLING
// granularity. Measured on a three-chapter fixture, one document carried rebill_units:15 two lines
// under total_units:6, and the ratio is not something a consumer can divide out — it is
// len(Members)·nDraftStages + nEditStages, varying unit by unit inside one book, with its factors never
// crossing the seam. A consumer sizing a re-pass from rebill_units and handing that number to
// --max-units would buy several times the book it meant to: this pack's own defect, one layer down.
// rebill_units is NOT renamed — it is established and its meaning is unchanged; this is the number a
// caller should size a purchase from.
RebillOutputUnits int `json:"rebill_output_units"`
// RebillBasis says WHAT the two figures above are a projection of, because the number alone cannot
// carry that and a reader must never have to guess:
//
// pending — the fold of the book's decision FILES: the bank the NEXT `translate` will build, so the
// figures answer "what would a re-pass cost" for an edit that has been applied but not yet
// run. This is the ordinary answer.
// stored — the fold could not be built (a broken seed, a collision that would abort ReplaceBank),
// so the figures fall back to the glossary the LAST run stored. They are then a fact about
// the past, not a projection of the next run; the reason is on the WARN log.
// none — no stored row exists, so there is nothing already-billed to re-pay. Genuinely $0.
// failed — the projection itself errored. The figures are zero because they are UNKNOWN.
RebillBasis string `json:"rebill_basis"`
TotalUnits int `json:"total_units"` TotalUnits int `json:"total_units"`
Done int `json:"done"` Done int `json:"done"`
@ -492,17 +525,6 @@ func (r *Runner) Status(ctx context.Context) (*StatusReport, error) {
if len(editStages) > 0 { if len(editStages) > 0 {
rep.Progress.Edit.Total = len(units) rep.Progress.Edit.Total = len(units)
} }
// The unsigned-bank exposure, same definition as the quality report's (a row with a rendering that is
// not approved). Read-only, like everything else here; a read failure says so rather than reporting 0.
if rows, gerr := r.Store.GlossaryForBook(r.Book.BookID); gerr != nil {
r.Log.WarnContext(ctx, "status: could not read the bank; the unsigned-term count is unknown, not zero", "err", gerr)
} else {
for _, e := range rows {
if e.Status != "approved" && strings.TrimSpace(e.Dst) != "" {
rep.UnsignedBankTerms++
}
}
}
passports := map[int]*ChapterPassport{} passports := map[int]*ChapterPassport{}
var chapterOrder []int var chapterOrder []int
@ -631,33 +653,57 @@ func (r *Runner) Status(ctx context.Context) (*StatusReport, error) {
rep.Snapshot = s rep.Snapshot = s
} }
} }
if !rep.SnapshotDrift && (len(draftSnaps) > 0 || len(editSnaps) > 0) { // The bank BOTH projections below are computed against, materialized ONCE.
if err := r.projectStoredMemory(); err != nil { //
// Do not stay silent: a projection failure used to be read silently as "no drift" (chain audit). // It is hoisted out of the drift branch it used to sit inside, and that move is a fix rather than a
r.Log.WarnContext(ctx, "config-drift check failed; drift state unknown (reported as none)", "err", err) // tidy-up: the re-bill projection is computed unconditionally further down, and on a book with
} else { // snapshot drift this branch never ran — leaving r.memory nil, so `current(w)` in projectRebill
checkWave := func(snaps map[string]bool, w wave) { // rendered the wave snapshots over an EMPTY bank (memoryVersion falls back to the hash of no rows,
if len(snaps) != 1 { // snapshot.go), every stored row then differed from it, and status reported the WHOLE book as a
return // re-payment. Materializing before either consumer removes the state where one of them runs without it.
} memBasis, memWarn := r.foldMemoryForRead()
var stored string if memWarn != nil {
for s := range snaps { r.Log.WarnContext(ctx, "the bank fold refused, so the projections below are computed against the glossary the LAST run stored — they are a fact about the past, not a projection of the next run", "basis", memBasis, "err", memWarn)
stored = s }
} // The unsigned-bank exposure, same definition as the quality report's (a row with a rendering that is
cur, _, serr := r.snapshotIDForWave(w) // not approved), counted off THE BANK THIS DOCUMENT JUST FOLDED.
if serr != nil { //
r.Log.WarnContext(ctx, "config-drift check failed for a wave; drift state unknown", "err", serr) // ⚠ It used to be read straight from the stored glossary, and after the fold landed that made one
return // status document describe TWO banks: the money figures spoke about the bank the next run will build
} // while this counter spoke about the bank the last run stored. An operator asking "is there anything
if cur != stored { // to sign before I keep paying?" was told zero while the very next translate would inject an unsigned
rep.ConfigDrift = true // proposal sitting in the auto-bank file and bill for the unit it changed. One document, one bank.
rep.CurrentSnapshot = cur if r.bankRows == nil {
} r.Log.WarnContext(ctx, "status: no bank could be materialized; the unsigned-term count is unknown, not zero")
} else {
for _, e := range r.bankRows {
if e.Status != "approved" && strings.TrimSpace(e.Dst) != "" {
rep.UnsignedBankTerms++
} }
checkWave(draftSnaps, waveDraft)
checkWave(editSnaps, waveEdit)
} }
} }
if memBasis != RebillBasisFailed && !rep.SnapshotDrift && (len(draftSnaps) > 0 || len(editSnaps) > 0) {
checkWave := func(snaps map[string]bool, w wave) {
if len(snaps) != 1 {
return
}
var stored string
for s := range snaps {
stored = s
}
cur, _, serr := r.snapshotIDForWave(w)
if serr != nil {
r.Log.WarnContext(ctx, "config-drift check failed for a wave; drift state unknown", "err", serr)
return
}
if cur != stored {
rep.ConfigDrift = true
rep.CurrentSnapshot = cur
}
}
checkWave(draftSnaps, waveDraft)
checkWave(editSnaps, waveEdit)
}
// One re-pricing read serves both money projections below (reprice.go): the re-payment amount and // One re-pricing read serves both money projections below (reprice.go): the re-payment amount and
// projected_book_usd, which is also the base of the consent threshold — computing them from two // projected_book_usd, which is also the base of the consent threshold — computing them from two
@ -670,15 +716,17 @@ func (r *Runner) Status(ctx context.Context) (*StatusReport, error) {
// not only under ConfigDrift — because SnapshotDrift (rows split across snapshots within one wave) // not only under ConfigDrift — because SnapshotDrift (rows split across snapshots within one wave)
// re-bills too, and that is precisely the case the boolean pair leaves unpriced. $0 and read-only: // re-bills too, and that is precisely the case the boolean pair leaves unpriced. $0 and read-only:
// projectRebill re-renders the wave snapshots and re-prices stored usage; it reaches no provider. // projectRebill re-renders the wave snapshots and re-prices stored usage; it reaches no provider.
if len(statuses) > 0 { var proj RebillProjection
if proj, perr := r.projectRebill(statuses, chunks, withText, rp); perr != nil { var projErr error
if projectable(len(statuses) > 0, memBasis) {
proj, projErr = r.projectRebill(statuses, chunks, withText, rp)
if projErr != nil {
// Same discipline as the drift check above: a failed projection is reported, never // Same discipline as the drift check above: a failed projection is reported, never
// silently rendered as "nothing to re-pay". // silently rendered as "nothing to re-pay".
r.Log.WarnContext(ctx, "re-bill projection failed; the re-payment cost of the drift is unknown (reported as none)", "err", perr) r.Log.WarnContext(ctx, "re-bill projection failed; the re-payment cost of the drift is unknown (reported as unknown, not as none)", "err", projErr)
} else {
rep.RebillUnits, rep.RebillUSD = proj.Rows, proj.USD
} }
} }
rep.RebillBasis, rep.RebillUnits, rep.RebillUSD, rep.RebillOutputUnits = rebillOutcome(len(statuses) > 0, memBasis, proj, projErr)
// Money. // Money.
committed, reserved, err := r.Store.SpentUSD(r.Book.BookID) committed, reserved, err := r.Store.SpentUSD(r.Book.BookID)
@ -730,16 +778,77 @@ func (r *Runner) Status(ctx context.Context) (*StatusReport, error) {
return rep, nil return rep, nil
} }
// projectStoredMemory materializes r.memory from the STORED glossary (read-only: no re-seed, no write) — // projectable reports whether the re-payment projection is worth computing at all.
// the side effect the wave-snapshot projections need. A seed-FILE edit not yet re-run is NOT reflected here //
// (the STORED glossary is projected, not the seed file) — that drift surfaces on the next translate's // It is FALSE when no bank could be materialized, and skipping it then is the point rather than an
// re-seed, exactly as it does for `translate` itself. // optimization. The projection compares every stored row's snapshot against what the CURRENT config
// renders — and with r.memory nil, memoryVersion falls back to the hash of an EMPTY bank (snapshot.go),
// so every row differs from it and the whole book comes back as a re-payment. Running it anyway would put
// the largest figure the field can hold beside a basis that says "unknown", and the CLI would print "the
// figures below are zero because they could not be computed" next to it.
func projectable(hasRows bool, memBasis string) bool {
return hasRows && memBasis != RebillBasisFailed
}
// rebillOutcome is the ONE decision about what the report says on the re-payment axis: the basis and the
// two figures, always together, so a branch cannot set one and forget the other.
//
// It exists as a function rather than as a switch inside Status because the invariant it carries is not
// reachable from a fixture: "failed" needs BOTH the fold and the stored materialization to refuse, which
// no healthy store will do. The first version of this logic left the basis at "failed" while a
// successfully-computed whole-book figure sat beside it — a basis contradicting its own numbers, which is
// worse than having no basis at all. Here the rule is a value, and a table test can hold it to it.
func rebillOutcome(hasRows bool, memBasis string, proj RebillProjection, projErr error) (basis string, units int, usd float64, outputUnits int) {
switch {
case !hasRows:
return RebillBasisNone, 0, 0, 0 // nothing already-billed exists, so nothing can be re-paid
case memBasis == RebillBasisFailed, projErr != nil:
// UNKNOWN, and zeroes that say so rather than figures that lie. EVERY figure, including the
// output-unit count: one number surviving here would be the same contradiction the basis exists to
// prevent, just in a different field.
return RebillBasisFailed, 0, 0, 0
default:
return memBasis, proj.Rows, proj.USD, proj.OutputUnits
}
}
// projectStoredMemory materializes r.memory from the STORED glossary (read-only: no re-seed, no write).
//
// ⚠ IT IS NO LONGER THE READ PATH'S FIRST ANSWER — it is foldMemoryForRead's FALLBACK, and the demotion
// is backlog row 231 (errata 28.08-к). What it materializes is LAST run's fold, and its own comment used
// to justify that with "a seed-FILE edit not yet re-run is NOT reflected here … exactly as it does for
// `translate` itself". The second half was false: TranslateBook calls seedGlossary BEFORE
// checkRebillConsent (bookrun.go), so `translate` re-seeds from the FILES and its consent gate sees the
// edit — while every read-only surface said "nothing moved" for exactly the work the next run would bill
// for. Right after a `bank-apply`, which writes decision FILES and no store row, that is the whole of the
// blind window: the platform had nothing to show a buyer before charging them.
//
// ⚠ IT NOW MATERIALIZES BOTH BANKS, and that is a fix this pack's own test caught rather than a
// generalization. It used to call membank.Materialize (rows alone) and set r.memory only, leaving
// r.baseMemory nil — and the re-bill projection's re-pin branch renders the DRAFT wave against exactly
// that bank (rebill.go → renderedContentHashes), where repin.go skips every position when it is nil. The
// positions then had no reproducible hash, "cannot conclude" took the conservative branch, and a bank move
// that touched BASE rows made status report the whole draft wave as a re-payment — while `translate`,
// which has both banks, charged for a fraction of it. status.go's own promise that it "can never quote a
// different number than the one translate enforces" was false in precisely that state.
//
// It reads the voice and address rows for the same reason: the run materializes with them, and a read that
// materializes without them is a second definition of the same bank. They do not move the version hash
// while InjectVoice is false (membank.ComputeVersionScopedIn), so nothing re-pays for this.
func (r *Runner) projectStoredMemory() error { func (r *Runner) projectStoredMemory() error {
rows, err := r.Store.GlossaryForBook(r.Book.BookID) rows, err := r.Store.GlossaryForBook(r.Book.BookID)
if err != nil { if err != nil {
return err return err
} }
r.memory = membank.Materialize(rows, r.Pipeline.Gates.Glossary.PostcheckGate) voices, err := r.Store.VoiceProfilesForBook(r.Book.BookID)
if err != nil {
return err
}
pairs, err := r.Store.AddressPairsForBook(r.Book.BookID)
if err != nil {
return err
}
r.materializeBanks(rows, voices, pairs)
return nil return nil
} }
@ -911,7 +1020,9 @@ func (r *Runner) Redrive(ctx context.Context, sel RedriveSelector) (*RedriveSumm
if err != nil { if err != nil {
return summary, nil, fmt.Errorf("pipeline: redrive re-bill projection: %w", err) return summary, nil, fmt.Errorf("pipeline: redrive re-bill projection: %w", err)
} }
if err := r.checkRebillConsent(ctx, rebillChunks); err != nil { // nil scope: `redrive` does not take --max-units (invocation.go refuses it there), so there is no
// volume ceiling to narrow this projection by and the book-wide figure is the honest one.
if err := r.checkRebillConsent(ctx, rebillChunks, nil); err != nil {
return summary, nil, err return summary, nil, err
} }

View file

@ -67,11 +67,33 @@ func TestStatusPricesTheDriftWithTheGatesOwnNumber(t *testing.T) {
} }
// The same number, from the enforcement side: the gate's refusal must quote what status showed. // The same number, from the enforcement side: the gate's refusal must quote what status showed.
err = r2.checkRebillConsent(ctx, chunksOf(t, r2)) err = r2.checkRebillConsent(ctx, chunksOf(t, r2), nil)
if err == nil { if err == nil {
t.Fatal("the gate must refuse a fully-superseded book without consent") t.Fatal("the gate must refuse a fully-superseded book without consent")
} }
if want := fmt.Sprintf("~$%.6f", rep.RebillUSD); !strings.Contains(err.Error(), want) { if want := fmt.Sprintf("~$%.6f", rep.RebillUSD); !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not quote the projection %s:\n%v", want, err) t.Errorf("the refusal does not quote the projection %s:\n%v", want, err)
} }
// ⚠ AND THE SAME AGAIN UNDER A VOLUME CEILING, because that is where the two sides could drift apart
// and status.go's docstring promises out loud that they cannot: "status can never quote a different
// number than the one translate enforces". A bounded run re-pays only its own slice, but the question
// the THRESHOLD answers is about the book — so the figure the refusal quotes must still be the book's,
// which is the one status reports. Passing nil here (the shape this pin had while the pack was being
// built) tests only the unbounded case and would miss a divergence entirely.
// A scope that ADMITS the book's single unit — i.e. a real bounded purchase that genuinely re-pays.
// (A scope admitting nothing is a different case and is handled before the threshold: a run that
// re-pays nothing has no concrete spend to consent to.)
bounded := &volumeScope{
admitted: map[chunkKey]bool{{1, 0}: true},
leader: map[chunkKey]bool{{1, 0}: true},
stop: VolumeStop{MaxUnits: 1},
}
err = r2.checkRebillConsent(ctx, chunksOf(t, r2), bounded)
if err == nil {
t.Fatal("a bounded run over a fully-superseded book must still meet the gate: a threshold a caller can shrink by splitting is not a threshold")
}
if want := fmt.Sprintf("~$%.6f", rep.RebillUSD); !strings.Contains(err.Error(), want) {
t.Errorf("under a volume ceiling the refusal stopped quoting the BOOK figure status reports (%s):\n%v", want, err)
}
} }

View file

@ -0,0 +1,581 @@
package pipeline
import (
"context"
"fmt"
"textmachine/backend/internal/chunk"
"textmachine/backend/internal/membank"
"textmachine/backend/internal/store"
)
// volume.go: the VOLUME ceiling — the run's second stop, beside the money one (D39.165 §1б, backlog row
// «потолок объёма»).
//
// WHY A SECOND CEILING AT ALL. The platform sells CHAPTERS and hands the engine a DOLLAR bound
// (--ceiling-usd), which is the only bound the engine had. Measured on real ledgers (D39.165 §1), a
// chapter costs $0.0115$0.0190 and the constant sold against it is $0.03 — so a purchase of ten chapters
// hands over $0.30, and $0.30 buys sixteen to twenty-six. The knob said chapters and meant money. This
// gives the engine a bound in the unit it actually ships, so the seller's promise and the engine's stop
// are the same quantity.
//
// WHAT IT IS MEASURED IN, and why not the other two candidates. The unit is the OUTPUT UNIT — editUnit,
// the granularity `outputUnits` returns and `buildManifest` publishes as the manifest's units_total, which
// is where the platform's per-chapter unit count comes from. So "N units" means the same thing on both
// sides of the seam, and the platform's chapters→units conversion is EXACT rather than an estimate: the
// manifest is $0, needs no keys and exists before the first paid call.
// - NOT chunk×stage, which is the BILLING unit (the one the re-payment estimate counts in). Cutting the
// wave in the unit one PAYS in, while selling the unit one SHIPS in, reproduces the very defect above
// one layer down: a unit costs len(Members)·nDraftStages + nEditStages positions, so "10" would buy a
// different amount of book on a different pipeline shape.
// - NOT the chunk, which is an internal artefact of the cut: the buyer never sees it and the manifest
// does not count it.
//
// ⚠ The unit count DOES depend on the pipeline's shape — an editor pipeline groups draft chunks into
// coarse edit units, a draft-only pipeline ships one unit per draft chunk (outputUnits). That is a
// property of the configuration, not a branch per language pair, and it is safe because the manifest the
// platform counted and the run it bounds come from the SAME deployment. It would stop being safe if a
// ceiling were carried across two deployments with different pipelines.
//
// HOW REPINS AND RETRIES COUNT — the question this design has to answer out loud.
// - A RETRY and an ESCALATION hop do NOT count. They live inside one unit (the attempt loop in
// stagerun.go writes ONE chunk_status row per chunk×stage however many attempts it took), and the
// volume ceiling bounds DELIVERY while the money ceiling bounds SPEND. The measured quality tail
// (retries+escalations swinging 1.5%→23% between runs) is a fact about money and already has its own
// bound; if it burned volume, a buyer of ten chapters would receive eight because two of them
// stuttered — paying in BOOK for the engine's own tail, which is the substitution this pack exists to
// remove.
// - A RE-PIN and a $0 RESUME do NOT count. A re-pass that re-pins five hundred units for $0 (rebill.go)
// would otherwise exhaust the ceiling having delivered nothing.
//
// So: THE CEILING COUNTS OUTPUT UNITS FOR WHICH THIS RUN WILL MAKE AT LEAST ONE PROVIDER CALL. Units that
// resolve for free are admitted regardless — they cost nothing, and holding them back would leave the
// book's snapshots stale for no saving.
//
// AND IT COUNTS THEM IN TWO KINDS, which is the difference between a report and a true report. A paying
// unit is either book that did not exist (DELIVERY) or book that existed and is being made again under a
// moved snapshot (REWORK). unitClass below is that distinction, and VolumeStop carries it all the way to
// the operator: a purchase spent entirely on rework delivers no chapter, and neither the counters nor the
// line the CLI prints may let that read as a delivery. The remainder splits the same way — units never
// delivered are the only ones anybody may be invited to buy, while delivered-but-not-re-made units are
// unrefreshed, not unbought.
//
// ⚠ ON THE PREDICATE'S RELATION TO projectRebill: the free/paying test here is runStage's, not the
// re-payment estimate's, and they are NOT the same test. projectRebill deliberately ignores the CONTENT
// axis (its own doc comment says so: a source edit moves content_hash but not the snapshot, and is
// invisible there). This one checks the rendered content hash on every row, so a source edit correctly
// makes a unit paying here while the estimate would call it free. The difference is in the safe
// direction — the ceiling over-counts cost rather than under-counting it — and it is why this predicate
// is written against the executor rather than borrowed from the projection.
//
// CHECKED BEFORE, NOT AFTER. The scope is computed ONCE, before either wave starts, and the waves simply
// do not begin an item outside it. That is structural rather than a guard: there is no moment at which a
// unit past the ceiling has started, so the ceiling cannot be overshot by one call the way a post-hoc
// token check always is (LiteLLM's known behaviour, named in D39.165 §1б). It is also why the admission
// is not a counter the wave workers decrement — the waves are parallel, and a worker refused a slot would
// have to abandon its item permanently while a slot freed by a $0 re-pin went to nobody.
//
// THE STOP IS A COMPLETION, NOT A HALT — ratified by the orchestrator with this pack and NOT re-decided
// here. A money ceiling stops the run in the MIDDLE of work it meant to do: that is exit 4, the platform
// records `paused`, and the remedy is more money. A volume ceiling means the run did exactly what was
// bought: exit 0, the frozen exit-code dictionary (cmd/tmctl/main.go) is not touched at all, and the
// platform records the run `ready` — which does NOT declare the book finished, because the book's progress
// travels as unit events, so the next purchase starts a new run normally. What DOES have to be said is
// WHICH ceiling stopped it (OpenHands' iteration limit is silent, and that is a standing complaint named
// in the same note): that lives in the run's result, its log line and the CLI's rendering, not in a new
// exit code and not in a new word of the event stream's outcome vocabulary.
// VolumeStop is a run's report that it stopped on the volume ceiling rather than on the end of the book.
// It is a RESULT, not an error: the run did what it was granted, so it travels on BookResult and never
// through the error path, where the exit-code mapper would have to invent a word for it.
//
// ⚠ IT COUNTS PAID UNITS IN TWO KINDS, and the split is a MODEL rather than a nicety. A paying unit is
// either book that did not exist before (DELIVERY) or book that existed and is being made again under a
// moved snapshot (REWORK), and the two are not the same product: the first is what "buy ten chapters"
// means, the second is what a re-pass means. The first version of this struct had one counter for both,
// and every consequence of that was a lie the operator could act on — a purchase that went entirely into
// re-translating the opening chapters reported ten units bought and exited 0 like a delivery, while the
// remaining count invited buying chapters the reader already owned. A struct that cannot tell the two
// apart cannot report either one honestly, so it carries both.
type VolumeStop struct {
// MaxUnits is the ceiling that was in force.
MaxUnits int
// Delivered is paying units that had never been completed before — NEW book, the thing a purchase of
// chapters is actually for.
Delivered int
// Reworked is paying units that WERE already completed and are being paid for a second time because
// their snapshot moved. Real work and a real charge, but not new book: a purchase made entirely of
// these advances the reader not at all.
Reworked int
// Flagged is paid units that resolved FLAGGED and shipped no text. They cost money and delivered
// nothing, and calling them delivered is the lie this counter exists to stop: a purchase of two units
// one of which flagged used to print "2 NEW unit(s) delivered" over a single readable unit.
//
// ⚠ It is filled AFTER the waves, from what actually happened, not from the plan. Everything else on
// this struct is a plan-time decision about admission — correct for bounding money, and unable on its
// own to know whether the work it authorised produced a chapter.
Flagged int
// Free is units that rode along at $0 — resumed or re-pinned. Reported because otherwise the operator
// sees a run that touched far more units than it was granted and cannot tell why.
Free int
// LeftFresh is undelivered units still in the book. THIS is what "there is more to buy" honestly
// means, and it is the only one of the two remainders a buyer should ever be invited to purchase.
LeftFresh int
// LeftRework is completed units still carrying a superseded snapshot. They are "unrefreshed", not
// "unbought", and presenting them as stock for sale is how re-payment becomes a product.
LeftRework int
}
// Paid is every unit this run was charged for, of either kind.
func (v VolumeStop) Paid() int { return v.Delivered + v.Reworked }
// reconcile corrects the plan-time counters against what the run actually produced.
//
// Admission is a decision made BEFORE the work, and it has to be — that is what bounds the money. But a
// decision to pay for a unit is not a fact about a chapter existing: a unit can be paid for and come back
// flagged with nothing shippable. Reporting the plan as though it were the outcome is how "2 NEW unit(s)
// delivered" ended up printed over one readable unit. So the counters are trued up here, from the
// outcomes, before anyone reads them.
func (s *volumeScope) reconcile(outcomes []ChunkOutcome) {
if s == nil {
return
}
for _, oc := range outcomes {
if oc.Disposition != DispFlagged || oc.FinalText != "" {
continue // it shipped something readable; the plan's word for it stands
}
switch {
case s.stop.Delivered > 0 && s.class[chunkKey{oc.Chapter, oc.ChunkIdx}] == unitFresh:
s.stop.Delivered--
case s.stop.Reworked > 0:
s.stop.Reworked--
case s.stop.Delivered > 0:
s.stop.Delivered--
default:
continue // it was never counted as paid (a free unit that flagged on resume); nothing to move
}
s.stop.Flagged++
}
}
// Left is every unit the ceiling held back, of either kind.
func (v VolumeStop) Left() int { return v.LeftFresh + v.LeftRework }
func (v VolumeStop) String() string {
// ⚠ EVERY COUNT HERE IS IN OUTPUT UNITS, AND THE WORDS MUST SAY SO. An earlier version of this line
// read "%d NEW chapter(s) delivered" over a counter that increments per editUnit — and a chapter is
// more than one unit whenever the cut closes a unit at the edit ceiling, and is many units in a
// draft-only pipeline. That is the unit↔chapter substitution this whole ceiling exists to remove,
// committed by the ceiling's own report. Chapters are the PLATFORM's word; it converts them through
// the manifest. The engine speaks units and nothing else.
s := fmt.Sprintf("stopped on the VOLUME ceiling (--max-units %d), not on money and not at the end of the book: %d paying output unit(s) — %d NEW unit(s) delivered, %d already-delivered unit(s) re-made under a moved snapshot",
v.MaxUnits, v.Paid()+v.Flagged, v.Delivered, v.Reworked)
if v.Flagged > 0 {
s += fmt.Sprintf(", and %d PAID FOR BUT FLAGGED — money spent, no readable text produced (buying more will not fix them; `tmctl redrive` re-attacks a flag)", v.Flagged)
}
if v.Free > 0 {
s += fmt.Sprintf("; %d rode along at $0 (resumed or re-pinned)", v.Free)
}
s += fmt.Sprintf(". Still in the book: %d unit(s) NEVER delivered", v.LeftFresh)
if v.LeftRework > 0 {
s += fmt.Sprintf(" and %d already delivered but not yet re-made (those are unrefreshed, NOT unbought)", v.LeftRework)
}
return s
}
// volumeScope is the run's admitted set of output units, decided before any work begins. A nil scope
// means no volume ceiling is in force and nothing anywhere changes behaviour.
type volumeScope struct {
admitted map[chunkKey]bool // unit leader key → admitted
leader map[chunkKey]bool // member chunk key → its unit is admitted (flattened for the draft wave)
stop VolumeStop
// class is what each unit was judged to be at PLAN time, kept because the edit wave has to re-judge
// the free ones (see rescopeEditWave).
class map[chunkKey]unitClass
// editSnapshot is the edit-wave snapshot the classification above was made AGAINST. The run can move
// it after planning — the bank-mining stop re-seeds mid-run — so this is what rescopeEditWave compares
// with to know whether the plan is still standing on the ground it was made on.
editSnapshot string
// editBlocked are units whose EDIT must not run even though they were admitted: they were judged free,
// the bank then moved under them, and no grant was left to pay for what they turned out to cost.
editBlocked map[chunkKey]bool
}
// allows reports whether a DRAFT chunk may be started: it may when the unit it belongs to was admitted.
func (s *volumeScope) allows(ch chunk.Chunk) bool {
if s == nil {
return true
}
return s.leader[chunkKey{ch.Chapter, ch.ChunkIdx}]
}
// allowsUnit reports whether an EDIT unit may be started.
func (s *volumeScope) allowsUnit(u editUnit) bool {
if s == nil {
return true
}
key := chunkKey{u.Chapter, u.FirstChunkIdx}
return s.admitted[key] && !s.editBlocked[key]
}
// admittedStatuses narrows stored rows to the units this run is actually going to work on, so a money
// projection over them describes THIS run rather than the whole book. A nil scope is the identity, which
// is what keeps every un-bounded run's figures byte-identical to what they were before the ceiling
// existed. An edit row lives at its unit's LEADER chunk, and a leader is one of the unit's members, so
// the member map covers both waves' rows.
func (s *volumeScope) admittedStatuses(in []store.ChunkStatus) []store.ChunkStatus {
if s == nil {
return in
}
out := in[:0:0]
for _, cs := range in {
if s.leader[chunkKey{cs.Chapter, cs.ChunkIdx}] {
out = append(out, cs)
}
}
return out
}
// bound reports whether the ceiling actually held work back. A run granted more than the book had left is
// an ordinary complete run and must not be reported as a volume stop — otherwise every generously-bounded
// run would end claiming it had been cut short.
func (s *volumeScope) bound() bool { return s != nil && s.stop.Left() > 0 }
// planVolume decides, before any wave starts, which output units this run may work on.
//
// It is $0 by construction: store reads, string rendering and hashing, no provider anywhere. It is also
// skipped entirely when no ceiling is set, so a run without --max-units pays nothing for this existing —
// not even the content-hash reproduction, which is the expensive half.
func (r *Runner) planVolume(ctx context.Context, chunks []chunk.Chunk, stickySel []membank.Selection) (*volumeScope, error) {
if r.MaxUnits <= 0 {
return nil, nil
}
// ⚠ REFUSED on a pipeline whose SHIPPING stage is a translator while editor stages also exist, because
// in that one shape the two granularities this ceiling straddles stop being the same object. outputUnits
// keys off finalStageWave and returns a singleton per DRAFT CHUNK there, while the edit wave still
// groups those chunks into buildEditUnits — so a ceiling granted in singletons would admit one member's
// chunk and then let the edit wave run the whole multi-member unit over drafts that were never made,
// paying full price for half-empty input. config validates stage ROLES but imposes no order, so the
// shape loads; refusing loudly is the only answer that neither miscounts a purchase nor silently
// changes what a unit means. (It is also the shape in which the platform's chapters→units conversion
// would stop being exact, which is the whole basis of measuring in units at all.)
if len(r.waveStagesIndexed(waveEdit)) > 0 && r.finalStageWave() == waveDraft {
return nil, fmt.Errorf("pipeline: --max-units cannot bound this pipeline: its last stage is a translator while editor stages exist, so the shipping unit is the draft CHUNK while the edit wave still works in grouped edit UNITS — a ceiling counted in one would admit work measured in the other. Put the shipping stage last (an editor/other role), or run this pipeline without a volume ceiling")
}
// ⚠ THE ONE COMPOSITION THIS CEILING MAKES WORSE, said out loud where an operator will see it.
//
// A bounded purchase drafts only part of the book, and the bank-mining stop consolidates over the
// chunks drafted SO FAR — so the auto-bank it writes grows with every purchase. The next purchase folds
// that larger auto-bank into the ENRICHED bank, which moves the edit-wave snapshot, and the edit jobs
// the PREVIOUS purchase created are then pinned to a superseded one: without --resnapshot the run stops
// loudly, and with it the units whose injected bytes the new terms changed are re-translated and re-paid.
//
// None of that is new machinery — it is the mine D39.165 §3 named and errata 28.08-и narrowed to "an
// edit made AFTER edit jobs exist". What IS new is the frequency: without a volume ceiling that state
// needs an interrupted run (which usually has no edit jobs yet), while WITH one it is the ordinary
// shape of selling a book a few chapters at a time. Warned rather than refused, because the run is
// still correct — it re-pays through the consent gate like any other re-payment — and because refusing
// would take the ceiling away from exactly the books that most need selling in parts.
if r.pack != nil && r.Pipeline.Mining.ContrastPath != "" {
r.Log.WarnContext(ctx, "a VOLUME ceiling on a book that MINES its bank: each purchase drafts more, so the bank-mining stop writes a larger auto-bank, and the next purchase moves the edit-wave snapshot the previous purchase's edit jobs are pinned to. Expect that run to need --resnapshot and to re-pay the units the new terms actually touch (the re-payment consent gate still bounds it)",
"book", r.Book.BookID, "max_units", r.MaxUnits)
}
units := r.outputUnits(chunks)
statuses, err := r.Store.ChunkStatusesForBook(r.Book.BookID)
if err != nil {
return nil, fmt.Errorf("pipeline: read chunk_status for the volume ceiling: %w", err)
}
class, curEditAtPlan, err := r.classifyUnits(units, chunks, statuses, stickySel)
if err != nil {
return nil, err
}
s := &volumeScope{
admitted: make(map[chunkKey]bool, len(units)),
leader: make(map[chunkKey]bool, len(chunks)),
stop: VolumeStop{MaxUnits: r.MaxUnits},
class: class,
editSnapshot: curEditAtPlan,
editBlocked: map[chunkKey]bool{},
}
admit := func(u editUnit) {
key := chunkKey{u.Chapter, u.FirstChunkIdx}
s.admitted[key] = true
for _, m := range u.Members {
s.leader[chunkKey{m.Chapter, m.ChunkIdx}] = true
}
}
// ⚠ A MISSING KEY READS AS unitFresh, and that is deliberate rather than incidental: classifyUnits
// returns an EMPTY map for a book with no stored rows, so the zero value has to be the class such a
// book's units actually are. unitFresh is iota 0 for exactly this reason — do not reorder the enum.
classOf := func(u editUnit) unitClass { return class[chunkKey{u.Chapter, u.FirstChunkIdx}] }
// Free units first and unconditionally: they spend nothing, so holding them back would only leave
// their rows pinned to a superseded snapshot for no saving at all.
for _, u := range units {
if classOf(u) == unitFree {
s.stop.Free++
admit(u)
}
}
// ⚠ DELIVERY BEFORE REWORK, and this ORDER is the answer to "what did the buyer actually get".
//
// Both kinds are paying work, so a single pass in book order admits whichever comes first — and after
// any bank edit the already-delivered units ARE the early ones. Measured consequence, on a fixture
// where one chapter is one unit: a purchase of two units on a five-unit book with two delivered
// re-made units 1 and 2 and never started unit 3, reporting Delivered=0, Reworked=2, LeftFresh=3. The
// buyer paid for two units, received none, and three units that had never been translated at all sat
// untouched. A ceiling sold as "the seller's promise and the engine's stop are the same quantity"
// cannot behave that way.
//
// So the grant goes to NEW book while any is left, and only then to re-making. The reason it is safe to
// serve units out of book order is that nothing in a unit's rendered bytes depends on which other units
// ran: the sticky-window selection is precomputed over the WHOLE book before either wave
// (precomputeSticky), so running unit 3 before unit 1 renders exactly what running them in order would.
//
// ⚠ THE TRADE IS NAMED, not hidden: a purchase whose PURPOSE was a re-pass will now spend its grant on
// undelivered units if the book still has any. That is the right default — an undelivered unit is
// worth more to a reader than a re-made one, and the re-payment consent gate bounds rework by money in
// its own right — but it IS a default, and a caller who needs "re-make only" needs a flag that says so.
// Reported to the orchestrator as a chosen default rather than assumed.
for _, pass := range []unitClass{unitFresh, unitRework} {
for _, u := range units {
if classOf(u) != pass {
continue
}
if s.stop.Paid() >= r.MaxUnits {
if pass == unitFresh {
s.stop.LeftFresh++
} else {
s.stop.LeftRework++
}
continue
}
if pass == unitFresh {
s.stop.Delivered++
} else {
s.stop.Reworked++
}
admit(u)
}
}
return s, nil
}
// rescopeEditWave re-judges the FREE units against the snapshot the edit wave is actually about to run
// under, and makes them pay for a grant slot if they turn out to cost money.
//
// ⛔ WITHOUT THIS THE CEILING DOES NOT HOLD, and the report lies while it fails. planVolume classifies
// before the draft wave, but the bank-mining stop sits between the two waves and RE-SEEDS the bank
// mid-run (mining.go, the auto-continue branch), after which waverun recomputes the edit-wave snapshot.
// Every unit admitted as FREE — and free units are admitted OUTSIDE the grant, because free work costs
// nothing — was judged against a snapshot the run itself then replaced. Any of them whose injected bytes
// the new bank changes gets a fresh PAID editor call that no grant ever authorised, and the stop line
// calls it "rode along at $0". Found by the acceptance hunter, who measured four provider calls under a
// grant of one and $0.007280 of spend reported as free.
//
// The fix is not a guard but a re-plan: the free/paying question is asked again, against the snapshot
// that is now real, at the last moment before the edit wave begins — so "check before the unit" still
// holds. A unit that has become paying takes a slot if one is left; if none is, its edit does not run at
// all and it is reported as an already-delivered unit still awaiting its re-make, which is what it is.
//
// It is a no-op on the ordinary path: with no mid-run re-seed the snapshot is unchanged and every free
// unit stays free, so a book that does not mine pays nothing for this existing.
func (r *Runner) rescopeEditWave(ctx context.Context, s *volumeScope, units []editUnit, chunks []chunk.Chunk, stickySel []membank.Selection, editSnapshot string) error {
if s == nil || editSnapshot == s.editSnapshot {
return nil // the ground the plan was made on is still the ground the wave runs on
}
statuses, err := r.Store.ChunkStatusesForBook(r.Book.BookID)
if err != nil {
return fmt.Errorf("pipeline: re-read chunk_status for the volume ceiling's edit-wave re-plan: %w", err)
}
draftStages, editStages := r.waveStagesIndexed(waveDraft), r.waveStagesIndexed(waveEdit)
draftNames, editNames := stageNameSet(draftStages), stageNameSet(editStages)
curDraft, _, err := r.snapshotIDForWave(waveDraft)
if err != nil {
return fmt.Errorf("pipeline: render the draft-wave snapshot for the edit-wave re-plan: %w", err)
}
// The memo was dropped by the re-seed that moved the snapshot (materializeBanks clears it), so this
// renders against the bank the edit wave will actually use.
hashes := r.cachedRenderedContentHashes(chunks, stickySel)
byChunk := map[chunkKey][]store.ChunkStatus{}
for _, cs := range statuses {
byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}] = append(byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}], cs)
}
moved, paid, blocked := 0, 0, 0
for _, u := range units {
key := chunkKey{u.Chapter, u.FirstChunkIdx}
if !s.admitted[key] || s.class[key] != unitFree {
continue // only the units admitted WITHOUT a slot can be wrong about being free
}
rows := unitRows(u, byChunk)
if r.rowsResumeFree(rows, draftNames, editNames, curDraft, editSnapshot, hashes) {
continue // still free under the snapshot that is now real
}
moved++
s.stop.Free--
if s.stop.Paid() < s.stop.MaxUnits {
// It costs money now, and there is grant left to pay for it: it becomes what it is — an
// already-delivered unit being re-made.
s.stop.Reworked++
s.class[key] = unitRework
paid++
continue
}
// No grant left. Its edit does not run; it stays delivered and stale, which is LeftRework.
s.editBlocked[key] = true
s.stop.LeftRework++
s.class[key] = unitRework
blocked++
}
if moved > 0 {
r.Log.WarnContext(ctx, "the bank moved between planning and the edit wave, so units judged FREE are no longer free: they have been re-judged against the snapshot the edit wave actually uses",
"book", r.Book.BookID, "no_longer_free", moved, "took_a_grant_slot", paid, "held_back_for_want_of_grant", blocked,
"planned_against", s.editSnapshot[:12], "running_under", editSnapshot[:12])
}
s.editSnapshot = editSnapshot
return nil
}
// unitClass is what this run would DO to an output unit, which is the distinction the ceiling both
// admits and reports on.
type unitClass int
const (
// unitFresh — never completed. Paying for it DELIVERS a chapter that did not exist.
unitFresh unitClass = iota
// unitFree — fully recorded and every row resumes or re-pins at $0. Costs nothing, so it rides along
// regardless of the ceiling.
unitFree
// unitRework — fully recorded, but at least one row will be paid for again under a moved snapshot.
// Real work and a real charge, and NOT new book: it replaces a chapter the reader already has.
unitRework
)
// classifyUnits sorts every output unit into the three things this run can do to it.
//
// The free/paying predicate is deliberately the one the run itself applies, read off runStage's resume
// fast-path: a stored row is served for free when its rendered content hash is unchanged AND its snapshot
// either matches the current one or is a re-pinnable bank-only move. A unit is free when every one of its
// rows is, and when every position it will execute already HAS a row — a half-done unit resumes what it
// has and pays for the rest.
//
// The fresh/rework split falls straight out of the same completeness test, which is why it costs nothing
// to make: a unit that was never fully recorded has never been delivered, so paying for it is delivery; a
// unit that WAS fully recorded has been delivered once already, so paying for it again is rework.
//
// Every uncertainty resolves to "this unit will cost", never to "this unit is free". That direction is
// forced: mistaking a paying unit for a free one lets the run spend past the ceiling, which is the defect
// the ceiling exists to prevent, while the opposite merely makes a run stop one unit early.
func (r *Runner) classifyUnits(units []editUnit, chunks []chunk.Chunk, statuses []store.ChunkStatus, stickySel []membank.Selection) (map[chunkKey]unitClass, string, error) {
class := map[chunkKey]unitClass{}
if len(statuses) == 0 {
return class, "", nil // a book nothing has run is entirely fresh; skip the rendering entirely
}
draftStages, editStages := r.waveStagesIndexed(waveDraft), r.waveStagesIndexed(waveEdit)
draftNames, editNames := stageNameSet(draftStages), stageNameSet(editStages)
curDraft, _, err := r.snapshotIDForWave(waveDraft)
if err != nil {
return nil, "", fmt.Errorf("pipeline: render the draft-wave snapshot for the volume ceiling: %w", err)
}
curEdit := curDraft
if len(editStages) > 0 {
if curEdit, _, err = r.snapshotIDForWave(waveEdit); err != nil {
return nil, "", fmt.Errorf("pipeline: render the edit-wave snapshot for the volume ceiling: %w", err)
}
}
// The same reproduction the re-payment estimate uses (repin.go): what the run WOULD render for every
// live position. A position whose inputs cannot be reproduced is simply absent, and an absent hash is
// read below as "cannot conclude", i.e. the paying answer.
hashes := r.cachedRenderedContentHashes(chunks, stickySel)
byChunk := map[chunkKey][]store.ChunkStatus{}
for _, cs := range statuses {
byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}] = append(byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}], cs)
}
for _, u := range units {
rows := unitRows(u, byChunk)
key := chunkKey{u.Chapter, u.FirstChunkIdx}
if !unitFullyRecorded(u, rows, draftStages, editStages) {
class[key] = unitFresh // a position with no row is a position this run will call for
continue
}
if r.rowsResumeFree(rows, draftNames, editNames, curDraft, curEdit, hashes) {
class[key] = unitFree
continue
}
class[key] = unitRework
}
return class, curEdit, nil
}
// unitFullyRecorded reports whether EVERY position this run would execute for the unit already has a
// stored row — the completeness half of "this unit costs nothing".
//
// ⚠ IT DELIBERATELY DOES NOT USE resolveChunkState, and the difference is money. That resolver answers
// "what does a reader call this unit", and it returns ChunkFlagged the moment ANY row is flagged, before
// it ever tests whether the expected positions are all present. That is right for a progress projection
// and wrong here: a unit whose draft flagged for one member and whose EDIT row does not exist yet — the
// state every book is left in by a bank-signing stop, a Ctrl-C, or any abort in the edit wave — would be
// read as terminal, judged free, admitted without consuming a grant, and then charged for a full editor
// call. The run would pay for more units than were bought and, with Deferred still 0, would not even
// report a volume stop. Found by this pack's own adversarial pass, reproduced before it was fixed.
//
// Positions are counted, not dispositions: a completed unit always leaves a row for every position it
// has — ok, flagged, or the `skipped` rows recordSkippedStages writes downstream of a flag, including the
// all-members-flagged case where the editor never runs. So full coverage IS terminality, and it is the
// property that actually predicts "no provider call".
func unitFullyRecorded(u editUnit, rows []store.ChunkStatus, draftStages, editStages []wavedStage) bool {
type pos struct {
chapter, chunkIdx int
stage string
}
have := make(map[pos]bool, len(rows))
for _, cs := range rows {
have[pos{cs.Chapter, cs.ChunkIdx, cs.Stage}] = true
}
for _, m := range u.Members {
for _, ws := range draftStages {
if !have[pos{m.Chapter, m.ChunkIdx, ws.st.Name}] {
return false
}
}
}
// The unit's edit rows all live at its LEADER chunk — that is where the read-models look for them.
for _, ws := range editStages {
if !have[pos{u.Chapter, u.FirstChunkIdx, ws.st.Name}] {
return false
}
}
return true
}
// rowsResumeFree is the per-row half of the predicate above.
func (r *Runner) rowsResumeFree(rows []store.ChunkStatus, draftNames, editNames map[string]bool,
curDraft, curEdit string, hashes map[chunkKey]map[string]string) bool {
for _, cs := range rows {
if cs.Disposition == string(DispSkipped) {
continue // a skipped stage is re-derived from the flag above it; it never reaches a provider
}
var cur string
var w wave
switch {
case draftNames[cs.Stage]:
cur, w = curDraft, waveDraft
case editNames[cs.Stage]:
cur, w = curEdit, waveEdit
default:
continue // a stage this pipeline no longer runs cannot cost anything (rebill.go's rule)
}
h, ok := hashes[chunkKey{cs.Chapter, cs.ChunkIdx}][cs.Stage]
if !ok || h != cs.ContentHash {
return false // the wire bytes moved, or could not be reproduced — either way, a fresh call
}
if cs.SnapshotID != cur && !r.repinnable(cs.SnapshotID, cur, w) {
return false
}
}
return true
}

View file

@ -0,0 +1,149 @@
package pipeline
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"textmachine/backend/internal/chunk/chunktest"
"textmachine/backend/internal/obs"
)
// volume_midrun_bankmove_test.go: the SCENARIO behind rescopeEditWave — a purchaser is charged twice for
// a chapter they already own, because the run moved the bank after deciding what was free.
//
// It is deliberately not the same assertion as the invariant test. That one pins "the re-plan runs before
// the wave"; this one pins "the buyer is not billed for a unit no grant paid for", which is the thing the
// buyer's money is actually exposed to. The two can come apart: a re-plan that ran but judged wrongly, a
// third path that moves the bank, or a later refactor that re-orders the seed — each keeps the invariant
// and breaks this.
//
// THE FIXTURE'S ONE TRICK, and the reason a homogeneous book will not show the defect. The mined delta has
// two sides. The WHICH side (the term list) is mined from the SOURCE of the whole book, so it is complete
// after the first purchase and never moves again. The WHAT side (each term's dst) is drafted-side: it is
// folded from the banknote blocks the translator emitted, so it grows with every purchase. So the source
// here is IDENTICAL in every chapter — the asymmetry is in what the drafts PROPOSE:
//
// - chapters 1-2 emit a banknote naming only 青茅山;
// - chapters 3-4 also propose a dst for 方源, which occurs in every chapter's source, chapters 1-2
// included.
//
// Purchase 1 buys chapters 1-2, so the auto-bank it writes holds 方源 with NO rendering. Purchase 2 drafts
// chapter 3, whose banknote supplies one — and the re-seed at the bank-mining stop folds it into the
// ENRICHED bank the editor selects over. Chapters 1-2 were admitted as free against the bank as it was at
// planning; their editor injection now carries a term it did not carry, so their content hash moves and
// they cost a fresh paid call. That is production's ordinary shape (mining.go says so where it re-seeds:
// "a term whose drafts disagreed can carry a different proposal than it did before ... that is bank
// CONTENT"), and it needs no heterogeneous source at all.
func TestAMidRunBankMoveNeverBillsBeyondTheGrant(t *testing.T) {
const chapters, grant2 = 4, 1
rec := &reqRec{}
srv := newJSONProvider(rec, func(body string) (string, string) {
if isEditBody(body) {
return "ОТРЕДАКТИРОВАННЫЙ ПЕРЕВОД", "stop"
}
// The early chapters' drafts propose no rendering for 方源; the later ones do.
note := bankSeparator + "\n青茅山\tгора Цинмао\tplace"
if !strings.Contains(body, "MARK1") && !strings.Contains(body, "MARK2") {
note = bankSeparator + "\n方源\tФан Юань\tname\n青茅山\tгора Цинмао\tplace"
}
return "Фан Юань пришёл к горе Цинмао.\n" + note, "stop"
})
defer srv.Close()
bookPath := miningBookOfNChapters(t, srv.URL, chapters)
ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
// PURCHASE 1: two units. They are delivered, and the auto-bank written at the mining stop holds 方源
// with no rendering yet.
r1 := newRunner(t, bookPath)
r1.MaxUnits = 2
if _, err := r1.TranslateBook(ctx); err != nil {
t.Fatalf("purchase 1: %v", err)
}
r1.Close()
// PURCHASE 2: ONE unit. --resnapshot because the previous purchase's edit jobs are pinned to the
// snapshot this run's own re-seed will move — the state planVolume warns about by name.
before := rec.count()
r2 := newRunner(t, bookPath)
defer r2.Close()
r2.MaxUnits = grant2
r2.Resnapshot = true
res2, err := r2.TranslateBook(ctx)
if err != nil {
t.Fatalf("purchase 2: %v", err)
}
newBodies := rec.all()[before:]
// (1) THE MONEY. A grant of N units buys at most N drafts and N edits. Anything more is a unit the
// buyer did not pay for being billed to them.
if want := grant2 * 2; len(newBodies) != want {
t.Errorf("a grant of %d output unit(s) made %d provider call(s), want exactly %d (draft+edit each): the surplus is work no grant authorised",
grant2, len(newBodies), want)
}
if maxUSD := float64(grant2*2) * fakeCallUSD; res2.TotalUSD > maxUSD+1e-9 {
t.Errorf("the run spent $%.6f on a grant of %d unit(s); at most $%.6f can be owed", res2.TotalUSD, grant2, maxUSD)
}
// (2) WHICH unit was billed, named. The already-delivered chapters must not be re-edited at all.
for _, b := range newBodies {
if !isEditBody(b) {
continue
}
for _, owned := range []string{"MARK1", "MARK2"} {
if strings.Contains(b, owned) {
t.Errorf("chapter %s was delivered by the previous purchase and was EDITED again on a grant that never covered it — the buyer paid twice for a chapter they already owned", owned)
}
}
}
// (3) THE REPORT. A unit that was billed must never be counted among the ones that rode along at $0 —
// that is the line an operator reads to decide the run was cheap.
if res2.Volume == nil {
t.Fatalf("the run stopped short of the book and said nothing about which ceiling did it")
}
if res2.Volume.Free != 0 {
t.Errorf("VolumeStop reports %d unit(s) as riding along at $0, but the bank moved under every one of them: %+v",
res2.Volume.Free, *res2.Volume)
}
if res2.Volume.Paid() > res2.Volume.MaxUnits {
t.Errorf("VolumeStop admits paying for %d unit(s) under a ceiling of %d: %+v",
res2.Volume.Paid(), res2.Volume.MaxUnits, *res2.Volume)
}
t.Logf("purchase 2 (grant %d): %d call(s), $%.6f — %v", grant2, len(newBodies), res2.TotalUSD, res2.Volume)
}
// miningBookOfNChapters is the mining-stop fixture over N EPUB chapters (setupMiningStopProject is
// single-source, and a volume ceiling needs a book with more than one output unit to bound). The chapter
// bodies are byte-identical apart from a MARK<n> tag the provider stub keys off — see the file comment:
// the defect needs asymmetric BANKNOTES, not an asymmetric source.
func miningBookOfNChapters(t *testing.T, providerURL string, n int) string {
t.Helper()
body := strings.Repeat("方源来到青茅山。方源很强。花家很大。花家的人。", 6)
var eps []chunktest.Chapter
var spine []string
for i := 1; i <= n; i++ {
id := fmt.Sprintf("c%d", i)
eps = append(eps, chunktest.Chapter{ID: id, Href: id + ".xhtml", Body: fmt.Sprintf("<p>MARK%d %s</p>", i, body)})
spine = append(spine, id)
}
bookPath := setupProjectOpts(t, providerURL, projectOpts{
epub: eps, spine: spine, regenerate: 1,
gatesYAML: "\nmining:\n contrast_path: mining-contrast.txt\ngates:\n banknote:\n enabled: true\n",
})
dir := filepath.Dir(bookPath)
writeFile(t, filepath.Join(dir, "mining-contrast.txt"), miningContrastData)
packRoot, err := filepath.Abs("../../configs/langpacks")
if err != nil {
t.Fatal(err)
}
raw, err := os.ReadFile(bookPath)
if err != nil {
t.Fatal(err)
}
writeFile(t, bookPath, strings.Replace(string(raw), "source_lang: ja", "source_lang: zh\nlangpack_root: "+packRoot, 1))
return bookPath
}

File diff suppressed because it is too large Load diff

View file

@ -102,7 +102,7 @@ func (e *WaveSignatureStop) Error() string {
// translateBookWaves is the wave driver (R1). It runs the draft wave (draft ∥), the bank-mining stop, the edit wave (edit ∥) and // translateBookWaves is the wave driver (R1). It runs the draft wave (draft ∥), the bank-mining stop, the edit wave (edit ∥) and
// assembles the BookResult. chunks + stickySel come from the precompute pass (chunk.SplitChunks + precomputeSticky), computed by // assembles the BookResult. chunks + stickySel come from the precompute pass (chunk.SplitChunks + precomputeSticky), computed by
// the caller (TranslateBook) after ingest/seed/eager-build. Returns a *WaveSignatureStop when the bank-mining stop stops. // the caller (TranslateBook) after ingest/seed/eager-build. Returns a *WaveSignatureStop when the bank-mining stop stops.
func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, stickySel []membank.Selection) (*BookResult, error) { func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, stickySel []membank.Selection, scope *volumeScope) (*BookResult, error) {
draftStages := r.waveStagesIndexed(waveDraft) draftStages := r.waveStagesIndexed(waveDraft)
editStages := r.waveStagesIndexed(waveEdit) editStages := r.waveStagesIndexed(waveEdit)
workers := r.Pipeline.Waves.Workers workers := r.Pipeline.Waves.Workers
@ -126,6 +126,12 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
draftResults := make([]stageSeqResult, len(chunks)) draftResults := make([]stageSeqResult, len(chunks))
draftUnits := newDraftUnitTracker(r.outputUnits(chunks), chunks) draftUnits := newDraftUnitTracker(r.outputUnits(chunks), chunks)
if err := r.runWave(ctx, workers, len(chunks), func(ctx context.Context, i int) error { if err := r.runWave(ctx, workers, len(chunks), func(ctx context.Context, i int) error {
// The VOLUME ceiling (volume.go), applied where the item BEGINS rather than after it: a chunk whose
// output unit is outside this run's granted scope is never started, so no ceiling can be overshot by
// one call. A nil scope means no ceiling is in force and this is a no-op.
if !scope.allows(chunks[i]) {
return nil
}
d, err := r.runDraftChunk(ctx, draftSnapshot, chunks[i], stickySel[i], draftStages, !editWave) d, err := r.runDraftChunk(ctx, draftSnapshot, chunks[i], stickySel[i], draftStages, !editWave)
if err != nil { if err != nil {
return err return err
@ -158,6 +164,18 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
} }
res := &BookResult{BookID: r.Book.BookID} res := &BookResult{BookID: r.Book.BookID}
// The stop NAMES its ceiling (D39.165 §1б: OpenHands' silent iteration limit is the anti-pattern being
// avoided). It rides on the result rather than on an error because a volume stop is a COMPLETION — the
// run did what was bought — so it must not reach the exit-code mapper at all. Only set when the ceiling
// actually held work back: a run granted more than the book had left simply finished.
if scope.bound() {
res.Volume = &scope.stop
r.Log.InfoContext(ctx, "the run is bounded by the VOLUME ceiling: it will stop having done what was granted, not at the end of the book",
"book", r.Book.BookID, "max_units", scope.stop.MaxUnits,
"paid_units", scope.stop.Paid(), "delivered_new", scope.stop.Delivered, "re_made", scope.stop.Reworked,
"free", scope.stop.Free,
"left_never_delivered", scope.stop.LeftFresh, "left_delivered_not_re_made", scope.stop.LeftRework)
}
// The terminologist's spend is THIS run's spend and belongs in this run's total. It is not a chunk cost, // The terminologist's spend is THIS run's spend and belongs in this run's total. It is not a chunk cost,
// so the per-chunk accumulation below cannot see it — and a paid call that no total reports is a call // so the per-chunk accumulation below cannot see it — and a paid call that no total reports is a call
// the operator cannot notice. (Zero when the gate is off or the calls replayed from checkpoints.) // the operator cannot notice. (Zero when the gate is off or the calls replayed from checkpoints.)
@ -166,6 +184,9 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
// --- Draft-only pipeline: the draft IS the shipping output; assemble per draft chunk (no edit units) --- // --- Draft-only pipeline: the draft IS the shipping output; assemble per draft chunk (no edit units) ---
if !editWave { if !editWave {
for i, ch := range chunks { for i, ch := range chunks {
if !scope.allows(ch) {
continue // outside the volume scope: never drafted, so there is no outcome to assemble
}
oc := r.draftOnlyOutcome(ch, draftResults[i]) oc := r.draftOnlyOutcome(ch, draftResults[i])
res.Chunks = append(res.Chunks, oc) res.Chunks = append(res.Chunks, oc)
res.TotalUSD += oc.CostUSD res.TotalUSD += oc.CostUSD
@ -173,6 +194,7 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
res.Flagged++ res.Flagged++
} }
} }
scope.reconcile(res.Chunks)
r.Log.InfoContext(ctx, "book run finished (draft-only)", "book", r.Book.BookID, r.Log.InfoContext(ctx, "book run finished (draft-only)", "book", r.Book.BookID,
"chunks", len(res.Chunks), "flagged", res.Flagged, "run_usd", fmt.Sprintf("%.6f", res.TotalUSD)) "chunks", len(res.Chunks), "flagged", res.Flagged, "run_usd", fmt.Sprintf("%.6f", res.TotalUSD))
return res, nil return res, nil
@ -187,6 +209,21 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
return nil, fmt.Errorf("pipeline: upsert edit-wave snapshot %.12s: %w", editSnapshot, err) return nil, fmt.Errorf("pipeline: upsert edit-wave snapshot %.12s: %w", editSnapshot, err)
} }
units := buildEditUnits(chunks) units := buildEditUnits(chunks)
// ⛔ THE PLAN IS RE-JUDGED HERE, because the bank-mining stop above may have re-seeded the bank and
// moved the snapshot this wave runs under. Units the plan called FREE were admitted outside the grant;
// any that the new bank makes paying must take a slot or not run. See rescopeEditWave.
if err := r.rescopeEditWave(ctx, scope, units, chunks, stickySel, editSnapshot); err != nil {
return nil, err
}
// ⚠ AND THE RE-PLAN IS NOT OPTIONAL: the wave refuses to run a scope that was planned against a
// different snapshot than the one it is about to use. Without this the call above is merely a call —
// deleting it leaves every test green while the ceiling silently stops holding on any book that
// re-seeds mid-run. With it, the invariant is the code's own, and removing the re-plan makes the run
// say so instead of overspending quietly.
if scope != nil && scope.editSnapshot != editSnapshot {
return nil, fmt.Errorf("pipeline: the volume scope was planned against edit-wave snapshot %.12s but the wave is running under %.12s — units admitted as free were judged on ground this run has since replaced, and paying for them would escape the ceiling (wave sequencing bug: rescopeEditWave did not run)",
scope.editSnapshot, editSnapshot)
}
draftByKey := make(map[chunkKey]stageSeqResult, len(chunks)) draftByKey := make(map[chunkKey]stageSeqResult, len(chunks))
for i, ch := range chunks { for i, ch := range chunks {
draftByKey[chunkKey{ch.Chapter, ch.ChunkIdx}] = draftResults[i] draftByKey[chunkKey{ch.Chapter, ch.ChunkIdx}] = draftResults[i]
@ -195,6 +232,11 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
"units", len(units), "workers", workers, "edit_stages", len(editStages)) "units", len(units), "workers", workers, "edit_stages", len(editStages))
unitOutcomes := make([]*ChunkOutcome, len(units)) unitOutcomes := make([]*ChunkOutcome, len(units))
if err := r.runWave(ctx, workers, len(units), func(ctx context.Context, i int) error { if err := r.runWave(ctx, workers, len(units), func(ctx context.Context, i int) error {
// Same admission as the draft wave, and the SAME set: a unit is either in this run's scope for both
// waves or in neither, so the edit wave can never be handed a unit whose members were not drafted.
if !scope.allowsUnit(units[i]) {
return nil
}
oc, err := r.runEditUnit(ctx, editSnapshot, units[i], draftByKey, editStages) oc, err := r.runEditUnit(ctx, editSnapshot, units[i], draftByKey, editStages)
if err != nil { if err != nil {
return err return err
@ -207,12 +249,17 @@ func (r *Runner) translateBookWaves(ctx context.Context, chunks []chunk.Chunk, s
return nil, err return nil, err
} }
for _, oc := range unitOutcomes { for _, oc := range unitOutcomes {
if oc == nil {
continue // outside the volume scope: the unit was never started this run
}
res.Chunks = append(res.Chunks, *oc) res.Chunks = append(res.Chunks, *oc)
res.TotalUSD += oc.CostUSD res.TotalUSD += oc.CostUSD
if oc.Disposition == DispFlagged { if oc.Disposition == DispFlagged {
res.Flagged++ res.Flagged++
} }
} }
// The plan's counters become the run's counters only now, corrected by what actually shipped.
scope.reconcile(res.Chunks)
r.Log.InfoContext(ctx, "book run finished", "book", r.Book.BookID, r.Log.InfoContext(ctx, "book run finished", "book", r.Book.BookID,
"units", len(res.Chunks), "flagged", res.Flagged, "run_usd", fmt.Sprintf("%.6f", res.TotalUSD)) "units", len(res.Chunks), "flagged", res.Flagged, "run_usd", fmt.Sprintf("%.6f", res.TotalUSD))
return res, nil return res, nil

File diff suppressed because one or more lines are too long

View file

@ -22,7 +22,7 @@
| Роль | Активный промт | Статус | | Роль | Активный промт | Статус |
|---|---|---| |---|---|---|
| Оркестратор | [ORCHESTRATOR_SESSION_PROMPT.md](ORCHESTRATOR_SESSION_PROMPT.md) | роль и нормы; счётчик роли — CURRENT-STATE | | Оркестратор | [ORCHESTRATOR_SESSION_PROMPT.md](ORCHESTRATOR_SESSION_PROMPT.md) | роль и нормы; счётчик роли — CURRENT-STATE |
| Бэкенд | [BACKEND_MONEY_PACK_SESSION_PROMPT.md](BACKEND_MONEY_PACK_SESSION_PROMPT.md) | **НАПИСАН, ждёт запуска владельцем.** Пак «деньги»: потолок ОБЪЁМА в движке рядом с денежным (D39.165 §1б) + $0-глагол сметы «оценить, ничего не покупая» (строка **231**). Оба рубежа пройдены: механический + опровергатель, **9 находок применены**, включая три фатальные — форма ответа через шов (словарь кодов ЗАМОРОЖЕН; решено: стоп по объёму = ЗАВЕРШЕНИЕ, код 0), выдуманный мной якорь, и вырожденное репро, повторявшее эррату 28.08-и. ⚠ Честная граница вписана в §0: проводка числа глав до движка паком НЕ заказана | | Бэкенд | активного НЕТ | пак «деньги» ПРИНЯТ И ЗАЛЕНДЖЕН 29.08 (**D39.170**): потолок ОБЪЁМА оплаченной работы + читающий путь `status` сворачивает банк, поэтому смета доезжает до покупателя ДО покупки. Промт отработан — `archive/prompts/`. ⚠ Следующий заказ уже назван строкой **232** (ось отгрузки на `once_key`), проводка `--max-units` в платформу ГЕЙЧЕНА |
| Платформа | активного НЕТ | пак **P11** ПРИНЯТ И ЗАЛЕНДЖЕН 29.08 (**D39.169**): `PD-379` закрыт живой пробой оркестратора (отзыв → поток гаснет за секунду кадром `session_ended` → переподключение `401`), видимость застрявших денег, денежная группа. Промт отработан — `archive/prompts/`. ⚠ Открытым остаётся денежный `major` `PD-425` (дверь коррекций теряет пост-verb факт при обрыве клиента), отсрочка на оркестраторе | | Платформа | активного НЕТ | пак **P11** ПРИНЯТ И ЗАЛЕНДЖЕН 29.08 (**D39.169**): `PD-379` закрыт живой пробой оркестратора (отзыв → поток гаснет за секунду кадром `session_ended` → переподключение `401`), видимость застрявших денег, денежная группа. Промт отработан — `archive/prompts/`. ⚠ Открытым остаётся денежный `major` `PD-425` (дверь коррекций теряет пост-verb факт при обрыве клиента), отсрочка на оркестраторе |
| Полигон | [POLYGON_EXP2223_REDO_SESSION_PROMPT.md](POLYGON_EXP2223_REDO_SESSION_PROMPT.md) (отложенный — [POLYGON_PACKAGE4_SESSION_PROMPT.md](POLYGON_PACKAGE4_SESSION_PROMPT.md), строка 85) | фаза Д ИДЁТ; ⚠ живой носитель курса — в `eval/dovodka/`, какой именно называет зона (⚠ [POLYGON_PHASE_D_HANDOFF.md](POLYGON_PHASE_D_HANDOFF.md) — перекрытый снимок, читать не как курс) | | Полигон | [POLYGON_EXP2223_REDO_SESSION_PROMPT.md](POLYGON_EXP2223_REDO_SESSION_PROMPT.md) (отложенный — [POLYGON_PACKAGE4_SESSION_PROMPT.md](POLYGON_PACKAGE4_SESSION_PROMPT.md), строка 85) | фаза Д ИДЁТ; ⚠ живой носитель курса — в `eval/dovodka/`, какой именно называет зона (⚠ [POLYGON_PHASE_D_HANDOFF.md](POLYGON_PHASE_D_HANDOFF.md) — перекрытый снимок, читать не как курс) |
| Фронт | активного НЕТ | **ЗОНА ЗАМОРОЖЕНА** (D39.136 п.2 + D39.147: разморозка отдельным словом владельца, не привязана к P7); перечень первого касания — в зонном журнале | | Фронт | активного НЕТ | **ЗОНА ЗАМОРОЖЕНА** (D39.136 п.2 + D39.147: разморозка отдельным словом владельца, не привязана к P7); перечень первого касания — в зонном журнале |

View file

@ -1,4 +1,4 @@
# Реестр D-нот — карта актуальности v2 (D1D39.169; # Реестр D-нот — карта актуальности v2 (D1D39.170;
> ⚠ **СЛАБОЕ МЕСТО, КОТОРОЕ БЫЛО ЗДЕСЬ (вписано 22.08, ЗАКРЫТО 24.08 — D39.157 п.6).** Колонка ТЕЛА > ⚠ **СЛАБОЕ МЕСТО, КОТОРОЕ БЫЛО ЗДЕСЬ (вписано 22.08, ЗАКРЫТО 24.08 — D39.157 п.6).** Колонка ТЕЛА
> у нот D39.107…D39.123 говорила «жив», хотя тела уехали в слайс подрезкой D39.139; семнадцать строк > у нот D39.107…D39.123 говорила «жив», хотя тела уехали в слайс подрезкой D39.139; семнадцать строк
@ -230,3 +230,4 @@
| D39.167 | 28.08 | **Аудит документации (хребет)**: очередь пять дней звала активным исполненный промт и держала на владельце решённую развилку; регистр платформы расходился с актами лендинга ЧЕТЫРЬМЯ строками, одна major — класс «open, а лекарство в дереве» без гейта (строка 225); реестр D-нот потерял две колонки на восьми моих строках; справочники отстали от двух суток миноров. Журнал ужат 943 → 347 строк, прожитая проза бэкенда — в слайс. | жив | ЖИВОЕ: строка 225 (гейт класса); три участка обхода не пройдены | доки процесс регистр аудит | | D39.167 | 28.08 | **Аудит документации (хребет)**: очередь пять дней звала активным исполненный промт и держала на владельце решённую развилку; регистр платформы расходился с актами лендинга ЧЕТЫРЬМЯ строками, одна major — класс «open, а лекарство в дереве» без гейта (строка 225); реестр D-нот потерял две колонки на восьми моих строках; справочники отстали от двух суток миноров. Журнал ужат 943 → 347 строк, прожитая проза бэкенда — в слайс. | жив | ЖИВОЕ: строка 225 (гейт класса); три участка обхода не пройдены | доки процесс регистр аудит |
| D39.168 | 28.08 | **Аудит доков, часть 2 — архитектура и ресёрчи.** Две НОРМЫ требовали построенного (закон шва: «`status --json` версию не несёт — закрыть», а несёт); доки врали про `accepts_labels` («пусты у всех» при трёх заполненных эндпоинтах — ось аккаунта); денежный вход цитировал грант $5 при коде 0. Вынесены в `archive/research/` четыре мёртвых отчёта (290 КБ) по графу входящих ссылок; двум ресёрчам без шапки вовсе поставлены ревю-шапки. | жив | ЖИВОЕ: системный класс line-якорей в архитектуре; часть 3 не пройдена | доки ресёрчи архив аудит | | D39.168 | 28.08 | **Аудит доков, часть 2 — архитектура и ресёрчи.** Две НОРМЫ требовали построенного (закон шва: «`status --json` версию не несёт — закрыть», а несёт); доки врали про `accepts_labels` («пусты у всех» при трёх заполненных эндпоинтах — ось аккаунта); денежный вход цитировал грант $5 при коде 0. Вынесены в `archive/research/` четыре мёртвых отчёта (290 КБ) по графу входящих ссылок; двум ресёрчам без шапки вовсе поставлены ревю-шапки. | жив | ЖИВОЕ: системный класс line-якорей в архитектуре; часть 3 не пройдена | доки ресёрчи архив аудит |
| D39.169 | 29.08 | **Платформенный пак P11 принят и заленджен + контрактный минор 0.8.0.** Отзыв сессии гасит открытый SSE-поток (`PD-379`, единственная уязвимость `major`) и объявляет причину кадром `session_ended`, а не молчаливым обрывом в `401`; застрявший расчёт видим и закрываем; денежная группа `PD-384/391/394/397/376`. Второй рубеж сессии поймал СЕМЬ ложных подтверждений в её же отчёте (включая выдуманную мутационную посадку) — код цел, достоверность рассказа нет. Охотник вне карты нашёл ЧЕТЫРЕ регресса, введённых самим паком, все закрыты до сдачи. Две мои ошибки в каноне: схема нового кадра была сиротой вне союза `anyOf`, перечень кадров соединения противоречил телу схемы. Линтер якорей нашёл 21 указатель на исчезнувший код — норма: гонять его ПОСЛЕДНИМ шагом, после того как код замер. | жив | ЖИВОЕ: `PD-425` — открытый денежный major, отсрочка на оркестраторе; `PD-424`/`PD-426` отложены с доводом; `PD-423` — условие батареи, которого не знал рецепт | платформа контракт деньги сессии приёмка | | D39.169 | 29.08 | **Платформенный пак P11 принят и заленджен + контрактный минор 0.8.0.** Отзыв сессии гасит открытый SSE-поток (`PD-379`, единственная уязвимость `major`) и объявляет причину кадром `session_ended`, а не молчаливым обрывом в `401`; застрявший расчёт видим и закрываем; денежная группа `PD-384/391/394/397/376`. Второй рубеж сессии поймал СЕМЬ ложных подтверждений в её же отчёте (включая выдуманную мутационную посадку) — код цел, достоверность рассказа нет. Охотник вне карты нашёл ЧЕТЫРЕ регресса, введённых самим паком, все закрыты до сдачи. Две мои ошибки в каноне: схема нового кадра была сиротой вне союза `anyOf`, перечень кадров соединения противоречил телу схемы. Линтер якорей нашёл 21 указатель на исчезнувший код — норма: гонять его ПОСЛЕДНИМ шагом, после того как код замер. | жив | ЖИВОЕ: `PD-425` — открытый денежный major, отсрочка на оркестраторе; `PD-424`/`PD-426` отложены с доводом; `PD-423` — условие батареи, которого не знал рецепт | платформа контракт деньги сессии приёмка |
| D39.170 | 29.08 | **Движковый пак «деньги» принят и заленджен**: потолок ОБЪЁМА оплаченной работы (`--max-units`) + читающий путь `status` теперь СВОРАЧИВАЕТ банк, поэтому смета пере-прохода впервые доезжает до покупателя ДО покупки, оставаясь $0. ⚠ Пак СНЯЛ ПОСЫЛКУ чужой зоны: платформа не берёт `rebill_*` по доводу «status читает ноль сразу после apply» (`ingest/resync.go:37-43`) — довод устарел. Четыре круга приёмки; блокирующая находка охотника: потолок ПРОБИВАЛСЯ пере-сидом банка посреди прогона (грант 1 → 4 вызова, $0.0073 вместо $0.0036, две доставленные главы оплачены дважды и названы бесплатными). Лечение структурное — пере-план + отказ волны работать с планом чужого снапшота. Сессия не воспроизвела сценарий ПЯТЬ раз при верных прогонах: дельту двигает не текст, а предложения черновиков, а `spoilerBlocked` (`membank/memory.go:640-647`) режет термин с поздним `since_ch` навсегда. | жив | ЖИВОЕ: ось отгрузки на `once_key` — отдельный заказ; терминолог вне потолка — вход в калибровку цены; проводка `--max-units` ГЕЙЧЕНА | движок деньги потолок приёмка шов |

View file

@ -1,4 +1,4 @@
# Журнал решений оркестратора — контракт D1D39.169 (живой файл: карта · эрраты · живые тела · голова D39.124+ (подрезка D39.139); тела закрытых эр — в слайсах `docs/archive/architecture/`, указатель ниже; реестр всех нот — `05-decisions-index.md`) # Журнал решений оркестратора — контракт D1D39.170 (живой файл: карта · эрраты · живые тела · голова D39.124+ (подрезка D39.139); тела закрытых эр — в слайсах `docs/archive/architecture/`, указатель ниже; реестр всех нот — `05-decisions-index.md`)
> **КАРТА АКТУАЛЬНОСТИ (ревизия D31, продлена до D38.2 [12.07]; исторические записи ниже НЕ переписываются — дисциплина D23.3).** Работая с контрактом (греп номера: живой файл → слайсы, целиком НЕ читать — D39.125), держи под рукой, что чем перекрыто: > **КАРТА АКТУАЛЬНОСТИ (ревизия D31, продлена до D38.2 [12.07]; исторические записи ниже НЕ переписываются — дисциплина D23.3).** Работая с контрактом (греп номера: живой файл → слайсы, целиком НЕ читать — D39.125), держи под рукой, что чем перекрыто:
> ⚠ **Эррата 09.08 (D39.125):** D39.111 п.1 предписывал промту S3 «максимум = баланс МИНУС открытые холды» — формула ОШИБОЧНА (вычитание дважды), исправлена D39.115 п.2(а): максимум = Balance КАК ЕСТЬ; тело D39.111 живёт ниже в этом файле (голова D39.106+). > ⚠ **Эррата 09.08 (D39.125):** D39.111 п.1 предписывал промту S3 «максимум = баланс МИНУС открытые холды» — формула ОШИБОЧНА (вычитание дважды), исправлена D39.115 п.2(а): максимум = Balance КАК ЕСТЬ; тело D39.111 живёт ниже в этом файле (голова D39.106+).
@ -1235,3 +1235,100 @@ D39.165 «смета уже публикуется в `status`» верна то
**Строки.** Регистр платформы: 426 строк, 105 open, 7 major. Семь заказанных строк закрыты с телами; **Строки.** Регистр платформы: 426 строк, 105 open, 7 major. Семь заказанных строк закрыты с телами;
девять новых заведены тем же деревом. девять новых заведены тем же деревом.
## D39.170 — ДВИЖКОВЫЙ ПАК «ДЕНЬГИ» ПРИНЯТ И ЗАЛЕНДЖЕН: потолок ОБЪЁМА оплаченной работы + смета пере-прохода, которая наконец доезжает до покупателя (29.08, оркестратор №19). ✅
**Что заленджено.** Движок останавливается по ОБЪЁМУ оплаченной работы, а не только по деньгам
(`--max-units`, `VolumeStop`) — покупатель платит за N юнитов и получает ровно N · читающий путь
`status` СВОРАЧИВАЕТ банк, а не проецирует сохранённый глоссарий, и потому впервые отвечает на вопрос
«сколько будет стоить пере-проход» ДО покупки, оставаясь $0-глаголом без записи · денежный отчёт
разведён на `Delivered`/`Reworked`/`LeftFresh`/`LeftRework`, и приглашение купить произносится только
про никогда-не-доставленное · авто-банк пишется атомарно.
⚠ **ГЛАВНОЕ, ЧЕГО НЕ НАЗВАЛА НИ ОДНА СЕССИЯ: этот пак СНЯЛ ПОСЫЛКУ, на которой стоит решение ЧУЖОЙ
зоны.** Платформа сознательно не берёт `rebill_units`/`rebill_usd` через шов, и её основание записано в
коде (`platform/internal/ingest/resync.go:37-43`): «status проецирует СОХРАНЁННУЮ память, и сразу после
`bank-apply` — в единственный момент, когда согласие хотело бы цифру, — он честно читает ноль». Это было
верно и ратифицировано эрратой 28.08-к. **Теперь неверно:** `foldMemoryForRead` стал ПЕРВЫМ ответом
читающего пути, а `projectStoredMemory` понижена до фолбэка (`status.go:817-824`, комментарий самого
пака это и объявляет). Слепое окно закрыто. Следствия проведены этим же лендингом: строка бэклога
**231** закрывается, комментарий платформы получает строку в её регистр, а проводка полей через шов
становится ВОЗМОЖНОЙ — но не выполняется, потому что гейчена вместе с `--max-units` (ниже).
**Приёмка — четыре круга, и первые три сдачи были неверны.** Мой первый проход дал 4 денежных дефекта
(обход гейта согласия дроблением покупок; потолок, тративший покупку на пере-делку вперёд доставки;
расхождение `status`/`translate`; неатомарный авто-банк). Второй круг сессии — ещё шесть её собственных,
включая то, что **её же строка стопа сообщала ЮНИТЫ как ГЛАВЫ**, то есть подмену, ради устранения которой
пак и заведён, совершал его собственный отчёт. Четвёртый круг — денежный охотник вне карты, пять
подтверждённых прогоном.
⛔ **БЛОКИРУЮЩАЯ НАХОДКА ЧЕТВЁРТОГО КРУГА: потолок объёма ПРОБИВАЛСЯ, и пробивал его сам прогон.**
`planVolume` классифицирует юниты ДО волн и по снапшоту, который берёт один раз (`volume.go:353-355`);
free-юниты допускаются БЕЗУСЛОВНО, вне гранта (`:259-265`) — бесплатная работа ничего не стоит. Но между
планированием и редакторской волной стоит стоп майнинга, который пере-сеивает банк ПОСРЕДИ прогона
(`mining.go:242`), после чего edit-снапшот берётся заново (`waverun.go:203`). Значит каждый «бесплатный»
юнит судился по снапшоту, который прогон сам же и заменил. Замер: грант 1 → **4 вызова вместо 2**,
$0.007280 вместо максимум $0.003640, две уже доставленные главы отредактированы повторно — и строка
отчёта назвала их «rode along at $0».
**Лечение — не гард, а ПЕРЕ-ПЛАН:** вопрос «бесплатен ли юнит» задаётся заново против снапшота, ставшего
реальным, перед самой волной; ставший платным берёт слот или не идёт. **И оно структурно:** волна
ОТКАЗЫВАЕТСЯ работать со скоупом, спланированным против другого снапшота — без этого удаление вызова
оставляло батарею зелёной при молча переставшем держать потолке.
⚠ **ЧЕТЫРЕ КРУГА ПОТРЕБОВАЛИСЬ НЕ ПОТОМУ, ЧТО СЕССИЯ ПЛОХО РАБОТАЛА — А ПОТОМУ ЧТО СЦЕНАРИЙ
КОНТРИНТУИТИВЕН, И ЭТО САМОЕ ЦЕННОЕ ЗНАНИЕ ПАКА.** Сессия пыталась воспроизвести дефект ПЯТЬ раз и не
смогла ни разу; её прогоны были ВЕРНЫ, ошибочна была гипотеза о причине. Две ловушки, обе измерены:
1. **Не та сторона дельты.** Список терминов майнится из ИСХОДНИКА всей книги (`mining.go:80-98`) — он
полон после первой покупки и не растёт. Байты банка двигает другая сторона: `dst` термина, который
складывается из banknote-блоков ЧЕРНОВИКОВ (`mining.go:109`) и растёт с каждой покупкой **даже при
побайтово однородном источнике**. Асимметрию надо строить в том, что предлагает МОДЕЛЬ, а не в тексте.
2. **Условия тянут в РАЗНЫЕ стороны.** Естественный способ заставить дельту расти — дать поздним главам
новый термин — ровно этим выталкивает `since_ch` за пределы уже доставленного, а
`spoilerBlocked` (`membank/memory.go:640-647`) — жёсткий гейт `chapter < since_ch`: такой термин не
попадёт в инъекцию ранней главы НИКОГДА, какой бы `dst` он ни получил. Нужна ОБРАТНАЯ асимметрия:
термин в раннем исходнике, рендеринг поздно.
**Урок, годный за пределами этого пака:** пять верных прогонов при неверной гипотезе неотличимы от
«дефекта нет». Различил их только тот, у кого сценарий уже работал.
**Доказательства, которые я снял САМ, а не принял.** Батарея: 21 пакет из 21, 0 FAIL, полнота сверена
списком `go list` против вердиктов. Линтер `0 issues`. Четыре ключевых пина зелёные поимённо. **Две мои
мутационные посадки на сценарном тесте:** снятие вызова пере-плана → структурный отказ адресным
сообщением; снятие вызова И гарда → дефект целиком, всеми четырьмя денежными утверждениями. То есть
находка настоящая, тест ловит ПЕРЕОПЛАТУ, а не факт вызова, и фикс её закрывает.
**Статус находки уточнён ПРОТИВ автора, обеими сторонами.** Возражение сессии («сдвиг `memory_version`
ничего не доказывает — мерить надо `content_hash` юнита») принято, и охотник показал, что мерил именно
его. Но он же признал, что **завысил срочность**: дефект требует конъюнкции трёх условий, в его
собственной фикстуре два термина из трёх её не выполняют и остаются инертными; «покупка №50
переоплачивает 490 юнитов» — потолок тяжести, когда сработало, а не ожидаемый случай, и частоту на
реальной книге он не мерил и назвать не может. Лечение от узости входа не зависит — оно структурно.
**ПРОВОДКА `--max-units` В ПЛАТФОРМУ ОСТАЁТСЯ ГЕЙЧЕНОЙ, и основание усилилось.** Я ставил гейт на
доводе «вторая покупка на майнящей книге падает без `--resnapshot`». Настоящее основание оказалось
сильнее — **с `--resnapshot` потолок не держал**, — и нашёл его охотник, не я. Записываю как есть:
везение, а не прозорливость.
**ЧТО ОТЛОЖЕНО СТРОКАМИ, диспозиции мои.** (1) **Ось «свежий/пере-делка» выведена из ПОЛНОТЫ СТРОК, а
не из факта отгрузки** — поэтому добавление стадии превращает дочитанную книгу в «никогда не
доставлявшуюся», а юнит, прерванный между волнами, тратит слот дважды. Носитель у движка уже есть —
реестр анонсов `events_outbox.once_key` (`unitOnceKey` = книга+волна+глава+юнит), монотонный на всю
жизнь книги; не хватает читающего метода поверх готовой константы `onceKeyLookup`. Отдельный заказ:
сессия законно не полезла в чужой пакет в хвосте четвёртого круга. ⚠ Формулировка строки — «ошибается
не только слово»: отчёт, зовущий дочитанную книгу недоставленной, ПРИГЛАШАЕТ купить её снова.
(2) **Терминолог вне объёмного потолка** — место траты в ЦЕНЕ, как договорено, но цифра меняет вес
договорённости: накладные масштабируются КНИГОЙ, а не грантом (три покупки по одному юниту дали три
полнокнижных консолидации по $0.005460 каждая, при том что юнит дешевле). Книга на 500 юнитов,
проданная по одному, оплатит 500 полнокнижных проходов — это вход в калибровку цены, а не сноска.
(3) **Майнящие книги под потолком** — корень в джоб-гарде ратифицированного Р6-контура, чужой предмет.
**Названная граница, которую сессия записала, а не умолчала:** флагнутый юнит следующей покупкой едет
как `Free` и в остатке стопа не виден. Я проверил цепь до конца и границу принимаю: движок отчитывает
его в `status` (`flagged`, пер-главные паспорта), платформа берёт это аллоулистом (`Flagged`,
`UnitsFlagged``ingest/resync.go`), а читателю он доезжает как `withheld` (`ingest/export.go:47`,
канон §UnitState). Покупателю не показывают фальшивое «доставлено» — цепь цела, дублировать её в стопе
покупки незачем.
**Сужение D20.2-Q2 — подтверждена ДЕЙСТВУЮЩАЯ редакция:** порог судится по ВСЕЙ книге (дробить
бесполезно), именованный кап — по тому, что заплатит ЭТОТ прогон (законная работа не отклоняется),
прогон без пере-оплаты гейта не встречает. Первая редакция отозвана самой сессией и помечена отозванной,
а не переписана молча.

View file

@ -0,0 +1,240 @@
> ⚠ **ОТРАБОТАН И ЗАКРЫТ 29.08 — D39.170.** Пак принят и заленджен. Исход: потолок ОБЪЁМА оплаченной
> работы построен; читающий путь `status` сворачивает банк, поэтому смета пере-прохода впервые доезжает
> до покупателя ДО покупки, оставаясь $0-глаголом. Приёмка шла ЧЕТЫРЕ круга: первые три сдачи были
> неверны, и блокирующую находку — потолок пробивался пере-сидом банка посреди прогона — дал охотник вне
> карты, а не отчёт. Отложены строками **232** (ось отгрузки на `once_key`) и **233** (терминолог вне
> потолка); проводка `--max-units` в платформу ГЕЙЧЕНА. **Инструкции отсюда НЕ исполнять** — файл
> историчен.
# Промт: бэкенд, пак «деньги» — продажа обретает настоящий стоп, а смета становится доступной до покупки
> **Выдан оркестратором №19, 28.08.2026.** Основание — **D39.165** §1 (три продуктовых развилки сняты
> словом владельца) и строка бэклога **231**. Пак идёт ПАРАЛЛЕЛЬНО платформенному.
> ⚠ **ФАЙЛЫ вы не делите** — модули отдельные, платформа кода движка НЕ компилирует и его тестов НЕ
> запускает (проверено). **Но ШОВ общий и ведётся РУКАМИ в трёх точках:** таблица кодов выхода ·
> argv из строковых литералов · аллоулист полей `status --json`. **Твой §3.1 трогает первые две**
> форму ответа я решил за тебя (§3.1), а не оставил на согласование.
## §0. Какая проблема и что решит твой результат
**1. Ручка «купить N глав» ничего не ограничивает.** Платформа продаёт ГЛАВЫ, а движку передаёт
только денежный потолок: `--ceiling-usd` (`backend/cmd/tmctl/invocation.go:129`), и тот каппит
**КУМУЛЯТИВНУЮ** трату по книге (`backend/internal/store/ledger.go:67`
`bookTotal+estimate > c.BookUSD`). **Про число глав движок не знает вовсе**, флага такого нет.
**Измерено приёмкой 28.08 (леджеры реальных прогонов, $0):** настоящая цена главы — средняя
**$0.01150.0190**, максимум одной главы **$0.0375**; константа продажи платформы — **$0.03**. То
есть покупка «10 глав» отдаёт движку $0.30, а $0.30 на этом материале покупает **1626 глав**.
Пользователь просит одно, получает другое, и полоса каппится на купленном.
**И хвост цены порождает КАЧЕСТВО, а не объём** (наблюдение владельца, подтверждено данными):
вызов может отработать успешно и вернуть мусор — деньги списаны, юнит уезжает на ретрай, иногда на
эскалационный хоп к дорогой модели. В леджере это видно с именами причин (`empty`, `length`,
`cjk_artifact`, `sanitizer_defect`), и доля ретраев+эскалаций гуляет **1.5% → 23%** между прогонами.
**2. Смету пере-прохода показать НЕЧЕМ — и это названная цена вчерашнего минора 0.7.0.** Движок уже
считает её (`projectRebill`, `backend/internal/pipeline/rebill.go:111`) и публикует в `status --json`.
Но `status` материализует банк из СОХРАНЁННОГО глоссария
(`backend/internal/pipeline/status.go:733-744`, `projectStoredMemory`его собственный комментарий:
«A seed-FILE edit not yet re-run is NOT reflected here»), а `bank-apply` пишет только ФАЙЛЫ решений.
Свёртка происходит внутри СЛЕДУЮЩЕГО `translate`. **Итог: сразу после правки банка движок честно
отвечает «ничего не двигалось», и платформе нечего показать покупателю.** Носитель — строка **231**.
**Что решит результат:** движок получает СПОСОБНОСТЬ остановиться по объёму, а пользователь получает
возможность увидеть, что затронет пере-проход, ДО оплаты.
**ЧЕСТНАЯ ГРАНИЦА, которую первая редакция промта скрывала (нашёл опровергатель).** «Купил N глав —
получил N глав» этим паком НЕ становится правдой, потому что **число глав до движка не доезжает**:
`CeilingChapters` живёт на платформе (`platform/internal/runs/runs.go:232`) до самого спавна, а
`runner.TranslateArgs` собирает argv литералами и канала для него не имеет. **Проводка — платформенная
половина, и она в этом паке НЕ заказана.** Ты строишь механизм; труба к нему — отдельный заказ, и я
его завожу строкой. Не считай пак провалившимся оттого, что после него обещание §0 ещё не выполнено.
## §1. Зона записи и git
**Твоя зона — `backend/`.** Итоги и находки — своя секция журнала `docs/PROGRESS.md` («Бэкенд»).
- **Ты НЕ коммитишь.** Лендит оркестратор после адверсариальной приёмки.
- ⚠ **`platform/` НЕ ТРОГАТЬ.** Там параллельно работает платформенный пак. Нужна правка на той
стороне — это ПИНГ, а не правка.
- ⚠ **`docs/` — не твоя зона**, кроме своей секции журнала. ⚠ Перед записью в неё **перечитай файл**:
журнал правят две сессии, и 28.08 оркестратор уже унёс чужую секцию своим коммитом из-за общего
окна записи.
- ⚠ **Стендовый `tmctl` собирай из ЗАФИКСИРОВАННОЙ копии дерева, а не из рабочего**, если он тебе
понадобится: рабочее дерево грязное твоими же правками, и бинарь из него мерит непонятно что.
- В дереве незакоммиченная работа полигона (20 позиций) — не касаться.
## §2. Карта чтения — ≤5 позиций, ЗАКОН
1. **`backend/cmd/tmctl/main.go:252-262`** — три денежных флага и почему они ОРТОГОНАЛЬНЫ; там же
предупреждение «`--ceiling-usd`НЕ бюджет прогона» · **`invocation.go:120-210`** (парсинг и
валидация флагов).
2. **`backend/internal/store/ledger.go:30-80`** — где потолок реально судит трату (`Reserve`).
3. **`backend/internal/pipeline/rebill.go`** — `RebillProjection`, `projectRebill` (`:111`), порог
согласия и его текст отказа (`:318-324`).
4. **`backend/internal/pipeline/status.go:733-744`** — `projectStoredMemory`; **читать ВНИМАТЕЛЬНО:
это и есть слепое окно из §0.2**.
5. **`docs/architecture/05-decisions-log.md`, тело D39.165** (греп `^## D39.165`) — основание пака;
**и эррата 28.08-к в шапке того же файла** — там названа моя ошибка про смету, не повтори её.
## §3. Состав пака
### §3.1. Потолок ОБЪЁМА рядом с денежным — ЗАКАЗ; форма свободна
**Заказано:** движок умеет остановиться по объёму работы, а не только по деньгам, и **говорит, по
какому потолку встал**.
**Ратифицировано D39.165 §1б и обсуждению не подлежит:** потолок живёт в ДВИЖКЕ. Резать волну на
стороне платформы нельзя — она волной не владеет и пер-юнитной цены заранее не знает.
**Свободен и обязан обосновать — В ЧЁМ мерить. ⚠ Кандидатов ТРИ, а не два (поправка опровергателя):**
(а) **выходной юнит**`ManifestUnit`, та самая гранулярность, которую движок отгружает, и она же
платформенный `chapters.units_total` с интейка; (б) **чанк**; (в) **`chunk×stage` — БИЛЛИНГОВАЯ
единица движка**, ровно та, в которой считает смета (`rebill.go:322`: «%d chunk×stage unit(s)»).
**Ловушка ровно здесь:** режущий волну естественно считает то, за что ПЛАТЯТ, то есть (в), а
платформа продаёт (а) — и купивший десять глав получит их долю. **Дефект пака воспроизведётся слоем
ниже, в той же форме.** Выбор объясни и назови, как он ложится на платформенную единицу продажи.
⚠ **И ещё одно, чего первая редакция не спрашивала: как против объёмного потолка считаются РЕПИНЫ и
РЕТРАИ.** Пере-проход, пере-привязывающий пятьсот юнитов за $0 (`rebill.go:315-320`), сожжёт объёмный
потолок, не потратив цента. Ответь на это явно.
⛔ **ФОРМА ОТВЕТА ЧЕРЕЗ ШОВ РЕШЕНА МНОЙ, и это НЕ предмет твоего выбора — потому что выбор здесь
ломает деньги в обе стороны (нашёл опровергатель).** Словарь кодов выхода объявлен ЗАМОРОЖЕННЫМ самим
движком (`backend/cmd/tmctl/main.go:125-127`: «both bands are frozen seams and a fresh code would be a
word added to a ratified dictionary»), а платформа незнакомый код читает как ОТКАЗ
(`platform/internal/ingest/exit.go:168-169`, `default: OutcomeFailed`). Отсюда:
- переиспользуешь код денежного потолка → платформа объявит прогон ПРИОСТАНОВЛЕННЫМ и возобновит его
за уже купленный объём — дефект, ради которого пак заведён, воспроизведётся слоем ниже;
- заведёшь новый код → пользователь, получивший РОВНО купленное, увидит «ошибка сервиса».
**РЕШЕНИЕ: остановка по объёму — это НЕ остановка, а ЗАВЕРШЕНИЕ.** По деньгам прогон встал ПОСРЕДИ
работы, там честно «приостановлен» и возобновление. По объёму он сделал ровно то, что куплено, — это
УСПЕХ, код **0**, замороженный словарь не трогается вовсе, платформа читает штатное «готово».
Различение, которого требует пункт ниже, живёт тогда не в КОДЕ ВЫХОДА, а в отчёте/логе. Если найдёшь,
что это ломает что-то, чего я не вижу, — **пинг, и я пере-решаю**: на этой развилке уже стоял выбор,
ломающий деньги в обе стороны.
⚠ **Две грабли, забранные у внешних систем — исполнить обе:**
- **остановка обязана НАЗЫВАТЬ свой потолок.** У OpenHands (`max_budget_per_task` + `max_iterations`)
агент упирается в лимит итераций МОЛЧА — известная жалоба. У нас «встал по деньгам» и «встал по
объёму» — разные ответы пользователю и разные ремеди;
- **проверять ПЕРЕД началом единицы работы, а не после.** У LiteLLM пост-фактум-проверка токенного
потолка всегда пробивает его на один вызов. Денежный потолок у нас проверяется в `Reserve` до
вызова — объёмный обязан вести себя так же.
**Не сломай ортогональность.** `--ceiling-usd` — кумулятивный КНИЖНЫЙ кап, не бюджет прогона
(`main.go:256-260`). Новый потолок — про работу ЭТОГО прогона. Смешаешь семантику — сломаешь
платформенный расчёт холдов, который на кумулятивности и стоит.
### §3.2. Смета БЕЗ покупки — ЗАКАЗ; форма свободна
**Заказано:** платформа может узнать «сколько юнитов затронет пере-проход», НЕ запуская `translate` и
ничего не покупая.
**Что знать** (проверено мной, пере-проверь): смета уже считается (`projectRebill`) и уже публикуется
в `status --json` полями `rebill_units`/`rebill_usd`. Мешает ровно одно — **тайминг свёртки**:
`status` читает СОХРАНЁННЫЙ глоссарий, а решения лежат в ФАЙЛАХ до следующего `translate`.
**Форма свободна.** Очевидные кандидаты, оба со своей ценой:
- **флаг у `status`** («сверни решения в память для расчёта и не пиши») — дёшево, но `status`
ратифицирован как ЧИСТО ЧИТАЮЩИЙ ремонтный глагол, и свёртка внутри него это свойство ломает;
- **новый $0-глагол** — честнее по разделению, но новая поверхность CLI и новый документ.
**Родня, которую надо знать: `resnapshot --dry-run`** — строка **124(в)**, поименована в законе шва
как будущая дверь. Она просит ТУ ЖЕ способность «оценить, ничего не покупая». **Если твоя форма
закрывает обе — скажи это прямо, это удешевляет проект.**
⛔ **Границы, обе жёсткие, но сформулированы ТОЧНО (первая редакция обе переоценила — нашёл
опровергатель):**
- **$0 — абсолютно:** ни одного провайдерского вызова.
- **Read-only — с ОДНОЙ названной оговоркой:** «ни одной записи в стор» абсолютом быть НЕ МОЖЕТ —
первое касание проекта создаёт и мигрирует базу движка по построению (`backend/internal/pipeline/runner.go:193-197`;
ратифицировано D39.122, у `status` тот же побочный эффект). Требование верное: **никаких записей
СВЕРХ этого first-touch**, и ни одной записи в банк/глоссарий/чекпойнты.
- **Ключей НЕ требует.** ⚠ Якорь первой редакции (`dotenv.go:53-55`) я ВЫДУМАЛ — там середина
докблока про другое. Настоящий носитель: `backend/cmd/tmctl/invocation.go:152-155` (текст отказа,
перечисляющий читающие глаголы) и пин `backend/cmd/tmctl/keysfile_test.go`.
**ТРЕТИЙ КАНДИДАТ, ДЕШЕВЛЕ ОБОИХ МОИХ — его нашёл опровергатель, и я его не видел.** У `bank-apply`
УЖЕ ЕСТЬ режим проекции (`invocation.go:126`: «print the projection of the decisions and write
NOTHING»), он берёт тот же арбитр-флок, ничего не пишет и печатает JSON-отчёт. **И у него уже на
руках ровно недостающее состояние**: `readBookState` поднимает bank + seed + delta + rejects ДО
свёртки. Не хватает только statuses и manifest для `projectRebill`. Рассмотри этот путь ПЕРВЫМ: он не
трогает «`status` — чисто читающий» вовсе.
**ЛОВУШКА, КОТОРАЯ УБЬЁТ ВЕСЬ СМЫСЛ, если её не увидеть.** Свернуть решения В СТОР ты не сможешь
(`store.go:117-128`, `OpenReadOnly` ставит `PRAGMA query_only=1`), значит фолд будет В ПАМЯТИ. Но
канонический фолд идёт ЧЕРЕЗ стор: `seeding.go` пишет банк, читает его обратно `ORDER BY src, sense,
` и материализует. Порядок НЕ безразличен — `membank/memory.go:610-617` сортирует стабильно «в
порядке замороженных entries», а дальше токенный бюджет РЕЖЕТ хвост. **Материализуешь «будущий банк»
в порядке добавления — получишь другой набор строк глоссария, другие отрендеренные байты, другой
content-hash: бесплатная смета назовёт одно число, а платный прогон выставит другое.** Это ровно тот
дефект, который пак продаёт как решённый. Докажи тождество, а не предположи его.
**И мина формы:** оба поля сметы объявлены `omitempty` (`status.go:208-209`). D39.166 п.2 уже
заплатил за это однажды: «$0-цена мурует дверь — юниты по нулевой цене приходят БЕЗ цены». Если
переиспользуешь форму, «ноль юнитов» и «не считалось» на проводе станут неотличимы.
**Факт из строки 231, который тебе полезен:** поля `rebill_units`/`rebill_usd` СНЯТЫ с аллоулиста
шва до появления потребителя — вернутся вместе с твоим глаголом.
### §3.3. Чего в паке НЕТ — пропуски подписаны
- **Калибровка константы $0.03 — НЕ твоя и НЕ сейчас.** Она платформенная и гейчена строкой **202**
(живой прогон текущего стека). Ты даёшь МЕХАНИЗМ, цену считает не движок.
- **Цена от объёма исходника** (D39.165 §1а) — платформенная половина, носитель `chapters.units_total`.
- **Строки 228 / 230 / 141-остаток / 131** — соседи, не заказ. Увидишь — назови строкой, не чини.
## §4. Самопроверка ИСПОЛНЕНИЕМ — «перечитал сам» её не удовлетворяет
1. **Батарея зоны** под `-race`, линтер 0 issues; числа — с командами.
2. **Потолок объёма доказывается ПОСТ-ФИКСНЫМ инвариантом:** прогон с потолком в N единиц
останавливается на N, отвечает признаком «встал по ОБЪЁМУ» (отличимым от денежного), и
останавливается ПЕРЕД началом N+1, а не после.
3. ⛔ **Смета доказывается $0 — и РЕПРО ПЕРВОЙ РЕДАКЦИИ БЫЛО ВЫРОЖДЕННЫМ, я повторил ошибку, о
которой сам же написал эрратy 28.08-и.** Состояние «`translate` не запускался» негодно: у такой
книги строк `chunk_status` нет вовсе, и `projectRebill` отдаёт ПУСТО и до фикса, и после
(`rebill.go:132-134` пропускает строки без снапшота). **Верное состояние: прогон СОСТОЯЛСЯ**
(строки `chunk_status` есть, снапшоты записаны) → **ПОТОМ применена правка банка** → и твой
механизм отдаёт НЕнулевой счёт затронутых юнитов, тогда как сегодняшний `status` на том же
состоянии отвечает нулём. Разница «сегодня ноль / после фикса N» и есть деливерабл.
При этом леджер не двинулся ни на цент, а в стор не ушло ничего сверх first-touch.
4. **Свои посадки мутаций:** минимум по одной на пункт; вердикт — по ДЕЛЬТЕ против чистой базовой
линии и по ТОПИЧНОСТИ упавшего теста, не по цвету батареи.
5. **Адверсариальный проход по своей готовой работе — заказан; глубину и веер выбираешь сама.**
⚠ Куда смотреть в ЭТОМ паке: деньги (перерасход либо отказ законной работе) · тождество бесплатной сметы и платного прогона · шов (форма ответа, argv).
Дефекты, которые внёс сам пак, называй первыми — приёмке они не видны.
**Fable 5** — обычно хватает одного-двух агентов, но это РЕКОМЕНДАЦИЯ, не потолок: веер под предмет выбирает сессия. Модель задавай ЯВНО и знай, сколько их работает.
## §5. Оси ревью — 13, вправе заменить с аргументом
1. **Ось денег:** может ли новый потолок дать перерасход или, наоборот, отказать законной работе;
не сломана ли кумулятивная семантика денежного капа.
2. **Ось «пара, которой в репо нет»:** ветвления по паре/книге в Go быть не должно.
3. **Ось детерминизма:** трогаешь сборку запроса или снапшот — не перекупается ли прогон.
## §6. Записка-план, комплектность
**До правок** — записка-план в своей секции журнала. **В конце** — таблица комплектности против §3:
пункт → что сделано → каким ИСПОЛНЕНИЕМ подтверждено. Слово вместо команды = пункт НЕ сделан.
## §7. Эхо-протокол старта
ДО работы — ≤10 строк: **скоуп · инварианты · не-делать**. Первое действие — вписать свой блок в файл
канала (§10), сразу второе — послать мне эхо.
## §8. Obstacle — обязательная секция
**«Что НЕ удалось и что НЕ проверено»** отдельной секцией, не россыпью.
## §9. Канал вопросов и твоё право отказаться
Конфликт промта с кодом или доками — **пинг, не интерпретация в свою пользу**.
**У тебя есть право сказать «этого делать не надо» — с аргументом.** За последние сутки исполнители
опровергли ЧЕТЫРЕ моих заказа и все четыре раза были правы; два я пере-проверил своей посадкой и
отозвал. Отказ с разбором дороже послушного исполнения.
**Тесты и гейты не подгонять под зелень** (D39.121).
## §10. Как со мной связаться
**Адрес — в файле `/tmp/textmachine-channel`.** Впиши свой блок первым делом, чужие не трогай.
**Канала нет ⇒ НЕ искать:** вопрос секцией в отчёт, работа продолжается.