Land the accepted tmctl migrate pack: a write open with no run, the typed schema_mismatch refusal at exit 13, restore point under the lock

This commit is contained in:
heaven 2026-08-15 18:55:33 +03:00
parent 3b1fe9aff9
commit d55edd48f1
10 changed files with 1260 additions and 48 deletions

View file

@ -8,10 +8,10 @@
| Пакет | Что делает | Ключевые файлы |
|---|---|---|
| `cmd/tmctl` | CLI: `translate` / `status` (read-only N/M+паспорта глав+деньги, `--json`; работает ПРИ живом прогоне — store без flock) / `report` (request-log+леджер книги **+ per-run quality-report**: структурный KPI предл./нарратив-абзац, тире, cosmetic-strip/echo-rates, глоссарий-промахи, trust-gated — наблюдаемость, НЕ гейт; D39.2-T4) / `export` (**экспорт-поверхность для полигона**: final_hash→checkpoint→checks.ExportNormalize, manifest-join c pending/ghost-учётом, ConfigDrift-поле, `--plaintext`; D39.5) / `redrive` (переатака флагнутых: `--chapter/--chunk/--reason/--dry-run`, D15.3) / `manifest` (**$0-производитель персиста манифеста глав/чанков**, строка 100: нужен ДО первого прогона — дерево глав разобранной, но не запущенной книги; `--json` печатает сам документ). Денежный аргумент прогона: `--ceiling-usd` (строка 145) перекрывает книжный `ceilings.book_usd` ТОЛЬКО на этот прогон, в book.yaml не пишется, ноль/отрицательное = отказ запуска. main — тонкая обвязка: разбор/exit-коды/.env/рендеры в тестируемых функциях | `main.go`, `invocation.go`, `render.go`, `dotenv.go` |
| `cmd/tmctl` | CLI: `translate` / `status` (read-only N/M+паспорта глав+деньги, `--json`; работает ПРИ живом прогоне — store без flock) / `report` (request-log+леджер книги **+ per-run quality-report**: структурный KPI предл./нарратив-абзац, тире, cosmetic-strip/echo-rates, глоссарий-промахи, trust-gated — наблюдаемость, НЕ гейт; D39.2-T4) / `export` (**экспорт-поверхность для полигона**: final_hash→checkpoint→checks.ExportNormalize, manifest-join c pending/ghost-учётом, ConfigDrift-поле, `--plaintext`; D39.5) / `redrive` (переатака флагнутых: `--chapter/--chunk/--reason/--dry-run`, D15.3) / `manifest` (**$0-производитель персиста манифеста глав/чанков**, строка 100: нужен ДО первого прогона — дерево глав разобранной, но не запущенной книги; `--json` печатает сам документ) / `migrate` (**$0-write-open без прогона**, строка 174: read-only команды требуют ТОЧНОГО совпадения схемы и не мигрируют, поэтому апгрейд бинаря запирал все существующие книги — платформа зовёт `status --json` перед каждым спавном, а write-команда, которая мигрировала бы, не наступала никогда. Грузит ТОЛЬКО `book.yaml` — ни цен, ни ключей на $0-пути (строка 146); restore point берёт ПОД ЛОКОМ и только когда шаг реально применяется (имя `<метка>-pre-migrate.db` — вне секундного неймспейса платного пути, строка 173); несовпадение схемы у read-only путей — типизированный отказ **exit 13** с машинным токеном `schema_mismatch found=N expected=M`). Денежный аргумент прогона: `--ceiling-usd` (строка 145) перекрывает книжный `ceilings.book_usd` ТОЛЬКО на этот прогон, в book.yaml не пишется, ноль/отрицательное = отказ запуска. main — тонкая обвязка: разбор/exit-коды/.env/рендеры в тестируемых функциях | `main.go`, `invocation.go`, `render.go`, `dotenv.go` |
| `internal/llm` | OpenAI-совместимый транспорт + retry/backoff, **capability-слой** (budget_field/temperature/reasoning per-модель), провайдеры openai/local (no-proxy)/anthropic (DEPRECATED-референс); `failover.go` удалён паком-17 — маршрутизация лейблами живёт в config/pipeline (канал B, один хоп `chain[0]`, fail-closed) | `httpllm.go`, `capability.go`, `provider_*.go` |
| `internal/ledger` | Цены по usage (вкл. reasoning/cache-поля), `PriceForResponse` по фактической модели | `pricing.go` |
| `internal/store` | SQLite (modernc, CGO-free), цепочка миграций `schema_version` (⚠ ALTER-шаги v8+ не идемпотентны вопреки шапке — бэклог-строка 49а), reserve/settle+checkpoint, chunk_status, глоссарий (+подписной цикл терминолога), ruby, retrieval_state, request_log; `OpenReadOnly` — без flock/миграций/recovery для status/report/export | `ledger.go`, `migrate.go`, `glossary.go`, `store.go` |
| `internal/store` | SQLite (modernc, CGO-free), цепочка миграций `schema_version` (⚠ ALTER-шаги v8+ не идемпотентны вопреки шапке — бэклог-строка 49а), reserve/settle+checkpoint, chunk_status, глоссарий (+подписной цикл терминолога), ruby, retrieval_state, request_log; `OpenReadOnly` — без flock/миграций/recovery для status/report/export (схема не совпала — типизированный `*store.SchemaMismatchError{Found,Expected}`, обе стороны); `Migrate` (+шов `beforeApply` под локом: restore point берётся там) — поверхность деплой-шага строки 174, `SchemaHead` — та самая `Expected`; write-open БД новее бинаря теперь ОТКАЗ, а не тихое открытие | `ledger.go`, `migrate.go`, `glossary.go`, `store.go` |
| `internal/config` | fail-fast загрузка models/pipeline/book; **эхо-мина-гейт** `echoMineViolation`; `CheckRunnable` блокирует неисполнимое (C2/fanout/judge); **пар-слой** (D39.23): `configs/pairs/<пара>.yaml` — калибровка нарезки + коридор длины пары; **промпт-резолв КОНВЕНЦИЕЙ** `prompts/<пара>/<роль>.md` из `Book.LangPair()`+роли, **fail-loud на пару без пакета** (ja-книга не едет молча через zh-промпт), осознанное исключение — `prompt_override` на стадии | `models.go`, `pipeline.go`, `pair.go` |
| `internal/text` | Нормализация и рун-примитивы, общие для банка/майнера/чекеров (script-agnostic слой, `lang` НЕ импортирует): `NormalizeSourceKey`/`NormalizeTargetForm` (версионированный артефакт `NormVersion()`, фолдится в снапшот), `NormalizeSource`, `TokenizeScript` (алфавит цели из данных, Mn-фолд U+0301; `TokenizeCyrillic` = ru-дефолт-обёртка), `DenseScript` — единый дом sizing-таксономии (D39.64) | `norm.go`, `runes.go`, `source.go` |
| `internal/seed` | Схема сид-YAML книги (`File`/`Term`/`Decl`/`Alias`) — общий артефакт: майнер её ПИШЕТ (mined-дельта), банк ЧИТАЕТ (`LoadGlossarySeed`); тип общий, поэтому эмиссия не может разъехаться с загрузчиком | `seed.go` |
@ -65,6 +65,7 @@ go run ./cmd/tmctl status --config example/book.yaml --json # $0, N/M+пасп
go run ./cmd/tmctl report --config example/book.yaml # $0, quality-report (KPI/rates)
go run ./cmd/tmctl export --config example/book.yaml # $0, экспорт-JSON для полигона (--plaintext для человека)
go run ./cmd/tmctl manifest --config example/book.yaml # $0, пере-строить персист манифеста глав/чанков (--json — сам документ)
go run ./cmd/tmctl migrate --config example/book.yaml # $0, довести схему проекта до головы бинаря (деплой-шаг: стоп прогонов → НОВЫЙ бинарь на место → migrate ИМ по каждой книге без открытых попыток → прогоны в работу)
go run ./cmd/tmctl translate --config example/book.yaml --ceiling-usd 0.5 # потолок ТОЛЬКО на этот прогон, book.yaml не пишется
# live-conformance (реальные провайдеры, платно, вне CI):
set -a; . ./.env; set +a; TM_LIVE=1 go test -tags live -run TestLive -v ./internal/pipeline/

View file

@ -1,4 +1,5 @@
// tmctl is the TextMachine CLI: translate / report / status / export / redrive / manifest / backup.
// tmctl is the TextMachine CLI: translate / report / status / export / redrive / manifest / backup /
// migrate.
// main.go — thin wiring (package №4): argument parsing — invocation.go,
// output renderers — render.go, .env — dotenv.go; here just the
// «parse → env → ctx → fetch → render» wiring and exit-code mapping.
@ -45,6 +46,7 @@ const (
exitConfigInvalid = 10 // the configuration will not run: unreadable, unparseable, invalid, no key
exitSourceUnreadable = 11 // the BOOK's source cannot be read or decoded — the one class about the text
exitProjectLocked = 12 // another tmctl owns this project right now; come back later
exitSchemaMismatch = 13 // this project's schema is not this binary's: run `tmctl migrate` (row 174)
exitRefusedOther = 19 // a refusal class this build of tmctl has no number for
)
@ -53,6 +55,7 @@ var refusalExit = map[pipeline.RefusalClass]int{
pipeline.RefusalBadConfig: exitConfigInvalid,
pipeline.RefusalSourceUnreadable: exitSourceUnreadable,
pipeline.RefusalProjectLocked: exitProjectLocked,
pipeline.RefusalSchemaMismatch: exitSchemaMismatch,
}
// exitCode maps a run() error onto the ratified shell contract (Milestone 2 / R1-FL-A):
@ -183,10 +186,12 @@ func run() error {
return manifestCmd(inv.cfgPath, inv.asJSON)
case "backup":
return backupCmd(inv.cfgPath, os.Stdout)
case "migrate":
return migrateCmd(inv.cfgPath, os.Stdout)
case "seed-lint":
return seedLint(inv.seedPath)
default:
return fmt.Errorf("unknown command %q (want translate|report|status|export|redrive|manifest|backup|seed-lint)", inv.cmd)
return fmt.Errorf("unknown command %q (want translate|report|status|export|redrive|manifest|backup|migrate|seed-lint)", inv.cmd)
}
}

View file

@ -0,0 +1,108 @@
package main
// migrate.go: the $0 `tmctl migrate` verb — a project's WRITE open with no run behind it (backlog
// row 174).
//
// The deadlock it breaks: the read-only commands (status, report, export, manifest) demand this
// binary's exact schema and never migrate — only a write open applies the chain. So upgrading the
// engine binary locks every existing book at once: the platform calls `tmctl status --json` before each
// spawn and for the money, that call refuses the older schema, and the write command that WOULD have
// migrated the project never comes. The deploy step is "stop the runs → put the NEW binary in place →
// run THIS command over the books with it → let the runs start again", and a caller that meets exit 13
// in flight can also self-heal from it.
//
// ⚠ WHICH books: the sweep belongs on projects with no OPEN attempt. A migrated project is above the
// schema of every older binary, and the platform settles a finished attempt with the binary that
// attempt was PINNED to — so migrating underneath an attempt whose settle has not happened yet would
// trade this deadlock for a hold that can never be closed. The engine has no notion of an attempt, so
// that rule lives in the caller's sweep (row 175); what the engine guarantees is that a project a live
// run holds is refused rather than migrated (exit 12).
//
// It deliberately loads the BOOK config only. A migration touches no prompt, no provider and no price,
// and pulling in the rest of the stack would drag its gates onto a $0 path — `prices_checked` refuses a
// models.yaml whose prices are over 120 days old (backlog row 146), and an engine deployed onto a stand
// with stale prices would then meet the deadlock again in exactly the shape this command exists to
// remove. Same reason the command needs no provider key: it is an operator safety verb, like `backup`.
import (
"errors"
"fmt"
"io"
"io/fs"
"os"
"time"
"textmachine/backend/internal/config"
"textmachine/backend/internal/pipeline"
"textmachine/backend/internal/store"
)
// migrateSuffix names what the restore point below is FOR, and it is what keeps this command out of the
// paid path's namespace. `translate`'s pre-flight guard writes backups/<stamp>.db with a stamp of
// second precision, and BackupSQLite refuses to overwrite an existing one (backlog row 173) — so
// sharing the name would make the very sequence this command enables (migrate, then start the run it
// unblocked) fail inside the pre-flight guard whenever both land in the same second, turning a healthy
// deploy into exit 1 before anything reached a provider.
const migrateSuffix = "-pre-migrate"
// migrateCmd brings the book's project database to this binary's schema and reports the transition.
// $0: no LLM call, no network, no ledger movement — the only writes are the migration steps themselves
// and the stale-reservation recovery every write open performs.
//
// Idempotent: a project already at head is opened, found current and closed — no step applied, no
// restore point taken, no money touched.
func migrateCmd(cfgPath string, w io.Writer) error {
book, err := config.LoadBook(cfgPath)
if err != nil {
// Like the pre-flight backup guard, this runs before any runner exists, so it is the first thing
// a broken config meets — and a refusal classified here rather than collapsed onto exit 1 is the
// whole of PD-196 (see pipeline/refusal.go).
return pipeline.RefuseConfig(err)
}
// Only ABSENT means "never run": a database this call is about to CREATE has no history to protect,
// and it is the one case that gets no restore point. Any other stat error read that way would skip
// the guard on a file that is there.
dbExists := true
if _, err := os.Stat(book.ProjectDB); err != nil {
if !errors.Is(err, fs.ErrNotExist) {
return fmt.Errorf("tmctl migrate: cannot stat the project database %s: %w", book.ProjectDB, err)
}
dbExists = false
}
// The restore point is taken INSIDE the project lock, and only when a step is actually going to run
// (store.Migrate's seam). A migration is the one $0 operation that rewrites the file holding the
// owner's signed bank, the checkpoints and the ledger, and the ALTER steps do not converge on a
// half-applied database (row 49а) — a damaged file is not something a re-run repairs. Taking it
// before the lock instead would copy the whole database of a project a live run owns and then refuse
// the migration anyway, once per retry.
backup := func(m store.Migration) error {
if !dbExists {
return nil
}
path, err := store.BackupSQLite(book.ProjectDB,
backupDirFor(book.ProjectDB), backupStamp(time.Now())+migrateSuffix)
if err != nil {
return fmt.Errorf("tmctl migrate: refusing to migrate a project it could not back up first: %w", err)
}
fmt.Fprintf(w, "pre-migrate: backed up the project DB at schema v%d to %s (integrity_check green)\n", m.From, path)
return nil
}
applied, err := store.Migrate(book.ProjectDB, backup)
if err != nil {
// The two classified open failures a caller acts on: another tmctl holds the project (come back
// later, exit 12) and a database NEWER than this binary (upgrade tmctl, exit 13 — a migration
// cannot repair that direction, so a caller looping on "migrate and retry" has to be told).
return pipeline.RefuseStoreOpen(err)
}
switch {
case !dbExists:
fmt.Fprintf(w, "migrate: created %s at schema v%d\n", book.ProjectDB, applied.To)
case applied.From == applied.To:
fmt.Fprintf(w, "migrate: %s already at schema v%d\n", book.ProjectDB, applied.To)
default:
fmt.Fprintf(w, "migrate: %s schema v%d -> v%d\n", book.ProjectDB, applied.From, applied.To)
}
return nil
}

View file

@ -0,0 +1,398 @@
package main
import (
"bytes"
"context"
"database/sql"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
"strings"
"syscall"
"testing"
"time"
"textmachine/backend/internal/pipeline"
"textmachine/backend/internal/store"
)
// migrate_cli_test.go is the CLI half of backlog row 174: the deploy deadlock is a SHELL fact — the
// platform learns what happened from an exit code and a line of stderr — so the contract is exercised
// where it is a contract. The migration mechanics themselves are pinned generically in
// internal/store/migrate_test.go.
// staleTheProjectSchema rolls the project database's recorded version back one step, which is what an
// older engine binary left behind.
//
// ⚠ This fixture re-applies the newest migration on the next write open, which is fine while that step
// is `CREATE … IF NOT EXISTS` and is NOT guaranteed for a future ALTER step (backlog row 49а). It is
// used here because these tests are about the exit codes and the output, and the deep fixture — the
// migration chain truncated to an older vintage — is only reachable from inside the store package,
// where those properties are tested. If a future migration makes this fail, move the setup rather than
// weakening the assertions.
func staleTheProjectSchema(t *testing.T, dbPath string) int {
t.Helper()
db, err := sql.Open("sqlite", "file:"+dbPath)
if err != nil {
t.Fatal(err)
}
defer db.Close()
if _, err := db.Exec(`DELETE FROM schema_version WHERE version = (SELECT MAX(version) FROM schema_version)`); err != nil {
t.Fatal(err)
}
return schemaVersionOf(t, dbPath)
}
// schemaVersionOf reads the recorded schema version through its own connection — the state under test
// must be asserted independently of the product code that reports it.
func schemaVersionOf(t *testing.T, dbPath string) int {
t.Helper()
db, err := sql.Open("sqlite", "file:"+dbPath)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var v int
if err := db.QueryRow(`SELECT COALESCE(MAX(version), 0) FROM schema_version`).Scan(&v); err != nil {
t.Fatal(err)
}
return v
}
// holdTheProject takes the project's exclusive flock the way another tmctl process would, WITHOUT
// opening the store — because a write open would migrate the database, and the state under test is a
// project that is both stale and held.
func holdTheProject(t *testing.T, dbPath string) {
t.Helper()
f, err := os.OpenFile(dbPath+".lock", os.O_CREATE|os.O_RDWR, 0o644)
if err != nil {
t.Fatal(err)
}
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
_ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
f.Close()
})
}
func projectDBOf(bookPath string) string {
return filepath.Join(filepath.Dir(bookPath), "cli-book.db")
}
func backupsOf(t *testing.T, dbPath string) []string {
t.Helper()
entries, err := os.ReadDir(backupDirFor(dbPath))
if os.IsNotExist(err) {
return nil
}
if err != nil {
t.Fatal(err)
}
var names []string
for _, e := range entries {
names = append(names, e.Name())
}
return names
}
// mockProvider answers every call with a usable draft, so a `translate` really settles money.
func mockProvider(t *testing.T) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.ReadAll(r.Body)
fmt.Fprint(w, `{"id":"fake","model":"fake-model","choices":[{"message":{"content":"ЧЕРНОВИК ПЕРЕВОДА"},"finish_reason":"stop"}],
"usage":{"prompt_tokens":1000,"completion_tokens":500}}`)
}))
t.Cleanup(srv.Close)
return srv
}
// TestTheDeployDeadlockIsBrokenAtTheShell drives the real binary through the exact sequence the
// platform meets on an engine upgrade: `status --json` before a spawn refuses the older schema, and the
// contract must make that refusal actionable — its own number inside the refusal band, and the two
// versions machine-readable — so the caller can migrate and start the run instead of stopping the world.
func TestTheDeployDeadlockIsBrokenAtTheShell(t *testing.T) {
if testing.Short() {
t.Skip("builds and runs the binary")
}
bin := buildTmctl(t)
bookPath := setupCLIProject(t, mockProvider(t).URL)
dbPath := projectDBOf(bookPath)
// A book an older binary left behind.
st, err := store.Open(dbPath)
if err != nil {
t.Fatal(err)
}
st.Close()
from := staleTheProjectSchema(t, dbPath)
var stderr bytes.Buffer
status := exec.Command(bin, "status", "--config", bookPath, "--json")
status.Stderr = &stderr
if got := exitCodeOf(t, status.Run()); got != exitSchemaMismatch {
t.Fatalf("`status --json` on an older schema exited %d, want %d", got, exitSchemaMismatch)
}
// The number says WHAT; these two say which way and how far — without them a caller cannot tell the
// direction a migration repairs from the one where it would loop forever.
want := fmt.Sprintf("schema_mismatch found=%d expected=%d", from, store.SchemaHead())
if !strings.Contains(stderr.String(), want) {
t.Fatalf("the refusal must carry %q machine-readably; got:\n%s", want, stderr.String())
}
if exitSchemaMismatch < refusalFirst || exitSchemaMismatch > refusalLast {
t.Fatalf("%d is outside the refusal band [%d,%d] every consumer keys on", exitSchemaMismatch, refusalFirst, refusalLast)
}
out, err := exec.Command(bin, "migrate", "--config", bookPath).Output()
if got := exitCodeOf(t, err); got != 0 {
t.Fatalf("`migrate` exited %d: %s", got, out)
}
if want := fmt.Sprintf("schema v%d -> v%d", from, store.SchemaHead()); !strings.Contains(string(out), want) {
t.Fatalf("migrate must report the transition %q; got:\n%s", want, out)
}
if got := exitCodeOf(t, exec.Command(bin, "status", "--config", bookPath, "--json").Run()); got != 0 {
t.Fatalf("`status --json` after the migration exited %d, want 0 — the deadlock is not broken", got)
}
// …and the run the migration unblocked actually starts. It follows within milliseconds, which is
// what put migrate's restore point and the paid path's pre-flight guard in the same second.
if run := exec.Command(bin, "translate", "--config", bookPath); exitCodeOf(t, run.Run()) != 0 {
t.Fatal("the run spawned right after the migration must start, not die in its pre-flight guard")
}
}
// TestTheMigrationRestorePointNeverCollidesWithThePaidPath is the deterministic half of the same fact:
// both producers are handed the SAME stamp, because in the deploy sequence they really do land in the
// same second. `store.BackupSQLite` refuses to overwrite a restore point (backlog row 173), so sharing
// the name would make the run refuse to start with exit 1 — outside the refusal band, and after a
// migration that already succeeded.
func TestTheMigrationRestorePointNeverCollidesWithThePaidPath(t *testing.T) {
bookPath := setupCLIProject(t, "http://127.0.0.1:1")
dbPath := projectDBOf(bookPath)
st, err := store.Open(dbPath)
if err != nil {
t.Fatal(err)
}
st.Close()
staleTheProjectSchema(t, dbPath)
// PRODUCTION takes the restore point — the point of the test. Supplying the name here instead would
// only pin that the constant is non-empty, and the regression to catch is the call site dropping it.
if err := migrateCmd(bookPath, io.Discard); err != nil {
t.Fatal(err)
}
names := backupsOf(t, dbPath)
if len(names) != 1 {
t.Fatalf("restore points after the migration: %v, want exactly one", names)
}
// Strip the tag production chose and the bare second-stamp must still be free, so translate's
// pre-flight guard can take its own backup in the SAME second. Untagged, the two names are equal and
// BackupSQLite refuses to overwrite — the run the migration just unblocked would die in its
// pre-flight guard with exit 1. No wall clock is involved: the collision is decided by the name.
tagged := strings.TrimSuffix(names[0], ".db")
bare := strings.TrimSuffix(tagged, migrateSuffix)
if bare == tagged {
t.Fatalf("the migration's restore point %q carries no tag of its own — it is inside the paid path's namespace", names[0])
}
if _, err := store.BackupSQLite(dbPath, backupDirFor(dbPath), bare); err != nil {
t.Fatalf("the paid path's pre-flight backup in the SAME second must still succeed: %v", err)
}
}
// TestMigrateNeedsNeitherPricesNorModels: the $0 path must not carry the paid path's gates. A
// models.yaml whose prices are older than 120 days refuses to load (backlog row 146), and if the
// migration were built on the full config stack a stand with stale prices would meet the same deadlock
// this command exists to remove — the read command refuses, and the one thing that could migrate
// refuses too.
func TestMigrateNeedsNeitherPricesNorModels(t *testing.T) {
bookPath := setupCLIProject(t, mockProvider(t).URL)
modelsPath := filepath.Join(filepath.Dir(bookPath), "models.yaml")
raw, err := os.ReadFile(modelsPath)
if err != nil {
t.Fatal(err)
}
stale := time.Now().UTC().AddDate(0, 0, -200).Format("2006-01-02")
writeCLIFile(t, modelsPath, strings.Replace(string(raw), time.Now().UTC().Format("2006-01-02"), stale, 1))
// The contrast: the read-only projection is refused by the price gate before it reads a schema.
if got := exitCode(status(context.Background(), bookPath, true)); got != exitConfigInvalid {
t.Fatalf("setup: `status` on stale prices exited %d, want %d", got, exitConfigInvalid)
}
var out bytes.Buffer
if err := migrateCmd(bookPath, &out); err != nil {
t.Fatalf("migrate must not consult prices: %v", err)
}
if !strings.Contains(out.String(), fmt.Sprintf("v%d", store.SchemaHead())) {
t.Fatalf("migrate must report the head it brought the project to; got %q", out.String())
}
// The structural version of the same claim, which also covers the provider keys the models file
// carries: with no models.yaml at all the migration still runs, so it cannot be consulting either.
if err := os.Remove(modelsPath); err != nil {
t.Fatal(err)
}
if got := exitCode(status(context.Background(), bookPath, true)); got != exitConfigInvalid {
t.Fatalf("setup: `status` without models.yaml exited %d, want %d", got, exitConfigInvalid)
}
if err := migrateCmd(bookPath, io.Discard); err != nil {
t.Fatalf("migrate must not load the model stack at all: %v", err)
}
}
// TestMigrateKeepsTheMoneyAndRepeatsAsANoOp is the money half at the command level: a real run's
// settled spend, a reservation its process never settled, and a migration between them. `committed` is
// what the platform's ceiling argument is computed from (PD-158) and what a settle reconciles against,
// so it must survive the deploy step untouched; `reserved` is advisory and the write open's recovery
// pass is entitled to zero it.
func TestMigrateKeepsTheMoneyAndRepeatsAsANoOp(t *testing.T) {
bookPath := setupCLIProject(t, mockProvider(t).URL)
dbPath := projectDBOf(bookPath)
if err := translate(context.Background(), bookPath, false, pipeline.RebillConsent{}, false, 0); err != nil {
t.Fatalf("setup run: %v", err)
}
st, err := store.Open(dbPath)
if err != nil {
t.Fatal(err)
}
committed, _, err := st.SpentUSD("cli-book")
if err != nil {
t.Fatal(err)
}
if committed <= 0 {
t.Fatalf("setup: the run must have settled money, committed=%v", committed)
}
// The reservation a killed run leaves behind.
if _, verdict, err := st.Reserve("cli-book", 0.25, store.Ceilings{BookUSD: 10, DayUSD: 10}); err != nil || verdict != store.ReserveOK {
t.Fatalf("stale reserve: %v %v", verdict, err)
}
st.Close()
from := staleTheProjectSchema(t, dbPath)
for i := range 2 {
var out bytes.Buffer
if err := migrateCmd(bookPath, &out); err != nil {
t.Fatalf("call %d: %v", i, err)
}
ro, err := store.OpenReadOnly(dbPath)
if err != nil {
t.Fatalf("call %d: the read path must work after migrating: %v", i, err)
}
gotCommitted, gotReserved, err := ro.SpentUSD("cli-book")
ro.Close()
if err != nil {
t.Fatal(err)
}
if gotCommitted != committed {
t.Fatalf("call %d: committed %v -> %v; a migration must not move settled money by a cent", i, committed, gotCommitted)
}
if gotReserved != 0 {
t.Fatalf("call %d: reserved=%v, want the recovery pass to have zeroed it", i, gotReserved)
}
// The restore point is taken for the call that MIGRATES and for no other, and it holds the
// database as it was BEFORE the step — a copy taken afterwards is not a restore point at all.
names := backupsOf(t, dbPath)
if len(names) != 1 {
t.Fatalf("call %d: restore points %v, want exactly the one the migrating call took", i, names)
}
if v := schemaVersionOf(t, filepath.Join(backupDirFor(dbPath), names[0])); v != from {
t.Fatalf("call %d: the restore point is at schema v%d, want the pre-migration v%d", i, v, from)
}
if i == 1 && !strings.Contains(out.String(), "already at schema") {
t.Fatalf("the repeat call must report a no-op; got %q", out.String())
}
}
}
func TestMigrateRefusesWithTheSameVocabularyAsEveryOtherCommand(t *testing.T) {
t.Run("a broken config", func(t *testing.T) {
bookPath := setupCLIProject(t, "http://127.0.0.1:1")
writeCLIFile(t, bookPath, "book_id: cli-book\n bad indentation: [\n")
if got := exitCode(migrateCmd(bookPath, io.Discard)); got != exitConfigInvalid {
t.Fatalf("exit %d, want %d", got, exitConfigInvalid)
}
})
t.Run("a project another process holds", func(t *testing.T) {
// The deploy step runs while the operator believes the runs are stopped. If one is not, the
// answer is "come back later" — never a migration under a live writer's feet, whose reservations
// the recovery pass would zero. And nothing may be written for that refusal: a restore point
// taken before the lock would copy the whole database once per retry.
bookPath := setupCLIProject(t, "http://127.0.0.1:1")
dbPath := projectDBOf(bookPath)
st, err := store.Open(dbPath)
if err != nil {
t.Fatal(err)
}
st.Close()
staleTheProjectSchema(t, dbPath)
holdTheProject(t, dbPath)
if got := exitCode(migrateCmd(bookPath, io.Discard)); got != exitProjectLocked {
t.Fatalf("exit %d, want %d", got, exitProjectLocked)
}
if got := backupsOf(t, dbPath); len(got) != 0 {
t.Fatalf("a refused migration wrote %v; nothing may be written for a project it does not own", got)
}
// The retry the refusal invites reaches the lock too, instead of dying on the leftovers of the
// previous attempt.
if got := exitCode(migrateCmd(bookPath, io.Discard)); got != exitProjectLocked {
t.Fatalf("retry exited %d, want %d again", got, exitProjectLocked)
}
})
t.Run("a project newer than this binary", func(t *testing.T) {
// The direction a migration cannot repair. It must not read as success, or a caller looping
// "status refused → migrate → retry" would loop forever on a rolled-back binary.
bookPath := setupCLIProject(t, "http://127.0.0.1:1")
dbPath := projectDBOf(bookPath)
st, err := store.Open(dbPath)
if err != nil {
t.Fatal(err)
}
st.Close()
db, err := sql.Open("sqlite", "file:"+dbPath)
if err != nil {
t.Fatal(err)
}
if _, err := db.Exec(`INSERT INTO schema_version (version) VALUES (?)`, store.SchemaHead()+7); err != nil {
t.Fatal(err)
}
db.Close()
err = migrateCmd(bookPath, io.Discard)
if got := exitCode(err); got != exitSchemaMismatch {
t.Fatalf("exit %d, want %d", got, exitSchemaMismatch)
}
if !strings.Contains(err.Error(), fmt.Sprintf("schema_mismatch found=%d expected=%d", store.SchemaHead()+7, store.SchemaHead())) {
t.Fatalf("the refusal must name both versions machine-readably: %v", err)
}
if got := backupsOf(t, dbPath); len(got) != 0 {
t.Fatalf("a refused migration wrote %v", got)
}
})
t.Run("a book that has never been run", func(t *testing.T) {
// The deploy sweep does not have to know which books have a database yet. Nothing is backed up:
// a database this call creates has no history to lose.
bookPath := setupCLIProject(t, "http://127.0.0.1:1")
var out bytes.Buffer
if err := migrateCmd(bookPath, &out); err != nil {
t.Fatal(err)
}
if !strings.Contains(out.String(), "created") {
t.Fatalf("migrate must say it created the project; got %q", out.String())
}
if v := schemaVersionOf(t, projectDBOf(bookPath)); v != store.SchemaHead() {
t.Fatalf("created at v%d, want v%d", v, store.SchemaHead())
}
if got := backupsOf(t, projectDBOf(bookPath)); len(got) != 0 {
t.Fatalf("a project being created was backed up: %v", got)
}
})
}

View file

@ -1,5 +1,11 @@
package pipeline
import (
"errors"
"textmachine/backend/internal/store"
)
// refusal.go: the shell contract's REFUSAL classes (row 165 / PD-196 of the platform).
//
// The defect this closes was not cosmetic. `tmctl` mapped every failure it did not recognise onto exit
@ -27,6 +33,17 @@ const (
// RefusalProjectLocked is another tmctl process owning the project. Nothing is wrong with anything —
// the answer is to come back later.
RefusalProjectLocked RefusalClass = "project_locked"
// RefusalSchemaMismatch is a project database whose schema version is not this binary's. Nothing is
// wrong with the book either: the answer is `tmctl migrate` (or, in the other direction, a newer
// binary), and the two numbers a caller needs to tell those apart ride the message as the stable
// token `schema_mismatch found=<N> expected=<M>` (store.SchemaMismatchError).
//
// It is its own class because read-only opens never migrate, so an engine upgrade refuses every
// existing book until a write command touches it — and the platform's own pre-spawn call is one of
// those read-only opens, so that write command never came (the deploy deadlock, backlog row 174).
// A caller that can SEE this class self-heals ("caught it → migrate → retry") where it used to stop
// the world over what looked like a broken project.
RefusalSchemaMismatch RefusalClass = "schema_mismatch"
)
// Refusal is an invocation the engine turned down before doing any work of its own: nothing reached a
@ -57,6 +74,27 @@ func RefuseConfig(err error) error {
return refuse(RefusalBadConfig, err)
}
// RefuseStoreOpen classifies a store-open failure. It is exported and it is the ONE place the mapping
// lives, because two callers open a project store: openRunner below, and `tmctl migrate`, which opens
// one with no runner at all. A second copy of this switch is how "another process holds it" and "this
// binary does not match the schema" drift back into a single exit 1 on one of the two paths.
//
// An unrecognised failure is returned UNCHANGED rather than swept into the band: the band means "turned
// down before doing any work", and a caller destroys or waits on data because of it.
func RefuseStoreOpen(err error) error {
var mismatch *store.SchemaMismatchError
switch {
case err == nil:
return nil
case errors.Is(err, store.ErrLocked):
return refuse(RefusalProjectLocked, err)
case errors.As(err, &mismatch):
return refuse(RefusalSchemaMismatch, err)
default:
return err
}
}
// refuseSource classifies a failure to obtain the book's text, and it classifies almost all of them as a
// CONFIG fault. That is deliberate, and it is the most consequential decision in this file.
//

View file

@ -0,0 +1,58 @@
package pipeline
import (
"errors"
"fmt"
"testing"
"textmachine/backend/internal/store"
)
// refusal_test.go pins RefuseStoreOpen, the one place a store-open failure becomes a shell class. Two
// callers share it (openRunner and `tmctl migrate`), and its arms decide what an automated caller DOES:
// wait, migrate, or treat the project as broken.
func TestRefuseStoreOpenClassifiesWhatACallerActsOn(t *testing.T) {
locked := fmt.Errorf("opening the project: %w", store.ErrLocked)
mismatch := fmt.Errorf("opening the project: %w", &store.SchemaMismatchError{Path: "/b.db", Found: 14, Expected: 15})
for _, tc := range []struct {
name string
err error
want RefusalClass
}{
{"another process holds it", locked, RefusalProjectLocked},
{"the schema is not this binary's", mismatch, RefusalSchemaMismatch},
} {
t.Run(tc.name, func(t *testing.T) {
var refusal *Refusal
if !errors.As(RefuseStoreOpen(tc.err), &refusal) {
t.Fatalf("%v was not classified as a refusal", tc.err)
}
if refusal.Class != tc.want {
t.Fatalf("class %q, want %q", refusal.Class, tc.want)
}
// Both arms must survive a %w-wrap, because that is how they arrive: neither caller opens the
// store at the top of its own stack.
if !errors.Is(refusal, store.ErrLocked) && !errors.As(refusal, new(*store.SchemaMismatchError)) {
t.Fatal("the classification dropped the cause it was made from")
}
})
}
// The safety rule, and the reason the default arm is a passthrough: the refusal band means "turned
// down before doing ANY work", and the platform's intake acts on it — it waits instead of failing,
// and one number in it deletes the user's upload. A disk error during an open is not that; it must
// stay exit 1 (infra failure) rather than borrow a class that promises nothing happened.
disk := errors.New("store: open write pool: input/output error")
got := RefuseStoreOpen(disk)
if !errors.Is(got, disk) {
t.Fatalf("an unrecognised open failure must come back with its cause intact, got %v", got)
}
if errors.As(got, new(*Refusal)) {
t.Fatal("an unrecognised open failure was swept into the refusal band")
}
if RefuseStoreOpen(nil) != nil {
t.Fatal("a nil failure must stay nil: the classifier never invents one")
}
}

View file

@ -262,10 +262,7 @@ func openRunner(bookPath string, logger *slog.Logger, forWrite bool) (*Runner, e
}
}
if err != nil {
if errors.Is(err, store.ErrLocked) {
return nil, refuse(RefusalProjectLocked, err)
}
return nil, err
return nil, RefuseStoreOpen(err)
}
r := &Runner{
Book: book,

View file

@ -2,6 +2,7 @@ package store
import (
"context"
"database/sql"
"fmt"
)
@ -463,17 +464,155 @@ var migrations = []string{
// which is membank.SpoilerLeaks over the selection's Rejected set, no schema at all. Scope amended
// by D39.55 (form, not intent).
// migrate runs all pending migrations on the write pool, one transaction per
// step, recording each in schema_version.
func (s *Store) migrate(ctx context.Context) error {
if _, err := s.w.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS schema_version (version INTEGER PRIMARY KEY)`); err != nil {
return fmt.Errorf("store: schema_version: %w", err)
// SchemaHead is the schema version this binary migrates a project database to. It is the version every
// read-only open demands (OpenReadOnly) and the version `tmctl migrate` brings a project to.
func SchemaHead() int { return len(migrations) }
// SchemaMismatchError is a project database whose recorded schema version is not this binary's. It is a
// TYPE rather than a message because it is the one open failure a caller can repair by itself, and it
// could not act on the difference while every failure arrived as the same text and the same exit 1.
//
// The engine's read-only opens never migrate — only a write open does — so upgrading the binary leaves
// every existing book unreadable until something writes to it. The platform calls `tmctl status --json`
// before each spawn and for the money, that call refuses the older schema, and the write command that
// would have migrated the project never comes: the deploy deadlock of backlog row 174.
//
// The numbers therefore travel BOTH in the type (for a Go caller) and in the message (for the shell),
// where they carry the stable token
//
// schema_mismatch found=<N> expected=<M>
//
// Found < Expected is the direction `tmctl migrate` repairs — "caught it, migrated, retried" instead of
// stopping the world. Found > Expected is a binary OLDER than the project, which no command here can fix
// (the operator upgrades tmctl), so a caller that blindly re-ran the migration would loop forever.
type SchemaMismatchError struct {
Path string
Found int
Expected int
}
func (e *SchemaMismatchError) Error() string {
repair := "the database is NEWER than this tmctl — update the binary (an old build must neither read nor write a newer schema)"
if e.Stale() {
repair = "the database is older than this tmctl — run `tmctl migrate --config <book.yaml>` (read-only opens never migrate)"
}
return fmt.Sprintf("store: schema_mismatch found=%d expected=%d path=%s — %s", e.Found, e.Expected, e.Path, repair)
}
// Stale reports whether the database is BEHIND the binary — the one direction a migration repairs.
func (e *SchemaMismatchError) Stale() bool { return e.Found < e.Expected }
// Migration is the schema transition a write open performed: From is the version the project database
// was recorded at, To the version it is at now. From == To is a database that was already at head.
type Migration struct {
From int
To int
}
// Migrate applies a project database's pending migrations and reports the transition — the WRITE open
// (exclusive flock · migration chain · stale-reservation recovery) with no run behind it, which is all
// `tmctl migrate` is (backlog row 174). It sends nothing, spends nothing and holds the flock only for
// as long as the migration takes.
//
// A database already at head is a no-op; one that does not exist is CREATED at head, exactly as the
// first touch of a project by any other write command creates it. A database a NEWER binary wrote is
// refused (SchemaMismatchError) before anything of this process reaches the file.
//
// beforeApply, when non-nil, runs INSIDE the project's exclusive lock, after the pending set is known
// and before the first step is applied — and only when there IS a step to apply. That is where a caller
// takes its restore point, and the placement is the whole point of the parameter: a backup taken before
// the lock is a full copy written for a migration that a live run may then refuse (litter on every
// retry of an exit-12), and one taken after the steps is not a restore point at all. An error from it
// aborts the migration with nothing applied.
func Migrate(path string, beforeApply func(Migration) error) (Migration, error) {
s, err := open(path, beforeApply)
if err != nil {
return Migration{}, err
}
applied := s.applied
if err := s.Close(); err != nil {
return applied, fmt.Errorf("store: close %s after migrating: %w", path, err)
}
return applied, nil
}
// schemaVersion reads the recorded schema version through db.
//
// A database that exists but has no schema_version TABLE — a fresh file, or a process killed between
// creating one and committing the first migration — is version 0, which is what it is: nothing has been
// applied. That is asked as a separate question rather than read off a failed SELECT, so a file that is
// not a database at all still fails loudly instead of reading as an empty project.
func schemaVersion(ctx context.Context, db *sql.DB) (int, error) {
var recorded int
if err := db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'schema_version'`).Scan(&recorded); err != nil {
return 0, err
}
if recorded == 0 {
return 0, nil
}
var current int
if err := s.w.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM schema_version`).Scan(&current); err != nil {
return fmt.Errorf("store: read version: %w", err)
if err := db.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM schema_version`).Scan(&current); err != nil {
return 0, err
}
return current, nil
}
// migrate runs all pending migrations on the write pool, one transaction per
// step, recording each in schema_version.
//
// Every phase gets its own opTimeout budget, and the caller's seam gets NONE. That is not tidiness: the
// seam exists to take a restore point, whose cost scales with the book, and while it shared one deadline
// with the steps a large project made the migration fail on the copy it had just paid for (see open).
func (s *Store) migrate(path string, beforeApply func(Migration) error) error {
current, err := s.schemaBaseline()
if err != nil {
return err
}
// A project a NEWER binary wrote is REFUSED rather than opened. The loop below would silently do
// nothing with it — there is no step left to apply — and the caller would then read and, worse,
// WRITE a schema through code that predates it. OpenReadOnly has always refused that direction
// ("reading a newer schema with old code is unsafe"); the write path used to accept it, which is
// the more dangerous half of the same fact. It is refused here, BEFORE recoverReservations, so a
// refused open leaves the file exactly as it found it.
if current > len(migrations) {
return &SchemaMismatchError{Path: path, Found: current, Expected: len(migrations)}
}
// The caller's seam (Migrate): the lock is held, the pending set is known, nothing has been applied,
// and no deadline of this package is running — the work is the caller's and so is its bounding.
if beforeApply != nil && current < len(migrations) {
if err := beforeApply(Migration{From: current, To: len(migrations)}); err != nil {
return err
}
}
for v := current; v < len(migrations); v++ {
if err := s.applyStep(v); err != nil {
return err
}
}
s.applied = Migration{From: current, To: len(migrations)}
return nil
}
// schemaBaseline makes sure the version table exists and reports what is recorded in it.
func (s *Store) schemaBaseline() (int, error) {
ctx, cancel := opContext()
defer cancel()
if _, err := s.w.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS schema_version (version INTEGER PRIMARY KEY)`); err != nil {
return 0, fmt.Errorf("store: schema_version: %w", err)
}
current, err := schemaVersion(ctx, s.w)
if err != nil {
return 0, fmt.Errorf("store: read version: %w", err)
}
return current, nil
}
// applyStep applies migrations[v] and records v+1 in ONE transaction (DDL is transactional in SQLite),
// so a process that dies mid-step leaves the database at its previous version rather than half-applied.
func (s *Store) applyStep(v int) error {
ctx, cancel := opContext()
defer cancel()
tx, err := s.w.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("store: begin migration %d: %w", v+1, err)
@ -489,6 +628,5 @@ func (s *Store) migrate(ctx context.Context) error {
if err := tx.Commit(); err != nil {
return fmt.Errorf("store: commit migration %d: %w", v+1, err)
}
}
return nil
}

View file

@ -0,0 +1,450 @@
package store
import (
"database/sql"
"errors"
"os"
"path/filepath"
"testing"
"time"
)
// migrate_test.go pins the mechanics `tmctl migrate` stands on (backlog row 174): a database an OLDER
// binary wrote converges to head and unlocks the read-only path, the MONEY it holds survives the
// conversion untouched, a database a NEWER binary wrote is refused before anything is written, and a
// step that fails leaves nothing of itself behind.
// atVersion runs fn with the migration chain TRUNCATED to v steps, so anything opened inside it sees
// exactly the database a binary of that vintage would have written and worked with.
//
// The fixture truncates the chain instead of deleting rows from schema_version because those are not the
// same thing: rolling the counter back leaves HEAD's tables in place and makes the next open re-apply
// the newest step, which backlog row 49а says is not idempotent for an ALTER. Truncation also keeps the
// fixture correct when a new migration lands — it never names a version or a table.
//
// It mutates a package-level var, which is safe because these tests do not run in parallel (nothing in
// this package calls t.Parallel).
func atVersion(t *testing.T, v int, fn func()) {
t.Helper()
full := migrations
migrations = migrations[:v]
defer func() { migrations = full }()
fn()
}
// openAtVersion creates a project database exactly as a binary with v migrations would have left it.
func openAtVersion(t *testing.T, path string, v int) {
t.Helper()
atVersion(t, v, func() {
s, err := Open(path)
if err != nil {
t.Fatal(err)
}
if err := s.Close(); err != nil {
t.Fatal(err)
}
})
}
// rawVersion reads MAX(schema_version) through its own connection, deliberately NOT through the
// product's own reader: the state a migration left behind must be asserted independently of the code
// that reports it, and this way it can also be read from a database the product is currently refusing.
func rawVersion(t *testing.T, path string) int {
t.Helper()
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var v int
if err := db.QueryRow(`SELECT COALESCE(MAX(version), 0) FROM schema_version`).Scan(&v); err != nil {
t.Fatal(err)
}
return v
}
func TestMigrateBringsAnOlderProjectToHeadAndUnlocksTheReadPath(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
openAtVersion(t, path, head-1)
// The deadlock itself: the read-only projections the platform calls before every spawn refuse a
// database an older binary wrote, and they never migrate it.
_, err := OpenReadOnly(path)
var mismatch *SchemaMismatchError
if !errors.As(err, &mismatch) {
t.Fatalf("a read-only open of an older schema must return the typed refusal, got %v", err)
}
if mismatch.Found != head-1 || mismatch.Expected != head || !mismatch.Stale() {
t.Fatalf("found=%d expected=%d stale=%v, want %d/%d/true", mismatch.Found, mismatch.Expected, mismatch.Stale(), head-1, head)
}
applied, err := Migrate(path, nil)
if err != nil {
t.Fatalf("migrate: %v", err)
}
if applied.From != head-1 || applied.To != head {
t.Fatalf("transition %+v, want v%d -> v%d", applied, head-1, head)
}
if got := rawVersion(t, path); got != head {
t.Fatalf("schema is at v%d after migrating, want v%d", got, head)
}
ro, err := OpenReadOnly(path)
if err != nil {
t.Fatalf("the read-only path must work after a migration: %v", err)
}
ro.Close()
}
func TestMigrateIsANoOpOnAProjectAlreadyAtHead(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
openAtVersion(t, path, head)
for i := range 2 {
applied, err := Migrate(path, nil)
if err != nil {
t.Fatalf("call %d: %v", i, err)
}
if applied.From != head || applied.To != head {
t.Fatalf("call %d: transition %+v, want a no-op at v%d", i, applied, head)
}
}
}
func TestMigrateCreatesAProjectThatWasNeverRun(t *testing.T) {
// The deploy sweep runs over every book, including one that has been accepted but never translated.
// Creating it at head is what every other write open already does on first touch, and it keeps the
// sweep uniform instead of making the caller ask which books exist yet.
path := filepath.Join(t.TempDir(), "fresh.db")
applied, err := Migrate(path, nil)
if err != nil {
t.Fatal(err)
}
if applied.From != 0 || applied.To != SchemaHead() {
t.Fatalf("transition %+v, want v0 -> v%d", applied, SchemaHead())
}
ro, err := OpenReadOnly(path)
if err != nil {
t.Fatalf("a created project must be readable at head: %v", err)
}
ro.Close()
}
// TestMigrateKeepsCommittedMoneyAndZeroesOnlyTheStaleReservation is the money test.
//
// It is the one property the platform's ratified ceiling formula rests on (PD-158: the argument is
// committed + increment, WITHOUT reserved, precisely because a write open zeroes leftover reservations
// and a read-only status does not). A migration is a write open, so it performs that recovery — and it
// must move `committed` by not one cent while doing it, or every ceiling computed from a status taken
// after a deploy would be wrong.
func TestMigrateKeepsCommittedMoneyAndZeroesOnlyTheStaleReservation(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
caps := Ceilings{BookUSD: 10, DayUSD: 10}
// A book an older binary ran: one settled call, and one reservation whose process died before it
// settled — exactly the state a crashed or killed run leaves behind.
atVersion(t, head-1, func() {
s, err := Open(path)
if err != nil {
t.Fatal(err)
}
defer s.Close()
job := mustSnapshotAndJob(t, s)
res, verdict, err := s.Reserve("book", 0.10, caps)
if err != nil || verdict != ReserveOK {
t.Fatalf("reserve: %v %v", verdict, err)
}
cp := Checkpoint{RequestHash: "h1", JobID: job.ID, Stage: "draft", Role: "translator",
ModelRequested: "m", ModelActual: "m", ResponseText: "текст", UsageJSON: "{}", CostUSD: 0.04}
if err := s.SettleWithCheckpoint(res, 0.04, cp, nil); err != nil {
t.Fatal(err)
}
if _, verdict, err = s.Reserve("book", 0.25, caps); err != nil || verdict != ReserveOK {
t.Fatalf("stale reserve: %v %v", verdict, err)
}
committed, reserved, err := s.SpentUSD("book")
if err != nil || committed != 0.04 || reserved != 0.25 {
t.Fatalf("fixture: committed=%v reserved=%v err=%v", committed, reserved, err)
}
})
if _, err := Migrate(path, nil); err != nil {
t.Fatalf("migrate: %v", err)
}
// Twice: the second call is the idempotency half — a deploy sweep that runs over a book it already
// migrated must not move money either.
for i := range 2 {
if i == 1 {
if _, err := Migrate(path, nil); err != nil {
t.Fatalf("repeat migrate: %v", err)
}
}
ro, err := OpenReadOnly(path)
if err != nil {
t.Fatalf("call %d: %v", i, err)
}
committed, reserved, err := ro.SpentUSD("book")
ro.Close()
if err != nil {
t.Fatal(err)
}
if committed != 0.04 {
t.Fatalf("call %d: committed moved to %v — a migration must not touch settled money", i, committed)
}
if reserved != 0 {
t.Fatalf("call %d: reserved=%v, want 0 — the write open owes the recovery pass", i, reserved)
}
}
if got := rawVersion(t, path); got != head {
t.Fatalf("schema v%d after migrating, want v%d", got, head)
}
// The checkpoint the money is anchored to is still there: `committed == SUM(checkpoints)` is the
// invariant the platform settles from, so a migration that kept the number and lost the row would
// pass the assertion above and still have broken the book.
ro, err := OpenReadOnly(path)
if err != nil {
t.Fatal(err)
}
defer ro.Close()
cp, err := ro.GetCheckpoint("h1")
if err != nil || cp == nil || cp.CostUSD != 0.04 {
t.Fatalf("checkpoint after migration: %+v err=%v", cp, err)
}
}
// TestOpenRefusesAProjectNewerThanTheBinary: the write path used to accept a database it could not
// understand and would then have written through pre-dating code. It is refused in both open modes, and
// the refusal happens BEFORE the recovery pass — a live reservation still standing afterwards is the
// proof that nothing of the refused process reached the file.
func TestOpenRefusesAProjectNewerThanTheBinary(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
s, err := Open(path)
if err != nil {
t.Fatal(err)
}
if _, verdict, err := s.Reserve("book", 0.25, Ceilings{BookUSD: 10, DayUSD: 10}); err != nil || verdict != ReserveOK {
t.Fatalf("reserve: %v %v", verdict, err)
}
s.Close()
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatal(err)
}
if _, err := db.Exec(`INSERT INTO schema_version (version) VALUES (?)`, head+7); err != nil {
t.Fatal(err)
}
db.Close()
for name, open := range map[string]func(string) (*Store, error){"write": Open, "read-only": OpenReadOnly} {
st, err := open(path)
if err == nil {
st.Close()
t.Fatalf("%s open of a newer schema must be refused", name)
}
var mismatch *SchemaMismatchError
if !errors.As(err, &mismatch) {
t.Fatalf("%s open: %v, want a typed SchemaMismatchError", name, err)
}
if mismatch.Found != head+7 || mismatch.Expected != head || mismatch.Stale() {
t.Fatalf("%s open: found=%d expected=%d stale=%v", name, mismatch.Found, mismatch.Expected, mismatch.Stale())
}
}
if _, err := Migrate(path, nil); !errors.As(err, new(*SchemaMismatchError)) {
t.Fatalf("migrate of a newer schema must be refused with the typed error, got %v", err)
}
// Nothing was written: the reservation the fixture left standing is still standing, so the recovery
// pass never ran on a schema this binary does not know.
ro, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatal(err)
}
defer ro.Close()
var reserved float64
if err := ro.QueryRow(`SELECT COALESCE(SUM(reserved_usd), 0) FROM spend WHERE book_id = 'book'`).Scan(&reserved); err != nil {
t.Fatal(err)
}
if reserved != 0.25 {
t.Fatalf("reserved=%v after three refused opens, want the fixture's 0.25 untouched", reserved)
}
}
// TestAFailedMigrationStepLeavesTheDatabaseAtItsPreviousVersion answers, by execution, the question
// `tmctl migrate` raises: is APPLYING a step atomic? It is — DDL is transactional in SQLite and the
// version row is inserted in the SAME transaction as the step — so a process that dies mid-migration
// leaves a database at its previous version rather than a half-applied one whose ALTER steps (row 49а)
// will not converge on the next try. The migration bodies themselves are untouched by this pack.
func TestAFailedMigrationStepLeavesTheDatabaseAtItsPreviousVersion(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
// A step whose FIRST statement succeeds and whose second does not: if application were not atomic,
// `probe` would outlive the failure. Restored with defer, so a panic inside Open cannot leave the
// package-level chain rewritten for whatever runs next.
full := migrations
defer func() { migrations = full }()
migrations = append(append([]string(nil), migrations...), `CREATE TABLE probe (x INTEGER); THIS IS NOT SQL;`)
s, err := Open(path)
if err == nil {
s.Close()
t.Fatal("a migration step that does not execute must fail the open")
}
if got := rawVersion(t, path); got != head {
t.Fatalf("schema recorded v%d after a failed step, want the previous v%d", got, head)
}
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var probes int
if err := db.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'probe'`).Scan(&probes); err != nil {
t.Fatal(err)
}
if probes != 0 {
t.Fatal("the failed step's first statement survived: applying a migration is not atomic")
}
}
// TestADatabaseWithNothingAppliedIsAVersionAndNotAnError: a file that exists but has no
// schema_version table — a process killed between creating one and committing the first migration —
// used to fail the read path with prose and exit 1, i.e. as an unrepairable project. It is version 0:
// the SAME typed refusal as any other mismatch, so the same "caught it → migrate → retry" repairs it.
func TestADatabaseWithNothingAppliedIsAVersionAndNotAnError(t *testing.T) {
dir := t.TempDir()
empty := filepath.Join(dir, "empty.db")
if err := os.WriteFile(empty, nil, 0o644); err != nil {
t.Fatal(err)
}
_, err := OpenReadOnly(empty)
var mismatch *SchemaMismatchError
if !errors.As(err, &mismatch) {
t.Fatalf("a database with nothing applied must be the typed refusal, got %v", err)
}
if mismatch.Found != 0 || mismatch.Expected != SchemaHead() || !mismatch.Stale() {
t.Fatalf("found=%d expected=%d stale=%v, want 0/%d/true", mismatch.Found, mismatch.Expected, mismatch.Stale(), SchemaHead())
}
if _, err := Migrate(empty, nil); err != nil {
t.Fatalf("and a migration must repair it: %v", err)
}
ro, err := OpenReadOnly(empty)
if err != nil {
t.Fatalf("read path after the repair: %v", err)
}
ro.Close()
// A file that is not a database at all must NOT read as "version 0, migrate me".
junk := filepath.Join(dir, "junk.db")
if err := os.WriteFile(junk, []byte("this is not a database, it is a text file"), 0o644); err != nil {
t.Fatal(err)
}
if _, err := OpenReadOnly(junk); err == nil || errors.As(err, &mismatch) {
t.Fatalf("a corrupt file must fail loudly and NOT as a schema mismatch, got %v", err)
}
}
// TestTheMigrationSeamRunsUnderTheLockAndOnlyWhenAStepWillRun pins the placement `tmctl migrate` takes
// its restore point in. The three properties are what make a backup a backup: it runs BEFORE the first
// step (a copy taken afterwards is not a restore point), only when there IS a step (else a repeat call
// litters, and its second-precision stamp collides), and inside the lock (else a project a live run
// owns is copied in full for a migration that is refused anyway — once per retry).
func TestTheMigrationSeamRunsUnderTheLockAndOnlyWhenAStepWillRun(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
openAtVersion(t, path, head-1)
var seen []Migration
versionAtHook := -1
applied, err := Migrate(path, func(m Migration) error {
seen = append(seen, m)
versionAtHook = rawVersion(t, path)
// The lock is ours while the hook runs: a second opener must be turned away, which is what
// makes "back up here" different from "back up before calling".
if s, err := Open(path); !errors.Is(err, ErrLocked) {
if err == nil {
s.Close()
}
t.Errorf("the seam must run INSIDE the project lock; a concurrent open got %v", err)
}
return nil
})
if err != nil {
t.Fatal(err)
}
if len(seen) != 1 || seen[0] != (Migration{From: head - 1, To: head}) {
t.Fatalf("the seam saw %+v, want exactly one v%d -> v%d", seen, head-1, head)
}
if versionAtHook != head-1 {
t.Fatalf("the database was at v%d when the seam ran, want the PRE-migration v%d", versionAtHook, head-1)
}
if applied.From != head-1 || applied.To != head {
t.Fatalf("transition %+v", applied)
}
// Nothing left to apply → the seam does not run at all.
seen = nil
if _, err := Migrate(path, func(m Migration) error { seen = append(seen, m); return nil }); err != nil {
t.Fatal(err)
}
if len(seen) != 0 {
t.Fatalf("the seam ran %d time(s) on a project already at head", len(seen))
}
}
// TestTheSeamIsNotChargedToTheStoreOperationBudget: the seam's work is the CALLER's (a restore point
// whose cost scales with the book), and opTimeout bounds operations that are O(1) in the size of the
// book. While the two shared one deadline, a large project failed the migration on the copy it had just
// paid for — «context deadline exceeded» on the first step, exit 1 outside the refusal band, and a
// retry that copied the whole database again (round-2 review, reproduced on 2.6 GB).
//
// The budget is shrunk instead of the seam sleeping ten real seconds: same property, no dead time in
// the battery. It is restored on the way out and nothing here runs in parallel.
func TestTheSeamIsNotChargedToTheStoreOperationBudget(t *testing.T) {
head := SchemaHead()
defer func(original time.Duration) { opTimeout = original }(opTimeout)
opTimeout = 150 * time.Millisecond
path := filepath.Join(t.TempDir(), "book.db")
openAtVersion(t, path, head-1)
applied, err := Migrate(path, func(Migration) error {
time.Sleep(3 * opTimeout) // a restore point on a book far larger than the budget
return nil
})
if err != nil {
t.Fatalf("a slow seam must not fail the migration it protects: %v", err)
}
if applied.From != head-1 || applied.To != head {
t.Fatalf("transition %+v, want v%d -> v%d", applied, head-1, head)
}
if got := rawVersion(t, path); got != head {
t.Fatalf("schema at v%d, want v%d", got, head)
}
}
// TestASeamFailureAbortsTheMigrationWithNothingApplied: a caller that cannot take its restore point is
// telling the engine not to migrate — the answer is to stop with the database exactly as it was, not to
// proceed unprotected.
func TestASeamFailureAbortsTheMigrationWithNothingApplied(t *testing.T) {
head := SchemaHead()
path := filepath.Join(t.TempDir(), "book.db")
openAtVersion(t, path, head-1)
refuse := errors.New("no room for a restore point")
if _, err := Migrate(path, func(Migration) error { return refuse }); !errors.Is(err, refuse) {
t.Fatalf("the seam's error must abort the migration, got %v", err)
}
if got := rawVersion(t, path); got != head-1 {
t.Fatalf("schema moved to v%d after a refused seam, want v%d", got, head-1)
}
// And the lock is released, so the next attempt is not locked out by the failed one.
if _, err := Migrate(path, nil); err != nil {
t.Fatalf("a retry after a refused seam: %v", err)
}
}

View file

@ -29,13 +29,24 @@ import (
// opTimeout bounds every store operation: SQLite is a local file and
// effectively never blocks; the cap keeps a wedged filesystem from hanging a
// pipeline goroutine forever.
const opTimeout = 10 * time.Second
//
// It is a PER-OPERATION budget and only holds while every operation under it is O(1) in the size of the
// book — which is what the migration chain and the recovery pass are (metadata-only DDL and one UPDATE).
// A caller's own work must therefore never be charged to it; see the seam in migrate.
//
// A var, not a const, so a test can shrink it: the property "the seam is not charged to this budget" is
// otherwise only testable by sleeping ten real seconds in the battery.
var opTimeout = 10 * time.Second
// Store is one project database.
type Store struct {
w *sql.DB // single-connection write pool; all mutations go through it
r *sql.DB // read pool
lock *os.File
// applied is the schema transition this open performed — the zero value on a read-only open, which
// never migrates. It is recorded because a write open is the only thing that MAY migrate, so it is
// also the only thing that knows what it migrated (`tmctl migrate` reports it, row 174).
applied Migration
}
// Open opens (or creates) the project database at path, applies pending
@ -45,7 +56,11 @@ type Store struct {
// owns the project file" is enforced here, not assumed. Without the lock a
// concurrent `tmctl report` during a translation would zero a running process's live
// reservations via the recovery pass and blind its ceilings (review finding).
func Open(path string) (*Store, error) {
func Open(path string) (*Store, error) { return open(path, nil) }
// open is Open with the migration seam exposed: beforeApply, when non-nil, runs after the lock is held
// and the pending set is known, and before the first step is applied. See Migrate.
func open(path string, beforeApply func(Migration) error) (*Store, error) {
lock, err := acquireLock(path + ".lock")
if err != nil {
return nil, err
@ -80,12 +95,18 @@ func Open(path string) (*Store, error) {
r.SetMaxOpenConns(4)
s := &Store{w: w, r: r, lock: lock}
ctx, cancel := context.WithTimeout(context.Background(), opTimeout)
defer cancel()
if err := s.migrate(ctx); err != nil {
// Each phase takes its OWN budget rather than sharing one across the whole open. The migration is
// the only phase that can host caller work of unbounded size (the seam below), and a single shared
// deadline made that work eat the budget of the steps it was called to protect: on a large project
// the restore point finished and the first step then failed «context deadline exceeded» — an exit 1
// outside the refusal band, after an expensive write, with every retry copying the database again
// (round-2 review, reproduced on a 2.6 GB project).
if err := s.migrate(path, beforeApply); err != nil {
s.Close()
return nil, err
}
ctx, cancel := opContext()
defer cancel()
if err := s.recoverReservations(ctx); err != nil {
s.Close()
return nil, err
@ -104,9 +125,9 @@ func Open(path string) (*Store, error) {
//
// Guarantees: connections are strictly read-only (PRAGMA query_only=1 — an accidental
// write fails loudly rather than corrupting a live writer's state); a missing DB is a loud
// error (we don't create an empty one); a schema older than the binary is a loud error "run a
// write command" (only the writer applies migrations). A known, honest boundary:
// after a run crashes, reserved_usd stays non-zero until the next WRITE command
// error (we don't create an empty one); a schema this binary does not match is a TYPED refusal naming
// `tmctl migrate` (only a write open applies migrations — see SchemaMismatchError). A known, honest
// boundary: after a run crashes, reserved_usd stays non-zero until the next WRITE command
// (the recovery pass runs only there) — status will show this leftover as reserved.
func OpenReadOnly(path string) (*Store, error) {
if _, err := os.Stat(path); err != nil {
@ -128,20 +149,18 @@ func OpenReadOnly(path string) (*Store, error) {
s := &Store{w: r, r: r, lock: nil}
ctx, cancel := context.WithTimeout(context.Background(), opTimeout)
defer cancel()
var current int
if err := s.r.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM schema_version`).Scan(&current); err != nil {
current, err := schemaVersion(ctx, s.r)
if err != nil {
s.Close()
return nil, fmt.Errorf("store: read schema version of %s (empty/corrupt database?): %w", path, err)
return nil, fmt.Errorf("store: read schema version of %s (not a database?): %w", path, err)
}
if current < len(migrations) {
// A version this binary does not match is a TYPED refusal, in both directions: the caller's repair
// differs ("migrate this project" versus "upgrade this binary"), and while both arrived as prose and
// exit 1 an automated caller could act on neither — which is what turned a binary upgrade into a
// deadlock across every existing book (row 174, SchemaMismatchError).
if current != len(migrations) {
s.Close()
return nil, fmt.Errorf("store: %s is at schema v%d, this binary expects v%d — run a write command (`tmctl translate`/`redrive`) to migrate first (read-only open never migrates)",
path, current, len(migrations))
}
if current > len(migrations) {
s.Close()
return nil, fmt.Errorf("store: %s is at schema v%d, NEWER than this binary's v%d — update tmctl (reading a newer schema with old code is unsafe)",
path, current, len(migrations))
return nil, &SchemaMismatchError{Path: path, Found: current, Expected: len(migrations)}
}
return s, nil
}