textmachine/backend/internal/pipeline/volume.go

package pipeline

import (
	"context"
	"fmt"

	"textmachine/backend/internal/chunk"
	"textmachine/backend/internal/membank"
	"textmachine/backend/internal/runevents"
	"textmachine/backend/internal/store"
)

// volume.go: the VOLUME ceiling — the run's second stop, beside the money one (D39.165 §1б, backlog row
// «потолок объёма»).
//
// WHY A SECOND CEILING AT ALL. The platform sells CHAPTERS and hands the engine a DOLLAR bound
// (--ceiling-usd), which is the only bound the engine had. Measured on real ledgers (D39.165 §1), a
// chapter costs $0.0115–$0.0190 and the constant sold against it is $0.03 — so a purchase of ten chapters
// hands over $0.30, and $0.30 buys sixteen to twenty-six. The knob said chapters and meant money. This
// gives the engine a bound in the unit it actually ships, so the seller's promise and the engine's stop
// are the same quantity.
//
// WHAT IT IS MEASURED IN, and why not the other two candidates. The unit is the OUTPUT UNIT — editUnit,
// the granularity `outputUnits` returns and `buildManifest` publishes as the manifest's units_total, which
// is where the platform's per-chapter unit count comes from. So "N units" means the same thing on both
// sides of the seam, and the platform's chapters→units conversion is EXACT rather than an estimate: the
// manifest is $0, needs no keys and exists before the first paid call.
//   - NOT chunk×stage, which is the BILLING unit (the one the re-payment estimate counts in). Cutting the
//     wave in the unit one PAYS in, while selling the unit one SHIPS in, reproduces the very defect above
//     one layer down: a unit costs len(Members)·nDraftStages + nEditStages positions, so "10" would buy a
//     different amount of book on a different pipeline shape.
//   - NOT the chunk, which is an internal artefact of the cut: the buyer never sees it and the manifest
//     does not count it.
//
// ⚠ The unit count DOES depend on the pipeline's shape — an editor pipeline groups draft chunks into
// coarse edit units, a draft-only pipeline ships one unit per draft chunk (outputUnits). That is a
// property of the configuration, not a branch per language pair, and it is safe because the manifest the
// platform counted and the run it bounds come from the SAME deployment. It would stop being safe if a
// ceiling were carried across two deployments with different pipelines.
//
// HOW REPINS AND RETRIES COUNT — the question this design has to answer out loud.
//   - A RETRY and an ESCALATION hop do NOT count. They live inside one unit (the attempt loop in
//     stagerun.go writes ONE chunk_status row per chunk×stage however many attempts it took), and the
//     volume ceiling bounds DELIVERY while the money ceiling bounds SPEND. The measured quality tail
//     (retries+escalations swinging 1.5%→23% between runs) is a fact about money and already has its own
//     bound; if it burned volume, a buyer of ten chapters would receive eight because two of them
//     stuttered — paying in BOOK for the engine's own tail, which is the substitution this pack exists to
//     remove.
//   - A RE-PIN and a $0 RESUME do NOT count. A re-pass that re-pins five hundred units for $0 (rebill.go)
//     would otherwise exhaust the ceiling having delivered nothing.
//
// So: THE CEILING COUNTS THE OUTPUT UNITS THIS RUN TAKES A SLOT FOR — new book it starts paying for, and
// delivered book it re-makes under a moved snapshot (unitRework, which takes a slot per re-make by the
// DELIVERY BEFORE REWORK order below). Units that resolve for free are admitted regardless — they cost
// nothing, and holding them back would leave the book's snapshots stale for no saving. And a unit an
// EARLIER run started and never shipped is admitted outside the grant too, though it costs: charging a
// slot again from every run that advances the same unit is how a buyer of N units receives fewer than N
// chapters (backlog row 232, second half). The record of "started" is the unit's own stored rows at the
// positions this run executes, which is why the rule survives a restart without a carrier of its own.
// ⚠ What the engine does NOT know is whether a slot was ever taken for it: the run that started the unit
// may have carried no ceiling at all. So the rule is stated as what it is — this run does not charge its
// grant for a unit it did not start — and never as a conservation law the store could not support.
// Such a unit is CARRIED (unitCarried), reported apart from new starts, and — because it does cost —
// still bounded by the money ceiling, which is the bound that was always meant to hold the dollars.
//
// ⚠ AND THE CARRY IS ITSELF BOUNDED BY THE GRANT — at most MaxUnits of them, in book order. Without that
// bound the ceiling stops being one: a run that drafted three hundred units and died before the edit wave
// (it may have carried NO ceiling at all) would hand every one of them to the next purchase, and a buyer
// of ten chapters would be charged for three hundred editor calls. Measured on a fixture: a grant of one
// paid for five units. The bound has a principle rather than a number — a purchase of N may finish as many
// interrupted units as it was granted new ones, so the work a grant authorises stays proportional to it —
// and what it holds back is reported as never delivered, which is what those units are.
//
// ⚠ THE RULE STOPS AT «NEVER SHIPPED». A delivered unit whose RE-MAKE was interrupted between the waves
// — its draft row re-bought under the current snapshot, its edit row still on the old one — has a row at
// every position, is classified unitRework again and takes a slot again. Telling that shape from an
// ordinary bank-only edit move (draft free, edit re-paid: the very definition of rework) needs a record of
// which run bought which row, and nothing stores one; it is the rework twin of the residual deliveredUnits
// names.
//
// AND IT COUNTS THEM IN TWO KINDS, which is the difference between a report and a true report. A paying
// unit is either book that did not exist (DELIVERY) or book that existed and is being made again under a
// moved snapshot (REWORK). unitClass below is that distinction, and VolumeStop carries it all the way to
// the operator: a purchase spent entirely on rework delivers no chapter, and neither the counters nor the
// line the CLI prints may let that read as a delivery. The remainder splits the same way — units never
// delivered are the only ones anybody may be invited to buy, while delivered-but-not-re-made units are
// unrefreshed, not unbought.
//
// ⚠ ON THE PREDICATE'S RELATION TO projectRebill: the free/paying test here is runStage's, not the
// re-payment estimate's, and they are NOT the same test. projectRebill deliberately ignores the CONTENT
// axis (its own doc comment says so: a source edit moves content_hash but not the snapshot, and is
// invisible there). This one checks the rendered content hash on every row, so a source edit correctly
// makes a unit paying here while the estimate would call it free. The difference is in the safe
// direction — the ceiling over-counts cost rather than under-counting it — and it is why this predicate
// is written against the executor rather than borrowed from the projection.
//
// CHECKED BEFORE, NOT AFTER. The scope is computed ONCE, before either wave starts, and the waves simply
// do not begin an item outside it. That is structural rather than a guard: there is no moment at which a
// unit past the ceiling has started, so the ceiling cannot be overshot by one call the way a post-hoc
// token check always is (LiteLLM's known behaviour, named in D39.165 §1б). It is also why the admission
// is not a counter the wave workers decrement — the waves are parallel, and a worker refused a slot would
// have to abandon its item permanently while a slot freed by a $0 re-pin went to nobody.
//
// THE STOP IS A COMPLETION, NOT A HALT — ratified by the orchestrator with this pack and NOT re-decided
// here. A money ceiling stops the run in the MIDDLE of work it meant to do: that is exit 4, the platform
// records `paused`, and the remedy is more money. A volume ceiling means the run did exactly what was
// bought: exit 0, the frozen exit-code dictionary (cmd/tmctl/main.go) is not touched at all, and the
// platform records the run `ready` — which does NOT declare the book finished, because the book's progress
// travels as unit events, so the next purchase starts a new run normally. What DOES have to be said is
// WHICH ceiling stopped it (OpenHands' iteration limit is silent, and that is a standing complaint named
// in the same note): that lives in the run's result, its log line and the CLI's rendering, not in a new
// exit code and not in a new word of the event stream's outcome vocabulary.

// VolumeStop is a run's report that it stopped on the volume ceiling rather than on the end of the book.
// It is a RESULT, not an error: the run did what it was granted, so it travels on BookResult and never
// through the error path, where the exit-code mapper would have to invent a word for it.
//
// ⚠ IT COUNTS PAID UNITS IN TWO KINDS, and the split is a MODEL rather than a nicety. A paying unit is
// either book that did not exist before (DELIVERY) or book that existed and is being made again under a
// moved snapshot (REWORK), and the two are not the same product: the first is what "buy ten chapters"
// means, the second is what a re-pass means. The first version of this struct had one counter for both,
// and every consequence of that was a lie the operator could act on — a purchase that went entirely into
// re-translating the opening chapters reported ten units bought and exited 0 like a delivery, while the
// remaining count invited buying chapters the reader already owned. A struct that cannot tell the two
// apart cannot report either one honestly, so it carries both.
type VolumeStop struct {
	// MaxUnits is the ceiling that was in force.
	MaxUnits int
	// Delivered is paying units that had never been completed before — NEW book, the thing a purchase of
	// chapters is actually for. It INCLUDES the carried units below: they are new book and this run pays
	// for them; what they do not do is take a slot of this grant.
	Delivered int
	// Carried is the part of Delivered that an EARLIER run had already started — rows on file for some of
	// the unit's positions, none of them shipped — and this run finishes OUTSIDE its grant, the slot having
	// been taken by the run that started them (the file comment, backlog row 232). Reported apart so the
	// grant arithmetic (granted) and the operator can see why a run delivered more than it was granted.
	Carried int
	// Reworked is paying units that WERE already completed and are being paid for a second time because
	// their snapshot moved. Real work and a real charge, but not new book: a purchase made entirely of
	// these advances the reader not at all.
	Reworked int
	// Flagged is paid units that resolved FLAGGED and shipped no text. They cost money and delivered
	// nothing, and calling them delivered is the lie this counter exists to stop: a purchase of two units
	// one of which flagged used to print "2 NEW unit(s) delivered" over a single readable unit.
	//
	// ⚠ It is filled AFTER the waves, from what actually happened, not from the plan. Everything else on
	// this struct is a plan-time decision about admission — correct for bounding money, and unable on its
	// own to know whether the work it authorised produced a chapter.
	Flagged int
	// Free is units that rode along at $0 — resumed or re-pinned. Reported because otherwise the operator
	// sees a run that touched far more units than it was granted and cannot tell why.
	Free int
	// LeftFresh is undelivered units still in the book. THIS is what "there is more to buy" honestly
	// means, and it is the only one of the two remainders a buyer should ever be invited to purchase.
	LeftFresh int
	// LeftRework is completed units still carrying a superseded snapshot. They are "unrefreshed", not
	// "unbought", and presenting them as stock for sale is how re-payment becomes a product.
	LeftRework int
}

// Paid is every unit this run was charged for, of either kind.
func (v VolumeStop) Paid() int { return v.Delivered + v.Reworked }

// granted is how many of the grant's slots this run has taken: every paying unit except those an earlier
// run started, whose slot was taken then. It is the number the ceiling is compared against — Paid() would
// charge a carried unit a second slot, which is the defect row 232 names.
func (v VolumeStop) granted() int { return v.Delivered - v.Carried + v.Reworked }

// reconcile corrects the plan-time counters against what the run actually produced.
//
// Admission is a decision made BEFORE the work, and it has to be — that is what bounds the money. But a
// decision to pay for a unit is not a fact about a chapter existing: a unit can be paid for and come back
// flagged with nothing shippable. Reporting the plan as though it were the outcome is how "2 NEW unit(s)
// delivered" ended up printed over one readable unit. So the counters are trued up here, from the
// outcomes, before anyone reads them.
func (s *volumeScope) reconcile(outcomes []ChunkOutcome) {
	if s == nil {
		return
	}
	for _, oc := range outcomes {
		if oc.Disposition != DispFlagged || oc.FinalText != "" {
			continue // it shipped something readable; the plan's word for it stands
		}
		// ⚠ CARRIED IS A PART OF DELIVERED, and every branch here keeps it one. granted() subtracts Carried
		// from Delivered, so a branch that moves Delivered past Carried makes the grant arithmetic negative
		// and the stop line say «0 NEW unit(s) delivered (1 of the new ones had been started by an earlier
		// run)» — a sentence that cannot be true. The two fall-through branches therefore ask which kind of
		// delivered unit is still there to move, rather than assuming a non-carried one is.
		class := s.class[chunkKey{oc.Chapter, oc.ChunkIdx}]
		switch {
		case class == unitFresh && s.stop.Delivered > s.stop.Carried:
			s.stop.Delivered--
		case class == unitCarried && s.stop.Carried > 0:
			s.stop.Delivered--
			s.stop.Carried--
		case s.stop.Reworked > 0:
			s.stop.Reworked--
		case s.stop.Delivered > s.stop.Carried:
			s.stop.Delivered--
		case s.stop.Carried > 0:
			s.stop.Delivered--
			s.stop.Carried--
		default:
			continue // it was never counted as paid (a free unit that flagged on resume); nothing to move
		}
		s.stop.Flagged++
	}
}

// Left is every unit the ceiling held back, of either kind.
func (v VolumeStop) Left() int { return v.LeftFresh + v.LeftRework }

func (v VolumeStop) String() string {
	// ⚠ EVERY COUNT HERE IS IN OUTPUT UNITS, AND THE WORDS MUST SAY SO. An earlier version of this line
	// read "%d NEW chapter(s) delivered" over a counter that increments per editUnit — and a chapter is
	// more than one unit whenever the cut closes a unit at the edit ceiling, and is many units in a
	// draft-only pipeline. That is the unit↔chapter substitution this whole ceiling exists to remove,
	// committed by the ceiling's own report. Chapters are the PLATFORM's word; it converts them through
	// the manifest. The engine speaks units and nothing else.
	// ⚠ TWO OPENINGS, because the report is attached for TWO different reasons and one sentence cannot
	// carry both. The grant may have held work back — the run stopped short of the book — or it may have
	// held nothing back and the run still did work outside the grant, by finishing units an earlier run
	// began. Said with the first opening, the second case reads «not at the end of the book» in the same
	// breath as «0 unit(s) NEVER delivered»: an operator is told the run was cut short and that nothing
	// remains, and both cannot be true. Which one applies is Left(), the same question bound() asks.
	head := "stopped on the VOLUME ceiling (--max-units %d), not on money and not at the end of the book"
	if v.Left() == 0 {
		head = "reached the END of the book under a VOLUME ceiling (--max-units %d), which held nothing back"
	}
	s := fmt.Sprintf(head+": %d paying output unit(s) — %d NEW unit(s) delivered, %d already-delivered unit(s) re-made under a moved snapshot",
		v.MaxUnits, v.Paid()+v.Flagged, v.Delivered, v.Reworked)
	if v.Carried > 0 {
		s += fmt.Sprintf(" (%d of the new ones had been started by an earlier run and were finished OUTSIDE this grant — a unit takes a slot once, in the run that starts it)", v.Carried)
	}
	if v.Flagged > 0 {
		s += fmt.Sprintf(", and %d PAID FOR BUT FLAGGED — money spent, no readable text produced (buying more will not fix them; `tmctl redrive` re-attacks a flag)", v.Flagged)
	}
	if v.Free > 0 {
		s += fmt.Sprintf("; %d rode along at $0 (resumed or re-pinned)", v.Free)
	}
	if v.Left() == 0 {
		return s + ". Nothing is left in the book"
	}
	s += fmt.Sprintf(". Still in the book: %d unit(s) NEVER delivered", v.LeftFresh)
	if v.LeftRework > 0 {
		s += fmt.Sprintf(" and %d already delivered but not yet re-made (those are unrefreshed, NOT unbought)", v.LeftRework)
	}
	return s
}

// volumeScope is the run's admitted set of output units, decided before any work begins. A nil scope
// means no volume ceiling is in force and nothing anywhere changes behaviour.
type volumeScope struct {
	admitted map[chunkKey]bool // unit leader key → admitted
	leader   map[chunkKey]bool // member chunk key → its unit is admitted (flattened for the draft wave)
	stop     VolumeStop
	// class is what each unit was judged to be at PLAN time, kept because the edit wave has to re-judge
	// the free ones (see rescopeEditWave).
	class map[chunkKey]unitClass
	// editSnapshot is the edit-wave snapshot the classification above was made AGAINST. The run can move
	// it after planning — the bank-mining stop re-seeds mid-run — so this is what rescopeEditWave compares
	// with to know whether the plan is still standing on the ground it was made on.
	editSnapshot string
	// editBlocked are units whose EDIT must not run even though they were admitted: they were judged free,
	// the bank then moved under them, and no grant was left to pay for what they turned out to cost.
	editBlocked map[chunkKey]bool
}

// allows reports whether a DRAFT chunk may be started: it may when the unit it belongs to was admitted.
func (s *volumeScope) allows(ch chunk.Chunk) bool {
	if s == nil {
		return true
	}
	return s.leader[chunkKey{ch.Chapter, ch.ChunkIdx}]
}

// allowsUnit reports whether an EDIT unit may be started.
func (s *volumeScope) allowsUnit(u editUnit) bool {
	if s == nil {
		return true
	}
	key := chunkKey{u.Chapter, u.FirstChunkIdx}
	return s.admitted[key] && !s.editBlocked[key]
}

// admittedStatuses narrows stored rows to the units this run is actually going to work on, so a money
// projection over them describes THIS run rather than the whole book. A nil scope is the identity, which
// is what keeps every un-bounded run's figures byte-identical to what they were before the ceiling
// existed. An edit row lives at its unit's LEADER chunk, and a leader is one of the unit's members, so
// the member map covers both waves' rows.
func (s *volumeScope) admittedStatuses(in []store.ChunkStatus) []store.ChunkStatus {
	if s == nil {
		return in
	}
	out := in[:0:0]
	for _, cs := range in {
		if s.leader[chunkKey{cs.Chapter, cs.ChunkIdx}] {
			out = append(out, cs)
		}
	}
	return out
}

// bound reports whether this run's report has anything to say about the grant: work was held back, or work
// was done OUTSIDE the grant. A run granted more than the book had left is an ordinary complete run and
// must not be reported as a volume stop — otherwise every generously-bounded run would end claiming it had
// been cut short.
//
// ⚠ CARRIED COUNTS AS SOMETHING TO SAY, and that is disclosure rather than bookkeeping: a run that finishes
// interrupted units delivers MORE than its grant, and the one line explaining why is the stop line. Judged
// by the remainder alone, the run whose whole remainder is carried — the largest discrepancy there is —
// would report nothing at all.
func (s *volumeScope) bound() bool { return s != nil && (s.stop.Left() > 0 || s.stop.Carried > 0) }

// planVolume decides, before any wave starts, which output units this run may work on.
//
// It is $0 by construction: store reads, string rendering and hashing, no provider anywhere. It is also
// skipped entirely when no ceiling is set, so a run without --max-units pays nothing for this existing —
// not even the content-hash reproduction, which is the expensive half.
func (r *Runner) planVolume(ctx context.Context, chunks []chunk.Chunk, stickySel []membank.Selection) (*volumeScope, error) {
	if r.MaxUnits <= 0 {
		return nil, nil
	}
	// ⚠ REFUSED on a pipeline whose SHIPPING stage is a translator while editor stages also exist, because
	// in that one shape the two granularities this ceiling straddles stop being the same object. outputUnits
	// keys off finalStageWave and returns a singleton per DRAFT CHUNK there, while the edit wave still
	// groups those chunks into buildEditUnits — so a ceiling granted in singletons would admit one member's
	// chunk and then let the edit wave run the whole multi-member unit over drafts that were never made,
	// paying full price for half-empty input. config validates stage ROLES but imposes no order, so the
	// shape loads; refusing loudly is the only answer that neither miscounts a purchase nor silently
	// changes what a unit means. (It is also the shape in which the platform's chapters→units conversion
	// would stop being exact, which is the whole basis of measuring in units at all.)
	if len(r.waveStagesIndexed(waveEdit)) > 0 && r.finalStageWave() == waveDraft {
		return nil, fmt.Errorf("pipeline: --max-units cannot bound this pipeline: its last stage is a translator while editor stages exist, so the shipping unit is the draft CHUNK while the edit wave still works in grouped edit UNITS — a ceiling counted in one would admit work measured in the other. Put the shipping stage last (an editor/other role), or run this pipeline without a volume ceiling")
	}
	// ⚠ THE ONE COMPOSITION THIS CEILING MAKES WORSE, said out loud where an operator will see it.
	//
	// A bounded purchase drafts only part of the book, and the bank-mining stop consolidates over the
	// chunks drafted SO FAR — so the auto-bank it writes grows with every purchase. The next purchase folds
	// that larger auto-bank into the ENRICHED bank, which moves the edit-wave snapshot, and the edit jobs
	// the PREVIOUS purchase created are then pinned to a superseded one: without --resnapshot the run stops
	// loudly, and with it the units whose injected bytes the new terms changed are re-translated and re-paid.
	//
	// None of that is new machinery — it is the mine D39.165 §3 named and errata 28.08-и narrowed to "an
	// edit made AFTER edit jobs exist". What IS new is the frequency: without a volume ceiling that state
	// needs an interrupted run (which usually has no edit jobs yet), while WITH one it is the ordinary
	// shape of selling a book a few chapters at a time. Warned rather than refused, because the run is
	// still correct — it re-pays through the consent gate like any other re-payment — and because refusing
	// would take the ceiling away from exactly the books that most need selling in parts.
	if r.pack != nil && r.Pipeline.Mining.ContrastPath != "" {
		r.Log.WarnContext(ctx, "a VOLUME ceiling on a book that MINES its bank: each purchase drafts more, so the bank-mining stop writes a larger auto-bank, and the next purchase moves the edit-wave snapshot the previous purchase's edit jobs are pinned to. Expect that run to need --resnapshot and to re-pay the units the new terms actually touch (the re-payment consent gate still bounds it)",
			"book", r.Book.BookID, "max_units", r.MaxUnits)
	}
	units := r.outputUnits(chunks)
	statuses, err := r.Store.ChunkStatusesForBook(r.Book.BookID)
	if err != nil {
		return nil, fmt.Errorf("pipeline: read chunk_status for the volume ceiling: %w", err)
	}
	class, curEditAtPlan, err := r.classifyUnits(units, chunks, statuses, stickySel)
	if err != nil {
		return nil, err
	}
	s := &volumeScope{
		admitted:     make(map[chunkKey]bool, len(units)),
		leader:       make(map[chunkKey]bool, len(chunks)),
		stop:         VolumeStop{MaxUnits: r.MaxUnits},
		class:        class,
		editSnapshot: curEditAtPlan,
		editBlocked:  map[chunkKey]bool{},
	}
	admit := func(u editUnit) {
		key := chunkKey{u.Chapter, u.FirstChunkIdx}
		s.admitted[key] = true
		for _, m := range u.Members {
			s.leader[chunkKey{m.Chapter, m.ChunkIdx}] = true
		}
	}
	// ⚠ A MISSING KEY READS AS unitFresh, and that is deliberate rather than incidental: classifyUnits
	// returns an EMPTY map for a book with no stored rows, so the zero value has to be the class such a
	// book's units actually are. unitFresh is iota 0 for exactly this reason — do not reorder the enum.
	classOf := func(u editUnit) unitClass { return class[chunkKey{u.Chapter, u.FirstChunkIdx}] }

	// Free units first and unconditionally: they spend nothing, so holding them back would only leave
	// their rows pinned to a superseded snapshot for no saving at all.
	for _, u := range units {
		if classOf(u) == unitFree {
			s.stop.Free++
			admit(u)
		}
	}

	// Then the CARRIED units — the ones an earlier run started and never shipped. They pay, but not out of
	// this grant's slots: charging a slot again for the same unit is what turns four bought units into two
	// chapters (backlog row 232). What they are bounded BY is the grant's size, in book order: a purchase
	// of N may finish at most N interrupted units beside the N it starts. Unbounded, this is not a ceiling
	// at all — see the file comment. What does not fit is held back as what it is: never delivered.
	for _, u := range units {
		if classOf(u) != unitCarried {
			continue
		}
		if s.stop.Carried >= r.MaxUnits {
			s.stop.LeftFresh++
			continue
		}
		s.stop.Delivered++
		s.stop.Carried++
		admit(u)
	}

	// ⚠ DELIVERY BEFORE REWORK, and this ORDER is the answer to "what did the buyer actually get".
	//
	// Both kinds are paying work, so a single pass in book order admits whichever comes first — and after
	// any bank edit the already-delivered units ARE the early ones. Measured consequence, on a fixture
	// where one chapter is one unit: a purchase of two units on a five-unit book with two delivered
	// re-made units 1 and 2 and never started unit 3, reporting Delivered=0, Reworked=2, LeftFresh=3. The
	// buyer paid for two units, received none, and three units that had never been translated at all sat
	// untouched. A ceiling sold as "the seller's promise and the engine's stop are the same quantity"
	// cannot behave that way.
	//
	// So the grant goes to NEW book while any is left, and only then to re-making. The reason it is safe to
	// serve units out of book order is that nothing in a unit's rendered bytes depends on which other units
	// ran: the sticky-window selection is precomputed over the WHOLE book before either wave
	// (precomputeSticky), so running unit 3 before unit 1 renders exactly what running them in order would.
	//
	// ⚠ THE TRADE IS NAMED, not hidden: a purchase whose PURPOSE was a re-pass will now spend its grant on
	// undelivered units if the book still has any. That is the right default — an undelivered unit is
	// worth more to a reader than a re-made one, and the re-payment consent gate bounds rework by money in
	// its own right — but it IS a default, and a caller who needs "re-make only" needs a flag that says so.
	// Reported to the orchestrator as a chosen default rather than assumed.
	for _, pass := range []unitClass{unitFresh, unitRework} {
		for _, u := range units {
			if classOf(u) != pass {
				continue
			}
			if s.stop.granted() >= r.MaxUnits {
				if pass == unitFresh {
					s.stop.LeftFresh++
				} else {
					s.stop.LeftRework++
				}
				continue
			}
			if pass == unitFresh {
				s.stop.Delivered++
			} else {
				s.stop.Reworked++
			}
			admit(u)
		}
	}
	return s, nil
}

// rescopeEditWave re-judges the FREE units against the snapshot the edit wave is actually about to run
// under, and makes them pay for a grant slot if they turn out to cost money.
//
// ⛔ WITHOUT THIS THE CEILING DOES NOT HOLD, and the report lies while it fails. planVolume classifies
// before the draft wave, but the bank-mining stop sits between the two waves and RE-SEEDS the bank
// mid-run (mining.go, the auto-continue branch), after which waverun recomputes the edit-wave snapshot.
// Every unit admitted as FREE — and free units are admitted OUTSIDE the grant, because free work costs
// nothing — was judged against a snapshot the run itself then replaced. Any of them whose injected bytes
// the new bank changes gets a fresh PAID editor call that no grant ever authorised, and the stop line
// calls it "rode along at $0". Found by the acceptance hunter, who measured four provider calls under a
// grant of one and $0.007280 of spend reported as free.
//
// The fix is not a guard but a re-plan: the free/paying question is asked again, against the snapshot
// that is now real, at the last moment before the edit wave begins — so "check before the unit" still
// holds. A unit that has become paying takes a slot if one is left; if none is, its edit does not run at
// all and it is reported as an already-delivered unit still awaiting its re-make, which is what it is.
//
// It is a no-op on the ordinary path: with no mid-run re-seed the snapshot is unchanged and every free
// unit stays free, so a book that does not mine pays nothing for this existing.
func (r *Runner) rescopeEditWave(ctx context.Context, s *volumeScope, units []editUnit, chunks []chunk.Chunk, stickySel []membank.Selection, editSnapshot string) error {
	if s == nil || editSnapshot == s.editSnapshot {
		return nil // the ground the plan was made on is still the ground the wave runs on
	}
	statuses, err := r.Store.ChunkStatusesForBook(r.Book.BookID)
	if err != nil {
		return fmt.Errorf("pipeline: re-read chunk_status for the volume ceiling's edit-wave re-plan: %w", err)
	}
	draftStages, editStages := r.waveStagesIndexed(waveDraft), r.waveStagesIndexed(waveEdit)
	draftNames, editNames := stageNameSet(draftStages), stageNameSet(editStages)
	curDraft, _, err := r.snapshotIDForWave(waveDraft)
	if err != nil {
		return fmt.Errorf("pipeline: render the draft-wave snapshot for the edit-wave re-plan: %w", err)
	}
	// The memo was dropped by the re-seed that moved the snapshot (materializeBanks clears it), so this
	// renders against the bank the edit wave will actually use.
	hashes := r.cachedRenderedContentHashes(chunks, stickySel)
	byChunk := map[chunkKey][]store.ChunkStatus{}
	for _, cs := range statuses {
		byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}] = append(byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}], cs)
	}

	moved, paid, blocked := 0, 0, 0
	for _, u := range units {
		key := chunkKey{u.Chapter, u.FirstChunkIdx}
		if !s.admitted[key] || s.class[key] != unitFree {
			continue // only the units admitted WITHOUT a slot can be wrong about being free
		}
		rows := unitRows(u, byChunk)
		if r.rowsResumeFree(rows, draftNames, editNames, curDraft, editSnapshot, hashes) {
			continue // still free under the snapshot that is now real
		}
		moved++
		s.stop.Free--
		if s.stop.granted() < s.stop.MaxUnits {
			// It costs money now, and there is grant left to pay for it: it becomes what it is — an
			// already-delivered unit being re-made.
			s.stop.Reworked++
			s.class[key] = unitRework
			paid++
			continue
		}
		// No grant left. Its edit does not run; it stays delivered and stale, which is LeftRework.
		s.editBlocked[key] = true
		s.stop.LeftRework++
		s.class[key] = unitRework
		blocked++
	}
	if moved > 0 {
		r.Log.WarnContext(ctx, "the bank moved between planning and the edit wave, so units judged FREE are no longer free: they have been re-judged against the snapshot the edit wave actually uses",
			"book", r.Book.BookID, "no_longer_free", moved, "took_a_grant_slot", paid, "held_back_for_want_of_grant", blocked,
			"planned_against", s.editSnapshot[:12], "running_under", editSnapshot[:12])
	}
	s.editSnapshot = editSnapshot
	return nil
}

// unitClass is what this run would DO to an output unit, which is the distinction the ceiling both
// admits and reports on.
type unitClass int

const (
	// unitFresh — never completed. Paying for it DELIVERS a chapter that did not exist.
	unitFresh unitClass = iota
	// unitFree — fully recorded and every row resumes or re-pins at $0. Costs nothing, so it rides along
	// regardless of the ceiling.
	unitFree
	// unitRework — fully recorded, but at least one row will be paid for again under a moved snapshot.
	// Real work and a real charge, and NOT new book: it replaces a chapter the reader already has.
	unitRework
	// unitCarried — started by an earlier run and never shipped: rows on file for some of the positions
	// this run executes, not all, and no delivery on the shipping wave. It COSTS (the missing positions
	// are provider calls), it is new book when it lands, and it takes no slot of this grant — the run that
	// started it took one (the file comment, backlog row 232).
	unitCarried
)

// unitShipped reports whether the unit still HAS shippable text — the second half of "delivered".
//
// FinalHash is the request_hash of the authoritative checkpoint, and export reads the unit's text through
// it: an `ok` row must carry one (export.go fails loud on an empty one), a flagged row carries none and
// exports "". So a final hash on the SHIPPING row is exactly "a reader could open this unit", expressed in
// the field the read models already use for it rather than in a second opinion about dispositions.
//
// ⛔ THE SHIPPING ROW AND NOT ANY ROW, and the difference is a whole class of unit. The first version of
// this helper scanned every row of the unit, DRAFT rows included — and the ordinary failure shape is a
// draft that succeeded (its row carries a final hash) under an EDIT that flagged and shipped nothing.
// That unit exports "" and its reader has no text, yet any-row would call it an already-delivered re-make
// and the stop line would print «0 unit(s) NEVER delivered» over a hole. The pack's own flagged-unit test
// did not catch it because its flag lands on the DRAFT, where no row carries a hash at all. Found by an
// adversarial pass over this pack's finished work; it is the same lie the axis exists to remove, one
// stage deeper.
//
// A unit with NO row for the shipping stage — the shape a newly added stage produces, which is the whole
// case this axis was built for — is judged by the row of the stage that shipped BEFORE it, so adding a
// stage does not retroactively un-deliver a book. That is why the lookup walks the shipping wave's stages
// from the last backwards and answers on the first one the unit actually has a row for.
func unitShipped(u editUnit, rows []store.ChunkStatus, shipStages []wavedStage) bool {
	for i := len(shipStages) - 1; i >= 0; i-- {
		for _, cs := range rows {
			if cs.Stage != shipStages[i].st.Name || cs.Chapter != u.Chapter || cs.ChunkIdx != u.FirstChunkIdx {
				continue
			}
			return cs.FinalHash != ""
		}
	}
	return false
}

// deliveredUnits is the set of output units a reader has ALREADY been told shipped, read off the
// announce-once ledger (store.AnnouncedOnceKeys) rather than re-derived from rows.
//
// WHICH WAVE COUNTS AS SHIPPING, and why the question has one answer rather than a choice. The ledger's
// key carries the WAVE, so "delivered" is a per-wave fact, and only ONE wave ships: the one that owns the
// pipeline's last stage (finalStageWave). On the ordinary editor pipeline that is the edit wave; on a
// draft-only pipeline the draft itself ships. Reading the DRAFT announcement on an editor pipeline would
// be the opposite lie and a worse one — a book whose draft wave finished and whose edit wave never ran
// (the state a signing stop, a ceiling halt or a Ctrl-C leaves) would report every unit as an
// already-delivered re-make while the reader has no text for any of them.
//
// ⚠ POSITIVE EVIDENCE ONLY. An announcement present means delivered; an announcement ABSENT means
// nothing, and the classification falls back to the row test exactly as before. That direction is
// forced, not stylistic: the ledger is only written when the line actually reached the journal file
// (MarkAnnounced), so a run whose journal could not be opened at all — an in-process caller with no trace
// id, a directory that refused the write — legitimately has no keys, and reading absence as "delivered"
// would print an unread book as re-work and hide real work from its buyer.
//
// ⚠ AND AN ANNOUNCEMENT IS NOT ENOUGH ON ITS OWN — the caller pairs it with unitShipped, and BOTH halves
// were found by something going red rather than by design.
//
// A unit_done line is written for every resolved unit, shipped or FLAGGED (waverun.go passes `shipped` as
// a payload field, not as a condition), and what was delivered can afterwards be destroyed:
// ResetChunkStages, the redrive's own primitive, DELETES the unit's chunk_status rows AND its checkpoints
// (store/chunkstatus.go). So an announcement alone answers "was a reader told about this unit", which is
// not the same as "does that unit have text". Both failure modes are the SAME lie mirrored — calling a
// unit with nothing to show for it an already-delivered re-make — and both would print «0 unit(s) NEVER
// delivered» over a book the reader cannot read.
//
// The pair is therefore: a reader was told, AND the thing they were told about still exists. The first
// version of this check had only the first half and turned a redriven unit into rework, which the zone's
// own TestARunThatRePaysNothingIsNotAskedForConsent caught; the flagged-unit half was found by walking the
// announcement path afterwards, and is why the test is unitShipped and not len(rows) > 0.
//
// ⚠ TWO RESIDUALS, named rather than implied, and both need a record nothing stores today:
//   - A pipeline that gains its FIRST editor stage moves the shipping wave from draft to edit, and its
//     already-read units have indeed never shipped an EDITED unit — they are reported fresh again. That is
//     the honest answer for the edited unit and the wrong-sounding one for the reader who already has
//     text; closing it needs a record of what the shipping wave WAS.
//   - A delivered unit whose re-make was interrupted between the waves takes a slot from each run that
//     advances it (the file comment): a never-shipped unit's slot is answered from its rows (unitCarried),
//     a re-made unit's cannot be, because a half-finished re-make and an ordinary bank-only edit move
//     leave the same rows.
//
// A read failure is REPORTED and degrades to "nothing is known to be delivered" rather than failing the
// plan: the ceiling's job is to bound money, and a projection detail must not be able to stop a run.
func (r *Runner) deliveredUnits(units []editUnit) map[chunkKey]bool {
	waveName := runevents.WaveEdit
	if r.finalStageWave() == waveDraft {
		waveName = runevents.WaveDraft
	}
	announced, err := r.Store.AnnouncedOnceKeys()
	if err != nil {
		r.Log.Warn("volume: the announce ledger could not be read, so delivery is judged by row completeness "+
			"alone and an already-delivered unit may be reported as NEW book", "book", r.Book.BookID, "err", err)
		return nil
	}
	out := make(map[chunkKey]bool, len(units))
	for _, u := range units {
		key := chunkKey{u.Chapter, u.FirstChunkIdx}
		// The emitter's own derivation, called and not re-spelled — see unitOnceKey.
		if announced[unitOnceKey(r.Book.BookID, unitWave{waveName, u.Chapter, u.FirstChunkIdx})] {
			out[key] = true
		}
	}
	return out
}

// classifyUnits sorts every output unit into the four things this run can do to it.
//
// The free/paying predicate is deliberately the one the run itself applies, read off runStage's resume
// fast-path: a stored row is served for free when its rendered content hash is unchanged AND its snapshot
// either matches the current one or is a re-pinnable bank-only move. A unit is free when every one of its
// rows is, and when every position it will execute already HAS a row — a half-done unit resumes what it
// has and pays for the rest.
//
// The fresh/rework split falls straight out of the same completeness test, which is why it costs nothing
// to make: a unit that was never fully recorded has never been delivered, so paying for it is delivery; a
// unit that WAS fully recorded has been delivered once already, so paying for it again is rework.
//
// Every uncertainty resolves to "this unit will cost", never to "this unit is free". That direction is
// forced: mistaking a paying unit for a free one lets the run spend past the ceiling, which is the defect
// the ceiling exists to prevent, while the opposite merely makes a run stop one unit early.
func (r *Runner) classifyUnits(units []editUnit, chunks []chunk.Chunk, statuses []store.ChunkStatus, stickySel []membank.Selection) (map[chunkKey]unitClass, string, error) {
	class := map[chunkKey]unitClass{}
	if len(statuses) == 0 {
		return class, "", nil // a book nothing has run is entirely fresh; skip the rendering entirely
	}
	draftStages, editStages := r.waveStagesIndexed(waveDraft), r.waveStagesIndexed(waveEdit)
	draftNames, editNames := stageNameSet(draftStages), stageNameSet(editStages)

	curDraft, _, err := r.snapshotIDForWave(waveDraft)
	if err != nil {
		return nil, "", fmt.Errorf("pipeline: render the draft-wave snapshot for the volume ceiling: %w", err)
	}
	curEdit := curDraft
	if len(editStages) > 0 {
		if curEdit, _, err = r.snapshotIDForWave(waveEdit); err != nil {
			return nil, "", fmt.Errorf("pipeline: render the edit-wave snapshot for the volume ceiling: %w", err)
		}
	}
	// The same reproduction the re-payment estimate uses (repin.go): what the run WOULD render for every
	// live position. A position whose inputs cannot be reproduced is simply absent, and an absent hash is
	// read below as "cannot conclude", i.e. the paying answer.
	hashes := r.cachedRenderedContentHashes(chunks, stickySel)

	byChunk := map[chunkKey][]store.ChunkStatus{}
	for _, cs := range statuses {
		byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}] = append(byChunk[chunkKey{cs.Chapter, cs.ChunkIdx}], cs)
	}
	// Has a READER already been told these units are done? Asked once, for the whole book, because the
	// answer is what separates "new book" from "book being re-made" — see deliveredUnits.
	delivered := r.deliveredUnits(units)
	// The stages whose rows carry the text a reader opens — the shipping wave's, last first.
	shipStages := r.waveStagesIndexed(r.finalStageWave())
	for _, u := range units {
		rows := unitRows(u, byChunk)
		key := chunkKey{u.Chapter, u.FirstChunkIdx}
		if !unitFullyRecorded(u, rows, draftStages, editStages) {
			// A position with no row is a position this run will call for — the unit COSTS, and that half
			// is unchanged. What it is NOT, when a reader has already been told this unit shipped, is NEW
			// BOOK. Row completeness answers "will this run call a provider"; it cannot answer "has this
			// unit ever been delivered", because it is re-evaluated against whatever stages the config
			// runs TODAY: add a stage and every finished unit loses a row it never had, and a fully read
			// book reports "N unit(s) NEVER delivered" and invites its buyer to purchase it again
			// (unified backlog row 232). The announce ledger answers the second question and is monotone.
			if delivered[key] && unitShipped(u, rows, shipStages) {
				class[key] = unitRework
				continue
			}
			// Nor is it a new START when an earlier run already has rows for it: it was begun and the buyer
			// got no chapter, so this run does not charge its own grant for it (backlog row 232, second
			// half; the carry is bounded by the grant — planVolume). Any row at a position this run executes
			// is the evidence — a unit cut off between the waves (a signing stop, a money halt, a Ctrl-C) or
			// inside one (some members drafted) alike. ⚠ A REDRIVE does not necessarily undo it:
			// ResetChunkStages deletes only the FLAGGED and SKIPPED stages (status.go), so redriving a unit
			// whose EDIT flagged leaves its ok DRAFT row on file and the unit stays carried. Only a unit
			// whose rows are gone from every position this run executes starts anew.
			if unitStarted(u, rows, draftStages, editStages) {
				class[key] = unitCarried
				continue
			}
			class[key] = unitFresh
			continue
		}
		if r.rowsResumeFree(rows, draftNames, editNames, curDraft, curEdit, hashes) {
			class[key] = unitFree
			continue
		}
		class[key] = unitRework
	}
	return class, curEdit, nil
}

// unitFullyRecorded reports whether EVERY position this run would execute for the unit already has a
// stored row — the completeness half of "this unit costs nothing".
//
// ⚠ IT DELIBERATELY DOES NOT USE resolveChunkState, and the difference is money. That resolver answers
// "what does a reader call this unit", and it returns ChunkFlagged the moment ANY row is flagged, before
// it ever tests whether the expected positions are all present. That is right for a progress projection
// and wrong here: a unit whose draft flagged for one member and whose EDIT row does not exist yet — the
// state every book is left in by a bank-signing stop, a Ctrl-C, or any abort in the edit wave — would be
// read as terminal, judged free, admitted without consuming a grant, and then charged for a full editor
// call. The run would pay for more units than were bought and, with Deferred still 0, would not even
// report a volume stop. Found by this pack's own adversarial pass, reproduced before it was fixed.
//
// Positions are counted, not dispositions — with ONE exception, and it is the property this predicate
// exists for: a completed unit always leaves a row for every position it has (ok, flagged, or the
// `skipped` rows recordSkippedStages writes downstream of a flag, including the all-members-flagged case
// where the editor never runs), so full coverage IS terminality and is what actually predicts "no
// provider call".
//
// ⛔ THE EXCEPTION IS A STOP MARK, AND WITHOUT IT THIS PREDICATE PREDICTS THE OPPOSITE OF WHAT IT SAYS. A
// `cancelled` or `retry_unaffordable` row is a position the next run RE-DOES, for money — so a unit whose
// last position carries one is fully COVERED and not at all terminal. Counted as recorded, it skipped the
// `unitCarried` branch below and landed in `unitRework`, which TAKES A GRANT SLOT (granted() above) and is
// queued behind fresh book — exactly the charge the carried class exists to prevent, and the thing the
// `--max-units` help text promises never happens («a unit an EARLIER run started and never shipped is
// finished outside the grant»). Measured on the mark fixture before this line: class `rework` for a unit
// whose attempt 0 was paid and whose text the export does not ship.
func unitFullyRecorded(u editUnit, rows []store.ChunkStatus, draftStages, editStages []wavedStage) bool {
	answered, _, want := unitPositionsOnFile(u, rows, draftStages, editStages)
	return answered == want
}

// unitStarted reports whether ANY position this run would execute for the unit already has a stored row —
// the evidence that an earlier run began the unit and took its slot (unitCarried). It is the same
// enumeration as unitFullyRecorded's, asked the other way round, so the two cannot disagree about what a
// position is.
func unitStarted(u editUnit, rows []store.ChunkStatus, draftStages, editStages []wavedStage) bool {
	// ⚠ ROWS, not answers — the opposite half of the pair above, and deliberately so: a STOP MARK is the
	// strongest evidence there is that an earlier run began this unit and spent money on it, while being
	// no answer at all. Asked about answers, a unit whose only row is a mark would read «never started»
	// and be charged as fresh book.
	_, started, _ := unitPositionsOnFile(u, rows, draftStages, editStages)
	return started > 0
}

// unitPositionsOnFile counts the positions this run would execute for the unit — every member × every
// draft stage, plus every edit stage at the LEADER chunk, where the read-models look for the unit's edit
// rows — and how many of them already have a stored row. A row for a stage the pipeline no longer runs is
// not a position and is not counted: it is neither evidence of a start under this pipeline nor work this
// run would do.
//
// It returns TWO counts over the same enumeration, because its two callers ask different questions of the
// same rows: `answered` is positions whose row is a verdict (will this run call a provider — see
// unitFullyRecorded), `started` is positions with ANY row (did an earlier run begin this unit — see
// unitStarted). They differ exactly on the two stop marks, and one enumeration is what keeps the pair from
// disagreeing about what a position IS.
func unitPositionsOnFile(u editUnit, rows []store.ChunkStatus, draftStages, editStages []wavedStage) (answered, started, want int) {
	type pos struct {
		chapter, chunkIdx int
		stage             string
	}
	stored := make(map[pos]bool, len(rows))
	answers := make(map[pos]bool, len(rows))
	for i := range rows {
		cs := rows[i]
		p := pos{cs.Chapter, cs.ChunkIdx, cs.Stage}
		stored[p] = true
		if cs.Disposition != string(DispFlagged) || resolvedForResume(&cs) {
			answers[p] = true
		}
	}
	count := func(p pos) {
		want++
		if stored[p] {
			started++
		}
		if answers[p] {
			answered++
		}
	}
	for _, m := range u.Members {
		for _, ws := range draftStages {
			count(pos{m.Chapter, m.ChunkIdx, ws.st.Name})
		}
	}
	for _, ws := range editStages {
		count(pos{u.Chapter, u.FirstChunkIdx, ws.st.Name})
	}
	return answered, started, want
}

// rowsResumeFree is the per-row half of the predicate above.
func (r *Runner) rowsResumeFree(rows []store.ChunkStatus, draftNames, editNames map[string]bool,
	curDraft, curEdit string, hashes map[chunkKey]map[string]string) bool {
	for _, cs := range rows {
		if cs.Disposition == string(DispSkipped) {
			continue // a skipped stage is re-derived from the flag above it; it never reaches a provider
		}
		var cur string
		var w wave
		switch {
		case draftNames[cs.Stage]:
			cur, w = curDraft, waveDraft
		case editNames[cs.Stage]:
			cur, w = curEdit, waveEdit
		default:
			continue // a stage this pipeline no longer runs cannot cost anything (rebill.go's rule)
		}
		h, ok := hashes[chunkKey{cs.Chapter, cs.ChunkIdx}][cs.Stage]
		if !ok || h != cs.ContentHash {
			return false // the wire bytes moved, or could not be reproduced — either way, a fresh call
		}
		if cs.SnapshotID != cur && !r.repinnable(cs.SnapshotID, cur, w) {
			return false
		}
		// ⛔ THE SAME QUESTION runStage ASKS, ASKED THROUGH THE SAME PREDICATE. A `cancelled` row records
		// a stop, not an answer: the resume re-does that call, for money. Read as free, a unit whose last
		// row is one needed no slot — so a grant of one unit paid for two and the volume report, which
		// speaks only when something was carried, said nothing at all.
		//
		// ⚠ AND IT IS ASKED LAST, AFTER the switch above has dropped the stages this pipeline no longer
		// runs. Asked first — where it was written — it made a unit PAID for a cancelled row of a stage
		// that will never be called again, which is the same error in the opposite direction: an operator
		// who edits the pipeline over a stopped run would spend volume slots on nothing.
		//
		// ⚠ SINCE THE COMPLETENESS TEST LEARNED THE SAME QUESTION, THIS IS THE SECOND LINE AND NOT THE
		// FIRST: unitFullyRecorded counts positions whose row is an ANSWER, so a unit carrying a stop mark
		// no longer reaches this function at all (it is `carried` before it gets here). The guard stays
		// because it is the one that makes the ANSWER to «does this row resume for free» correct on its own
		// terms — it is pinned directly (TestACancelledRowIsNotAFreeResume) rather than through the
		// classifier — and because a caller that ever asks this without asking completeness first must not
		// be handed the wrong answer.
		if !resolvedForResume(&cs) {
			return false
		}
	}
	return true
}