textmachine/backend/internal/runevents/runevents.go

// Package runevents is the ENGINE side of the run-event seam (row 103): the vocabulary of the
// NDJSON stream the platform tails, and the append-only journal it is written to.
//
// The transport is ratified and not open (D39.106 §2, research/25 §Форма): the engine is a transient
// systemd unit per run, the platform is NOT its parent, and the stream is `events.jsonl` in the BOOK's
// directory — an outbox projection of rows the engine has already committed to its SQLite. Delivery is
// at-least-once; a re-read line is normal (D39.119 п.3 / PD-105). What this package owns is only the
// FORM: the envelope, the payloads and the file discipline. WHEN an event happens is the driver's
// (internal/pipeline), and the durable sequencing is the store's (internal/store, events_outbox).
//
// The payload shapes mirror the platform's reader (`platform/internal/ingest/events.go`), which is the
// platform's PROPOSAL written as code, "so the engine zone can answer it with a diff". Where this file
// differs from that one, the difference is DELIBERATE and named in the doc comment of the type:
// `Ceiling` adds `scope` and `Finished` widens its outcome vocabulary. (`Progress` was going to omit
// `eta_seconds`; measurement showed omission NULLs the consumer's column, so it is carried — see there.)
//
// Nothing here is language-, pair- or book-specific, and nothing may become so: an event carries
// counters, ordinals and engine-side enums only (общность §0.1 — a pair that is not in the repository
// must stream identically without a line of Go changing).
package runevents

import (
	"encoding/json"
	"fmt"
	"math"
	"time"
)

// StreamVersion is the version of the stream this build writes. The rule is terraform's, ratified by
// D39.85: a MINOR bump adds fields and event types — a reader ignores the ones it does not know; a
// MAJOR bump is refused by the reader outright. So: adding a field or an event type bumps the minor,
// changing what an existing field MEANS bumps the major.
//
// 1.1, and the bump is the point rather than a formality. This build adds a field (`Ceiling.Scope`) and
// two outcome values, so by the rule above it is not 1.0 — and a version is a fact about the BYTES on
// the wire, not about whether the vocabulary diff has been accepted yet. Leaving it at 1.0 would make
// two streams that differ in content claim the same version, which is the one thing a version exists to
// prevent; the reader compares only the major, so the bump costs nothing and is safe in both directions
// (if `scope` is later declined, removing it is another minor — that is what minors are for).
const StreamVersion = "1.1"

// Type is the event name.
type Type string

const (
	// TypeHello is always the first line a process writes: the version handshake.
	TypeHello Type = "hello"
	// TypeProgress carries the per-phase counters. Per phase because a unit is done only once its edit
	// resolved, so one end-to-end counter reads zero for the whole draft wave (row 99 / D39.122 п.2б).
	TypeProgress Type = "progress"
	// TypeUnitDone is one resolved output unit.
	TypeUnitDone Type = "unit_done"
	// TypeBankStop is the book-wide signing stop before the edit wave.
	TypeBankStop Type = "bank_stop"
	// TypeCeiling is the resumable halt on a spend ceiling: the fact only, no figures.
	TypeCeiling Type = "ceiling"
	// TypeSpend is the cumulative spend counter.
	TypeSpend Type = "spend"
	// TypeFinished is the terminal line of a run that ended on purpose.
	TypeFinished Type = "finished"
)

// Envelope is one line of the stream. Seq is per PROCESS and starts at 1: a resumed run is a NEW
// process that appends a second hello to the same file and numbers from 1 again, which is why the
// ratified idempotency key is (engine_run_id, seq) and not the platform's run id.
type Envelope struct {
	Seq  int64           `json:"seq"`
	Type Type            `json:"type"`
	Time time.Time       `json:"time"`
	Data json.RawMessage `json:"data"`
}

// Hello is the handshake payload.
type Hello struct {
	StreamVersion string `json:"stream_version"`
	// EngineRunID is this PROCESS's identity — the trace id, which since row 102 the caller may supply
	// (TM_TRACE_ID). It must be non-empty: it is half of the idempotency key, and an empty one would
	// collapse every run's events into one namespace instead of failing.
	EngineRunID string `json:"engine_run_id"`
	BookID      string `json:"book_id"`
	// ChunkerVersion lets a reader notice that the chapter manifest it persisted was produced by a
	// different chunker — the case that silently re-numbers chapters.
	ChunkerVersion string `json:"chunker_version"`
}

// Counter is one phase's done/total pair, in OUTPUT UNITS — the granularity every engine read model
// counts in (status.go, the manifest). Counting the draft wave in CHUNKS instead would put the stream
// and the `status --json` resync on two different scales, and the platform folds both into one column.
type Counter struct {
	Done  int `json:"done"`
	Total int `json:"total"`
}

// Progress is the run's per-wave counters. Total is 0 for a wave this pipeline does not have, which is
// how a reader tells "no such phase" from "none of it is done yet".
//
// ETASeconds is carried, and the reason is worth recording because the first version of this pack
// omitted it on an argument that MEASUREMENT destroyed. The argument was: the engine has one ETA
// definition already (pipeline/status.go, a book-lifetime mean over stored latencies), computing it here
// would cost a store aggregate per unit, and the field is optional to the reader anyway. The last clause
// is false. The consumer's progress handler ASSIGNS the column unconditionally —
// `update runs set … eta_seconds = $6` with `etaOrNil(p.ETASeconds)` (platform/internal/pgstore/sink.go)
// — so an absent field decodes to 0 and NULLS the estimate on every single progress line, erasing what
// the `status --json` resync had just written. Omitting a field is not leaving it alone.
//
// So it is emitted, computed from THIS RUN's own throughput (see pipeline/events.go). That deliberately
// differs from status.go's book-lifetime mean: this one is what the run is achieving now, it costs no
// query, and the alternative on the table was not "a second opinion" but "no estimate at all".
type Progress struct {
	Draft      Counter `json:"draft"`
	Edit       Counter `json:"edit"`
	ETASeconds int     `json:"eta_seconds,omitempty"`
}

// UnitDone is one resolved output unit. Chapter is the engine's dense 1-based ordinal and Unit is the
// unit's LEADER chunk index — together the join key the manifest publishes as `first_chunk_idx`, which
// is how a reader maps this onto its own opaque ids.
//
// Shipped and Flagged are BOTH carried and neither implies the other: a flagged unit legally ships text
// (a cosmetic sanitizer strip, a c-lite member drop), and the pair is exactly the contract's derivation
// of unit state.
//
// It is emitted ONLY for a unit THIS process resolved — a unit already resolved when the process started
// is walked again at $0 on every resume and re-announcing it would make a counting reader count it twice
// (see pipeline/events.go, `resolvedAtStart`).
type UnitDone struct {
	Chapter int    `json:"chapter"`
	Unit    int    `json:"unit"`
	Wave    string `json:"wave"` // draft | edit
	Shipped bool   `json:"shipped"`
	Flagged bool   `json:"flagged"`
	// Reason is the ENGINE's flag reason (glossary_miss, sanitizer_stripped, …) — stored by a reader,
	// never projected verbatim, so a reason it has never heard of still gets a neutral phrase.
	Reason string `json:"reason,omitempty"`
}

// BankStop is the signing stop. The full table travels as an artifact (row 101), not through the
// stream: a thousand rows are not an event.
type BankStop struct {
	TermsProposed int `json:"terms_proposed"`
}

// Ceiling is the ceiling halt. It carries the FACT and nothing else: money never reaches the platform's
// wire or its INFO logs (D39.84), and the stop is resumable, so it is NOT a failure — which is the whole
// of PD-113.
//
// ⚠ DIFF against the platform's proposal: `scope` is added. It is not money — it names WHICH ceiling
// stopped the run, book or day — and it closes the diagnosis half of PD-157: a book whose `day_usd` the
// platform never chose stops the run on a limit the platform cannot even see, and today it cannot tell
// that from the ceiling it set itself.
type Ceiling struct {
	Halted bool   `json:"halted"`
	Scope  string `json:"scope"` // book | day
}

// Spend is the freshness channel for money and ONLY that: the balance is protected by the platform's
// hold and by the per-book ceiling the engine enforces itself, so a lost tail costs an indicator its
// accuracy and never costs an account its correctness. Building enforcement on this event is forbidden —
// the stream is at-least-once and a crash truncates it (D39.106 §2: холд+потолок = защита, события =
// свежесть).
//
// CUMULATIVE, not a delta, so a re-delivered line is harmless to a reader that keeps the maximum. It is
// the book's LIFETIME committed spend — the same figure `status --json` reports as `committed_usd`, so
// the two channels can never quote different numbers. Integer micro-USD: money never travels as a float
// (PD-79), and the engine's ledger is a lower bound, so the conversion rounds UP.
type Spend struct {
	CommittedMicroUSD int64 `json:"committed_micro_usd"`
}

// Finished is the terminal line of a run that ended on purpose. Its absence is meaningful: a stream
// without it ended without finishing a book run — a crash, or a command that never started one.
//
// ⚠ DIFF against the platform's proposal: the outcome vocabulary gains `ceiling` and `stopped`. The
// proposal has clean|flagged|bank_stop|failed, and a ceiling halt fits none of them — `failed` is exactly
// what the contract forbids for a resumable stop (PD-113), and leaving the stream unterminated would make
// "stopped on purpose" indistinguishable from "truncated by a crash", which is the same confusion one
// level down. `stopped` is its sibling for a caught SIGTERM (PD-152).
type Finished struct {
	Outcome string `json:"outcome"`
}

// The outcomes this engine writes. They mirror the shell contract of cmd/tmctl (0/2/3/4/5/1) so a reader
// never has to do exit-code archaeology over a stream that ended cleanly.
const (
	OutcomeClean    = "clean"
	OutcomeFlagged  = "flagged"
	OutcomeBankStop = "bank_stop"
	OutcomeCeiling  = "ceiling"
	OutcomeStopped  = "stopped"
	OutcomeFailed   = "failed"
)

// The waves a unit can be resolved by.
const (
	WaveDraft = "draft"
	WaveEdit  = "edit"
)

// The ceiling scopes.
const (
	ScopeBook = "book"
	ScopeDay  = "day"
)

// Line renders one journal line — the envelope, WITHOUT its newline. The bytes it returns are the line:
// they are stored verbatim and re-projected verbatim, because a reader compares a re-read line against
// the sha256 it recorded and a re-render with a fresher timestamp would read as a payload conflict
// (tail.go: ErrPayloadConflict) and quarantine the projection.
func Line(seq int64, t Type, at time.Time, data any) ([]byte, error) {
	payload, err := json.Marshal(data)
	if err != nil {
		return nil, fmt.Errorf("runevents: marshal %s payload: %w", t, err)
	}
	line, err := json.Marshal(Envelope{Seq: seq, Type: t, Time: at.UTC(), Data: payload})
	if err != nil {
		return nil, fmt.Errorf("runevents: marshal %s envelope: %w", t, err)
	}
	return line, nil
}

// MicroUSD converts a ledger figure to the integer micro-USD the seam carries, rounding UP: the ledger
// is a lower bound on what a provider billed (a 2xx whose body did not decode is settled at its
// estimate), so the seam must never round the shortfall away. A negative figure cannot exist in the
// ledger and is clamped rather than sign-extended into a nonsense counter.
func MicroUSD(usd float64) int64 {
	if !(usd > 0) { // also catches NaN
		return 0
	}
	return int64(math.Ceil(usd*1e6 - 1e-6))
}