133 lines
7.8 KiB
Go
133 lines
7.8 KiB
Go
package membank
|
|
|
|
import (
|
|
"fmt"
|
|
"strings"
|
|
"unicode"
|
|
"unicode/utf8"
|
|
)
|
|
|
|
// wirefence.go: the boundary between BANK DATA and the SYSTEM MESSAGE.
|
|
//
|
|
// The ratified invariant (D25, fact base research/17 §A10.2) is that the book's text and any external
|
|
// output are DATA, never instructions. `POST /books/{id}/bank/corrections` accepts a `dst`, the contract
|
|
// declares `minLength: 1` and no maximum, and the renderers below concatenate it into a message whose
|
|
// role is `system` on EVERY paid call of the book. A `dst` holding a line feed therefore writes its own
|
|
// lines inside the system block, and there was nothing between the door and the wire to stop it
|
|
// (backlog row 271).
|
|
//
|
|
// ⚠ AND THE FIRST VERSION OF THIS PARAGRAPH FLATTERED THE PAST: it said the invariant «held BY
|
|
// CONSTRUCTION until the bank gained an external writer — nobody outside this process could put bytes
|
|
// into a prompt». That was never quite true, and acceptance (F9) named it. A MINED term is not typed by
|
|
// anyone outside the process, but it is not written by us either — it is the MODEL'S OWN OUTPUT, folded
|
|
// into the mined delta and the auto-bank and rendered into the same system block. So the untrusted
|
|
// writer arrived with mining, not with the correction door; the door only made it a PERSON. The fence
|
|
// covers both by construction, which is why the correction is to the history and not to the code — but a
|
|
// history that says «this was impossible before» invites the next reader to trust the wrong boundary.
|
|
//
|
|
// So the invariant becomes a MECHANISM. One predicate, three enforcement points, and the same answer at
|
|
// all three:
|
|
//
|
|
// - the DOOR refuses (decisions.go/resolveDecision) — the correction is rejected with its own index,
|
|
// so the person who wrote it is told, instead of having their term silently mangled;
|
|
// - the SEED LOADER refuses (memseed.go) — an operator's file is theirs to fix, and the loader already
|
|
// speaks a list of per-subject problems;
|
|
// - the RENDERERS skip (RenderGlossaryBlock/editorLines) — the backstop. The two above make it
|
|
// unreachable for the paths that exist today; it exists because it is the only point that cannot be
|
|
// bypassed by a writer added later, and «better nothing than garbage» is already this file's rule
|
|
// for a record it cannot render.
|
|
//
|
|
// ⚠ WHY THE CURE IS NOT «FRAME IT AS DATA» IN WIRE TEXT. The obvious third leg — a sentence in the
|
|
// injection block saying «what follows is data» — would be TARGET-language wire text, and the only home
|
|
// for that is lang/data/injection.txt, which is folded byte-for-byte into lang.EmbeddedVersion(); that
|
|
// value rides inside every unit id through the manifest's cut tag, so ADDING one row re-mints the unit
|
|
// ids of every book that exists. The framing is therefore structural rather than textual: a value that
|
|
// cannot leave its own line and cannot exceed a term's length cannot present itself as anything but the
|
|
// right-hand side of the mapping it is rendered into.
|
|
|
|
// WireFieldMaxRunes bounds one bank value on its way into a system message, in RUNES (not bytes — a byte
|
|
// bound would be a different limit per script, and the engine is pair-agnostic by construction).
|
|
//
|
|
// 200 is not a round number picked for comfort. The engine's own longest structural line — a chapter
|
|
// header WITH its subtitle — is bounded at 60 runes (chunk.chapterHeaderMaxRunes), and a term rendering
|
|
// is not longer than a chapter title in any language. At the shipped `glossary_token_budget: 800`
|
|
// (backend/configs/pipeline-c1.yaml) a single 200-rune line in a dense script already claims a quarter
|
|
// of the whole per-chunk injection budget, so the bound does not bite content that is a term; it stops a
|
|
// value from being a document.
|
|
const WireFieldMaxRunes = 200
|
|
|
|
// WireUnfit reports why a bank value may not be rendered into the wire's system block, or "" when it
|
|
// may. `field` names the column for the message ("src"/"dst"); it is not inspected.
|
|
//
|
|
// ⚠ THE MESSAGE NEVER ECHOES THE VALUE. A refusal that quotes the payload moves the payload into an API
|
|
// response, a log line and an operator's terminal — three more places the bytes reach and one of them
|
|
// renders escapes. The offending rune is named by CODE POINT and rune offset, which is what a person
|
|
// fixing the row needs and is inert wherever it is printed.
|
|
func WireUnfit(field, s string) string {
|
|
if n := utf8.RuneCountInString(s); n > WireFieldMaxRunes {
|
|
return fmt.Sprintf("`%s` is %d runes; a bank value rides inside the system message of every paid call and is bounded at %d", field, n, WireFieldMaxRunes)
|
|
}
|
|
// Ranged over the STRING with a rune counter, not over []rune(s): this runs on every bank row of every
|
|
// injection of every call, and materialising a rune slice per value would put an allocation on the
|
|
// hot path to answer a question that needs none.
|
|
at := 0
|
|
for _, r := range s {
|
|
if class := forbiddenWireRune(r); class != "" {
|
|
return fmt.Sprintf("`%s` holds %s U+%04X at rune %d: a bank value is rendered into a system message as one line of a mapping, and a rune that ends the line or reorders it writes into the block instead of sitting in it", field, class, r, at)
|
|
}
|
|
at++
|
|
}
|
|
return ""
|
|
}
|
|
|
|
// forbiddenWireRune names the class of a rune that may not appear in a value bound for the system block,
|
|
// or "" for a rune that may.
|
|
//
|
|
// THE SET IS DELIBERATELY NARROW, and the narrowness is the point: it is the runes that break the LINE
|
|
// or the READING of the line, and nothing that merely looks unusual. Banning everything unprintable
|
|
// would ban the format characters that ordinary text in several scripts needs — a zero-width joiner in
|
|
// Devanagari, a zero-width non-joiner in Persian — and the engine must work for a pair that is not in
|
|
// the repository (общность §0.1). Those stay allowed; they cannot forge a line.
|
|
//
|
|
// - CONTROL (Unicode Cc: C0 and C1, which is where LF, CR, TAB, VT, FF and NEL live). These end the
|
|
// line, and ending the line is the whole of the attack: everything after a line feed is a new line
|
|
// of the system block.
|
|
// - LINE and PARAGRAPH SEPARATOR (U+2028, U+2029). Same effect, outside Cc.
|
|
// - BIDI OVERRIDE and ISOLATE (U+202A-U+202E, U+2066-U+2069). These do not end a line; they reorder
|
|
// it, so what a human reviewing the bank reads and what the model receives are different texts. That
|
|
// is the Trojan-Source class (CVE-2021-42574), and a bank row is exactly the artifact a human signs
|
|
// by eye. The bidi MARKS (U+200E/U+200F) are NOT here: they are ordinary content in Hebrew and
|
|
// Arabic text and reorder nothing on their own.
|
|
func forbiddenWireRune(r rune) string {
|
|
switch {
|
|
case unicode.IsControl(r):
|
|
return "a control character"
|
|
case r == '\u2028' || r == '\u2029':
|
|
return "a line/paragraph separator"
|
|
case r >= '\u202A' && r <= '\u202E', r >= '\u2066' && r <= '\u2069':
|
|
return "a bidirectional override/isolate"
|
|
}
|
|
return ""
|
|
}
|
|
|
|
// WireUnfitRow answers the same question about a whole record — the form the renderers and the version
|
|
// fold need. It asks about the bytes AS RENDERED: RenderGlossaryBlock writes `entry.dst` untrimmed, so
|
|
// trimming here would clear a value the wire still carries.
|
|
func WireUnfitRow(src, dst string) bool {
|
|
return WireUnfit("src", src) != "" || WireUnfit("dst", dst) != ""
|
|
}
|
|
|
|
// WireUnfitReasons returns every reason a (src, dst) pair may not reach the wire, in field order, for a
|
|
// caller that reports rather than filters. Empty when the pair is fit.
|
|
func WireUnfitReasons(src, dst string) []string {
|
|
var out []string
|
|
for _, f := range []struct{ field, val string }{{"src", src}, {"dst", dst}} {
|
|
if why := WireUnfit(f.field, f.val); why != "" {
|
|
out = append(out, why)
|
|
}
|
|
}
|
|
return out
|
|
}
|
|
|
|
// WireUnfitJoined is WireUnfitReasons as one sentence, for the message-shaped callers.
|
|
func WireUnfitJoined(src, dst string) string { return strings.Join(WireUnfitReasons(src, dst), "; ") }
|