package terminology import "strings" // evidence.go: the EVIDENCE view of a candidate — every context the source holds for a surface, as opposed // to the subset AttachKWIC hands the model. // // ⛔ WHY THIS IS NOT KWIC, AND WHY THE TWO MUST NOT SHARE A KNOB. KWIC answers "how much context does the // model need to translate this well", and it is a PROMPT-SHAPE question: it is capped (maxPer), it takes // the FIRST occurrences in (chapter, chunk, offset) order, and the day a stratified selection lands it // will take a different subset of the same text. EvidenceWindows answers a different question — "what does // the book say about this surface" — so it is uncapped and selection-free by construction. // // The distinction is money. A fingerprint built over the KWIC subset would move for every book on earth // the moment anyone turned kwic_per_term, changed the selection rule, or widened the model's context, and // each of those moves would re-buy a whole bank out of a LIFETIME role budget. Built over every // occurrence, it moves only when the SOURCE around the surface moves — which is the event that genuinely // invalidates a decision. // EvidenceWindows returns the ±width-rune context around EVERY occurrence of key in nsource, in document // order, non-overlapping — the same window shape KWIC shows, without the cap and without the selection. // // nsource is the book's NORMALIZED source (the same space keys live in), so a caller must not pass raw // text: a surface would then be looked up in one space and matched in another. // // An empty result means the surface does not occur in the source at all, which is honest rather than // empty-by-accident: it is what a draft-invented term looks like, and a caller that hashes it gets a // value distinct from every attested surface. // // ⛔ IT DOES NOT REUSE window(), AND THAT IS ABOUT COST RATHER THAN TASTE. window() re-derives the rune // view of its input on every call — `[]rune(s)` plus two prefix counts — which is free at the size its own // comment declares ("the strings are chunk-sized") and quadratic here, where the input is the WHOLE book // and the call happens once per occurrence. Measured on a synthetic source at the occurrence density of the // bought cold run: a source four times longer took FIFTEEN times as long, and a million-rune book would // have spent minutes — plus tens of gigabytes of garbage — inside a $0 pass that runs before every // purchase, whether or not it ends up saving anything. So the source is converted ONCE per call and the // windows are cut from that view. // // ⚠ The window CONTENT is identical to window()'s by construction: same ±width in runes, same clamp at the // ends, same TrimSpace. That equality is pinned rather than asserted here, because a fingerprint built on // a subtly different window would be a silent, book-wide re-purchase the first time anything compared them. func EvidenceWindows(nsource, key string, width int) []string { if key == "" || width <= 0 { return nil } runes := []rune(nsource) // Byte offset → rune index, built by ONE forward pass. `range` over a string visits rune STARTS, which // is exactly the set of offsets this function indexes: strings.Index of a valid UTF-8 key inside valid // UTF-8 text returns a rune-aligned offset, and so does that offset plus the key's byte length. // Continuation bytes are left at zero on purpose — indexing one would be a bug in the caller, not a // case to handle here, and giving them a plausible value would hide it. runeAt := make([]int, len(nsource)+1) n := 0 for bi := range nsource { runeAt[bi] = n n++ } runeAt[len(nsource)] = n var out []string for from := 0; from < len(nsource); { i := strings.Index(nsource[from:], key) if i < 0 { break } at := from + i lo := runeAt[at] - width if lo < 0 { lo = 0 } hi := runeAt[at+len(key)] + width if hi > len(runes) { hi = len(runes) } out = append(out, strings.TrimSpace(string(runes[lo:hi]))) from = at + len(key) } return out }