Replace the placeholder chunker with a real sentence-aware source segmenter, txt/epub ingest, and ruby-reading capture persisted to a new v4 table

2026-07-05 02:41:57 +03:00 · 2026-07-05 02:41:57 +03:00 · b26e4df465
commit b26e4df465
parent 073ed62791
11 changed files with 1582 additions and 99 deletions
--- a/backend/internal/pipeline/chunker.go
+++ b/backend/internal/pipeline/chunker.go
@ -1,18 +1,32 @@
 package pipeline

-import "strings"
+import (
+	"strings"
+	"unicode"
+)

-// chunker.go: the Веха-2 segmenter SEAM. Its only job right now is to turn a
-// normalized source into an ordered list of (chapter, chunk) units so the runner
-// loop, resume and disposition machinery can be exercised over MANY chunks — the
-// hardcoded "chapter=1/chunk=0" of Фаза 0 is gone.
+// chunker.go: the шаг-3a SOURCE segmenter. It turns the per-chapter normalized
+// text produced by ingest.go into an ordered list of (chapter, chunk) units for
+// the runner loop. It replaced the Веха-2 placeholder (chapter split on \f +
+// blank-line paragraph packing to a CHAR budget) with real segmentation: pack
+// whole paragraphs to a ~1–2k-TOKEN budget, and when a single paragraph exceeds
+// the budget descend to SENTENCE boundaries — never splitting a sentence.
 //
-// This is a deliberate PLACEHOLDER. The real linguistic chunker of шаг 3
-// (sentence segmentation, read-only overlap, ruby name-reading extraction for
-// ja) replaces SplitChunks WITHOUT touching the loop — it only has to keep the
-// Chunk contract and bump chunkerVersion. chunkerVersion is folded into the
-// snapshot (render.go), so changing the segmentation is an explicit
-// re-translation (a loud --resnapshot), never a silent cache miss (R6/D5.2).
+// Contract kept intact for the loop (runner.go depends on it): SplitChunks emits
+// Chunk{Chapter,ChunkIdx,Text}; ChunkIdx resets to 0 per chapter; the function is
+// a PURE, deterministic function of its input (no time/rand, no map iteration).
+// Its behavior is versioned by chunkerVersion (render.go), folded into the job
+// snapshot — so changing the segmentation is a loud --resnapshot re-translation,
+// never a silent cache miss (§3.4/R6/D5.2). The chunk target is a CONST covered by
+// that version (Р2 = code); were it ever made config-tunable it MUST be folded into
+// the top-level snapshot like coverageSnap (§7d), not left to the version string.
+//
+// IMPORTANT: this sentence splitter is a SEPARATE, independent piece of logic from
+// coverage.go's splitSentences. That one is a bit-for-bit port of the Python oracle
+// (refusal_bench.py) over the RUSSIAN OUTPUT and is Python-locked (D12 Q1). THIS one
+// segments the SOURCE (zh/ja/en), is versioned by chunkerVersion (Р2 = code), and is
+// free to be more faithful (closing-bracket absorption, an abbreviation guard). Do
+// not couple them.

 // Chunk is one ordered unit of translation work.
 type Chunk struct {
@ -21,68 +35,357 @@ type Chunk struct {
 	Text     string
 }

-// chapterSep splits the source into chapters. Form feed (U+000C) is the ASCII
-// "page/section break" — semantically a chapter boundary, invisible in prose,
-// and untouched by NormalizeSource (BOM/CRLF/NFC/outer-trim leave an internal
-// \f intact). A source with no \f is a single chapter (Фаза-0 backward compat:
-// the one-file example stays one chapter).
-const chapterSep = "\f"
+// targetChunkTokens is the packing budget: paragraphs (or, inside an oversize
+// paragraph, sentences) are greedily packed until the WHOLE chunk's estimated
+// tokens (EstimateTokens' formula, applied once — see appendChapterChunks) would
+// exceed this. ~1500 sits in the plan's "1–2k токенов по границам абзацев" corridor
+// (02-mvp Фаза 1); a chunk of ordinary prose lands well above the coverage gate's
+// 500-char floor, so the excision gate is not silently disabled (§7b). A chunk can
+// still fall under the floor only when a whole chapter (or its tail) is genuinely
+// that short — a documented boundary. A const, not config: the size is a code
+// segmentation rule (Р2), covered by chunkerVersion.
+const targetChunkTokens = 1500

-// targetChunkChars is the placeholder chunk size (≈ the plan's "1–2k tokens";
-// for a CJK source ~1 char ≈ 1 token). Paragraphs are packed up to this size and
-// never split, so the 264-char example remains a single chunk.
-const targetChunkChars = 1500
-
-// SplitChunks segments a NORMALIZED source (post-NormalizeSource) into an ordered
-// chunk list. Rule: split on \f into 1-based chapters; within a chapter pack
-// blank-line paragraphs greedily into chunks of ≤ targetChunkChars, never
-// splitting a paragraph (a paragraph larger than the target is its own chunk);
-// drop empty pieces. Fully deterministic.
-func SplitChunks(source string) []Chunk {
+// SplitChunks segments the ordered, per-chapter NORMALIZED text of a Document
+// (ingest.go already applied NormalizeSource and split chapters) into an ordered
+// chunk list. Chapters that yield no text (blank/whitespace-only, e.g. an epub
+// cover/nav page) do NOT consume a chapter number, so numbering stays dense over
+// real content (Фаза-0 backward compat). Fully deterministic.
+func SplitChunks(chapters []string) []Chunk {
 	var out []Chunk
 	chapterNo := 0
-	for _, chapRaw := range strings.Split(source, chapterSep) {
-		paras := splitParagraphs(chapRaw)
+	for _, chapText := range chapters {
+		paras := splitParagraphs(chapText)
 		if len(paras) == 0 {
-			continue // an empty chapter block does not consume a chapter number
+			continue // an empty chapter does not consume a chapter number
 		}
 		chapterNo++
-		chunkIdx := 0
-		var buf strings.Builder
-		flush := func() {
-			if buf.Len() == 0 {
-				return
-			}
-			out = append(out, Chunk{Chapter: chapterNo, ChunkIdx: chunkIdx, Text: buf.String()})
-			chunkIdx++
-			buf.Reset()
-		}
-		for _, p := range paras {
-			// Start a new chunk if adding this paragraph would exceed the target
-			// and the current chunk is non-empty (never split a paragraph).
-			if buf.Len() > 0 && buf.Len()+len("\n\n")+len(p) > targetChunkChars {
-				flush()
-			}
-			if buf.Len() > 0 {
-				buf.WriteString("\n\n")
-			}
-			buf.WriteString(p)
-		}
-		flush()
+		out = appendChapterChunks(out, chapterNo, paras)
 	}
 	return out
 }

-// splitParagraphs breaks a chapter block on blank lines and trims each paragraph,
-// dropping empties. Deterministic; whitespace-only paragraphs vanish.
-func splitParagraphs(block string) []string {
+// appendChapterChunks packs one chapter's paragraphs into chunks and appends them.
+// Rule: pack whole paragraphs while the running token estimate stays within
+// targetChunkTokens (prefer paragraph boundaries); a paragraph that alone exceeds
+// the budget is flushed on its own and then SENTENCE-packed (never splitting a
+// sentence; a single sentence over budget is its own chunk).
+//
+// The running budget is tracked as ADDITIVE char-class counts (CJK vs other),
+// applying EstimateTokens' formula (and its 16-token floor) ONCE to the whole
+// chunk — NOT by summing per-unit EstimateTokens. Summing per-unit floors every
+// tiny paragraph up to 16 tokens, so a dialogue chapter of many short lines would
+// flush a chunk at ~⅓ its true token content, needlessly dropping real content
+// under the coverage gate's 500-char floor (self-review finding). Counting classes
+// and flooring once reproduces EstimateTokens(joined text) exactly (the "\n\n"
+// separators are whitespace, which EstimateTokens ignores).
+func appendChapterChunks(out []Chunk, chapterNo int, paras []string) []Chunk {
+	chunkIdx := 0
+	var buf []string // paragraphs accumulated for the current chunk
+	var bufCJK, bufOther int
+	flush := func() {
+		if len(buf) == 0 {
+			return
+		}
+		if text := strings.TrimSpace(strings.Join(buf, "\n\n")); text != "" {
+			out = append(out, Chunk{Chapter: chapterNo, ChunkIdx: chunkIdx, Text: text})
+			chunkIdx++
+		}
+		buf = buf[:0]
+		bufCJK, bufOther = 0, 0
+	}
+	for _, p := range paras {
+		pCJK, pOther := tokenClassCounts(p)
+		if estTokensFrom(pCJK, pOther) > targetChunkTokens {
+			// Oversize paragraph: close any open chunk at this paragraph boundary,
+			// then split the paragraph at sentence boundaries into its own chunks.
+			flush()
+			for _, sub := range packSentences(p) {
+				if text := strings.TrimSpace(sub); text != "" {
+					out = append(out, Chunk{Chapter: chapterNo, ChunkIdx: chunkIdx, Text: text})
+					chunkIdx++
+				}
+			}
+			continue
+		}
+		// Normal paragraph: start a new chunk if adding it would exceed the budget
+		// and the current chunk is non-empty (never split a paragraph here).
+		if len(buf) > 0 && estTokensFrom(bufCJK+pCJK, bufOther+pOther) > targetChunkTokens {
+			flush()
+		}
+		buf = append(buf, p)
+		bufCJK, bufOther = bufCJK+pCJK, bufOther+pOther
+	}
+	flush()
+	return out
+}
+
+// packSentences splits one oversize paragraph into sentence segments (lossless
+// tiling — concatenation reproduces the paragraph) and packs consecutive segments
+// into ≤ targetChunkTokens sub-chunks, never splitting a sentence. A single
+// sentence larger than the budget becomes its own (oversize) sub-chunk — the only
+// way to honor "never split a sentence". Returns the sub-chunk texts in order. Uses
+// the same additive char-class budget as appendChapterChunks (floor applied once).
+func packSentences(paragraph string) []string {
+	segs := splitSourceSentences(paragraph)
+	var chunks []string
+	var buf strings.Builder
+	var bufCJK, bufOther int
+	flush := func() {
+		if buf.Len() == 0 {
+			return
+		}
+		chunks = append(chunks, buf.String())
+		buf.Reset()
+		bufCJK, bufOther = 0, 0
+	}
+	for _, s := range segs {
+		sCJK, sOther := tokenClassCounts(s)
+		if buf.Len() > 0 && estTokensFrom(bufCJK+sCJK, bufOther+sOther) > targetChunkTokens {
+			flush()
+		}
+		buf.WriteString(s)
+		bufCJK, bufOther = bufCJK+sCJK, bufOther+sOther
+	}
+	flush()
+	return chunks
+}
+
+// tokenClassCounts counts the two token-estimate character classes of EstimateTokens
+// (render.go): CJK ideographs/kana/hangul, and "other" non-space runes (whitespace
+// folds into neighbours). Kept separate from EstimateTokens so the packer can sum
+// counts ADDITIVELY across units and floor once at the chunk level. Mirrors
+// EstimateTokens' classes; a recalibration there that changes packing granularity
+// is itself a chunkerVersion bump (both are behaviour changes covered by the snapshot).
+func tokenClassCounts(s string) (cjk, other int) {
+	for _, r := range s {
+		switch {
+		case unicode.In(r, unicode.Han, unicode.Hiragana, unicode.Katakana, unicode.Hangul):
+			cjk++
+		case unicode.IsSpace(r):
+		default:
+			other++
+		}
+	}
+	return cjk, other
+}
+
+// estTokensFrom applies EstimateTokens' formula (cjk + other/3, floored at 16) to
+// pre-counted class totals — identical to EstimateTokens over the joined text.
+func estTokensFrom(cjk, other int) int {
+	est := cjk + other/3
+	if est < 16 {
+		est = 16
+	}
+	return est
+}
+
+// splitParagraphs breaks a chapter into paragraphs on blank lines and trims each,
+// dropping empties. Deterministic; whitespace-only paragraphs vanish. (Ingest emits
+// a blank line between block-level epub elements, so paragraphs survive extraction.)
+func splitParagraphs(chapter string) []string {
 	var paras []string
-	for _, raw := range strings.Split(block, "\n\n") {
-		// Collapse a run of blank lines: a "paragraph" that is itself only blank
-		// lines (from 3+ consecutive newlines) trims to "".
+	for _, raw := range strings.Split(chapter, "\n\n") {
 		if p := strings.TrimSpace(raw); p != "" {
 			paras = append(paras, p)
 		}
 	}
 	return paras
 }
+
+// --- source sentence segmentation (zh/ja/en) — chunkerVersion-versioned ---------
+
+// splitSourceSentences segments one paragraph at sentence boundaries, returning
+// substrings that TILE the input (their concatenation is exactly the paragraph —
+// no character is lost or moved), so a packed sub-chunk is an exact source slice.
+// A boundary is a run of sentence terminators, plus any closing quotes/brackets, at:
+//
+//	(1) a CJK terminator (。！？) — zero-width, CJK prose has no inter-sentence space —
+//	    but ONLY at quote-depth 0. A terminator inside a quote is dialogue-internal:
+//	    「止まれ！」と言った。 is ONE sentence ending at the outer 。, not two (the same
+//	    for zh “…！”）— this is the dialogue case §3.7/D12-Q1 defers for the coverage
+//	    oracle, but our source splitter is free to get it right;
+//	(2) an ASCII terminator (.!?) or ellipsis (…) — only when followed by whitespace
+//	    or end-of-paragraph (so a decimal/URL dot mid-token does not split, and en
+//	    dialogue "Stop!" She ran. still splits on the space after the quote).
+//
+// Trailing whitespace after a boundary is attached to the LEFT segment (the next
+// segment starts at the next content char), keeping the tiling exact. An
+// abbreviation guard suppresses a split after a lone "." following a known
+// abbreviation or a single-letter initial (never split "Mr. Smith" / "J. R. R.").
+// When unsure it prefers NOT to split (over-merge is safe — units just pack
+// together; over-split cuts a sentence, the forbidden case). Unbalanced quotes are
+// tolerated (depth floors at 0), erring toward merge.
+func splitSourceSentences(paragraph string) []string {
+	runes := []rune(paragraph)
+	var segs []string
+	start, i, depth := 0, 0, 0
+	for i < len(runes) {
+		r := runes[i]
+		if isOpenQuote(r) {
+			depth++
+			i++
+			continue
+		}
+		if isCloseQuote(r) {
+			if depth > 0 {
+				depth--
+			}
+			i++
+			continue
+		}
+		if !isSourceTerminator(r) {
+			i++
+			continue
+		}
+		// Consume a run of terminators (collapse "。。。" / "!?" into one boundary).
+		j := i
+		cjk := false
+		for j < len(runes) && isSourceTerminator(runes[j]) {
+			if isCJKTerminator(runes[j]) {
+				cjk = true
+			}
+			j++
+		}
+		// The terminator's quote-depth is the depth HERE — before absorbing the
+		// closing quote that may follow it (「…！」: the ！ is at depth 1, the 」 that
+		// closes the quote comes after and must not retro-promote it to depth 0).
+		depthHere := depth
+		// Absorb a run of closing quotes/brackets that belong to the sentence,
+		// decrementing quote-depth for the depth-tracked closers among them.
+		e := j
+		for e < len(runes) && isClosingBracket(runes[e]) {
+			if isCloseQuote(runes[e]) && depth > 0 {
+				depth--
+			}
+			e++
+		}
+		boundary := false
+		if cjk {
+			boundary = depthHere == 0 // dialogue-internal terminators do not end a sentence
+		} else {
+			// ASCII terminator / ellipsis: a boundary only before whitespace or EOS.
+			boundary = e >= len(runes) || isASCIISpace(runes[e])
+			// Abbreviation guard: a lone "." after an abbreviation/initial is not a end.
+			if boundary && j-i == 1 && runes[i] == '.' && isAbbrevBefore(runes[start:i]) {
+				boundary = false
+			}
+		}
+		if !boundary {
+			i = e
+			continue
+		}
+		// Attach the following whitespace run to the left segment (keeps tiling exact).
+		w := e
+		for w < len(runes) && isASCIISpace(runes[w]) {
+			w++
+		}
+		segs = append(segs, string(runes[start:w]))
+		start, i = w, w
+	}
+	if start < len(runes) {
+		segs = append(segs, string(runes[start:]))
+	}
+	return segs
+}
+
+// isOpenQuote / isCloseQuote are the DEPTH-TRACKED paired quote/bracket delimiters
+// (CJK brackets + curly quotes with distinct open/close glyphs). Straight quotes
+// (" ') are NOT tracked — same glyph opens and closes, so depth is undecidable; en
+// relies on the whitespace rule instead, which is correct for spaced prose.
+func isOpenQuote(r rune) bool {
+	switch r {
+	case '「', '『', '（', '【', '《', '〈', '［', '｛', '“', '‘':
+		return true
+	}
+	return false
+}
+
+func isCloseQuote(r rune) bool {
+	switch r {
+	case '」', '』', '）', '】', '》', '〉', '］', '｝', '”', '’':
+		return true
+	}
+	return false
+}
+
+// isSourceTerminator is the sentence-terminator class for the SOURCE: CJK 。！？,
+// ASCII .!?, and the ellipsis … (U+2026).
+func isSourceTerminator(r rune) bool {
+	switch r {
+	case '。', '！', '？', '.', '!', '?', '…':
+		return true
+	}
+	return false
+}
+
+// isCJKTerminator is the zero-width-splitting subset (fullwidth CJK). The ellipsis
+// is deliberately NOT here: "……" mid-CJK-sentence must not split, but "……。" still
+// splits on the 。 in the same run.
+func isCJKTerminator(r rune) bool {
+	switch r {
+	case '。', '！', '？':
+		return true
+	}
+	return false
+}
+
+// isClosingBracket is the closing-quote/bracket class absorbed into the left
+// sentence after its terminator (ja/zh brackets + straight/curly quotes).
+func isClosingBracket(r rune) bool {
+	switch r {
+	case '」', '』', '】', '）', ')', '》', '〉', '］', ']', '｝', '}',
+		'"', '\'', '”', '’':
+		return true
+	}
+	return false
+}
+
+// isASCIISpace is the whitespace used for ASCII-terminator boundary detection and
+// tiling — the ordinary prose separators (space, tab, newline, CR, form feed,
+// vertical tab, and the ideographic space U+3000).
+func isASCIISpace(r rune) bool {
+	switch r {
+	case ' ', '\t', '\n', '\r', '\f', '\v', '　':
+		return true
+	}
+	return false
+}
+
+// sourceAbbrevs are trailing tokens after which a lone "." is treated as an
+// abbreviation, not a sentence end (case-insensitive). A pragmatic set for prose;
+// the acceptance path is ja→ru where CJK terminators dominate, so this only guards
+// the en source. Single-letter initials are handled separately (isAbbrevBefore).
+var sourceAbbrevs = map[string]bool{
+	"mr": true, "mrs": true, "ms": true, "dr": true, "prof": true, "st": true,
+	"jr": true, "sr": true, "vs": true, "no": true, "vol": true, "ch": true,
+	"fig": true, "col": true, "gen": true, "sgt": true, "capt": true, "lt": true,
+	"rev": true, "gov": true, "sen": true, "rep": true, "etc": true, "inc": true,
+	"ltd": true, "co": true, "mt": true, "ave": true, "rd": true,
+}
+
+// isAbbrevBefore reports whether the text immediately before a lone "." ends in a
+// known abbreviation or a single-letter initial (so the "." is not a boundary).
+func isAbbrevBefore(left []rune) bool {
+	// Trailing run of ASCII letters (the "word" before the period).
+	end := len(left)
+	k := end
+	for k > 0 && isASCIILetter(left[k-1]) {
+		k--
+	}
+	word := left[k:end]
+	if len(word) == 0 {
+		return false
+	}
+	// Single-letter initial ("J.") — only when it stands alone (start or preceded by
+	// a space/opening punctuation), never a one-letter word ending a real sentence
+	// mid-flow would be rare; over-merge is the safe side.
+	if len(word) == 1 {
+		return k == 0 || !isASCIILetter(left[k-1])
+	}
+	return sourceAbbrevs[strings.ToLower(string(word))]
+}
+
+func isASCIILetter(r rune) bool {
+	return (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z')
+}
--- a/backend/internal/pipeline/chunker_test.go
+++ b/backend/internal/pipeline/chunker_test.go
@ -7,7 +7,7 @@ import (
 )

 func TestSplitChunksSingleParagraph(t *testing.T) {
-	got := SplitChunks("静かな図書館の朝。")
+	got := SplitChunks([]string{"静かな図書館の朝。"})
 	want := []Chunk{{Chapter: 1, ChunkIdx: 0, Text: "静かな図書館の朝。"}}
 	if !reflect.DeepEqual(got, want) {
 		t.Fatalf("single paragraph = %+v, want %+v", got, want)
@ -15,8 +15,7 @@ func TestSplitChunksSingleParagraph(t *testing.T) {
 }

 func TestSplitChunksChapters(t *testing.T) {
-	src := "Глава один." + chapterSep + "Глава два." + chapterSep + "Глава три."
-	got := SplitChunks(src)
+	got := SplitChunks([]string{"Глава один.", "Глава два.", "Глава три."})
 	want := []Chunk{
 		{Chapter: 1, ChunkIdx: 0, Text: "Глава один."},
 		{Chapter: 2, ChunkIdx: 0, Text: "Глава два."},
@ -28,23 +27,22 @@ func TestSplitChunksChapters(t *testing.T) {
 }

 func TestSplitChunksPacksToTarget(t *testing.T) {
-	// Two paragraphs that together exceed the target must split into two chunks,
-	// each never splitting a paragraph.
-	p1 := strings.Repeat("a", 800)
-	p2 := strings.Repeat("b", 800)
-	got := SplitChunks(p1 + "\n\n" + p2)
+	// Two paragraphs that together exceed the TOKEN target (each ~1000 tokens for
+	// cyrillic: 3000 chars / 3) must split into two chunks, each kept whole.
+	p1 := strings.Repeat("а", 3000)
+	p2 := strings.Repeat("б", 3000)
+	got := SplitChunks([]string{p1 + "\n\n" + p2})
 	want := []Chunk{
 		{Chapter: 1, ChunkIdx: 0, Text: p1},
 		{Chapter: 1, ChunkIdx: 1, Text: p2},
 	}
 	if !reflect.DeepEqual(got, want) {
-		t.Fatalf("packing = chapters/chunks %d, want 2 chunks; got %+v", len(got), summarize(got))
+		t.Fatalf("packing = %d chunks, want 2 whole paragraphs; got %+v", len(got), summarize(got))
 	}
 }

 func TestSplitChunksPacksSmallParagraphsTogether(t *testing.T) {
-	// Small paragraphs pack into one chunk (under the target), joined by a blank line.
-	got := SplitChunks("Абзац один.\n\nАбзац два.\n\nАбзац три.")
+	got := SplitChunks([]string{"Абзац один.\n\nАбзац два.\n\nАбзац три."})
 	if len(got) != 1 {
 		t.Fatalf("small paragraphs must pack into one chunk, got %d: %+v", len(got), summarize(got))
 	}
@ -53,19 +51,62 @@ func TestSplitChunksPacksSmallParagraphsTogether(t *testing.T) {
 	}
 }

-func TestSplitChunksOversizeParagraphIsOwnChunk(t *testing.T) {
-	big := strings.Repeat("ы", 2000) // > targetChunkChars but never split
-	got := SplitChunks(big)
+func TestSplitChunksOversizeParagraphSplitsAtSentences(t *testing.T) {
+	// One paragraph far over the token target, built of many whole sentences, must
+	// split into >1 chunk AND never cut a sentence: every original sentence must
+	// survive intact and in order across the chunks.
+	sentence := strings.Repeat("слово ", 40) + "конец." // ~80 tokens each
+	var sb strings.Builder
+	for i := 0; i < 40; i++ { // ~3200 tokens total ≫ 1500
+		sb.WriteString(sentence)
+		sb.WriteByte(' ')
+	}
+	para := strings.TrimSpace(sb.String())
+	got := SplitChunks([]string{para})
+	if len(got) < 2 {
+		t.Fatalf("oversize paragraph must split into >1 chunk, got %d", len(got))
+	}
+	wantSentences := trimAll(splitSourceSentences(para))
+	var gotSentences []string
+	for _, c := range got {
+		gotSentences = append(gotSentences, trimAll(splitSourceSentences(c.Text))...)
+	}
+	if !reflect.DeepEqual(gotSentences, wantSentences) {
+		t.Fatalf("sentence integrity broken:\n got %d sentences\nwant %d sentences", len(gotSentences), len(wantSentences))
+	}
+}
+
+func TestSplitChunksShortLinesPackByTrueTokenBudget(t *testing.T) {
+	// Many short dialogue paragraphs must pack by TRUE token content, not by summing
+	// the per-unit 16-token floor. 150 lines of 「はい。」 are ~450 true tokens (cjk=300,
+	// other=450 → 300+150), so they belong in ONE chunk. Summing the floor (150×16)
+	// would wrongly flush into ~2 sub-500-char chunks the coverage gate then skips.
+	var paras []string
+	for i := 0; i < 150; i++ {
+		paras = append(paras, "「はい。」")
+	}
+	got := SplitChunks([]string{strings.Join(paras, "\n\n")})
+	if len(got) != 1 {
+		t.Fatalf("short lines must pack by true token budget into 1 chunk, got %d chunks", len(got))
+	}
+	if EstimateTokens(got[0].Text) > targetChunkTokens {
+		t.Fatalf("packed chunk overshoots the budget: %d tokens", EstimateTokens(got[0].Text))
+	}
+}
+
+func TestSplitChunksOversizeSentenceIsOwnChunk(t *testing.T) {
+	// A single sentence (no internal terminator) larger than the budget can only be
+	// its own chunk — never split.
+	big := strings.Repeat("ы", 6000) // ~2000 tokens, one un-terminated run
+	got := SplitChunks([]string{big})
 	if len(got) != 1 || got[0].Text != big {
-		t.Fatalf("an oversize paragraph must be one un-split chunk, got %+v", summarize(got))
+		t.Fatalf("an oversize sentence must be one un-split chunk, got %+v", summarize(got))
 	}
 }

 func TestSplitChunksDropsEmpties(t *testing.T) {
-	// Empty chapter blocks and whitespace-only paragraphs vanish; chapter numbers
-	// only advance for non-empty chapters.
-	src := "A" + chapterSep + "   \n\n  " + chapterSep + "B"
-	got := SplitChunks(src)
+	// A blank/whitespace-only chapter vanishes and does NOT consume a chapter number.
+	got := SplitChunks([]string{"A", "   \n\n  ", "B"})
 	want := []Chunk{
 		{Chapter: 1, ChunkIdx: 0, Text: "A"},
 		{Chapter: 2, ChunkIdx: 0, Text: "B"},
@ -76,13 +117,94 @@ func TestSplitChunksDropsEmpties(t *testing.T) {
 }

 func TestSplitChunksDeterministic(t *testing.T) {
-	src := "Абзац.\n\nЕщё абзац." + chapterSep + strings.Repeat("длинный ", 300)
-	a, b := SplitChunks(src), SplitChunks(src)
+	chapters := []string{
+		"Абзац.\n\nЕщё абзац.",
+		strings.Repeat("слово ", 400) + "конец.",
+	}
+	a, b := SplitChunks(chapters), SplitChunks(chapters)
 	if !reflect.DeepEqual(a, b) {
 		t.Fatal("SplitChunks must be deterministic")
 	}
 }

+func TestSplitChunksNeverSplitsAbbreviation(t *testing.T) {
+	// Two long en paragraphs each ending mid-flow with "Mr. Smith" must not chunk
+	// between "Mr." and "Smith" — the abbreviation guard keeps the sentence whole.
+	p := strings.Repeat("The road went on and on. ", 100) + "It was Mr. Smith who arrived."
+	got := SplitChunks([]string{p})
+	for _, c := range got {
+		if strings.HasSuffix(strings.TrimSpace(c.Text), "Mr.") {
+			t.Fatalf("chunk ended at an abbreviation 'Mr.' — a sentence was cut: %q", c.Text)
+		}
+	}
+}
+
+// --- source sentence splitter (zh/ja/en) ---------------------------------------
+
+func TestSplitSourceSentencesTilesLosslessly(t *testing.T) {
+	// The segments must TILE the input: concatenation reproduces it byte-for-byte.
+	inputs := []string{
+		"静かな朝。彼は歩いた。「止まれ！」と叫んだ。",
+		"他说。“不要！”然后离开了。",
+		"Hello world. This is a test!  Really?  Yes.",
+		"Mr. Smith met Dr. J. R. R. Brown. They talked.",
+		"末尾に句点なし",
+		"",
+	}
+	for _, in := range inputs {
+		segs := splitSourceSentences(in)
+		if got := strings.Join(segs, ""); got != in {
+			t.Fatalf("tiling broke for %q: rejoined to %q", in, got)
+		}
+	}
+}
+
+func TestSplitSourceSentencesJapanese(t *testing.T) {
+	// CJK terminators split zero-width; a closing quote is absorbed into its sentence.
+	got := trimAll(splitSourceSentences("静かな朝。「止まれ！」と彼は言った。"))
+	want := []string{"静かな朝。", "「止まれ！」と彼は言った。"}
+	if !reflect.DeepEqual(got, want) {
+		t.Fatalf("ja split = %q, want %q", got, want)
+	}
+}
+
+func TestSplitSourceSentencesEnglish(t *testing.T) {
+	got := trimAll(splitSourceSentences(`He said "Stop!" She ran. Done.`))
+	want := []string{`He said "Stop!"`, "She ran.", "Done."}
+	if !reflect.DeepEqual(got, want) {
+		t.Fatalf("en split = %q, want %q", got, want)
+	}
+}
+
+func TestSplitSourceSentencesEllipsisNotOverSplitInCJK(t *testing.T) {
+	// "……" mid-CJK-sentence must not split; the trailing 。 still ends it.
+	got := trimAll(splitSourceSentences("そう……だった。次へ。"))
+	want := []string{"そう……だった。", "次へ。"}
+	if !reflect.DeepEqual(got, want) {
+		t.Fatalf("ellipsis split = %q, want %q", got, want)
+	}
+}
+
+func TestSplitSourceSentencesAbbreviations(t *testing.T) {
+	// Abbreviations and single-letter initials do not end a sentence.
+	got := trimAll(splitSourceSentences("Dr. J. R. R. Tolkien wrote it. The end."))
+	want := []string{"Dr. J. R. R. Tolkien wrote it.", "The end."}
+	if !reflect.DeepEqual(got, want) {
+		t.Fatalf("abbrev split = %q, want %q", got, want)
+	}
+}
+
+// trimAll TrimSpaces each segment and drops empties (test convenience).
+func trimAll(segs []string) []string {
+	var out []string
+	for _, s := range segs {
+		if t := strings.TrimSpace(s); t != "" {
+			out = append(out, t)
+		}
+	}
+	return out
+}
+
 func summarize(cs []Chunk) []string {
 	out := make([]string, len(cs))
 	for i, c := range cs {
--- a/backend/internal/pipeline/ingest.go
+++ b/backend/internal/pipeline/ingest.go
@ -0,0 +1,389 @@
+package pipeline
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"fmt"
+	"io"
+	"net/url"
+	"os"
+	"path"
+	"strings"
+)
+
+// ingest.go: the шаг-3a import layer. Ingest turns a source file (txt or epub)
+// into an ordered list of per-chapter NORMALIZED text plus the ruby/furigana
+// readings captured on the way (04-unhappy §4 / D9). It is the ONLY place that
+// reads the source; the runner then feeds doc.Chapters to SplitChunks and persists
+// doc.Ruby (runner.go). It is offline and deterministic — no LLM, no time/rand — so
+// the whole path is $0 and reproducible.
+//
+// epub v1 is a text extraction (02-mvp:25): read the chapters in spine order,
+// strip tags, drop inline markup/styling. The ONE thing we do NOT drop is ruby:
+// <ruby>base<rt>reading</rt></ruby> carries an author reading of a name/term that a
+// plain text export would silently lose (the documented hole, 04-unhappy §4). We
+// keep the BASE in the body and hand the READING to the caller for a glossary lock
+// (memory v2, шаг 4) — never injecting it into a prompt here (§7d).
+
+// chapterSep splits a TXT source into chapters. Form feed (U+000C) is the ASCII
+// "page/section break": semantically a chapter boundary, invisible in prose, and
+// untouched by NormalizeSource. A txt with no \f is a single chapter (Фаза-0 backward
+// compat: the one-file example stays one chapter). epub carries real chapter
+// structure in its spine, so it does not use this.
+const chapterSep = "\f"
+
+// RubyReading is one captured (base, reading) pair and the chapter it appeared in.
+// The pipeline aggregates these (min chapter, count) before persisting (runner.go).
+type RubyReading struct {
+	Base    string // the ruby body: the kanji/base surface form
+	Reading string // the <rt> reading (furigana)
+	Chapter int    // 1-based chapter where this occurrence was found
+}
+
+// Document is the ingested source: per-chapter normalized text (in reading order)
+// plus every ruby occurrence found. Chapters is what SplitChunks consumes.
+type Document struct {
+	Chapters []string
+	Ruby     []RubyReading
+}
+
+// Ingest reads a source file and returns its Document. Dispatch is by extension:
+// .epub → the epub reader; anything else → plain text (the example uses .txt; an
+// unknown extension is treated as text rather than rejected).
+func Ingest(p string) (*Document, error) {
+	if strings.EqualFold(extOf(p), ".epub") {
+		return ingestEPUB(p)
+	}
+	return ingestTXT(p)
+}
+
+// extOf returns the lower-cased file extension incl. the dot ("" if none).
+func extOf(p string) string {
+	i := strings.LastIndexByte(p, '.')
+	if i < 0 || strings.ContainsAny(p[i:], "/\\") {
+		return ""
+	}
+	return strings.ToLower(p[i:])
+}
+
+// ingestTXT reads a plain-text source: split on \f into chapters, NormalizeSource
+// each (BOM/CRLF/NFC/outer-trim). No ruby in txt.
+func ingestTXT(p string) (*Document, error) {
+	raw, err := os.ReadFile(p)
+	if err != nil {
+		return nil, fmt.Errorf("pipeline: ingest txt: %w", err)
+	}
+	var chapters []string
+	for _, part := range strings.Split(string(raw), chapterSep) {
+		chapters = append(chapters, NormalizeSource(part))
+	}
+	return &Document{Chapters: chapters}, nil
+}
+
+// --- epub ----------------------------------------------------------------------
+
+// containerXML is META-INF/container.xml: it points at the OPF package file. Tags
+// are matched by LOCAL name (no namespace in the struct tags), so a namespaced or
+// prefixed container still binds.
+type containerXML struct {
+	Rootfiles []struct {
+		FullPath  string `xml:"full-path,attr"`
+		MediaType string `xml:"media-type,attr"`
+	} `xml:"rootfiles>rootfile"`
+}
+
+// opfPackage is the OPF: the manifest (id→href→media-type) and the spine (the
+// reading order of itemref idrefs).
+type opfPackage struct {
+	Manifest []struct {
+		ID        string `xml:"id,attr"`
+		Href      string `xml:"href,attr"`
+		MediaType string `xml:"media-type,attr"`
+	} `xml:"manifest>item"`
+	Spine []struct {
+		IDRef string `xml:"idref,attr"`
+	} `xml:"spine>itemref"`
+}
+
+// ingestEPUB reads chapters in spine order and captures ruby. Every structural
+// problem (bad zip, missing container/opf, empty spine, no readable chapter) is a
+// loud error, never a panic (a truncated/foreign epub must not crash a run).
+func ingestEPUB(p string) (*Document, error) {
+	zr, err := zip.OpenReader(p)
+	if err != nil {
+		return nil, fmt.Errorf("pipeline: ingest epub: open zip %s: %w", p, err)
+	}
+	defer zr.Close()
+
+	files := map[string]*zip.File{}
+	for _, f := range zr.File {
+		files[path.Clean(f.Name)] = f
+	}
+
+	// 1) container.xml → OPF path.
+	cdata, err := readZipEntry(files, "META-INF/container.xml")
+	if err != nil {
+		return nil, fmt.Errorf("pipeline: ingest epub: %w", err)
+	}
+	var container containerXML
+	if err := xml.Unmarshal(cdata, &container); err != nil {
+		return nil, fmt.Errorf("pipeline: ingest epub: parse container.xml: %w", err)
+	}
+	opfPath := ""
+	for _, rf := range container.Rootfiles {
+		if strings.TrimSpace(rf.FullPath) != "" {
+			opfPath = path.Clean(rf.FullPath)
+			break
+		}
+	}
+	if opfPath == "" {
+		return nil, fmt.Errorf("pipeline: ingest epub: container.xml has no rootfile full-path")
+	}
+
+	// 2) OPF → manifest + spine.
+	odata, err := readZipEntry(files, opfPath)
+	if err != nil {
+		return nil, fmt.Errorf("pipeline: ingest epub: %w", err)
+	}
+	var opf opfPackage
+	if err := xml.Unmarshal(odata, &opf); err != nil {
+		return nil, fmt.Errorf("pipeline: ingest epub: parse opf %s: %w", opfPath, err)
+	}
+	if len(opf.Spine) == 0 {
+		return nil, fmt.Errorf("pipeline: ingest epub: opf %s has an empty spine", opfPath)
+	}
+	byID := map[string]struct{ href, mediaType string }{}
+	for _, it := range opf.Manifest {
+		byID[it.ID] = struct{ href, mediaType string }{it.Href, it.MediaType}
+	}
+	opfDir := path.Dir(opfPath)
+
+	// 3) Read each spine document in order (= one chapter). A dangling idref or a
+	// non-(x)html item is skipped, not fatal; a spine that yields zero readable
+	// content documents is a loud error.
+	doc := &Document{}
+	denseNo := 0 // matches SplitChunks: only a NON-empty chapter takes a number
+	read := 0
+	for _, ref := range opf.Spine {
+		item, ok := byID[ref.IDRef]
+		if !ok || !isXHTML(item.mediaType, item.href) {
+			continue
+		}
+		entry := resolveHref(opfDir, item.href)
+		data, err := readZipEntry(files, entry)
+		if err != nil {
+			return nil, fmt.Errorf("pipeline: ingest epub: spine item %q → %s: %w", ref.IDRef, entry, err)
+		}
+		read++
+		text, ruby, err := extractXHTML(data)
+		if err != nil {
+			return nil, fmt.Errorf("pipeline: ingest epub: extract %s: %w", entry, err)
+		}
+		norm := NormalizeSource(text)
+		doc.Chapters = append(doc.Chapters, norm)
+		// The chapter number ruby carries MUST equal the number SplitChunks assigns
+		// (dense — empty chapters are skipped), otherwise memory v2's since_ch is off
+		// by the count of empty spine items (cover/nav) before the reading. A
+		// ruby-bearing chapter is never empty (its base sits in the body), so denseNo
+		// is well-defined here; empty chapters carry no ruby.
+		if len(splitParagraphs(norm)) == 0 {
+			continue
+		}
+		denseNo++
+		for i := range ruby {
+			ruby[i].Chapter = denseNo
+		}
+		doc.Ruby = append(doc.Ruby, ruby...)
+	}
+	if read == 0 {
+		return nil, fmt.Errorf("pipeline: ingest epub: spine resolved to no readable (x)html chapters")
+	}
+	return doc, nil
+}
+
+// isXHTML reports whether a manifest item is an (x)html content document. It
+// checks the media-type (parameters like "; charset=utf-8" stripped), then falls
+// back to the href extension for ANY unrecognized/absent type — real epubs ship
+// xhtml chapters typed application/xml or text/xml, so a content-document extension
+// is the reliable signal. A genuine non-content asset keeps its own extension
+// (.jpg/.css/.ncx/.svg), so the extension fallback does not misclassify it
+// (self-review: a present-but-unrecognized type used to silently drop the chapter).
+func isXHTML(mediaType, href string) bool {
+	mt := strings.ToLower(strings.TrimSpace(mediaType))
+	if i := strings.IndexByte(mt, ';'); i >= 0 {
+		mt = strings.TrimSpace(mt[:i])
+	}
+	switch mt {
+	case "application/xhtml+xml", "text/html", "application/html":
+		return true
+	}
+	switch extOf(href) {
+	case ".xhtml", ".html", ".htm":
+		return true
+	}
+	return false
+}
+
+// resolveHref joins an OPF-relative href to the OPF directory (percent-decoded,
+// slash-cleaned) into a zip entry path.
+func resolveHref(opfDir, href string) string {
+	if h, err := url.PathUnescape(href); err == nil {
+		href = h
+	}
+	// Drop any fragment (chapter.xhtml#frag).
+	if i := strings.IndexByte(href, '#'); i >= 0 {
+		href = href[:i]
+	}
+	return path.Clean(path.Join(opfDir, href))
+}
+
+// readZipEntry reads a zip entry by cleaned path.
+func readZipEntry(files map[string]*zip.File, name string) ([]byte, error) {
+	f, ok := files[path.Clean(name)]
+	if !ok {
+		return nil, fmt.Errorf("missing zip entry %q", name)
+	}
+	rc, err := f.Open()
+	if err != nil {
+		return nil, fmt.Errorf("open zip entry %q: %w", name, err)
+	}
+	defer rc.Close()
+	return io.ReadAll(rc)
+}
+
+// --- xhtml text + ruby extraction ----------------------------------------------
+
+// blockTags emit a paragraph break (blank line) so the chunker's paragraph packing
+// survives extraction; without them a whole chapter collapses into one paragraph.
+var blockTags = map[string]bool{
+	"p": true, "div": true, "h1": true, "h2": true, "h3": true, "h4": true,
+	"h5": true, "h6": true, "li": true, "blockquote": true, "section": true,
+	"article": true, "tr": true, "table": true, "ul": true, "ol": true, "dl": true,
+	"dd": true, "dt": true, "pre": true, "figure": true, "figcaption": true,
+	"header": true, "footer": true, "aside": true, "nav": true, "hr": true,
+	"main": true, "td": true, "th": true, "caption": true,
+}
+
+// skipRoots are subtrees whose text is not prose (CSS/JS/metadata).
+var skipRoots = map[string]bool{"script": true, "style": true, "head": true}
+
+// extractXHTML strips tags to text and captures ruby. Rules:
+//   - block tag → blank line; <br> → newline (paragraph structure for the chunker);
+//   - <ruby>base<rt>reading</rt></ruby>: BASE goes to the body, READING is captured
+//     as a (base,reading) pair, <rp> parenthesis fallbacks are dropped;
+//   - <script>/<style>/<head> subtrees are dropped.
+//
+// Non-strict HTML decoding (HTMLEntity/HTMLAutoClose) tolerates the loose xhtml
+// real epubs ship. A non-UTF-8 declared charset is a loud error (epub v1 = UTF-8).
+// Ruby is returned with Chapter unset (0); the caller stamps the dense chapter no.
+func extractXHTML(data []byte) (string, []RubyReading, error) {
+	dec := xml.NewDecoder(bytes.NewReader(data))
+	dec.Strict = false
+	dec.Entity = xml.HTMLEntity
+	dec.AutoClose = xml.HTMLAutoClose
+	dec.CharsetReader = func(charset string, input io.Reader) (io.Reader, error) {
+		switch strings.ToLower(strings.TrimSpace(charset)) {
+		case "", "utf-8", "utf8", "us-ascii", "ascii":
+			return input, nil
+		}
+		return nil, fmt.Errorf("unsupported xhtml charset %q (epub v1 expects UTF-8)", charset)
+	}
+
+	var body strings.Builder
+	var ruby []RubyReading
+	var base, reading strings.Builder
+	var rubyDepth, rtDepth, rpDepth, skipDepth int
+
+	for {
+		tok, err := dec.Token()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			return "", nil, err
+		}
+		switch t := tok.(type) {
+		case xml.StartElement:
+			if skipDepth > 0 {
+				skipDepth++
+				continue
+			}
+			name := strings.ToLower(t.Name.Local)
+			switch {
+			case skipRoots[name]:
+				skipDepth++
+			case name == "ruby":
+				rubyDepth++
+				if rubyDepth == 1 {
+					base.Reset()
+					reading.Reset()
+				}
+			case name == "rt":
+				rtDepth++
+			case name == "rp":
+				rpDepth++
+			case name == "br":
+				body.WriteByte('\n')
+			case blockTags[name] && rubyDepth == 0:
+				body.WriteString("\n\n")
+			}
+		case xml.EndElement:
+			if skipDepth > 0 {
+				skipDepth--
+				continue
+			}
+			name := strings.ToLower(t.Name.Local)
+			switch {
+			case name == "ruby":
+				if rubyDepth > 0 {
+					rubyDepth--
+					if rubyDepth == 0 {
+						b := strings.TrimSpace(base.String())
+						r := strings.TrimSpace(reading.String())
+						if b != "" && r != "" {
+							ruby = append(ruby, RubyReading{Base: b, Reading: r})
+						}
+						base.Reset()
+						reading.Reset()
+					}
+				}
+			case name == "rt":
+				if rtDepth > 0 {
+					rtDepth--
+				}
+			case name == "rp":
+				if rpDepth > 0 {
+					rpDepth--
+				}
+			case blockTags[name] && rubyDepth == 0:
+				body.WriteString("\n\n")
+			}
+		case xml.CharData:
+			if skipDepth > 0 {
+				continue
+			}
+			text := string(t)
+			if rubyDepth > 0 {
+				switch {
+				case rpDepth > 0:
+					// ruby parenthesis fallback — drop.
+				case rtDepth > 0:
+					reading.WriteString(text)
+				case strings.TrimSpace(text) == "":
+					// Formatting whitespace BETWEEN ruby base segments (pretty-printed
+					// jukugo <rb>旧</rb> <rb>字</rb> …) must not enter the captured base
+					// (it would mis-key the glossary reading) nor the body (it would
+					// inject stray whitespace between CJK glyphs into ch.Text) —
+					// self-review finding. Drop it; real base glyphs are non-whitespace.
+				default:
+					base.WriteString(text)
+					body.WriteString(text)
+				}
+				continue
+			}
+			body.WriteString(text)
+		}
+	}
+	return body.String(), ruby, nil
+}
--- a/backend/internal/pipeline/ingest_test.go
+++ b/backend/internal/pipeline/ingest_test.go
@ -0,0 +1,332 @@
+package pipeline
+
+import (
+	"archive/zip"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// --- txt ------------------------------------------------------------------------
+
+func TestIngestTXTSingleChapter(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "src.txt")
+	if err := os.WriteFile(path, []byte("静かな図書館の朝。"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	doc, err := Ingest(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(doc.Chapters) != 1 || doc.Chapters[0] != "静かな図書館の朝。" {
+		t.Fatalf("chapters = %#v", doc.Chapters)
+	}
+	if len(doc.Ruby) != 0 {
+		t.Fatalf("txt has no ruby, got %#v", doc.Ruby)
+	}
+}
+
+func TestIngestTXTFormFeedChapters(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "src.txt")
+	if err := os.WriteFile(path, []byte("Глава A"+chapterSep+"Глава B"+chapterSep+"Глава C"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	doc, err := Ingest(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	want := []string{"Глава A", "Глава B", "Глава C"}
+	if strings.Join(doc.Chapters, "|") != strings.Join(want, "|") {
+		t.Fatalf("chapters = %#v, want %#v", doc.Chapters, want)
+	}
+}
+
+func TestIngestTXTNormalizes(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "src.txt")
+	// UTF-8 BOM + CRLF + surrounding whitespace must be normalized away.
+	if err := os.WriteFile(path, []byte("\uFEFF  строка один\r\nстрока два  "), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	doc, err := Ingest(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if doc.Chapters[0] != "строка один\nстрока два" {
+		t.Fatalf("normalized = %q", doc.Chapters[0])
+	}
+}
+
+// --- epub builder ---------------------------------------------------------------
+
+type epubChapter struct {
+	id    string
+	href  string // relative to the OPF dir (OEBPS/)
+	mtype string // "" → application/xhtml+xml
+	body  string // inner xhtml of <body>
+}
+
+// buildEPUB writes a minimal but real epub (container.xml → OPF → spine → xhtml)
+// to a temp .epub file and returns its path. spineIDs gives the reading order
+// (may differ from manifest order to prove the spine drives it).
+func buildEPUB(t *testing.T, chapters []epubChapter, spineIDs []string) string {
+	t.Helper()
+	path := filepath.Join(t.TempDir(), "book.epub")
+	buildEPUBAt(t, path, chapters, spineIDs)
+	return path
+}
+
+// buildEPUBAt writes the epub to a caller-chosen path (used by the runner e2e).
+func buildEPUBAt(t *testing.T, path string, chapters []epubChapter, spineIDs []string) {
+	t.Helper()
+	f, err := os.Create(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer f.Close()
+	zw := zip.NewWriter(f)
+	add := func(name, content string) {
+		w, err := zw.Create(name)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if _, err := w.Write([]byte(content)); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	add("mimetype", "application/epub+zip")
+	add("META-INF/container.xml", `<?xml version="1.0"?>
+<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
+  <rootfiles><rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"/></rootfiles>
+</container>`)
+
+	var manifest, spine strings.Builder
+	for _, c := range chapters {
+		mt := c.mtype
+		if mt == "" {
+			mt = "application/xhtml+xml"
+		}
+		manifest.WriteString(`<item id="` + c.id + `" href="` + c.href + `" media-type="` + mt + `"/>` + "\n")
+	}
+	for _, id := range spineIDs {
+		spine.WriteString(`<itemref idref="` + id + `"/>` + "\n")
+	}
+	add("OEBPS/content.opf", `<?xml version="1.0" encoding="utf-8"?>
+<package xmlns="http://www.idpf.org/2007/opf" version="3.0" unique-identifier="uid">
+  <metadata xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title>Test</dc:title></metadata>
+  <manifest>`+manifest.String()+`</manifest>
+  <spine>`+spine.String()+`</spine>
+</package>`)
+
+	for _, c := range chapters {
+		// The FILE is (x)html when its href says so, regardless of how the manifest
+		// media-type labels it — real epubs mislabel xhtml as application/xml, etc.
+		switch extOf(c.href) {
+		case ".xhtml", ".html", ".htm":
+			add("OEBPS/"+c.href, `<?xml version="1.0" encoding="utf-8"?>
+<html xmlns="http://www.w3.org/1999/xhtml"><head><title>c</title><style>.x{color:red}</style></head>
+<body>`+c.body+`</body></html>`)
+		default:
+			add("OEBPS/"+c.href, c.body) // non-xhtml asset (e.g. image bytes stand-in)
+		}
+	}
+	if err := zw.Close(); err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestIngestEPUBSpineOrder(t *testing.T) {
+	// Manifest order (c3,c1,c2) differs from spine order (c1,c2,c3): the spine wins.
+	chapters := []epubChapter{
+		{id: "c3", href: "ch3.xhtml", body: `<p>第三章。</p>`},
+		{id: "c1", href: "ch1.xhtml", body: `<p>第一章。</p>`},
+		{id: "c2", href: "ch2.xhtml", body: `<p>第二章。</p>`},
+	}
+	doc, err := Ingest(buildEPUB(t, chapters, []string{"c1", "c2", "c3"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(doc.Chapters) != 3 {
+		t.Fatalf("want 3 chapters, got %d: %#v", len(doc.Chapters), doc.Chapters)
+	}
+	for i, want := range []string{"第一章。", "第二章。", "第三章。"} {
+		if strings.TrimSpace(doc.Chapters[i]) != want {
+			t.Fatalf("chapter %d = %q, want %q (spine order broken)", i+1, doc.Chapters[i], want)
+		}
+	}
+}
+
+func TestIngestEPUBStripsTagsAndEntities(t *testing.T) {
+	body := `<p>Hello &amp; <b>bold</b> world.</p><p>Second&#160;paragraph.</p>`
+	doc, err := Ingest(buildEPUB(t, []epubChapter{{id: "c1", href: "ch1.xhtml", body: body}}, []string{"c1"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	txt := doc.Chapters[0]
+	if !strings.Contains(txt, "Hello & bold world.") {
+		t.Fatalf("entities/tags not handled: %q", txt)
+	}
+	if strings.Contains(txt, "<b>") || strings.Contains(txt, "color:red") || strings.Contains(txt, "<title>") {
+		t.Fatalf("markup/style/head leaked into text: %q", txt)
+	}
+	// Two <p> blocks must be separated by a blank line so the chunker sees paragraphs.
+	if paras := splitParagraphs(NormalizeSource(txt)); len(paras) != 2 {
+		t.Fatalf("want 2 paragraphs from 2 <p>, got %d: %#v", len(paras), paras)
+	}
+}
+
+func TestIngestEPUBRubyCaptureAndBaseInBody(t *testing.T) {
+	body := `<p><ruby>漢字<rt>かんじ</rt></ruby>を読む。</p>` +
+		`<p><ruby>東<rp>(</rp><rt>とう</rt><rp>)</rp>京<rp>(</rp><rt>きょう</rt><rp>)</rp></ruby>タワー。</p>` +
+		`<p><ruby>字<rt></rt></ruby>だけ。</p>` // empty reading → not captured, base kept
+	doc, err := Ingest(buildEPUB(t, []epubChapter{{id: "c1", href: "ch1.xhtml", body: body}}, []string{"c1"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	txt := doc.Chapters[0]
+	// Base stays in the body; readings and rp parens do NOT.
+	for _, want := range []string{"漢字を読む", "東京タワー", "字だけ"} {
+		if !strings.Contains(txt, want) {
+			t.Fatalf("base missing from body %q (want %q)", txt, want)
+		}
+	}
+	for _, bad := range []string{"かんじ", "とう", "きょう", "(", ")"} {
+		if strings.Contains(txt, bad) {
+			t.Fatalf("reading/paren %q leaked into body %q", bad, txt)
+		}
+	}
+	// Two readings captured (mono-ruby merged to one base+reading); empty rt skipped.
+	got := map[string]string{}
+	for _, r := range doc.Ruby {
+		got[r.Base] = r.Reading
+		if r.Chapter != 1 {
+			t.Fatalf("ruby chapter = %d, want 1", r.Chapter)
+		}
+	}
+	if len(doc.Ruby) != 2 || got["漢字"] != "かんじ" || got["東京"] != "とうきょう" {
+		t.Fatalf("ruby capture = %#v", doc.Ruby)
+	}
+}
+
+func TestIngestEPUBRubyChapterMatchesDenseNumbering(t *testing.T) {
+	// An empty cover page in the spine before a ruby-bearing chapter must NOT shift
+	// ruby's first_chapter off the number SplitChunks assigns (dense — cover skipped).
+	chapters := []epubChapter{
+		{id: "cover", href: "cover.xhtml", body: `<div>   </div>`}, // whitespace only → no chapter number
+		{id: "c1", href: "ch1.xhtml", body: `<p>ふつうの文。</p>`},
+		{id: "c2", href: "ch2.xhtml", body: `<p><ruby>朱雀<rt>すざく</rt></ruby>が舞う。</p>`},
+	}
+	doc, err := Ingest(buildEPUB(t, chapters, []string{"cover", "c1", "c2"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	// The ruby is in the 3rd spine item but the 2nd NON-empty chapter → chapter 2,
+	// exactly what SplitChunks(doc.Chapters) labels it.
+	if len(doc.Ruby) != 1 || doc.Ruby[0].Chapter != 2 {
+		t.Fatalf("ruby dense chapter = %#v, want chapter 2", doc.Ruby)
+	}
+	chunks := SplitChunks(doc.Chapters)
+	var rubyChunkChapter int
+	for _, c := range chunks {
+		if strings.Contains(c.Text, "朱雀") {
+			rubyChunkChapter = c.Chapter
+		}
+	}
+	if rubyChunkChapter != doc.Ruby[0].Chapter {
+		t.Fatalf("ruby first_chapter %d != chunk chapter %d — memory-v2 since_ch would be off",
+			doc.Ruby[0].Chapter, rubyChunkChapter)
+	}
+}
+
+func TestIngestEPUBAcceptsGenericAndParameterizedMediaTypes(t *testing.T) {
+	// Real epubs mislabel xhtml chapters as application/xml or text/xml, or add a
+	// "; charset=utf-8" parameter. All must still be read — not silently dropped.
+	chapters := []epubChapter{
+		{id: "c1", href: "ch1.xhtml", mtype: "application/xml", body: `<p>第一章。</p>`},
+		{id: "c2", href: "ch2.xhtml", mtype: "text/xml", body: `<p>第二章。</p>`},
+		{id: "c3", href: "ch3.xhtml", mtype: "application/xhtml+xml; charset=utf-8", body: `<p>第三章。</p>`},
+	}
+	doc, err := Ingest(buildEPUB(t, chapters, []string{"c1", "c2", "c3"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(doc.Chapters) != 3 {
+		t.Fatalf("mislabeled/parameterized media-types must all be read, got %d: %#v", len(doc.Chapters), doc.Chapters)
+	}
+	for i, want := range []string{"第一章。", "第二章。", "第三章。"} {
+		if strings.TrimSpace(doc.Chapters[i]) != want {
+			t.Fatalf("chapter %d = %q, want %q", i+1, doc.Chapters[i], want)
+		}
+	}
+}
+
+func TestIngestEPUBRubyBaseWhitespaceCollapsed(t *testing.T) {
+	// Pretty-printed jukugo ruby: whitespace BETWEEN <rb> base segments must not
+	// enter the captured base (mis-keys the glossary) nor the body (pollutes ch.Text).
+	body := "<p><ruby>\n  <rb>旧</rb>\n  <rb>字</rb>\n  <rb>体</rb>\n  <rt>きゅう</rt><rt>じ</rt><rt>たい</rt>\n</ruby>の話。</p>"
+	doc, err := Ingest(buildEPUB(t, []epubChapter{{id: "c1", href: "ch1.xhtml", body: body}}, []string{"c1"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(doc.Ruby) != 1 || doc.Ruby[0].Base != "旧字体" || doc.Ruby[0].Reading != "きゅうじたい" {
+		t.Fatalf("ruby base/reading not clean: %#v", doc.Ruby)
+	}
+	if !strings.Contains(doc.Chapters[0], "旧字体の話") {
+		t.Fatalf("clean base missing from body: %q", doc.Chapters[0])
+	}
+	for _, bad := range []string{"旧 字", "旧\n"} {
+		if strings.Contains(doc.Chapters[0], bad) {
+			t.Fatalf("ruby-internal whitespace leaked into body: %q", doc.Chapters[0])
+		}
+	}
+}
+
+func TestIngestEPUBSkipsNonXHTML(t *testing.T) {
+	chapters := []epubChapter{
+		{id: "img", href: "cover.jpg", mtype: "image/jpeg", body: "\xff\xd8not-really-jpeg"},
+		{id: "c1", href: "ch1.xhtml", body: `<p>本文。</p>`},
+	}
+	doc, err := Ingest(buildEPUB(t, chapters, []string{"img", "c1"}))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(doc.Chapters) != 1 || strings.TrimSpace(doc.Chapters[0]) != "本文。" {
+		t.Fatalf("non-xhtml spine item must be skipped: %#v", doc.Chapters)
+	}
+}
+
+func TestIngestEPUBBrokenFailsLoud(t *testing.T) {
+	t.Run("not a zip", func(t *testing.T) {
+		p := filepath.Join(t.TempDir(), "bad.epub")
+		os.WriteFile(p, []byte("this is not a zip"), 0o644)
+		if _, err := Ingest(p); err == nil {
+			t.Fatal("a non-zip .epub must error, not panic")
+		}
+	})
+	t.Run("missing container", func(t *testing.T) {
+		p := filepath.Join(t.TempDir(), "noc.epub")
+		f, _ := os.Create(p)
+		zw := zip.NewWriter(f)
+		w, _ := zw.Create("OEBPS/content.opf")
+		w.Write([]byte("<package/>"))
+		zw.Close()
+		f.Close()
+		if _, err := Ingest(p); err == nil {
+			t.Fatal("missing container.xml must error")
+		}
+	})
+	t.Run("empty spine", func(t *testing.T) {
+		p := buildEPUB(t, []epubChapter{{id: "c1", href: "ch1.xhtml", body: `<p>x</p>`}}, nil)
+		if _, err := Ingest(p); err == nil {
+			t.Fatal("an empty spine must error")
+		}
+	})
+	t.Run("dangling spine idref", func(t *testing.T) {
+		// spine references a missing manifest id → resolves to zero chapters → error.
+		p := buildEPUB(t, []epubChapter{{id: "c1", href: "ch1.xhtml", body: `<p>x</p>`}}, []string{"ghost"})
+		if _, err := Ingest(p); err == nil {
+			t.Fatal("a spine resolving to no readable chapters must error")
+		}
+	})
+}
--- a/backend/internal/pipeline/render.go
+++ b/backend/internal/pipeline/render.go
@ -29,13 +29,18 @@ import (
 // match) промахивается.

 // chunkerVersion versions the segmentation/ingestion rules (включая
-// нормализацию источника NormalizeSource); it is part of the snapshot, so
-// re-chunking a book is an explicit re-translation, not a silent cache miss
-// (§3.4 — [НУЖНО РЕШЕНИЕ] п.2в). Веха 2 replaced the whole-file "one chunk" of
-// Фаза 0 with the chapter/chunk splitter (chunker.go), so the version bumps:
-// any Фаза-0 project DB re-pins loudly via the --resnapshot gate (a re-chunk is
-// a deliberate re-translation), never a silent divergent re-pay.
-const chunkerVersion = "chunker-v2-ff-para-pack"
+// нормализацию источника NormalizeSource, ingest-слой txt/epub и целевой размер
+// чанка targetChunkTokens); it is part of the snapshot, so re-chunking a book is
+// an explicit re-translation, not a silent cache miss (§3.4 — [НУЖНО РЕШЕНИЕ]
+// п.2в). Шаг 3a replaced the Веха-2 char-budget paragraph packer with the real
+// source segmenter (sentence-aware, ~1–2k-token packing over ingest-split
+// chapters — chunker.go/ingest.go), so the version bumps: any earlier project DB
+// re-pins loudly via the --resnapshot gate (a re-chunk is a deliberate
+// re-translation), never a silent divergent re-pay. NOTE: the chunk target is a
+// const covered by THIS string; if it ever becomes config-tunable it must ALSO be
+// folded into the top-level snapshot (like coverageSnap), never left to the
+// version alone (§7d).
+const chunkerVersion = "chunker-v3-sentence-pack"

 // estimatorVersion versions EstimateTokens: его выход входит в max_tokens и
 // через него в request-hash, поэтому перекалибровка весов — тоже явная
--- a/backend/internal/pipeline/runner.go
+++ b/backend/internal/pipeline/runner.go
@ -8,7 +8,7 @@ import (
 	"errors"
 	"fmt"
 	"log/slog"
-	"os"
+	"sort"
 	"strings"
 	"time"

@ -473,14 +473,11 @@ func (e *CompletedWithFlags) Error() string {
 // (ceiling, config change without --resnapshot, unbilled call failure, store
 // error) aborts with an error, from which resume continues.
 func (r *Runner) TranslateBook(ctx context.Context) (*BookResult, error) {
-	srcRaw, err := os.ReadFile(r.Book.SourceFile)
+	// Ingest reads + normalizes the source (txt/epub) into ordered per-chapter text
+	// and captures ruby readings (ingest.go). Offline and deterministic ($0, no LLM).
+	doc, err := Ingest(r.Book.SourceFile)
 	if err != nil {
-		return nil, fmt.Errorf("pipeline: read source: %w", err)
-	}
-	// Нормализуем (BOM/CRLF/NFC) — request-hash стабилен между ОС/редакторами.
-	source := NormalizeSource(string(srcRaw))
-	if source == "" {
-		return nil, fmt.Errorf("pipeline: source file %s is empty", r.Book.SourceFile)
+		return nil, err
 	}

 	// Snapshot is book-level (brief + stage plan + memory), computed once and
@ -493,11 +490,19 @@ func (r *Runner) TranslateBook(ctx context.Context) (*BookResult, error) {
 		return nil, err
 	}

-	chunks := SplitChunks(source)
+	chunks := SplitChunks(doc.Chapters)
 	if len(chunks) == 0 {
 		return nil, fmt.Errorf("pipeline: source file %s produced no chunks after normalization", r.Book.SourceFile)
 	}

+	// Persist captured ruby readings (idempotent; consumed by memory v2 for a
+	// glossary name-lock — шаг 4). Never injected into a prompt here (§7d), so it
+	// touches neither the snapshot nor request_hash; a resume re-persists the same
+	// rows at $0.
+	if err := r.persistRuby(doc.Ruby); err != nil {
+		return nil, fmt.Errorf("pipeline: persist ruby readings: %w", err)
+	}
+
 	res := &BookResult{BookID: r.Book.BookID}
 	for _, ch := range chunks {
 		outcome, err := r.translateChunk(ctx, snapID, ch)
@ -515,6 +520,55 @@ func (r *Runner) TranslateBook(ctx context.Context) (*BookResult, error) {
 	return res, nil
 }

+// persistRuby aggregates the ingested ruby occurrences into one row per
+// (base, reading) — first_chapter = MIN, occurrences = full-book count — and
+// upserts them (v4 schema). Idempotent: the aggregation is recomputed identically
+// on every ingest, so a resume re-writes the same rows (MIN keeps the earliest
+// chapter, occurrences is replaced with the recomputed count). The write order is
+// sorted for deterministic, test-stable behavior; correctness does not depend on it
+// (the upsert is commutative). Memory v2 (шаг 4) consumes ruby_readings into a
+// glossary name-lock (D9); nothing here injects into a prompt (§7d).
+func (r *Runner) persistRuby(readings []RubyReading) error {
+	if len(readings) == 0 {
+		return nil
+	}
+	type agg struct {
+		first int
+		count int
+	}
+	seen := map[[2]string]*agg{}
+	order := make([][2]string, 0, len(readings))
+	for _, rr := range readings {
+		key := [2]string{rr.Base, rr.Reading}
+		a, ok := seen[key]
+		if !ok {
+			a = &agg{first: rr.Chapter}
+			seen[key] = a
+			order = append(order, key)
+		}
+		if rr.Chapter < a.first {
+			a.first = rr.Chapter
+		}
+		a.count++
+	}
+	sort.Slice(order, func(i, j int) bool {
+		if order[i][0] != order[j][0] {
+			return order[i][0] < order[j][0]
+		}
+		return order[i][1] < order[j][1]
+	})
+	for _, key := range order {
+		a := seen[key]
+		if err := r.Store.UpsertRubyReading(store.RubyReading{
+			BookID: r.Book.BookID, Base: key[0], Reading: key[1],
+			FirstChapter: a.first, Occurrences: a.count,
+		}); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
 // translateChunk drives ONE chunk through the stage list. Stages run in order,
 // each feeding the next; the FIRST flagged stage stops the chunk — later stages
 // are recorded `skipped` (no paid edit over a garbage draft, D2). Returns an
--- a/backend/internal/pipeline/runner_test.go
+++ b/backend/internal/pipeline/runner_test.go
@ -16,6 +16,7 @@ import (
 	"time"

 	"textmachine/backend/internal/obs"
+	"textmachine/backend/internal/store"

 	_ "modernc.org/sqlite"
 )
@ -101,6 +102,8 @@ const fakeCallUSD = (800*1.0 + 200*0.1 + 500*2.0) / 1e6

 type projectOpts struct {
 	source       string
+	epub         []epubChapter // when set, the source is an epub (source.epub) instead of txt
+	spine        []string      // spine order for epub
 	regenerate   int
 	minMaxTokens int
 	bookUSD      float64
@ -149,7 +152,13 @@ stages:
  - { name: edit, role: editor, model: fake-model, prompt: prompts/editor.md, prompt_version: v-test, temperature: 0.4, reasoning: "off" }
 %s`, o.minMaxTokens, o.regenerate, o.gatesYAML))

-	writeFile(t, filepath.Join(dir, "source.txt"), o.source)
+	sourceName := "source.txt"
+	if len(o.epub) > 0 {
+		sourceName = "source.epub"
+		buildEPUBAt(t, filepath.Join(dir, sourceName), o.epub, o.spine)
+	} else {
+		writeFile(t, filepath.Join(dir, "source.txt"), o.source)
+	}
 	writeFile(t, filepath.Join(dir, "book.yaml"), fmt.Sprintf(`
 book_id: test-book
 title: Тест
@ -163,9 +172,9 @@ transcription: polivanov
 footnotes: minimal
 pipeline: pipeline.yaml
 models: models.yaml
-source_file: source.txt
+source_file: %s
 ceilings: { book_usd: %g, day_usd: 2.0 }
-`, o.bookUSD))
+`, sourceName, o.bookUSD))
 	return filepath.Join(dir, "book.yaml")
 }

@ -245,6 +254,73 @@ func TestRunnerEndToEndWithResume(t *testing.T) {
 	}
 }

+func TestRunnerEPUBEndToEndAndRubyPersist(t *testing.T) {
+	rec := &reqRec{}
+	srv := newJSONProvider(rec, draftEdit)
+	defer srv.Close()
+	chapters := []epubChapter{
+		{id: "c1", href: "ch1.xhtml", body: `<p><ruby>朱雀<rt>すざく</rt></ruby>は南を守る。</p>`},
+		{id: "c2", href: "ch2.xhtml", body: `<p><ruby>羅生門<rt>らしょうもん</rt></ruby>の下で待つ。</p>`},
+	}
+	bookPath := setupProjectOpts(t, srv.URL, projectOpts{epub: chapters, spine: []string{"c1", "c2"}, regenerate: 1})
+	ctx := obs.WithReqInfo(context.Background(), obs.ReqInfo{TraceID: obs.NewTraceID()})
+
+	r1 := newRunner(t, bookPath)
+	res1, err := r1.TranslateBook(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+	// Two spine chapters → two chunks (one each) on the chapter axis, both OK.
+	if len(res1.Chunks) != 2 {
+		t.Fatalf("want 2 chunks from 2 chapters, got %d: %+v", len(res1.Chunks), res1.Chunks)
+	}
+	if res1.Chunks[0].Chapter != 1 || res1.Chunks[1].Chapter != 2 ||
+		res1.Chunks[0].ChunkIdx != 0 || res1.Chunks[1].ChunkIdx != 0 {
+		t.Fatalf("chapter/chunk axis wrong: %+v", res1.Chunks)
+	}
+	if res1.Flagged != 0 || res1.Chunks[0].FinalText != "ОТРЕДАКТИРОВАННЫЙ ПЕРЕВОД" {
+		t.Fatalf("run = %+v", res1)
+	}
+	if rec.count() != 4 { // draft+edit per chunk × 2 chunks
+		t.Fatalf("want 4 provider calls, got %d", rec.count())
+	}
+	// Ruby captured on ingest and persisted with the right first_chapter.
+	assertRuby := func(s *store.Store, wantRows int) {
+		t.Helper()
+		rows, err := s.RubyReadingsForBook("test-book")
+		if err != nil {
+			t.Fatal(err)
+		}
+		if len(rows) != wantRows {
+			t.Fatalf("ruby rows = %d, want %d: %#v", len(rows), wantRows, rows)
+		}
+		got := map[string]store.RubyReading{}
+		for _, rr := range rows {
+			got[rr.Base] = rr
+		}
+		if got["朱雀"].Reading != "すざく" || got["朱雀"].FirstChapter != 1 {
+			t.Fatalf("ruby 朱雀 = %+v", got["朱雀"])
+		}
+		if got["羅生門"].Reading != "らしょうもん" || got["羅生門"].FirstChapter != 2 {
+			t.Fatalf("ruby 羅生門 = %+v", got["羅生門"])
+		}
+	}
+	assertRuby(r1.Store, 2)
+	r1.Close()
+
+	// Resume (new process): $0, no new provider calls, ruby persist stays idempotent.
+	r2 := newRunner(t, bookPath)
+	defer r2.Close()
+	res2, err := r2.TranslateBook(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if rec.count() != 4 || res2.TotalUSD != 0 {
+		t.Fatalf("resume must be $0 with no new calls: calls=%d usd=%v", rec.count(), res2.TotalUSD)
+	}
+	assertRuby(r2.Store, 2)
+}
+
 func TestRunnerSnapshotPinning(t *testing.T) {
 	rec := &reqRec{}
 	srv := newJSONProvider(rec, draftEdit)
--- a/backend/internal/store/migrate.go
+++ b/backend/internal/store/migrate.go
@ -133,6 +133,33 @@ var migrations = []string{
 	// in the same tx as the settle), NOT request_log, so the sum is durable across
 	// resume / kill -9 (telemetry is fire-and-forget; money-path budgeting is not).
 	`ALTER TABLE checkpoints ADD COLUMN escalation INTEGER NOT NULL DEFAULT 0;`,
+	// v4: ruby/furigana readings captured on ingest (шаг 3a, 04-unhappy §4 / D9). A
+	// ja epub's <ruby>base<rt>reading</rt></ruby> carries the AUTHOR's reading of a
+	// name/term; epub-v1 drops inline markup (02-mvp:25), which would silently kill
+	// this layer, so ingest EXTRACTS the readings into this table instead. It is NOT
+	// injected into any prompt (that is memory v2 + determinism-load-bearing) — this
+	// step only CAPTURES + PERSISTS; memory v2 (шаг 4) consumes it into a glossary
+	// name-lock (base→dst via reading, first_chapter→since_ch — D7 schema). PK
+	// (book_id, base, reading): the same name may carry several readings across a
+	// book (variant yomi), each a distinct row. first_chapter = the MIN chapter the
+	// pair appears in (a glossary "since"); occurrences = the pair's full-book count
+	// (a memory-v2 confidence signal). The persist is idempotent: ingest aggregates
+	// each (base,reading) once over the WHOLE book, so re-ingesting the same source
+	// re-derives identical rows and BOTH first_chapter and occurrences are REPLACED
+	// with those authoritative full-book values (never an increment or a MIN-pin that
+	// would drift on resume or stale-early after a source edit).
+	`
+	CREATE TABLE IF NOT EXISTS ruby_readings (
+		book_id       TEXT NOT NULL,
+		base          TEXT NOT NULL,   -- the ruby BODY: the kanji/base surface form
+		reading       TEXT NOT NULL,   -- the <rt> reading (furigana) captured for it
+		first_chapter INTEGER NOT NULL,-- 1-based full-book MIN chapter the pair appears in (glossary "since"; idempotent REPLACE)
+		occurrences   INTEGER NOT NULL DEFAULT 0, -- full-book count (idempotent REPLACE, not increment)
+		created_at    TEXT NOT NULL DEFAULT (datetime('now')),
+		PRIMARY KEY (book_id, base, reading)
+	);
+	CREATE INDEX IF NOT EXISTS ruby_readings_book_idx ON ruby_readings (book_id);
+	`,
 }

 // migrate runs all pending migrations on the write pool, one transaction per
--- a/backend/internal/store/ruby.go
+++ b/backend/internal/store/ruby.go
@ -0,0 +1,67 @@
+package store
+
+// ruby.go: the ruby/furigana readings captured on ingest (шаг 3a, v4 schema).
+// This is CAPTURE-ONLY storage — the reading layer that epub-v1 text extraction
+// would otherwise silently drop (04-unhappy §4 / D9). Memory v2 (шаг 4) reads it
+// to seed a glossary name-lock; nothing here injects into a prompt, so no field
+// is snapshot- or wire-load-bearing (it never touches request_hash).
+//
+// The store is dumb storage (mirrors chunkstatus.go): the aggregation "one row per
+// (base,reading) with first_chapter=MIN and occurrences=full-book count" happens in
+// the pipeline (ingest.go); this layer only upserts and reads back.
+
+// RubyReading is one (book, base, reading) row.
+type RubyReading struct {
+	BookID       string
+	Base         string // the ruby body (kanji/base surface form)
+	Reading      string // the <rt> reading (furigana)
+	FirstChapter int    // MIN 1-based chapter the pair appears in
+	Occurrences  int    // full-book count of the pair
+}
+
+// UpsertRubyReading writes (or converges) one ruby row. Idempotent by design: the
+// caller (persistRuby) recomputes the WHOLE-book aggregate on every ingest, so it
+// passes the authoritative first_chapter (the full-book MIN) and occurrences (the
+// full-book count) already. Both columns are therefore REPLACED on conflict —
+// re-ingesting the identical source rewrites identical values (idempotent), and a
+// source edit that moves the pair's first appearance re-converges to the truth. A
+// MIN-merge here would instead pin first_chapter to a stale-early chapter the pair
+// no longer occupies after such an edit (self-review finding), asymmetric with
+// occurrences. Callers pass one row per (base,reading); the ON CONFLICT only fires
+// across separate ingests of the same book.
+func (s *Store) UpsertRubyReading(rr RubyReading) error {
+	ctx, cancel := opContext()
+	defer cancel()
+	_, err := s.w.ExecContext(ctx, `
+		INSERT INTO ruby_readings (book_id, base, reading, first_chapter, occurrences)
+		VALUES (?, ?, ?, ?, ?)
+		ON CONFLICT (book_id, base, reading) DO UPDATE SET
+			first_chapter = excluded.first_chapter,
+			occurrences   = excluded.occurrences`,
+		rr.BookID, rr.Base, rr.Reading, rr.FirstChapter, rr.Occurrences)
+	return err
+}
+
+// RubyReadingsForBook returns every captured reading for a book, ordered
+// deterministically (first_chapter, base, reading) for a stable report/consumer.
+func (s *Store) RubyReadingsForBook(bookID string) ([]RubyReading, error) {
+	ctx, cancel := opContext()
+	defer cancel()
+	rows, err := s.r.QueryContext(ctx, `
+		SELECT base, reading, first_chapter, occurrences
+		FROM ruby_readings WHERE book_id = ?
+		ORDER BY first_chapter, base, reading`, bookID)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+	var out []RubyReading
+	for rows.Next() {
+		rr := RubyReading{BookID: bookID}
+		if err := rows.Scan(&rr.Base, &rr.Reading, &rr.FirstChapter, &rr.Occurrences); err != nil {
+			return nil, err
+		}
+		out = append(out, rr)
+	}
+	return out, rows.Err()
+}
--- a/backend/internal/store/ruby_test.go
+++ b/backend/internal/store/ruby_test.go
@ -0,0 +1,87 @@
+package store
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestRubyReadingsUpsertAndRead(t *testing.T) {
+	s, _ := openTemp(t)
+	rows := []RubyReading{
+		{BookID: "b", Base: "漢字", Reading: "かんじ", FirstChapter: 2, Occurrences: 5},
+		{BookID: "b", Base: "東京", Reading: "とうきょう", FirstChapter: 1, Occurrences: 3},
+		{BookID: "other", Base: "京", Reading: "きょう", FirstChapter: 1, Occurrences: 1},
+	}
+	for _, rr := range rows {
+		if err := s.UpsertRubyReading(rr); err != nil {
+			t.Fatal(err)
+		}
+	}
+	got, err := s.RubyReadingsForBook("b")
+	if err != nil {
+		t.Fatal(err)
+	}
+	// Ordered by (first_chapter, base, reading): 東京(ch1) before 漢字(ch2).
+	want := []RubyReading{
+		{BookID: "b", Base: "東京", Reading: "とうきょう", FirstChapter: 1, Occurrences: 3},
+		{BookID: "b", Base: "漢字", Reading: "かんじ", FirstChapter: 2, Occurrences: 5},
+	}
+	if !reflect.DeepEqual(got, want) {
+		t.Fatalf("read = %#v, want %#v (other book must not leak)", got, want)
+	}
+}
+
+func TestRubyReadingsUpsertIsIdempotentAndConverges(t *testing.T) {
+	s, _ := openTemp(t)
+	rr := RubyReading{BookID: "b", Base: "漢字", Reading: "かんじ", FirstChapter: 3, Occurrences: 4}
+	if err := s.UpsertRubyReading(rr); err != nil {
+		t.Fatal(err)
+	}
+	// Re-ingesting the IDENTICAL full-book aggregate is a no-op (idempotent): one row,
+	// same values, occurrences never doubled.
+	if err := s.UpsertRubyReading(rr); err != nil {
+		t.Fatal(err)
+	}
+	got, err := s.RubyReadingsForBook("b")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(got) != 1 || got[0].FirstChapter != 3 || got[0].Occurrences != 4 {
+		t.Fatalf("identical re-upsert must be a no-op: %#v", got)
+	}
+	// A re-ingest after a source edit (e.g. a foreword inserted) moves the pair's
+	// first appearance to ch 4 and changes the count. persistRuby recomputes the
+	// authoritative full-book aggregate, so the store must REPLACE BOTH columns —
+	// it must NOT MIN-pin first_chapter to the stale-early 3 where the pair no
+	// longer occurs (self-review finding).
+	if err := s.UpsertRubyReading(RubyReading{BookID: "b", Base: "漢字", Reading: "かんじ", FirstChapter: 4, Occurrences: 2}); err != nil {
+		t.Fatal(err)
+	}
+	got, err = s.RubyReadingsForBook("b")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(got) != 1 || got[0].FirstChapter != 4 || got[0].Occurrences != 2 {
+		t.Fatalf("re-ingest must REPLACE (converge) both columns, not MIN-pin: %#v", got)
+	}
+}
+
+func TestRubyReadingsVariantReadingsAreDistinctRows(t *testing.T) {
+	// The same base with two readings (variant yomi) are two rows, not a collision.
+	s, _ := openTemp(t)
+	for _, rr := range []RubyReading{
+		{BookID: "b", Base: "海", Reading: "うみ", FirstChapter: 1, Occurrences: 2},
+		{BookID: "b", Base: "海", Reading: "かい", FirstChapter: 4, Occurrences: 1},
+	} {
+		if err := s.UpsertRubyReading(rr); err != nil {
+			t.Fatal(err)
+		}
+	}
+	got, err := s.RubyReadingsForBook("b")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(got) != 2 {
+		t.Fatalf("variant readings must be distinct rows, got %#v", got)
+	}
+}
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@ -380,6 +380,27 @@ keep-alive (Ф1–2), инъекция глоссария (`selective`), пор

 **Веха 2.5 ПОЛНОСТЬЮ закрыта** (реализация + селфревью 14 + внешнее ревью 7). Дальше — D12-хвосты по приоритету оркестратора (память v2 → tmctl status → F3).

+### 2026-07-05 — Шаг 3a: настоящий чанкер + импорт txt/epub + ruby-захват
+
+Снят плейсхолдер-чанкер: книга (D9 ja→ru ранобэ epub) гоняется end-to-end на настоящих границах. Всё зелёное `go build/vet/test -race`, `tmctl report` $0, живые LLM не звались (моки + $0-report). Три части + селфревью с фиксами.
+
+**A. Настоящий сорс-чанкер (`chunker.go`).** `SplitChunks(source string)`→`SplitChunks(chapters []string)` (ingest уже режет главы). Паковка **по границам предложений/абзацев** в **коридор ~1–2k токенов** (`targetChunkTokens=1500`, **const покрыт `chunkerVersion`**, не конфиг — Р2=код; если станет конфигом — обязателен snapshot-fold как `coverageSnap`, §7d). Отдельный сорс-сплиттер `splitSourceSentences` (zh/ja/en): CJK zero-width `。！？`, **quote-depth** (терминатор внутри `「…！」と…。` — НЕ конец предложения, диалоговый кейс §3.7/D12-Q1), эллипсис `…`, абсорбция закрывающих скобок, guard аббревиатур/инициалов, **lossless-тайлинг** (конкатенация сегментов = исходный абзац, текст не теряется/не сдвигается). **Никогда не режет предложение**; абзац>цели → дробится по предложениям, предложение>цели → свой чанк. **ОТДЕЛЬН от coverage-`splitSentences`** (тот Python-locked по РУССКОМУ ВЫХОДУ; этот — по ИСХОДНИКУ, `chunkerVersion`-versioned). `chunkerVersion` бампнут `v2-ff-para-pack`→**`v3-sentence-pack`** (в snapshot → перечанкование = громкий `--resnapshot`).
+
+**B. Импорт txt/epub (`ingest.go`, новый).** `Ingest(path)→*Document{Chapters []string, Ruby []RubyReading}`, диспетчер по расширению. **txt:** главы по `\f` (обратная совместимость example), `NormalizeSource` каждой. **epub — чисто stdlib** (CGO-free, без новых зависимостей): `archive/zip`→`container.xml`→OPF→**spine-порядок**→`encoding/xml` (`Strict=false`, `HTMLEntity`, `HTMLAutoClose`)→стрип тегов, скип `head/script/style`, block-теги→абзацные разрывы. Битый/пустой/чужой epub — **fail-loud, не паника** (нет container/opf/spine, dangling idref, не-zip). epub v1 = текст по spine, инлайн-разметка не сохраняется (осознанно, 02-mvp:25). Раннер: `os.ReadFile+NormalizeSource+SplitChunks` → `Ingest→SplitChunks(doc.Chapters)→persistRuby` (минимальный след, шов цикла цел; `ch.Text` в 3 местах не тронут).
+
+**C. Ruby-захват + персист (`store/ruby.go`, миграция v4).** `<ruby>base<rt>reading</rt></ruby>` → **base в тело, reading захвачен, `<rp>` скобки-фолбэк дропнуты**, mono-ruby склеен. Персист в **v4 `ruby_readings(book_id, base, reading, first_chapter, occurrences)`** (PK `(book_id,base,reading)` — вариантные чтения = разные строки). **Идемпотентный upsert:** `persistRuby` агрегирует по всей книге раз на инжест → оба поля **REPLACE** (ре-инжест того же источника = те же строки, правка источника = сходится к правде). **НЕ инъектится в промпт** (§7d, память v2 + детерминизм-load-bearing) → не в snapshot/`request_hash`. Память v2 (шаг 4) потребит → глоссарий-лок имён (D9).
+
+**Селфревью (адверсариальный Workflow: 5 дименсий → находки → независимая CONFIRMED/REFUTED-верификация с конкретным сценарием инпут→неверный-выход).** Сам нашёл до ревью 1 баг (dense-numbering, ниже). Ревью: 9 находок, 4 подтверждены (≥1 верификатором), **все 4 исправлены + регресс-тесты мутационно-проверены** (реверт → падает именно его тест):
+1. *[dense-numbering, до ревью]* ruby.Chapter брал spine-read-index, а `SplitChunks` нумерует главы **плотно** (пустые cover/nav скипаются) → `first_chapter` расходился с осью глав раннера (сдвиг `since_ch` памяти v2). Фикс: ingest нумерует тем же плотным правилом.
+2. *[segmentation]* паковка **суммировала per-unit `EstimateTokens`** (флор 16 на каждый абзац/предложение) → глава из коротких реплик дробилась в суб-500-симв. чанки, которые coverage-гейт молча скипает. Фикс: аддитивный подсчёт классов символов, флор-16 **один раз на весь чанк** (= `EstimateTokens` от склейки).
+3. *[epub]* `isXHTML` фолбэк на расширение только при ПУСТОМ media-type → глава с `application/xml`/`text/xml`/`…; charset=utf-8` молча терялась (реальные epub так мислейблят). Фикс: стрип параметров + фолбэк на расширение при ЛЮБОМ нераспознанном типе.
+4. *[epub]* форматирующий whitespace МЕЖДУ base-сегментами ruby (pretty-printed jukugo `<rb>`) тёк в base (мис-ключ глоссария) и в `ch.Text`. Фикс: whitespace-only CharData внутри ruby-base дропается.
+5. *[ruby-persist]* `first_chapter` был MIN-merge, `occurrences` — REPLACE (асимметрия): после правки источника, сдвигающей первое появление позже, `first_chapter` застревал на устаревшей ранней главе. Фикс: оба REPLACE (агрегат `persistRuby` — авторитетный full-book).
+
+**Отклонено (обоснованно, оба верификатора REFUTED):** голый `<` в прозе валит весь epub — это **корректный fail-loud** на невалидном xhtml (альтернатива = тихая потеря главы, против инварианта); zip-bomb LimitReader — вне threat-model MVP (оператор-файл), в докстринг как future-hardening. Dense-numbering-находка ревью REFUTED — потому что уже был исправлен (верификаторы читали пофикшенный код).
+
+**Границы/техдолг (не блокеры):** чанк <500 симв. молча выключает excision-гейт — **документированная граница §7b** (после фикса #2 случается только на реально-крошечной главе/хвосте); overlap-инъекция и ruby→глоссарий-лок отложены (§3, ждут msgs-поверхности памяти v2 — строим один раз); epub v1 не сохраняет инлайн-разметку; zip-bomb hardening — future. Форму `ruby_readings` согласовать с памятью v2 при шаге 4 (контракт: base→dst через reading, first_chapter→since_ch, occurrences=confidence). Тесты: чанкер (сегментация/паковка/детерминизм/аббревиатуры/quote-depth), ingest (txt/epub/spine-порядок/entity/ruby/битый-epub/dense-numbering/media-types), ruby-persist (идемпотентность+сходимость), e2e мульти-глава epub→чанки→мок-перевод→resume $0.
+
 ## Полигон
 (секция параллельной сессии — записи добавлять сюда)