237 lines
12 KiB
Text
237 lines
12 KiB
Text
# Target-language (ru) readability checker DATA (pair-14 data-out): the wordlists and DETECTION patterns the
|
||
# TARGET-GENERAL checkers/sanitizer run on ANY →ru output (regardless of source pair). Engine/target data —
|
||
# a no-langpack →ru book (the ja→ru golden) still runs these — so it is embedded here, not a book pack; the
|
||
# checker/sanitizer ALGORITHMS stay generic Go. `category<TAB>value`; value VERBATIM (a regex or a literal);
|
||
# lines are grouped by category IN ORDER (order matters for the sanitizer preamble preview). Add a target =
|
||
# add data/target-<tgt>.txt to the go:embed manifest (the target registry DERIVES from the file name).
|
||
#
|
||
# word_script: the target's WORD ALPHABET — the Unicode letter script the lexical checkers tokenize words on
|
||
# (whole-word wordlist match, broken-word/yofikation scan, latin-residue detection). The engine resolves the
|
||
# NAME to a Unicode range; it never branches on a language. `cyrillic` is the range for ru (unicode.Cyrillic).
|
||
word_script cyrillic
|
||
#
|
||
# broken-word: a Russian word ending in this suffix is a mangled infinitive (no valid word ends in «-йть»).
|
||
broken_suffix йть
|
||
# speech-verb attribution: the verbs that make a chevron-quoted line SPOKEN dialogue rather than a
|
||
# thought. The dialogue-dash MIXING rule counts a chevron line only when its attribution carries one of
|
||
# these, i.e. the DEFAULT for an unrecognised attribution is "not a style clash" — precision over recall,
|
||
# the gate's stated bias. The inverse list (thought verbs) was tried first and proved unbounded: a live
|
||
# 2-chapter run immediately produced «понимал» and «размышлял», which no enumeration of thought verbs
|
||
# would have covered, while the speech-verb side is small and closed. Lower-cased, substring match.
|
||
speech_verb сказал
|
||
speech_verb сказала
|
||
speech_verb ответил
|
||
speech_verb ответила
|
||
speech_verb спросил
|
||
speech_verb спросила
|
||
speech_verb крикнул
|
||
speech_verb крикнула
|
||
speech_verb воскликнул
|
||
speech_verb воскликнула
|
||
speech_verb прошептал
|
||
speech_verb прошептала
|
||
speech_verb пробормотал
|
||
speech_verb пробормотала
|
||
speech_verb проговорил
|
||
speech_verb проговорила
|
||
speech_verb продолжил
|
||
speech_verb продолжила
|
||
speech_verb добавил
|
||
speech_verb добавила
|
||
speech_verb возразил
|
||
speech_verb возразила
|
||
speech_verb позвал
|
||
speech_verb позвала
|
||
speech_verb объявил
|
||
speech_verb объявила
|
||
speech_verb произнёс
|
||
speech_verb произнес
|
||
speech_verb произнесла
|
||
# inner-speech VETO markers: a qualifier that makes a line inner speech even when its attribution verb
|
||
# is a speech verb — «пробормотал ПРО СЕБЯ» is a thought, not a spoken turn. Checked after speech_verb,
|
||
# so the pair (verb + veto) reproduces the writer's actual intent instead of the verb alone. Found by the
|
||
# 26.07 live verification: it was the single flag surviving on the real 10-chapter export.
|
||
inner_marker про себя
|
||
inner_marker мысленно
|
||
inner_marker в мыслях
|
||
inner_marker себе под нос
|
||
inner_marker в уме
|
||
# yofikator homograph whitelist: е-spellings that are DISTINCT words from their ё-counterpart (все≠всё,
|
||
# небо≠нёбо, берет≠берёт …), so the inconsistency rule must NOT flag «все»+«всё» as one word two ways.
|
||
yo_homograph все
|
||
yo_homograph всех
|
||
yo_homograph всем
|
||
yo_homograph всеми
|
||
yo_homograph небо
|
||
yo_homograph небом
|
||
yo_homograph узнаем
|
||
yo_homograph узнаете
|
||
yo_homograph узнает
|
||
yo_homograph падеж
|
||
yo_homograph падежа
|
||
yo_homograph совершенный
|
||
yo_homograph совершенное
|
||
yo_homograph совершенная
|
||
yo_homograph совершенно
|
||
yo_homograph совершенны
|
||
yo_homograph чем
|
||
yo_homograph тем
|
||
yo_homograph тема
|
||
yo_homograph теме
|
||
yo_homograph темы
|
||
yo_homograph берет
|
||
yo_homograph берете
|
||
yo_homograph берета
|
||
yo_homograph осел
|
||
yo_homograph осла
|
||
yo_homograph ослы
|
||
yo_homograph ослов
|
||
yo_homograph мел
|
||
yo_homograph мела
|
||
yo_homograph лен
|
||
yo_homograph лена
|
||
yo_homograph нес
|
||
yo_homograph несем
|
||
yo_homograph несет
|
||
yo_homograph вел
|
||
yo_homograph ведет
|
||
yo_homograph ведем
|
||
# magnitude_stem: Russian magnitude WORD stem → base-10 exponent, for the number-magnitude gate (a stem
|
||
# covers [base, base+2] since an unseen multiplier can lift it two orders). `magnitude_stem<TAB>stem<TAB>exp`.
|
||
magnitude_stem тысяч 3
|
||
magnitude_stem миллион 6
|
||
magnitude_stem миллиард 9
|
||
magnitude_stem триллион 12
|
||
# sanitizer DETECTION patterns (pair-14 data-out). The sanitizer is isRuTarget-gated, so these are
|
||
# ru-target data; the strip/detect ALGORITHM stays in sanitizer.go. `sanitizer_preamble` is ORDERED
|
||
# (detectLeadingPreamble returns the FIRST match's preview). Values VERBATIM (regexes).
|
||
sanitizer_preamble (?is)^\s*(?:вот|представляю|привожу|держите)\s+[^\n:]{0,40}?(?:отредактированн|исправленн|улучшенн)[а-яё]+\s+(?:перевод|текст|вариант)[а-яё]*[^\n:]{0,120}?:
|
||
sanitizer_preamble (?is)^\s*(?:отредактированн|исправленн|улучшенн)[а-яё]+\s+(?:перевод|текст|вариант)[а-яё]*[^\n:]{0,120}?:
|
||
sanitizer_preamble (?is)^\s*(?:вот\s+)?перевод\s+(?:фрагмента|текста|отрывка|главы|черновика)\s*:
|
||
sanitizer_preamble (?is)^\s*ниже\s+(?:привед[её]н|представлен|дан|след)[а-яё]*\s+[^\n:]{0,40}?(?:перевод|отредактированн|текст)[а-яё]*[^\n:]{0,20}?:
|
||
sanitizer_trailing_note (?i)^[*_>\s-]{0,4}(?:(?:примечани[ея]|заметк[аи]|комментари[йяю]|пояснени[ея]|сноск[аи])\s*:|(?:примечани[ея]|заметк[аи]|комментари[йяю])\s+(?:переводчик|редактор)[а-яё]*|прим\.\s*(?:перев|ред)\.?)
|
||
sanitizer_edit_meta (?im)^[*_>\s-]{0,4}(?:(?:внесённ|внесен)[а-яё]+\s+правк|основн[а-яё]+\s+правк[а-яё]*(?:\s+для\s+справк[а-яё]+|\s*:)|что\s+(?:было\s+)?(?:изменен|исправлен)|список\s+(?:правок|изменений)|список\s+внесённых)
|
||
sanitizer_invalid_sign (?i)(?:^|\P{L})[ъь][а-яё]|[ъь][ъь]
|
||
# --- pack-19 voice flagger (T/V + attributed-reply checks) -----------------------------------------
|
||
# The T/V distinction is a fact about the TARGET language, so it lives here: a target that ships no
|
||
# tv_informal/tv_formal rows runs the whole voice flagger inert, with no Go edit. Surfaces are matched
|
||
# as WHOLE WORDS on the normalized output, lower-cased; only PRONOUNS and possessives are listed —
|
||
# verb agreement is morphology and belongs to the ru-target layer, not to a $0 flagger.
|
||
tv_informal ты
|
||
tv_informal тебя
|
||
tv_informal тебе
|
||
tv_informal тобой
|
||
tv_informal тобою
|
||
tv_informal твой
|
||
tv_informal твоя
|
||
tv_informal твоё
|
||
tv_informal твое
|
||
tv_informal твои
|
||
tv_informal твоего
|
||
tv_informal твоей
|
||
tv_informal твоих
|
||
tv_informal твоим
|
||
tv_informal твою
|
||
tv_formal вы
|
||
tv_formal вас
|
||
tv_formal вам
|
||
tv_formal вами
|
||
tv_formal ваш
|
||
tv_formal ваша
|
||
tv_formal ваше
|
||
tv_formal ваши
|
||
tv_formal вашего
|
||
tv_formal вашей
|
||
tv_formal ваших
|
||
tv_formal вашим
|
||
tv_formal вашу
|
||
# tv_plural_veto: Russian «вы» is ALSO plural-you, so a reply addressing several people legitimately
|
||
# mixes «ты» and «вы» («Ты иди, а вы оба ждите»). A reply carrying one of these markers is not counted
|
||
# as a T/V contradiction — the same veto shape inner_marker has over speech_verb, and for the same
|
||
# precision-over-recall reason: a miss here is silence on a real clash, not noise on good prose.
|
||
tv_plural_veto оба
|
||
tv_plural_veto обе
|
||
tv_plural_veto вдвоём
|
||
tv_plural_veto втроём
|
||
tv_plural_veto все вы
|
||
tv_plural_veto вы все
|
||
tv_plural_veto господа
|
||
tv_plural_veto каждый из вас
|
||
# self_ref_generic: the PLAIN first-person surfaces. They are the evidence that a character referred to
|
||
# herself at all — the check is «she said "я" where her profile says "ваша служанка"», i.e. the measured
|
||
# flattening class (research/15 §Пробы, marker 奴婢), never «she said "я"» on its own.
|
||
self_ref_generic я
|
||
self_ref_generic меня
|
||
self_ref_generic мне
|
||
self_ref_generic мной
|
||
self_ref_generic мною
|
||
# reply segmentation: how this target sets direct speech. reply_dash opens a dash-marked turn at line
|
||
# start; reply_open/reply_close bound a quoted turn. Data, not a Go literal, so a target with other
|
||
# conventions (ja 「」) states its own without touching the segmentation algorithm.
|
||
reply_dash —
|
||
reply_open «
|
||
reply_close »
|
||
# translit-interjection blocklist (04-unhappy §9): JP/EN fillers that must be TRANSLATED/adapted, not
|
||
# transliterated into the target. Only surfaces that are NOT ordinary target words are listed (ара/уму were
|
||
# removed as high-FP collisions). The blocklist is TARGET data now (was a Go literal); the whole-word,
|
||
# case-insensitive matching ALGORITHM stays generic. D20.4: only the exact hyphenated reduplications listed
|
||
# fire — a generic «X-X» rule was rejected as too false-positive-prone. Extend per corpus finding.
|
||
translit_interjection ара-ара
|
||
translit_interjection маа
|
||
translit_interjection хмф
|
||
translit_interjection нани
|
||
translit_interjection ауч
|
||
translit_interjection упс
|
||
translit_interjection кья
|
||
translit_interjection десу
|
||
# decl_suffix: the productive Russian inflectional endings the decl-aware post-check stems a word by
|
||
# (bank-quality §3, primitive C). It lets dstFormPresent accept a term rendered in an oblique case
|
||
# («Фан Юаню» for «Фан Юань») WITHOUT the seed listing every form, closing the measured decl-142/142-null
|
||
# false-miss noise. TARGET data — the suffix set is ru-specific; the stemmer ALGORITHM (strip the longest
|
||
# listed ending that leaves a stem ≥ min length) stays generic Go and a target with no decl_suffix rows
|
||
# gets no stemming (inert). It is a FLAGGER aid, biased to ACCEPT declensions; it never gates. Ordered
|
||
# longest-first in code, so listing order here is free.
|
||
decl_suffix ами
|
||
decl_suffix ями
|
||
decl_suffix ого
|
||
decl_suffix его
|
||
decl_suffix ому
|
||
decl_suffix ему
|
||
decl_suffix ыми
|
||
decl_suffix ими
|
||
decl_suffix ах
|
||
decl_suffix ях
|
||
decl_suffix ов
|
||
decl_suffix ев
|
||
decl_suffix ом
|
||
decl_suffix ём
|
||
decl_suffix ем
|
||
decl_suffix ой
|
||
decl_suffix ей
|
||
decl_suffix ый
|
||
decl_suffix ий
|
||
decl_suffix ая
|
||
decl_suffix яя
|
||
decl_suffix ое
|
||
decl_suffix ее
|
||
decl_suffix ым
|
||
decl_suffix им
|
||
decl_suffix ых
|
||
decl_suffix их
|
||
decl_suffix ую
|
||
decl_suffix юю
|
||
decl_suffix а
|
||
decl_suffix я
|
||
decl_suffix у
|
||
decl_suffix ю
|
||
decl_suffix е
|
||
decl_suffix ы
|
||
decl_suffix и
|
||
decl_suffix о
|
||
# NB (fix-pack §а, D39.71): the bare SOFT SIGN «ь» is deliberately NOT a decl_suffix. It would close the
|
||
# Palladius soft-sign case («Фан Юаню»~«Фан Юань» via the stem «юан»), but that strip is byte-identical in
|
||
# shape to a HARMFUL collision — «Синь»/«Линь» (top zh surnames) collapse onto «син»/«лин», the stems of the
|
||
# common words «синий»/«линия», which fires a spurious Bank.SpoilerLeaks false-positive (the noise direction).
|
||
# «юань»→«юан» and «синь»→«син» are the same 4-rune-soft-sign→3-rune-stem shape, so no min-stem rule separates
|
||
# them. The soft-sign nominative gap needs seed decl-form enumeration or an anchor-gated match, not this global
|
||
# suffix — see the fix-pack report §а ping.
|