35 lines
747 B
Text
35 lines
747 B
Text
# Source-language sentence-splitter abbreviations (pair-14 §4): trailing tokens after which a lone "." is an
|
|
# abbreviation, not a sentence end (case-insensitive). Per SOURCE language — a CJK source uses 。 and needs
|
|
# none; `en` is the one that guards an ASCII period. `lang<TAB>token`. Add a source = add its section.
|
|
# NOTE: the chunker currently consumes the `en` section universally (its historical en-only design); making
|
|
# the splitter consume the BOOK's source section is a shallow follow-up (thread sourceLang into SplitChunks).
|
|
# --- en ---
|
|
en mr
|
|
en mrs
|
|
en ms
|
|
en dr
|
|
en prof
|
|
en st
|
|
en jr
|
|
en sr
|
|
en vs
|
|
en no
|
|
en vol
|
|
en ch
|
|
en fig
|
|
en col
|
|
en gen
|
|
en sgt
|
|
en capt
|
|
en lt
|
|
en rev
|
|
en gov
|
|
en sen
|
|
en rep
|
|
en etc
|
|
en inc
|
|
en ltd
|
|
en co
|
|
en mt
|
|
en ave
|
|
en rd
|