package runevents import ( "bytes" "errors" "fmt" "io" "os" "path/filepath" ) // JournalFile is the name of the event journal in the BOOK's directory. It is the platform's constant // too (`platform/internal/ingest/tail.go`) and is ratified by D39.106 §2 — one journal per BOOK, one // stream per PROCESS, appended to it. const JournalFile = "events.jsonl" // maxPartialTail bounds the repair scan below. It is the reader's own line cap: a tail longer than that // with no newline in it is not a torn line of ours, it is a file that is not what we think it is. const maxPartialTail = 1 << 20 // Journal is the append-only event journal of one book. // // FORMAT (the fork row 103 left open — single JSONL vs per-file events vs CRC framing): a single // `events.jsonl`, appended with one write(2) per line. The reader is already built for exactly this // (tail.go), so any other shape is a coordinated change of BOTH zones, not an engine decision — and the // objection recorded against it (research/25: JSONL is blind to a torn record that happens to end in a // valid newline) does not survive contact with the reader's own contract: // // - a torn record CANNOT acquire a newline while this process lives — the line and its newline are one // write(2), so the only torn tail is the one a dead process left behind; // - the reader deliberately leaves a trailing line WITHOUT a newline alone ("still being written"), so // it never MATERIALIZES one. // // So the torn tail is repaired at open, before this process's own hello. The repair PADS it into a blank // line — it does NOT truncate, and that distinction is load-bearing rather than stylistic. An earlier // version cut back to the last newline, reasoning that a reader's cursor can never stand past an // unterminated line. That reasoning covers only a cursor EARNED BY READING. The ratified consumer also // seeds one from `stat`: a new attempt starts at `journalSize(workdir)` — the raw size, torn fragment // included (platform/internal/runs/spawn.go, runs.go, reconcile.go). Truncating below it makes the very // next drain report "journal shrank … it is not append-only" (platform/internal/ingest/tail.go) and // quarantine the attempt's projection — on a healthy resumed run. Padding keeps the file monotone in // length, and a run of spaces plus a newline is exactly what that reader skips as a blank line. // // DURABILITY (the fsync fork): a line is handed to the kernel immediately and is NOT fsynced. The journal // is a projection of SQLite commits taken at synchronous=NORMAL — "survives kill -9; power loss is // outside the MVP threat model" (store/store.go) — and a projection cannot be more durable than the truth // it projects, so an fsync per event would buy nothing and cost one on every unit of a 4000-unit book. // What IS synced is the directory, exactly once, when the journal is created: the file's NAME is the one // thing no later write re-creates. // // There is no user-space buffer, and that is the answer to PD-61(а) rather than a discipline about it: // `os.Exit`, `log.Fatal` and a panic all skip defers, so a buffered writer would lose precisely the last // events — `finished` and `ceiling`, the two that say why the run ended. Nothing is retained, so nothing // can be lost by not flushing. Events are per unit and per stop, not per token, so the syscall is free. type Journal struct { f *os.File // w is where lines go — f itself, except in tests that need a writer which stops part-way. A short // write is the failure this type's retry logic exists for, and it cannot be provoked on a real file // without filling a disk. w io.Writer // pending is the tail of a line whose write stopped part-way (a full disk is the realistic cause). // Those bytes are ALREADY on the file, so the retry must send only the remainder: re-sending the // whole line would concatenate a prefix and a copy, and that is a malformed line no repair can // recognise later — the reader would refuse the journal from there on, permanently. pending []byte // pendingOf is the full buffer `pending` belongs to, so a caller that moves on to a DIFFERENT line // while a remainder is outstanding is detected instead of silently splicing two lines together. pendingOf []byte } // OpenJournal opens (creating if needed) the journal in dir and repairs a torn tail left by a previous // process. dir is the BOOK's directory — where book.yaml lives and where the platform tails. func OpenJournal(dir string) (*Journal, error) { path := filepath.Join(dir, JournalFile) created := false if _, err := os.Stat(path); os.IsNotExist(err) { created = true } else if err != nil { return nil, fmt.Errorf("runevents: stat %s: %w", path, err) } // The repair READS the tail and OVERWRITES it in place, so it needs a descriptor without O_APPEND — // with O_APPEND the kernel forces every write to the end, WriteAt included. The appending descriptor // is opened afterwards, once the file is sound. if !created { rw, err := os.OpenFile(path, os.O_RDWR, 0o644) if err != nil { return nil, fmt.Errorf("runevents: open %s: %w", path, err) } err = padTornTail(rw) rw.Close() if err != nil { return nil, err } } f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) if err != nil { return nil, fmt.Errorf("runevents: open %s: %w", path, err) } if created { // Only on creation: an append changes no directory entry, so this is once per book, ever. if d, derr := os.Open(dir); derr == nil { _ = d.Sync() d.Close() } } return &Journal{f: f, w: f}, nil } // padTornTail turns an unterminated trailing line into a blank one, in place. A file that ends in a // newline (or is empty) is left untouched, which is every ordinary case. // // It never shortens the file — see the type comment: a consumer may hold an offset taken from the SIZE, // and shrinking below it reads to that consumer as a journal that is not append-only. func padTornTail(f *os.File) error { st, err := f.Stat() if err != nil { return fmt.Errorf("runevents: stat journal: %w", err) } size := st.Size() if size == 0 { return nil } window := int64(maxPartialTail) if size < window { window = size } buf := make([]byte, window) if _, err := f.ReadAt(buf, size-window); err != nil && !errors.Is(err, io.EOF) { return fmt.Errorf("runevents: read journal tail: %w", err) } if buf[len(buf)-1] == '\n' { return nil } i := bytes.LastIndexByte(buf, '\n') if i < 0 && window < size { return fmt.Errorf("runevents: journal ends with a partial line longer than %d bytes — refusing to repair a file this cannot have written", maxPartialTail) } from := size - window + int64(i) + 1 // i == -1 (the whole file is one partial line) ⇒ from 0 blank := bytes.Repeat([]byte{' '}, int(size-from)) if _, err := f.WriteAt(blank, from); err != nil { return fmt.Errorf("runevents: blank the torn journal tail: %w", err) } if _, err := f.WriteAt([]byte{'\n'}, size); err != nil { return fmt.Errorf("runevents: terminate the torn journal tail: %w", err) } return nil } // Append writes one line and its newline. The whole buffer goes in ONE write(2) — that syscall is what // makes a torn line impossible while the process lives, and O_APPEND is what makes the position atomic // without a seek. // // A write that stops part-way (ENOSPC is the realistic cause) leaves those bytes on the file, so the // remainder is remembered and the retry sends only that. The caller retries by re-offering the SAME // line, because a failed append leaves the projection cursor where it was. func (j *Journal) Append(line []byte) error { buf := make([]byte, 0, len(line)+1) buf = append(buf, line...) buf = append(buf, '\n') out := buf if len(j.pending) > 0 { if !bytes.Equal(j.pendingOf, buf) { // Appending a DIFFERENT line after a half-written one splices two records into a malformed // third, and no later repair can tell where the seam was. It cannot happen through the // projection — a failed append leaves the cursor where it was, so the retry always re-offers // the same line — so this is an assertion, not a recovery path. The half-written line stays // unterminated, which the reader treats as "still being written", and the next open blanks it. return fmt.Errorf("runevents: refusing to append a new line while %d bytes of another are half-written", len(j.pending)) } out = j.pending // continue the line whose write stopped part-way } n, err := j.w.Write(out) if n > 0 && n < len(out) { j.pending, j.pendingOf = out[n:], buf } if err != nil { return fmt.Errorf("runevents: append to journal: %w", err) } if n != len(out) { return fmt.Errorf("runevents: short append to journal: wrote %d of %d bytes", n, len(out)) } j.pending, j.pendingOf = nil, nil return nil } func (j *Journal) Close() error { return j.f.Close() }