.omni v2 Container Format

The OmniLoop journal format has been upgraded to a more robust v2 format, featuring mid-file corruption resistance, sparse indexing, and embedded metadata.

When to Use

You rarely need to parse this format yourself. Use the Python or Rust APIs unless you are building a custom language binding or third-party analyzer.

Container Format

All integers in the format are little-endian.

Binary Layout

File header (32 bytes)
  [0..8)   magic            b"OMNILOG2"
  [8..12)  version          u32 = 2
  [12..16) flags            u32 (bit 0 reserved for compression)
  [16..24) created_wall_ns  u64 unix-epoch nanos
  [24..32) reserved         u64

Record (repeated; 36-byte header + payload)
  [0..4)   payload_len  u32
  [4..5)   kind         u8   0=telemetry 1=mutation 2=control 3=lifecycle 4=hash
  [5..8)   padding
  [8..16)  tick         u64  monotonic frame counter, stamped by the Rust core
  [16..24) mono_ns      u64  nanos since journal creation (monotonic clock)
  [24..32) wall_ns      u64  unix-epoch nanos
  [32..36) crc32        u32  IEEE CRC-32 of the payload
  [36..)   payload

Index (written at close; one entry per 64 records)
  magic b"OMNIIDX2" | count u64 | count x { tick u64, offset u64 }

Trailer (last 24 bytes)
  record_count u64 | index_offset u64 | magic b"OMNIEND2"

Record Kinds

The kind field in the record header maps to the following enum:

0 = telemetry
1 = mutation
2 = control
3 = lifecycle
4 = hash

Design Properties

Untrusted Payloads: Ticks and timestamps are stamped in Rust, not trusted from the payload.
Resilience: The IEEE CRC-32 per record means mid-file corruption raises an IOError, while truncation is handled gracefully.
Fast Seeking: The sparse index and trailer make seek_to_tick() $O(\log n)$ on finalized files. If a run crashes, it falls back to a sequential scan.

API Usage

Python

The Python API provides tools for parsing and inspecting journal files:

player = JournalPlayer("run.omni")
player.version()          # 2 (or 1 for legacy files)
player.record_count()     # total records, from the trailer (None if crashed)
player.seek_to_tick(4000) # next read returns first record with tick >= 4000

Rust Surface

In the Rust core (omniloop-core/src/journal.rs), the main primitives are:

StateJournal
JournalPlayer
FrameKind
FrameRecord