WebSocket Protocol

The OmniLoop WebSocket protocol provides the bidirectional control plane for telemetry, replay, and live variable mutation.

Connection

Endpoint: ws://127.0.0.1:8000/ws

Protocol Version: 1

Note

The protocol version is bumped on breaking changes only. Additive messages and fields do not require a version bump. The canonical version lives in protocol/omniloop_protocol.json, loaded by both the Python server and the dashboard client at startup.

Handshake

The server’s first message on every connection is hello:

{
  "type": "hello",
  "protocol": 1,
  "server": "OmniLoop",
  "auth_required": false,
  "capabilities": ["mutation", "watch", "replay", "fork", "export"],
  "principal": {"id": "local", "display_name": "Local user"}
}

• capabilities reflects what this server will actually allow — e.g. fork is omitted when timeline forking is disabled, so clients hide the control instead of hitting runtime errors.
• principal is who the server believes this connection is.
• Clients should reply with {"client_hello": {"protocol": 1, "client": "<name>/<version>"}}; clients that never do are treated as legacy and remain fully supported. The server never rejects a connection over a version mismatch.

Authentication

With no token configured (the default), every connection is the implicit local principal. When the server is started with OMNILOOP_AUTH_TOKEN, the first hello arrives with "auth_required": true and no capabilities; the client must then send {"auth": {"token": "..."}} within 5 seconds (or connect with ?token=... in the URL). On success a second, fully populated hello follows; on failure the socket closes with code 4401. HTTP endpoints accept the same token as Authorization: Bearer <token>. Websocket handshakes from browsers are additionally subject to an Origin allow-list (OMNILOOP_ALLOWED_ORIGINS); disallowed origins close with 4403.

HTTP endpoints

The websocket carries the control plane. A few things are HTTP instead, because they are either too large for a latency-sensitive channel or need to work before one is open.

Endpoint	Method	Purpose
/health	GET	Cheap readiness probe. No auth, no shared-memory read.
/stats	GET	IPC counters, pending watch trip, latest limit clamp, session id.
/trace/upload	POST	Upload an .omni (multipart file) from the dashboard’s picker or drop zone. Returns {"name", "size"}; the name feeds straight into the load_trace action. Capped at 512 MB; refuses with 400 and a detail message the client shows verbatim.
/	GET	The packaged dashboard, when this install has one. Mounted last so it cannot shadow the routes above.

Capabilities

The protocol supports the following capabilities:

Capability	Description
mutation	Live parameter editing
watch	Watchpoint support
replay	Journal replay
fork	Timeline forking
export	State export

Messages

Server-to-Client Messages

Messages sent from the OmniLoop server to the connected client.

Message	Description
hello	Handshake: protocol version, capabilities, principal, and the current live_events buffer (always first)
telemetry	State frame from the target process. Also carries live_events on any frame where the server’s event buffer has grown, so the Event Console updates during a session rather than only on reconnect.
schema	Parameter schema declaration
replay_info	Journal metadata + events for timeline
state_exported	Response to export_state action
info	Informational message
error	Error message
trace_list	Response to list_traces action with available journal files
go_to_line	Source navigation request
set_execution_node	Execution node update
causal_result	Response to cause_of / effects_of with the traced record chain
ack	Generic acknowledgement ({"action", "ok", "detail"}) for watch, set_bounds and dump_flight_recorder. Sent only to a client that supplied a request_id.
mutation_ack	Response to mutation: the batch’s correlation_id, the values actually applied, and anything rejected by a registered bound. Sent only to a client that supplied a request_id.
halt_context	Response to get_halt_context / wait_for_halt: why the loop is frozen — trip, exception, file, line, surrounding source, state frame
session_list	Response to list_sessions: channel state, publisher PIDs, and collision warnings per session
channel_health	Response to get_channel_health: IPC counters plus what they imply about the trustworthiness of live telemetry
timing_report	Response to analyze_timing: tick-duration distribution and deadline overshoots
clamp_log	Response to get_clamps: recent clamps/refusals and the registered bounds
watchpoint_list	Response to list_watchpoints: armed tripwire specifications and latched states
session_selected	Response to select_session: confirmation of session binding

Client-to-Server Messages

Messages sent from the client to the OmniLoop server.

Message	Format	Description
client_hello	{"client_hello": {"protocol": 1, "client": "..."}}	Client introduction after hello
auth	{"auth": {"token": "..."}}	First-message token authentication (see Handshake)
mutation	{"mutation": {"<name>": <value>, ...}}	One or more parameter edits, keyed by variable name; the reserved key sys carries system commands (halt/resume/step)
watch	{"watch": {"name": ..., "condition": ..., "threshold": ...}} or {"watch": "clear"}	Arm/clear watchpoints
action	{"action": "...", ...params}	Trigger an action (see Actions below)

Request correlation (request_id)

Any client-to-server message may carry a request_id. The server echoes it on every message it sends in direct response, and never on anything it pushes unprompted (telemetry, broadcasts). A client with one request in flight at a time can ignore this entirely — the dashboard does. A client driving the whole surface over one socket cannot: message type is not unique per request, so without an id an error cannot be attributed to the question that caused it.

Supplying a request_id additionally opts the sender into the acknowledgement messages: mutation answers with mutation_ack, watch with ack. Without one, both stay fire-and-forget, which is what a slider being dragged wants.

{"action": "get_halt_context", "request_id": "7"}

Mutation Value Typing

Incoming mutation values are untyped JSON (usually strings from a UI control) and are coerced server-side before being applied. The target param_type is resolved from the variable’s declared schema, falling back to the Python type of its current telemetry value, and is one of:

• float
• int
• bool — accepts "true"/"1"/"yes" (case-insensitive) as true and "false"/"0"/"no" as false
• string

A value that fails coercion for its resolved type is rejected with an error message (see below) instead of being applied.

Error Message Shape

error messages are a flat JSON object:

{
  "type": "error",
  "code": "coercion",
  "message": "<human-readable description>"
}

code is a stable, machine-readable category. Switch on it to render targeted remediation; treat message as human-facing prose that may change between releases. An unrecognised code should be handled as internal.

Code	Meaning
auth	Missing, invalid, or expired token
capability	Authenticated, but the principal lacks the capability (or the server has the feature disabled)
coercion	A mutation value could not be coerced to its declared type
path	A trace/journal path is missing, disallowed, or unreadable
quota	A configured limit was exceeded (e.g. concurrent forks)
protocol	Malformed message, bad JSON, or unknown action
internal	Unhandled server-side failure

The set is defined by ERROR_CODES in omniloop/server/connection.py. Codes are append-only: an existing code is never renumbered or repurposed. Clients written against an older server must tolerate an error with no code field at all.

Actions

The following actions can be triggered by the client using the action message:

Action	Description
start_replay	Begin journal replay
stop_replay	Stop journal replay
get_frame	Request a specific frame by frame_index
fork_timeline	Restart worker from snapshot state
list_traces	List available .omni journal files
load_trace	Load a .omni file for replay
export_state	Export current state
clear_mutations	Clear pending mutations
reset_gains	Reset to initial values
trigger_exception	Deliberately trigger an exception (for testing)
go_to_line	Navigate to source line
set_execution_node	Set active execution node
cause_of	Trace what caused a record, by correlation_id
effects_of	Trace what a record caused, by correlation_id
get_param_schema	The declared parameter schema on demand, rather than only on connect. Answers schema.
list_sessions	Every routed session with its channel state (live/stale/absent), publisher PIDs and collision warnings, plus which one this connection is driving (selected). Answers session_list.
select_session	Point this connection at a different session, when the relay watches several. Every subsequent mutation, halt and step goes there. Unknown ids are refused, because a client that thinks it switched and did not would send its next Halt to the wrong controller. Answers session_selected.
get_halt_context	Why the loop is frozen: tripped watchpoint, exception type and message, file, line, the ±5 lines of source around it, the traceback (read from the journal), and the state frame at freeze. Answers halt_context.
wait_for_halt	Blocks up to timeout seconds (default 30, max 600) until the loop halts, then answers halt_context — so a client never has to poll a control loop.
get_channel_health	IPC counters plus the delivered/skipped ratio and what it means for the data. Answers channel_health.
analyze_timing	Tick-duration distribution, jitter and deadline overshoots over the observed window. Optional deadline_ns. Answers timing_report.
get_clamps	Recent limit clamps and refusals, plus the registered bounds. Optional limit. Answers clamp_log.
list_watchpoints	The tripwires the target has armed: name (a variable, a prefix_* group, or * for the global non-finite switch), condition, threshold, latched, and members. Answers watchpoint_list with target_reports_watchpoints, which separates “nothing is armed” from “the target never said”.
set_bounds	Register a hard [min, max] for a variable, in the target’s core and as a relay-side refusal. Answers ack with target_confirmed and the layers actually armed.
dump_flight_recorder	Capture a black box now. Asks the target to dump its own ring; falls back to the frames this server observed. Optional bare name, optional source: "relay". Answers ack with the path, record count and source.

Check which layer answered

set_bounds reaches two enforcement points, and only one of them protects against the loop’s own code:

• the target’s registry, over the command ring, where the core clamps every write into registered memory — confirmed by the target republishing its whole limit set under __bounds__;
• the relay, which refuses an out-of-range mutation before publishing it.

target_confirmed: false means only the relay guard is armed: the target is halted (a parked loop drains no commands), not publishing, or running an SDK that predates the command. Treat it as “the envelope is not installed”.

dump_flight_recorder likewise reports source. target is the loop’s own ring — every frame it published, at full rate. relay is the fallback: the frames this server managed to read, with frames_skipped_by_channel counting the ones it never saw.

This table is asserted against the server’s registered handlers by test_documented_actions_match_registered_handlers, so a new action cannot ship undocumented.

Target-published readouts

Two reserved telemetry keys carry state only the target process can report, because only it can perform the operation behind them. Both are folded onto frames repeatedly rather than once: the telemetry channel is a single-slot latest-wins slot, so a value present on exactly one frame is dropped outright — not merely delayed — on any loop publishing faster than its reader polls.

Key	Meaning
__bounds__	Every limit the target’s core is enforcing, as {name: {min, max}}. Present on every frame while anything is bounded, so an envelope can be verified at any moment — including limits the loop declared itself via TrainingLoop(limits=...), which are otherwise invisible from outside the process.
__blackbox_dump__	Where the target wrote its flight-recorder ring and how many records it held. Republished for ~3 s after the dump.

Correlation IDs

Every mutation batch arriving from a client is assigned a server-generated UUID v4 correlation_id. This ID is threaded through the IPC command ring, the state registry, and into journal event records, so any state change can be traced back to the exact user action that caused it.

• The ID is generated per mutation message, not per key — all keys in a single {"mutation": {"lr": 0.01, "bs": 32}} share one ID.
• Journal records written as a result of the mutation include "correlation_id": "<uuid>" in their JSON payload.
• The dashboard’s Event Console displays the first 8 characters of the ID next to each event entry.
• Post-hoc queries: effects_of(journal_path, correlation_id) returns every record whose payload contains the given ID.

Jitter Tracking

The SDK instruments tick wall-clock duration to detect deadline overruns:

Field	Source	Description
__tick_duration_ns__	Telemetry payload	Wall-clock nanoseconds of the most recent tick, injected by TrainingLoop._build_payload when > 0

Deadline Overshoot Watchpoint

Arm via the watch message:

{
  "watch": {
    "name": "__tick_duration__",
    "condition": "deadline_overshoot",
    "threshold": 0.005
  }
}

When a tick exceeds threshold seconds, the watchpoint trips and the loop halts (same behavior as other watchpoints). The threshold also configures the registry’s tick_deadline_ns so the check runs on every tick.

Journal Record Types

The v2 journal format supports the following FrameKind values:

Kind	ID	Description
telemetry	0	State snapshot (one per tick)
mutation	1	Parameter change event
control	2	System command (halt/resume/step)
lifecycle	3	Session start/stop
hash	4	State-hash checkpoint for divergence detection
input	5	External inputs for deterministic replay (IPC commands, RNG seeds)

Input Records

input records capture the exact external inputs received during a tick so that a replay can feed them back in the same order. They are written automatically by TrainingLoop when record_inputs=True (the default when a journal is attached).

Deterministic Replay Engine

The replay engine re-executes a recorded journal by feeding back the exact inputs that were captured during the original run. State-hash checkpoints are verified on each tick to detect divergences.

Rust Core: InputReplayer

InputReplayer::open(path) pre-indexes all input records, hash checkpoints, and telemetry states into per-tick hash maps for O(1) lookup:

Method	Returns	Description
tick_count()	u64	Total ticks in the journal
inputs_for_tick(tick)	&[Vec<u8>]	Input payloads recorded at tick
expected_hash(tick)	Option<u64>	Hash checkpoint, if one was recorded
state_at_tick(tick)	Option<&[u8]>	Telemetry state snapshot at tick
verify_tick(tick, actual_hash)	TickVerification	Match, Diverged{expected,actual}, or NoCheckpoint
hash_checkpoint_count()	usize	Number of ticks with hash checkpoints
all_inputs_ordered()	Vec<(u64, Vec<u8>)>	All inputs sorted by tick

Python: DeterministicReplay

DeterministicReplay(journal_path, output_path=None) wraps InputReplayer with JSON parsing and a ReplayReport accumulator:

from omniloop import DeterministicReplay

replay = DeterministicReplay("run.omni")
for tick in range(replay.tick_count):
    mutations = replay.mutations_for_tick(tick)
    # ... apply mutations, step sim, compute state_hash ...
    replay.submit_hash(tick, state_hash)
report = replay.report()

TrainingLoop.replay()

Factory classmethod that creates a TrainingLoop in replay mode — mutations are sourced from the recorded journal instead of live IPC, and hash checkpoints are verified automatically:

loop = TrainingLoop.replay("run.omni", cfg, tunable=["lr"])
for _ in range(loop.replay_tick_count):
    with loop.tick():
        loop.log(reward_mean=compute_reward())
print(loop.replay_report())

Fork Timeline with Journal Verification

When fork_timeline is invoked and a journal file exists for the session, the forked process receives --replay-journal <path> and --fork-tick <N> arguments. This allows the forked process to verify its starting state against the recorded journal before beginning independent execution.

LTTng Kernel Tracepoint Import

lttng_to_journal(lttng_path, journal_path, ...) converts babeltrace2 JSON output into .omni journal event records so kernel-level scheduling events can be viewed alongside control-loop telemetry in the same replay timeline.

Supported tracepoints: sched_switch, sched_wakeup, sched_wakeup_new, irq_handler_entry, irq_handler_exit, softirq_entry, softirq_exit, hrtimer_expire_entry, hrtimer_expire_exit.

Options: pid_filter (only import events involving a specific PID), tracepoints (subset of tracepoint names), tick_duration_ns (synthesize tick boundaries from timestamps).

Journal Query API

Two query functions are available for post-hoc causal analysis:

• cause_of(path, tick) — Returns all causal events (mutation, control, lifecycle, input) at or before the given tick, in journal order.
• effects_of(path, correlation_id) — Returns all records whose JSON payload contains the given correlation ID (string search).