omniloop doctor

When the dashboard looks stale, there are usually three main suspects:

1. Loop not publishing: The training loop has stopped sending data.
2. Server can’t keep up: The server is polling too slowly, causing frames to be overwritten.
3. Session mismatch: The loop and server are communicating on different OMNILOOP_SESSION_IDs.

The OmniLoop IPC engine maintains strict counters for all traffic, allowing you to quickly diagnose these issues.

Diagnostic Counters

The following counters are tracked per engine handle:

Counter	Meaning
telemetry_published	Frames this handle wrote.
telemetry_last_bytes	Size of the last written frame.
telemetry_delivered	Frames this handle received (genuinely new frames).
telemetry_skipped	Frames that were overwritten before the consumer polled them.
telemetry_torn_reads	Number of seqlock retries due to concurrent writes.
commands_published	Commands written to the command ring.
commands_received	Commands read from the command ring.
command_ring_full	Rejected command publishes because the ring was full.
command_corruption_errors	Errors reading the command ring (corrupted cursor/length).

Checking Health

There are three ways to access these statistics depending on where you are debugging.

1. From Python (SDK)

You can inspect the counters for the current process’s handle:

stats = tracker.stats()
print(stats["telemetry_skipped"])

2. From the Server API

You can curl the stats endpoint of a running server:

curl http://127.0.0.1:8000/stats

3. Using the CLI

The fastest way to diagnose an issue is the omniloop doctor command, which provides human-readable diagnostics of the IPC state.

omniloop doctor

If it detects an issue (like high telemetry_skipped or stagnant publishing), omniloop doctor will exit with a nonzero status code, making it highly scriptable for CI or automated monitoring setups.