Halt & Step

Halting a loop means one thing in a training run and something entirely different on a machine that can move. This page is the contract for both.

The short version

A control loop that stops emitting commands does not pause — it trips a watchdog and the machine falls. If your loop drives actuators, set halt_policy="handoff" and give it a fallback controller. The default "block" policy freezes the calling thread, which is correct for simulation and wrong for hardware.

The three policies

The loop’s author declares up front what a halt is allowed to do. The barrier then reports a decision rather than acting on one.

Policy	At the barrier	Use for
block (default)	Blocks the calling thread until something releases the halt.	RL training, offline sim, replay — anything with no actuator downstream.
handoff	Returns immediately, reporting run_safety. Your loop keeps running at rate and drives a fallback controller instead of its normal body.	Hardware. The machine keeps being commanded and stays still.
advisory	Returns immediately, always reporting run. The halt is recorded, published and journaled but never changes control flow.	Bring-up, and loops where a halt must be observable but must never be actionable.

block remains the default deliberately. Every loop written before this existed already behaves that way, and silently changing halt semantics under running code would be worse than requiring an opt-in. It is also why handoff warns at startup if you did not give it a fallback: running nothing while halted is the exact failure the policy exists to prevent.

Confirm the policy is armed, don’t assume it

TrainingLoop reads the policy back from the core after setting it and warns if the two disagree, the same way a limits= envelope is confirmed by the target republishing what it enforces. A safety property you cannot observe is one nobody can verify.

It is also published, as __halt_policy__, and the dashboard header shows a HALT: HANDOFF / HALT: ADVISORY chip for any non-default policy. Whether a halt will stop commanding the machine is exactly the thing an operator standing at that machine needs on screen, not on stderr at startup.

If the tracker in use cannot enforce policies at all, the warning names which layer is missing support — the native extension is absent, the native extension predates halt policies, or the Python tracker in front of it does not forward set_halt_policy. The three have three different fixes, and a message naming the wrong one costs a rebuild that cannot help.

The policy is not remotely settable

Every other knob here can be driven over IPC by the dashboard or an MCP agent. This one cannot, and there is no command in the wire vocabulary that sets it.

A safety policy an operator could lower from outside is not a safety property, it is a suggestion. The process holding the actuators decides how it may be stopped.

Python

from omniloop import TrainingLoop

loop = TrainingLoop.from_dataclass(
    cfg.controller,
    tunable=["kp", "kd"],
    limits={"kp": (0.0, 400.0)},
    halt_policy="handoff",              # never stop actuating
    safety_controller=damping_hold,      # what runs instead
    halt_timeout=5.0,                    # deadman on a lost operator
    journal="run.omni",
)

while running:
    loop.step(lambda: controller.update(dt))

step() is the whole contract in one call: it opens the tick, asks the barrier what to do, runs your body or the fallback accordingly, and freezes on an exception the way tick() does. Prefer it on hardware — the branch is easy to forget, and forgetting it means commanding nothing while halted.

If you need the branch yourself:

with loop.tick() as t:
    if t.run_safety:
        damping_hold()               # halted: hold the machine
    else:
        controller.update(dt)        # running normally
    t.log(tracking_error=err)

t.halted, t.escalated and t.halt (the raw decision dict) are available for finer control.

C++

omniloop::Config cfg;
cfg.halt_policy  = omniloop::HaltPolicy::Handoff;
cfg.halt_timeout = std::chrono::seconds(5);
omniloop::Loop loop(cfg);

while (running_) {
    loop.step([&] { controller_.update(dt); },    // normal body
              [&] { damping_.hold(dt); });        // fallback controller
}

Full surface: C++ SDK.

Why the loop is halted

A fallback controller usually wants to treat a deliberate pause differently from a fault, so the reason travels with the halt.

Reason	Raised by
operator	A human or an agent — dashboard Halt, MCP omniloop_halt, loop.halt().
watchpoint	A tripwire fired.
exception	The loop body raised and was intercepted.
step	Single-step mode re-armed the barrier after one iteration.

with loop.tick() as t:
    if t.run_safety:
        if t.halt["reason"] == "watchpoint":
            emergency_ramp_down()     # something measured went out of envelope
        else:
            damping_hold()            # somebody just pressed pause

Freeze-on-exception is policy-aware

When the loop body raises, OmniLoop still intercepts it, journals it with the traceback, dumps the flight recorder, and publishes the fault. What differs is what happens next:

• Under block, the loop freezes for inspection — the historical behaviour.
• Under handoff, the halt is set but the loop is not stopped. The next iteration reports run_safety, so the fallback controller takes the machine while you read the traceback.

This is the difference between a robot that holds position after an encoder glitch and one that goes limp.

The deadman

A halt is a promise that somebody is coming back. halt_timeout is what happens when that promise expires — an operator’s laptop sleeps, the websocket drops, the tab closes.

The two policies diverge, because the dangerous state is different in each:

• Under block the danger is staying frozen: a process nobody will ever release is hung. Escalation releases the halt so the loop runs again, and latches the escalation so the release is attributable rather than mysterious.
• Under handoff the danger is silently resuming. The fallback is already holding the machine, so nothing is on fire; handing control back to the body an operator deliberately stopped would be the fault. Escalation keeps the fallback running and reports itself, leaving the decision — ramp down, power off, page a human — to the application, which is the only layer that knows what a safe end state is.

loop = TrainingLoop.from_dataclass(
    cfg.controller,
    halt_policy="handoff",
    safety_controller=damping_hold,
    halt_timeout=5.0,
    on_halt_escalation=lambda detail: pager.alert(detail),
)

Escalations are journaled as a halt_escalation control event, counted (loop.halt_escalations()), and published on telemetry as __halt_escalation__. Timeouts are opt-in and default to disabled, so a loop that never sets one behaves exactly as before.

What the deadman does not cover

The halt clock starts when the barrier first observes the halt, and it only advances when your loop calls the barrier. That makes this a deadman for an absent operator, not for an absent loop: a thread that stops polling entirely — deadlocked, starved, or killed — will never escalate, because the only code that could notice is the code that stopped running.

That is true of any in-process timer, and it is why this is not the outermost safety layer. A robot still needs a watchdog that outlives the process holding it. This covers the failure the control plane itself introduces, and nothing beyond it.

Step mode

Step advances the loop by exactly one iteration and then re-arms the barrier.

Under handoff, a step is spent only on an iteration that ran your real body — never on one that ran the fallback. Otherwise “step” would advance the safety controller while the thing you are trying to debug stayed where it was. step() handles this; if you write the branch by hand, call loop._tracker.step_barrier() only on the non-fallback path, or just use step().

Halt visibility

A halted loop publishes thread_state: "LOCKED"; a faulted one also sets is_safety_tripped. Under handoff the loop keeps publishing frames while handed over, which is the point — a frozen loop that publishes nothing is a loop you cannot diagnose.

The declared policy rides on every frame as __halt_policy__, so the dashboard and an MCP agent can both see whether this loop is one that can be stopped. On a machine that can move, that is worth knowing before you press anything.

Migrating an existing loop

Nothing is required. block is the default and the old behaviour is unchanged, including wait_if_halted() on the raw primitives.

To adopt handoff:

1. Write the fallback. It must be safe to run indefinitely and must command the machine every iteration — a damping hold, a gravity-compensation hold, or a controlled ramp to a rest pose.
2. Pass halt_policy="handoff" and safety_controller=.
3. Set halt_timeout= and decide, in on_halt_escalation=, what an abandoned halt should end in.
4. Switch the body to loop.step(...), or branch on t.run_safety.
5. Verify on the bench: run it, press Halt in the dashboard, and confirm the loop keeps ticking and the fallback is driving. Do this before it is anywhere near something expensive.

Related

• Watchpoints — the tripwires that raise a watchpoint halt.
• Limit Enforcement — hard envelopes, enforced inside the target process.
• C++ SDK — the safe-halt contract for non-Python control loops.