Skip to content
OmniLoop

Isaac Lab (rsl_rl / rl_games)

Libraries like rsl_rl and rl_games own their training loop internally (runner.learn() or agent.train()). Instead of asking you to restructure them, OmniLoop wraps the per-update step in place.

A single call turns an existing runner or agent into a live-tunable session — no edits to the library source code and no with block to insert.

Usage

Section titled “Usage”

rsl_rl

Section titled “rsl_rl”

Instrument the OnPolicyRunner before calling learn():

from rsl_rl.runners import OnPolicyRunner
from omniloop.integrations.rsl_rl import instrument_rsl_rl


runner = OnPolicyRunner(env, train_cfg, log_dir, device=device)


# learning_rate, entropy_coef, etc. become live
instrument_rsl_rl(runner)


runner.learn(num_learning_iterations=1500)

rl_games

Section titled “rl_games”

Instrument the agent before calling train():

from omniloop.integrations.rsl_rl import instrument_rl_games


instrument_rl_games(agent)
agent.train()

How it works

Section titled “How it works”

The instrumentation monkeypatches the algorithm’s update() method. Each iteration, pending dashboard mutations are applied before the update, and a telemetry frame containing the update’s loss stats is published after.

Default Tunables

Section titled “Default Tunables”
  • rsl_rl: learning_rate, entropy_coef, clip_param, value_loss_coef.
  • rl_games: last_lr, entropy_coef, e_clip.