Stable-Baselines3

Live-tune a Stable-Baselines3 (SB3) training run directly from the OmniLoop dashboard.

When you add the OmniLoop callback to an SB3 .learn() call, its learning rate, entropy coefficient, and clip range become live sliders. Rollout stats stream automatically as readouts, and an exception inside a rollout freezes the run for inspection instead of crashing.

Usage

You can instrument any SB3 algorithm by passing OmniLoopCallback to the learn() method:

from stable_baselines3 import PPO
from omniloop.integrations.sb3 import OmniLoopCallback

model = PPO("MlpPolicy", env)
model.learn(total_timesteps=1_000_000, callback=OmniLoopCallback())

Installation

Install the SB3 extra:

pip install omniloop[sb3]

Why an SB3-aware binding is needed

SB3 stores several hyperparameters (like learning rate) as schedules — callables of training progress — not plain numbers. It re-applies these schedules every rollout, so naively setting model.learning_rate = x gets overwritten on the next update.

This adapter binds each tunable to the place SB3 actually reads:

• learning_rate -> writes to model.policy.optimizer.param_groups[*]["lr"] and pins model.lr_schedule to a constant so SB3 won’t clobber it.
• ent_coef -> model.ent_coef (PPO/A2C read this float directly).
• clip_range -> pins model.clip_range to a constant schedule (for PPO).

Configuration

OmniLoopCallback accepts the following parameters:

Parameter	Type	Description
tunable	List of strings	Hyperparameter names to expose (e.g., ["learning_rate", "ent_coef"]).
bounds	Dictionary	Min, max, and step for sliders {name: (min, max, step)}.
publish_every	Integer	Stream telemetry every N rollouts.
freeze_on_exception	Boolean	Whether to freeze the run if an exception occurs during training.
tracker	Object	Optional custom OmniLoop tracker instance to inject.