Control v2 by MatthieuDarcy · Pull Request #291 · BasisResearch/dynestyx

MatthieuDarcy · 2026-07-30T19:00:17Z

This PR adds at first attempt at support for closed loop control in dynastyx.

Added in /dynestyx/control

DiscreteControlLoopSimulator

API:

    def model():
        with DiscreteControlLoopSimulator(
            control_policy=policy,
            policy_state_init=None,
            filter_config=FilterConfig(record_filtered_states_mean=True),
        ):
            return dsx.sample("loop", dynamics, predict_times=predict_times)

MPPI

Basic MPPI controller

API

mppi = MPPI(
    dynamics_model=dynamic_rollout
    loss_fn=quadratic_loss,
    horizon=horizon,
    n_samples=n_samples,
    noise_std=1.0,
    temperature=1.0,
)

Added in tutorials
3. Notebook controller_demo.ipynb

Additional elements I will work on

Verifying compatibility with different filters.
Verifying gradient computation.

Closes #288

MatthieuDarcy · 2026-07-30T19:24:32Z

The biggest addition to other parts of the code is in cuthbert/discrete_filter.py

compute_cuthbert_filter_update now computes a one step filtering update: takes in the previous estimated state, a control, and observation and provides a new estimated state.

mattlevine22

This looks great, thank you @MatthieuDarcy !

Can we have @cooijmanstim look at the Jupyter notebook demo, and get his take?
My preliminary comments/questions (haven't looked carefully through the code yet):

I think DiscreteControlLoopSimulator should be made available directly through the Simulator() handler via simulation/auto.py (and, hence, numpyro-free dsx.simulate) interface.
Should check that we can autodiff through a ClosedLoopSimulation.
Should probably support n_simulations > 1.
-DiscreteControlLoopSimulator should another argument for an "approximate model" (assuming the "true" model is in the dsx.sample statement)?
I like that one can feed very generic controllers (including MPPI); however, I think the current MPPI setup should be better curated so that the user doesn't have to do a bunch of the extra manual work that appears in the demo. I think the following should be possible:

sim_mppi = DiscreteControlLoopSimulator(
    control_policy_config=MPPIConfig(
                                                      loss_fn="quadratic",  # or a callable
                                                      horizon=horizon,
                                                      n_samples=n_samples,
                                                      noise_std=1.0, # is this needed?
                                                      temperature=1.0,
                                                ),
    policy_state_init=mppi_initial_state(horizon, control_dim),
    filter_config=KFConfig(record_filtered_states_mean=True),
)

# MPPI should have make_mppi_rollout internally (don't expose user to it)

cooijmanstim · 2026-07-30T20:42:55Z

Looks great, this interface works for us. For the notebook it would be good besides no control vs control to see the effectiveness of the controller as the noise level on the observations changes. Just three variants in total will do (no control, control with lots of noise, control with little noise).

DanWaxman

This looks like a good start! Some early high-level comments.

DanWaxman · 2026-08-03T13:58:51Z

I think it might be beneficial to just implement these directly in Simulator classes. The issue here is just that we made a split to actively recommend against Simulator for parameter inference...

I think it might be beneficial to just implement these directly in Simulator classes. The issue here is just that we made a split to actively recommend against Simulator for parameter inference...

This is okay because we aren't recommending/supporting numpyro-based parameter inference; just auto differentiable simulation results.

DanWaxman · 2026-08-03T14:00:57Z

+    r"""One-step FilterUpdate: state_k + u_k + y_{k+1} -> state_{k+1}.
+
+    Unlike `compute_cuthbert_filter` (whole-trajectory), this performs exactly
+    one predict+update step using cuthbert's `Filter.filter_prepare`/
+    `filter_combine` primitives directly, without requiring future
+    observations. This is what makes online control possible: the state
+    returned here can be consumed by a policy to choose the next control
+    before the next observation exists.
+
+    Pass `prev_state=None` for the bootstrap call (computing the filtering
+    state after only the first observation, with no control history yet);
+    this internally calls the cuthbert filter's `init_prepare` first.
+
+    Control convention (important, and different from `compute_cuthbert_filter`
+    / `DiscreteTimeSimulator`): `u` is the control that drove the transition
+    *into* the state being filtered, i.e. u_k when producing state_{k+1} from
+    state_k and y_{k+1} -- matching `FilterUpdate(x_hat_k, u_k, y_{k+1}, ...)`
+    in the control-loop equations. `compute_cuthbert_filter`/
+    `DiscreteTimeSimulator` instead pair `ctrl_values[t]` with *both* the
+    observation and the outgoing transition at the same index t, which is
+    only valid when the whole control trajectory is already known in advance.
+    For online control this is impossible: u_{k+1} cannot exist before
+    y_{k+1} is observed, since it is computed by the policy from the filtered
+    state that itself depends on y_{k+1}. So `u` here is used for both
+    `CuthbertInputs.u` and `CuthbertInputs.u_prev` in the single-row input
+    built for this step. Pass `u=None` for the bootstrap call, matching y_0's
+    lack of a control argument in the control-loop equations (numerically
+    equivalent to zeros for models with a control-input matrix, since D=None
+    or u=None are both treated as "no control contribution").
+    """


too much docstring for an internal function

Cleaned this up.

DanWaxman · 2026-08-03T14:02:26Z

+def compute_cuthbert_filter_update(
+    dynamics: DynamicalModel,
+    filter_config: BaseFilterConfig,
+    prev_state,
+    key: jax.Array,
+    *,
+    y: jax.Array,
+    u: jax.Array | None,
+    t: jax.Array,
+    t_prev: jax.Array | None = None,
+):


can probably change this a bit and then scan over it instead of duplicating prepare/combine code?

Cleaned this up. Now can either receive a config or a filter object (but not both, will raise an error). The ControlDiscretizer builds the filter outside the loop/scan and passes it to compute_cuthbert_filter_udpate.

Seems best to pick one contract---and a filter object seems like a good one. Any reason to keep the config option here?

MatthieuDarcy · 2026-08-03T18:17:47Z

Addressed some of the points made (thanks @mattlevine22 and @DanWaxman !).

Changes

DiscreteControlLoopSimulator now works with dsx.simulate
Example syntax:

dsx.simulate( dynamics, rng_key=key, predict_times=predict_times, control_policy=policy, filter_config=KFConfig(record_filtered_states_mean=True) )

~~Policy is any callable that takes in x_hat (estimated filtered state) and s (internal state) and returns u and s_new.~~

~~u can be an array or a numpyro distribution, in which case the simulator will draw a sample from the distribution.~~

3. MPPI is more robust and has better defaults. It's only required parameters at initialization are loss_function and dynamics (thus allowing for dynamics different from the dynamics evolving the state itself).

The notebooks controller_demo.ipynb and mpc_demo.ipynbhave been improved and now expose some of these choices. The MPPI example is now exclusively in mpc_demo.ipynb.

Explained in the notebooks are:

how to define a policy
how to use DiscreteControlLoopSimulator with non-linear dynamics
how to use ``MPPI` (highlighting that the dynamics used by MPPI have to be defined outside the simulator and thus can be different from the ones used by the simulator).

Outstanding questions

Currently policy does not use time $t$. This actually might be relevant (such as MPC combined with time dependent dynamics). Should we have policies take 3 arguments instead?

The simulator needs an initial state $s_0$. Internally, the simulator will check if policy has an initial_state attribute, defaulting to None otherwise. This makes it more convenient for the user to implement simple policies. In the notebook, I have the policy explicitly return None to highlight that, in general, policies might need to return an initial state. Is this alright?

Todos

Address some of @DanWaxman's requested changes, especially improving the internal docs.
~~Simplify the test suite.~~
~~Test gradients and policy optimization.~~
Expand the policies we package.

MatthieuDarcy · 2026-08-03T20:37:42Z

Update:

The control loop requires that the policy be any callable (e.g. a learned neural policy, an model predictive controller...) that accepts 4 arguments:

x_hat ~~(an estimate of the filtered state, provided by the filter)~~ (a numpyro distribution infered from the states).
t_now the current time.
t_next the next time at which the dynamics will advance.
s internal state.

It must return

A control u, a jax.numpy.array.
A new state s.

If u is a distribution, will raise an error and instruct the user to sample from the distribution instead. Tutorial have been updated.

Sharp edges

The state returned by the filter must be convertible to a numpyro distribution. The current implementation uses filter_state_dist to do this: supports (mean, cov) -> Gaussian and particles -> weighted/uniform particles. Other filters might break this.

mattlevine22 · 2026-08-04T19:20:00Z

+    return filter_obj, parallel
+
+
+def build_cuthbert_filter(


this is a classic "thin function" that LLMs like to build. I think it is only twice and is basically 2 lines of code!

Moreover, maybe its usage can be removed from compute_cuthbert_filter_update (why not always pass that a filter object?)

put differently, I would scrap _build_cuthbert_filter_obj and put it all inside this function (and let this function also return parallel)

"I would scrap _build_cuthbert_filter_obj and put it all inside this function." Could you clarify which function you're referring to?

I personally like the thin function, I think it makes other parts of the code more readable. Happy to remove it otherwise.

described above. I'd keep this function and get rid of _build_cuthbert_filter_obj

Got it, will remove _build_cuthbert_filter_obj (this will also require to update compute_cuthbert_filter).

mattlevine22 · 2026-08-04T19:20:56Z

+
+def compute_cuthbert_filter_update(
+    dynamics: DynamicalModel,
+    filter_config: BaseFilterConfig | None,


Why have both options? Should probably just take a filter object right? @DanWaxman

I agree, I would require to pass the filter itself.

mattlevine22 · 2026-08-04T19:21:43Z

+def compute_cuthbert_filter_update(
+    dynamics: DynamicalModel,
+    filter_config: BaseFilterConfig,
+    prev_state,
+    key: jax.Array,
+    *,
+    y: jax.Array,
+    u: jax.Array | None,
+    t: jax.Array,
+    t_prev: jax.Array | None = None,
+):


Seems best to pick one contract---and a filter object seems like a good one. Any reason to keep the config option here?

mattlevine22 · 2026-08-04T19:23:14Z

I think it might be beneficial to just implement these directly in Simulator classes. The issue here is just that we made a split to actively recommend against Simulator for parameter inference...

This is okay because we aren't recommending/supporting numpyro-based parameter inference; just auto differentiable simulation results.

mattlevine22 · 2026-08-04T20:06:03Z

+    if isinstance(filter_config, PFConfig):
+        if key is None:
+            raise ValueError(
+                "Particle filter requires a PRNG key: set 'crn_seed' in the filter config, "
+                "or run inside a NumPyro seeded context (e.g., with numpyro.handlers.seed)."
+            )
+        filter_obj = _cuthbert_filter_pf(dynamics, filter_kwargs)
+    elif isinstance(filter_config, EnKFConfig):
+        if key is None:
+            raise ValueError(
+                "Ensemble Kalman filter requires a PRNG key: set 'crn_seed' in the filter config, "
+                "or run inside a NumPyro seeded context (e.g., with numpyro.handlers.seed)."
+            )
+        filter_obj = _cuthbert_filter_enkf(dynamics, filter_kwargs)
+    elif isinstance(filter_config, KFConfig):
+        filter_obj = _cuthbert_filter_kalman(dynamics, filter_kwargs)
+    elif isinstance(filter_config, EKFConfig):
+        filter_obj = _cuthbert_filter_taylor_kf(dynamics, filter_kwargs)
+    else:
+        raise ValueError(
+            f"Unsupported cuthbert config: {type(filter_config).__name__}. "
+            "Expected KFConfig, EKFConfig, EnKFConfig, PFConfig."
+        )
+
+    parallel = (
+        want_parallel
+        and isinstance(filter_config, KFConfig)
+        and filter_config.associative
+    )
+    if parallel and not filter_obj.associative:
+        raise ValueError(
+            "Associative filtering was requested, but the constructed cuthbert "
+            f"filter is not associative: {type(filter_config).__name__}."
+        )
+    return filter_obj, parallel


I would just cut all of this code and drop it into build_cuthbert_filter

MatthieuDarcy · 2026-08-11T19:22:02Z

@mattlevine22 @DanWaxman what do you think of the PR in it's current state regarding the DiscreteControlLoopSimulator only?

I have updated MPPI so that the loss function takes in a ControlledSimulationResult and returns a scalar. The implementation is clunky under the hood but works. My feeling is that it would be nice to have the overall structure merged and to create a seperate PR to impore the MPPI implementation (as this is separate and might involve changes to other parts of the code) and other MPC algorithms. Thoughts?

mattlevine22 · 2026-08-12T05:09:51Z

I opened PR Control v2 Cleanup #313 to try to update some stuff here.

main things were to beef up the typing, add API documentation, do a few more checks / error-raises, and massage some of the variable names to be more clear/consistent

Realized that there is an off-by-one incompatibility of the ClosedLoopSimulator vs dsx.simulate when called with the selected controls specifically when the observation depends on a control. I opened issue Define observation/control alignment for discrete-time models #312 about this. I'm thinking we let this slide for now, and fix it more generally separately.
I still don't love that this has a TYPE_CHECKING variable to avoid circular imports, but oh well haha.

Implements the discrete-time control loop from the design issue: a new DiscreteControlLoopSimulator (alongside DiscreteTimeSimulator) interleaves simulation, observation, filtering, and control online, deciding each u_k from the filtered belief via a user-supplied policy rather than requiring the whole control trajectory up front. FilterUpdate is powered by a new compute_cuthbert_filter_update, which drives cuthbert's existing Filter.filter_prepare/filter_combine primitives one step at a time instead of over a whole pre-supplied trajectory -- generic across KFConfig/EKFConfig/EnKFConfig/PFConfig. Verified PFConfig and EnKFConfig support genuinely black-box state transitions (no log_prob required), matching the design issue's black-box dynamics requirement. Includes a demo notebook (docs/tutorials/control/controller_demo.ipynb) showing a linear feedback policy driving a 1D linear-Gaussian system to 0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…transitions Adds a tutorial section demonstrating a genuinely continuous-time SDE (ContinuousTimeStateEvolution + Discretizer(discretize=euler_maruyama)) composed with DiscreteControlLoopSimulator, showing the loop only ever sees discrete times while the underlying dynamics are a true SDE solved between them. Building this surfaced a real bug: the bootstrap FilterUpdate call left t_prev defaulting to t (dt=0). For fixed-covariance transitions this is harmless, but for a dt-scaled transition (like the Euler-Maruyama discretization here), it constructs a zero-covariance distribution whose NaN log-density leaks through the gradient in EKF's Taylor linearization (jnp.where evaluates both branches, unlike jax.lax.cond) -- corrupting the filtered state and, downstream, the policy's control and the simulated trajectory itself. Fixed by giving the bootstrap call a non-degenerate t_prev, borrowing the width of the first real interval (matching compute_cuthbert_filter's own dummy-row convention). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…ccuracy Documents how the continuous-time integration is actually defined: Discretizer takes exactly one Euler-Maruyama step over the whole gap between requested times (needed so filtering gets an explicit one-step transition density), unlike SDESimulator/solve_sde's substepped solvers, which only support pure forward simulation without filtering. Adds a section 6 demonstrating the same recipe on a genuinely nonlinear 2D system with drift = A x^2 + u: x=0 is an unstable equilibrium (x^2 >= 0 always pushes away from the origin), so the uncontrolled run diverges while the same linear feedback policy as before stabilizes it locally. Uses a finer time grid (dt=0.1 vs 1.0) since the one-step EM approximation needs a small enough gap to stay accurate for a nonlinear drift. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…rt_filter_update 35 tests across 7 groups: filter_state_mean unit tests, compute_cuthbert_filter_update correctness (including regression tests for the dt=0 EKF NaN bug), validation/error paths, end-to-end shape/output-key tests (including the stateless-policy regression), behavioral/control correctness (closed-loop stabilization, the control-index convention, determinism, eqx.Module policies), continuous-time/Discretizer composition, and black-box transition compatibility. Surfaced one more real gap while writing these: a genuinely nested-pytree policy_state_init (e.g. a dict of arrays) can't actually round-trip today -- BaseSimulator's shared _run_single_member_simulation enforces a dict[str, Array] | None return type via jaxtyping at runtime, so policy_states must stay a flat Array. Documented in the corresponding test rather than widening the shared base class's type contract, which is out of scope here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…ulator there Consolidates control-loop + control-policy code under dynestyx/control/: moves discrete_controller_simulators.py there, and adds a basic MPPI (Model Predictive Path Integral) controller (dynestyx/control/mppi.py) that plugs into the same control_policy= slot. Deliberately simple: samples candidate control sequences as Gaussian perturbations around a nominal sequence, rolls each through a user-supplied dynamics_model, and returns the softmax-weighted mean control (standard MPPI weighting). dynamics_model can be batched (one call handles all samples) or not (driven via jax.lax.map, so it never needs to support batching itself). MPPI needs fresh randomness every step, which PolicyCallable's signature didn't provide -- added a key parameter to __call__(x_hat, s, key), passed in by DiscreteControlLoopSimulator's per-step scan. This also keeps MPPI's own policy state a flat array (just the nominal control sequence), avoiding a real limitation found while testing last time: a nested-pytree policy_state_init can't actually be recorded as output today, since BaseSimulator's shared return-type check requires flat Array values. Verified: existing linear-feedback policies (tests, tutorial notebook) updated for the new signature, full regression suite still passes (117 tests), and a smoke test confirms MPPI stabilizes a marginally-unstable linear system identically under both batched and non-batched dynamics_model. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…lation) main/control_v2 landed a major refactor (commit 3fb1cd7, "Numpyro-Free Usage") that deleted dynestyx/simulators.py entirely, replacing it with a new dynestyx/simulation/ package: BaseSimulator subclasses now implement a public `simulate(dynamics, *, rng_key, ...) -> SimulatedResult` (not `_simulate(name, ..., obs_times=..., ...) -> dict`), take rng_key directly instead of calling numpyro.prng_key() internally, and numpyro site registration happens via a generic deferred callback (dataclasses.fields() iteration over the returned SimulatedResult) rather than base-class magic over a free-form dict. Ported DiscreteControlLoopSimulator to the new contract: - Plain class with explicit __init__ (matching DiscreteTimeSimulator's new shape) instead of a bare @dataclasses.dataclass. - New ControlledSimulatedResult(SimulatedResult) subclass carrying the extra controls/filtered_states_mean/policy_states fields -- confirmed by reading base.py directly that _run_single_member_simulation's common path is a bare `return self.simulate(...)` with no reconstruction, so subclass fields flow through untouched and get registered generically the same way as SimulatedResult's own fields (None-valued fields are auto-skipped). - Dropped the now-redundant manual `numpyro.prng_key()`/obs_values checks: the base class already validates/rejects these before simulate() is ever called. compute_cuthbert_filter_update/_build_cuthbert_filter_obj (added to discrete_filter.py on the `control` branch) merged in via `git merge control` with a single trivial conflict (re-adding the `jax.random` import) -- confirmed via git merge-tree dry run and direct diffing that every symbol they depend on exists byte-identical post-refactor, just re-imported from dynestyx.inference.configs.filter instead of dynestyx.inference.filter_configs. Discretizer required no changes at all (already ported upstream). Verified: full regression suite passes on control_v2 (118 tests: this file's 35 plus test_filters.py/test_filter_simulator.py/test_discretizers.py/ test_predictive_filter_simulator_shapes.py), the tutorial notebook re-executes cleanly end-to-end, and MPPI produces numerically identical results to its control-branch run under both batched and non-batched dynamics_model. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Section 7 demonstrates dynestyx.control.MPPI plugged into the same control_policy= slot as the earlier linear-feedback policy, on the same 1D system from section 1. Builds a planning rollout directly from dynamics.state_evolution's deterministic mean (a standard MPPI simplification -- the planner doesn't need a faithful stochastic simulation, just a reasonable prediction of where a candidate control sequence leads) and a simple quadratic loss, then compares against the K=0 no-control baseline already computed in section 3. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

DynamicalModel.state_evolution is declared as the broader ContinuousTimeStateEvolution | DiscreteStateTransition union, so `ty` couldn't statically narrow continuous_dynamics.state_evolution to StochasticContinuousTimeStateEvolution before passing it to euler_maruyama() (which requires the narrower type) -- even though this is exactly how Discretizer._sample_ds itself resolves it at runtime. Added the same isinstance narrowing check used there. Also includes an incidental ruff-format tweak to mppi.py (collapsing a one-line call that had been wrapped unnecessarily). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…book Section 8 demonstrates that DiscreteControlLoopSimulator is differentiable end-to-end: rolls out a short horizon with the linear policy from section 2, defines a loss as the norm of the final true state, and takes jax.grad of it with respect to the gain K -- no special machinery needed beyond plain JAX autodiff. Verified independently against a finite-difference check before writing the notebook cells (gradient matched to ~1e-4), and shows one gradient-descent step meaningfully reducing the loss (0.62 -> 0.18). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Section 3 (differentiating through the closed loop) computed grad_K via jax.grad but never independently verified it. Added a cell checking a central finite difference on K[0, 0] against the autodiff value -- confirms the gradient through the whole closed loop (policy -> transition -> observation -> filter update) is actually correct, not just plausible. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Section 3's rollout_final_state_norm now calls sim.simulate(dynamics, rng_key=..., predict_times=...) directly instead of going through dsx.sample/numpyro.handlers.seed -- unneeded here since there's no conditioning/MCMC, and it makes the PRNG key an explicit argument instead of an implicit fixed rng_seed=0. More importantly, this fixes a real issue in section 4's training loop: every epoch previously reused the same fixed seed, so gradient descent could overfit K to one specific noise realization rather than the underlying dynamics. Now each epoch draws a fresh key via jax.random.split, so K_opt has to generalize across realizations. Verified the loop still converges to a sensible stabilizing gain with this fresh-key-per-epoch setup before editing the notebook. Section 4's final comparison plot (run()/dsx.sample) is left unchanged -- there, reusing the same seed for both the unoptimized and optimized rollouts is what makes the before/after comparison fair. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

run() (updated separately) now returns a ControlledSimulatedResult from sim.simulate() directly instead of a numpyro trace dict, so the plotting cell needed the equivalent attribute-access update: trace["loop_X"]["value"] -> result.X. Also removed a leftover duplicate plotting cell still using the old trace-dict access, which was left after the updated one and caused the notebook to fail on execution. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Replaces the with sim: dsx.sample(...) + numpyro.handlers.seed/trace pattern across every section (discrete demo, SDE, nonlinear 2D, MPPI) with direct sim.simulate(dynamics, rng_key=key, predict_times=...) calls returning ControlledSimulatedResult objects -- no NumPyro handler or model function needed, since every section here is a pure generative rollout with nothing to condition on. Renamed trace_* variables to result_* throughout, since they're no longer NumPyro trace dicts, and updated all downstream plotting cells from trace["name_field"]["value"] to result.field attribute access. For the continuous-time sections, this also removes the Discretizer handler composition: instead of wrapping dsx.sample in `with Discretizer(...)`, the continuous-time DynamicalModel is discretized once up front via euler_maruyama(state_evolution) into a plain discrete-time DynamicalModel, which is then simulated exactly like any other discrete-time model. Updated the surrounding markdown to match (no more handler-chain explanation needed). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Unified non-linear dynamics, black box and partial observations example

Make filter_obj/filter_config mutually exclusive; add filter_state_dist compute_cuthbert_filter_update now takes filter_config: BaseFilterConfig | None and requires exactly one of filter_config/filter_obj, matching how DiscreteControlLoopSimulator.simulate() already calls it with a pre-built filter_obj. Also add filter_state_dist(state) -> provides a numpyro distribution from the states. This will try to infer the right distribution given the state (mean, cov -> Gaussian, particles -> weighted/uniform particles. Modified the control loop to take this into account as well as the MPPI definition and the example notebooks

Removed the ability to give a filter config to the one step filter update. Now always requires a filter object

Updated the control optimization notebook to use the new syntax. Also added a check to see that gradients flow correctly through both the dynamics and the MPC planning loop.

Removed the need/requirement to have an initial state attribute to a policy. Instead this must be passed explicitly to the simulator. This will avoid hidden operations and is more in line with other coding practices.

Dan said he's happy with the incoming PR as a cleanup to this

mattlevine22

@DanWaxman is happy and so am I ! Let's do this thing---congrats @MatthieuDarcy on your first big Dynestyx / Basis PR haha, great work! Let's hope @cooijmanstim and the R-ADA team enjoy the fruits of your labor :) I'm sure many controls people will !

mattlevine22 reviewed Jul 30, 2026

View reviewed changes

DanWaxman previously requested changes Aug 3, 2026

View reviewed changes

mattlevine22 requested changes Aug 4, 2026

View reviewed changes

mattlevine22 reviewed Aug 4, 2026

View reviewed changes

MatthieuDarcy requested review from DanWaxman and mattlevine22 August 11, 2026 19:22

This was referenced Aug 13, 2026

Tutorial on online control #316

Open

Support: Plating with Online Closed Loop Controls #318

Open

MatthieuDarcy and others added 16 commits August 13, 2026 16:52

added notebook on optimizing the policy using gradients

928f555

Improved the control_optimization tutorial

340ef2c

Improved controller demo

7838a31

Unified non-linear dynamics, black box and partial observations example

MatthieuDarcy added 15 commits August 13, 2026 16:52

Update controller_demo.ipynb

5f1fe53

updated notebooks

62ef808

One step filter updates requires filter object

d1293ea

Removed the ability to give a filter config to the one step filter update. Now always requires a filter object

Updated the discrete filters constructor for cuthbert filters

0689a10

Update test_discrete_control.py

ec21aa5

improved tutorials

5799ec7

Update controller_demo.ipynb

44f8ff7

Updated control optimization: added gradients through MPC

10207ef

Updated the control optimization notebook to use the new syntax. Also added a check to see that gradients flow correctly through both the dynamics and the MPC planning loop.

Added initial policy state to simulator

efd6123

Removed the need/requirement to have an initial state attribute to a policy. Instead this must be passed explicitly to the simulator. This will avoid hidden operations and is more in line with other coding practices.

Update controller_demo.ipynb

0e81078

Updated cuthbert filters building

cdcb93f

MPPI loss now uses the result from dsx.sim

874271a

improved MPPI doc

1372be4

Update discrete_filter.py

a70c572

MatthieuDarcy force-pushed the control_v2 branch from 8c31c0b to a70c572 Compare August 13, 2026 22:27

mattlevine22 and others added 9 commits August 13, 2026 18:27

revised typing and small fixes

1e7bfcd

require cuthbert backend for filter in control simulator

90e5819

reduce intrusiveness; rollback some changes

360ac1e

add back a long comment

7c55807

make cuthbert filter source explicit

9c57a37

Update mppi.py

c709a26

Linked to #314

d20ec6d

add links to outstanding issues

f4b8ee0

bootrap -> initial filter setup with no prev state

e6bf79d

mattlevine22 removed the request for review from DanWaxman August 14, 2026 03:29

mattlevine22 approved these changes Aug 14, 2026

View reviewed changes

mattlevine22 merged commit 2d5e307 into main Aug 14, 2026
6 checks passed

MatthieuDarcy mentioned this pull request Aug 14, 2026

Discrete time, closed loop control #288

Closed

Uh oh!

Conversation

MatthieuDarcy commented Jul 30, 2026

Uh oh!

MatthieuDarcy commented Jul 30, 2026

Uh oh!

mattlevine22 left a comment

Choose a reason for hiding this comment

Uh oh!

cooijmanstim commented Jul 30, 2026

Uh oh!

DanWaxman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MatthieuDarcy Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MatthieuDarcy commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Outstanding questions

Todos

Uh oh!

MatthieuDarcy commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sharp edges

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MatthieuDarcy commented Aug 11, 2026

Uh oh!

mattlevine22 commented Aug 12, 2026

Uh oh!

mattlevine22 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MatthieuDarcy Aug 4, 2026 •

edited

Loading

MatthieuDarcy commented Aug 3, 2026 •

edited

Loading

MatthieuDarcy commented Aug 3, 2026 •

edited

Loading