Stochastic Multiple Shooting with Local Feedback Improves Sample Efficiency in Trajectory Optimization

Stochastic single shooting trajectory optimization methods such as Model Predictive Path Integral control (MPPI) have been widely adopted in robotics due to their ability to reason about probabilistic dynamics and provide solutions where model gradients are noisy, costly to evaluate, or unavailable. However, when shooting over long action sequences, the satisfaction of terminal constraints is often sample-inefficient, requiring a large number of iterations for convergence.

To overcome this limitation, we present a stochastic multiple shooting method that optimizes short control action sequences connected via local feedback policies. By dividing the long-horizon problem into shorter segments, each stabilized by a feedback policy, the method significantly improves sample efficiency and convergence to the terminal set, as the local policies provide better guidance for each segment.

A key innovation of our approach is the synthesis of approximate system Jacobians purely from rollouts, without requiring explicit model gradients. This makes the method directly applicable to model-based reinforcement learning with black-box dynamics, where gradients are often unavailable or expensive. This capability broadens the method's utility to a wider range of robotic systems.

We validate the algorithm on three nonlinear, underactuated optimization problems: a classic cartpole swingup task with analytical dynamics, the same task with learned neural network dynamics, and a VTOL quadplane performing a high angle-of-attack, precision post-stall landing maneuver. In all cases, our method demonstrates improved sample efficiency and terminal set convergence compared to stochastic single shooting baselines.

The results indicate that stochastic multiple shooting with local feedback policies offers a robust and efficient framework for trajectory optimization in complex, uncertain environments. Future work will explore extending this approach to real-time control and larger-scale problems.