Abstract Recent advances in solutions to Hybrid MDPs with discrete and continuous state and action spaces have signi?cantly extended the class of MDPs for which exact solutions can be derived, albeit at the expense of a restricted transition noise model. In this paper, we work around limitations of previous solutions by adopting a robust optimization approach in which Nature is allowed to adversarially determine transition noise within pre-speci?ed con?dence intervals. This allows one to derive an optimal policy with an arbitrary (user-speci?ed) level of success probability and signi?cantly extends the class of transition noise models for which Hybrid MDPs can be solved. This work also signi?cantly extends results for the related “chance-constrained” approach in stochastic hybrid control to accommodate state-dependent noise. We demonstrate our approach working on a variety of hybrid MDPs taken from AI planning, operations research, and control theory, noting that this is the ?rst time robust solutions with strong guarantees over all states have been automatically derived for such problems.