Abstract Specifying the reward function of a Markov decision process (MDP) can be demanding, requiring human assessment of the precise quality of, and tradeoffs among, various states and actions. However, reward functions often possess considerable structure which can be leveraged to streamline their speci?cation. We develop new, decisiontheoretically sound heuristics for eliciting rewards for factored MDPs whose reward functions exhibit additive independence. Since we can often ?nd good policies without complete reward speci?cation, we also develop new (exact and approximate) algorithms for robust optimization of imprecise-reward MDPs with such additive reward. Our methods are evaluated in two domains: autonomic computing and assistive technology.