Abstract Before deploying a software system we need to assure ourselves (and stakeholders) that the system will behave correctly. This assurance is usually done by testing the system. However, it is intuitively obvious that adaptive systems, including agent-based systems, can exhibit complex behaviour, and are thus harder to test. In this paper we examine this “obvious intuition” in the case of Belief-Desire-Intention (BDI) agents, by analysing the number of paths through BDI goalplan trees. Our analysis confirms quantitatively that BDI agents are hard to test, sheds light on the role of different parameters, and highlights the enormous difference made by failure handling.