Abstract
Artificial Intelligence (AI) applications are being
used to predict and assess behaviour in multiple
domains which directly affect human well-being.
However, if AI is to improve people’s lives, then
people must be able to trust it, by being able to understand what the system is doing and why. Although transparency is often seen as the requirement in this case, realistically it might not always
be possible, whereas the need to ensure that the system operates within set moral bounds remains.
In this paper, we present an approach to evaluate
the moral bounds of an AI system based on the
monitoring of its inputs and outputs. We place a
‘Glass-Box’ around the system by mapping moral
values into explicit verifiable norms that constrain
inputs and outputs, in such a way that if these remain within the box we can guarantee that the system adheres to the value. The focus on inputs and
outputs allows for the verification and comparison
of vastly different intelligent systems; from deep
neural networks to agent-based systems.
The explicit transformation of abstract moral values into concrete norms brings great benefits in
terms of explainability; stakeholders know exactly
how the system is interpreting and employing relevant abstract moral human values and calibrate
their trust accordingly. Moreover, by operating at
a higher level we can check the compliance of the
system with different interpretations of the same
value