In this paper I investigate which of the main conditions proposed in the moral responsibility literature are the ones that spell trouble for the idea that Artificial Intelligence Systems (AISs) could ever be full-fledged responsible agents. After arguing that the standard construals of the control and epistemic conditions don't impose any in-principle barrier to AISs being responsible agents, I identify the requirement that responsible agents must be aware of their own actions as the main locus of resistance to attribute that kind of agency to AISs. This is because this type of awareness is thought to involve first-person or de se representations, which, in turn, are usually assumed to involve some form of consciousness. I clarify what this widespread assumption involves and conclude that the possibility of AISs' moral responsibility hinges on what the correct theory of de se representations ultimately turns out to be.
Keywords: Artificial intelligence; Consciousness; De Se representation; First-person representation; Moral agency; Moral responsibility.
© The Author(s), under exclusive licence to Springer Nature B.V. 2021, corrected publication 2021.