Even where an act appears to be responsible, and satisfies all the conditions for responsibility laid down by society, the response to it may be unjust where that appearance is false, and where those conditions are insufficient. This paper argues that those who want to place considerations of responsibility at the centre of distributive and criminal justice ought to take this concern seriously. The common strategy of relying on what Susan Hurley describes as a 'black box of responsibility' has the (...) advantage of not taking responsibility considerations to be irrelevant merely because some specific account of responsibility is mistaken. It can, furthermore, cope perfectly well with an absence of responsibility, even of the global sort implied by hard determinism and other strongly sceptical accounts. Problems for the black box view come in where responsibility is present, but in a form that is curtailed in one significant regard or another. The trick, then, is to open the box of responsibility just enough that its contents can be the basis for judgements of justice. I identify three 'moderately sceptical' forms of compatibilism that cannot ground judgements of justice, and are therefore expunged by the strongest 'grey box' view. (shrink)