Our research is a value sensitive based approach to designing autonomous systems in a military context. Value sensitive design is an iterative process of conceptual, empirical and technical considerations. We enhance value sensitive design with Participatory Value Evaluation. This allows us to mine values of a large unorganized stakeholder group relevant to our context of research, namely Australian citizens. We found that value prioritizations differ depending on the context of use and that no one value fits all autonomous systems. General (...) ethical frameworks can serve as high level guiding principles, but when it comes to actual technologies, a more nuanced and granulated approach to selecting and incorporating values is needed. Our research contributes to a greater awareness of the importance of inclusion of values in autonomous systems design in the context of defence. We suggest relevant frameworks and methods for values-based autonomous military systems design in the context of Australia, which may serve as a blueprint for other countries. Finally, we provide suggestions on how to operationalise values in systems design, as this is underrepresented in the current literature. (shrink)

The question of human control is a key concern in autonomous military systems debates. Our research qualitatively and quantitatively investigates values and concerns of the general public, as they relate to autonomous military systems, with particular attention to the value of human control. Using participatory value evaluation (PVE), we consulted 1980 Australians about which values matter in relation to two specific technologies: an autonomous minesweeping submarine and an autonomous drone that can drop bombs. Based on value sensitive design, participants were (...) tasked to enhance the systems with design features that can realize values. A restriction (limited budget) in each design task forced participants to make trade-offs between design options and the values that these options realize. Our results suggest that the ‘general public’ has diverse and nuanced stances on the question of human control over autonomous military systems. A third of participants that is opposed to autonomous military systems when asked directly, selected different combinations of design features realizing varying degrees of human control. Several contextual factors, technology-specific concerns, and certain values seemed to explain these different choices. Our research shows that a focus on human control might overlook other important values that the general public is concerned about, such as system reliability, verifiability, and retrievability. (shrink)