Modern weaponry is often too complex for unaided human operation, and is largely or totally controlled by computers. But modern software, particularly artificial intelligence software, exhibits such complexity and inscrutability that there are grave dangers associated with its use in non-benign applications. Recent efforts to make computer systems more accessible to military personnel through natural language processing systems, as proposed in the Strategic Computing Initiative of the Defense Advanced Research Projects Agency, increases rather than decreases the dangers of unpredictable behavior. (...) Defense systems constitute, in fact, a paradigm case of the wrong kind of application for this technology. This cannot be expected to change, since the unpredictability stems from inherent properties of computer systems and of natural languages. (shrink)

This essay warns of eroding accountability in computerized societies. It argues that assumptions about computing and features of situations in which computers are produced create barriers to accountability. Drawing on philosophical analyses of moral blame and responsibility, four barriers are identified: 1) the problem of many hands, 2) the problem of bugs, 3) blaming the computer, and 4) software ownership without liability. The paper concludes with ideas on how to reverse this trend.

A proof of ‘correctness’ for a mathematical algorithm cannot be relevant to executions of a program based on that algorithm because both the algorithm and the proof are based on assumptions that do not hold for computations carried out by real-world computers. Thus, proving the ‘correctness’ of an algorithm cannot establish the trustworthiness of programs based on that algorithm. Despite the (deceptive) sameness of the notations used to represent them, the transformation of an algorithm into an executable program is a (...) wrenching metamorphosis that changes a mathematical abstraction into a prescription for concrete actions to be taken by real computers. Therefore, it is verification of program executions (processes) that is needed, not of program texts that are merely the scripts for those processes. In this view, verification is the empirical investigation of: (a) the behavior that programs invoke in a computer system and (b) the larger context in which that behavior occurs. Here, deduction can play no more, and no less, a role than it does in the empirical sciences. (shrink)

Prototyping is not a new approach to computer-based information system development. It is just one technique among many used in system design. What might be new is for what purpose prototyping is used. The purpose could be to achieve a more user controlled system development and to give the future users a tool that will enable them to fully participate in not only the work with requirements specifications, but also in the actual systems design and organisational change. This paper describes (...) a working model of an approach-the PROTEVS model-in which it is recommended that the future system users design their own prototype systems. The prototypes may act as both requirements specifications and solutions for actual change. This alternative approach aims to offer a basis for new ways of action for future users as participants in the design, evaluation, and change of a workplace. A suitable environment for the approach to act within is also described—a local design shop. (shrink)

A debate over the theoretical capabilities of formal methods in computer science has raged for more than two years now. The function of this paper is to summarize the key elements of this debate and to respond to important criticisms others have advanced by placing these issues within a broader context of philosophical considerations about the nature of hardware and of software and about the kinds of knowledge that we have the capacity to acquire concerning their performance.

Una imagen muy generalizada a la hora de entender el software de computador es la que lo representa como una “caja negra”: no importa realmente saber qué partes lo componen internamente, sino qué resultados se obtienen de él según ciertos valores de entrada. Al hacer esto, muchos problemas filosóficos son ocultados, negados o simplemente mal entendidos. Este artículo discute tres unidades de análisis del software de computador, esto es, las especificaciones, los algoritmos y los procesos computacionales. El objetivo central es (...) entender las prácticas cientficas e ingenieriles detrás de cada unidad de software, así como analizar su metodología, ontología y epistemología. (shrink)