Curiosity is evident in humans of all sorts from early infancy, and it has also been said to appear in a wide range of other animals, including monkeys, birds, rats, and octopuses. The classical definition of curiosity as an intrinsic desire for knowledge may seem inapplicable to animal curiosity: one might wonder how and indeed whether a rat could have such a fancy desire. Even if rats must learn many things to survive, one might expect their learning must be driven (...) by simpler incentives, such as hunger. One might also wonder what proximal signals could guide animals towards knowledge itself, or how something as abstract as knowledge could ever be a motivational target for an unreflective animal. Taking a cue from recent work in reinforcement learning, I argue that surprise functions as a reward signal for the curious animal, and then show how this amounts to a desire for knowledge gain, where knowledge is conceived of as a cognitive adaptation to reality. This adaptation results in a mental state whose existence depends essentially on the truth of its contents, a factive mental state. Curious creatures benefit from an interaction between the prediction-error correction processes of basic learning and the active surprise-seeking force of their curiosity. This internally adversarial interaction accelerates knowledge gain in ways that are very helpful for agents with the restrictions of biological creatures, in environments with the complexity of our natural world. (shrink)

There is an apparent connection between reinforcement learning and agency. Artificial entities controlled by reinforcement learning algorithms are standardly referred to as agents, and the mainstream view in the psychology and neuroscience of agency is that humans and other animals are reinforcement learners. This article examines this connection, focusing on artificial reinforcement learning systems and assuming that there are various forms of agency. Artificial reinforcement learning systems satisfy plausible conditions for minimal agency, and those which use models of the environment (...) to perform forward search are capable of a form of agency which may reasonably be called action for reasons. (shrink)