Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants’ behavior in this task, we find evidence that a substantial number of them balanced exploration and exploitation by (...) considering the probability that an option offers the maximum reward out of all the available options. (shrink)

Two variables are usually recognised as determinants of human causal learning: the contingency between a candidate cause and effect, and the temporal and/or spatial contiguity between them. A common finding is that reductions in temporal contiguity produce concomitant decrements in causal judgement. This finding had previously (Shanks & Dickinson, 1987) been interpreted as evidence that causal induction is based on associative learning processes. Buehner and May (2002, 2003, 2004) have challenged this notion by demonstrating that the impact of temporal delay (...) depends on expectations about the timeframe between cause and effect. A corollary of this knowledge-mediation account is that in certain situations longer delays could facilitate, while shorter delays should impair, causal learning. Here we present two experiments involving a physical apparatus that demonstrate a detrimental effect of contiguity under certain conditions. In contrast to all previous studies, delays universally promoted causal learning. This evidence is clearly at variance with the notion of a bottom-up contiguity bias in causal induction. A new, more general timeframe bias is discussed. (shrink)