In multi-armed bandit problems, information acquired from experimentation is valuable because it tells the agent whether to select a particular option again in the future. This article tests whether people undervalue this information because they are ambiguity averse, or have a distaste for uncertainty about the average quality of each alternative. It is shown that ambiguity averse agents have lower than optimal Gittins indexes, appearing to undervalue information from experimentation, but are willing to pay more than ambiguity neutral agents to (...) learn the true mean of the payoff distribution, appearing to overvalue objectively given information. This prediction is tested with a laboratory experiment that elicits a Gittins index and a willingness to pay on six two-armed bandits. Consistent with the predictions of ambiguity aversion, the Gittins indexes are significantly lower than optimal and willingnesses to pay are significantly higher than optimal. (shrink)