Mouse tracking, a new action‐based measure of behavior, has advanced theories of decision making with the notion that cognitive and social decision making is fundamentally dynamic. Implicit in this theory is that people's decision strategies, such as discounting delayed rewards, are stable over task design and that mouse trajectory features correspond to specific segments of decision making. By applying the hierarchical drift diffusion model and the Bayesian delay discounting model, we tested these assumptions. Specifically, we investigated the extent to which (...) the “mouse‐tracking” design of decision‐making tasks (delay discounting task, DDT and stop‐signal task, SST) deviate from the standard “keypress” design of decision making tasks. We found remarkable agreement in delay discounting rates (intertemporal impatience) obtained in the keypress and mouse‐tracking versions of DDT (ρ = 0.90) even though these tasks were given about 1 week apart. Rates of evidence accumulation converged well in the two versions (DDT, ρ =.86; SST, ρ =.55). Omission/commission error in SST showed high agreement (ρ =.42, ρ =.53). Mouse‐motion features such as maximum velocity and AUC (area under the curve) correlated well with nondecision time (ρ = −.42) and boundary separation (ρ =.44)—the amount of information needed to accumulate prior to making a response. These results indicate that the response time (RT) and motion‐based decision tasks converge well at a fundamental level, and that mouse‐tracking features such as AUC and maximum velocity do indicate the degree of decision conflict and impulsivity. (shrink)

Affective computing research has advanced emotion recognition systems using facial expressions, voices, gaits, and physiological signals, yet these methods are often impractical. This study integrates mouse cursor motion analysis into affective computing and investigates the idea that movements of the computer cursor can provide information about emotion of the computer user. We extracted 16–26 trajectory features during a choice-reaching task and examined the link between emotion and cursor motions. Participants were induced for positive or negative emotions by music, film clips, (...) or emotional pictures, and they indicated their emotions with questionnaires. Our 10-fold cross-validation analysis shows that statistical models formed from “known” participants could predict nearly 10%–20% of the variance of positive affect and attentiveness ratings of “unknown” participants, suggesting that cursor movement patterns such as the area under curve and direction change help infer emotions of computer users. (shrink)