Login |
    Reinforcement learning and beyond: neural systems for valuation and learning in the human brain.
    John O’Doherty, California Institute of Technology
    Reinforcement learning and beyond: neural systems for valuation and learning in the human brain.
    Interest in the computational and neural underpinnings of learned valuation and choice has surged in recent years. This interest can be attributed in large part to the observation that the phasic activity of dopamine neurons bears a remarkable similarity to prediction error learning signals derived from a family of abstract computational models collectively known as reinforcement learning (RL). In RL, prediction error signals are used to update predictions of future reward for different actions. These values are then compared in order to implement action selection. In this presentation I will outline evidence from functional neuroimaging studies in humans for the existence of computational learning signals such as prediction errors in the human brain. In particular, I will show that RL-related activity in the human dorsal striatum appears to be strongly linked to behavioral expression of instrumental learning and choice. I will then consider situations under which simple RL is unlikely to be sufficient to account for choice behavior. One form of learning that cannot be accounted for by simple RL is “latent-learning”, whereby an animal can learn to take actions for reward on the basis of acquired “latent” knowledge about the structure of the environment even in the absence of explicit reinforcement during training. Yet another form of learning that is difficult to account for in terms of simple RL is when one must take into account knowledge about the intentions of an intelligent opponent during competitive strategic interactions. Using fMRI, I will provide evidence for the existence of distinct computational signals in the brain that could underpin each of these RL-independent types of learning. Finally, I will review the implications of these findings for our current understanding of the neural and computational basis of valuation and choice
    ZI Mannheim