Learning to Predict in Networks with Heterogeneous and Dynamic Synapses

Learning to Predict in Networks with Heterogeneous and Dynamic Synapses. Daniel Burnham, Eric Shea-Brown, Stefan Mihalas. bioRxiv 2021.05.18.444107; doi: https://doi.org/10.1101/2021.05.18.444107

Models of Decision-Making During Probabilistic Foraging

Salient features of the environment are often obscured and must be inferred and integrated into an internal model used to drive appropriate behavior. This generality can be applied to many cognitive tasks, but for our purposes we focus on the decision-making paradigm of optimal foraging for depleting resources. Successful performance in this task requires an actor to understand the relevant environmental parameters and relate these features, via an internal model, to anticipated action outcomes. In the absence of such an internal model of the environment there are heuristic processes (e.g. model free reinforcement learning) that can also be used to motivate behavior though these do not guarantee optimal performance.

There is substantial evidence for animals adaptively employing at least these two (model free and model based) decision-making frameworks yet the underlying neural processes that mediate switching between these approaches has yet to be clearly described. One hypothesis is that neuromodulatory signals guiding attention (specifically serotonin) are key drivers of decision-making strategy switching. Our goal is to investigate this process in the context of a probabilistic foraging task, whereby mice sought rewards at either one of two water port foraging sites. Crucially, optimal performance on this task requires the animal to utilize an internal model of hidden task structure, namely to assume Markovian water port on-off switching dynamics. Previous work has demonstrated that both model free and optimal model based strategies can be readily inferred from animal behavior and neural activity based on differences in how these two approaches integrate action outcomes.