We employed a choice task previously used to demonstrate strategic exploratory decision-making behavior in healthy humans (Badre et al, 2012; Frank et al, 2009). All groups show a conserved effect of valence such that exploration was higher in the reward domain compared with the loss domain. Indeed, in the loss domains, subjects showed a consistently negative exploration parameter, meaning that they were averse to uncertainty when there was some prospect of losing even more. These findings potentially reflect the asymmetrical influence of gains and losses on choice behavior (Kahneman and Tversky, 1979) imposed by the strength of loss aversion as a consistent mediator of choice (Tom et al, 2007).
Exploratory behavior in subjects with AUD was reduced across gain and loss environments, in favor of more repetitive or exploitative choices. Obese subjects with and without BED did not differ from HV in their exploratory choices. However, when compared with each other, there was greater exploratory behaviors in BED subjects compared with those without BED. There was a trend toward a group × valence interaction driven by greater exploratory behaviors to losses in BED subjects compared with those without BED. Similarly, BED subjects had greater exploratory behaviors particularly to losses compared with AUD. Furthermore, we investigated the influence of smoking in HV on a pilot basis: current smokers showed an enhancement of the influence of valence with greater exploration to gain outcomes and less exploration to loss outcomes compared with non-smokers. Exploratory behavior in HV was associated with an underlying network including FPC and ventral striatal connectivity in the context of reward and FPC and precuneus for losses.
Compared with HV, AUD subjects had restricted exploratory behaviors and were more likely to avoid uncertainty across both gain and loss stimulus-outcomes in a task that is independent of learning. AUD subject have been shown to have abnormalities in decision making under ambiguity or uncertainty as measured using the Iowa Gambling Task (Goudriaan et al, 2005; Bechara et al, 2001). Our findings extend these results to suggest either intolerance/avoidance of uncertainty, or a reduced tendency to use a controlled strategy that searches for uncertain outcomes so as to maximize information gain. The current findings of reduced exploration in an unknown environment dovetail with findings suggesting that the effects of alcohol are selective for uncertainty-related anxiety rather than certainty-related fear (Hefner and Curtin, 2012), the former being hypothesized to drive the negative-reinforcement cycle of alcohol use (Edwards and Koob, 2010). An alternate explanation may be that changes in outcome sensitivity, rather than uncertainty avoidance, may engender reluctance to explore. However, decreased sensitivity to outcome may be more likely to manifest as greater exploration to sample further stimulus-outcome contingencies. Although we do not explicitly measure the role of novelty, decreased exploration may relate to the possible presence of novel environments. Ethanol withdrawal in rodents indeed causes reduced exploration of brightly lit chambers (Hascoet et al, 2001).
Furthermore, like HV, AUD subjects had decreased exploratory behaviors to losses compared with gains suggesting sensitivity to their differential influences. Current smokers also have an enhancement of this differential effect of valence with greater exploratory behaviors to gains and the opposite to losses relative to non-smokers. The enhancement in exploration for gains is in line with enhanced reward sensitivity related to nicotine use (Rose et al, 2013). This finding invites the suggestion that participants who are more likely to explore the potential hedonic benefits of smoking are those that become smokers. The findings in the loss domain suggest a potential role for enhanced loss aversion in smokers with greater avoidance of uncertainty in a loss context, perhaps facilitating sustained smoking in the presence of perceived small losses associated with immediate health consequences, rather than explore alternative strategies that would require giving up smoking for potentially other (eg social) losses. Although losses in the form of social and health cost are difficult to model, the secondary reinforcer of money can act as a proxy. These findings in AUD and smokers may be consistent with the negative reinforcement model of addiction (Koob, 2013; Koob and Le Moal, 2005) whereby a negative context may drive exploitative repetitive behaviors to avoid losses. Reduced exploration, or more repetitive choices, in the face of losses is consistent with theories that neuroadaptive systems driving aversive states lead to repetitive drug-seeking behaviors (Edwards and Koob, 2010). Indeed, negative affect in smokers is associated with craving severity (Robinson et al, 2011). Together with the current findings, this may explain how particular environmental influences (ie negative outcomes in the form of financial, social, or health losses) may facilitate the repetition of behaviors with certain, known outcomes, such as pathological drinking and smoking behaviors. Although these findings are intriguing, we caution that the findings in smokers are preliminary as the sample size of current users is small, and we cannot rule out an impact of nicotine etc. on exploration rather than the other direction of causality.
That subjects display reduced exploration for losses contrasts with the observation of enhanced ambiguity seeking in the face of losses in healthy humans (Ho et al, 2002; Chakravarty and Roy, 2009). However, this discrepancy is also similar to the observation of ambiguity aversion in the face of gains, despite exploration toward uncertain options in that case. The main difference is that in a learning task, choosing an ambiguous option can serve to reduce subsequent ambiguity, ie exploration drives learning. In the case of losses, it is thus perhaps surprising that subjects do not seek uncertain options to reduce subsequent ambiguity. In addition, the current study deals with explicit and experienced uncertainty rather than hypothetical ambiguity. The effect of valence on risky choice has been shown to be reversed when choices are either experience or description-based, with the former reducing risk-seeking for losses (Ludvig and Spetch, 2011) consistent with our findings. Furthermore, there may be at least two strategies for approaching an explore–exploit dilemma: choice biased toward information seeking; and random exploratory decisions involving chance (Wilson et al, 2014) and perhaps subjects adopted a strategy to simply increase random choices in the case of losses rather than rely on uncertainty.
Our findings show decreased exploration in obese subjects without BED as compared with BED suggesting differences as a function of greater avoidance of uncertainty. BED subjects appear to be more biased toward exploratory behavior but particularly in the context of losses and not to gains, that is, the opposite profile from smokers. These findings are similarly evident in the comparison of AUD and BED subjects in which BED have greater exploratory behaviors and particularly in the loss domain. This dissociation of valence coincides with previous work showing that BED subjects demonstrate greater risk taking for high probability losses only (Voon et al, 2014c) possibly suggesting less of an influence of loss aversion. These findings suggest differences between AUD and BED subjects particularly in the loss domain. Whether the distinct rewards of choice (natural or drug) are responsible for causing increased or decreased exploration in the face of loss or whether they are a product of an inherent attraction or aversion to exploration, remains a question for future studies. The suggestion that neuroadaptive negative reinforcement systems are initiated or propagated by excessive reward system activation (Koob, 2013), may explain the current finding of heightened sensitivity to losses in smokers and individuals with AUD, but not in BED, whereby nicotine and alcohol hijack the reward system to a greater degree than food. Moreover, we note that the negative consequences of binge eating on weight gain are far more immediate than those of smoking, which are perceived to be delayed and subject to potential quitting.
Our findings further highlight a role for an intrinsic network of FPC connectivity in exploration biases. The FPC sits at the outermost periphery of the hierarchical prefrontal control regions (Christoff and Gabrieli, 2000; Koechlin and Hyafil, 2007), being well poised to mediate higher level strategic switches rather than behavioral sequence control. Accumulating evidence suggests that through interactions with social/emotional network (orbitofrontal cortex, amygdala), cognitive network (dorsolateral prefrontal cortex) and default mode network (precuneus, anterior cingulate cortex; Liu et al, 2013), the FPC orchestrates more flexible and self-relevant behavioral control in the pursuit of optimal decision-making (Koechlin and Hyafil, 2007). We show that FPC and ventral striatal connectivity is associated with exploration in the context of a rewarding environment. This coincides with the notion that the FPC coordinates voluntary and adaptive switching based on uncertainty and expected value (Badre et al, 2012; Daw et al, 2006). Exploration may depend on the probability that an explored choice will provide a better outcome than expected based on previous experiences (a positive prediction error; Frank et al, 2009). It is thus possible that the FPC-VS connectivity implies a reward value assignment to the potential for exploring. This would not be expected in the context of losses because the value of exploring is only to reduce loss values rather than provide a positive outcome.
We also show that FPC and precuneus connectivity positively correlates with exploration in the loss domain. Although the precuneus has been traditionally associated with integration of visuo-spatial imagery (Selemon and Goldman-Rakic, 1988), converging evidence suggests a role in integration of external and self-relevant information (Cavanna and Trimble, 2006). Furthermore, goal-directed hand movements (Karnath and Perenin, 2005) and voluntary attentional shifts between targets even in the absence of an overt motor response (Culham et al, 1998), are mediated by the precuneus. Functional links between FPC and the default mode network (Liu et al, 2013) support its role in processing internal rather than external generation of information (Christoff and Gabrieli, 2000) to guide future-focused (Okuda et al, 2003) decision-making. The current findings suggest that although assignment of perceived agency to actions and encoding and organizing of intentions is mediated by the precuneus, it may interact with the FPC (Liu et al, 2013) which in turn processes internally-generated goals for behavioral control (Ramnani and Owen, 2004; Okuda et al, 2003). Further evidence of the role of the precuneus in exploratory choices comes from studies of foraging behavior. Humans may alternate between economic decisions and choices governed by sequential ‘engage or search elsewhere’ foraging choices (Kolling et al, 2012). Foraging choices (compared with decisions between two options) have been associated with activations in the precuneus extending to posterior cingulate cortex (PCC; Kolling et al, 2012) and PCC seems to be sensitive to risker compared with safer choices (Kolling et al, 2014). That this region is associated with risker choices suggests why it may be associated with exploratory choices losses rather than rewards.
Although recent evidence implicates both FPC and inferior parietal cortex in exploratory choices (Daw et al, 2006; Boorman et al, 2009), we did not find significant correlations for inferior parietal cortex. In a previous study, activity in both FPC and the inferior parietal sulcus correlated with the ratio between an unchosen and chosen action probability, or the relative unchosen probability (Boorman et al, 2009). However, the inferior parietal sulcus was only recruited when a switch in choice occurred (Boorman et al, 2009). Therefore, the FPC seems to track information accumulation relevant to switching to an alternate choice—here to reduce uncertainty—but engages the parietal cortex immediately before switching, which implements the switch itself. In line with this hypothesis, a recent study examining negative outcomes implicated the inferior parietal cortex in encoding actions and outcome objects but a more medial region, similar to that implicated in the current study, in encoding the action × object interaction reflecting the appropriate or inappropriate action (Morrison et al, 2013).
Our findings suggest biases in exploratory behaviors in the context of an uncertain environment across the misuse of drug and natural rewards. We also highlight the conserved effect of valence on exploration across groups with enhanced uncertainty avoidance to losses possibly reflecting an interaction with underlying loss aversion tendencies. Although we do not currently examine the neural correlates of exploration in the pathological groups, we build upon the understanding of the role of the FPC in guiding higher order and flexible decision-making, illustrating the possible means through which it coordinates behavioral processes in HV. Together, the findings further the characterization of overlapping disorders of natural and drug rewards by maintaining the use of dimensional facets of compulsivity.
Открыв полку над головой, он вспомнил, что багажа у него. Времени на сборы ему не дали, да какая разница: ему же обещали, что путешествие будет недолгим - туда и обратно. Двигатели снизили обороты, и самолет с залитого солнцем летного поля въехал в пустой ангар напротив главного терминала.
Вскоре появился пилот и открыл люк. Беккер быстро допил остатки клюквенного сока, поставил стакан на мокрую столешницу и надел пиджак.