Can Differentiate Between Seeing and Doing in Rats

Classical conditioning mechanisms

can differentiate between seeing and doing in rats

Munir G. Kutlu and Nestor A. Schmajuk

Supplementary Online Material

The attentional-associative model

Schmajuk, Lam, & Gray model (SLG, 1996) presented an attentional-associative model of classical conditioning. The network incorporates (a) a mechanism that modulates attention to the conditioned stimuli (CSs) in proportion to the total novelty detected in the environment, and (b) a network that forms CS-CS and CS-unconditioned stimulus (US) excitatory and inhibitory associations, according to a real-time competitive rule. The model assumes that total novelty increases when (a) a predicted CS or predicted the US is absent, or (b) an unpredicted CS or unpredicted US is present.

Figure 1 shows a simplified diagram of the model that illustrates the different mechanisms involved in the generation of a conditioned response (CR) when a given CS is presented. Node 1 receives input from a short-term memory trace of the CS, τCS, and the prediction of that CS, BCS, by other CSs or the context (CX). In order to modulate attention to the CSs in proportion to the novelty detected in the environment, the output of Node 1, (τCS + BCS), becomes associated through zCS with the normalized value of the total novelty detected in the environment, Novelty’. Node 3 receives input from Node 2, XCS, as well as from the error term (US - BUS). The synaptic weight connecting Node 2 to Node 3, VCS-US, reflects the (excitatory or inhibitory) association of XCS with the US. Changes in VCS-US are proportional to a common error term (US - BUS), which reflects the difference between the predicted, BUS, and the real value of the US. The model describes forward (and backward sensory preconditioning in the version used in this paper), because a CS can be predicted by another CS through BCS in Node 1. Activation of Node 1 by BCS results in the activation of XCS (through zCS) in Node 2, and of BUS (through VCS-US) in Node 3. Finally, the CR is a non-linear function of BUS.

Figure 1 about here

1. Short-term memory and feedback. Node 1 in Figure 1 is activated by a short-term memory trace, τCS, of the CS. Changes in τCS are given by

d τCS/dt = K1 (λCS τCS), [ A1 ]

where K1 is the rate of increase and decay of τCS and λCS is the salience of the CS. By Equation A1, τCS increases over time from zero to a maximum when the CS is present and then gradually decays back to its initial value when the CS is absent. In order to make the trace of the CSs and the Bar Press long enough to have an effect after 40 t.u. of their offset, we assumed that K1 is larger when λCS > τCS than when λCS < τCS, that is, τCS decays at a slower rate than it growths.

The output of Node 1 is given by (τCS + K3 BCS), where BCS is the prediction of the CS by (a) itself, through CSi-CSi associations, (b) other CSs, through CSi-CSj associations, and (c) the context (CX), through CX-CSi associations. In the parenthesis, K3 is a feedback constant, K3 < 1 to avoid instabilities.

2.Attention. Attentional memory, zCS, reflects the association between (τCS + K3 BCS) and Novelty’. Changes in zCS are given by

d zCS /dt = (τCS + K3 BCS) ( K5 Novelty’ (1- zCS) - K6 ( 1 + zCS) ), [ A2 ]

where K5 is the rate of increase of zCS, K6 is the rate of decay of zCS, and Novelty’ is given by Equation A8. The synaptic weight (represented by the triangle in Figure 1) connecting Node 1 to Node 2 reflects the positive value of attention zCS, which can vary between 1 and –1. The initial value of zCS is zero. When Novelty’ is larger than a certain value, zCSgradually increases; when Novelty’ is smaller than another value, zCS decreases. Notice that Novelty’ need not change for zCS to increase or decrease. Because attention zCS is controlled by Novelty’, changes in zCS always lag behind changes in Novelty’. Therefore, zCS reflects the history of exposure of the CS to previous values of Novelty’.

The output of Node 2 is the attention-modulated representation of the CS, XCS. Importantly, the more negative zCS becomes the longer it will take to become positive again and have an effect on XCS. Therefore, whereas positive values of zCS can be interpreted as a measure of the attention directed to CS, negative values of zCS can be interpreted as a measure of the inattention to CS (see Lubow, 1989, page 192). Representation XCS, is given by

XCS = K2 (τCS + K3 BCS) (K4 + zCS). [ A3 ]

When zCS becomes negative, input (τCS + K3 BCS) activates XCS only through an unmodifiable connection K4 (not shown in Figure 1). This connection ensures that Node 1 and Node 2 are always minimally connected.

Latent inhibition and compound preexposure. Equation A3 allows the model to describe latent inhibition as well as its many properties (Lubow, 1989). In addition, the model describes the retardation in conditioning to an element of a compound following compound preexposure (Honey & Hall, 1989), which is similar to the case when Light-Tone presentations precede Light-Food presentations in the CC case and when Tone-Light presentations precede Light-Food presentations in the CH case.

3. Novelty. The novelty of event k (a CS, CX or the US) is computed as the absolute value of the difference between the average observed value of that event, and the average of the sum of all predictions of that event by all active CSs and CXs:

Noveltyk = |k - k|, [ A4 ]

where k is the average observed value of event k, and k is the average aggregate prediction of event k.

The average observed value of event k is given by

k = ( 1 k ) λk - K8k, [ A5 ]

where K8 is the rate of decay of k.

The average aggregate prediction of event k is given by

k = ( 1 k ) Bk - K8k, [ A6 ]

where K8 is the rate decay of k.

Total novelty, Novelty, at a given time is given by the sum of the novelty of all stimuli present or predicted at a given time. Novelty is given by

Novelty = Σk |k - k|, [ A7 ]

where k includes all CSs and the US.

The normalized value of Novelty, Novelty’, is given by

Novelty’ = Novelty2 / ( K92 + Novelty2 ). [ A8 ]

Novelty’ is used by the attentional system to define the value of zCS (Equation A2).

4. Changes in CS-US associations. Figure 1 shows that Node 3 receives input from Node 2, XCS, as well as from the US. The synaptic weight connecting Node 2 to Node 3, VCS-US, reflects the (excitatory or inhibitory) association of XCS with the US. Changes in this association, VCS-US, are given by

dVCS-US /dt = K7 XCS (λUS - BUS) ( 1 - |VCS-US|), [ A9 ]

where XCS is the internal representation of the CS, BUS is the aggregate prediction of the US by all X's active at a given time (See Equation A3), and λUS is the strength of the US.

As in the Rescorla-Wagner (R-W) (1972) model, changes in VCS-US are proportional to a common error term given by the difference between predicted and real values of US, (λUS - BUS). As in the Rescorla-Wagner model, conditioned inhibition is the result of presenting an excitatory CS together with a neutral CS. As suggested by Zimmer-Hart and Rescorla (1974), in contrast to the Rescorla-Wagner model and in order to prevent the extinction of conditioned inhibition (Zimmer-Hart and Rescorla, 1974) or the generation of an excitatory CS by presenting a neutral CS with an inhibitory CS (Baker, 1974), our model assumes that BUS (and BCS) takes on only positive values (BUS = 0 when BUS < 0).

According to Equation A9, changes in associations VCS-US are also proportional to (1 - |VCS-US|). This term (a) limits VCS-US, -1< VCS-US < +1, and (b) makes changes in VCS-US relatively slow when VCS-US approaches 1 or –1 (and relatively fast when VCS-US is close to 0).

Mediated acquisition and extinction. Because XCS in Equation A9 is proportional to τCS + K3 BCS (see Equation A3), changes in VCS-US might take place even in the absence of the CS, if the CS is predicted by other CSs or the CX. That is the model is able to described mediated excitatory and inhibitory acquisition (Holland & Sherwood, 2008) and mediated extinction (Shevill & Hall, 2004).

Protection from extinction. Notice that by Equation A9, dVCS-US /dt = 0 whenever λUS = BUS, that is when the US is perfectly predicted. In that case, if an excitatory CS is present together with an inhibitory CS and BUS = 0 (see Equation A10), the excitatory association will not decrease even when the US is absent, λUS = 0. We use this mechanism, referred to as “protection from extinction” (Chorazyna, 1962; Soltysik, 1985; Rescorla, 2003), to explain why responding during Observation is relatively strong even after testing in the Intervention condition.

5. Changes in CS-CS associations. Although not shown in Figure 1, the model also stores (1) associations of each XCS with its corresponding CS, e.g., VCS1-CS1; (2) associations between XCS with other CSs, e.g., VCS1-CS2 (see Rescorla and Durlach, 1981); (3) associations of XCS with the context (CX), VCS-CX (see Rescorla, 1984); (4) associations of XCX with the CS, VCX-CS (see Marlin, 1982); and (5) associations of XCX with the US, VCX-US (see Baker et al., 1981). Changes in the CS1-CS2 associations, VCS1-CS2, are given by

dVCS1-CS2 /dt = K7 XCS1 (λCS2 - BCS2) ( 1 - |V CS1-CS2|), [ A9’]

where XCS1 is the internal representation of CS1, BCS2 is the aggregate prediction of CS2 by all XCS 's active at a given time, and λCS2 is the value of CS2.

Forward and backward sensory preconditioning. Equation A9’ allows the original model to describe forward sensory preconditioning, by establishing CS1-CS2 associations which can be combined with CS2-US associations. This is accomplished because the output of Node 1 is given by (τCS2 + K3 BCS2), and BCS2 = XCS1 VCS1-CS2. Therefore, when CS1 is presented in the absence of CS2, the output of Node 1 is XCS1 VCS1-CS2, which activates zCS2 and VCS2-US, to generate a CR to CS1. Even if the model is able to combine CS1-CS2 with CS2-US associations and describe forward preconditioning when CS1 preceded CS2, Equation A9’ does not yield the CS2-CS1 associations needed to describe backward sensory preconditioning. In the present study, we initially thought that backward sensory preconditioning was needed to explain why Light-Tone and Light-Food presentations (CC training) result in the Tone being able to predict the Food and the conditioned nose pokes controlled by that prediction.

Schmajuk and Larrauri (2006, page 10) proposed to modify Equation A9’ by replacing the actual value of the predicted CS2 (Tone), λCS2, by its trace, CS2.

dVCS1-CS2 /dt = K7 XCS1 (CS2 - BCS2) ( 1 - |V CS1-CS2|), [ A9’’]

This change allows the model to build VCS2-CS1 (Tone-Light) associations even when CS1 (Light) precedes CS2 (Tone) because their traces overlap. Schmajuk and Larrauri (2006) demonstrated that computer simulations run with this modification yield backward sensory preconditioning (Ward-Robinson and Hall, 1996). Therefore, in the present study we use the Schmajuk-Larrauri (2006) modification of the SLG model to describe associations formed during CC training. Notice that the same CS1-CS2 associations that describe sensory preconditioning can describe second-order conditioning, in which CS1-US presentations precede CS1-CS2 presentations. For consistency, λUS in Equation A9 was also replaced by λUS which allows the model to describe backward conditioning.

6. Aggregate predictions of the US and CS.The output of Node 3 in Figure 1 is the aggregate prediction of the US by all CSs with representations active at a given time, BUS, given by

BUS = CS BCS-US = CS XCS VCS-US, [ A10]

where BCS-US is the prediction of the US by CS and VCS-US is the association of XCS with the US. BUS is used to compute VCS-US in Equation A9, and determines the magnitude of the CR. As mentioned, although VCS-US can be either positive (excitatory) or negative (inhibitory), BUS is always positive.

Not shown in Figure 1, the aggregate prediction of CS by all CSs with representations active at a given time, BCS, given by

BCS = i BCSi-CS = i XCSi VCSi-CS, [ A10' ]

where BCSi,CS is the prediction of CS by CSi and VCSi, CS is the association of XCSi with CS.

7.CR strength. The output of Node 3 is BUS, which directly controls the CR: according to a non-linear function

CR = (1- K10 OR) BUS2 / (K112 + BUS2). [ A11 ]

Although the original model assumed that the orienting response (OR = Novelty’) inhibits the CR, Schmajuk and Larrauri (2006) explained that this is not the case when (a) a suppression paradigm is used because both appetitive and aversive behaviors are affected and the effects possibly cancel out, or (b) when the conditioned response is freezing, because both the CR and the OR work in the same direction. Therefore, they assumed that K10= 0.

For simplicity, in the present paper we assume that

CR = BUS. [ A11’ ]

Similar results are obtained with Equation A11.

Parameter values used in the simulations.

Parameters values used in the simulations are K1= .2, K2= 2, K3= .4, K4= .1, K5= .02, K6= .005, K7= .005, K8= .005, K9= .75, K10= 0, and K11= .15. These values are identical to those used in Schmajuk et al. (1996), Schmajuk and Larrauri (2006), Schmajuk et al. (2007) and Larrauri and Schmajuk (2008), with exception of K1. As in previous publications, the rate of growth of all short-term memory traces, K1 = .2 when λCS > τCS or λBAR > τBAR. In order to extend the duration of their short-term memory traces, the rate of decay of all CS traces, including that of the Bar Press, is K1 = .02.

Interventions but not exogenous cues affect responding following common-cause training

Leising et al.’s (2008) Experiment 1 showed that, following CC training responding was decreased following Intervention but not following Observation or the presentation of a new, “Exogenous” Cue.

Causal-model theory vs. attentional-associative theory. According to the causal reasoning approach, whereas Interventions result in the removal of the Tone-Light link, that connection remains intact in the Observation and Exogenous-Cue condition (Figure 2, Upper Panel). Causal models expect that observing the barometer reading (Tone) leads to the assumption that atmospheric pressure changes (Light) were also present but missed—either in the absence or the presence of the Exogenous Cue—, and therefore expect weather changes (Food).

Our associative model predicts stronger responding to Exogenous Cues than to Interventions because XEXOGENOUS CUE at the time of the Tone presentation (Figure 2, Lower Panel) is much weaker than XBAR PRESS (Figure 4, Upper Panel) thereby providing less activation to similarly weak inhibitory connections with Food.

Figure 2 about here

Experimental data. In Leising et al.’s (2008) Experiment 1, rats received 24 Light→Tone pairings followed by 24 Light→Food pairings (CC design). After training, a lever was placed into the testing chamber for the Intervention and Observation groups, with no consequence for the latter. Subjects in the Intervention Group received a Tone presentation with every Bar Press, while subjects in the Observation Group were yoked pairs of the Intervention Group. Subjects in an Exogenous-Cue Group were presented with a 10-s Clicker followed immediately after its offset by a 10-s Tone during the test trials. Testing consisted of 60 min sessions over 1 day. For all groups, the number of nose pokes into the food magazine was recorded. Figure 3 (Upper Panel) shows that the number of nose pokes was significantly lower for the Intervention Group than those of the other two groups. Responses of the Exogenous-Cue and Observation groups did not differ.

Figure 3 about here

Simulated Results. Simulations with the SLG model consisted of 48 Light→Tone trials followed by 48Light→Food trials and (CC design), and 4 test trials either with (a) the Exogenous Cue followed by Tone, (b) Bar Press followed by Tone, or (c) Tone alone.

During Light→Tone trials, the 40-t.u. Light preceded the 40-t.u. Tone. During Light→Food trials, the 40-t.u. Light preceded the 40-t.u. Food US. During testing of the Intervention Group, the 40-t.u. Bar Press CS started simultaneously and fully overlapped with the 40-t.u. Tone. During testing of the Exogenous-Cue group, a 40-t.u. Clicker was presented preceding the Tone. During testing of the Observation Group, the 40-t.u. Tone was presented alone. Testing consisted of 3 test trials over “1 day.” Because Bar Pressing had no consequence in the Observation groups, during the test trials a Bar lever CS was simulated in the Intervention, but not for the Observation groups. All the CS (including the Bar Press and Exogenous Cue), Magazine saliences, US strength, and ITI were as in the simulations for Leising et al.’s (2009) Experiment 3. Figure 3 (Lower Panel) shows that the CR is weaker in the Intervention case than in the Exogenous-Cue and Observation cases, which did not differ.

Exogenous Cue Group. As explained so far for the Intervention groups, VBAR PRESS-FOOD becomes inhibitory during the test trials. As shown in Figure 4, VEXOGENOUS CUE-FOOD also becomes inhibitory because the absent Food is predicted by adding VMAG-FOOD, VTONE-FOOD and the combination of VTONE-MAG VMAG-FOOD associations. Although the magnitudes of the inhibitory VBAR PRESS-FOOD and VEXOGENOUS CUE-FOOD associations are very similar (Figure 4, Lower Panel), the internal representation of the Bar Press (XBAR PRESS) is much stronger than the internal representation of the Exogenous Cue (XEXOGENOUS CUE) during the Tone presentation (Figure 4, Upper Panel), because the Bar Press CS is more salient than those of the Exogenous Cue. Therefore, the inhibition exerted by XBAR PRESS on responding to the Tone (Equation 2) is much stronger than that exerted by XEXOGENOUS CUE in Equation 7.

CRCC-EXOGENOUS CUE (Tone) = XMAG VMAG-FOOD + XTONE VTONE-MAG VMAG-FOOD –

XBEXOGENOUS CUE VEXOGENOUS CUE-FOOD [7]

Predictions of the model. A prediction of our associative model, but not of causal-model theory, is decreased responding by (a) increasing the number of test trials using the Exogenous Cue, (b) presenting the Exogenous Cue starting simultaneously and perfectly overlapping with the Tone and (c) increasing the intensity of the Exogenous Cue. The last change would make the Exogenous Cue more similar to the Bar Press CS. However, because salience is a function not only of the intensity of a CS, but also of attentional, emotional, motivational and cognitive (e.g., sensory-motor feedback) factors, it is possible that even an intense Exogenous Cue will not decrease nose poking to the low level attained by a Bar Press. Conversely, but also expected by causal-model theory, the model predicts that increasing the Bar Press-Tone interval (making the Intervention more like the Exogenous Cue used in Leising et al.’s Experiment 1) will decrease the inhibition exerted by the Intervention on responding.

References

Baker, A.G., Mercier, P., Gabel, J.Baker, P.A.(1981). Contextual conditioning and the US preexposure effect in conditioned fear. Journal of Experimental Psychology: Animal Behavior Processes, 7, 109–128.

Chorazyna, H. (1962). Some properties of conditioned inhibition. Acta Biologiae Experimentalis, 22, 5–13.

Holland, P.C., & Sherwood, A. (2008). Formation of excitatory and inhibitory associations between absent events. Journal of Experimental Psychology: Animal Behavior Processes, 34, 324-335.

Honey, R. C., & Hall, G. (1989). Attenuation of Latent Inhibition after Compound Pre-exposure: Associative and Perceptual Explanations. The Quarterly Journal of Experimental Psychology, 418, 355-368.