2Nd Term Topic: Operant Conditioning with Sniffy

210 Winter 2014Operant Conditioning1

Psychology 210: Learning

Assignment #2 Lab Manual

2nd Term Topic: Operant Conditioning with Sniffy

Due Friday, April 4th (in class)

Manual Index

Purpose and background info

Starting the program and conducting the experiments

Your cumulative records: What to print and attach with your assignment

Complete instructions for training Sniffy on the different schedules

Writing your report

Grading rubric

Checklist – make sure you have everything listed before you pass in your assignment

Plagiarism/cheating policy

Lab Experiment: Operant Conditioning with Sniffy the Virtual Rat

Lab Report Due Date: Friday, April 4th, in class

Purpose

The Sniffy program is meant to give you first-hand experience with operant conditioning so that you can humanely learn about observational techniques, magazine training, shaping behavior, reinforcement schedules, acquisition, and extinction.

The Sniffy program contains Sniffy the virtual rat, an operant conditioning chamber, and a cumulative record (CR) that will allow you to track Sniffy’s training progress. The operant conditioning chamber in the program has a water spout, a bar, and a food dispenser (called a magazine). By default, any time Sniffy presses the bar, a food pellet will drop into the food hopper.

You’re going to train Sniffy to bar-press for food pellets on a continuous reinforcement (CRF) schedule, then four different partial reinforcement (PRF) schedules. There will be things you should make note of to report in your results section, or to write on your printed records. This is all outlined in the instructions below.

Background Information

Operant conditioning increases the rate of a desired behavior by contingently reinforcing that behavior once it occurs. In our lab, we will condition a virtual rat named “Sniffy” to bar-press, by rewarding him with food pellets. Sniffy may not spontaneously perform this behavior on his own so we will first need to shape his behavior through successive approximation, where we will reward him for performing actions that are increasingly similar to the one we want to train. Once Sniffy does learn to associate bar-pressing with food-pellets there are two different ways he can be reinforced: Continuously or Partially. When on a Continuous Reinforcement (CRF) Schedule, Sniffy receives a pellet every time he bar-presses (this is the default setting in the Sniffy program and this is how he’ll be reinforced when he first learns to bar-press). When on a Partial Reinforcement (PRF) Schedule, Sniffy receives a pellet sometimes, but not every time he bar-presses. The timing of reward on a PRF schedule differs depending on whether it is a Fixed Ratio (FR), Variable Ratio (VR), Fixed Interval (FI), or Variable Interval (VI) schedule. Ratio refers to a number of bar-presses being the determinant for reward. Interval refers to a period of time elapsing as the determinant for reward. Fixed means that there is a specific, set amount of bar-presses, or time required to pass, before reward will be given. Variable means that the number of presses or the period of time, changes slightly, and reward will be given around an average amount of presses or time. For example, if you are training on a VR5 schedule, this means that after about 5 bar-presses Sniffy will get a reward. In other words, he may get rewarded after 3 presses, after 4 presses, after 5 presses, or after 6, but the average will be 5. Sniffy does not know exactly when he will be rewarded. If it were a FR5 schedule, he would get reward after exactly 5 presses every time. On a VI5 schedule, reward would be given when Sniffy responds after about 5 seconds. On a FI5 schedule, reward would be given when Sniffy responded exactly 5 seconds after previous reinforcement.

Steps to Completing this Lab

1. Observe Sniffy’s baseline behaviors
2. Magazine Training
3. CRF Training

SAVE YOUR CRF-TRAINED RAT TO YOUR H: DRIVE AND DON’T SAVE OVER THIS FILE

Always save a new file after subsequent steps when you’re saving you progress – It’s not a bad idea to save a 2nd copy of your CRF Sniffy in case you accidentally hit ‘save’ instead of ‘save as’ at some point. If you lose your CRF-trained Sniffy file, you won’t have anything to work from for the next steps, because they all start with the CRF-trained Sniffy.

1. Extinguish the CRF-trained Sniffy
2. Observe Sniffy’s Spontaneous Recovery

Steps 6-9 involve a PRF schedule.

1. Open original CRF-Trained Sniffy  Train on a VR 20 Schedule and Extinguish
2. Open original Sniffy  Train on a VI 20 Schedule and Extinguish
3. Open original Sniffy  Train on a FR 20 Schedule and Extinguish
4. Open original Sniffy Train on a FI 20 Schedule and Extinguish

To Start the Program

After you log on to a computer on campus, from the “Start” menu select “My Computer”, then “Apps on ‘caesar’(S:)”. Double-click on the “Sniffy” folder, and then on the Sniffy icon.

To deliver food pellets to the magazine, press the spacebar or click on the bar with your mouse.

Menu Information

- Under File, you can start a new Sniffy, open an experiment file, save your files, and exit.

- Under Parameters, you can define your training schedules, and study spontaneous recovery with “Remove Sniffy for Time-Out”. You will not need to change the parameters until after Sniffy has been fully trained to bar-press (CRF).

- Windows: There are two windows, one with the operant chamber and one with the cumulative record. The cumulative record window has vertical lines that represent five-minute intervals. Up to five pages of data can be saved by the recorder; each record runs for two hours. After 2 hours, a new cumulative record file will open.

Materials Needed

When you go to a lab to get started, bring a stopwatch or a stopwatch app with you. You should have a pen and some paper to make notes for your results section as needed.

Attaching your Cumulative Records

You will print records for:

1. CRF Training, Extinction, and Spontaneous Recovery (Steps 1-5)
2. VR 20 Training and Extinction (Step 6)
3. VI 20 Training and Extinction (Step 7)
4. FR 20 Training and Extinction (Step 8)
5. FI 20 Training and Extinction (Step 9)

Label each set of records at the top of the first page, with a pen. On each set of records indicate when acquisition was complete, when extinction started, and when extinction was complete. You should print your records and make notes on them as you go, rather than waiting until you’re finished all the steps. You will attach these CRs at the end of your assignment in the order listed above. Please avoid attaching multiple copies of the same steps. In other words, don’t attach FR training, and then FR training with extinction; just attach the latter copy that includes both steps. (You can print as often as you choose, just don’t include extras in the assignment)

Complete Training Instructions

Step 1: Observing Baseline Behaviors

First, observe what actions your untrained subject naturally exhibits (baseline behaviour). Observe Sniffy for 10 minutes to become familiar with the rat’s repertoire of behaviour. This will help you identify what you’ll be reinforcing later, during magazine training and during CRF-training.

Second, find Sniffy’s Baseline Bar-Pressing Rate. After 10 minutes of observation, establish the baseline frequency of bar pressing per minute. The cumulative record shown at the bottom of the screen shows a tick mark every time the rat was reinforced for bar-pressing. The vertical lines represent five-minute blocks. Therefore, count the number of ticks from the start of the cumulative record up to the second vertical line and divide by 10 to calculate the number of bar presses per minute. Keep this number in your records. (Note: His baseline rate might be zero, or he could bar-press several times during those 10 minutes.)

Step 2: Magazine-Training Sniffy

Wait until Sniffy is away from the magazine (where food pellets are delivered), then mouse click on the bar above the magazine (or press the space bar) to deliver a food pellet. After Sniffy finds and eats the pellet, immediately give him another pellet. Continue training for 10 to 15 more approaches (i.e. 20 – 30 food pellets). Wait until the rat has left the magazine area and subsequently returned before initiating another training trial. Your rat should now be trained to approach the magazine whenever a food pellet is delivered. To test that your rat has been trained, wait until Sniffy is in a corner away from the food magazine, then deliver another pellet. If he is magazine trained, he will make his way over to the magazine to eat right away. Record how long magazine training took to accomplish.

Step 3: CRF-Training Sniffy

We’re going to train Sniffy to bar press through shaping by successive approximations. It involves reinforcing activities that are more and more similar to the bar pressing response until the animal begins to make that response. To be effective, the reinforcer must be delivered to Sniffy immediately after the target behaviour has started. The skill involved is to identify and move through the appropriate steps. If you reinforce one action too often, Sniffy may fixate on it. You must now decide which activities approximate bar pressing and how closely they do so. So, you may start by rewarding him for reaching up on a side wall, for reaching up on the back wall (gradually begin to reward more often for the back wall than for the side wall but not so much that he will fixate), and/or for drinking water, and also if he, by chance, presses the bar on his own, or even if he just walks by the magazine. Gradually get him to “realize” that bar pressing equals reward by rewarding him for increasingly similar behaviour. You must pay careful attention to the screen at all times for this step. If he performs behaviour you should reward and you miss it (and thus miss rewarding it) this could impede training. I encourage you to take a break if needed before starting CRF training, because you absolutely need to devote 100% of your attention to CRF-training or it will take you a lot longer.

When you are successful, you will see Sniffy press the bar four or five times within 1 minute. When that happens, stop delivering pellets with your space bar and let Sniffy press the bar himself. Over the next several minutes, the response rate will climb; the cumulative record will grow steeper. When the cumulative record line reaches the top of the record, it will reset down to the bottom and continue climbing back up. Conditioning is considered complete when it resets a second time (i.e., once it climbs up and drops down two times). **Observe and record Sniffy’s behaviour and rate of responding within a 2 minute time period, upon acquisition of this response.

Now save your trained Sniffy. It is suggested that you save your trained rat as CRF.sdf (or some name that identifies this is the Sniffy you’ll be working from). Now you have your own Sniffy to use in the remaining steps of the experiment. You need to save it on your H: drive (there is a pull-down menu at the bottom of your save window). After saving, exit the program and restart. Open your file of a successfully trained Sniffy before extinction and let Sniffy run for one minute under a Continuous Reinforcement schedule (CRF) then begin Step 4 below.

Step 4: Extinguishing Behaviour after CRF Training

Open your CRF trained Sniffy and allow him to run for one minute on a CRF schedule. Then:

 Select “Training Schedules” in the “Parameters” menu and click “Extinction”. This means that Sniffy will no longer be reinforced for bar pressing. Then click OK.

Time and record the period from when you stopped reinforcing bar-pressing until extinction is complete (this is the extinction baseline) and count and report the number of non-reinforced bar presses during the first two minutes. Bar pressing behavior is considered to be extinguished when Sniffy presses the bar fewer than three times in five minutes.

** Select “SAVE AS” in the file menu and save this file under a new name (e.g. “Ext.cdf”) **

Step 5: Observing Spontaneous Recovery:

To observe the effects of a time-out after extinction, select “Remove Sniffy for Time-Out” in the “Parameters menu”, select “OK”. Be certain that the Training schedule remains on “extinction”. Count and record the bar press responses for two minutes. Measure and report the time from when Sniffy returns to the cage until bar pressing is extinguished again (i.e. Less than three presses in five minutes).

Save this as a new file, and now Print a copy of your cumulative record to be handed in with your assignment. This record will include your behaviour observation, magazine training, CRF training, extinction, and spontaneous recovery. Indicate when acquisition was complete, when extinction started, and when extinction was complete. Note where you removed Sniffy for time-out and where extinction occurred after SR.

Beginning of PRF Training

PRF schedules are changed using the Training Schedules under the Parameters menu. Fixed schedules are chosen by clicking the “Fixed” button and Variable schedules are chosen by clicking the “Variable” button. Ratio schedules are chosen by clicking the “Responses” button and typing an integer in the number-entry box. Interval schedules are chosen by clicking the “Seconds” button and typing an integer in the number-entry box.

Open the program following the above directions, and open a copy of your CRF trained rat (un-extinguished). Systematically work through training and extinguishing your CRF trained rat on all four PRF schedules using the following instructions.

Step 6: Training Sniffy on a FR 20 Schedule

Open your original CRF Trained Sniffy (that hasn’t gone through Extinction). Once you have let Sniffy run untouched on a CRF schedule for one minute you can begin to make the transition to the FR 20 target by changing Sniffy to a FR schedule. Do this by selecting “Fixed” and “Responses” in the Parameters menu. Note that you cannot just put 20 in as your first value; you MUST start with a small PRF schedule and gradually increase it after the animal’s behaviour stabilizes at each value. It is probably best to start with a very low value (3 or 4) and work in small interval increases (intervals of 3-5, perhaps – you’ll have to start over if it’s not working to take your time and use small increments). Sniffy’s behaviour will be extinguished if you choose too large a value when you first place him on a PRF schedule. If Sniffy is maintaining bar pressing at a stable rate for 15 food pellets on that schedule (there will be 15 ticks on your CR), then he has learned the new schedule and can then be trained on the next interval until the target value is acquired (i.e., wait until Sniffy receives at least 15 pellets before moving up to the next interval). For example, you may wish to start Sniffy out on a schedule of FR 5. Wait until he receives 15 pellets on that schedule, and then try moving him up to FR 10. Wait another 15 pellets, and so on until he has received 15 pellets on the FR 20 schedule.

**When Sniffy has received at least 15 pellets on the FR 20 schedule he is considered trained.

* Save your file under a new name

Extinction:

Next, you need to measure extinction after the schedule has been acquired. Use the same method as in Step 4. Restart the program and open your saved FR20 Sniffy. Don’t forget to time and record the period from when you instituted extinction until extinction is complete (the extinction baseline) and count and record the number of non-reinforced bar presses during the first two minutes.

*You do not need to do spontaneous recovery for the PRF schedules.

** Save and Print a copy of your cumulative record**

Step 7: Training Sniffy on a VR 20 Schedule

Open your original CRF Trained Sniffy and let him run for one minute. In the Parameters Menu select “Variable” and “Responses”. Remember that you cannot start at 20 seconds; you must start with a small number and gradually increase it in intervals just as you did in the previous step. You can use the same incremental increases you used in the previous step. When Sniffy receives at least 15 pellets on the target schedule of VR20, save your file under a new name.

Exit the program and restart. Open your saved VR 20 Sniffy. Follow the same extinction instructions as in Step 4. Time and record the period from when you instituted extinction to when it was complete (extinction baseline) and count and record the number of non-reinforced bar-presses in the first two minutes. Save and print your cumulative records