An Examination of the Role of Exogenous Events in a Multiple Object Tracking Task

Divided attention task

1

AN EXAMINATION OF THE ROLE OF EXOGENOUS EVENTS IN A MULTIPLE OBJECT TRACKING TASK.

ALEXANDER KUSHNIER

A thesis submitted to

The Department of Psychology

RutgersUniversity

Written under the direction of

Dr. Zenon Pylyshyn

Of the Department of Psychology

RutgersUniversity

New Brunswick, New Jersey

April 2003
Abstract

Visual Indexing Theory assumes that events like flashes cause indexes to be automatically assigned, and that indexes stay assigned to the objects as they move around the visual field. In our study we ask whether assigned indexes can be draw away by flashing non-targets. Observers tracked 4 among 8 identical randomly-moving targets. Midway through, 4 objects flashed: two initial targets and two non-targets. We asked whether the dropped target was more often one that had flashed than one that had not, and was the erroneously substituted non-target one of the flashing distractor objects. The results showed the target lost was no more likely to be one that flashed initially, and the non-target erroneously substituted was significantly more often one that had flashed. We interpreted this to suggest that once objects were indexed, flashing them did not cause the index to be dropped, rather flashing non-targets drew indexes away from targets.

An examination of the role of exogenous events in a multiple object tracking task.

Vision involves the activity of one’s eyes as they bring one part of the visual field and then another into the foveal region, where vision is most acute. These jerky and constant movements of the eye are called saccades. During saccadic eye movements, the eyes can take in and differentiate between both static and dynamic images. Objects in the visual field that are at the center of our focus are those that we are most aware of, and that receive the greatest part of our cognitive attention. “It is widely accepted that there exists a region or locus of maximal resource allocation in visual perception – sometimes referred to as the spotlight of attention” (Pylyshyn, 1994).

The reputed psychologist and philosopher William James wrote of attention: “It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought” (James, 1890). As one can pay attention to only so many pieces of information at once, the mind and its attentional abilities must therefore have a finite limit. Thus, attention must be divided among the most important pieces of information at any given time.

In the 1950’s, Donald Broadbent proposed that attention was like a filter to prevent short-term memory from being filled up with unnecessary and irrelevant information (Brand, 1997). It was known at the time that short-term memory had a limited capacity, so a way of sifting items of little consequence, and therefore preventing a strain on one’s memory, made sense. Attention is not able to filter everything out, however. Salient information can still make itself noticed in spite of inattention, as in the case of familiar names and buzz words that pop into our awareness without our searching for them.

In the 1970’s “Focus shifted . . . to determining those conditions under which attention would operate selectively (‘selective attention’) and those in which attention would be divided (‘divided attention’)” (Brand, 1997). In dual-task testing, a subject is instructed to focus only on one task in order to test selective attention, and more than one task in order to test divided attention.

Given that there is a limit to the amount of visual information that one can keep track of at any given moment, many researchers have concluded that mental groupings are created by the cognitive system. Some have “referred to this process of keeping track of objects as ‘marking’” but, “What this terminology suggests . . . is that we have a geostable . . . icon somewhere in our heads where we can place a marker” and, “there are many reasons to refrain from hypothesizing such a metrical iconic display in the head.” (Pylyshyn, 1996). According to Pylyshyn, index would be a more suitable term than marker.

A small number (4-6) of temporary visual objects, called a visual index, can be accounted for over a period of time. An index allows the objects to be tracked while their locations and relative positions are changing, thereby making it possible for subjects to undergo visual routines (Pylyshyn, 1996). Also referred to as a FINST, visual indexes function by placing a mental “finger” on each target to be followed. The FINST does not record characteristics of the objects, but tracks them so that they can be found later, at which point they can be examined for features (Pylyshyn, 1989).

Endogenous drive and exogenous signals each have the ability to influence object tracking. In endogenous attention, the subject voluntarily controls his or her own attention (as per instruction given by the experimenter, for example). Thus, endogenous attention is goal driven with top-down processing; it first takes into account global expectations for a situation based on the subject’s past experiences and breaks them down into smaller segments. In exogenous attention, awareness is grabbed automatically by an event such as a color change or flash. “Abrupt onsets may capture attention even when the cues were not informative of target location and even when subjects were instructed to ignore them” (Chun and Wolfe, 2000). Unlike endogenous attention, exogenous attention is stimulus-driven and uses bottom-up processing; the subject takes in and compiles only the most basic slices of information, at first, and builds them into more and more complex pieces of data.

Some researchers have claimed the existence of attentional control settings (ACS) which “are conceptualized as endogenously generated rules that determine which exogenous signals will result in orienting” (Klein and Shore, 2000). If the ACS is appropriately strong it can prevent irrelevant exogenous signals from taking hold of the subject’s attention.

The above methods and theories all lend themselves to a type of experiment called Multiple Object Tracking (or MOT). MOT allows one to study the mechanisms involved in grabbing and keeping attention. The basic Multiple Object Tracking experiment (of which there are numerous variations) often involves a set of 8 circles on a computer screen. A set of circles (4-5) within this larger set is momentarily singled out as the targets (by a series of flashes or some other indicator) while the rest of the set is designated non-targets, by default. After designation, the circles move randomly about the screen for a period of about 15 seconds, and then stop where they are; the subject is then prompted to select the target circles with a mouse pointer. The basic purpose of the experiment is to measure the accuracy with which subjects are able to correctly track and distinguish the original targets from the non-targets and it has been generally found that 4 or 5 objects (Pylyshyn and Storm, 1988) moving in this manner can be tracked with relative ease.

Variations that have been performed on the basic experimental design include color, shape, size, and luminance changes, placing occluders on the screen behind which the objects could disappear temporarily, and making the non-flashed items the targets.

It has, therefore, been established that while visually tracking multiple objects, the cognitive system is capable of creating an index for a limited number of singled out objects. For our study, we asked if a second flash occurring during the motion, but without stopping or interfering with the motion in any way, could have any significant effect on the subjects’ perceptions of the index in question. Would the second set of flashed items reset or replace the original index formed, or would the strength of the initial index be strong enough to withstand flashing distractors? Would the cognitive system be more likely to lose a target that was not flashed a second time, or one that was? In conjunction, would one be more likely to pick up a non-target that was flashed halfway through the trial, or one that remained static throughout the entire length of the trial? Also, would an endogenous instruction at the beginning of a trial be able to overpower an exogenous cue occurring during the trial?

We proposed that the second bout of flashing, which simulated the first, (and which was responsible for telling the subjects which targets were to be tracked) would interfere with the subjects’ abilities to remain locked on the initial index and that the exogenous cue would overpower any endogenous instruction. Furthermore, we postulated that the two targets that were flashed in both instances would be reinforced, while those targets flashed only once would be weakened within the index, and would more likely break off and be replaced by non-targets. In addition, it seemed likely that the non-target distractors that were flashed would be most likely to be picked up by the index than the non-targets that never flashed. We proposed that although attention was assigned to the target objects (in the form of an index), flashes in the non-targets would draw attention away and re-shape the index.

Method

Subjects

The study involved 29 college students, both male and female ranging between the ages of 17 and 21, who participated in the trials in partial fulfillment of a class requirement. Subjects were naïve to visual attention and tracking experiments. They were instructed to remember the identities of four initially flashed circles that were the targets, then to ignore a second, different flashed set of four, and subsequently to click on the initial four target circles with a mouse pointer. The subjects were told to hit any one of the keyboard keys to advance to the next trial and, finally, they were also asked to focus on a target square located center-screen rather than watch the individual objects. Subjects were told they could take a break whenever they were tired, or otherwise felt a break was needed.

Apparatus

The entire experiment was performed on a Macintosh iMac with a 9 X 12 inch screen. Participants were seated in a quiet, darkened room, and used a mouse pointer to select responses and a keyboard to advance trials. The monitor was placed at eye level on a desk at a viewing distance of approximately 15-20 inches. The task was to follow targeted items as they moved on-screen, while ignoring identical distractor items and flashes that occurred. Both target and non-target objects were programmed to have independent and random motion on the screen. The experiment was programmed using the Macintosh visual tracking application Code-Warrior.

Procedure

The experiment consisted of three sets of 50 trials each, and lasted between 30 and 45 minutes. In each trial, eight stationary white circles were presented on a black screen. Subjects were instructed to focus on a small, white, 10 X 10 pixel fixation box located center-screen. This kept the subjects’ eyes from following individual objects; rather, they watched the whole screen. The box remained on-screen through out the trial.

12

34

56

(1)Eight white circles and fixation box presented on a black screen. (2)The four target circles are flashed 3 times, as indicated by the rays, for 855 ms. (The letters T for Target and N for non-target do not appear on screen, rather they are for the reader’s convenience.) (3)The objects begin moving independently and randomly. (4)Two of the initial targets and two of the non-targets flash, as indicated by the rays. (5)The objects continue to move independently and randomly. (The tick marks denote the objects flashed the second time and did not appear in the trials, rather they are for the reader’s convenience.) (6)Finally, the subjects select four objects as the ones they believe to have been in the initial index

When the eight circles, which were each 47 pixels in diameter, appeared on the screen, four of them flashed three times initially for a duration of 855 ms, thus designating them as the targets. The circles all flashed simultaneously. The other four remained static, and thus were designated as non-targets. Immediately after, all eight white circles began moving independently and randomly on the screen. After about three seconds, two of the target items and two of the non-flashed non-target items flashed on and off three times for a second time (without stopping or changing their speed or motion) for 855 ms, after which they continued moving for an additional three seconds before finally coming to rest.

Each trial was about six seconds long. The subjects were given the verbal instructions to ignore the second occurrence of the objects’ flashing, and select the initial four targets by clicking on them with the mouse pointer.

After this was done, hitting any key on the keyboard advanced the experiment to the next trial, when the subjects were ready to continue. Breaks were permitted whenever the subjects felt it was necessary.

An output file recorded the identities of the four target circles, the four distractor circles, the second set of four flashed circles, and the circles ultimately selected by the subject as the initial targets. We examined only trials of 75% accuracy or, in other words, those in which exactly one of the four targets was lost and replaced by one of the non-targets. By choosing to examine only those instances, ambiguity was avoided regarding which target was lost and which non-target was erroneously chosen to replace it. For these trials we asked whether the target dropped was more often the one that had been flashed a second time than the one that had not, and if the erroneously substituted non-target was more often one of the distractor objects that had flashed than one that had not.

Results

The results of the experiment revealed that it was the flashing non-targets that had the greatest impact on the subjects’ tendencies to switch targets with non-targets within the index; the flashing and non-flashing targets were not found to be a significant factor in target loss. The exogenous signals (flashing) were stronger than the endogenous instructions given (to ignore the second flashing).

Using only the trials where participants scored with 75% accuracy (which turned out to be 37% of the total trials of our experiment) it was found that subjects significantly picked the non-targets that flashed the second time over the non-flashed non-targets, to replace the target lost from the index. This conclusion is evident from an analysis of variance (ANOVA) [F(1,20)= 26.70, p < .000]. The mean and standard deviation for incorrectly picked non-flash items were (M = 21.29, SD = 10.27) and (M = 33.57, SD = 10.21) for incorrectly picked items that did flash.

In measuring whether a subject was more likely to lose a target that was flashed initially only or one that was flashed on both occasions, there was no significant difference found in the second ANOVA performed. [F(1,20)= 2.85, p = .107]. The mean and standard deviation for lost items that flashed a second time were (M = 29.57, SD = 8.82) and (M = 26.19, SD = 10.21) for lost target objects that did not flash the second time. Thus, once grabbed by the cognitive system, whatever disturbances and distractions that occurred to the initial targets were immaterial, as the second flash target group and second non-flash target group were both equally likely to be dropped in favor of a flashing non-target.

Discussion

In general, it was found that the double exposure to flashing targets and non-targets caused some notable disturbance in subjects’ abilities to keep track of the initial target indexes.

For the subjects, the difference between losing a target that was flashed only once or one that was flashed in both instances was negligible. Once the index was established, changes made to the features of the objects within it did not cause any of the items to become more likely to be dropped, nor did flashing again reinforce the index for those targets that were flashed twice.

More notable were the subjects’ reactions to the flashing distractors. These objects were picked up by the cognitive system and added to the index quite readily, at the expense of targets in the already established index.

Although “subjects can track multiple independent moving targets in a field of identical distractors [and] that the enhanced ability to detect changes occurring on these targets does not accrue to nontargets. . .” (Pylyshyn et al., 1994), we have found that when a change in a non-target is a significant enough occurrence, it is not only detected, but can distract the subject from the target index. Others have found that, “some early exogenously driven processes can be immune to endogenous control” (Klein and Shore, 2000).