Change Behavior Coding
This document describes the scheme I used when coding the change stories for change behavior. The goal was to learn more about how a change was informed and implemented, rather than the motivation for change or whether it was successful. The coding scheme used the 10 tags shown at the leaves of the tree below:
When coding, I started by identifying the details of the change in a given story and determining their origin. The three sources shown in the diagram were sufficient to tag the set of change stories. Authors either drew upon published materials to inform the details of their change, had a personal interaction with an individual or group, or invented the details themselves without outside influence. (If personal interaction existed, I categorized the story as PI even if other sources were also used.)
Next I looked at how the source of the details had been found. In stories where the source was personal interaction or published materialsI used a set of four categories: Authors either came across the resource while involved in an active search (AS), found it serendipitously but while browsing in a manner that indicated they were open to influence (OI), already knew about the resource when the need for it arose (PF), or stumbled across the resource completely by accident without having intended to make a change (UP). AS corresponds to the Guzdial model, where faculty search available resources to inform a change. OI was described by Sally Fincher as “putting oneself in the way” of influence — browsing the SIGCSE proceedings, for example, or reading an education blog. One isn’t actively searching for a solution to a particular pedagogical problem, but reasonably expects to learn something of interest. PF is used in cases where the author indicates that they were already familiar with the appropriate literature or knew an individual with expert knowledge when the need for change arose. (One could argue that PF is unnecessary and that we should label instead with either AS or OI depending upon how the resource had initially been found, but I found it useful to contrast pre-existing information with sources that were encountered only after the need for them was realized.) Sometimes sources were found in an “unplanned” manner — these are the cases where one has an unexpected conversation with an expert, for example, that reveals shortcomings in one’s teaching while at the same time providing the details of an alternate approach.
Wherethe details of change were locally created, I further divided the stories based on the amount of control the author had over the change. At one extreme, a change might be imposed upon the author in such detail that the author isn’t responsible for the specifics. (E.g. A department head specifies that a particular active learning activity must be integrated into a course.) I considered this to be different from the case where the author was responsible for generating the details of the new approach. For example, the instructor decides to alter the layout of his or her PowerPoint slides in response to negative evaluations, or a department head requests the use of active learning but leaves the implementation details up to the instructor. (The BA label stands for “By Author”.)
Clarifications and Tips
Make sure you’re focusing on the details of the change, and not the motivation. An author might describe a conversation they had with a colleague or student that made them realize the need for change, but unless that individual was passing along specific details on a revised pedagogical approach I didn’t consider it to be PI. This also makes it unlikely that students will play the PI role, as they’re typically not experienced enough to be handing out useful details on pedagogical approaches.
The PI label is appropriate for any sort of personal interaction, even if it’s mediated by technology. The key attribute, for me, was the expectation that the author could ask questions and expect a personal response. This is true of an email exchange, for example, but I’d also consider attending a keynote address as PI since the instructor could potentially ask a question and receive a personal response. (I don’t think it ever came up, but I’d consider viewing a video recording of a keynote as being in the “published materials” category because there’s no possibility of interaction with the expert.)
In general, if someone takes the initiative to attend a conference or workshop, I consider them to be open to influence (OI).
Several of the stories describe changes that the author instigated in the practice of someone else. In coding these stories, I coded from the perspective of the instructor actually making the change rather than from the author of the story. For example, if an author describes being invited to sit in on a colleague’s lecture and then suggests pedagogical changes that are later adopted, I would code it as PIOI — PI because the instructor making the change received the details of the improved technique via personal interaction (with the story’s author), and OI because one can reasonably expect to be influenced if one invites a colleague to sit in on a lecture.
Limitations
Stories lacking in detail can be tagged as LCBA (change created by the author without outside influence) when additional detail might have placed them in other categories. For example, an instructor might simply say that they integrated active learning activities in response to student evaluations, without mentioning when or where they heard about active learning. One can’t know whether this belongs in the PI or PM top-level categories without more detail, or if they invented the approach themselves. If this coding scheme is used to tag stories from a follow-up study, we’ll want to ensure that participants are asked specifically about their sources.