DRAFT 3/15/03

DRAFT (3/15/03)

Judy Arzt

Session Title: “Electronic Portfolios’ Transformative Effects on Assessment”

Panel Title: Technology Transforms Writing Assessment,

College Composition and Communication Conference, 2003, March 21, 200, New York City

Electronic Portfolios’ Transformative Effects on Assessment

Statement of Issue

Two divergent trends have evolved in the linked fields of writing assessment and electronic media, both using the umbrella term electronic portfolios. One is the assessment driven and seeks to explore ways in which new technologies can support large-scale portfolio programs. The other is classroom-based and centers on students creating websites that need to be assessed. These two trends look at the terms “assessment” and “electronic media” from opposite vantage points. Perhaps the trends could be thought of as two horizontal lines intersecting at a point, and although they share some vocabulary, they have distinct characteristics. One effort seeks to use technology to fine-tune an assessment program—to make it more manageable and user-friendly—whereas the other is THE technology in need of an appropriate assessment. The short history of web authoring in the curriculum coupled with the evolving technology asks us to redefine constantly what makes a good website.

Ergo, one movement aims to solve an assessment problem—storage and access to a large body of knowledge—while the other searches for a vocabulary to describe and thereby assess students’ accomplishments. One is a top-down approach, a solution to an administrative problem, and the other is grassroots, emanating from classroom needs. How these fields merge, as well as diverge, demands scrutiny as both are becoming increasingly present on our campuses.

Debate over the Term E-Portfolios

The distinction between the use of technology to serve assessment and the use of technology for students to create products has led to some debate, including what the very term “electronic portfolios” means. The TechRhet and Writing Programs Administrators listservs recently saw some robust discussion on this topic. One discussion thread unfolded after Trent Batson posted on TechRhet a link to his article “The Electronic Portfolio Boom: What’s It All About,” where he states the term “electronic portfolios” should be used only to refer to large-scale, database-driven portfolio models. He calls portfolios of this kind “dynamic,” refers to the kind that students create in courses as “static,” and suggests the course kind be called “webfolios.”

Batson’s article, although helpful for providing a framework for distinguishing between the two kinds of portfolios, upset some who saw their students’ course portfolios as “dynamic.” Steven Krause writes of Batson’s article, “It was a good introduction and interesting … [b]ut there’s one part of it I don’t know if I get or agree with,” and he quotes:

Since the mid-90s, the term “eportfolios” or “electronic portfolios” has been used to describe collections of student work at a Web site. Within the field of composition studies, the term “Webfolio” has also been used. In this article, we are using the current, general meaning of the term, which is a dynamic Web site that interfaces with a database of student work artifacts. Webfolios are static web sites where functionality derives from HTML links. “E-portfolios” therefore now refers to database-driven, dynamic Web sites, not static, HTML-driven sites.

Krause finds the distinction between “dynamic” and “static” misleading, and Rich Rice adds Batson’s distinction between the two kinds of portfolios is “not healthy.” Bill Condon chimes in that historic precedent already has defined a portfolio as involving “collection, reflection, and … selection… and there are other distinctions but none that preclude putting an electronic portfolio (eportfolio) into HTML,” plus “changing the definitions of things that are already fully and firmly defined [is] like playing Tweedle Dum and Tweedee Dee.”

Lee Honeycutt writes: “Webportfolios tend to support greater student ownership of the process and can be much more creative than what you might see on a template-driven system. But they also have the potential to be incredibly unorganized and … individualistic to support matrix competency program assessment.” To clarify, Honeycutt offers the example of a departmental assessment of students’ portfolios across fifteen competencies, remarking: “I don’t know how faculty assessors could go about rating, much less, finding, examples of these [competencies] in a batch of Webfolios that were all over the map in terms of their site architecture and labeling systems.” Honeycutt’s use of the term “webportfolios” suggests he accepts Batson’s nomenclature.

Kathleen Yancey, Carl Whithaus, and Steven Krause assert that the websites that students create in courses are more “dynamic” than database-driven portfolios. Krause proffers: “If you make students responsible for their own websites—even simple ones—they learn another set of skills—I see it as more ‘dynamic’ than [the] database way” even if these ventures have the “downside” of “tremendous inconsistency” that make them more difficult to assess. Whithaus, speaking of first-year composition, stresses the idea that students need exposure to electronic tools for creating hypermedia rather than experience dumping work into an “IT [information technology] designed system.” Students, he seems to think, should do more than fill in boxes; they should practice with real composing tools and not just “interface/database programs.” Yancey offers that although a number of institutions are opting for database-web interfaces, student-created websites provide a richer learning environment. She enumerates:

·  form and content are related, and that if students are to exert agency, they need to have some control over the form

·  the interface and navigation design are a major part of self-representation (and thus self-construction)

·  hyperlinking is related to cognitive thinking

In fact, Yancey suggests that the database model might more aptly deserve the appellation “static.” In referring to this model, she remarks, “…much of what is being called a portfolio is really simply a ‘capture’ of information—no reflection, no student choice, but indeed a lot of stuff. That, to me, seems totally “static.”

Yancey’s comments on the TechRhet listserv echo her earlier comments on the Writing Program Administrators list in December. Returning from the American Association of Higher Education e-portfolio conference held in November, she reports on WPA the emerging use of large-scale database models: These folios as places where students “drag and drop” work into a “template,”; “[i]t is more a response … to a set of prompts than a creation or product or a composition.” Having “reservations” about the value of these experiments, she pines for student-designed portfolios even if that means variations across departments and programs. These portfolios, as she sees it, are “better for student learning” and “more interesting to read.” In short, Yancey advices “more composition people” to stay “involved” in the conversation about e-portfolios on their campuses.

A Sampling of Large-Scale Electronic Portfolios

A review of some of the large-scale models reveals commonalities and a shared vocabulary. One model used by the English Department at Georgia State University in the Rhetoric and Composition Program requires students concentrating in Professional and Technical Writing to complete e-portfolios as an exit requirement. George Pullman notes that “an electronically stored version of the portfolio” does not require students “to know anything about making web pages” and can be made using “Word documents, WordPerfect documents, Rich Text Formatted documents, PDfs, even images like GIFs and JPGs.” He explains that the “electronic portfolio has many advantages over a regular paper portfolio.” Students can quickly and easily revise “any of the documents … from nearly any computer connected to the Internet” and can show them to anyone “without having to print copies or pay postage.” In summary, an e-portfolio is “like a resume” that offers “proof” of a student’s “skill set,” Pullman adds.

In the Alverno College model, which is touted as the “first-of-its-kind, web-based system,” a student “anyplace, anytime” can “follow her learning process throughout her years of study.” This portfolio system, called Diagnostic Digital Portfolio, enables a student to receive feedback from faulty, peers and others, and, as claimed on Alverno’s website, helps a student “look for patterns in her academic work so she can take more control of her development and become a more autonomous learner.” A screen capture at the site shows the gateway into a student’s portfolio as a table. A column on the left lists competencies (e.g., communication, analysis, problem-solving, etc.), and a column at top gives levels: level 1, level 2, level 3, and level 4. Remaining cells on the table have course prefixes and numbers. The implication seems to be that if one clicks in a cell, a portal to viewable evidence of a student’s competency at the level will appear. Although this system appears to manage a large collection of work well and provides a way for others to critique a student’s work, it is difficult to tell as a visitor to the site if the technology engenders student invention and learning or merely accomplishes making competencies observable. In fact, perhaps the only experience students have using hypertext in this environment is filling out forms and clicking on buttons to access feedback. The advantages seem much like those of working in a BlackBoard or WebCT except now the technology is customized to fit the institution’s specific assessment program.

The financial expense of Alverno’s startup of the Diagnostic Digital Portfolio Program is suggested by the list of sponsors who funded the inception of the project, among them the W.K. Kellogg Foundation, the Pew Charitable Trusts, and a U.S. Department of Education Title III Grant. It would appear that this kind of investment bodes that the model will be transportable to other campuses.

A similar model used at the University of Washington provides a space for students to collect and store “work that they have done as part of their academic career.” The website describing the project notes the e-portfolio system enables students to “interact with instructors and advisers” and “present …work on the Web for a variety of audiences.” A screen capture pictures “the main interface that students … use to interact with the tool.” The electronic portfolio is touted as a means “to collect” and “annotate” a variety of “digital ‘artifacts’” in an “organized” way by using the navigational features built into the program.

Stanford University uses a similar system called “E-Folios,” short for Electronic Learning Portfolios. The literature describing the program states: “E-folios are ubiquitous, portable, electronic knowledge databases that are private, personalized and sharable, and are easily accessible via the web.” Screen previews again reveal a form in which students drop work. Although the screens on the Stanford, U. of Washington and Alveno sites look slightly different, the programs seem to serve the same basic purpose: provide a place for students to drop work and access feedback. These programs make it possible for faculty, administrators, and staff to view student work from virtually any computer and provide immediate feedback. The programs also seem to help with monitoring student progress toward achieving specific outcomes and appear similar to those that are being designed on the state level to help with assessing skills for teacher certification programs wherein a range of competencies must be demonstrated.

As we well know, large databases help us manage large amounts of information, but how well do these programs engender student creativity? To what extent do these programs promote creativity and help students learn aside from the guided feedback provided? How much invention and play students have in these environments is not readily apparent by viewing the websites with visitor status. Furthermore, as many of these database e-portfolios are password driven, they do not allow for the kind of accessibility that occurs when students mount their work in a generic way on the World Wide Web. So, herein is another distinction between the two divergent trends in the field of assessment and technology that fall under the umbrella term e-portfolios. One operates in an environment that keeps snoopers out, and the other freely permits anyone in.

Databases versus HTML Sites: What about the Waters between the Two Rivers?

With these two distinct movements unfolding on our college campuses, we have to wonder about students getting caught in the middle. Will students be able to drop an existing website into a newly implemented database portfolio? How will assessors using a grid respond to student creativity? Will students be able to readily extract work created in a database websites and export it to html websites? In effect, how well do the two models co-exist, and can they be successfully wedded? Institutions engaged in both enterprises will provide fertile grounds for answering these questions, but students will also have to be engaged in the conversations to see how they are affected.

If a February 1, 2002 Chronicle of Higher Education article prediction proves true, we will increasingly see database-driven portfolios models on our campuses. John Ittelson, professor of communication science and technology at California State University-Monterey Bay, director of the California State Idea Lab, and lead figure in e-portfolios nationally, notes in Chronicle article: “‘A few years ago the question was, Was every student going to have an e-mail account…[ now]...everyone is going to want to have some kind of Web space that represents their [sic] learning and their assessment’” (as cited in Young). This statement about exponential rate of growth underscores the fact that students do and will continue to create websites regardless of whether the technology is part of the college curriculum.

Those of us at institutions using, or about to use, both flavors of electronic portfolios should be thinking about the interconnectedness of a student’s education. Although both forms of portfolio development share the common principle of being a way for students to demonstrate learning and academic accomplishments in an integrated, seamless environment viewable to many in many locations, will students be able to synthesize these learning experiences and make connections? Perhaps their educations will be enriched by participating in both. But like all other college experiences, we still need to wonder about how all of the pieces fit together to form an integrated, holistic experience as opposed to a frustrating, fragmented education. Further, will students who have labored over the creative design of a website fare well with assessment number crunchers looking to assess students’ skills across a matrix of skills and then punch results into a canned database?