Shaken and stirred: ASIS&T 2011 attendee reactions to
Shaking it up: embracing new methods for publishing, finding, discussing, and measuring our research output

Alex Garnett
University of Victoria
Electronic Textual Cultures Lab

@axfelix
Heather Piwowar
DataONE and Dryad
National Evolutionary Synthesis Center

@researchremix / Kim Holmberg
Åbo Akademi University
Åbo, Finland

@kholmber
Jason Priem
University of North Carolina
at Chapel Hill
School of Library and
Information Science

@jasonpriem / Christina K. Pikas
University of Maryland
College of Information Studies

@cpikas
Nicholas Weber
University of Illinois, U-C
Graduate School of Library & Information Science

@nniiicc

ABSTRACT

What does the Information Science community think about new, open methods for publishing, finding, discussing, and measuring our research output? This poster will summarize audience member participation and reaction to an ASIS&T 2011 panel discussing these issues.

Reaction data will consist of several Likert-scale and open-ended responses. The data will be collected only a day or two before the poster is displayed: classification and visualization will be done openly to accomplish a rapid summary of the data.

The tight timeline and attendees-as-data-source will heighten the relevance of these exploratory results. Likert-scale response distributions will be displayed in dot-plots to facilitate additional Write-On-The-Poster contributions from poster-viewers, further increasing engagement.

Through this process we hope to raise awareness of these new open methods, discuss their strengths and weaknesses for the Information Science community, experiment with new methods for face-to-face group scholarly communication, and build community.

Keywords

scholarly communication, information dissemination,
data archiving, open science, collaborative filtering,
alt-metrics.

INTRODUCTION

The ASIS&T 2011 program includes a panel session, “Shaking it up: embracing new methods for publishing, finding, discussing, and measuring our research output.”

The scholarly communication ecosystem is changing. Scholars produce and publish a wider range of products than ever before, and scholars and others increasingly interact with these diverse products in new ways within the online ecosystem.

The associated ASIS&T panel will engage the audience on these issues, soliciting audience feedback in out-of-your-seat icebreakers and feedback forms.

Audience icebreaker viewpoints

As discussed in more detail in the panel proposal, the panel session will begin with an audience-participation icebreaker. The moderator will make a statement and ask all audience members to stand in a position along an invisible line in the room that best describes their reaction to the statement: one side of the room is for those who “strongly agree” and the other side is for those who “strongly disagree,” with intermediate opinions between. Proposed statements:

  1. We should thoroughly understand the implications of new methods of scholarly communication before adopting them.
  2. The scholarly reward system rewards the wrong things.
  3. Information Science should to be more progressive than other disciplines in experimenting with new methods for scholarly communication.

Panel members will then present brief talks on four areas:

Nontraditional research products: Scholars are producing and publishing a wider range of products than ever before. Research projects produce not just journal and conference publications, but also datasets, source code, blog posts, preprints, videos, and educational resources.

Nontraditional forums for discussion: Scholars and others increasingly interact with research output in a complex online ecosystem. Research products are bookmarked in Mendeley and CiteULike and discussed in new forms of research communication like blog posts, on-article comments, Twitter, and Facebook.

Nontraditional processes for discovery: Widespread availability of these research products and discussion can inform new methods for finding diverse types of research output.

Nontraditional impact measures: Interaction with online tools can be tracked and used to inform new, faster, and broader metrics of impact and also create new areas of research and innovation.

REACTION DATA FOR POSTER

Attendee responses will contribute several types of data. Poster authors will record the overall distribution of audience members in the room for each of the three questions.

Also, audience will be given response forms encouraged to anonymously fill out a few scale and short response questions for each presentation topic. Likert scale response (on 7pt scale from strongly disagree to strongly agree)

  • Information science already does this
  • Adopting this is (or will be) important for information science
  • This is very risky
  • I want to try this out
  • I know how to do this

Short response:

  • Yeah, But….
  • I’d like to see this happen because….

POSTER ANALYSIS

Likert scale responses will be presented as graphical distributions and means. A selection of the distribution responses will be displayed prominently as dot-plot frequency distributions. Writing Implements will be available at the poster display: poster-viewers will be encouraged to contribute their own responses by adding Xs to the existing plots, as shown in Figure 1.

Figure 1: Dot-plot frequency distribution of Likert scale responses, displayed to facilitate additional contributions from the poster-viewing audience

Open-ended responses will be typed in to a shared, open document. The poster authors, hopefully contributions from some in the ASIS&T conference community and others online, will experiment with classifying and visualizing the comments. Given the short time available, the goal will be very light analysis that attempts to “show the data” transparently.

DISCUSSION

The data presented in this poster has clear limitations: it will be collected from a small number of self-selected participants. As such, it is appropriate for generating discussion and hypotheses rather than conclusions.

As Information Science scholars, new methods of publishing, finding, discussing, and measuring research influence both our research methods and also our own personal and community practices.

Through this process we hope to raise awareness of these new open methods, discuss their strengths and weaknesses for the Information Science community, experiment with new methods for face-to-face group scholarly communication, and build community.