Project proposal: Digital Reference KPIs
Digital Reference Key Performance Indicators
UNISON Reference & Information Services
Interest Group
PROJECT PLAN
PROJECT PLAN
1. PLANNING AND SCOPING
1.1 Project Description
1.2Project Drivers
1.3Outcomes and Benefits
1.4Project life cycle
2. KEY ELEMENTS
2.1Governance
2.1.1Steering Committee
2.1.2Implementation Committee
2.1.3Project Manager/Team
2.2Stakeholder/Client Management
2.3Risk Management
2.3.1Consequences and alternatives
2.4Issues Management
2.5Resource Management
2.6Timeline
2.7Status Reporting
2.8Closure of the Project
Performance Indicators for Information Services
A project proposal for the UNISON reference & Information Services Interest Group
1. PLANNING AND SCOPING
The move from traditional reference services to information desks, electronic services and virtual chat has provided the opportunity to assess the wide range of tasks performed by reference librarians. From the CAUL perspective, there is no breakdown of reference statistics, including no categorization of digital reference, therefore no analysis or history can be used to determine the trends, use and importance of digital reference. While individual institutions capture their electronic reference statistics, there are no standardised statistics recording format, making it difficult to measure and benchmark, no standardised user satisfaction surveys and no measure of staff accuracy and training methods. The University Librarians in the State of New South Wales (UNISON) have commissioned this project to investigate measures for Digital Reference Services.
Digital Reference is defined as:
Digital reference is the delivery of ‘point of need’ reference services using electronic information technologies. It may consist of synchronous or asynchronous electronic reference interactions, such aschat, co-browsing, voice over IP, SMS and emailbetween users seeking information and reference librarians. Users ask questions electronically and are answered electronically by reference librarians, regardless of geographic location(American Library Association, 2004; Saskatchewan Virtual Reference Working Group, 2001) .
Reference is defined as:
1.1 Project Description
The goal of the project is to develop 3 Key Performance Indicators for Digital Reference Services. At the present time there are no standardised measures for Digital Reference Services in Australian University Libraries. The Key Performance Indicators will provide a basis for comparison of Digital Reference Services in UNISON Libraries. The Key Performance Indicators will focus on Client Satisfaction, Staff training and Transaction activity. The ASK Model Key Performance Indicators were used as the basis for the Digital ReferenceIndicators. The Key PerformanceIndicators will be developed by the steering committee and then tested in a pilot data collection study. The final survey will be sent to all UNISON Libraries andthe findings will be collated. A final report on the project will be completed by November 2005. It is anticipated the Key Performance Indicator project will assist in the review and evaluation of Digital Reference Services.
The ASK model was used to develop the digital reference indicators below:
Key Performance Indicators(Based on the ASK Model)
ATTRIBUTES
Analytical skills / Ability to analyse the information needs of a range of users and provide solutions that meet the resource constraints of the situation / Client satisfaction survey
SUPPORT
Staffing the reference service / Level, quality and consistency of staffing that is provided to meet the information demands of the teaching, research and learning objectives of the institution. Development of staff skills through training, teamwork and continuing education / Training of staff for digital reference services
KNOWLEDGE
Matching resources to user needs / Providing access to information resources that meet the needs of the user / Completion of weekly digital reference transactions statistical survey
(CAVAL Reference Interest Group. Working Party on Performance Measures for Reference Services, 1998)
It will also build upon the report created by the CAVAL Reference Interest Group and the literature review on “Performance Indicators for Digital Research and Information Services” by the University of Newcastle(University of Newcastle Education Services, 2004).
1.2Project Drivers
• UNISONgroup has identified the need to ensure tasks performed are tasks required by Library Management and needed by Library Patrons
• The previous work by CAVAL
• Identified need for benchmarking of qualitative information
1.3Outcomes and Benefits
At the end of the project we will have:
• an agreed understanding of digital reference services
• an agreed data set of what is to be measured
• Indicators that provide benchmarking opportunities
• Indicators that provide a service measure
• Indicators that provide a method of monitoring technological change and service implication
• a set of categorical key performance indicators on:
-analytical skills of digital reference staff (Client Satisfaction Survey)
-staffing the reference services
-matching resources to user needs
1.4Project life cycle
The project life cycle is 12 months.
2. KEY ELEMENTS
2.1Governance
Project sponsor: UNISON
Champion: Felicity McGregor
2.1.1Steering Committee
Kerryn Amery
Linden Fairburn
Pam Freeland
Alison Rigby
Sally Schofield
Amanda Smithers
2.1.2Implementation Committee
Reference & Information Services Interest Group (RISIG)
2.1.3Project Manager/Team
Annette Dawes
Susan Zhang
2.2Stakeholder/Client Management
UNISON Reference & Information Services Interest Group (RISIG)
Feedback and input will be communicated by group emails and by regular steering group meetings.
2.3Risk Management
The project is to be completed in the given time frame. The Project Officer is to ensure that all participants are kept informed of timelines and that the final report is prepared in advance of need and circulated.
2.3.1Consequences and alternatives
The Risk Management Plan ensures that due consideration has been given to elements and issues which could have an adverse impact on the achievement of the project objectives. For each objective, the following elements should be considered.
Identified Risk / Risk Managemetn Strategies / ProcessThe goals of the project are not achievable
Resources are scarce
Project is too big / Ensure project is clearly defined
Ensure work flows are manageable & staff available
Ensure project is clearly defined / Refer issues to Steering Committee
Prioritise work flows and make optimal use of staff
No deviation from original scope of project except by the decision of the Steering Committee
2.4Issues Management
Issues management is related to risk management. Where concerns or issues arise throughout the life of the project, which may affect the outcomes, they should be raised initially with the Project Officer for analysis and / or referral to the Steering Committee.
An Issues Register will be maintained, and reviewed regularly. Unresolved issues which could have an impact on outcomes will need to be identified and reported via the Risk Management Plan, with accompanying strategies for management.
2.5Resource Management
Funds for the Project Officer are available and will be invoiced monthly to the UNISON account.
Project performance will be assessed towards the end of the project, as a Project Performance Review.
2.6Timeline
November, 2004 – November, 2005
November, 2004 – January, 2005
Working Group meetings to discuss project scope
February 18, 2005
Project Plan Completed and distributed to Working Group Members
March 21, 2005 – April 22, 2005
Testing of the Digital Reference Weekly Statistics Survey by the Working Group
Testing of the Staffing and Staff Training Survey by the Working Group
April, 2005
Project report and update to UNISON meeting
May 2, 2005 – June 24, 2005
Digital Reference Weekly Statistics Survey trialled by UNISON libraries
Staffing and Staff Training Survey trialled by UNISON libraries
July, 2005 – September, 2005
Data collection and compilation
Compiled data submitted to Working Group
Discussion of draft Key Performance Indicators
Key Performance Indicators prepared by Working Group
Report written and submitted to Working Group
September, 2005 –November, 2005
Wrap up and closure of the Project and the Working Group
Presentation of Key Performance Indicators to UNISON
Final Report to UNISON
2.7Status Reporting
It is anticipated that the project will take 12 months to complete. During that time monthly updates/status reports will be delivered.
Status Reports should include
- Status of the Project
Description
Milestones (significant points in the project) achieved for the last reporting period
Milestones expected for the next reporting period
- Budget or resources report
- Issues report
- General information and recommendations.
2.8Closure of the Project
Closure will include:
- Handover of the project deliverables to client/ stakeholder groups.
- Review of the project outcomes against the objectives of the project.
- Formal closure by the Steering Committee.
- Report on completion of the project.
Reference List
Alberta Library Online.Ask A Question Virtual Reference Service. Retrieved January 24, 2005, 2005, from
American Library Association. (2004). Definitions of a Reference Transaction. Retrieved January 24, 2005, 2005, from
CAVAL Reference Interest Group. Working Party on Performance Measures for Reference Services (1998). Final report. Prepared by Rosemary Cotter, David Cunnington, Eva Fisch & Barbara Paton. Bundoora, Vic: CAVAL Limited
Lipow, A. G. (2003). The virtual reference librarian's handbook. Berkley, California: Library Solutions Press.
MARS Digital Reference Guidelines Ad Hoc Committee, R. a. U. S. A. (2004, January 10, 2005). Guidelines for Implementing and Maintaining Virtual Reference Services. Retrieved January 24, 2005, 2005, from
McClure, R., Lankes, R. D., Gross, M., & Choltco-Devlin, B. (2002). Statistics, measures and quality standards for assessing digital reference library services : guidelines and procedures. New York: Information Institute of Syracuse, Syracuse University.
Saskatchewan Virtual Reference Working Group. (2001). What is virtual reference? Retrieved January 24, 2005, 2005, from
University of Newcastle Education Services. (2004). Performance Indicators for Digital Research and Information Services:A Literature Review. Newcastle.
Viles, A. (1999, October 13, 1999). The Virtual Reference Interview: Equivalencies: Discussion Group on Reference Work Report. Retrieved January 24, 2005, 2005, from
Virtual Reference Desk. (2000). Facets of Quality for Digital Reference Services
Version 4. Retrieved January 24, 2005, 2005, from
Last Updated: 16/01/2006 11:06:00 AMPage 1 of 8