Information Product Evaluation for KFS User Guide 1.0
Overview
Information Product Evaluation of the KFS User Guide
The Society for Technical Communication provided an expert analysis of the KFS User Guide v.1.0 during the Technical Communications Summit in May, 2007. These are the workshop results.
Contents
Executive Summary
Positive Feedback
Opportunities for Improvement
Documentation-Related
General or System-Related Feedback
Transcription of Recorded Analysis and Notes
More from STC:
Information Provided to Expert Evaluator
Your name
Your job title
Years of experience in technical writing
What is the format of your information product sample?
(Printed document, online document, online help)
Title of information product
Purpose of information product
Did you perform a task analysis?
An audience analysis?
Define the primary audience.
Define the secondary audience.
Identify your information sources (technical experts, users, publications, etc.).
How much time did you have to develop the information product?
How early in project did you become involved?
What type of feedback have you received from your audience?
What budget constraints did you have for the document?
Be prepared to explain the document’s purpose, its audience, and any constraints you had developing it.
30-Minute Expert Analysis Criteria & Questions
Audience
Who is the audience?
Are there several audiences for this material?
What are the implications for information organization?
Organization
What information does the sample include?
How have sections been divided?
How is information arranged within each section?
Page Layout Evaluation
Is it easy to locate information?
Is it easy to find your place again after you leave your current location?
Is there sufficient use of white space and chunking of information?
Is it easy to see different levels of information?
Writing Style Evaluation
Has the writer used language appropriate to the audience?
Does the text help the audience see the structure of the information?
Is the text easy to read and understand?
Graphics Evaluation
Are there enough graphics?
Are the graphics appropriate?
Do the graphics communicate their messages clearly?
Are the graphics positioned properly?
Appendix A: Announcement Invitation
Appendix B: Analysis Confirmation
Appendix C: IP Evaluation Form
Executive Summary
Overall, the user guide was referred to as a “very good piece of work.”
Positive Feedback
- System: a very thoroughly and thoughtfully designed system
- Descriptions: clearly-written
- Procedures: excellent job, follows standard practice
- Page Layout: good, well-segmented, it’s easy to find things on the page
- Writing Style: appropriate, clear
Opportunities for Improvement
In order to increase chances of winning an international competition, the following will need to be addressed:
Documentation-Related
- Page Numbering: title page should be considered “i”
- Table of Contents: too long, needs summary toc first
- Index: needs one, even in preliminary drafts
- Glossary: needs one, even in preliminary drafts
- Organization: sort common functions by process
- Usability: add more high-level process flow diagram graphics at beginning of guide that depict overall flows, with references to the more detailed ones in later sections.
- Screen Capture Graphics: ensure on-screen text is large enough to be readable when possible
- Swim Lane Charts: rotate left-column text display to read horizontally and match rest of text.
- Authoring Tool: use something more than just MS Word for such a large publication
- Figures: needs a List of Figures
General or System-Related Feedback
- Nomenclature: creative, novelty terms like “nervous” system and “scrubber” should be replaced with more standard, generally-accepted terms to avoid misinterpretation and increase user confidence
- Auditing Information/Reliability: Anything you can do to explain how the system or the process applies standard controls (auditing) would be very useful in demonstrating its adequacy as a reliable financial system. That could be a separate document for controllers, but it’s very useful for auditors.
Transcription of Recorded Analysis and Notes
The evaluator allowed me to record this workshop session for a podcast. Refer to Confluence for the recorded audio file in .mp3 format. -SC
Hi, this is Annette Riley and this is an evaluation session of the Kuali Financial Systems User Guide 1.0 at the STC Conference. The views on the evaluation don’t represent STC’s or my employers’ and are individual opinions purely based on an initial look and not a long-time understanding.
OK, in the interest of time, after just a very few minutes of looking at the document, if for example, you entered it in an STC competition, a judge might spend anywhere from 20 minutes to two hours going through at least one chapter of it, and a good part of the book in detail before giving a full evaluation.
It looks like a very thoroughly and thoughtfully designed system that you have documented in some detail with attention to covering all the functions.
The manual is laid out so that it’s easy to find things on the page, and the descriptions are generally clearly-written descriptions of what’s going on to give some understanding of the data entry required to maintain a detailed financial system for a university.
The things I’m going to suggest are suggestions to make what’s a very good piece of work perhaps even more useful for your intended audience. It has a very long table of contents, running over 9 pages, by the way, it’s customary for the title page to be considered page “i” instead of page “iii”. What might be useful for a book that has a table of contents of this length, is to give a summary table of contents on the first page, and then follow it with the detailed table of contents.
It’s a lot to scan through at the front. At this point the table of contents is kind of replacing the index, because there is no index yet. Me: “one is planned.” That’s a big job, but a book of this size with this much detail, especially when there’s several places where the same concept can appear as you work your way through the financial process, it’s really very valuable to have that and to do it thoroughly.
Is this available in electronic format with topic search capability? Me: “yes – pdf with electronic bookmarks and cross-references”
The title, I don’t know if you have much control over the terminology used here, but nervous system could be misinterpreted, people don’t like to be nervous about their financial system, so you might want to think of a better word for that concept.
What you might call it is Administrative or Security functions.
I think most users will already be familiar with financial procedures, but you might consider having a glossary if there’s anything in it unusual or unique here.
It really is easier to compile it as you go, bearing in mind there will be changes along the way, but looking for the key concepts or unusual terms and getting that set from the beginning will save time as compared to later.
Scrubber-this is not standard financial terminology, and another term that might be misinterpreted, people want to make sure their money is not laundered or their transactions washed out. Whenever in doubt, if you stick to generally accepted accounting principles and related terms, because people are really not looking for novelty in a finance system, albeit less creative.
Will there be a conversion manual?
Alphabetic order is appropriate for a reference guide, which is about the clearest way to search for a function, but another way if this were more of a training guide would be to sort the common functions by process, and it would be really helpful to have a process flow for standard accounting clerk or controller processes when using this system.
You might consider on your swim lane charts, the graphics are very small. The print quality is excellent, but anything that is smaller than the equivalent of 8pt arial, but in future versions you’ll decide whether you want that many screen shots in there or not. Returning to the swim lane charts, consider rotating the swim lane chart user name text so that everything can be read from one direction.
Did you use Framemaker?
The cross-references throughout the book appear to be complete and accurate, which takes a tremendous amount of attention to keep all that straight when using Microsoft Word.
I’m sure you’ve thought about the possibility of using a system that’s more friendly to the production of large documents like this. That’s a very difficult production process, and as you continue to add books, you’ll think about more advanced ways of handling that.
You seem to have done a good job of keeping those crossreferences straight.
You pointed out that there are swim lane process flows for the individual transactions, but having them for overall flows would give a clear idea
At the least consider separating out overall flows that show people where to find the functions for their specific area.
I think you’ve done an excellent job with this. It follows standard practice for procedures but there are a lot of step-by-step task flow, but in the long run people do very little hands-on work once it’s set up, with inquiry and reports, which is reasonable.
Another thing you might think of making as a separate process flow is the budget and a good explanation of how to handle encumbered accounts which is a big deal for universities.
It appears to cover the material that would be needed by the intended audience, however the detail sections are arranged alphabetically for reference, and we discussed the advantages of giving overview of process flow, and some idea of a task organization.
Page layout is good, material is well-segmented, figures are numbered and labeled, but it needs a List of Figures.
Writing style is appropriate, appears to be clear, uses numbered steps for procedures. Stick with standard accounting terminology.
Number of graphics are good for before users are familiar with the system, but undoubtedly may need to be reduced for the final publication, knowing customizations by implementation will occur that affect graphics.
Anything you can do to explain how the system or the process applies standard controls (auditing) would be very useful in demonstrating its adequacy as a reliable financial system. That could be a separate document, but it’s very useful for auditors.
About the Evaluator
Contact: , +1 301 214 3056
Annette Riley is a past president of the Society for Technical Communication who specializes in international standards for documentation. For her employer, she is the System Engineering Manager for HR, Payroll & Learning Systems. She also works with financial systems as a trustee of her church. Annette has been a judge in international technical publications competitions for 15 years.
More about Annette from STC:
Annette is the Standards Council manager. An STC Fellow, she is a member of the Washington, D.C. Chapter STC and the Management, Information Design and Architecture, Policies and Procedures, Single Sourcing, and Usability and User Experience SIGs. Annette is a past president of STC and recipient of the president's award and distinguished chapter service award. Other STC roles included director-sponsor for Region 2, INTECOM representative, head of the bylaws committee, and frequent judge in the international technical publications and online competitions. A regular presenter at STC's annual conference, she was program manager and stem manager for annual conferences. In the Washington, D.C. Chapter STC, she was president, treasurer, founder of the online competition, and most recently manager of the technical publications competition.
Annette is a senior manager of systems engineering at Lockheed Martin, where she began work in 1983. Her management experience includes systems integration programs, proposals, client-server operations, and documentation. Her work at Lockheed Martin began in documentation and training. She was the working group manager for IEEE 1063, Standard for software user documentation, and editor of ISO/IEC 15289, Systems and software engineering, Content of systems and software life cycle process information products (documentation). She is currently the editor for new standards project ISO 24765, Software and systems engineering vocabulary, and co-editor of ISO 26514, User documentation for designers and developers.
Information Provided to Expert Evaluator
Your name
Scott Cooley
Your job title
Lead
Years of experience in technical writing
10+
What is the format of your information product sample?
(Printed document, online document, online help)
printed document
Title of information product
Kuali Financial Systems User Guide 1.0
Purpose of information product
To assist software system users with accomplishing business tasks
Did you perform a task analysis?
yes
An audience analysis?
yes
Define the primary audience.
Software end-users
Define the secondary audience.
Implementing institutions
Identify your information sources (technical experts, users, publications, etc.).
Functional specifications, business rules
How much time did you have to develop the information product?
Approx. Four months
How early in project did you become involved?
Build phase
What type of feedback have you received from your audience?
Positive, via survey results
What budget constraints did you have for the document?
Approximately $58,000.00
Be prepared to explain the document’s purpose, its audience, and any constraints you had developing it.
30-Minute Expert Analysis Criteria & Questions
Audience
Who is the audience?
Financial software system end-users, roles include Initiators, Reviewers, Approvers, Delegates and Fiscal Officers.
Are there several audiences for this material?
Yes
What are the implications for information organization?
High-level process flows are important, more are needed
Organization
What information does the sample include?
Functional areas include modules for financial transactions, general ledger, chart of accounts, reporting/decision support, and nervous system/workflow.
How have sections been divided?
Each chapter has its own “mini” table of contents, but the main table of contents arranges sections alphabetically where possible, based on software portal appearance.
How is information arranged within each section?
In general, an introductory conceptual overview paragraph (text), followed by document layout, screen shot graphics, field description tables, applicable business rules summary, and finally a numbered list of task steps (procedure) with cross-references throughout.
Page Layout Evaluation
Is it easy to locate information?
Yes
Is it easy to find your place again after you leave your current location?
Yes.
Is there sufficient use of white space and chunking of information?
Yes.
Is it easy to see different levels of information?
Yes.
Writing Style Evaluation
Has the writer used language appropriate to the audience?
Yes
Does the text help the audience see the structure of the information?
Yes.
Is the text easy to read and understand?
Yes.
Graphics Evaluation
Are there enough graphics?
Yes, for a preliminary publication, but implementing institutions may want to reduce the number of screen shots after users are trained.
Are the graphics appropriate?
Yes, but some fonts on screen shots are too small to read.
Do the graphics communicate their messages clearly?
Yes, callouts could be better, and could be used more frequently
Are the graphics positioned properly?
Yes.
Appendix A: Announcement Invitation
-----Original Message-----
From: [mailto:
Sent: Monday, April 23, 2007 4:02 PM
To:
Subject: STC Annual Conference Information Products Evaluation Workshop
Dear Conference Attendees,
STC is pleased to announce it's conference session, "Information Products Evaluation Workshop" facilitated by Dia Burroughs, Connie Kiernan, and Linda Mikkelsen.
To participate in this session, you must sign up in advance for a 30-minute analysis of your information product by an expert. Your product will be evaluated for organization, style, layout, and use of graphics. Bring your product; supply your own laptop if your product is an online sample. Note: There will not be Internet access.
To sign up you must do two things.
1) Immediately send an email to Connie Kiernan () to request an evaluation. The subject line of your email should be "Request IP Evaluation Workshop Registration." We will accept the first 50 requests and by return email notify you of your registration. You will need to bring that email to the conference to use as your "ticket" for the evaluation.
2) You will also need to complete and mail (US Postal Service) the following form ( to Dia Burroughs as addressed by May 7th. Please also bring a photocopy of this form with your sample to the workshop.
Lloyd Tucker
Director of Education
Society for Technical Communication
ph +1-703-522-4114, Ext 1904
** ph +1 571-366-1904 (direct line)
fax: +1 703-522-2075
** PLEASE NOTE THE CHANGE IN PHONE EXTENSION AND DIRECT PHONE LINE**
Appendix B: Analysis Confirmation
-----Original Message-----
From: [mailto:
Sent: Monday, April 30, 2007 12:14 PM
To:
Cc:
Subject: Re: Request IP Evaluation Workshop Registration
Congratulations! You are confirmed to receive a 30-minute analysis of your information product during the Information Products Evaluation Workshop, DD 9R, scheduled for Wednesday, May 16th, 8:30 to 10 a.m.
Your 30-minute evaluation session will be from 9:30 - 10:00 a.m. You must arrive at the session between 8:15 a.m. and 8:30 a.m.to receive a ticket for your assigned time. Because of the many people on our waiting list, if you do not pick upyour ticket by 8:30 a.m., we will give your ticket to a stand-by attendee.
To ensure your participation, you must bring all of the following items to the session:
- Acopy of this confirmation email
- Acopy of the information sheet you mailed to Dia Burroughs
- Your information product
Remember, you must supply your own laptop if your product is an online sample. Note: There will not be Internet access, andelectrical outlet access is limited, so please charge your laptop in advance.