SPECIFICATIONS FOR HUMAN FACTORS REPORTS
(better known as Superman Solves a Puzzle)
by: Dr. Paul Green
University of Michigan (Ann Arbor)
Department of Industrial & Operations Engineering (IOE)
and the Transportation Research Institute
for IOE 333, 334, 433, 436, 491
August, 2003
1
TABLE OF CONTENTS
BACKGROUND...... 1
GENERAL REQUIREMENTS...... 1
DETAILED REQUIREMENTS...... 3
Cover...... 3
Table of Contents...... 3
Introduction...... 4
Test Plan...... 5
Test Participants...... 5
Test Materials and Equipment...... 5
Test Activities and Their Sequence...... 6
Results...... 7
Conclusions...... 11
References...... 12
Appendix...... 14
FOR MORE INFORMATION...... 14
SAMPLE JOURNAL ARTICLES...... 14
SOME WEB RESOURCES...... 15
LAST STEP...... 15
1
1
BACKGROUND
Human factors/ergonomics courses expose students to human performance data, research methods, analytic procedures, and report writing. Writing reports is an activity most engineers detest but all engineers must do well if they want to succeed. This document contains complete specifications for writing human factors/ergonomics research reports. It also includes an abbreviated hypothetical report ("Superman Solves a Puzzle").
Technical reports are used by society to define knowledge and by engineers, managers, and others to make design decisions. To make decisions based on technical reports, they need to know exactly what was done and how. Decision makers need to know what was investigated. They also need to be able to judge if the method was appropriate for their application of interest, and if not, how the data might be adjusted to be useful. Engineers and managers also need to know the complete results and any recommendations from the report authors.
Students still ask, "Good grief, why are you using this detailed format?" In the past, when what was desired was left unsaid, students did not know what to include in their reports and frequently omitted necessary information. Neither students nor faculty members were happy with that arrangement. These specifications have made it easier for everyone. Remember, a Pascal program will bomb if a semicolon is missing. If that level of perfection is required to communicate with computers, it should be reasonable to expect similar precision in communicating with people. The difference between good and poor engineering is in the details.
While other courses may ask for reports that differ superficially, the essence of how a report is written will not differ. Further, life after the University of Michigan will not involve taking tests, but there will be many reports to write, for which this document is a guide.
Read this document carefully before starting the first few reports you write. After the report is written, use this document as a checklist.
GENERAL REQUIREMENTS
1. All human factors/ergonomics reports will follow this format unless otherwise stated. If there are conflicts between the requirements in Superman and those you may seen elsewhere (in another class, in a book, etc.), the requirements of Superman take precedence for this class.
- THE GOAL IS TO PROVIDE ENOUGH INFORMATION IN THE REPORT SO THAT A REASONABLY KNOWLEDGEABLE PERSON COULD DUPLICATE YOUR METHODS AND RESULTS WITHIN THE LIMITS OF STATISTICAL ERROR.
(Scientists refer to this as "publicly replicable.") This includes engineers without specific training in human factors and no knowledge of the particular experiment. (The instructor is not the intended audience.) If in doubt about how to present something and specific instructions do not appear in this document or the course pack, rely on this general rule for guidance. Think about the decisions that might be made based on the report and what the reader should be told.
3. As you consider how to present information, bear in mind that “user friendliness/ reader friendliness” is important.
4. The entire report will be written in English in a sentence format. Each sentence should contain a subject, verb, and should be a complete thought. Ideas should occur in a logical sequence and be connected. Use plain, clear writing. "Flowery" language has no place in technical reports.
5. Since the report describes a completed experiment, use the past tense (for example, "Participants responded by...").
6. Write in the third person (he, she, they) not the first person (I, we). For example use - "Subjects 1 and 2 made fewer errors" - not - "We made fewer errors."
7. Make sure the pages of the report are securely fastened together. Pages from reports held together with paper clips or stuffed in folder pockets are easily lost (or recycled).
8. Said the king to the queen, "If thou knowest not each page by name, number thy pages." The cover does not get a number but pages in the appendix are numbered and are included in the total page count.
9. Consistently use one set of units, preferably metric (kilograms, meters, etc.) not English units (pounds, feet, inches), as the primary measurement system. To help the reader unfamiliar with the metric system, English units may also be shown in parentheses. If the equipment presented the data in English units, then provide the metric units in parentheses.
10. When referring to numbers, the accepted rule is to express them as words if they are the first word of a sentence (since digits cannot be capitalized). Elsewhere, use digits if any value in the sentence is 10 or more, or if more than 3 values are in the sentence. Personally, I always use digits except for the first word of a sentence because the text is easier for non-native English readers to understand. Choose either way, but be consistent.
11. Make reports faxable. Do not use color or gray scale (or gray backgrounds on figures). For some fax machines gray may become black or white, resulting in no contrast with other graphical or text elements.
12. The Engineering College assumes that students have learned how to use Word, Excel, PowerPoint, and a drawing application in high school (or before). If you need a demo of a drawing application, contact your instructor. If you do not know how to use Word, arrange to meet with either your instructor (along with any other students who may lack such knowledge) well before the first assignment is due.
13. Twelve point type is preferred for the body of the report, but 10 point might be acceptable in some cases. Larger sizes may be desired for the cover. Finally, be sure the text in figures and tables is big enough to be easy to read.
DETAILED REQUIREMENTS
:
1. Every laboratory report will have a cover. Use laser printer paper for the cover, not cardboard or plastic. On the cover will appear:
• the experiment title
Provide a title that is specific, complete, and brief. (For example use "High Temperatures and Short Term Memory Loss" and not "Heat" or "The Effects of Artificially Induced High Temperature on Recall in a Brown-Peterson Task in a Laboratory Setting." The best title, the title you should use, may not be the title given in the course pack. When in doubt, select titles that are specific over those that are brief. Imagine the reader is viewing a list of titles and sees yours. Can they determine, using only the tile, if your report pertains to the question they are trying to answer?
Experience has shown that creating the title should occur after the report is basically done. Consider the following approach.
a. The authors should brainstorm (that means no criticism) to identify key words that should appear in the title. The conclusions is a good source for such words. Think about the audience. Is the audience primarily researchers (interested in methods), practitioners (interested in conclusions), or a mixture of both?
b. From the list of potential key words, identify those that are most important.
c. Construct several possible titles using the key words and select the best one. Optional structures include asking a question ("How does age ..."), giving an answer, describing the factors examined ("Effects of ..."), or two phrases separated by a colon. Pay special attention to the first word (often used to search alphabetized lists), what happens if the title is truncated, and the use of obscure words in the title.
• the author(s) - the name of student or students who wrote it
• the instructor's name
• the course title, section number (including the University and Department name
...do not abbreviate), and the meeting day and time (for courses with multiple
sections)
• the date submitted
The section information is strictly for the grader's and instructor's convenience. More than once students have "lost" reports (for example, at the library), but, because the cover page was complete, their reports managed to wander to the appropriate faculty office.
:
2. Every laboratory report more than eight pages long (not counting the cover) will have a Table of Contents. It will be the first page after the cover page. On it will appear the words "Table of Contents" along with all of the section and subsection names and the pages where they start. For the convenience of the grader, put the initials of the authors of each section named in the Table of Contents. Do not include Figure or Table numbers as Table of Contents entries.
:
3. Every laboratory report will have four sections entitled "Introduction," "Test Plan," "Results," and "Conclusions," and may contain two sections entitled "References" and "Appendix" in that order.
4.The Introduction must describe (a) the problem, (b) why the problem is important, (c) the purpose or object of the study (the questions addressed) , and (d) review previous research. Be specific. (For students in IOE 333 and 334, reviewing previous research is not required.) Often example applications strengthen your arguments. The potential benefits of the research should also be mentioned. Remember, you are writing for a professional audience, so the objectives are the scientific objectives (e.g., "Are the lighting levels ... adequate?"), not the expected learning/educational objectives (e.g., learn how to use light meters). Stating the objectives as who, what, when, where, why, or how questions is essential. The research objectives, the questions to be addressed, may be explicitly stated in the assignment description. At other times, an assignment may only generally describe the questions, in which case refinement of the questions is necessary or because of the way the experiment was executed, some modification of the stated question or questions is needed. Often, after carrying out an experiment, something may be learned regarding a question that was not part of the original questions to answer (for example, how a test protocol might be improved). Adding those questions to the introduction is appropriate.
An example introduction section follows:
INTRODUCTION
Manufacturers are constantly seeking ways to reduce costs so as to remain in business. A significant fraction of those costs, especially for assembly operations, are related to labor, in particular the cost of hiring employees and employee productivity once hired. Hiring decisions are often made based on selection tests, tests that must be equitable to meet legal requirements and reliable for their intended purpose. To date, there have been few selection tests whose scores have been highly correlated with on-the-job evaluations (Sanders and McCormick, 1987). Given that it costs $10,000 to train each new worker, effective worker selection can lead to significant cost reductions for an employer.
Because human tests can be costly, Smart and Not (1971) have proposed using animals of different intelligence levels instead of people to evaluate the validity of selection tests. It is suspected, however, that differences between people and animals may decrease with practice. Others have proposed that it would be informative to test those who are allegedly very intelligent (Successful Superheros, 1968; Smith, Smith, Smith, and Smyth, 1991). In utilizing them as subjects, care must be taken to avoid their use of special powers (Pittenger, 1983).
This report examines several new tests that may overcome those limitations. Three questions were examined:
Are the overall results for animals, regular employees, and superheros the same?
Are the results for individuals consistent from trial to trial?
Do the test scores correlate with on-the-job performance?
A good report will have a strong connection between (1) the questions or issues raised in the introduction, (2) the test method described, (3) the results addressing those questions, and (4) the topics in the conclusions section.
::
- The Test Plan section should describe how the experiment was conducted. The Test Plan section will consist of 3 subsections: "Test Participants," "Test Materials and Equipment," and "Test Activities and Their Sequence." (Most professional journals use the terms: "methods," "subjects," "apparatus," and "procedure" for these sections. The substituted terms should be easier for students to understand.) Within the Test Plan section, choose an order for these three subsections that facilitates the flow of ideas.
4a. The Test Participants subsection will state who participated in the experiment, how many subjects there were, and how they were selected (volunteers? paid?). Selection procedures may relate to the motivation of subjects, which in turn could affect the quality of the data collected. Major demographic characteristics such as age and sex and always included and key physical descriptions (e.g., handedness) and incapacities (e.g., visual defects, etc.), may be included. If a human characteristic might have an effect on the data, then mentioned it. So, for example, if the experiment concerned sensitivity to various odors (a smelly study?), you probably would not mention how many participants were left-handed (unless they responded by pressing buttons) but you would verify that participants did not have olefactory disfunctions. Use text, not tables, to describe the participants. Except where participants are well known and what they are known for is important to the experiment (e.g., a study of the throwing biomechanics of baseball players), refer to participants by number (preferred) or by using their initials, not their names to assure privacy. If there are more than four subjects, consider using ranges and means to describe them instead of listing characteristics for each subject.
::
4b. The Test Materials and Equipment subsection will describe all of the materials used. In brief, if someone was going to order, build, or reconstruct the materials and equipment, what would they need to know? How can graphics be used? Reportable items include the size, color, contrast, and luminance of materials on slides or in test booklets and the frequency/intensity composition of auditory stimuli. If complex visual stimuli are used, examples should be included as figures. Information about stimuli (for example, font) is more important than equipment specifics (for example, which computer was used). The materials are more important than the equipment in most cases.
How the test hardware was arranged and connected should also be identified (possibly using a figure). Sometimes a logical arrangement is sufficient, but if viewing or listening distance is an issue, then a dimensioned drawing is necessary.
Also mention the software used and provide source information. If there is a user's manual for the package, give a citation for it. Use paragraphs, not an outline. Be sure to identify the manufacturer, model and/or model number, name, size, and arrangement of each piece of equipment. Since all equipment having the same model number (or catalog number) is assumed identical, do not include serial numbers. A serial number is generally not meaningful to others. Depending on the situation, the serial number may either uniquely identify that item (every item made by that company has a unique serial number) or do so only in combination with the model number. So a Sears model xyz 8-inch circular saw, serial number 125 may be the 125th copy of that saw. Think about where you might put some of the details on a figure to shorten the explanatory text. In general, if you get the same model number item (or a similar item), you should be able to satisfy the public replicability requirement.
::
4c. The Test Activities and Their Sequence subsection should summarize each step of the test. It should describe the experimental tasks (including stimulus duration, intertrial intervals, the nature of feedback to participants, etc.), the exact instructions to participants for key phrases (shown in quotes), the order of experimental conditions (e.g., which trial blocks occurred first, how many trials were in each block, etc.) and the method of randomization. One strategy for keeping this section short is to put some of the details describing each test block in a table. Where a standard procedure was followed (e.g., a manufacturer's calibration instructions), summarize it. Please do not rehash handouts or manuals or describe the process by which the protocol was developed (First, we brainstormed about the problem. Then...) in any detail. You may find it helpful to include critical phrasing given to subjects in quotes.