Making The Case For A Black-Box Approach To Calculation

by

Mark P. Rossow

Department of Civil Engineering

Southern IllinoisUniversity Edwardsville

Edwardsville, IL62026

Abstract

A task analysis such as is used in human-factors studies is applied to develop guidelines for using scientific/graphing calculators in a manner that saves time and gives confidence that the answer calculated is correct. The principles derived from the task analysis also apply, with appropriate interpretation, to calculation programs (for example Mathcad, MATLAB and Mathematica) running on a personal computer. An example is given illustrating how excessive concern with promoting students’ understanding of a calculation procedure can lead to advocating the use of inefficient and error-prone calculation techniques. A more balanced approach to calculation is proposed wherein concern for promoting understanding is weighed against the avoidance of error and the efficiency obtainable when using a black-box approach to calculation.

I. Introduction

Present day scientific/graphing hand-held calculators are very powerful, and many articles and books are available describing how to use these calculators to solve complicated numerical problems. Little attention seems to have been paid to the topic of training students how to use these calculators most effectively when solving the relatively small numerical problems arising in typical undergraduate engineering courses (and in the everyday operation of many engineering offices). “Relatively small problems” are defined here to mean problems that do not require programming but can be solved by using features built-in to the calculator. Many instructors assume that this topic is trivial because the problems are small, and “students can easily learn how to solve these problems by themselves”. The author, however, believes that the topic is non-trivial for two reasons:

  1. The amount of student time spent on “relatively small problems” is almost certainly far greater than the time spent on the more advanced features such as programming, since the latter tend to be concentrated in a few projects during a semester while homework problems may be given at all times and in all quantitative courses. Thus the calculations are non-trivial in the sense that they occupy a significant portion of a student’s time, and any improvement in the efficiency in using these calculators will save a non-trivial amount of the student’s and, later, practicing engineer’s time.
  1. The author has observed in his classes that left to themselves students do not naturally learn to take advantage of what calculators do best and what people do best. The problem is that before students can learn how to realize the full potential of their calculators, they first must “unlearn” some things that they have been taught in their previous schooling, and “unlearning” usually requires intervention by the instructor.

This paper has two purposes, one specific and the other broad. The specific purpose is to use task analysis, such as is used in human-factors studies, to develop guidelines for using the current generation of scientific/graphing hand-held calculators in the most efficient manner to solve small problems [3,4]. The author has found these guidelines helpful for his students, and he presents them with the hope that other instructors might find them helpful for their students as well. The broad purpose of the paper is to use the guidelines to make a case for acceptance of a black-box approach to using calculation devices (not just calculators but also calculation programs running on a computer) in engineering education.

II. Task Analysis

A. Set of Tasks

Task analysis will be used to provide a logical basis for the guidelines. Many of the resulting guidelines turn out to be what simple common sense would indicate. A few of the guidelines, however, may be controversial. This point will be discussed more in the second part of the paper.

The first step in the analysis is to identify the set of tasks to be performed when a student solves a homework problem using pencil, paper, and a calculator. The student

  1. reads the problem description in the textbook
  2. thinks about how to solve the problem, perhaps referring to notes or examples in the book
  3. looks up needed equations and perhaps also data such as physical properties, standards, codes, etc., from the textbook or reference book
  4. copies equations and data from the textbook and reference to paper
  5. re-writes the equations but with numerical values substituted for some of the parameters
  6. manipulates (and re-writes further) the equations algebraically or perhaps with calculus to get the equations into a form for entry into the calculator
  7. copies equations or perhaps mathematical expressions from paper to calculator
  8. presses calculator keys to get the calculator to produce a result
  9. copies the result from the calculator to paper

B. Errors Types

Having identified the steps in the analysis process, we can next identify what can go wrong, that is, what are the possible errors. Thus if errors are broadly defined as actions that lead to incorrect results, the following types of errors may arise:

  1. Transfer (from printed problem statement to student’s mind)—misreading of the problem description or the numerical data or the units of measurement
  2. Conceptual (occurs as student thinks)—applying the wrong theory or making incorrect assumptions
  3. Transfer (from student’s mind to writing on paper)—thinking one thing but writing another, for example, intending to write a positive sign but mistakenly writing a negative, or leaving out a factor in a mathematical expression.
  4. Transfer (from reference book or formula sheet to paper)—miscopying a formula or needed data (This description is really an over-simplification, since in finding, reading, and then copying the formula and data in a reference book, the student actually has two possible transfer errors and a possible conceptual error besides. Because these errors are not directly related to interaction with the calculator, for the purposes of the present paper the errors will all be lumped into one.)
  5. Transfer (from paper to paper)—making a mistake in manipulating symbols or in making algebraic substitutions and simplifications
  6. Transfer/Keystroke (from paper to calculator)—entering in the calculator something different than is present on the paper. Telephone operators have been found to press the wrong key about 6% to 8% of the time [3]. One would expect error rates of engineers and engineering students to be higher since the telephone operators in the cited study had much more experience and specialized in pressing only a few (ten) keys.
  7. Keystroke (human operation of the calculator)—pressing the wrong keys to operate the calculator. Two things can happen: (1) the error leads to an illegal operation, and the calculator displays a message such as “Syntax error” (This type of error is annoying but relatively benign, since the only detrimental effect is that the user wastes time re-entering commands), and (2) the error does not lead to an illegal operation and the program produces a result—that unfortunately the user may not immediately spot as erroneous, especially if the error occurs in a long chain of calculation. This type of error can be insidious, since it may be hard to detect.
  8. Hardware (electronic functioning of the calculator)—making a mistake in design of the electronic circuits
  9. Software (logic controlling the electronic functioning of the calculator)—making a mistake in programming
  10. Transfer (from calculator display to paper)—writing something on paper that differs from what is shown in the display

Of the five types of errors present in the list given above: (1) transfer, (2) keystroke, (3) conceptual, (4) software, and (5) hardware, errors in hardware and in software built-in to the calculator are sufficiently rare that the possibility of their existence can be ignored. Conceptual errors do not involve use of a calculator and so will not be discussed further in this section of the paper (but will be discussed later).

Thus only transfer errors and keystroke errors remain, and a seemingly obvious strategy for reducing these types of errors is simply to reduce the opportunities for them—by making the number of transfers and keystrokes as small as possible. This strategy has the twofold attraction of (1) increasing the probability of producing the correct answer and (2) saving time.

Further thought shows that matters are not quite as simple as merely attempting to minimize the number of transfers and keystrokes. Given the many possibilities for making errors and given that human actions are a part of the calculation procedure, the potential for errors will always be present. It follows that any error-reduction strategy must provide some means for gaining assurance that the result obtained at the end of a calculation is in fact the correct result. Accordingly, the guidelines that will be given in the next section have been designed to achieve three goals:

1. reduce transfer and keystroke errors

2. give confidence that the result calculated is correct

3. achieve goals 1 and 2 as quickly as possible

III. Guidelines

Guideline 1.Enter information in the order in which it appears in the original problem formulation.

Algebra manipulations should be done by machines, not by human beings. Consider the following example. Suppose the student has applied some physical principle, for instance an equation of static equilibrium, and has written the following equation on paper:

14.78 + 5X + 6(3.42) – 3.97X + 2.23(X – 4) = 0.(1)

Based on years of training and experience in solving similar equations, the student will first re-write the equation on paper, with the terms re-arranged so that the variable X is isolated:

[14.78 + 6(3.42) + 2.23(– 4)] + X[5 – 3.97 + 2.23] = 0.(2)

Then the student will use a calculator to calculate the quantities in the square brackets and write the result on paper as

[26.38] + X[3.26] = 0.(3)

Then the student will use the calculator once more to evaluate the quotient 26.38/3.26 and will copy it from the calculator display to the paper and write

X = -26.38/3.26

= -8.09.(4)

The traditional paper-and-pencil approach just described has two drawbacks. First there are many places where transfer errors may occur: (1) in transforming Eq. 1 into Eq. 2, (2) in transferring the numbers in the square brackets in Eq. 2 to the calculator, (3) in transferring the numbers back from the calculator to paper to write Eq. 3, (4) in transferring the numbers from Eq. 3 to the calculator, and (5) in transferring the final result back to paper to write Eq. 4. The transfers are tallied in Table 1.

Transfer
/ Characters transferred / Number of characters
Source / Destination
paper / paper / 14.78 + 6(3.42) + 2.23(– 4) / 22
paper / paper / 5 – 3.97 + 2.23 / 11
paper / calculator / 14.78 + 6(3.42) + 2.23(– 4) / 22
paper / calculator / 5 – 3.97 + 2.23 / 11
calculator / paper / 26.38 / 5
calculator / paper / 3.26 / 4
paper / paper / -26.38/3.26 / 11
paper / calculator / -26.38/3.26 / 11
calculator / paper / -8.09 / 5
Total number of characters transferred =
/ 102

Table 1. Tally of Character Transfers for Traditional Paper-and-Pencil Approach

The second drawback of the traditional paper-and-pencil approach is that verifying that the work is correct is made unnecessarily difficult because many transfers were made and each transfer must be checked. If fewer transfers were used in the first place, then verification would be correspondingly simpler.

Compare the situation just described with that which prevails if Guideline 1 is followed. The student enters Eq. 1, with no changes, into the calculator and then invokes the built-in equation solving routine (the “solver”) to produce the result shown in Eq. 4. The transfers are tallied in Table 2. The number of possible transfer errors (and thus the number of transfers to check) has been reduced from 102 to 39.

Transfer / Characters transferred / Number of characters transferred

Source

/ Destination
paper / calculator / 14.78 + 5X + 6(3.42) – 3.97X + 2.23(X – 4) = 0 / 34
calculator / paper / -8.09 / 5
Total number of characters transferred =
/ 39

Table 2. Tally of Character Transfers for Calculation Based on Guideline 1

Two pedagogical observations are in order here. The first is that Guideline 1 seems, to the author anyway, so obvious as to be almost hardly worth stating. Yet of the hundreds of students that he has taught over the years since scientific/graphing calculators have become available, he has never encountered even one student who has taught himself or herself to use the solver’s symbolic-algebraic capability routinely to replace pencil-and-paper algebraic manipulations. Furthermore most students, initially at least, resist following Guideline 1 when the author introduces it in his classes. Apparently, years of previous instruction and practice beginning in the early school years and continuing into college are not discarded lightly. Instructors who want their students to take advantage of the full potential of a calculator must first convince their students to unlearn something that they have previously learned.

The second observation is that heavy reliance on a calculator may have beneficial side effects that are not widely appreciated. For example, students who make heavy use of algebraic solvers such as that used to solve Eq. 1 soon realize that the solvers are not perfect. Sometimes they will not converge or will converge to an extraneous root. Mindless pressing of calculator keys (one of every instructor’s deepest fears) will not produce the correct answer. Thus once students begin to rely heavily on the solver, they must develop the ability to estimate the value of one or more of the unknowns that are to be calculated, so that the solver will start from a point “close” to the answer. In other words, use of the solver encourages the student to develop a sense of the magnitude of the answer expected, and this is surely one aspect of what we mean when we say we want students to “understand” a physical situation described through mathematical equations.

Guideline 2. Check for errors by proofreading rather than by repeating the calculation.

If Guideline 1 has been followed, then the equation on paper can be compared character-by-character with the equation appearing in the calculator display. That is, error checking reduces to proofreading.

Error checking by proofreading can be contrasted with the technique employed by most calculator users—at least those yet to be enlightened by Guidelines 1 and 2—that might be called “error checking by majority rule.” In the latter procedure, the user of the calculator enters data, performs calculator operations and writes down the result. To check for an error, the user next tries to repeat the sequence of steps of data entry and calculator operation. If the result obtained agrees with the previous result, the user assumes that the result is correct. If the result does not agree, then the user tries again to repeat the sequence of steps. If this third result agrees with either of the two previous results, then the two identical results form a majority and are accepted as the correct result. If the third result agrees with neither of the first two, then a fourth (or fifth or sixth or … ) calculation is performed until a majority (more accurately, a plurality) is established.

The drawbacks of the majority rule are that (1) no attempt is made to locate the error— agreement of the majority may only indicate that the user has repeated the same mistake in both calculations; and (2) the total number of required keystrokes is much higher (and the probability of making an error correspondingly greater) than the number required when employing error checking by proofreading. Following Guideline 1 makes error checking by proofreading easier since, as has been previously mentioned, there are fewer numbers and equations to proofread.

The author has found that Guidelines 1 and 2 are probably the most difficult ones to get students to accept. Students view them with impatience, believing that the best way to get the answer is simply to start calculating and then apply the majority rule. They generally resist the argument that applying the majority rule will almost always take them more rather than less time to obtain the correct answer and to have assurance that it is correct.

Guideline 3. Data that appear in several equations should be entered only once.

For example, if Avogadro’s Number, 6.022  1023, appears more than once in equations or expressions to be entered into the calculator, then the number should first be stored under the name of a variable, and that name should be used in all terms subsequently entered into the calculator. Doing so decreases the number of possibilities for transfer and keystroke errors. To make Guideline 3 compatible with Guideline 1, students should be encouraged to develop the habit, when writing equations on paper, of giving a name or letter to a frequently used numerical value, and then using that name instead of the numerical value when writing the equations. Guideline 3 is an obvious extension—employed for the same reason—of the practice followed by all scientific/graphing calculator designers of providing dedicated keys for the common constants  and e.

Guideline 4.Store intermediate results in the calculator rather than writing them on paper.

Just as in Guideline 3, storing intermediate results under the name of a variable in the calculator decreases the number of possibilities for transfer and keystroke errors.