Diagnostic Questions for Revising Content and Peer Review

10

Diagnostic Questions for Revising Content and Peer Review

IPHY 3700: Scientific Writing in Integrative Physiology

Diagnostic Questions for Revising Content

This handout presents 8 questions to guide a goal-based diagnosis of problems, as well as strengths, in the content of scientific papers. We'll apply these diagnostic questions in the process activity "Revising Content" for our first writing project, the discussion essay. Later in the semester, we'll apply these questions in the peer review assignment for our position paper.

_________________________________________________________________________________

1. How successfully does the draft's content accomplish appropriate power goals?

A productive approach to content-level revision essentially involves evaluating a draft's success in accomplishing appropriate power goals. So, in a goal-based approach to revising content this first diagnostic question is fundamental to all others. As addressed in the diagnostic questions that follow this one, the appropriateness of a writer's goals depends on factors such as the paper type and section as well as the intended audiences. The extent to which a draft is successful at accomplishing appropriate power goals depends on factors such as the clarity, accuracy, development, and logic of its content. Consider, for example, content that is inessential for the audience, unclear in meaning, inaccurate, or insufficiently developed. These sorts of problems, which we address in presenting the diagnostic questions that follow this one, ultimately lead to a similar outcome: They limit writers from successfully accomplishing power goals.

Experts in independent revision and peer review possess specialized discourse and scientific knowledge. They deeply understand

(1) the basic science content and advanced research issues on which papers are based, including technical, methodological, and statistical matters;

(2) the conventional goals, along with effective strategies for accomplishing them, for different types and sections of scientific papers; and

(3) how to evaluate the many qualities of content in scientific papers, through a goal-based, problem-solving, and systematic process.

The third item above reflects procedural knowledge that relies on core scientific and discourse knowledge. Below, I present a general procedure to guide a goal-based approach to diagnosing content.

Step 1: Determine whether the draft includes content to accomplish goals that (a) the writer set in planning and (b) are appropriate for the rhetorical situation. This first step involves assessing whether the writer has included content that you deem should be in a draft. One aspect of this assessment involves the writer's plans. Under the considerable cognitive load of drafting, writers may inadvertently overlook parts of their plans. The outcome is that the draft's content doesn't reflect all of the goals that the writer intended to accomplish. It's relatively easy to detect and diagnose this problem independently: Use your goal-based plan as a checklist for assessing the completeness of your draft. When you're revising your own draft, this diagnosis is especially simple and productive if your goal-based plan is complete—that is, if it included all of the best power goals and strategies before you started drafting. If you question whether your plan is complete, the first step to revising content is to revisit the process of developing a goal-based plan. When you're reviewing peers' drafts, you can begin by asking them for details about their plans, particularly about their power goals. You can then apply their plans as checklists for assessing whether they've included essential goal-based content.

Another aspect of this first step involves considering the rhetorical situation, which is defined by the paper type and section, scientific matters that are particular to the research project, and the audience. Sometimes writers overlook including content for accomplishing important power goals in their drafts. So in independent revision and peer review, you should make passes through drafts in which you're looking for content that you deem appropriate for the rhetorical situation. We cover this rhetorical analysis in several of the diagnostic questions below.

If a draft includes content for accomplishing the writer's plans and appropriate power goals, the next steps are to evaluate the quality of that content.

Step 2: Identify the goals that the writer was trying to accomplish in a draft's parts. This step is for focusing your attention as you begin evaluating content. The objective is to map a draft's paragraphs and sections to the goals that the writer was trying to accomplish in them. You may find that single paragraphs or whole sections are devoted to single goals. Or, a paragraph may contain content for accomplishing several goals. When you've identified what the writer was trying to accomplish in a given part, make a mental note about the goal or, in the draft's margin, write a short phrase that summarizes the goal. If, for example, you're reviewing the discussion section of a research paper, you might tag paragraphs with phrases like these: state and support conclusions, discuss limitations, relate to previous studies, and suggest future research.

Step 3: Use the "think-ahead" method to establish criteria for content that is necessary to accomplish the power goal successfully. As we'll discuss in class, the think-ahead method is a strategy for determining what you're looking for before you go looking for it. In independent revision and peer review, the think-ahead method involves brainstorming the information, ideas, and arguments that the writer should have included in order to accomplish a power goal and its strategies. Consider a scenario in which a scientist is writing a hypothesis-driven research paper on a low-carbohydrate diet and coronary heart disease (CHD) risk factors. For the introduction section, one of the writer's goals is to present the hypothesis and its rationale. Suppose that you've identified a paragraph that is devoted to this goal. To use the think-ahead method, before you read the paragraph closely and critically, you'll brainstorm what the writer needs to say and do to present the hypothesis and its rationale successfully. Then, in the next step, you'll use your think-ahead criteria to guide your critical reading of the draft.

Step 4: Use your think-ahead criteria to identify strengths and problems concerning how successfully the content accomplishes the power goal and its strategies. We'll practice this technique in class!

2. How successfully does the draft's content meet the key power goals for the paper type and section?

To apply this diagnostic question productively in the revision process, you have to know the conventional goals for different types and sections of scientific papers. In addition, you have to know general strategies for accomplishing the goals. Consider the following conventional power goal for the introduction section of a research paper: To identify the research questions and to explain the factors that motivated the scientists to raise them. To successfully accomplish this goal, scientific authors might apply one or more of the following general strategies:

· Directly identify the research question relatively early in the introduction (especially for long introductions).

· Explain the real-world problems that motivate the research.

· Discuss the knowledge gaps in the scientific field.

· Present unresolved and conflicting findings from previous studies on the issue.

· Discuss the barriers to answering the research question, such as methodological limitations, that have been overcome in the study.

In revising introduction sections of scientific papers, you can use these general strategies to think ahead about what information and ideas the author should include to accomplish the power goal above. We'll demonstrate this process in class.

3. How successfully does the draft's content meet the needs, expectations, and values of primary and secondary audiences as well as target populations for the research?

This diagnosis involves applying your audience analysis, from planning stages of the writing process, to evaluate whether a draft's content is appropriate for its intended readers. One aspect of the diagnosis involves checking for whether you've defined terms and concepts to match your readers' level of knowledge about your topic. Another aspect of the diagnosis involves questioning whether your ideas and arguments address your readers' views and values. For school papers, your most important reader is usually your professor. So, you should evaluate draft content for whether it meets the standards that he or she has established for excellence. In IPHY 3700, for example, your professor is looking for students to demonstrate advanced scientific knowledge, to go beyond summarizing and paraphrasing, to make original arguments, to "think out of the box," and to dig deep holes. In independent revision and peer review of papers in this class, you should use these criteria to guide your evaluation of a draft's content.

4. To what extent is the draft's content accurate and comprehensive in covering the research field?

We tend to think of accuracy in scientific writing in terms of whether authors define common terms correctly, reproduce statistics from primary sources exactly, and explain concepts precisely according to widely accepted views. So in independent revision and peer review, evaluating content for accuracy involves comparing information and ideas in a draft to well-established knowledge. Because the knowledge in many scientific fields is by no means well established, however, the criteria we use to judge accuracy must be extended. Consider research fields in which knowledge is continually evolving and in which study outcomes are conflicting. When you're revising papers on such dynamic and debatable issues, your diagnosis for accuracy must go beyond assessing whether the content is simply right or wrong in reference to established knowledge. For many scientific issues the extent to which content is accurate depends on how comprehensively writers cover their research fields, acknowledging data and concepts that represent all sides of arguments and documenting the consensus views of leading researchers.

One of the most severe and common problems in scientific writing is content that does not comprehensively represent the relevant research on dynamic and debatable issues. Consider the discussion section of a research paper in which the writer's power goals include arguing for the physiological mechanisms that underlie the study's results. Suppose that in the published literature scientists have written about a handful of mechanisms that are relevant to the writer's argument. If the writer does not acknowledge all of the mechanisms, knowledgeable peer reviewers may respond like this: You're saying that the mechanism is X, but according to so-and-so's 2004 paper, which is generally recognized as the gold standard, the mechanism is more likely to be Y. You're also overlooking mechanisms Z, A, and B. In this sort of comment, the reviewer is saying that the writer's argument is insufficient and weak. But the reviewer is also saying that the content isn't accurate in the sense that it doesn't cover the research field and document consensus views.

It's especially difficult to detect and diagnose problems involving accuracy independently. But it's worthwhile to make passes through your drafts for this aspect of content-level revision. The task is to tune your attention to information and ideas that you're not completely confident are accurate. Focusing in on such content, you can then check it against primary sources to confirm its accuracy and comprehensive treatment. Of course, you should also solicit feedback from peers and professors who know your research field thoroughly.

5. How clearly are the writer's ideas expressed?

Before you can diagnose qualities such as the appropriateness, accuracy, and development of a draft's content, you have to understand what you're reading. This diagnostic question focuses on the lack of deep understanding that readers may experience due to barriers at the sentence and paragraph levels. (The question doesn't concern the sort of misunderstanding that readers experience when they lack scientific knowledge.) These barriers include awkwardly phrased sentences, vague language, illogical expressions, misplaced and missing inferences, and incoherent development of ideas. These problems may be caused by the writer's lack of writing knowledge and skill or by the writer's lack of content knowledge and conceptual understanding. Whatever the cause, the problems lead readers to respond, "I just don't understand what you're trying to say here."

Like revising content for accuracy, revising for clarity of expression is especially difficult to do independently. We don't draft with the intention of confusing readers. And when we draft, our ideas usually seem clear enough to us. But, sometimes, content that is crystal clear to us as writers might be beyond baffling to our readers. In independent revision the challenge of diagnosing the clarity of content involves the subjective, or close, relationship between ideas in the writer's mind and draft. Consider the problem of missing inferences, or the lack of necessary information to clarify the relationship between ideas in successive sentences. The inference is in the writer's mind but not in his draft. When the writer reads the draft in the revision process, he may draw the inference naturally because his mind and draft are so closely woven together. A practical strategy for gaining objectivity is to allow sufficient time to pass between drafting and revising, during which you don't think about your writing project. Of course, to allow sufficient time, which might be days or weeks, you have to organize your project schedule to complete drafting early.

Another practical strategy for diagnosing clarity of content independently is to read your draft out loud, listening for phrases and sentences that sound awkward and that are difficult to get through. Often, awkwardly phrased sentences are the root of unclear content. Often, writers are able detect awkward phrasing in their drafts only when they speak and hear it. Perhaps the most powerful strategy for applying this diagnostic question in independent revision is to take the perspective of your readers—that is, to put yourself in their shoes. That requires using your audience analysis to image how your readers might respond to words, phrases, sentences, and ideas developed across paragraphs and sections.

It's usually simple for peer reviewers to detect unclear content. When they don't understand something, they'll let you know about it. But, often, peer reviewers are challenged to diagnose the problem, or to clearly explain why they don't understand. In the competitive review process (for professional scientists), referees might not be obliged to explain their misunderstanding. They'll just tell the writer that the content is confusing and it needs to be fixed. When you're reviewing a colleague's paper to help him or her, however, you'll serve the writer best by explaining, in detail if necessary, why you're having trouble understanding certain content. That might require commenting on your analysis of sentence grammar and paragraph structure, pointing out missing inferences directly, and explaining precisely how you interpreted the confusing content so that writer will understand why you're confused and how to rewrite the material clearly.