How Dentists Read a Technique Article: a Qualitative Study

1

How Dentists Read a Technique Article: A Qualitative Study

David W. Chambers, EdM, MBA, PhD; and Cindy Lyon, DDS

[This is an early draft intended for circulation among colleagues for comment. Areas known to need attention are highlighted in red. At this stage, references are embedded in the text and shown in blue. There is no abstract. The intended journal is JADA.]

Journals publish instructions for authors that prescribe standard structures, and the Vancouver Convention of the World Association of Medical Editors is one example of efforts to achieve consensus across publications. There is also advice from experts on how to read or write effectively [Lozano JM, Ruiz JG. Reading a journal article. Indian J Pediatr 2000; 67(1): 55-62; Piatti A, De Fiore L. How to read medical journals. From ad hoc perusal to proactive reading to keep abreast of newly emerging evidence. Recent Prog Med 2008; 99(12): 590-8. Wagner JD, Wagner SA. Keeping abreast of the medical/dental literature: a simplified approach. J Oral Maxillofac Surg. 1992 50 (2), 163-8.Subramanyam R. Art of reading a journal article: Methodically and effectively. J 2013, 17 (1), 65-70].

There is a small literature on the self-reportedreading habits of health professionals (whichjournals and how often?)[Barnard A, Kamien M. The reading habits of RACGP training program doctors. Aust Fam Physician 1994, 23(9): 1753-6,8,60; Saint S, Christakis DA, et al. Journal reading habits of internists. J Gen Intern Med 2000; 5(12): 881-4;Tenopir C, King DW, et al. Journal reading patterns and preferences of pediatricians. 2007; J Med Libr Assoc 95(1): 56-53;Zarshenas L, Danaei SM, Mazarei E, Najafi HZ, Shakour M. Study skills and habits in Shiraz dental students; strengths and weaknesses. J Educ; Fujimoto, Yuka; Hagel, Pauline; Turner, Paul; Kattiyapornpong, Uraiporn; Zutshi, Ambika. Helping university students to 'read' scholarly journal articles: the benefits of a structured and collaborative approach. Journal of University Teaching & Learning Practice. 2011, 8 (3), 1-12]. The most extensive literature in this area concerns the identification and training of physiological processes that accompany reading of passages of test, especially among those with reading difficulties [J Learn Disabil. 2015 Jul 30, e-pub; Bergey BW, Deacon SH, Parrila RK. Metacognitive Reading and Study Strategies and Academic Achievement of University Students with and Without a History of Reading. Difficulties.Health Promot. 2014, 5 (3), 44, e-pub].

However, there are no empirical investigations of how dentists or other healthcare professionals actually work their way through articles. There have been no studies of how readers allocate their attention to segments in an article and how they use their own structure and the structure of the article to navigate while reading under ad lib conditions. This paper describes a qualitative study where clinical dentists were videotaped commenting on what they were doing at various stages in reading a journal article describing a somewhat novel approach to fabricating a provisional restoration. This is a first attempt to identify patterns and goals of dentists when reading this type of literature.

Materials and Methods

This project was approved in the expedited category by the IRB at the University of the Pacific, #13-62.

The article used in this study was Terry DA. Developing a functional composite resin provisional in the American Journal of Esthetic Dentistry, 2012; 1 (2): 56-66. The article presents a step-by-step description of the fabrication of a resin provisional on the central maxillary incisors of a young patient with optimal periodontal and occlusal conditions. In addition to the short abstract, short conclusion, and 21 references, the main text of the article is 1,000 words in length. In addition to before and after photographs, there were 28 color figures showing various steps in the procedure. The most unusual feature of the procedure was injection of the flowable composite resin through a portal in a clear silicone matrix placed over each tooth. Four individuals not participating the study were asked to read the article without interruption and they did so in an average of 4 minutes and 42 seconds.

The sample consisted of 21 dentists, all of whom had at least 10 years’ experience with resin provisionals. Their average age was 55.1. Eleven were part-time dental school faculty members and 2 full-time faculty members no longer practicing. Eight were currently in private practice.

Sessions were videotapes. The camera was positioned on the journal article to record the page being attended to. The task for the participants was placed in context by the experiment says, “Imagine that you are in your office and have some time. You find this article. What would you do?” Minimal prompts were uses: “please proceed,” “tell me what you are looking at,” and “why did you turn back to the front of the article?” Both the researcher’s and the participants’ comments were transcribed verbatim. All participants reviewed the transcripts of their remarks within a few days of the recording (two making minor word changes). Three participants also reviewed and approved the first-level coding. The transcribed and coded dataset contained 30,000 words.A copy of the article and all the transcribed comments by “readers” are available online at

The customary approach to qualitative research was followed [Charmaz K. Constructing grounded theory. Los Angeles, Sage, 2006; Glaser BG. Basics of grounded theory analysis. Mill Valley, CA: Sociology Press, 1992; Glaser BG, Strauss AL. The discovery of grounded theory. Chicago: Aldine, 1967; Strauss A. Corbin J. Basics of qualitative research. Newbury Park, CA: Sage, 1990] This procedure is most useful for developing hypotheses rather than testing “yes/no” questions researchers have developed from the literature or from previous research. The intent was to open a preliminary understanding of the landmarks in clinicians’ approach to the technical literature.

At the first-level of coding participants’ sentences where converted into short phrases describing the content of each thought or action. First-level coded phrases we sorted into categories in two additional rounds of coding. Multiple rounds of coding were required because early approaches produced idiosyncratic categories used by some participants but not by many others. Eventually a two-level system was adopted. Four major categories (orientation, navigation, judgment, and distancing) were identified. These were used by from 100% (navigation) to 75% (distancing) of respondents. A second level of categorization within the major headings was also developed. For example, four types of navigation emerged. Between 20% and 95% of participants used each of these more fine-grained categories.

Seventeen additional subjects were recruited using the same inclusion criteria and questioned regarding the entire issue of the journal in order to determine reactions to the target article from respondents who were not specifically requested to read it.

In qualitative research, optimal sample size is not determined in advance by power analysis. Rather, coding begins as data become available and data collection ends when the “concepts are saturated.” No new coding categories were identified based on interviews with the final five respondents, so data collection was terminated.

Results

Readers engaged in four activities to organize their approach to the article describing a clinical procedure. These included orienting (identifying overall topic and style of the article); navigating (pivoting from elements in the text to search for other material that may or may not follow in sequence and may or may not be present in the article); judging (evaluating the value or applicability of parts of the material); and distancing (identifying limitations in the article and reasons why the article should not compel assent). Most readers used all four activities, and these on multiple occasions. Examples of comments from readers representing these categories are shown in Table 1. The values reported in Table 1 are proportions of readers noting each type of comment (in square brackets) and the average number of such comments (in curly braces). All readers used the navigating and justifying categories at least once. Only 5 percent skipped a general orienting activity, going straight to the detail. Eighty percent of readers engaged in some form of distancing from the procedures or conclusions presented in the paper.

Five transcripts were coded independently by the two authors. Agreement for the four major categories of activities was above 90 percent in each case. Agreement for the 14 subcategories was above 70 percent.

[Determine whether the four main categories will be nouns, navigation, or gerunds, navigating.]

Each of the four major activities was coded at a more detailed level, as shown in Table 1. For example, orientating could be recognized as taking place at the general level (as in a single-sentence global characterization of the paper) or as refinement placing limits on the article (as its being a case report or its having high esthetic content). A larger than typical number of comments is reported in Table 1 for readers’ impression of the central point of the article. Among the 21 readers, there were at least 9 different interpretations of the main point the author was making.

There were four subcategories of navigating activities, and more than half of the readers used each of these at least once. Skimming, the most popular navigation activity at 0.952, involved selective jumping for recognized landmarks to the next, while assuming that the intervening material could be filled in without surprise. Scanning always moved in forward in small segments. Hunting was another popular form of hopping from landmark to landmark, used by 19 of the 21 readers. But hunting could involve skipping from the introduction to the references and backward jumps. It was motivated by search for presumed information – “I wanted to compare the before and after pictures.” Confirmation occurred when a reader found what was being hunted for or reported that it could not be found. There were also comments specifically about the photographs. Eleven of the 21 readers remarked that they were using the progression of photos to guide their passage through the article.

There were also four subdivisions of the judging activity. Generally these comments involved readers attempting to place the article in the context of their personal practice experience and to determine whether anything in the technique could be adopted. The most common subcategory, selected at least once by 19 of the 21 readers, expressed an interest in trying the procedure or more generally recognition that it would be a valuable adjunct to their practices. Sometimes there was a more explicit and detailed side-by-side comparison between the procedure in the article and what the reader was familiar with – “this is like what I do, except I use different materials” or “I could not get patients to pay for the extra time this would take.” Fourteen of the readers commented on the finished outcome being very attractive, without in that comment making a comparison with their own practice. (Usually comments on the outcome were followed with further comments making comparisons with the reader’s practice.) Only four readers failed to remark on the excellent quality of the photography that documented the case. These comments strongly emphasized that such beautiful photography could be accepted as evidence the dental work was also of a high quality becauseanyone skilled and careful enough to take such photographs would be skilled and careful in performing dental procedures.

Many readers placed some distance between themselves and the article, although there was no obvious prompt to do so. An example used by more than half of the readers included comments that the article did not answer all questions about the procedure and that it was not possible to generalize from the case presented to other types of patients or to related procedures. There is no reason to make this comment as it is a truism. More than half of the readers also noted that the case was ideal and thus the procedure could not be used for all patients. About a third of readers raised questions about potential conflicts of interest or excessive commercialism. Although there was a disclaimer and no unusual pattern in identifying products by name, readers felt that even the potential for these factors would excuse them from having to take the report at face value. A few readers noted that the paper was of limited value because it was a case report and thus not high in the pyramid of best evidence. There was an association between commenting that the article demonstrated an attractive result or that it described a potentially useful technique and mentioning any of the four distancing postures (r = 0.451, p = .04).

Figure 1 illustrates the fact that the four major activities were prominent at different points in working with the article. The horizontal axis in this figure divides the number of comments for each reader individually into quartiles based on temporal position. Q1 represents the first 25 percent of comment, etc. Thevertical axis is the number of comments. It is apparent that orienting activities were few and most prominent when beginning to work with the article. Navigating activities werefrequent at the beginning, but tapered off sharply. The judging activities were present from the beginning and strongly dominated toward the end. In fact, it was typical for readers to engage in general reflection at some point, setting the article aside and reflecting on their own reflections. Distancing rose appreciably while working through the material.

The video camera was positioned so that correlating its image with the verbal commentary made it possible to identify what aspects of the article readers were attending to at any particular time. This made it possible to reconstruct personal timelines for attending to segments of the article. These are shown in Table 2. An “idealized” path through such a paper would involve sequentially working through the overall appearance of the journal, then the abstract, followed by the introduction and text, the photos and captions, a comparison between text and photos, the conclusion, and finally the reference. In terms of the numbered code in Table 2, this sequences would be 0,1,2,3,3c,4,5.

None of the readers followed the “idealized” sequence. About half began with the introduction and half started in the middle with the photos. Consulting references or the journal as a whole was relatively rare and could occur at any point. There was substantial backtracking. Most time was spent with the photographs.

It was implied in orienting participants that they should “do something” with the article to which their attention had been drawn. This constitutes a limitation on the generalizability of the findings. Seventeen new subjects were recruited and asked to comment in a general way regarding how they would spend their time with the journal issue (not the article) handed to them. (One claimed only to do online searches). All but one respondent began with the table of contents. All 16 read through the titles of the articles; 3 looked for familiar names among the authors. All 16 also thumbed through the journal from start to finish, looking at each article. Tables and photos were the items most commonly focused on. One or two abstracts were read. On average, subjects indicated an interest in reading just under 2 of the five articles, the one drawing the most attention being the article used in this study. Where comments were offered about which articles attracted attention, the dominant standard was topics the reader was already familiar with, especially because that was an area of interest in practice. Two respondents mentioned that it was good to see “evidence-based dentistry,” which they defined (when asked) as procedures they used being mentioned in peer-reviewed publications. Although the journal is peer reviewed, that point was not apparent or verified by those commenting on it.

Discussion

This is a first attempt, using qualitative research methods, to understand how dentists structure their “reading” of the clinical technique literature. Twenty-one dentists with private practice experience fabricating provisional restorations on upper anteriors were videotapes while working through an article documenting this technique and commenting on it.

It is apparent that readers actively engage with the professional literature – they personalize it. Even for such a straightforward technique paper, there was wide variation in readers’ understanding of the purpose of the paper and what theymight expect togain from reading it. Among the 21 participants who read some of the article, at least 9 distinct sets of expectations were generated as to why the article should be considered.

Readers also customized their navigation through the article. They created personalized structures upon which to frame their understanding. It is rare that they proceeded in a linear (page-to-page) format or used the conventional “introduction, material andmethods, results, andconclusion”required for such literature. The dominant organizing principles in this case appear to be personal relevance in practice and the visual pattern created by the photographs. Because this was a technique article, readers also use the well-known technical sequence as an organizing principle. The frequent jumping ahead and backtracking appear to stem from readers needing to verify that their template for the flow of the article (technique) was matching at each point the standardized structure for such literature. Most attention was focused on possible differences between these two patterns.