Analyzing TV Commercials
Roger D. Wimmer, Ph.D.
Preface
The purpose of this document is not to ridicule advertising agency personnel or advertisers who accept ad agency recommendations and/or commercials produced for them. Instead, the purpose is to describe a situation in mass media research that most people outside of the media, ad agency, and research business know anything about.
This document includes a brief discussion of television commercial analysis and a sample ratings sheet used to accomplish the task. The purpose of TV commercial analysis is to: (1) Analyze current or proposed TV commercials; and (2) Identify commercials that are ineffective with polychronic viewers. Recall that about 75% of adults 18-54 in the United States identify themselves as polychronic TV viewers.
It is important for anyone who plans to test TV commercials, or interpret data from the tests, to read two documents on the website for Mass Media Research: An Introduction, 9e (www.wimmerdominck.com)—"Research in Advertising" located in the Supplemental Information section, and "The Five Stages of Communication" located in the Readings section.
Background Information
In most areas of private sector mass media research, clients are usually delighted to receive information about their audience/consumers because the information provides information to help make the correct decisions about the product or service. However, the delight in receiving research information is not usually evident in TV commercials analyses because, in most situations, an advertising agency is involved. In most situations, advertising agency personnel do not agree with empirical tests of their commercials because they say things like, "It's not possible to test art." or "This is a creative product, not something you can test." or "We know from experience what works and doesn't work in television commercials and research is a waste of time." While testing more than 7,500 TV commercials during the past 35+ years in media research, I have learned several things about advertising, advertising research, and how companies, businesses, and advertising agencies deal with the advertising process. Among the many things I have learned, five stand out in reference to testing television commercials:
- Only a very small number of advertisers, including the top management of the largest companies, know what advertising is and how it works. When asked what they hope advertising will accomplish, most managers will say something like, "To increase sales." Because of the lack of understanding about advertising, the managers (CEOs included) of most companies that use advertising agencies rely on the agency personnel to "point them in the right direction." In essence, most managers give blind trust in their ad agency because "that's their business and they know what they're doing." That's not necessarily true.
- The problem is that most advertising agency personnel do not know much, if anything, about communication/persuasion, and what advertising really is. During the past 35+ years of conducting TV commercial analyses, I have experienced hundreds of situations where it was clear that an ad agency's goals in developing commercials for a client were to try to win an Addy Award and/or receive acclaim from their peers in other advertising agencies, most of whom also know nothing about communication and the purpose of advertising.
- Most advertising agency personnel discount the results of TV commercial testing until their clients (the advertisers) see actual research conducted, usually in focus group settings, and discover that many (or all) of the commercials produced for them by their advertising agency are worthless.
- Most advertising agency personnel and advertisers have never heard of the phenomenon called polychronic behavior, particularly in reference to watching television. Advertisers and ad agency personnel are shocked to hear that about 75% of all adults 18-54 in the United States do something else while watching television; that all people don't simply sit and watch television as though they were comatose. However, the shock of hearing the information is tempered when, during the course of discussing the phenomenon, about 75% of the advertisers and ad agency personnel admit to being a member of the polychronic viewing group.
- After years of hearing advertisers (and others) complain that their television advertising, as well as advertising in other media, seemed to be a waste of money, I determined that there was a need to develop a new test for TV commercials.
The remainder of this document focuses on a procedure for testing TV commercials.
Procedure:
In most situations, TV commercials are tested in focus group settings, although it is possible to test spots in larger groups, such as a group administration project in a hotel ballroom.
Respondents are given ratings sheets and told they will watch a number of TV commercials. They are asked to rate each commercial on three items:
1. Overall Rating on 1-10 Scale. Instructions: "You all have seen many TV commercials in your lifetime and you have your own ideas about what is a good commercial and what is a bad commercial. For the first rating, the Overall Rating, please rate the commercial on how good you think it is as compared to other commercials you have seen. Use a scale of 1 to 10, where '1' means 'Poor,' '10' means 'Excellent,' and 2 through 9 are in-between. Please record your rating on the short line, and in only a few words, explain your rating on the longer line that follows your rating."
2. Try Rating on 1-10 Scale. Instructions: "Next, I would like you to rate the commercial on what we call a 'Try Rating.' With this rating, please use another scale of 1 to 10, where the higher the number, the more likely you would be to try the product or service for the first time, or purchase the product or service again. Please record your rating on the short line, and in only a few words, explain your rating on the longer line that follows your rating."
3. Message. Instructions: "After you rate each commercial on the two items, please briefly explain what you learned from the commercial, or what the commercial told you."
The wording for the "Try Rating" changes depending on the type of commercial tested. For example, if the commercial is for a charitable organization asking for contributions, the wording might be: "With this rating, please use another scale of 1 to 10, where the higher the number, the more likely you would be to contribute to the organization or contribute once again."
After the ratings scales are explained, the respondents are told:
"I have one more explanation about your ratings sheet. As you see on the sheet, you will rate each commercial twice. For the first showing of each commercial, I would like you to turn your eyes away from the screen. I'll play the commercial and then ask you to rate it on the two items under the 'Not Watching Screen' label. For the second showing, I'll ask you to watch the screen while the commercial is playing. After the commercial is over, please rate it on the two items under the 'Watching Screen" label.' Keep in mind that your ratings and comments are your opinions and, therefore, cannot be wrong. By the way, you may have different ratings and explanations of what the commercial told you for the same commercial in the 'Not Watching Screen,' and the 'Watching Screen' areas. That's OK.
Finally, please answer the two questions on the bottom of the sheet . . . one about watching TV, and the second about your age and sex."
Analysis
After testing thousands of TV commercials, two measurement standards have emerged:
1. Ratings: Successful commercials have Overall and Try ratings of at least 7.0. The closer these ratings are to a perfect "10," the better. When ratings fall below the minimum of 7.0, the ad agencies and/or advertisers are told that the commercials do not meet minimum standards established by previous tests of more than 7,000 TV commercials.
2. Message: The goal in this part of the analysis is to verify that the message received/interpreted by respondents is the message the ad agency and/or advertiser wanted to convey. In many cases, communication errors are obvious. For example, the goals of a TV commercial for a Soft Rock radio station may be to: (a) explain that the radio station plays more Soft Rock music than any other area radio station; and, (b) make sure that the station's frequency and call letters (or slogan or moniker) are clear. A test of the commercial may indicate that the respondents describe the radio station's music as "Hard Rock." In this case, which is from an actual commercial test, there is clearly something wrong with the commercial.
Sample Ratings Sheet
The ratings sheet on the following pages is an example of a typical setup for a TV commercial test. The sample shows space for only three ratings, but it can be easily expanded to account for the number of commercials tested (no more than 10 in one research setting).
TV COMMERCIAL RATINGS
Commercial 1
Not Watching Screen
a. Rating (1-10):
b. Try (1-10):
c. Message:
Watching Screen
a. Rating (1-10):
b. Try (1-10):
c. Message:
Commercial 2
Not Watching Screen
a. Rating (1-10):
b. Try (1-10):
c. Message:
Watching Screen
a. Rating (1-10):
b. Try (1-10):
c. Message:
Commercial 3
Not Watching Screen
a. Rating (1-10):
b. Try (1-10):
c. Message:
Watching Screen
a. Rating (1-10):
b. Try (1-10):
c. Message:
Please answer the following questions:
Overall, when you watch television, which statement best describes you . . . (Circle a number.)
1. When I watch TV, I don't do anything other than watch TV
2. When I watch TV, I'm also doing something else at the same time
3. It depends. Both answers 1 & 2 describe me
4. Don't know/Not sure