Business Research Methods, 11e, Case Discussion Guide
Yahoo!: Consumer Direct
Yahoo!: Consumer Direct Marries Purchase Metrics to Banner Ads
Abstract:As little as two years ago, many advertising pundits were bemoaning the inevitable demise of the banner ad on the Internet. But maybe they were too quick to judge. This case reveals how Yahoo!, in combination with ACNielsen’s Homescan, has developed a methodology (Consumer Direct) to evaluate the true effectiveness of banner ads, from ad exposure to shopping cart. This case describes a multistage study aimed at tracking the efficiency and effectiveness of Internet advertising. Through the use of new metrics that serve to enhance Internet advertising of Consumer Packaged Goods (CPG), Consumer Direct was able to track the purchasing behavior of Yahoo visitors. It also reveals the role Dynamic Logic played in conducting post-exposure ad evaluation to discover how client exposure to CPG ads impacted sales activities.
URLs:
How/When to Use:
This case prepares students to outline and analyze the various components of a research design, from sampling through data collection (Chapters6-14). Used with Chapter6, the student will be able to map the research design and identify strengths and weaknesses of different designs, discuss methods of data collection and the pros and cons of using a particular technique. Used with Chapter2, you can address the ethical implications involved when dealing with the sometimes intrusive nature of conducting research. Here you can open the topic of the ethical treatment of participants and how best to debrief subjects when challenged by deception issues. Used with Chapters 11, 12, and13, you can discuss measurement tools and instrument design. This case can be used to discuss appropriate analysis techniques (Chapters15-19) based on the data collected and the objective of the study, as well as observation studies (Chapter8).
Discussion Questions:
- Describe the research design for Consumer Direct.
After identifying their existing dilemma, Yahoo chose to expand the scope of its research by partnering with ACNielsen to utilize its Homescan panel (sample frame), which provided extensive demographic and lifestyle data necessary to track household purchases. With the data available, it recruited panel members whose Internet activities were tracked and compared to a control group based on exposure to consumer-packaged-goods (CPG) ads.
A test group of approximately 2000 households was established based on two metrics, effectiveness of ad targeting and persuasiveness of the advertising. The test group was exposed to an advertisement and exposure was removed after a period of time, given the time frame of the ad campaign. Members from the control group (also panel members) were not exposed to “tracked” ads, but were able to view different ads.
Purchases were tracked from both groups and the results were compared and analyzed over a two-week period with the use of analysis of covariance (ANCOVA) methodology. Using data from previously tracked Homescan purchasing activities, comparisons (from both test and control groups) were analyzed for the 52-week pre-ad interval period. This process was replicated when Yahoo also partnered with Dynamic Logic, which provided additional metrics for further analysis of purchasing behavior.
2.Discuss the strengths and weaknesses of the research design.
The research design lacks a structured form of sampling plan. The Nielsen database allowed for the possibility of significant analysis of purchasing activities, owing to the available data. However, a more random sampling procedure, rather than having panelists “opt-in” to become subjects, would give more credibility to the results. Owing to the use of control groups, an integrated randomized method of choosing subjects is necessary in order to achieve equivalence between both groups (test and control). The case indicates the researchers were unsure initially as to how subjects would feel about their Internet activities being tracked. It states that panelists had previously agreed to allow the tracking of their purchases, however they chose the opt-in process so that participants could consent to allow the tracking of the Website activity.
Opting-in (prior consent) eliminates one ethical dilemma, but you can ask students to discuss the appropriateness of tracking Web behavior without such consent. Students may raise that even with prior consent participants might not fully understand the extent and nature of the research and what the possibilities of their on-line activities being tracked entailed. You can use this time to ask students to discuss the drawbacks of disclosing to customers the full extent of how tracking their activities will be used? Students may suggest that had ACNielsen fully revealed its intentions that participants may have been more hesitant to opt-in. You can suggest ways to overcome this dilemma, such as debriefing participants once the research is complete. This would involve explaining to subjects the purpose and goal of the study, sharing the results and explaining the reasons for the deception.
The use of prior Homescan data creates the assumption that secondary data was used to supplement the analysis of post ad exposure tracking. This is an important aspect of this study as it creates the opportunity to discuss the importance of utilizing secondary data in research, to avoid recreating the wheel. In this regard, the researcher can take advantage of previously collected data, which may not have been useful in some research areas, but proves applicable to this study. This is also the perfect time to discuss longitudinal vs. cross-sectional research designer.
3.How has the use of panels affected the research design?
The use of ACNielsen’s Homescan panels expanded the scope of the research design as it allowed for the collection of extensive purchase data from households on a global perspective. However, due to the non-randomized fashion in which panels were used, it creates the question of how representative is the sample of the general Yahoo visitor population (the population that is exposed to the CPG ads). While ACNielsen works diligently to build representative panels for their Homescan studies, this case can be used to discuss the use of nonprobability samples, which may have been used in this study. This sampling method (panels), though cost effective, opens the door for bias that potentially affects the findings. Nonetheless, the study made use of a control group, which typically provides comparison data for more precise control in research.
4.How have ethical issues influenced the research design?
The issue of ethics in research is an important one that should spark interesting discussions among students. Given the purpose of this study and the mechanisms it demanded in order to gather useful data, both ACNielsen and Yahoo were fully aware of the intrusiveness of the data collection method. This is why they invited Yahoo members to participate through the opt-in procedure. The 19,000 participants who opted-in were “obviously unconcerned.” Another way of putting this to a skeptical observer of the process is that they might be unaware of the extent of just how much is revealed about them by tracking their Internet activity. Such tracking allowed Yahoo the ability to match the subjects’ purchasing behavior and extensive demographic data with the Websites they visited and link all these to their ad exposure. This information could serve as potential marketing sites for Consumer Direct to further increase their knowledge of where to target their ads. The benefits and risks involved in deception (Chapter2) can be discussed to better prepare students to design ethical but meaningful research.
5.Define the various measurements collected in Consumer Direct.
The use of participants from the Homescan panel as both test and control groups, and the exposure of the test group to tracked CPG ads, implies that a repeated-measures test was utilized. Multiple measurements were collected during the ad campaign period and throughout the pre and post ad exposure stages, to assess the effectiveness of ad targeting and persuasiveness of the advertising. Matching was also employed, so as to ensure that both test and control groups possessed similar characteristics, such as household size and prior purchase behavior. Consumer Direct may have encountered difficulty randomly assigning HomeScan panel participants to particular groups.
Given that Consumer Direct had access to HomeScan panel data, which includes household demographic and lifestyle data, a group time series design may have also been utilized as a way of collecting and analyzing trends in customers’ purchasing behavior.
6.What analysis would be appropriate for the data collected through Consumer Direct?
This question can be used to discuss appropriate analysis techniques (Chapters 15-19) based on the data collected and the objective of the study. Here Consumer Direct has utilized different metrics to test effectiveness of ad targeting as well as the persuasiveness of their CPG advertising to determine effects on sales activities. Given that sales lift is an important factor in this study, a multiple regression analysis could be employed to make predictions about trends in sales based on the use of particular metrics. If researchers could predict the likelihood of persons making a purchase based on exposure to an ad, they could gauge increases in sales based on the amount of persons exposed to a particular advertisement. Furthermore, they could also use this analysis to estimate the effectiveness and persuasiveness of their advertising, based on corresponding increases in the sale of low penetration products, or increases in the number of households who now purchase a particular item.
Students are likely to make a case for the use of a bivariate analysis to compare the brand favorability with likelihood of buying an item. This is consistent with their statistical exposure at this point in their curriculum. However, it is worth pointing out that researchers who evaluate two-variable relationships exclusively, and avoid multivariate tools, ignore the complexity of the relationships involved in sales activities and the potentially valuable information that can be obtained through higher-level analysis.
1