Evaluating AIDS Prevention Programs Expanded Edition (1991) / Chapter Skim
Currently Skimming:
3 Evaluating Media Campaigns
3 Evaluating Media Campaigns
Pages 50-82
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 50... ...
Evaluation may also help reduce the wasting of resources on ineffective campaign activities; if a campaign phase does not produce ejects it can be withdrawn or replaced with more effective material. ~ discussing the need for rigorous evaluations, the pane} concluded that randomized experiments would produce the most valid account of program effects.
|
From page 51... ...
BACKGROUND AND OBJECTIVES Four phases of a national AIDS media campaign have already been launched. Tree phases have channeled messages nationwide through a series of public service announcements (PSAs)
|
From page 52... ...
to increase the availability and adoption of comprehensive school health culTicula.2 Aim (~) concerns the specific target audiences only: changes should be observed only among parents of appropriately aged adolescents and the adolescents themselves and not necessarily in the rest of the population.
|
From page 53... ...
FORMATIVE EVALUATION: WHAT WORKS BETTER? Formative evaluations of media campaigns are especially useful for obtaining detailed, documented evidence of effectiveness prior to wide or further deployment.
|
From page 54... ...
A campaign as large and as important as the National AIDS Information and Education Program does not suffer from the same resource deficits that hamper developmental efforts In more typical PSA campaigns. It is possible to do much better than the standard approach in PSA campaign development and to adopt the kinds of research strategies that are more typical of formative evaluations for commercial advertisements.
|
From page 55... ...
Yet even when ideas are solidly grounded in research, the first step in formative evaluation should be to evaluate the power of the idea. This step is particularly important in cases in which the idea is developed outside of the context of the lives of the people to whom the message will be delivered.
|
From page 56... ...
Thus far, the AIDS campaign has chosen different audiencesthe general public, women, adults with multiple sexual parmers, parents and youth for each campaign phase. The panel believes ~at, under these circumstances, themes and messages should continue to be tested for each new phase and each new target group of the media campaign.
|
From page 57... ...
Cost constraints often make it difficult to carry out systematic evaluations to select the set of story boards that is likely to make the most effective advertisement. Large and important campaigns, however, should continue the process of formative evaluation at this stage to help in the selection process.
|
From page 58... ...
Debriefing is typically a part of Me data collection activity because it allows researchers to pinpoint aspects of the various advertisements that lead to confusion, that convey messages other than those that were intended, or that fait to have the intended persuasive impact. This information can then be used to modify the story boards for another iteration of the process, which continues until a format is developed that has the desired characteristics.
|
From page 59... ...
Following formative evaluation and subsequent program development efforts, a campaigns Implementation and outcomes also need to be evaluated (as described below)
|
From page 60... ...
To inaugurate the America Responds to AIDS campaign, NATEP solicited proposals and presentations from 13 different agencies before choosing Ogilvy & Mather. That contract is due to expire this fall, and to compete for the next contract, ad agencies will be required to present a proposed campaign in story-board form.
|
From page 61... ...
. Given the enormous potential impact of such national campaigns as America Responds to AIDS, it is inappropriate to implement the campaign (or any of its main stages)
|
From page 62... ...
In standard test-market studies, a campaign is commonly aired from 6 to 12 months to be sure that most viewers will have been exposed to an ad at least three tunes. Similarly, Be test period for the national AIDS media campaign should be at least six months to provide a sound evaluation of the campaign's exposure effectiveness.
|
From page 63... ...
variations across different markets. AIDS campaign sponsors must decide whether there are, indeed, several potentially elective campaign designs when they carry out their own evaluations of story boards for future stages of the campaign.
|
From page 64... ...
To conduct such tests, air time on desirable shows (those likely to be viewed by the selected audience for the campaigns must be purchased In some random markets and not in others. A common alternative to the ideal approach of using multiple, randomly selected test and control markets is to use only one or two cooperative test markets and the same or a larger number of well-matched control sites.
|
From page 65... ...
for a $10 million campaign. Using this figure as a benchmark, we estimate that the resources needed for efficacy teals of an AIDS media campaign prior to national implementation will be 2 to 3 percent of the expected total campaign cost (including donated or Unkind costs from all sources)
|
From page 66... ...
There are three major approaches to process evaluations of media campaigns: audits of PSA broadcasts through Broadcast Advertisers Reports data; monitoring of AIDS information requests through cans to AIDS telephone hotlines; and general population surveys of campaign awareness. In the past, CDC's process evaluation efforts for media campaigns have primanly involved auditing PSA broadcasts through Broadcast Advertisers Reports (BAR)
|
From page 67... ...
A more expensive but considerably more effective method for assessing exposure to a media campaign is to take general population surveys of campaign awareness. A special version of this approach is to conduct a telephone survey of a random sample of television households that limits contact to the few minutes directly following the broadcast of a PSA in a particular market.
|
From page 68... ...
An evaluation of campaign exposure in all markets throughout the country might produce more accurate information by malting use of the AIDS hotline callers as respondents. It is known that hotline calls increase for a short period of time after the broadcast of a PSA.
|
From page 69... ...
The same strategy of monitoring hotline calls to determine indirectly when a PSA was broadcast in a particular market could allow interviewers working in conjunction with the hotline to pinpoint markets for random-dig~t-dialing coincidental interviews. A small team of no more than 3 or 4 interviewers added to the 68 who are already answering hotline cans could phone random households in areas with PSA broadcasts reported during tile prior 30-minute period.
|
From page 70... ...
provide strong, face-valid evidence that it is time to introduce another set of messages. To monitor the success of a media campaigns implementation, however, it is necessary to obtain more detailed information than is currently available regarding how often the campaign messages are broadcast on television stations throughout the country.
|
From page 71... ...
This section considers the panel's recommended approach to determining the electiveness of a campaign. Randomized Experiments As discussed in Chapter I, the panel recommends randomized experiments to determine the effectiveness of AIDS media campaigns.
|
From page 72... ...
Unfortunately, the data cannot be interpreted with any confidence because respondents were not randomly assigned to exposure to the campaign or to a nonexposure control condition. National campaigns like America Responds to AIDS make it possible to analyze naturally occurring variations in campaign exposure by comparing aggregate trends In the outcomes across television markets that differ in their frequencies of airing the PSAs.
|
From page 73... ...
The current version of the NHIS includes questions on the following: · sources of AIDS information; · self-assessed level of AIDS knowledge; · basic facts about the AIDS virus and how it is transmitted; · blood donation expenence; · awareness of and experience with the blood test for the AIDS virus; · perceived effectiveness of selected preventive measures; · self-assessed chances of getting the AIDS virus; · personal acquaintance with persons with AIDS or the AIDS virus; · willingness to take part in a proposed national seroprevalence survey; and · a general risk behavior question. The National Center for Health Statistics publication Advance Data No.163 provides comparisons of the August 1987 and August 1988 responses.
|
From page 74... ...
Other Archival Sources Evaluators of national AIDS prevention campaigns should collect or monitor data on several other AIDS-relevant archival indicators—such as condom sales and reports of STDs of all types—and use them to aid the interpretation of other evaluation data. These examples and others are all possible indicators of changes in societal norms and behaviors, at least when considered in combination.
|
From page 75... ...
This being the case, it is useful to distinguish between problems in effectiveness evaluation of current campaigns and phases, and problems In He evaluation of future campaigns and phases. Effectiveness Evaluation of Current Activities There are three levels of analysis that may be undertaken to judge the effectiveness of current campaign activities: aggregate analyses, partially disaggregated analyses, and disaggregated analyses.
|
From page 76... ...
Nevertheless, the monitoring of other associated activities needs to be an integral part of outcome evaluation for the campaign. Evaluators need to deterrence to what extent observed changes In knowledge, attitudes, and beliefs are the result of the national mass media campaign and not the accumulated effects of news coverage or of a great many local campaigns.
|
From page 77... ...
The NHIS is obviously of greatest relevance to the AIDS media campaigns in that a senes of AIDS-related items has been included since August 1987. These items are limited to fairly general constructs, however, and cannot be used to evaluate focused objectives.
|
From page 78... ...
that are He focus of Phase IV of He America Responds to AIDS campaign. Hotline Calls and Other Archival Data.
|
From page 79... ...
NATEP staff have indicated to the pane] that the national media campaign is expected to continue into the foreseeable future; the panel believes strongly that unless randomized tests of alternatives or of lagged implementations are conducted, there is little hope for anything more than educated guesses about the meanings of any observed trends.
|
From page 80... ...
The only additional cost is the production of alternative campaign materials; all over costs of campaign implementation and evaluation are the same as for the delayed implementation model. Even for the most expensive approaches, the costs of outcome evaluations for media campaigns will not exceed the costs of program development and win be only a small fraction of the total costs of the campaign.
|
From page 81... ...
(1981) Evaluation of mass media prevention campaigns.
|
From page 82... ...
American Journal of Public Health 78:801-805. Wright, P
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.