Evaluation Design for Complex Global Initiatives Workshop Summary (2014) / Chapter Skim
Currently Skimming:
5 Mapping Data Sources and Gathering and Assessing Data
5 Mapping Data Sources and Gathering and Assessing Data
Pages 43-54
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 43... ...
• The time frame and budget of an evaluation are critical fac tors in designing data collection and analysis for a complex evaluation. • Routinely collected program data can be a rich and efficient source of information for program evaluation.
|
From page 44... ...
The initial data mapping was driven by the need to understand what data sources were actually available and developed into an iterative process of matching data sources with evaluation questions and the data needed to answer them. DATA ISSUES IN THE GLOBAL FUND EVALUATION Workshop planning committee member Martin Vaessen, senior vice president at ICF International, discussed data issues involved in the Global Fund's evaluation of the impact of collective efforts in reducing the disease burden of HIV/AIDS, tuberculosis, and malaria in 18 countries.
|
From page 45... ...
"Those are the realities we have to deal with," Vaessen said. Overall, the household surveys and facility surveys provided data that were of reasonable or good quality, but for most other information data quality was uneven across countries.
|
From page 46... ...
In addition, she added, "The team members participated in most of the training at baseline and at endline in all eight of the pilots." The AMFm team also developed common data cleaning guidelines and analysis plans and gave responsibility for the analysis of the outlet survey data to the contractors who collected those data. The independent evaluation team then reviewed the results, performed the analysis of the changes over time between baseline and endline, and integrated the quantitative data with the qualitative country case study data to interpret and understand what was going on in each country.
|
From page 47... ...
In the end, the evaluation team relied on secondary data using some inclusion criteria. "We said that in order to be eligible as a baseline, a household survey had to be undertaken no more than 2 years before the beginning of the program and that the endline had to be at least 6 months after the arrival of the first co-paid drugs." In the end, five countries had appropriately timed endline data, but unfortunately, neither Kenya nor Ghana, the two countries that were believed to be fast moving, strong implementers, had appropriately timed household survey data.
|
From page 48... ...
This was a relatively straightforward activity, Simon explained, because the stakeholders had asked for the review and were readily accessible. For the second objective, which was to try to get at what the program was doing, the evaluators used a mixed methods approach because they needed to look both at quantitative data about the key interventions and at some qualitative data about strengthening health systems or capacity strengthening within national malaria control programs.
|
From page 49... ...
as a large PEPFAR implementing partner supporting the scale-up of HIV services in approximately 20 countries over the past 8 years, Batya Elul, assistant professor of clinical epidemiology at Columbia University's Mailman School of Public Health, spoke about the nuts and bolts of using routinely collected data and publically available data for program evaluation. In many of its evaluations and research studies on the delivery of HIV/AIDS services, ICAP regularly uses four data sources: 1.
|
From page 50... ...
To ensure that data are accessible for analysis, ICAP uses unique site codes, geocoding of sites, and data dictionaries, and it built its monitoring and evaluation system to easily export a standardized analytic file along with a standardized data dictionary to minimize the need for data managers or programmers to create analysis files. To address potentially problematic issues involving data ownership, ICAP has established principles of collaboration with each host government for its evaluation framework.
|
From page 51... ...
WORKING WITH FINANCIAL DATA As had been discussed previously, all data must be fit for purpose, and the purpose of financial data differs from that of programmatic data, said Victoria Fan, research fellow at the Center for Global Development. "While programmatic data is used to assess the effectiveness of activities, financial data helps us to assess the efficiency and the value for money of our investments.
|
From page 52... ...
Elias listed several reasons for engaging in this effort, including enabling comparative work, providing the ability to consolidate data to study rare groups, and enlarging studies beyond national boundaries. Issues that need to be addressed include increasing the discoverability of data, using new forms of data such as Google Flu Trends or data mined from store loyalty cards, and developing new methods of collecting data that are more costeffective than traditional survey methods.
|
From page 53... ...
He noted that many efforts are under way to address these issues. For example, the European Union funds the Council of European Social Science Data Archives that is now working to integrate large data archives in many countries.
|
From page 54... ...
The open data movement is putting pressure on institutions to release data and research results within given time frames, and universities can support this movement, for example, through line items for data publication and archiving. "It is on the academic community to push harder on these issues." Elias said that data archiving and accessibility are especially important with new forms of data, such as information gathered from online activities.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.