Evaluation of PEPFAR (2013) / Chapter Skim
Currently Skimming:
11 PEPFAR's Knowledge Management
11 PEPFAR's Knowledge Management
Pages 609-718
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 609... ...
. Knowledge management has been defined by Swan and colleagues as "any process or practice of creating, acquiring, capturing, sharing and using knowledge wherever it resides, to enhance learning and performance in organizations" (Swan et al., 1999, p.
|
From page 610... ...
Significant gaps remain in PEPFAR's knowledge management approach, especially in the realms of knowledge creation, dissemination, and utilization, and to date, OGAC has not articulated a clear and comprehensive strategy for managing knowledge to optimize PEPFAR's performance and effectiveness. This chapter shifts its focus away from assessing and addressing the limitations in the available information that affected the committee's ability to respond to the specific charge mandated by Congress; these were discussed in Chapter 2.
|
From page 611... ...
guidance advised mission teams that activities planned under the SI budget code should aim "to build individual, institutional, and organizational capacity in country" for strategic information activities (OGAC, 2011h, p.
|
From page 612... ...
. To accomplish the overall PEPFAR I targets, each partner country mission team was assigned a target to achieve during the initial 5-year implementation period (OGAC, 2003)
|
From page 613... ...
. Mission teams described working closely with implementing partners to set program targets (116-1-USG; 461-16-USG; 461-18-USG)
|
From page 614... ...
. At the partner country level, mission teams used targets for COP planning and to assess whether implementing partners met their goals (196-1-USG; 636-1-USG; 461-16-USG)
|
From page 615... ...
Use of Evidence to Prioritize Activities PEPFAR has emphasized the use of epidemiological data and intervention effectiveness data to determine which activities and target populations should be prioritized for implementation in partner countries (NCV-13-ML; NCV16-USG; NCV-27-ML; NCV-28-ML; NCV-29-ML) (see also the sections later in this chapter on PEPFAR support for epidemiological data and for evaluation and research)
|
From page 616... ...
. PEPFAR previously had a Scientific Steering Committee that met "regularly to ensure that PEPFAR programs [were]
|
From page 617... ...
. One mission team interviewee spoke of the need for increased coordination with partner country governments in setting PEPFAR targets: ‘Going forward we need to have more discussions and involvement with the Ministry of Health.
|
From page 618... ...
The following sections describe several interrelated aspects of PEPFAR's program monitoring system: collection and reporting, indicator selection and appropriateness, alignment and harmonization with partner countries and other stakeholders, data quality, and data use. Collection and Reporting Program monitoring data are collected by staff at PEPFAR-supported sites such as clinics and community-based programs.
|
From page 619... ...
From FY 2006 to FY 2009, COPs and program monitoring data were submitted from PEPFAR mission teams to OGAC via an electronic, Internet-based system called the Country Operational Plan Reporting Sys TABLE 11-2 Number of PEPFAR Indicators by Reporting Status and Year of Indicator Guidance 2005 2007 2009 Routinely Reported to OGAC 65 76 31 Not Routinely Reported to OGAC 23 23 123 Essential for PEPFAR mission teams -- -- 31 Recommended for PEPFAR mission teams 23 23 92 Total 88 99 154 NOTES: One indicator defined in the 2009 guidance is routinely reported only from programs that have signed a Partnership Framework with the partner country. One indicator that was previously not routinely reported was elevated to being routinely reported starting in FY 2011, increasing the total number of routinely reported indicators to 32.
|
From page 620... ...
Given the importance of program monitoring data for knowledge management, it is critically important for OGAC to have all program monitoring data available in a usable format. Conclusion: PEPFAR's system of indicators to monitor program ac tivities has faced technological challenges that have made it difficult for both PEPFAR and external stakeholders to utilize and access data, and it is critical for the ongoing monitoring of the program
|
From page 621... ...
. OGAC's selection of indicators for program monitoring therefore, clearly plays an important role in which program activities are prioritized in partner countries.
|
From page 622... ...
. Although it is appropriate that the indicators have evolved as PEPFAR has matured and its activities have changed, revising program monitoring indicators creates challenges for mission teams and implementing partners: ‘The transition from the old to the new indicators is a challenge; most existing registers and tools need to be adjusted and levels of disaggregation may need to be adjusted for proper reporting.
|
From page 623... ...
PEPFAR'S KNOWLEDGE MANAGEMENT 623 TABLE 11-3 PEPFAR Indicators Consistent Across the Duration of PEPFAR NGIs Reporting Area Indicator Definition Indicator Level Care Number of HIV-positive patients in PEPFAR Output HIV care or treatment (pre-ART or ART) who started TB treatment Health Systems Strengthening Number of testing facilities PEPFAR Output (laboratories)
|
From page 624... ...
However, this adaptability over time needs to be balanced with the reality that changes in indicators place a burden on partner country programs and limit the comparability of PEPFAR monitoring data, hamper ing the ability to monitor trends. Program Monitoring Indicator Appropriateness Program monitoring indicators are designed to be used by implementing partners and mission teams to assess program performance; the indicators are not designed to "adequately capture every aspect of a comprehensive program" (OGAC, 2009e, p.
|
From page 625... ...
. To address indicator limitations, some mission teams and their implementing partners have developed custom indicators to allow for better monitoring of partners and initiatives (196-1-USG; 116-1-USG; 116-7-USG; 461-17-PCNGO; 934-21-USG)
|
From page 626... ...
. Interviewees in partner countries described how, initially, PEPFAR emphasized developing M&E capacity to report for PEPFAR rather than improving existing partner country M&E systems (331-24-PCGOV; 587-2-USG; 587-9-USG; 636-1-USG; 636-9-USACA; 166-4-USG; 166-12-USG; 166-34-PCGOV; 272-27-USG; 461-11-PCGOV)
|
From page 627... ...
OGAC's shift to the NGIs for program monitoring, described earlier in the chapter, was done in part with the goal of increasing alignment with country M&E systems. As described in the NGI guidance, the shift to the NGIs "attempts to minimize PEPFAR-specific reporting requirements to allow PEPFAR mission teams more flexibility to design M&E plans in line with host countries and strikes a better balance between support for USG reporting needs and national M&E systems" (OGAC, 2009e, p.
|
From page 628... ...
. PEPFAR has articulated a goal of aligning its monitoring and evaluation system with partner country governments and has made some progress toward this goal; however, in general, alignment efforts have not yet fully succeeded, nor have they achieved the desired magnitude of reduction in reporting burden for partner countries and implementing partners.
|
From page 629... ...
. The lack of harmonization with global indicators may be a reflection of activities that are unique to PEPFAR; however, it contributes to the continued need for parallel M&E systems in order to collect PEPFAR program monitoring data.
|
From page 630... ...
However, further modifications could be made to improve the clar ity of indicators, including eliminating PEPFAR-specific language in the indicator guidance, further reducing the reporting burden, im proving indicator harmonization with global indicators, and better aligning with partner country HIV monitoring and health informa tion systems for data collection at the program and country level. Data Quality PEPFAR implementing partners are responsible for validating and aggregating program monitoring data and ensuring data quality from each of
|
From page 631... ...
Partners agree on who gets to count and report cases and services.' (461-1-USG) More broadly, implementing partners and mission teams described various data quality assessment initiatives (166-12-USG; 272-27-USG; 934-21-USG)
|
From page 632... ...
. Because of the lack of an operating program monitoring database as described earlier in this chapter, in fiscal years 2010 and 2011, mission teams reported program monitoring data to OGAC using spreadsheets (OGAC, 2009d, 2010c)
|
From page 633... ...
. Data Use by Mission Teams and Implementing Partners Mission team interviewees described using program monitoring data for multiple purposes.
|
From page 634... ...
. As described previously, in response to feedback from the field about the burden of reporting program data, the introduction of NGIs reduced the number of centrally reported indicators (see Table 11-2)
|
From page 635... ...
One major contributing factor is PEPFAR's requirement for collection and reporting of a large amount of program monitoring data, which has placed a large administrative burden on implementing partners and mission teams, which detracts from efforts for data analysis and use.
|
From page 636... ...
Although PEPFAR's emphasis on collecting data for monitoring and evaluation is commendable, its value is limited if the process is so cumbersome that it results in limited ability to utilize the data. OGAC HQ, mission teams, implementing partners, and partner country governments have different constraints that limit use of data.
|
From page 637... ...
In addition to the use of program monitoring data, as described earlier in this chapter, PEPFAR-supported surveillance and survey data have been used by partner country governments and other country stakeholders to better understand drivers and to monitor trends for country epidemics, to contribute to and influence planning for the national response, and to influence national policies, which, in some cases, has resulted in increased attention to previously underserved populations or service needs (396-6-PCGOV; 396-1920-USG; 396-53-USNGO; 272-22-USG; 272-25-USG; 166-23-USG; 196-11-USNGO; 331-10-PCGOV; 331-ES; . For example, a national HIV/AIDS behavioral risk survey 331-24-PCGOV; 196-ES)
|
From page 638... ...
. For example, mission teams described how information was gathered on who was infected and what behaviors were driving the epidemic and how interventions were developed based on those behaviors, including focusing on best practices in populations at elevated risk (331-14USG; 396-12-USG)
|
From page 639... ...
. In addition to supporting the collection of epidemiological data, which is discussed in detail in the next section of this chapter, PEPFAR has carried out several activities to increase data use among partner country governments.
|
From page 640... ...
. As PEPFAR programs and priorities evolved from an emergency response toward a more sustainable response to the HIV epidemic, PEPFAR leadership increasingly recognized the importance of evaluation and research in capturing, utilizing, and maximizing knowledge created through PEPFAR as well as in ensuring contributions to the global knowledge base on effective HIV/AIDS interventions and program implementation5 (OGAC, 2009f; Padian et al., 2011)
|
From page 641... ...
Descriptions of research in the PEPFAR I legislation and strategy seemed to proscribe against using PEPFAR funds for research, and many country mission teams and implementing partners perceived a ban on using PEPFAR funds for research (IOM, 2007a)
|
From page 642... ...
Targeted evaluations Targeted evaluations began in 2005 to provide an evidence base, beyond routine program monitoring and evaluation or surveillance, to inform program planning and implementation (OGAC, 2005b, 2006b)
|
From page 643... ...
. According to one interviewee, TWGs and implementing partners developed the research agendas based on gaps that they saw in the field (NCV-3-USG)
|
From page 644... ...
measurable, and post-test results with Targeted Evaluation focused questions a comparison or Funded through COP Subcommittee about program control group implementation to Minimal central improve services and oversight identify best practices Targeted Answer specific Rigorous assessments Funded through COP Scientific Steering Technical working group Evaluations questions around including pre- and Committee and driven (2007) efficacy and best post-test results with Targeted Evaluation practices to produce a comparison or Subcommittee generalizable results control group and contribute to Minimal central program sustainability oversight Public Health Answer questions Rigorous, Funded centrally Scientific Steering Globally significant Evaluations around program scientifically Committee and PHE priorities generated at (PHEs)
|
From page 645... ...
randomization, modeling, advanced statistical techniques, and comparison groups Implementation Focused on Scientifically rigorous Funded centrallya Implementation Scientific Advisory Science improving uptake, research methods Science Steering Boardb driven (2011–present) implementation, using randomized Concept proposals Committee (formerly and translation of experimental designs, required through known as PHE Align with country research findings into quasi-experimental implementing Subcommittee)
|
From page 646... ...
. b SAB is composed of members that represent academia, advocates, international experts, the HIV/AIDS community, partner country governments, multilateral and bilateral agencies, foundations, and nongovernmental organizations.
|
From page 647... ...
. One HQ interviewee noted the importance of the introduction of TEs within PEPFAR as important because it provided a mechanism to look at program effectiveness, which the program monitoring indicators did not address (NCV-3-USG)
|
From page 648... ...
. Study priorities were generated and driven by the central level, country mission teams, TWGs, and other members of the PEPFAR community (OGAC, 2006a)
|
From page 649... ...
. Additionally, the information OGAC provided listed the implementing partners for 73 of the 83 PHEs.
|
From page 650... ...
SOURCES: OGAC, 2011b,d. U.S.-Based Organizations 55% Partner Country-Based Organizations 43% Other Country Based Organizations 2% FIGURE 11-4 Organizations implementing ongoing PEPFAR Public Health Evaluation (PHE)
|
From page 651... ...
Without a more complete understanding of the outputs of the TE and PHE efforts supported by PEPFAR, it was not possible for the committee to draw any conclusions about the extent to which findings from these evaluations have been used and have affected PEPFAR-supported programs. Other PEPFAR-supported evaluation and research activities In addition to using TEs and PHEs to provide formal support for evaluation and research across a range of technical areas, PEPFAR has also provided support for a wide array of additional evaluation activities employed at the macro level, the HQ level, the implementing agency level, and the country level.
|
From page 652... ...
. Finally, at the country level PEPFAR mission teams and implementing partners not only participate in the routine data collection and reporting described previously in this chapter, but also carry out basic program evaluations.
|
From page 653... ...
. Finally, PEPFAR provides support to build partner country research capacity; several interviewees described how PEPFAR mission teams and implementing partners provide in-country research support to local institutions and government agencies conducting research (934-44-PCACA; 166-1-USG; 396-55-USG; 166-5-USG)
|
From page 654... ...
, provided evidence on microbicides (NCV-10-USG) , and produced evidence used to change implementing partners' perceptions on gender (272-25-USG)
|
From page 655... ...
These gaps included a need for more program evaluation at the country level (331-5-ML; 272-6-ML; 166-23-USG; 240-15-USG; 272-32-PCNGO) ; local evidence generation (240-15-USG)
|
From page 656... ...
Conclusion: Despite recent efforts to strengthen research and evalu ation activities, the manner in which PEPFAR initially approached research activities was a missed opportunity to establish, from its inception, robust mechanisms to evaluate its programs, assess impact, contribute to the global knowledge base, and develop in country research capacity. Summary Although OGAC has officially supported some form of research and evaluation in PEPFAR since 2005 (through TEs and PHEs)
|
From page 657... ...
. Additional goals include aligning with partner country national research priorities and building research capacity among individuals and institutions at the country level (Bouey and Padian, 2011)
|
From page 658... ...
. The SAB includes 51 members who represent the HIV/AIDS community, academia, international experts, partner country governments, multilateral and bilateral agencies, foundations, advocates, and NGOs (OGAC, 2012g)
|
From page 659... ...
• Global Fund (1) • Joint United Nations Programme on HIV/AIDS (UNAIDS)
|
From page 660... ...
, voluntary medical male circumcision (VMMC)
|
From page 661... ...
. These organizations were based in the United States, PEPFAR partner countries, and other non-PEPFAR countries (see Figure 11-6)
|
From page 662... ...
Proposals for impact evaluations are submitted through the COPs from the mission teams, and funding for the studies comes out of the individual mission team's budget (NCV-31-USG) (Goosby, 2012; OGAC, 2011e)
|
From page 663... ...
. A major limitation of the current vision for impact evaluations is that evaluation studies using randomized controlled methods, while important for some purposes such as establishing the efficacy of an intervention, are neither appropriate nor reasonable for evaluating most HIV interventions being implemented in real-world settings.
|
From page 664... ...
KNOWLEDGE TRANSFER AND LEARNING WITHIN PEPFAR A key aspect of knowledge management is the transferring of insights, experiences, strategic information, best practices, and lessons learned within an organization. The efficient and timely transfer of knowledge is critical to successfully using the knowledge an organization has acquired and created to improve and change the way that work is accomplished.
|
From page 665... ...
interagency mission teams comprised of representatives from the implementing agencies that coordinate and oversee PEPFAR program activities at the country level as well as implementing partners (IPs) , which can include U.S.- and partner country–based: universities or other academic organizations, governmental organizations, NGOs, international NGOs, multilaterals, and private-sector organizations that implement PEPFAR programs.
|
From page 666... ...
However, the types of knowledge transferred in PEPFAR and the mechanisms used to share knowledge go well beyond the reporting of program monitoring data and other routine reporting, as described in more detail below from the perspective of interviewees involved in PEPFAR implementation. The types of knowledge shared and the scale of sharing vary, depending on the pathway of transfer.
|
From page 667... ...
Types of Knowledge Transferred by Different Pathways Table 11-6 lists the types of knowledge, beyond routine program monitoring data, that interviewees described as being shared within PEPFAR along with the multiple different stakeholder pathways through which these types of knowledge are transferred. Interviewees described how information, data and evidence, and policies and guidelines are transferred from the HQ level to the country level as well as at the country level.
|
From page 668... ...
interagency mission teams in different countries (240 9-USG; 272-36-USG; 396-18-USG; 396-1920-USG; 934-43-USG; NCV-9-USG; NCV-17-USG; NCV-18-USG) • Across implementing partners in different countries (116-1-USG; 396-9 PCGOV; NCV-7-USG; NCV-18-USG)
|
From page 669... ...
Programmatic X X indicator data (other than routine reporting)
|
From page 670... ...
. Most interviewees described very similar mechanisms of knowledge transfer across interviews; interviewees did indicate, however, that the scale of use and the particular mechanism used depended on the pathway of knowledge transfer.
|
From page 671... ...
As one HQ interviewee explained it, HQ TWGs and advisors transfer knowledge to mission teams incountry, which then in turn transfer knowledge to implementing partners: "The technical working groups for PEPFAR are comprised of re ally experts in the field. And they [.
|
From page 672... ...
Implementing agencies X (934-43-USG; NCV-7-USG) PEPFAR-Supported and External Meetings Conference calls X X X (934-43-USG; 935-27-USG; NCV-11-USG; NCV-17-USG; NCV-18-USG)
|
From page 673... ...
Staff Rotation/Study Tours Staff rotation X X (396-18-USG; 396-23-USG; 935-27-USG; 935-28-USG; NCV-2-USG; NCV-9-USG; NCV18-USG; NCV-31-USG) Study tour X (240-9-USG; 396-9-PCGOV; NCV-9-USG)
|
From page 674... ...
. One HQ interviewee said that these meetings were originally designed to allow PEPFAR implementing partners to share information with each
|
From page 675... ...
. Similar to the implementers' meetings, some interviewees described periodic country-level partners meetings, either in person or by video, as a means to transfer information, data, and best practices among partners at the country level (272-36-USG; 636-1-USG)
|
From page 676... ...
and responding to ad hoc data requests (166-12-USG; 461-20-PCPS) as ways to share program monitoring data with implementing partners.
|
From page 677... ...
. In addition to PEPFAR's official online communication tools described above, HQ TWGs and implementing partners have used technology in innovative ways to facilitate knowledge transfer between HQ and the country level as well as at the country level (see Table 11-7)
|
From page 678... ...
. Interviewees described various forms of staff rotation that occur within PEPFAR, including staff members moving between PEPFAR countries, staff moving from the country level to the HQ level and vice versa, and staff working on detail between USG agencies, e.g., a staff member from CDC brought to work on detail at OGAC (396-18-USG; 396-23-USG; 935-27-USG; 935-28-USG; NCV-2-USG; NCV-9-USG; NCV-18-USG; NCV-31-USG)
|
From page 679... ...
. Although PEPFAR has been successful in establishing and using a wide variety of formal and informal mechanisms to transfer knowledge both systematically and intermittently throughout PEPFAR, barriers to knowledge transfer exist and there is still a perceived need for more formalized mechanisms for transferring experiences across countries and implementing partners.
|
From page 680... ...
Much of this perceived need was articulated by mission teams and country-level implementing partners who reported a need for more formalized mechanisms for transferring knowledge across countries, implementing partners, and implementation sites. Country-level interviewees recognized the opportunity to learn from the experiences of other countries and voiced a desire for information and best practice sharing regionally and globally (196-8-ML; 934-46-PCGOV)
|
From page 681... ...
Many of the informal knowledge transfer mechanisms currently used on a small scale by implementing agencies, technical working groups, mission teams, and implementing partners could be useful tools for more formal scale-up and routine use across PEPFAR to enhance knowledge transfer and overcome barriers in the current systems. Conclusion: Although a wide variety of mechanisms have been successfully established and used to transfer an array of knowledge throughout PEPFAR, more progress is needed to address limita tions in current processes and systems and to establish formalized mechanisms to transfer experiences across countries, implementing
|
From page 682... ...
This implementing partner shares knowledge acquired and created by the organization through both active and passive knowledge transfer systems, including a sophisticated database reporting system, reports and periodic bulletins, dissemination meetings, feedback meetings, conferences, and webinar series. PEPFAR'S KNOWLEDGE DISSEMINATION EXTERNAL TO PEPFAR As described earlier in the chapter, PEPFAR acquires and creates vast amounts of knowledge in the form of routine PEPFAR program monitoring data, additional program and clinical data collected by implementing partners, outcomes and results of evaluation and research activities (TEs, PHEs, basic program evaluations, IS, and impact evaluations)
|
From page 683... ...
Additionally, the organization HQ produces easy-to-read facility- and region-level feedback reports used for guiding program implementation. Meetings Organization X uses several types of meetings to transfer tacit and ex plicit knowledge across implementing partners and sites, including but not limited to: • Annual meetings of in-country teams, implementing partners, and sites for the purpose of sharing data, lessons learned, innovations, and tools across implementing partners and countries.
|
From page 684... ...
Webinars are archived online and available for viewing. • The organization HQ distributes a periodic electronic bulletin that is sent to implementing partners and sites to highlight aggregate reported program data and illustrate the utility of routinely collected data.
|
From page 685... ...
Reporting to Congress OGAC reports a portion of the PEPFAR program monitoring data and updates on progress to Congress on a routine basis through annual reports to Congress as well as ad hoc reports that meet special congressional or White House requests (NCV-2-USG; NCV-3-USG)
|
From page 686... ...
One HQ interviewee said that PEPFAR program monitoring data are circulated to some external HIV/AIDS partners by indirect means when these data are used to contribute to partner country national data, which are then reported to the United Nations (UN)
|
From page 687... ...
involved in the HIV/AIDS response. Knowledge Sharing with Partner Country Governments At the country level PEPFAR knowledge dissemination occurs when mission teams and implementing partners share knowledge with partner country governments through meetings and presentations as well as through routine and ad hoc reporting.
|
From page 688... ...
PEPFAR implementing partners routinely report data to partner government systems. Interviewees from mission teams in several countries described how implementing partners are required or encouraged to report and share PEPFAR program monitoring data directly with partner country governments at the national, regional, and district levels as well as with government agencies such as the ministry of health, national AIDS commissions, and provincial AIDS commissions (116-1-USG; 196-22-PCGOV; 272-36-USG; 331-34-USNGO; 496-19-USG; 461-18-USG)
|
From page 689... ...
. Although several mission team interviewees reported that implementing partners are required or encouraged to report program monitoring data to the national M&E systems (116-1-USG; 272-36-USG; 331-34-USNGO; 496-19-USG; 461-18-USG)
|
From page 690... ...
Conclusion: OGAC could contribute to increased coordination among partners in the HIV/AIDS response by developing official routine and systematic mechanisms for knowledge exchange with other partners involved in the response at both the global and country levels, including partner country governments, other do nors, and multilateral organizations. PEPFAR Knowledge Dissemination to the Public and Contribution to the Global Knowledge Base As one of the largest funders addressing the global HIV/AIDS epidemic, PEPFAR has both the capacity and the responsibility to play a significant leadership role in ensuring that knowledge created through the HIV/AIDS response is disseminated broadly for the greatest public health impact and that data, research results, and evaluation outcomes are available to other researchers, evaluators, and the public to help accelerate the pace of new knowledge creation.
|
From page 691... ...
Public Knowledge Dissemination Mechanisms PEPFAR stakeholders at the HQ level and the country level use various platforms to disseminate knowledge created in PEPFAR to the public and to contribute to the global knowledge base on effective HIV/AIDS interventions and program implementation; these platforms include online technology (NCV-7-USG; NCV-10-USG; NCV-11-USG; NCV-18-USG)
|
From page 692... ...
. The Health Affairs special issue "Assessing the President's Emergency Plan for AIDS Relief," released July 2012, examined PEPFAR successes, lessons learned, and next steps and the Journal of Acquired Immune Deficiency Syndromes
|
From page 693... ...
. In addition, HQ- and country-level interviewees described other dissemination of PEPFAR data, activities, research results, and evaluation outcomes through publications in professional journals (NCV-4-USACA; NCV-8-USACA; 272-22USG; 272-27-USG; 331-23-USNGO; 196-11-USNGO)
|
From page 694... ...
To assess the scale of PEPFAR's contribuUSG; 396-12-USG; 396-1920-USG; 396-53-USNGO) tion to the global knowledge base, the committee requested from each of the four Track 1.0 partners, as well as from OGAC, CDC, and USAID, a list of publications resulting from PEPFAR support; several of the entities noted that the lists they provided were not completely comprehensive back to the beginning of PEPFAR.
|
From page 695... ...
Measuring and Tracking Contributions to the Global Knowledge Base As previously noted, PEPFAR implementing agencies and partners have created and disseminated vast amounts of knowledge since the inception of the program, yet it is difficult to assess PEPFAR's contribution to the knowledge base because of the lack of a centralized system or approach to track publications, abstracts, guidelines, and reports that result from PEPFAR funding. In order to assess the scale of PEPFAR's contribution to the global knowledge base on effective HIV/AIDS interventions and program implementation, the committee requested a comprehensive list of publications that resulted from PEPFAR support from the beginning of PEPFAR.
|
From page 696... ...
. In addition to the lack of a system to track PEPFAR-supported publications, PEPFAR lacks a central repository or tool that the international HIV/AIDS community (e.g., partner countries, other funders, researchers, evaluators, and the public)
|
From page 697... ...
. As a result of PEPFAR's efforts, OGAC, implementing agencies, and implementing partners have successfully contributed vast amounts of evidence to the global knowledge base on effective HIV/AIDS interventions and program implementation in the form of publications, reports, guidelines, tools, and participation in conferences.
|
From page 698... ...
Furthermore, as previously mentioned, when the committee requested PEPFAR program monitoring data from OGAC for use in this evaluation, there were no data beyond the seven key indicators that are reported annually to Congress that were readily available. Next Generation Indicator (NGI)
|
From page 699... ...
Addition ally, PEPFAR implementing agencies and partners have contributed evidence and vast amounts of publications to the global knowledge base on effective HIV/AIDS interventions and program implemen tation. Despite these successes, more progress is needed in dis seminating knowledge external to PEPFAR, particularly in sharing knowledge with partner country governments and other partners involved in the HIV/AIDS response, increasing the amount of PEPFAR data (e.g., routinely collected program monitoring data, evaluation outcomes, and research results)
|
From page 700... ...
to drive program activities and inform efforts, and at the partner country level there are good examples of data use by PEPFAR stakeholders. Additionally, PEPFAR has carried out initiatives to build capacity and increase data use among partner country governments and PEPFAR implementing partners, contributing to the fostering of a culture of evidence use among partner countries.
|
From page 701... ...
Finally, PEPFAR reporting requirements place a large administrative burden on implementing partners and mission teams, which detracts from their ability to analyze and use data. Further modifications are needed to improve harmonization with global indicators and to better align with partner country systems in order to further reduce the number of PEPFARspecific indicators, reduce reporting burden, increase data use by PEPFAR partner countries, and, ultimately, contribute to country-owned HIV/AIDS responses.
|
From page 702... ...
PEPFAR has made progress in disseminating knowledge external to PEPFAR, but more progress is needed to maximize the impact of knowledge created in PEPFAR, particularly by sharing knowledge with partner country governments and other partners involved in the HIV/AIDS response, tracking and measuring PEPFAR's contribution to the global knowledge base, and increasing the availability of data collected with PEPFAR funds (routinely collected and reported program monitoring and other data from implementing partners and contractors, evaluation outcomes, and research results) for use by external researchers, evaluators, and other interested parties.
|
From page 703... ...
Support for epidemiological data collection through surveillance and special studies in partner countries has been a cornerstone of PEPFAR's contribution and should continue to be a critical component of knowledge management activities to support joint planning with partner countries. Additionally, as one of the largest funders addressing the global HIV/ AIDS epidemic, PEPFAR has both the capacity and the responsibility to play a significant leadership role in ensuring that knowledge created through the HIV/AIDS response is widely disseminated and available to outside partners, researchers, evaluators, and the public to spur innovation, accelerate the pace of knowledge creation, and maximize the public health impact of interventions.
|
From page 704... ...
increases the acquisition of knowledge external to PEPFAR, (3) improves the efficiency and effectiveness of knowledge transfer within and external to PEPFAR, and (4)
|
From page 705... ...
should develop a comprehensive knowledge management framework, including a program monitoring and evaluation strategy, a prioritized and targeted research portfolio, and systems for knowledge dissemination. This framework should adapt to emerging needs to assess PEPFAR's models of implemen tation and contribution to sustainable management of the HIV response in partner countries.
|
From page 706... ...
FIGURE 1 PEPFAR Knowledge Management Framework 706 EVALUATION OF PEPFAR Improve and Evolve Disseminate se Acquire U Transfer Create Synthesize FIGURE 11-9 Suggested elements of a PEPFAR comprehensive knowledge management framework.
|
From page 707... ...
Create knowledge through tacit knowledge experiences, routine program monitoring, within PEPFAR frequent program evaluation, periodic outcome and impact evaluations at different levels of PEPFAR implementation, and targeted, purposeful research activities. Synthesize Across all levels of PEPFAR, perform iterative Synthesize diverse syntheses of acquired and created knowledge to knowledge streams inform decision making and action.
|
From page 708... ...
o ier 3: Indicators selected by implementing partners to monitor T and manage program implementation and effectiveness that are not routinely reported to mission teams. Implementing partners should select appropriate indicators defined in OGAC guidance
|
From page 709... ...
FIGURE X PEPFAR Information Management Global PEPFAR Program Reporting Monitoring Reporting System Systems Tier 1 Data • Small set of key indicators reported Office by mission teams to OGAC of the • Aggregated across countries to Global AIDS monitor performance across whole Coordinator of PEPFAR and report to Congress (OGAC) Data Feedback Tier 2 Data Data Reporting Mechanism • Larger number of indicators reported by implementing partners Mission Teams to mission teams • Used by mission teams for monitoring the effectiveness Data of the in-country response and Data Reporting Feedback accountability to partner countries Partner Mechanism Country Health Implementing Partners Tier 3 Data Information • Broad set of Indicators collected and used by Systems implementing partners • Used to manage program implementation FIGURE 11-10 Recommended PEPFAR tiered reporting in the context of partner country and global reporting systems.
|
From page 710... ...
o GAC should create mechanisms for implementing partners, O mission teams, and agency HQ to mutually contribute to a periodic review across all tiers of indicator development, ap plicability, and utility and to make modifications if necessary. o ier 1 indicators should be harmonized whenever possible and T appropriate with existing global indicators and national indica tors.
|
From page 711... ...
Further considerations for implementation of Recommendation 11-1B: Research • OGAC should clearly define which activities and methodologies will be included under the umbrella of PEPFAR-supported research as distinguished from program evaluation. • OGAC should draw on input from implementing agencies, mis sion teams, partner countries, implementing partners, the SAB, and other experts to identify and articulate research priorities and appropriate research methodologies.
|
From page 712... ...
The purpose of this policy would be to ensure that, within a purposefully and reasonably defined scope, specified program monitoring data and financial data, evaluation outcomes, and research data and results generated with PEPFAR support by contractors, grantees, mission teams, and USG agencies be made available to the public, research community, and other external stakeholders. OGAC and the PEPFAR implementing agen cies should consult with both internal and external parties who would be affected by this policy to help identify the data that are most critical for external access and that can be reasonably subject
|
From page 713... ...
o developing the policy and specifying the scope of data to In be included, several key factors and potential constraints that can affect the implementation of the policy will need to be ad dressed. These include patient and client information confiden tiality; the financial resources, personnel, and time needed to make data available; and issues of data ownership, especially in the context of increasing responsibility in partner countries and the provision of PEPFAR support through country systems or through activities and programs supported by multiple funding streams.
|
From page 714... ...
for strengthening program implementation through the President's Emergency Plan for AIDS Relief (PEPFAR)
|
From page 715... ...
2002. Synergizing the learning organization and knowledge management.
|
From page 716... ...
2008b. The President's Emergency Plan for AIDS Relief: FY2009 country operational plan guidance.
|
From page 717... ...
2011h. The President's Emergency Plan for AIDS Relief: FY2012 country operational plan guidance appendices.
|
From page 718... ...
1999. Knowledge management: The next fad to forget people.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.