4
Ways of Addressing Current Data Limitations
Nancy Birdsall (Center for Global Development) served as moderator for this session. She began by underscoring how much the discourse on
population and family planning issues has evolved since the 1970s. Birdsall said that this session would set a standard for future research on these topics based on better data—perhaps of a longitudinal, collective-level, or aggregate nature.
WOMEN’S ECONOMIC EMPOWERMENT AND COVID-19: RESULTS FROM PERFORMANCE MONITORING FOR ACTION
Philip Anglewicz (Johns Hopkins University) noted that shocks, such as climate disasters or pandemics, have a profound impact on people’s health, well-being, and economic status. Shocks also change household bargaining dynamics and may influence women’s economic empowerment. The COVID-19 pandemic is a prime example of a shock with the potential to influence these dynamics. Anglewicz provided background on the COVID-19 pandemic in sub-Saharan Africa, where more than 1 million confirmed cases and more than 23,000 deaths were reported by mid-August. Many regional governments quickly enacted travel bans, mobility restrictions, limitations on public gatherings, and other measures to prevent the spread of COVID-19; consequently, greater than 75 percent of households monitored by the Performance Monitoring for Action (PMA) project lost at least some portion of their income, while 30 percent lost all income. The PMA project has been adapted to leverage previous and future waves of data collection to explore the impact of the COVID-19 shock.
PMA operates in eight countries: Burkina Faso, Côte d’Ivoire, Democratic Republic of the Congo (DRC), India, Kenya, Niger, Nigeria, and Uganda (Anglewicz notes that a PMA project unrelated to this presentation is also under way in Ethiopia). The PMA project in these countries recently shifted from monitoring by repeat cross-section to using a longitudinal-panel design, in which the same women are monitored and interviewed over time in three waves, between 2019 and 2022. During June and July of 2020, a phone survey was conducted on the topic of COVID-19 in four countries for which baseline data had already been collected: Burkina Faso, DRC, Kenya, and Nigeria. This phone survey measured COVID-19 awareness, media exposure, risk perception, knowledge, behavior change, and socioeconomic impact, as well as the disease’s impact on family planning, contraceptive use, and fertility intentions. Interviewers remained constant across waves when possible, and responses were recorded on smartphones via Open Data Kit. The Kenya and Burkina Faso samples are nationally representative, while the DRC sample is restricted to Kinshasa and the Nigerian sample is representative of Kano and Lagos states (although only data from Lagos are analyzed here). Follow-up response rates were high in all samples (74.7 to 93.9%).
Anglewicz focused on a commonly used measure of household decision-making to examine women’s economic empowerment in these samples over
time and after the emergence of COVID-19. The same question was posed to respondents on both the baseline and follow-up survey: “Who usually makes decisions about making household purchases for daily needs: you, your husband/partner, you and your husband/partner jointly, or someone else?” Multinomial logistic regression was performed to analyze characteristics associated with changes in decision-making roles, with a focus on women who made decisions at baseline and assessing the degree of change in that role; characteristics included in the analysis were modern contraceptive use, parity, age, education, household size, wealth tertile, job participation in the previous seven days, and household economic impact of COVID-19.
The analysis revealed that the percentage of households in which women were the decision-makers shrank from baseline to follow-up across the samples, while partner or external decision-making rose. Anglewicz shared results from additional analyses based on who made household decisions at baseline to interrogate the stability of these decision-making relationships and impacts of COVID-19. In households where women made decisions at baseline, approximately one-quarter of those women retained that role in Kenya, Burkina Faso, and Nigeria; when the role did change, it was more likely to shift toward joint decision-making than external or partner decision-making. Women’s decision-making was more stable in DRC, where more than one-half of the women retained their role as the primary decision-maker after the pandemic began; however, when roles did change it was more likely to shift toward partner or external decision-making than a joint process.
In Kenya and Burkina Faso, the decision-making role was more stable at follow-up among households in which decisions were made externally by a partner or someone else at baseline than among those in which women made decisions at baseline; when the roles did change, decision-making was more likely to become a joint process than to shift to the woman. In DRC, however, the most frequent scenario at follow-up in these households was for external or partner decision-making to shift to the woman entirely. Joint decision-making was the most stable scenario from baseline to follow-up; if roles changed in these households they were more likely to shift toward partner or external decision-making in Kenya, Burkina Faso, and Nigeria, and were equally likely to shift toward the woman or a partner in DRC.
The analysis also revealed some characteristic patterns related to decision-making dynamics across countries. For example, older women (ages 25–49) were more likely to retain decision-making power than younger women (ages 15–24). In addition, in households that experienced complete or partial income loss following the emergence of COVID-19, decision-making was more likely to shift to a joint process in Burkina Faso and to partner or external power in DRC. In Kenya and Burkina Faso,
larger households were more likely to shift away from women holding decision-making power from baseline to follow-up.
These data highlight a fair amount of change in household decision-making power as a result of COVID-19, in which the overall pattern indicates a shift away from women making decisions about daily purchases and toward a joint decision-making process, or toward decision-making by the husband or other household member. Joint decision-making appears to be the scenario experiencing the least change from baseline, while women making decisions by themselves is the most vulnerable to change over time, with teens and young adults experiencing the most change. Anglewicz said that the next measure to be analyzed from the PMA survey assesses whether women’s economic reliance on husbands or partners has increased following the implementation of COVID-19 restrictions in their countries. Analyses will also be stratified by baseline wealth quintile and COVID-19 economic impact, and community-level measures of economic development, financial literacy or resources, and women’s employment will be added.
Anglewicz said that these COVID-19 data represent only one piece of the intended output from the PMA research. The broad range of women’s economic empowerment measures collected at baseline encompasses numerous indicators of economic advancement, power, and agency and will be linked to data from future collection timepoints to produce a longitudinal view of the landscape of women’s economic empowerment in these countries. After the second wave of data has been collected, the data will be used to evaluate causal relationships between contraceptive use and women’s economic empowerment, as well as the factors that predict which women will gain or lose empowerment over time.
In the discussion, Anglewicz received several questions pertaining to the phone-based survey mode of the PMA study. Aletheia Donald inquired whether the study tracked whether the call was private or on speaker phone when women were contacted for the follow-up survey. Anglewicz clarified that women were asked at the onset to not use a speaker phone and to try to find a place to respond to the survey privately. Alex Ezeh asked how the change in survey mode from in-person (baseline) to phone (follow-up) may have affected the observed changes. To this point, Anglewicz responded that in most cases the surveys were conducted by the same interviewer at baseline and at follow-up to establish a rapport between the questioner and the respondent and to yield better quality data despite the change in survey mode. Furthermore, the influence of using phones as a survey mode was specifically assessed in other data collected in Burkina Faso, and results suggested that the use of phone surveys did not have a large impact on responses of interest to PMA.
Win Brown (Bill & Melinda Gates Foundation) commented that a methodological challenge in this study is the possibility of selection bias
due to characteristics associated with phone ownership and the potential need to apply sampling weights accordingly. Anglewicz acknowledged that PMA addressed this challenge by using inverse probability weighting. In this method, the inverse of a score based on the estimated likelihood of women owning a phone (based on certain characteristics) is used to generate a weight, which is then aggregated with weights designed to account for sampling and nonresponse bias. Anglewicz likened the procedure to propensity score matching. After weights were applied, the distributions of sociodemographic characteristics between the weighted and baseline samples were compared to confirm that the weights were effective. As a caveat, Anglewicz noted that women who do not own phones are systematically different from women who do own phones in ways that may be unobservable and unaccounted for by weighting methods. However, based on observed characteristics, it appears that this adjustment was successful in limiting bias.
Anita Raj noted the difficulty in interpreting responses to questions about decision-making and asked whether either joint decision-making or women’s unilateral decision-making is better than the other. Anglewicz responded that the answer is likely context-dependent and that more data—particularly longitudinal data that include outcome measures—are needed.
Raj also asked how the researchers can be sure that the observed changes were due to a COVID-19-associated shock rather than a testing effect or other factor. Anglewicz said that the next wave of data will shed light on this question. In addition, data on economic impact intensity and fear of COVID-19 infection can be used to assess overall COVID-19 impact, and PMA plans to analyze variation in changes over time for associations with COVID-19 impact.
Jocelyn Finlay suggested that the intersection of domestic violence outcomes and decision-making would be an interesting dimension to add to this study. Anglewicz responded that domestic violence was not addressed at baseline but that PMA will add a domestic violence module to upcoming survey waves.
MEASURING GENDER EQUITY IN FAMILY PLANNING: LEARNINGS, EXPERIENCES, AND THE WAY FORWARD
Anita Raj (University of California, San Diego) presented an overview of how gender equity measures are used in the context of family planning by the project known as EMERGE (Evidence-based Measures of Empowerment for Research on Gender Equality). The EMERGE Family Planning Project operates under the umbrella of the EMERGE Project, which examines measures used in gender equality and empowerment research more broadly. Its goals were threefold: to identify constructs and measures of
agency and norms that are currently used in family planning research, to evaluate the quality of these measures, and to determine outstanding needs in the field.
The EMERGE Family Planning Project has three phases. In Phase 1, which was performed in January 2020, key informant interviews were conducted with SRH experts (n=40) who had developed and operationalized measures for family planning research. In Phase 2, completed in July 2020, a literature review was performed to assess measures from 152 papers across multiple disciplines for psychometrics, cross-national validation, and ease of use. Raj noted that a white paper was produced based on Phases 1 and 2, and that people interested in learning more about the methodologies of this work can find information in that publication.1 In Phase 3, a conference of experts in the field was held on September 9, 2020, to discuss the results of Phases 1 and 2, as well as future directions for the field; Raj remarked that several speakers at this Virtual Workshop were present at that conference and that the conference will lead to a position paper.
The EMERGE Family Planning Project employed an empowerment framework influenced by psychology, sociology, and political science to conceptualize family planning goals and used measurement constructs that focused on norms, choice, and agency. Raj emphasized that empowerment is a process that pushes against power structures and is rooted in self-determined action, such that actions are empowered as long as they align with the values and preferences of the person performing the actions and especially if they challenge the norm. To this end, Raj cautioned that researchers must be cognizant of the fact that family planning has roots in population control and eugenics, lest the field slip backwards into this dangerous territory by being overly prescriptive regarding what people’s choices “should” be.
The project identified several measures of agency that perform well in family planning research, including measures related to contraceptive self-efficacy, sexual communication self-efficacy, and reproductive decision-making autonomy. Raj highlighted that decision-making measures that capture nuances of the decision-making process, such as a woman’s desire to make a decision or what happens when disagreements occur, are newer and provide more clarity about women’s agency in decision-making. Other high-performing measures currently in use include those that assess response to agency (e.g., male engagement in family planning communication, reproductive coercion) and those that assess family planning stigma
___________________
1 N. Bhan, E. Thomas, A. Dixit, et al., Measuring Women’s Agency and Gender Norms in Family Planning: What do we know and where do we go? (EMERGE [Evidence-based Measures of Empowerment for Research on Gender Equality], Center on Gender Equity and Health, 2020).
or rights (e.g., concerning abortion, adolescent SRH). All of these helpful measures tend to be brief measures that are used in cross-national contexts.
Raj introduced several measures that are promising but need strengthening. Often these measures are in early stages of testing or have been used in single-nation contexts, but they are nevertheless psychometrically rigorous. These promising measures are most commonly related to social norms and quality of care in family planning counseling. Social norm measures include measures of gender norms, male engagement in family planning norms, family or community engagement in FP norms, and fertility and contraceptive use norms. Measures of quality of care in family planning counseling include provider respect, coercion, and effects of incentives or targets on provider behaviors. Raj acknowledged that family planning providers—who are often women—who engage in disrespectful or coercive care are commonly bullied or themselves less respected, which itself is an important area of study.
Critical gaps in family planning research measurements were also identified in the EMERGE Family Planning Project. Measures of agency in fertility were among these gaps, because these measures often focus on agency in contraceptive use but fail to capture agency in reproduction. Contraceptive discontinuation measures appear to lack the nuance required to assess ambivalence or lack of planning. Measures that illustrate forms of resistance (e.g., covert use of contraception), sexuality for pleasure, and critical consciousness and choice (i.e., attributes of choice) were also described by Raj as underdeveloped or lacking in the family planning field. Experts consulted by the EMERGE Family Planning Project pointed to agency in relation to fertility and mistreatment or pressure in services as two of the highest-priority gaps to address in the realm of family planning measures. Raj ended by underscoring that the project suggested to her that there was a consensus in the field concerning what family planning is, namely that it is not contraception, and that the two terms must not be used interchangeably. In addition, Raj stated that empowerment should be the fundamental lens through which the entire community understands family planning in order to improve research surveys and measures.
In the discussion, Jocelyn Finlay asked whether the term “family planning” is even necessary, noting its unhelpfulness for adolescents and youths and for women who are ambivalent on the subject of family planning. Raj replied that this question arose during the September conference of the EMERGE Family Planning Project and that experts in the field support retiring use of the term “family planning.” However, Raj noted that it is challenging to change widely understood terminology, and she reiterated her desire to, at a minimum, see an end to the use of the term as synonymous with contraceptive use.
THE DEMOGRAPHIC AND HEALTH SURVEYS: DATA, GAPS, AND SOME THOUGHTS
Sunita Kishor (ICF) explained that the Demographic and Health Surveys (DHS) Program is a key source of data on gender, women’s empowerment, and reproductive health on an international scale. The DHS Program is a U.S. Agency for International Development (USAID)-funded global project that provides technical assistance to lower- and middle-income countries to collect, disseminate, analyze, and act on data related to demographics, health, and nutrition on the national and subnational levels. Launched in 1984, the DHS Program has conducted more than 400 surveys in 90 countries. Kishor said that because one of the program’s key objectives is to collect high-quality data that are comparable across countries and over time, questionnaires are designed according to the guiding principle of comparability, although country-specific adjustments are also made.
DHS household surveys include five questionnaires: household, woman, man, biomarkers, and fieldworker. Kishor remarked that the DHS questionnaire for men is largely a subset of the woman’s questionnaire; while both questionnaires collect information on topics including number of children, contraception, fertility preferences, and gender and women’s empowerment, the woman’s questionnaire includes additional questions questions related to birth (and now pregnancy) history and key indicators of maternal and child health.
Kishor presented an overview of empowerment data collected by the DHS Program since its inception. Between 1989 and 1993, the woman’s questionnaire was expanded to ask gender and employment questions. Kishor herself developed the Women’s Status Module, which was developed and piloted in Egypt in 1995. Around the year 2000, many empowerment variables were introduced into the core questionnaire, and optional modules on empowerment and domestic violence were developed. The number of gender-related questions in both the woman’s and man’s core questionnaires—as well as the number of optional modules—has continued to increase over time.
DHS questions on empowerment are designed to capture not only evidence of empowerment itself, meaning acts, attitudes, or behaviors that reveal agency, but also the contexts or settings in which women live that foster or detract from empowerment. These contextual variables include, for example, living in a household with educated members and freedom from violence, and can be used in conjunction with many indicators of evidence of empowerment itself, such as control of earnings, ownership of bank account, internet use, and control of sex.
Kishor remarked that in addition to data on empowerment, the DHS Program provides a wealth of data on other relevant topics, such as fertility
and family planning. However, Kishor also recognized areas in which the DHS Program can improve its measurement and conceptualization of empowerment. A long-standing gap that DHS data does not fill is the need for longitudinal data on empowerment that can describe the causes and process of empowerment. Kishor noted that the relevance of data collected on empowerment also varies across cultures and life stages. Although the DHS Program uses country-specific indicators to account for cultural differences, the data collection needs to be more sensitive to a woman’s age and marital status through the use of separate questions and methods; Kishor pointed to the fact that many DHS variables are currently applicable only to married women.
Kishor said that the DHS Program needs to collect more qualitative data to complement quantitative variables. These qualitative data will enable researchers to better decode answers related to agency, for example answers to the question, What does it mean to make a decision alone? And the question, How much value do joint actions have as evidence of empowerment? Kishor further highlighted gaps in the field’s current understanding of empowerment measures, including how these empowerment measures interact with one another. Two examples of such interaction are the intersection between gender attitudes and decision-making roles, as well as between employment as empowerment and who controls earnings.
Similarly, Kishor questioned whether contraception and empowerment can be effectively studied together given the wide variation in how they relate to each other throughout the world; in some countries the use of contraception is itself a measure of empowerment, while in others it may be entirely unrelated to empowerment. Kishor suggested that this fluid relationship underlies the rarity of observing a consistently positive relationship between contraceptive use and women’s empowerment across studies. Alternative “outside the box” measures of empowerment may address some of these gaps. Kishor cited leisure (or self-indulgence) as one such potential alternative measure, because it is a sign of self-worth and “true freedom.” Other nontraditional indicators that may be developed to measure empowerment include hours of sleep, experience of hunger relative to men’s experience, and empowerment in settings outside the household.
Kishor concluded the presentation with three key points. First, she emphasized that choice is a fundamental aspect of empowerment and challenged others to consider whether the path chosen matters for empowerment (e.g., having a sex-selective abortion, not using contraception when a child is not desired). Second, she questioned how many sets of empowerment indicators are needed to determine whether a woman is empowered, given the fact that evidence variables will differ by domain of study, location, and life stage. Finally, she posed two rhetorical questions: What does successful women’s empowerment research look like? and Will we know an
empowered woman when we see one—particularly in unfamiliar cultural settings? To this end, Kishor stressed the need for greater comprehension of the multidimensionality of empowerment.
During the discussion, Jocelyn Finlay asked whether researchers can propose new DHS measures to refine the work module or other survey components. Kishor said that accommodating new ideas and emerging needs is a priority for the DHS Program, and thus the questionnaires are revised approximately every five years. The program sought public feedback for the most recent revision. Although revisions are more difficult to incorporate when surveys are under way, some changes can be made at that time, particularly when a country specifically asks for an update based on its needs.
Finlay also asked whether longitudinal data can be enabled through more frequent data collection by phone or tablet. Kishor acknowledged that one lesson people are learning during the COVID-19 pandemic is how to conduct more business remotely, and that the DHS Program could look at survey approaches similar to the PMA study presented by Philip Anglewicz, in which after an in-person interview has been conducted follow-ups may be pursued by phone or with other technologies. However, Kishor said that the DHS is unlikely to ever be entirely conducted by phone because of its complexity and the sensitive nature of some of the content covered.
