Demographic Assessment Techniques in Complex Humanitarian Emergencies: Summary of a Workshop (2002)

The term complex humanitarian emergency refers to a situation in which a civilian population is displaced from their homes by war or conflict. In addition, there is often a deterioration of living conditions and sometimes a significant increase in mortality, either in the short or long term.¹

One of the primary and earliest challenges in a complex humanitarian emergency is to obtain accurate estimates of displaced populations and their mortality rates. Although many basic guidelines exist to help field personnel in demographic assessment techniques, the quickly changing and generally unstable context of a crisis can make it difficult to implement a rapid assessment. The differing causes and demographic effects of emergencies— refugees or internally displaced persons,² rapid onset or slow build-up, and mortality caused primarily by disease or mortality primarily due to human rights abuses—make a one-size-fits-all approach problematic. In addition, field staff may not be knowledgeable about the latest innovations in demographic techniques, which are continually being updated and improved.

The Roundtable on the Demography of Forced Migration under the aegis of the Committee on Population of the National Research Council held a workshop to examine some of these topics. This report to the roundtable summarizes the presentations and discussion, but does not make any attempt to provide guidelines, conclusions, or recommendations for researchers or policy makers.³ The report generally follows the order of the

presentations and discussion listed in the workshop agenda (see the appendices).

The phrase “demographic methods” refers to techniques for measuring population size, structure, distribution, characteristics, and rates. In complex humanitarian emergencies, these methods are used for the purposes of early warning, intervention planning (assistance to and protection of the population), monitoring and evaluation of these interventions, and baseline surveillance. Assessment techniques may be used at different points throughout a crisis and by international organizations, nongovernmental organizations (NGOs), government agencies, academics, or refugees themselves. There are a variety of settings and types of populations (e.g., refugee camps, self-settled refugees,⁴ internally displaced persons) in which demographic assessment is needed. Therefore, the array of techniques and methods that may be used is vast. The workshop aimed to improve their application to estimate vital population parameters by reducing sampling (selection of individuals) and nonsampling (selection of information) error.

The workshop began with a presentation by W. Courtland Robinson of Johns Hopkins University about the basics of demographic assessment in emergencies. At least three major organizations have published guidelines on how to conduct initial demographic assessments: Médecins Sans Frontières (MSF), the United Nations High Commissioner for Refugees (UNHCR), and the Sphere Project.⁵ Each of these sets of guidelines varies slightly, but their purpose is the same—to help field workers obtain a baseline estimate of the total population and its vital characteristics.

As Robinson explained, a variety of techniques are used individually or in combination to estimate total populations of displaced persons and their characteristics. Simple counting of individuals or shelters, administrative records, community estimations, mapping (either manually or using geographic positioning systems), aerial photography, screening of children under age five or vaccination surveys and consequent extrapolation, household surveys, and camp registrations or censuses are some of the many methods available. Rates—including mortality, fertility, and migration rates—can be estimated in these settings using the methods listed above, in combination with registration and medical records, sample surveys, and counting burial sites or bodies (for mortality rates). The rest of this report examines some of the advantages and disadvantages of a few of these meth

ods, considers where and when they are most useful, and discusses possible refinements.

When data are poor and conditions are difficult, there are two principal approaches for obtaining demographic estimates. First, consistency checks are a useful tool; one can compare two data sets for the same population, compare the data for the area of interest with data from a model or a neighboring area, or examine data for internal consistency. For example, are the numbers or percentages of men and women or of different ages similar to other estimates for the population or what one would expect?

Second, indirect estimation techniques have been developed by demographers to produce estimates of a demographic parameter on the basis of information that is only indirectly related to its value. Indirect estimation is commonly used to obtain estimates of rare or difficult-to-measure demographic events, e.g., maternal mortality. Some examples include:

• using the proportion of children ever born to women aged 20-24 who have died by the date of the survey to estimate the probability of dying by age 2;

• using the incidence of orphanhood to estimate adult mortality;

• interviewing a migrant regarding the disposition of the household left behind; and

• interviewing survivors about the survivorship of their siblings.⁶

These indirect methods can be used to estimate demographic events among displaced populations or those they left behind when it is difficult to access actual data for the populations. For example, Robinson led a study in northern China that asked migrants from North Korea about mortality among the households of their relatives still in North Korea to estimate mortality due to famine.⁷

Social scientists, epidemiologists, and statisticians alike are familiar with the principles of sampling. Sampling is selecting a subset of the population of interest in order to gain information about the entire population. A good sample will therefore be representative of the population. The design for a

sample, however, varies greatly depending on the time, money, and staff available, the logistical challenges encountered, and the intended use of the data. The greater the precision required of the estimate, generally, the larger the sample size will need to be. The most basic type of sampling is called simple random sampling. A simple random sample, a type of probability sample, is one in which each member of the population has an equal probability of selection.

Cluster sampling is an approach in which each member of the population is assigned to a group (cluster) and then clusters are randomly selected and all members of selected clusters are included in the sample. Often combined with stratification techniques (in which case it is called multistage sampling⁸), cluster sampling is the approach most often used by epidemiologists. A statistical software package called EPI-Info (Centers for Disease Control and Prevention, Atlanta, Georgia, USA) is commonly used by field staff in emergency settings. However, in conducting statistical tests, most of the EPI-Info program assumes that it is working with simple random samples unless one specifically adds a variable to take into account the design effect of cluster sampling.⁹ Although adding a variable to adjust for the design effect is a way to correct for the effects of deviation from simple random sampling schemes (the CSAMPLE module in the EPI-Info program can correct for this), many field staff are not well versed in sampling techniques and therefore unaware of the need to correct for sampling design. This often results in artificially low estimates of the true underlying variance within a population.¹⁰ There was a lot of concern expressed at the workshop about potential misuse or misinterpretation of EPI-Info results. In addition, the standardized approach used by EPI-Info calculations for required sample size may not be appropriate for other situations (i.e., measuring human rights abuses that may be heterogeneous or rare, such as torture).

Paul Spiegel of the Centers for Disease Control and Prevention gave a presentation on the problems encountered when using cluster sampling for the estimation of populations in refugee camps or other displaced populations. Cluster sampling is appropriate for situations in which there is no readily available sampling frame¹¹ of individuals (such as a camp census list), but for which it is easy to obtain lists of subgroups or clusters of individuals, e.g., compounds, buildings, or tents. It is generally cheaper and quicker than nonstratified sampling.

The EPI-Info module called CSAMPLE is commonly used for immunization and nutrition surveys, based on a sample of 30 clusters, which is

the minimum number of clusters generally used for these types of surveys by field staff. In the first stage, probability proportional to size sampling (PPS)—a method that ensures that smaller clusters are not overrepresented in the sample—is used to choose clusters, but this can be problematic. First, estimated population sizes for each cluster in complex emergencies are generally not accurate. Second, displacement among subpopulations is unlikely to be proportional across a region or district because of the chaotic nature of forced migration and the nearly constant inflows and outflows of migrants in many situations. Thus the adjustment for the displacement is often imprecise. One approach to adjust for population displacement is to use the latest information available (e.g., food distribution censuses or other data) to update baseline population figures before sampling.

In the second stage, once sample clusters are chosen, households are chosen. At this stage, the sample of households may not be truly probabilistic; the sample may be biased toward households on roads or in the center of the settlement if interviewers are not careful in their household selection methods. The interviewers’ judgment on sampling households may affect the validity of results. Some potential solutions to these problems are to use systematic sampling—the selection of every nth household from a prepared list of households in the sample clusters—or to use maps to create a sampling frame of households. The accuracy of the parameter estimate being measured can be increased by increasing the number of clusters sampled. However, this method also increases the costs and time for survey, both of which are scarce in most emergency settings.

During the final stage of the sampling process, all those in the sampled households are selected or one member of each household is selected. There was some question among the participants whether individuals or households should be counted as the unit of analysis (see below).¹² Practical considerations can create limits as well; nonresponse can make it very difficult to reach one’s predetermined sample size. Nonresponse may also make statistical inference questionable.¹³

Donna Brogan of Emory University commented on some of the statistical issues surrounding the use of cluster sampling in refugee camps. First, one must decide whether the unit of analysis is the individual, the household, or the family; this in turn will guide the sampling process. Cluster sampling is often used because the World Health Organization (WHO) Programme on Immunization has popularized it (using CSAMPLE in EPI-Info for analysis), but it is not the only option, and the assumptions underlying it may not be appropriate for every circumstance. The WHO cluster

survey method assumes an equal probability selection of individuals, which may be an invalid assumption if cluster size estimates are inaccurate. In this case, weighted analyses can be done to obtain unbiased (or approximately unbiased) estimates on the basis of updated information on cluster size.¹⁴ When cluster sampling is used, 30 is the minimum number of clusters in the sample (not 30 individuals).

Sampling methods ultimately depend on the event to be measured and the criteria that are valued most by the researcher, such as costs, time, and sampling error. Unfortunately, the WHO cluster survey method, which was originally developed to measure vaccination levels in a stable population, has been used indiscriminately in situations involving forced migration. Different techniques and types of samples need to be considered in crisis situations, so it is important that field staff understand the limitations and potential weaknesses of the various methodologies.¹⁵

Spatial sampling—which is a variant of cluster sampling and is also known as area probability sampling—is the use of geographic area and population density to estimate population size or proportions, often using handheld global positioning systems (GPS) units or geographical information systems (GIS).¹⁶ This method can be used when maps and censuses of the geographic area under study do not exist. Vincent Brown of Epicentre/ MSF in Paris, introduced a method of sampling that is frequently used in very acute emergency situations. The quadrate method—as Brown called it—counts the population in small square blocks of equal areas. Blocks are randomly chosen within the refugee camp (or other defined area). The average sample data is then extrapolated to the level of the total camp population.

The first stage is to draw a map of the camp with a compass, measuring all sides of the camp by foot or with a vehicle odometer. The angles between each side are measured and then a map with a square grid superimposed on top is constructed. The second stage is to select a random sample of blocks (again, usually 30) and collect population data for all of the households in each block. The average population per block is calculated and extrapolated to obtain the total. There are many potential challenges with this method, including choosing the size of the blocks, the number of blocks to sample, and taking differing population densities into account. According to Brown, at the field level, these decisions have been based mainly on

common sense. Although this approach is an important tool that can be used systematically at the beginning of a crisis, further research is needed to improve the statistical validity of the method.¹⁷

Denis Coulombier of the Institut de Veille Sanitaire (the French National Institute for Public Health Surveillance) compared a method known as T-square estimation with the quadrate method. Although the quadrate method is the typical method used for estimating populations in disasters or forced migrations, the T-square method—a method that is often used in agronomy—is a potential alternative. It involves sampling a number of random points, measuring the distance between each point and the nearest household or family unit and then measuring the distance between that household and the next closest one. In this way, one can estimate population density. Coulombier and colleagues tested this method at a festival in France and the results were very comparable with results obtained using the quadrate method and using exhaustive entry registration. The table below compares the advantages and disadvantages of the two methods.

Participants were optimistic about the potential of using the T-square method in future situations. At the very least, it could be used as a rapid, low-cost estimation technique in the early stages of a crisis, and it is a good alternative to relying on politically biased estimates from governments or other groups. It also may be the least intrusive method of estimating a population, which could be helpful in particularly sensitive situations. Further testing is planned at voluntary mass gatherings.

Although qualitative methods are not the first to come to mind when thinking about how to estimate populations in emergency settings, they can offer some important insights for researchers throughout the various phases of a crisis. William Weiss of Johns Hopkins University gave a brief presentation on how qualitative techniques can assist those who are attempting to estimate forced migrant populations and their characteristics.

A qualitative method is a dynamic process that involves purposive or random sampling and generally small samples.¹⁸ Key characteristics are triangulation, flexibility, iterativeness, and open-ended inquiry.¹⁹ Qualitative research can help researchers reduce non sampling bias by using team members with different perspectives and by crosschecking sources.

Weiss argued that qualitative methods can give insights into such processes as food sharing, shelter building and sharing, credit transactions, and the use of health care services. They also allow refugees to be a part of the planning and problem-solving process. There are a variety of techniques that can be quite useful in emergency settings and refugee camps. For example, participatory mapping is a technique that has been used for assessment among refugees in Kenya. Houses are drawn on a map and then refugees are interviewed to learn more information about each household, such as the age and gender of the household members. A walkabout is a simple observation method in which researchers walk around a camp to observe the layout and amenities; it can be used to confirm maps and create checklists of program needs. Creating a timeline of refugees’ migration history can give insight into their personal history and health status, and can also later be used to draw insights about events and their consequences.

Commenting on the potential uses of qualitative methods in emergencies, Giovanna Merli of the University of Wisconsin, Madison, argued that they should not be thought of as separate from quantitative methods, because the two are complementary. In fact, the various methods can be seen as part of a continuum. Different methods can be used in tandem to validate findings, elucidate processes, and refine other methods. Merli noted that many issues could be addressed using qualitative methods, including vulnerability, poverty, population processes, resource distribution, reasons for migration, and—especially—security. Some demographers—typically thought of as quantitative researchers—are now developing mixed-method approaches that use both quantitative and qualitative methods. Researchers must be careful when using qualitative participatory methods, however, to

make sure that they truly understand the underlying demographic and social processes they are trying to measure. They should also be fluent in the language of the displaced and understand the political and social contexts of the population.²⁰

In sum, the best sampling or estimation method for a given situation is context-specific. Several participants suggested that it would be helpful to have a manual that gives some guidance on the different methods and approaches and their use in different phases of and in different types of complex humanitarian emergencies. It could serve as a common reference point for researchers and field staff involved in these situations.

Many of the techniques described above also can be used to estimate various demographic rates. In a crisis, one of the most important is the mortality rate. Epidemiologists conventionally use the crude mortality rate (CMR)—the number of deaths per 10,000 persons per day—as a rough indicator of a population’s overall health condition in an emergency situation. (Demographers use the crude death rate (CDR)—the number of deaths per 1,000 persons per year—but these are interchangeable and convertible from one base to another.²¹)

Assessing excess mortality, which is mortality above the normal mortality rate, is a primary goal during the beginning stages of a crisis. This measure gives aid workers an idea of the magnitude of the crisis. However, if a country has been politically unstable and impoverished for many years, data on “normal” mortality conditions may be difficult to obtain or unreliable.

Les Roberts of Johns Hopkins University described mortality surveys conducted in eastern Democratic Republic of Congo (DRC) using spatial sampling techniques.²² Two different methods were used, grid-based (quadrate) sampling and multistage cluster sampling. A very brief survey was used to try to determine mortality rates during a recent period of violence and displacement. The researchers encountered a variety of problems, including a large percentage of vacant residences (people had fled since the war began or whole households had been killed or died) and a number of security issues and geographical barriers.

Using a handheld global positioning system unit, the team selected clusters of households from clinic catchment areas (the areas that individual clinics are supposed to serve). Rural areas tended to be oversampled, so weighting was used to compensate, although comparing the weighted and unweighted results showed no difference in mortality rates. The team concluded that in eastern Congo there was no correlation between population density and mortality, and therefore most mortality was likely caused by violence, rather than communicable diseases, which spread more quickly among denser populations. Most mortality was due to the disintegration of the social fabric, which led to physical insecurity and ultimately deaths from violence. However, the small number of children in the population relative to a “normal” sub-Saharan African population was also a clear indicator of poor health conditions.

Roberts found that spatial sampling techniques tended to bias the sample toward large households or families, in comparison with cluster sampling methods, because large households are more likely to be captured since they physically take up more room than do smaller households. Nevertheless, the technique was sensitive enough to reveal that mortality during the period of study had tripled over the baseline mortality rate for the population under study (as measured at the beginning of the study). Urban displaced populations had higher mortality than rural displaced populations, perhaps because of urban-centered violence.

This study showed the change in mortality over time by comparing estimates at the end of the study to those at the beginning of the study (1999-2001). Commenting on Roberts’ presentation, Fritz Scheuren of the Urban Institute noted that it is important to distinguish between people who refuse to participate in a survey and those who cannot be reached and to use repeated measures to check the accuracy of estimates. One problem with mortality surveys is that respondents often have trouble with recall and may tend to displace deaths or situate them in the recall interval when they actually occurred in other periods not covered by the survey. This may be especially problematic if respondents are suffering from trauma. Overall, however, Scheuren felt that the survey was excellent considering the grueling circumstances in which it was conducted.²³

In a small group discussion, participants discussed the advantages and disadvantages of various sampling methods for estimating mortality rates. The group agreed that epidemiologists could benefit substantially from consultations with sample survey statisticians, rather than using standardized methods in every crisis. There should be more data management in the

field, and surveillance techniques need to be adapted for use in open populations (e.g., self-settled refugees). If time and resources make it feasible, an event history should be collected from each person in a survey, using a memorable point in the past as a starting point (such as a national or religious holiday).²⁴ Indirect methods are helpful for reducing bias, but they have so far been underused in emergency settings. Although there are two different types of data needs in crises—immediate and historical—a hybrid type of estimation method can address both needs.

Statisticians pointed out that some epidemiologists currently do not keep track of those in the sample who are not reached (noncontacts), but simply keep sampling in order to reach a quota of households. The noncontact rate should actually be estimated in advance and used to calculate the total sample size needed, with no substitutions allowed for noncontacts. This approach obviously creates practical problems because the staff needed to carry out surveys with large sample sizes in the field is lacking. Yet reporting the noncontact, nonresponse, and vacancy rates when a study is published would help to deflect some criticisms of the methodology and hence, of the estimates themselves.

Human rights abuses—violations of international standards of human rights, such as kidnapping, rape, torture, and genocide—are traditionally documented through testimonies and clinical or forensic evidence. Now these methods are also being used in combination with epidemiological and demographic methods. Cluster sampling is not appropriate for measuring mortality in cases of human rights abuses because they are not random events. Some nonprobability sampling techniques that might work in these situations include: network sampling, in which more than one individual can report on another individual in the survey, multiplicity or “snowball” sampling, which uses a process of chain referral by members of the population of interest, and adaptive sampling, which is a technique often used for studying disease patterns among wildlife in which if a diseased animal is discovered, then additional animals are sampled in the same area.

Some types of probability sampling may still be useful for measuring human rights abuses, but it depends on how much knowledge a researcher has about the probability of the events he or she is trying to measure. Finally, those who perpetrate human rights abuses sometimes record demographic data that can later be used by human rights investigators and crimi

nal tribunals or truth commissions. Collecting and working with this type of data may be more akin to working with administrative data for historical demographic research. Obviously, as with any political data, the reliability of such lists is somewhat suspect. Researchers have to take the source of the data into account, especially when the source is the group that perpetrated the abuses or those who oppose that group. They both might have reasons for falsifying data. Depending on the ultimate use of the data, precision and reliability may be more or less important: for example, data precision is generally very important in legal cases, such as human rights tribunals.

Patrick Ball of the American Association for the Advancement of Science presented some of his work on measuring human rights abuses in a variety of settings. One effective method is called capture-recapture, in which one member of the sample is selected, “returned” to the population, and if the same person is selected again, she or he is “tagged.” Originally developed for use in counting populations of animals or fish, for human rights research the method uses administrative data that were kept by the perpetrators. Because more than one witness may report a killing, some killings may go unreported, and there are often several sources of testimonies, multiple lists of killings are often available and must be used to estimate the actual number of murders. According to the theory behind the capture-recapture method, the level of overlap, or the number of killings reported to two independent lists, makes it possible for investigators to estimate the number of killings that occurred in the total population including those killings that were not recorded in either list. The method was used in Guatemala following the civil war in that country (1954-1996). Total killings were calculated by region, ethnicity, period, and perpetrator. The study estimated that approximately 200,000 people were murdered during the conflict.²⁵

Ball and his colleagues also conducted a study during the recent war in Kosovo (1999) using lists from three different organizations. They found that killings and migration during the conflict were highly correlated. The total estimates of deaths corresponded to estimates from other sources, so these findings appeared to be valid.²⁶ There are limitations to this method, however. It requires three large lists for comparison and there is a “heterogeneous catch ability”: in other words, the lists must be randomly ordered or the validity of the method cannot be ensured. The advantages of this method are that in most of these situations lists are easy to find and hence estimating the magnitude of mortality is relatively easy.

Noting the huge value of the work in Guatemala, William Seltzer of

Fordham University observed that it was so well done that even the Guatemalan army did not argue with the results. Capture-recapture methodology should be compared with typical demographic analyses (such as cluster sampling techniques, etc.), and postenumeration surveys can also be used to check the validity of estimates.²⁷ Capture-recapture is a standard technique among wildlife researchers, but it has been adapted for special application in this instance, so there are some additional problems with the technique. As Seltzer noted, sometimes it can be difficult to match records from the different lists if names are spelled differently or especially if several people have the same name. Some reports of abuses may be outside of the period of interest because of recall biases—for example, if those who reported the abuse remembered the wrong date. In addition, certain classes of events are unlikely to be “captured;” therefore, it is important to stratify the lists (perhaps by date or location) and use different types of estimation within different strata.

A small group discussed different approaches for collecting data on human rights abuses. In order to prevent bias, they suggested surveying many houses to validate reports of an event (this is called many-to-one reporting) and using indirect methods. Education and training in human rights are vital for humanitarian aid workers so that they can recognize human rights abuses, report them properly, and learn how to collect data on them. Human rights data are not just important for historical analysis, but could be used in real time to prevent genocide or other abuses.

The discussion at the workshop touched on just a few of the issues surrounding demographic estimation in complex humanitarian emergencies. In the past, there has been a dearth of research aiming to evaluate the use of various demographic methods in different types of emergencies. However, there are many issues that can be explored relatively easily, as long as methods are properly documented. Many of the workshop participants, in conjunction with their own institutes and universities, are pursuing further research on these topics, and new techniques will continue to develop over time. In the meantime, participants agreed that statisticians who work on sample surveys in stable settings could serve as valuable resources to those in the field doing rapid estimation surveys. Although several basic manuals already exist on estimation techniques for field workers,²⁸ further elaboration of these methods and guidance on which method is best in a

particular setting are important, needed resources. In addition, training on how to implement various methods for field workers is vital.

The human rights research field and the humanitarian relief field also have much to learn from one another. Human rights research techniques may be adaptable to crisis settings, and rapid estimation techniques may be useful for documenting human rights abuses in the early stages of an emergency. A recent conference in Mérida, Mexico, examined these issues, and the human rights community continues to work actively in this area of research.²⁹

This workshop was the beginning of a dialogue between many different groups—humanitarian aid workers, epidemiologists, human rights researchers, demographers, sample survey statisticians, policy makers, and anthropologists. Further discussion can only help to increase their knowledge of these methods which may help to improve their response to emergencies and ultimately to their understanding of the root causes and consequences of crises.

Asher, J., P. Ball, W. Betts, J. Dudukovich, and F. Scheuren 2002 Killings and Refugee Flow in Kosovo, March-June 1999. Washington, DC: American Association for the Advancement of Science.

Ball, P. 2000 Policy or Panic? The Flight of Ethnic Albanians from Kosovo, March-May 1999. Washington, DC: American Association for the Advancement of Science.

Ball, P., P. Kobrak, and H.F. Spirer 1999 State Violence in Guatemala, 1960-1996: A Quantitative Reflection. Washington, DC: American Association for the Advancement of Science.

Binkin, N., K. Sullivan, N. Staehling, and P. Nieburg 1995 Rapid nutrition surveys: How many clusters are enough? Disasters 16(2):97-103.

Boss, L.P., M.J. Toole, and R. Yip 1994 Assessments of mortality, morbidity, and nutritional status in Somalia during the 1991-1992 famine: Recommendations for the standardization of methods. Journal of the American Medical Association 272(5):371-376.

Brown, V., G. Jacquier, D. Coulombier, S. Balandine, F. Belanger, and D. Legros 2001 Rapid assessment of population size by area sampling in disaster situations. Disasters 25(2):164-171.

Henry, G.T. 1990 Practical Sampling. Applied Social Research Methods Series Vol. 21. Newbury Park, CA: Sage Publications, Inc.

Keely, C.B., H.E. Reed, and R.J. Waldman 2001 Understanding mortality patterns in complex humanitarian emergencies. Pp. 1-51 in National Research Council, Forced Migration and Mortality. Roundtable on the Demography of Forced Migration, Committee on Population. Holly E. Reed and Charles B. Keely, eds. Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Kish, L. 1965 Survey Sampling. New York: John Wiley.

Malilay, J., W.D. Flanders, and D. Brogan 1995 A modified cluster-sampling method for post-disaster rapid assessment of needs. Bulletin of the World Health Organization 74(4):399-405.

Médecins Sans Frontières 1997 Refugee Health: An Approach to Emergency Situations. London: Macmillan Education Ltd.

National Research Council 2001 Forced Migration and Mortality. Roundtable on the Demography of Forced Migration, Committee on Population. Holly E. Reed and Charles B. Keely, eds. Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Perrin, P. 1996 War and Public Health: Handbook on War and Public Health. Geneva: International Committee of the Red Cross.

Roberts, L., C. Hale, F. Belyakdoumi, L. Cobey, R. Ondeko, M. Despines, International Rescue Committee Democratic Republic of Congo Bukavu/Kisangani, and J. Keys 2001 Mortality in Eastern Democratic Republic of Congo: Results from Eleven Mortality Surveys. New York: International Rescue Committee.

Robinson, W.C., M.K. Lee, K. Hill, and G.M. Burnham 1999 Mortality in North Korean migrant households: A retrospective study. Lancet 354:291-295.

Sphere Project 2000 Humanitarian Charter and Minimum Standards in Disaster Response. Geneva: Sphere Project.

Spiegel, P.B., and P. Salama 2000 War and mortality in Kosovo, 1998-99: An epidemiological testimony. Lancet 355:2204-2209.

United Nations 1983 Manual X: Indirect Techniques for Demographic Estimation. Population Division of the Department of International Economic and Social Affairs of the United Nations Secretariat and the Committee on Population and Demography of the National Research Council, United States National Academy of Sciences. New York: United Nations.

United Nations High Commissioner for Refugees 1994 Registration: A Practical Guide for Field Staff. Geneva: United Nations High Commissioner for Refugees.