|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. C
|
||||||||||||||||
 |
 |
Page 259 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 259
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
C
Measuring Democracy
This appendix contains three sections to support and expand the material in Chapter 3. The statistical analysis presented in the first section was carried out by Ramziya Shakirova, a graduate student at George Mason University, on behalf of the committee. The second section contains the agenda and participants list for a committee workshop, “Democracy Indicators for Democracy Assistance,” held at Boston University in January 2007. The last section is an “Outline for a Disaggregated Meso-level Democracy Index” by John Gerring, which contains additional material related to the index proposed in Chapter 3.
STATISTICAL ANALYSIS
Spearman vs. Pearson Coefficients
The comparison of Spearman and Pearson correlation coefficients shows that on the whole, they are quite similar. However, in some cases the Spearman correlation coefficients are not significant (probably, the Pearson coefficients are too, but Stata does not display a significance level for the Pearson coefficients), which means that the Freedom House (FH) and Polity scores are in fact independent. The countries in “Partially Free Group” with insignificant correlations are:
Cambodia: Pearson is 0.3281; Spearman is 0.3453, not significant
Armenia: Pearson is 0.1632; Spearman is 0.1615, not significant
OCR for page 260
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Azerbaijan: Pearson is −0.0808; Spearman is 0.2864, but not significant
Moldova: Pearson is 0.6019; Spearman is 0.4550, not significant
Ukraine: Pearson is −0.3344; Spearman is −0.3015, not significant
Afghanistan: Pearson is 0.1832; Spearman is 0.2388, not significant
Egypt: Pearson is −0.2036; Spearman is −0.0889, but not significant
Yemen: Pearson is −0.0096; Spearman is −0.2060, not significant
Tunisia: Pearson is −0.0265; Spearman is −0.0452, not significant
Mexico: Pearson is 0.4544; Spearman is 0.2681, not significant
Greece: Pearson is 0.896; Spearman is 0.1609, not significant
Macedonia: Pearson is 0.373; Spearman is 0.3924, not significant
Sierra Leone: Pearson is 0.5094; Spearman is 0.2858, not significant
Zimbabwe: Pearson is 0.2791; Spearman is 0.2612, but not significant
Burundi: Pearson is 0.4269; Spearman is 0.2823, not significant
Cameroon: Pearson is −0.1538; Spearman is −0.1018, not significant
Comoros: Pearson is −0.0408; Spearman is 0.2358, not significant
Kenya: Pearson is 0.1287; Spearman is −0.1646, not significant
For two countries (Colombia and Côte d’Ivoire), the coefficients are close in their magnitude, although the Spearman coefficients are significant only at the 10 percent level, but not the 5 percent level.
Correlation of First Differences
The average correlation coefficients for the first differences in the group of “Partially Free” countries are low for the Former Soviet Union and the Middle East (Table C-1).
For the Former Soviet Union, the average correlation coefficient is equal to 0.148. Particularly, the coefficients are low for Armenia (0.1871) and Tajikistan (0.1320), and close to zero for the Ukraine (0.0891). Negative coefficients are observed for Kazakhstan (−0.1562), Moldova (−0.1800), and Russia (−0.5188). Satisfactory coefficients for the first differences are found in this group only for Azerbaijan, Belarus, and Georgia.
For the Middle East, the average is 0.285392. In this group, negative coefficients are observed for Egypt (−0.1292), Iran (−0.0531), and Yemen North (−0.0408), and cloze to zero, but positive, for Tunisia (0.0840).
In other regional groups there are also some countries with negative or zero coefficients: (Malaysia (−0.083), Panama (−0.2525), Angola (−0.0788), Côte d’Ivoire (−0.091), Liberia (0.000), Madagascar (0.0589), Rwanda (−0.2813), Togo (−0.0195), Uganda (0.0211), Chad (−0.218), Comoros (−0.3467), and Equatorial Guinea (−0.3725). The average correlation
OCR for page 261
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
coefficients for other regional groups are the following: Asia (0.4829), Latin America (0.54092105), and Africa (0.408083).
In the group of “Democratic” countries (Table C-2), negative correlations for the first differences are observed for Cyprus (−0.6930), France (−0.0197), and Mauritius (−0.0197), and close to zero coefficient for Trinidad (0.0163). The average for this group is also very low, and equal to 0.11855714.
For “Autocratic” countries (Table C-3), negative coefficients are observed for China (−0.0113), Oman (−0.0496), Yemen South (−0.4123), and Mauritania (−0.0197), and zero correlation for Syria.
There are several countries where the correlation coefficients for the first differences are positive, although correlations between FH and Polity scores are negative (Bahrain, Iraq, and Morocco).
The average correlation coefficient for “autocratic” countries is 0.296829.
TABLE C-1 “Partially Free” Countries (Polity Scores −5 to +7)— Correlations with FH Scores
Country
Number of Observations
Years
Pearson Correlation Coefficient
Spearman Rank Order Coefficient
Correlation Coefficient for First Differences
Asia
Cambodia
21
1972-1978; 1988-2002
0.32810
0.3453*1
0.2763
Fiji
31
1972-2002
0.86190
0.9474
0.6204
Indonesia
31
1972-2002
0.78120
0.6197
0.5473
Malaysia
31
1972-2002
0.64410
0.6942
–0.0830
Mongolia
31
1972-2002
0.98480
0.9694
0.6717
Philippines
31
1972-2002
0.93820
0.8965
0.5049
South Korea
31
1972-2002
0.96250
0.8702
0.4825
Taiwan
31
1972-2002
0.93100
0.9355
0.5105
Thailand
31
1972-2002
0.72970
0.6443
0.8155
Average
0.79572
0.769167
0.4829
Variance
0.04410
0.043645
0.06668676
Standard deviation
0.20999
0.208915
0.2582378
Former Soviet Union
Armenia
11
1992-2002
0.16320
0.1615*
0.1871
Azerbaijan
11
1992-2002
–0.08080
0.2864*
0.6191
Belarus
11
1992-2002
0.97200
0.9877
0.7319
Georgia
11
1992-2002
0.81110
0.7709
0.4286
Kazakhstan
11
1992-2002
0.54070
0.6992
–0.1562
Moldova
11
1992-2002
0.60190
0.4550*
−0.1800
OCR for page 262
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Country
Number of Observations
Years
Pearson Correlation Coefficient
Spearman Rank Order Coefficient
Correlation Coefficient for First Differences
Russia
11
1992-2002
−0.77130
−0.7287
−0.5188
Tajikistan
11
1992-2002
0.75710
0.7944
0.1320
Ukraine
11
1992-2002
−0.33440
−0.3015*
0.0891
Average
0.29550
0.347211
0.148
Variance
0.34806
0.317729
0.16145
Standard deviation
0.58997
0.563674
0.40181
Middle East
Afghanistan
24
1972-2002 (7 missing values)
0.18320
0.2388*
0.3839
Algeria
31
1972-2002
0.59750
0.5458
0.6766
Bangladesh
31
1972-2002
0.83010
0.7977
0.5921
Egypt
31
1972-2002
−0.20360
−0.0889*
−0.1292
Iran
31
1972-2002
−0.43260
−0.4784
−0.0531
Jordan
31
1972-2002
0.87020
0.9045
0.3692
Yemen North
18
1972-1989
0.64240
0.5127
−0.0408
Yemen
13
1990-2002
−0.00960
−0.2060*
0.4714
Nepal
31
1972-2002
0.77640
0.7313
0.2268
Pakistan
31
1972-2002
0.81720
0.8421
0.6368
Sri Lanka
31
1972-2002
0.75830
0.7932
0.2070
Tunisia
31
1972-2002
−0.02650
−0.0452*
0.0840
Average
0.40025
0.378967
0.285392
Variance
0.21838
0.227563
0.07862994
Standard deviation
0.46731
0.477035
0.2804103
Latin America
Argentina
31
1972-2002
0.84850
0.6961
0.5757
Bolivia
31
1972-2002
0.86500
0.7738
0.1415
Brazil
31
1972-2002
0.72550
0.6021
0.3323
Chile
31
1972-2002
0.97890
0.8875
0.8631
Colombia
31
1972-2002
0.38110
0.3248**
0.3981
Dominican
31
1972-2002
0.41910
0.3863
0.5327
Republic
Ecuador
31
1972-2002
0.96720
0.8145
0.8878
Honduras
31
1972-2002
0.91660
0.5534
0.3793
Uruguay
31
1972-2002
0.96570
0.9126
0.8292
Venezuela
31
1972-2002
0.91890
0.8858
0.6999
Guatemala
31
1972-2002
0.52950
0.4147
0.7231
Guyana
31
1972-2002
0.61740
0.6223
0.4895
Haiti
31
1972-2002
0.77960
0.5672
0.8299
El Salvador
31
1972-2002
0.44230
0.4156
0.4314
Mexico
31
1972-2002
0.45440
0.2681*
0.1765
OCR for page 263
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Country
Number of Observations
Years
Pearson Correlation Coefficient
Spearman Rank Order Coefficient
Correlation Coefficient for First Differences
Nicaragua
31
1972-2002
0.69900
0.7018
0.7508
Panama
31
1972-2002
0.78820
0.8845
−0.2525
Paraguay
31
1972-2002
0.93820
0.8731
0.8007
Peru
31
1972-2002
0.90660
0.8867
0.6885
Average
0.74430
0.656363
0.5409211
Variance
0.04356
0.046605
0.08946278
Standard deviation
0.20870
0.215881
0.2991033
European Union
Turkey
31
1972-2002
0.45220
0.6856
0.5807
Spain
31
1972-2002
0.97670
0.8546
0.5638
Greece
31
1972-2002
0.89570
0.1609*
0.6981
Macedonia
11
1992-2002
0.37300
0.3924*
0.6713
Portugal
31
1972-2002
0.95720
0.8370
0.7478
Albania
31
1972-2002
0.95260
0.9623
0.2263
Bulgaria
31
1972-2002
0.99360
0.9750
0.9330
Croatia
12
1991-2002
0.92370
0.7648
0.5851
Czechoslovakia
21
1972-1992
0.99360
0.8176
0.9911
Yugoslavia
31
1972-2002
0.87510
0.7940
0.3907
Hungary
31
1972-2002
0.98420
0.9123
0.7045
Poland
31
1972-2002
0.98750
0.9314
0.6476
Romania
31
1972-2002
0.93910
0.8766
0.5623
Slovakia
10
1993-2002
0.90800
0.9039
0.5754
Average
0.87229
0.776314
0.6341214
Variance
0.03957
0.05302
0.03728278
Standard deviation
0.19893
0.230261
0.19308749
Africa
Angola
26
1976-1991; 1993-2002
0.64430
0.6916
–0.0788
Côte d’Ivoire
31
1972-2002
0.21770
0.3316**
−0.0910
Kenya
31
1972-2002
0.12870
−0.1646*
0.5633
Liberia
25
1972-1989; 1996-2002
−0.05210
−0.0088
0.0000
Lesotho
30
1972-1997; 1999-2002
0.79260
0.8084
0.6109
Madagascar
31
1972-2002
0.90170
0.8745
0.0589
Malawi
31
1972-2002
0.99040
0.9925
0.9026
Mali
31
1972-2002
0.98390
0.8861
0.7004
Mozambique
28
1975-2002
0.95730
0.9106
0.6709
Nigeria
31
1972-2002
0.86180
0.7510
0.8536
Niger
31
1972-2002
0.94460
0.8791
0.7003
OCR for page 264
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Country
Number of Observations
Years
Pearson Correlation Coefficient
Spearman Rank Order Coefficient
Correlation Coefficientfor First Differences
Rwanda
31
1972-2002
−0.54360
−0.7643
−0.2813
South Africa
31
1972-2002
0.94290
0.8831
0.3889
Senegal
31
1972-2002
0.72580
0.6668
0.2634
Sierra Leone
27
1972-1996; 1998-2000
0.50940
0.2858*
0.7987
Sudan
31
1972-2002
0.54780
0.4058
0.8398
Tanzania
31
1972-2002
0.95040
0.8904
0.5234
Togo
31
1972-2002
0.81130
0.8327
−0.0195
Uganda
29
1972-1978; 1980-1984; 1986-2002
0.57990
0.6659
0.0211
Zambia
31
1972-2002
0.87650
0.8812
0.8956
Zimbabwe
31
1972-2002
0.27910
0.2612*
0.2902
Benin
31
1972-2002
0.98980
0.8889
0.9128
Burkina Faso
31
1972-2002
0.82510
0.8709
0.7828
Burundi
28
1972-1992; 1996-2002
0.42690
0.2823*
0.2181
Cameroon
31
1972-2002
−0.15380
−0.1018*
0.2964
Central African Republic
31
1972-2002
0.90760
0.8678
0.6987
Chad
26
1972-1977; 1984-2002
0.86520
0.8529
−0.218
Comoros
24
1976-1994; 1996-2002
−0.04080
0.2358*
−0.3467
Congo Brazzaville
31
1972-2002
0.88020
0.8362
0.7504
Equatorial Guinea
31
1972-2002
−0.43130
−0.4482
−0.3725
Ethiopia
29
1972-1973; 1974-1990; 1992-2002
0.83050
0.6848
0.1520
Gabon
31
1972-2002
0.92670
0.9295
0.6838
Gambia
31
1972-2002
0.92960
0.9447
0.8985
Ghana
31
1972-2002
0.91910
0.8917
0.5412
Guinea Bissau
27
1975-1997; 1999-2002
0.94260
0.8784
0.8501
Guinea
31
1972-2002
0.87330
0.9561
0.2320
Average
0.631697
0.598072
0.408083
Variance
0.182355
0.19153
0.165665
Standard deviation
0.427031
0.437642
0.40702
*Coefficient is not significant at 5 percent significance level.
**Coefficient is not significant at 5 percent level, but is significant at 10 percent level.
OCR for page 265
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
TABLE C-2 “Democratic” Countries (Polity Scores 8-10)—Correlations with FH Scores
Country
Number of Observations
Years
Correlation Coefficient
Correlation for First Differences
India
31
1972-2002
0.17940
0.5833
Israel
31
1972-2002
0.34690
0.5708
Jamaica
31
1972-2002
0.20880
0.3919
Trinidad
31
1972-2002
0.15910
0.0163
Cyprus
31
1972-2002
0.18380
−0.6930
France
31
1972-2002
0.02550
−0.0197
Mauritius
31
1972-2002
0.79870
−0.0197
Average
0.26446
0.11855714
Variance
0.067371
0.200422906
Standard deviation
0.259559
0.447686169
TABLE C-3 “Autocratic” Countries (Polity Scores −10 to −6)—Correlations with FH Scores
Country
Number of Observations
Years
Correlation Coefficient
Correlation for First Differences
China
31
1972-2002
0.34910
−0.0113
Burma
31
1972-2002
0.53420
0.3216
USSR
20
1972-1991
0.84570
0.7358
Bahrain
31
1972-2002
−0.12800
0.3623
Iraq
31
1972-2002
−0.06930
0.4152
Kuwait
30
1972-1989; 1991-2002
0.39630
0.8575
Morocco
31
1972-2002
−0.18060
0.4120
Oman
31
1972-2002
0.57210
–0.0496
Syria
31
1972-2002
−0.25580
0.0000
Yemen South
18
1972-1989
−0.78260
−0.4123
Eritrea
10
1993-2002
0.77170
0.3500
Mauritania
31
1972-2002
0.33160
–0.0197
Swaziland
31
1972-2002
0.78380
0.8647
Congo Kinshasa
20
1972-1991
0.66670
0.3294
Average
0.27392
0.296829
Variance
0.234796
0.136285
Standard deviation
0.484558
0.369168
OCR for page 266
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
WORKSHOP AGENDA AND PARTICIPANTS
Democracy Indicators for Democracy Assistance
January 26-27, 2007
Boston University
AGENDA
Friday, January 26, 2007
1:00 p.m.
Meeting begins
Opening remarks
Introductions
Brief project overview
Plan for the meeting
1:30 p.m.
Overview: USAID and Democracy Assistance Work
History of USAID Indicator Work
David Black, USAID
Applicability to USAID Programming and Evaluation
Margaret Sarles, USAID
2:00 p.m.
Extant Indicators. How good are they? To what degree do they fulfill USAID’s objectives, and to what extent do they fall short? Particular focus on Polity, Freedom House (with its newly released subcomponents), and the new (somewhat disaggregated) index from the Economist Intelligence Unit.
3:00 p.m.
Break
3:15 p.m.
Defining and Measuring Democracy. What is democracy? Can its dimensions and subcomponents be specified? What are the boundaries of what we choose to measure? Which important aspects of society (i.e. human rights, economic freedoms, and perhaps some things labeled governance) should fall outside our definition of democracy? Should the project also include aspects of governance that do not fall within the rubric of democracy (tout court)?
6:30 p.m.
Meeting Adjourns
7:00 p.m.
Committee Working Dinner
OCR for page 267
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Saturday, January 27, 2007
10:00 a.m.
The Aggregation Problem. Can aggregation rules be arrived at (a) within dimensions and (b) across dimensions? Can we provide some guidance to USAID on how to define “Big-D” democracy? Or is it advisable to avoid this highest level of aggregation?
11:00 a.m.
History. How important is the historical aspect of the index? What would have to be sacrificed from the current index in order for it to be extended back to 1960, 1900, or 1800?
11:30 a.m.
Management and Payoff. How to make this project work? Will the necessary data be available? How big a project is this, really? How much time would it take? How much money would it cost? How would it be organized? (Should we rely primarily on students or expert staff? If the latter, would they need to be paid, and if so how much?) What is the potential payoff of this project? Is it worth the money it would take?
12:00 p.m.
General Discussion (Lunch meeting). Revisit all issues to see what points of consensus have been reached and what points of disagreement remain. Try to resolve the latter. Return to issues that need more discussion.
1:30 p.m.
Final Recommendations and Conclusions
2:00 p.m.
Meeting Adjourns
PARTICIPANTS
David Black
U.S. Agency for International Development
Michael Coppedge
Notre Dame University
John Gerring
Boston University
Andrew Green
Georgetown University
Rita Guenther
National Academies
Jo Husbands
National Academies
Gerardo Munck
University of Southern California
Margaret Sarles
U.S. Agency for International Development
Frederic Schaffer
Harvard University
Richard Snyder
Brown University
Paul Stern
National Academies
Nicolas van de Walle
Cornell University
OCR for page 268
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
OUTLINE FOR A DISAGGREGATED MESO-LEVEL DEMOCRACY INDEX
John Gerring
Chapter 3 introduced the Committee’s proposal to develop a disaggregated index, which we believe will better serve USAID’s needs for strategic assessment and tracking. At the meso level, we identified 13 dimensions of democracy that may be independently assessed:
National Sovereignty: Is the nation sovereign?
Civil Liberty: Do citizens enjoy civil liberty in matters pertaining to politics?
Popular Sovereignty: Are elected officials sovereign relative to non-elected elites?
Transparency: How transparent is the political system?
Judicial Independence: How independent, clean, and empowered is the judiciary?
Checks on the Executive: Are there effective checks on the executive?
Election Participation: Is electoral participation unconstrained and extensive?
Election Administration: Is the administration of elections fair?
Election Results: Do results of an election indicate that a democratic process has occurred?
Leadership Turnover: Is there regular turnover in the top political leadership?
Civil Society: Is civil society dynamic, independent, and politically active?
Political Parties: Are political parties well institutionalized?
Subnational Democracy: How decentralized is political power and how democratic is politics at subnational levels?
The rest of this section of Appendix C elaborates on some of the issues related to the proposed index, concluding with a more detailed listing of the 13 dimensions listed above.
Components
Each dimension has multiple components, chosen with five criteria in mind: (a) centrality to the dimension, (b) centrality to the overall concept of democracy (defined minimally and maximally, as explained in the text), (c) the possible incorporation of existing data, (d) measurement precision, (e) accuracy (reliability), and (f) nonredundancy. Each component is stated in the form of a question or statement that may be coded numerically for
OCR for page 269
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
a given country or territory during a given year. Further work will be required in order to specify what these scales mean in the context of each question. The devil is always in the details.
Coding categories are dichotomous (yes/no), categorical (unranked), nominal (ranked), or interval. In certain cases, it may be possible to combine separate components into more aggregated nominal scales without losing information (Coppedge and Reinicke 1990). This is possible, evidently, only when the underlying data of interest are, in fact, nominal.
There are roughly 100 components in the index as currently constructed. While this may seem like quite a few, the reader is urged to consider that most of these questions—indeed, the vast majority—are very simple to answer. Thus, it should not take a country expert (or well-coached student assistant) very long to complete the questionnaire. Indeed, this is precisely the point. A longer set of questions is sometimes quicker to complete than a much shorter set of questions, if the latter are vague and ambiguous (due, we suppose, to a high level of aggregation).
For each datum, one should record (a) the coding (numerical or natural language), (b) the source(s) on which the coding was based, (c) the coder(s), (d) any revisions to the initial coding that may have been made in previous iterations of the dataset, (e) any further explanation that might be helpful, and (f) estimates of uncertainty (discussed below). Evidently, it is important that the data-storage software be capable of handling numerical and narrative responses (e.g., MS Access).
Objective/Subjective Measures
With respect to attaining greater accuracy, “hard” or “objective” indicators—based on what might be considered factual matters—are preferred over expert opinions. As one example, one might consider how to replace (or supplement) the opinion of country experts about how free the press is with a content analysis of major news outlets. Where the press is free, one would expect to find (a) a dispersion of views across news sources and (b) criticism of political leaders. Both signal the existence of the sort of open debate that is impossible if the press is constrained, and inevitable (one would think) if it is not.
At the same time, it is important to note that the development of an objective measure for a difficult concept such as press freedom is apt to be time-intensive and costly, and may not be possible at all for previous eras. Additionally, objective indicators are sometimes subject to the problem of “teaching to the test”; governments can attain higher scores by fulfilling some criterion that has little import for democracy.
The benefits of easy data collection thus must be balanced against the benefits of data efficiency, coverage, and conceptual validity.
OCR for page 274
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Pilot Tests
Before USAID, or any agency, undertakes a commitment to develop—and maintain—a new democracy index, it is important that it be confident of the yield. Thus, we recommend several interim tests of a “pilot” nature.
One of the principal claims of this index is that greater intercoder reliability will be achieved when the concept of democracy is disaggregated. This claim may be probed through intercoder reliability tests across the leading democracy indices. A pilot test of this nature might be conducted in the following manner: Train the same set of coders to code all countries (or a subset of countries) in a given year according to guidelines provided by Freedom House, Polity, and the present index. Each country-year would receive several codings by different coders, thus providing the basis for an intercoder reliability test. These would then be compared across indices. Since the coders would remain the same, varying levels of intercoder reliability should be illustrative of basic differences in the performance of the indices. Of course, there are certain methodological obstacles to any study of this sort. One must decide how much training to provide to the coders, and how much time to give them. One must decide whether to employ a few coders to cover all countries, or have separate coders for each country. One must decide whether to hire “naïve” coders (e.g., students) or coders well versed in the countries and regions they are assigned to code (the “country expert” model). In any case, we think the exercise worthwhile, not only because it provides an initial test of the present index but also because it may bring a level of rigor to a topic—political indicators—that has languished for many years in a highly unsatisfactory state.
THE INDEX
Dimensions
National Sovereignty: Is the nation sovereign?
Civil Liberty: Do citizens enjoy civil liberty in matters pertaining to politics?
Popular Sovereignty: Are elected officials sovereign relative to non-elected elites?
Transparency: How transparent is the political system?
Judicial Independence: How independent, clean, and empowered is the judiciary?
Checks on the Executive: Are there effective checks on the executive?
OCR for page 275
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Election Participation: Is electoral participation unconstrained and extensive?
Election Administration: Is the administration of elections fair?
Election Results: Do results of an election indicate that a democratic process has occurred?
Leadership Turnover: Is there regular turnover in the top political leadership?
Civil Society: Is civil society dynamic, independent, and politically active?
Political Parties: Are political parties well institutionalized?
Subnational Democracy: How decentralized is political power and how democratic is politics at subnational levels?
Clarifications
“Party” may refer to a longstanding coalition such as the CDU/CSU in Germany if that coalition functions in most respects like a single party. The identity of the party may be obscured by name changes. (If the party/coalition changes names but retains key personnel and is still run by and for the same constituency then it should be considered the same organization.)
“Executive” refers to the most powerful elective office in a country (if there is one)—usually a president or prime minister.
Wherever there is disparity between formal rules (constitutional or statutory) and actual practice, coding decisions should be based on the latter.
Unless otherwise specified, the geographic unit of analysis is the (sovereign or semi-sovereign) nation-state. Evidently, there is enormous heterogeneity within large nation-states, necessitating judgments about which level of coding corresponds most closely to the mean value within that unit. Where extreme heterogeneity exists vis-à-vis the variable of interest it may be important to include a companion variable that would indicate high within-country variance on that particular component. One thinks of contemporary Sri Lanka and Colombia—states where the quality of democracy is quite different across regions of the country.
Questions pertaining to elections may be disaggregated according to whether they refer to elections for the (a) lower house, (b) upper house, or (c) presidency. In some cases, (b) and/or (c) is nonexistent or inconsequential, in which case it should be ignored. If no election occurs in a given year, then many of these questions should be left unanswered (unless of course rules or norms pertaining to elections have changed in the interim). If more than one election occurs in a given year there will be two entries for that country in that year. (This complicates data
OCR for page 276
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
analysis, but it is essential to the purpose of the dataset, which is to provide primary-level data that can be used for further analysis.)
At some point, coding responses must be added to this questionnaire. Such responses may be dichotomous, multichotomous, or continuous, depending upon the question. However, we suggest that all original coding scales (where coding decisions are required) be comprised of no more than five categories. A larger number of options may create greater ambiguity. In any case, these response options should be as operational as possible. It should be clear what a “3” means with respect to the question at hand.
National Sovereignty
General question: Is the nation sovereign?
Is the territory independent of foreign domination? (Note: We are not concerned here with pressures that all states are subject to as part of the international system.)
Civil Liberty
General questions: Do citizens enjoy civil liberty in matters pertaining to politics?
Note: Civil liberties issues pertaining specifically to elections are covered in later sections.
Does the government directly or indirectly attempt to censor the major media (print, broadcast, Internet)? Indirect forms of censorship might include politically motivated awarding of broadcast frequencies, withdrawal of financial support, influence over printing facilities and distribution networks, selective distribution of advertising, onerous registration requirements, prohibitive tariffs, and bribery. (See recent index of Internet freedom developed by the Berkman Center for Internet and Society, Harvard University.)
Of the major media outlets, how many routinely criticize the government?
Are individual journalists harassed—i.e., threatened with libel, arrested, imprisoned, beaten, or killed—by government or nongovernmental actors while engaged in legitimate journalistic activities?
Is there self-censorship among journalists when reporting on politically sensitive issues?
Are works of literature, art, music, and other forms of cultural expression censored or banned for political purposes?
Do citizens feel safe enough to speak freely about political subjects in their homes and in public spaces?
OCR for page 277
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Is it possible to form civic associations, including those with a critical view of government?
Is physical violence (e.g., torture) and/or arbitrary arrest targeted at presumed opponents of the government widespread?
Are certain groups systematically discriminated against by virtue of their race, ethnicity, language, caste, or culture to the point where it impairs their ability to participate in politics on an equal footing with other groups? (Note: This question pertains to citizens only [not non-citizens] and does not cover issues of disenfranchisement, which are included in a later section.)
If so, how large (as a percentage of the total population) is this group(s)?
Popular Sovereignty
General question: Are elected officials sovereign relative to nonelected elites?
Are there national-level elections (even if only pro forma)?
If yes, are the governments that result from these elections fully sovereign—in practice, not merely in constitutional form—vis-à-vis any nonelective bodies whose members are not chosen by, or removable by, elected authorities (e.g., a monarchy, the military, and the church)? Note that this does not preclude extensive delegation of authority to nonelective bodies such as central banks and other agencies. But it does presume that the members of these nonelective authorities are chosen by, and may be removed, in circumstances of extreme malfeasance, by elective authorities. This power of removal must be real, not merely formal. Thus, while constitutions generally grant power to civilian authorities to remove military rulers, it is understood that in some countries, during some periods, an action of this nature would not be tolerated. In most cases, it will be clear to those familiar with the countries in question when this sort of situation obtains, though there may be questions about the precise dates of transition (e.g., when Chilean political leaders regained control over the military after the Pinochet dictatorship).
Transparency
General question: How transparent is the political system?
Note: this section pertains to the polity as a whole, while some other questions listed below pertain to particular sections of the polity (e.g., election administration).
Are government decisions made public in a timely fashion and otherwise made accessible to citizens?
Are decision-making processes open to public scrutiny, for example, through committee hearings?
OCR for page 278
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Judicial Independence
General question: How independent, clean, and empowered is the judiciary?
Is the judiciary independent of partisan-political pressures?
Is the judiciary noncorrupt?
Is the judiciary sufficiently empowered to enforce the laws of the land, including those pertaining to the ruling elite (or is its power so reduced that it cannot serve as a check on other branches of government)?
Checks on the Executive
General question: Are there effective checks—other than elections—on the exercise of power by the executive?
Note: Questions pertaining to electoral accountability are addressed elsewhere.
Constitutionality
Does the executive behave in a constitutional manner (i.e., according to written constitutional rules or well-established constitutional principles)?
Term limits
If the executive is elected directly by the general electorate (or through an electoral college), are there term limits?
If so, what are they?
Are they respected (at this point in time)?
The legislature
Is the executive able to control the legislature by undemocratic means (e.g., by manipulating legislative elections, by proroguing the legislature, by buying votes in the legislature)?
Is the executive able to make major policy decisions without legislative approval, i.e., without passing laws? Can the executive rule by fiat?
The judiciary
Is the executive accountable to the judiciary—which is to say, is the judiciary prepared to enforce the constitution, even when in conflict with the executive?
Election Participation
General question: Is electoral participation unconstrained and extensive?
Suffrage
What percent of citizens (if any) are subject to de jure and de facto eligibility restrictions based on ascriptive characteristics other than age (e.g., race, ethnicity, religion)?
What percent of the population are excluded from suffrage by virtue of being permanent residents (noncitizens)?
OCR for page 279
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Turnout
Note: This variable is meaningless in the absence of free and fair elections. Therefore, although data may be collected for all countries, it should be considered an aspect of democracy only where countries score above some minimal level on Election Administration.
What percent of the adult (as defined by the country’s laws) electorate turned out to vote?
Election Administration1
General question: Is the administration of elections fair?
Election law
At this time, are regularly scheduled elections—past and future—on course, as stipulated by election law or well-established precedent? (If the answer is no, the implication is that they have been suspended or postponed in violation of election law or well-established precedent.)
Are there clear and explicit sets of rules for the conduct of elections and are the rules clearly disseminated (at the very least, to political elites in the opposition)?
Election commission
Note: Election commission refers to whatever government bureau(s) is assigned responsibility for setting up and overseeing elections.
Is it unbiased and independent of partisan pressures or balanced in its representation of different partisans?
Does it have sufficient power and/or prestige to enforce its own provisions? (Are its decisions respected and carried out?)
Registration
Are electoral rolls updated regularly?
Do they accurately reflect who has registered? (If the election rolls are not made public, then the answer is assumed to be No.)
Do names of those registered appear on the rolls at their local polling station (as they ought to)?
Integrity of the vote
Are all viable political parties and candidates granted access to the ballot (without unduly burdensome qualification requirements)?
Are opposition candidates/parties subject to harassment (e.g., selective prosecution, intimidation)?
Is the election process manipulated through other means (e.g., changing age or citizenship laws to restrict opposition candidate’s access to the ballot, stalking horse candidates, snap elections scheduled without sufficient time for the opposition to organize)?
Are election choices secret (or are there violations)?
1
This section draws on Munck (2006).
OCR for page 280
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Is vote-buying (bribery) and/or intimidation of voters widespread?
Are other forms of vote fraud (e.g., ballot-stuffing, misreporting of votes) widespread?
What percent of polling stations did not open on time, experienced an interruption, ran out of voting materials, or experienced some other sort of irregularity?
What was the percentage of lost or spoiled ballots?
Media
Do all parties and candidates have equal access to the media? Equal access is understood as (a) all candidates or parties for a particular office are treated equally (thus granting an advantage to small parties or minor candidates) or (b) access to the media is in rough proportion to the demonstrated support of a party or candidate in the electorate.
Is election reportage (reportage about politics during election periods) biased against certain parties and/or candidates?
Campaign finance
Are there disclosure requirements for large donations?
If so, are these effective (i.e., are they generally observed)?
Is public financing available?
If so, does it constitute at least one-third of the estimated expenditures by candidates and/or parties during the course of a typical campaign?
Does the incumbent enjoy unfair advantages in raising money by virtue of occupying public office? Unfair advantage involves such things as (a) a levy on civil servants to finance the party’s campaigns, (b) widespread and organized use of civil servants for campaign purposes, or (c) use of government materiel for campaign purposes.
Is campaign spending heavily tilted in favor of the incumbent party or candidate(s)?
That is, does the incumbent party or candidate(s) expend more financial resources than their support in the electorate (as judged by polls or general impressions) or the legislature would indicate?
Note: Where campaign expenditures are unreported, or such reports are unreliable, they may be estimated from each party’s campaign activity, e.g., number of political advertisements on TV, radio, or billboards.
Election monitors
Were election monitors from all parties and/or from abroad allowed to monitor the vote at polling stations across the country?
How many polling stations (percent) were attended by election monitors (other than those representing the ruling party or clique)?
Election Results
General question: Do results of an election indicate that a democratic process has occurred?
OCR for page 281
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
What percent of the vote was received by the largest party or winning candidate in the final (or only) round?
Specify name of party or candidate:
What percent of the vote was received by the second largest party or second most successful candidate in the final round?
Specify name of party or candidate:
What percent of the seats in the lower/upper house was obtained by the largest party?
Specify name of party:
What percent of the seats in the lower/upper house was obtained by the second largest party?
Specify name of party:
Do the official results conform, more or less, to actual ballots cast (as near as that can be estimated)?
What was the general verdict by international election monitors and or the international press vis-à-vis the democratic quality of this election, i.e., how fair was it?
Note: If there was disagreement, then please report the mean (average) result, weighting each group by its level of involvement in overseeing this election.
Did losing parties/candidates accept the essential fairness of the process and the result?
Leadership Turnover
General question: Is there regular turnover in the top political leadership?
Note: Turnover may be regarded as a sufficient condition of effective electoral competition. If turnover occurs (by democratic instruments), contestation must be present—though it may of course still be flawed.
Executive
How many years has the current executive been in office? (Source: “YRSOFFC” variable from the DPI.)
How many consecutive terms has the current executive served?
Did the last turnover in power occur through democratic means (e.g., an election, a loss of confidence in the legislature, or a leader’s loss of confidence in his/her own party)?
Ruling party/coalition
How many years has the current ruling party or coalition been in office? (Source: “PRTYIN” variable from the DPI.)
How many consecutive terms has the current ruling party or coalition served?
Note: relevant only where elections fill the major offices.
Did the last turnover in power occur through democratic means (e.g.,
OCR for page 282
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
an election, a loss of confidence in the legislature, or a leader’s loss of confidence in his/her own party)?
Civil Society
General question: Is civil society dynamic, independent, politically active, and supportive of democracy?
Notes:
“Civil society organization” refers to any of the following: an interest group, a social movement, church group, or classic NGO, but not a private business, political party, or government agency. Must be at least nominally independent of government and the private sector.
Questions about civil liberties, of obvious significance to civil society, are covered in a separate section.
Existing indicators: the Civil Society Index compiled by the Global Civil Society Project.
How much support for democracy is there among citizens of the country? (Sources: World Values Surveys, Eurobarometer, Afrobarometer, Latinobarometer [see EIU].)
What is the level of literacy (a presumed condition of effective participation)? (Source: WDI.)
What percent of citizens regularly listen to or read the national news?
Are civil society organizations generally independent of direct government influence (or are they manipulated by the government and its allies such that they do not exercise an independent voice)?
Are there any sizeable civil society organizations that are routinely critical of the government?
Are major civil society organizations—representing key constituencies on an issue—routinely consulted by policymakers on policies relevant to their members (e.g., by giving testimony before legislative committees)?
Political Parties
General question: Are political parties well institutionalized?
Notes:
Questions about the freedom to form parties and participate in elections are included under Election Administration.
Questions below refer to all parties in a polity, considered as a whole. However, larger parties should be given greater weight in calculating answers so that the party system is adequately represented.
Are there well-understood rules governing each party’s business and, if so, are these rules generally followed?
OCR for page 283
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Is there a clearly identifiable group of party members and is this group relatively stable from year to year?
Do parties issue detailed policy platforms (manifestos)?
Do parties hold regular conventions and, if so, are these conventions sovereign (in the sense of making final decisions on party polity and procedure)?
Do parties have local sections (constituency groups), or are they centered on the capital and on a restricted group of local notables?
Subnational Government
General question: How democratic is politics at subnational levels?
Note: “Subnational government” refers to governments at regional and local levels.
How centralized is power within the polity, taking all factors into account (for a useful discussion of various relevant factors see Rodden 2004)? As a way of calibrating this, Switzerland may be said to define the decentralized extreme while New Zealand defines the centralized extreme among democratic polities. Most authoritarian regimes are highly centralized, but not all (e.g., failed states such as Afghanistan or Somalia). To clarify, the question refers to the relative power balance between national and subnational levels; it does not attempt to judge the actual strength of control at either level. That is, whether both levels of government are weak or strong is irrelevant; what is relevant is only their power relative to each other. The question pertains to practical power not to formal/constitutional power. Note that centralization is usually not considered a definitional component of democracy: New Zealand, most would agree, is no less democratic than Switzerland. However, if power is highly centralized in a very large country—say, India—one may infer a significant problem of local accountability. In any case, the degree of centralization/decentralization gives meaning to the next question.
How democratic are electoral politics at the subnational level? If practices differ appreciably between national and subnational levels, and perhaps even between regional and local levels, it may be necessary to complete the previous sections—Election Participation, Election Administration, Election Results, Leadership Turnover—for different levels of government.
References
Converse, N., and Kapstein, E.B. 2006. “The Economics of Young Democracies: Policies and Performance.” Working Paper No. 85, Center for Global Development (March).
Coppedge, M., and Reinicke, E.B. 1990. “Measuring Polyarchy.” Studies in Comparative International Development 25:51-72.
Europa Yearbook. [various years] The Europa Yearbook. London: Europa Publications.
OCR for page 284
Improving Democracy Assistance: Building Knowledge through Evaluations and Research
Gerring, J.; Bond. P.; Barndt, W.; and Moreno, C. 2005. Democracy and Growth: A Historical Perspective. World Politics 57(3):323-364.
Huntington, Samuel P. 1991. The Third Wave: Democratization in the Late Twentieth Century. Norman, OK: University of Oklahoma Press.
McFaul, M. 2005. Transitions from Postcommunism. Journal of Democracy 16(3): 5-19.
Munck, G. L. 2006. Standards for Evaluating Electoral Processes by OAS Election Observation Missions. Paper prepared for Organization of American States.
Persson, T., and Tabellini, G. 2006. Democratic Capital: The Nexus of Political and Economic Change. NBER Working Paper. No. 12175.
Rodden, J. 2004. Comparative Federalism and Decentralization: On Meaning and Measurement. Comparative Politics (July):481-500.