Drug Data Organization
Chapter 3 focused on the substantive innovations necessary to improve the quality, range, reliability, and usefulness of data on illegal drugs. Improving the data requires more than gifted survey researchers or fine questionnaires, however. The history of data collection indicates that the organizational structure of data collection is a critical feature of the overall system. This chapter briefly reviews the current organizational environment and makes recommendations on how the federal government can strengthen and advance the effort.
Effective policies to reduce the costs and damages associated with illegal drugs require accurate and timely data on all aspects of drug markets, consumption, prevention, and impacts. As this report documents, at present the quality of some data is poor, often data are simply unavailable, and policies in many areas are therefore poorly informed. Improving policies and introducing fruitful innovations will require a quantum improvement in the nation’s statistical efforts in this area.
The Office of National Drug Control Policy has reported that data on illegal drugs are collected by 18 federal agencies. In fact, the bulk of the data are collected by a much smaller number of agencies: the National Institute of Justice, the Drug Enforcement Administration, the National Institute on Drug Abuse, and Substance Abuse and Mental Health Service Administration. The Bureau of Justice Statistics and the National Center for Health Statistics have also collected some drug-related data in connection with their broader statistical responsibilities.
Each of the data collections is built on an infrastructure consisting of its professional staff, survey researchers (who in this area are mostly contractors or grantees), and analysts. They rely on the cooperation of respondents, schools, hospitals, treatment facilities, medical examiners, and the criminal justice system. In the committee’s judgment, regular professional and impartial reporting would greatly increase the cooperation of these key participants in the statistical processes that provide key indicators.
The ultimate goal of the recommendations in this chapter is to improve the quality and usefulness of the data so that they can better serve public understanding and policy making on illegal drugs. The central point of this chapter is that the substantive recommendations in this report cannot be implemented without improvements in the practices of statistical agencies and a consolidation of the federal effort along with determined leadership to implement these organizational improvements.
The scope of this chapter is limited to a review of the adequacy of the organizational structures for the collection, analysis, and reporting of statistical data on illegal drugs. This means that we are considering how to improve the quality and timeliness of data, for example, on the price or total consumption of cocaine, heroin, and marijuana. We do not consider the organizational issues involved in research on illegal drugs, such as how to best determine the relationship between interdiction and use or between crime and drug laws. Producing high-quality statistical data is a prerequisite for empirically informed research, but only data needs are considered here.
UPGRADING INDEPENDENCE AND PROFESSIONALISM
Regardless of the organizational structure of data collection on illegal drugs, it is essential that the federal government set a goal of protecting and strengthening the statistical reporting practices, independence, and scientific integrity of organizations charged with collecting the data. This section presents some principles that can serve as guidelines for ensuring that the institutional structure of agencies that collect and report drug data reflects the best practices for statistical agencies and is conducive to the collection of high-quality data. They apply both to existing agencies and to any new or consolidated agencies that might be created in the future.
A critical part of the process of improving the value of data and research on illegal drugs for policy purposes is to ensure that the agencies that collect the data have the resources, independence, and professional staff to engage in the range of activities necessary for producing high-quality data collection and statistical reports. The federal government has
considerable experience in this area. A 1992 report of the Committee on National Statistics of the National Academies has examined these issues (National Research Council, 1992), and we draw in part on that effort in setting out the following principles for agencies collecting statistical data on illegal drugs.
Improving the quality and usefulness of drug statistics requires that statistical data collection and reporting are assigned to agencies or offices within agencies that have statistical reporting as their major focus. High-quality statistical data collection cannot be an accident of activities carried out in support of some other mission—such as providing treatment or apprehending violators. It is essential that statistical organizations report results regardless of their programmatic impact, and their work not be subject to review by agencies or officials whose success will be judged based on the actual outcome of the data collection.
We divide the principles for strengthening statistical agencies into two components. First is a commitment to quality and professional standards. The following are guidelines for professionalism that should be maintained and strengthened in agencies providing data on illegal drugs:
To maintain a commitment to high standards, the agency head needs a strong professional background and reputation. The relevant skills would be either in survey research and statistics or in the subject field of the agency.
To encourage the highest standards of professionalism and adherence to providing independent and high-quality data, agency heads must be insulated from policy direction regarding the content of reported results. This can be enhanced when agency heads have fixed terms or are appointed as career members of the Senior Executive Service, so that they are not subject to political removal.
Adequate resources are needed to build up a professional staff that is knowledgeable and skilled in regular statistical reporting and in the design and implementation of surveys.
Improving the professionalism of an agency’s work requires that the staff and methodological research of the agency be integrated with the broader professional community and can draw on the full science base in the relevant field.
The second component is sufficient independence so that the statistical agency can provide credible and useful data to the government and to the public. Independence from policy direction regarding the issuance of reported results is particularly important when the data and subject are controversial and when data collection is plagued by methodological difficulties—both of which apply to the nth power to data on illegal drugs.
The following are guidelines for maintaining and strengthening independence for agencies providing data on illegal drugs:
When appropriate, consideration should be given to upgrading the status of statistical agencies within departments, including direct reporting to departmental secretaries, so that their personnel and results gain credibility. This will also serve to focus their mission on statistical reporting and distinguish it from research, service delivery, and policy analysis. This is particularly important if statistical data collection is consolidated into one or two lead agencies, as recommended in the section on consolidation below.
Statistical results should be released without prior political or administrative review. For certain key data, it may be useful to have data released on a predetermined schedule and to institute the practice (used for key economic data) of prohibiting comment by administration officials until one hour after the official release. It may also be useful to limit prior access to the statistical reports to the President and his or her immediate staff. All these steps would help to reduce or eliminate departmental and White House “spin” of data releases.
It may be useful for Congress to set up its own parallel analysis based on professional expertise to be released at the same time as the executive branch release, as is currently the case when the Joint Economic Committee analyzes key economic data.
In well-run statistical agencies, a most important principle governs the respective roles of policy makers and the statisticians who provide information. Policy makers articulate the need for information, and statisticians respond with answers based on best professional practices. In such agencies, the reports issued by statisticians are not subject to the kind of policy review that might alter their content or emphasis. Compared with other statistical efforts in the federal government (such as the economic agencies), the collection and reporting of statistical data on illegal drugs are not always free from such policy review. In fact, the release of major data series—such as results from the Monitoring the Future survey and the National Household Survey on Drug Abuse—has been the occasion for high-level press conferences and detailed prior review by policy makers. The following new standards should guide the release of these reports:
It is essential that statistical agencies have clearly defined and well-accepted missions. The mission is often defined by legislation or regulation and evolves over time. A clear mission is critical for planning the program of a statistical agency and for evaluating its performance.
Once it is assigned to produce high-quality statistical data in a given area, an agency should be able to issue statistical reports based on professionally designed methods. As is the practice at the Bureau of Labor Statistics and the Bureau of the Census, for example, these reports should be issued without intervening policy review. Direct reporting of results by expert statisticians is a key marker of professional integrity. Policy direction regarding data collection procedures or reporting of data reduces the credibility of reporting, slows down the release of the data, and reduces the usefulness of the data for policy and research. It also limits the research on statistical techniques and methods that is vital to improve the reliability, validity, and relevance of key statistical indicators.
Putting this principle in yet another way, there should be clear separation of the purposes of data collection—which should be decided by policy makers—from how the questions are to be asked and the final content of the report—which are best decided by the statisticians and other relevant specialists.
Many of the guidelines listed above for increasing the professionalism of agencies will also help ensure independence. For example, enhancing the professional qualifications for the heads of statistical agencies will help ensure that the direction of data collection has a professional rather than a political motivation. In addition, many of the guidelines on independence will reinforce the professionalism of agencies.
The usefulness and integrity of these statistical efforts would be greatly enhanced by improving the dissemination of data and results. There are many approaches available to make the results and data more widely available. Some agencies use data archives to release public-use files containing the original data from surveys or data compilations. Others provide this material to researchers upon request. In some cases, to protect the confidentiality of sensitive material, statistical agencies have established special procedures to give researchers access while requiring confidentiality agreements that ensure the privacy of respondents.
The committee emphasizes that open access to the original material (while always using practices consistent with protecting individual privacy) encourages use of the data and enlists the broad community of researchers in the task of analyzing and understanding current trends. Just as important is that external analysis provides the critical function of reviewing the data and methods of the agency and helps inform the research community and policy makers about the usefulness and limitations of the data. At the same time, it alerts policy makers to the need for improvements and the best applications of the results.
Several agencies currently make some or all of their material available
in this way. However, making data and research widely available is not universal practice. All drug-related agencies need to improve access to and encourage widespread analysis and reporting of their data. When access is not currently provided, steps need to be taken to provide it.
The committee recommends that public-use files of all major statistical series should be deposited in a data library. On a broader level, every agency sponsoring the collection of population-based data related to illegal drugs should require in their contracts and grants the timely deposit of public-use files in an appropriate data library or its dissemination in other ways.
In connection with this recommendation, see the committee’s recommendation in Chapter 3 specifically regarding the longitudinal file of data from the Monitoring the Future survey. Finally, we recognize the importance of ensuring that confidentiality be preserved. In the committee view, it is sponsors rather than data collectors who should make decisions regarding the level of detail released.
IMPROVING ECONOMIC DATA ON ILLEGAL DRUGS
One of the major messages of this report is that economic data relating to illegal drugs need to be substantially improved. Major gaps have been identified in the collection and reporting of reliable data about the prices, total consumption, and expenditure on illegal drugs. The deficiency in this area arises in part because economic data for the most part have been ancillary to law enforcement efforts. In addition, efforts to collect reliable economic data have been hampered by a lack of accepted procedures for collecting economic information about illegal activities. In the committee’s view, major efforts are needed to improve such data.
The methodologies for collecting economic data on illegal drugs are at present insufficiently developed to begin regular collection of statistical data. The first stage in improved collection of economic data would be to improve the methodologies in this area.
One approach would be to assign the tasks of designing, collecting, reporting, and validating statistical series on economic data, such as prices, expenditures, and consumption, to a newly constituted economic working group. This group would consist of professional statisticians located in economic statistics agencies, such as the Bureau of Labor Statistics and the Bureau of Economic Analysis, as well as economists and statisticians with expertise in these methods who are working in an existing statistical unit of an agency focused on illegal drugs.
The first assignment of the economic working group would be to develop, test, and validate methods and procedures to report on prices, expenditures, and total consumption. The working group would need
sufficient funds and independence to sponsor broadly based research activities. Its research would begin with thorough analysis of existing information and methodologies. The results of these efforts could be widely reported in the professional literature and subject to careful review and analysis before full-scale data collection begins.
It is likely that the data collection efforts for economic data will need to be separated from law enforcement activities. To accomplish this, after a review of the relevant law, either the executive branch would issue rules or the Congress would enact legislation to provide legal protection to researchers collecting economic data so that they can proceed without having to become a part of a law enforcement effort.
The committee recommends the formation of an executive branch board to review proposed data collection protocols that might be used as a part of a research effort to design, collect, report, and validate statistical series on economic data, such as prices, expenditures, and consumption. It may be necessary to have rules or legislation enabling the board to exercise its functions in a manner that clearly separates law enforcement from this research enterprise.
It is important to consider the coordination of statistical data collection related to illegal drugs in the broader context of such coordination across the entire federal government. One of the recurrent issues in the decentralized U.S. federal statistical system is how best to coordinate data collection by different agencies.
At a workshop held on February 17–18, 2000, the committee heard from representatives of the Office of Management and Budget (OMB), agencies that collect drug statistics, and other experts. The presentations, including comments from the chief statistician of the United States and from officials charged with reviewing data collection activities proposed by health and justice agencies, illuminated the roles of both OMB and the Office of National Drug Control Policy in overseeing data collection related to illegal drugs.
The presenters noted that the decentralization of the federal statistical system is reflected in the many agencies that report statistics relevant to illegal drugs. Since the 1930s, OMB and its predecessor agencies have been charged with reviewing agency proposals to collect statistical data from the public. The Federal Reports Act of 1940 provided that agencies were required to seek budget office review of all forms used to collect information from 10 or more persons. Subsequent legislation—in particular the Paperwork Reduction Act of 1980—defined and broadened OMB’s
authority in this area to include the coordination of all federal agency statistical data collection.
OMB can and does use its authority to encourage agencies to coordinate their data collection so as to improve the quality of the information collected and limit the burden on the public. In particular, it uses the authority of the Paperwork Reduction Act to encourage common terminology, constructs, and definitions in the data collected across the statistical system.
OMB has a very small staff available for review of statistical surveys. Its activities are limited to coordination and review of proposals to collect data. The OMB staff does not issue statistics or review results before they are issued. It frequently charters interagency groups to improve coordination of data collection among agencies. It also works to identify budget gaps and help to find ways to fill them.
Current legislation places responsibility for some coordination at the departmental level. For example, the Department of Health and Human Services Data Council has been examining ways of consolidating and improving statistical reporting and data collection concerning illegal drugs.
The Office of National Drug Control Policy (ONDCP) has authority to facilitate and coordinate data collection concerning illegal drugs. For example, it was at ONDCP’s suggestion that the National Household Survey of Drug Abuse was carried out each year rather than every 3 years, and its size markedly increased. Likewise ONDCP suggested that Monitoring the Future include 8th and 10th graders rather than just 12th graders. Its authority stems from several of its functions.
First, ONDCP is called on to certify the compliance of agency budgets with the President’s drug strategy. When agencies submit their budgets to OMB, the budgets go to ONDCP for certification that they comply with the drug strategy. Only the President can overrule ONDCP. In practice this procedure means that ONDCP is an influential part of the budget review process and can therefore require agencies to show how their budgets provide for the collection of needed statistical data. Alternatively, ONDCP can use this process to raise questions about the propriety of proposed data collection or its location. ONDCP uses the certification process to influence agency decisions regarding the initiation and maintenance of data collection efforts.
Second, OMB consults ONDCP during its review of the data collection proposals of the different agencies. ONDCP can use these occasions to review these proposals and suggest the inclusion of data items relevant to drug control policy.
Third, the ONDCP has chaired a Drug Control, Research Data and
Evaluation Advisory Committee since 1994. The committee has chartered a subcommittee on data, research, and interagency coordination, which has representatives from the major drug control program agencies. The subcommittee’s 1999 report on drug control research data and evaluation contains a useful inventory of federal data collection projects that contain information relevant to drug control policy (Executive Office of the President, 1999).
ONDCP currently is the lead agency for coordinating data collection on illegal substances and drug control in the federal government. In this role, ONDCP and its data subcommittee have analyzed the current structure of data collection, have identified some of the weaknesses in the current data system and have discussed design of survey instruments (Executive Office of the President, 1999). However, it has not seen a need to take any major initiatives to reorganize data collection efforts, nor is that the role of the data subcommittee.
At present, reporting, compilation, and analysis of the voluminous data on illegal drugs are largely spread through the many agencies that collect the data. While ONDCP does issue annual reports, they are largely concerned with setting policy and describing the results of federal activities. Many of the annual reports focus on policy goals and implementation strategies, and there is relatively little analysis of the underlying trends and data issues.
It would be useful to have an annual report on illegal drugs in the United States that presents and assesses the most important statistical series. Such a report would collect in one place the major data on the health, law enforcement, international, and economic facets of illegal drugs and related issues along with an appropriate commentary. Such a report could be modeled after similar reports in other areas, such as the Annual Energy Outlook of the Energy Information Administration and the Economic Report of the President prepared by the Council of Economic Advisers; these documents present the data in a scholarly fashion that can survive changing perspectives, policy approaches, and administrations. For this purpose, it would be important to have the report issued by an agency with an adequate staff of professional analysts and researchers.
CONSOLIDATING DATA COLLECTION
The 18 agencies that collect and report a wide variety of statistical indicators related to illegal (and closely related legal) drugs are currently located in the Department of Justice, the Department of State, Department of Defense, the Department of Labor, the Department of Housing and Urban Development, and several organizations within the Department of
Health and Human Services. In addition, the Office of National Drug Control Policy directly sponsors data collection. In many cases, these agencies collect statistics on illegal drugs as a part of a broader survey or data collection. For example, the National Center for Health Statistics has collected information relating to illegal drugs as a part of its National Health Interview Survey and the Bureau of Justice Statistics has collected such data as a part of the Census of Jails. Other systems are more targeted to illegal drug-related issues, such as the Arrestee Drug Abuse Monitoring Program of the National Institute of Justice and the National Household Survey on Drug Abuse of the Substance Abuse and Mental Health Services Administration.
In the long run, the quality of the statistical data on illegal drugs will be enhanced if data collection efforts are consolidated into a smaller number of agencies that either conduct or sponsor the major data collection. In the committee’s view, there are many different routes to consolidate data collection, and we recognize that the final form will depend on the focus of the effort and the evolution of capabilities among different agencies. The major point is not that there is a single right way to consolidate efforts. Rather, when decisions are made about the structure of data collection, a decision that over the long run furthers consolidation is preferable to one that further fragments or reinforces the existing fragmentation of data collection efforts.
The committee reviewed a number of different possible models for consolidation of data collection. One that appears attractive would organize the major current data collections into two lead statistical agencies: one for health-related data and one for justice-related data. For example, the health-related data might be consolidated in the National Center for Health Statistics, while data related to law enforcement might be collected primarily by the Bureau of Justice Statistics. These two agencies are identified because they have evolved into full-line statistical agencies with professional statistical staffs and a considerable degree of independence.
If data collection efforts are consolidated in a smaller number of agencies, it will be important to retain the expertise and continuity of effort of existing agencies. There are a number of ways to ensure that the transitional costs are minimized:
Statistical agencies could improve the quality of the data by placing actual data collection in agencies with the greatest expertise in the collection of data in particular formats. For example, school-based surveys could draw on the expertise of the National Center for Education Statistics, hospital-based surveys could draw on the expertise of the National Center for Health Statistics, and justice-based surveys could draw on the expertise of the Bureau of Justice Statistics.
One area of particular concern is producing price indices and other economic data on illegal drugs. To date, the efforts to construct these data have proceeded without drawing on the expertise of agencies that specialize in price and other economic data, such as the Bureau of Labor Statistics. The procedure described above for developing economic statistics will improve work in this area.
Consideration should be given to using interagency transfer of funds to obtain the services of professional statistical organizations so as to strengthen and extend the professional expertise pool for data collection, review, and analysis.
To maintain adequate resources for the collection of statistics on illegal drugs, policy, research, and operating agencies (such as the Substance Abuse and Mental Health Services Administration, the National Institute on Drug Abuse, the National Institute of Justice, and the Drug Enforcement Administration) could retain a role in funding (much as the National Institutes of Health does with the National Center for Health Statistics) and thereby ensure that the capacity to compile data on illegal drugs is maintained. Block grant money and other needed funds should be transferred to the lead statistical agencies using interagency transfers to ensure continuity of funding.
The lead statistical agencies should foster collaboration with drug policy and research agencies to ensure that their work is relevant to clinical and law enforcement needs. Such arrangements are currently in place in other fields. For example National Institute on Child Health and Human Development funds the National Survey of Family Growth. The staff of the National Institutes of Health plays a large role in shaping the survey, yet the statistical lead is assumed by the National Center for Health Statistics.
It is important to maintain the continuity of the professional staffs that currently work on surveys while their work is integrated into the work of the statistical agencies. Alternatively, they could continue to collect data under delegated authority from the statistical agencies, but operating within a framework that emphasizes the independence of statistical operations.
Consolidating the authority to collect and report statistical data on illegal drugs in a very small number of statistical agencies is a long-term goal. It should not be undertaken until these agencies acquire adequate financial resources as well as the requisite professional and technical expertise and staff. In the interim, the committee recommends strengthening the existing organizations for data collection, analysis, and reporting so that they would adhere more closely to appropriate professional standards for statistical units.
In summary, the quality, range, reliability, and utility of statistical data on illegal drugs can be improved by undertaking several measures to improve the institutional structure at the federal level. Two key steps are necessary for improving the quality of data on illegal drugs:
Efforts should be taken to strengthen the professional staffs and capabilities of the organizations responsible for data collection and analysis and to increase the independence of their operations.
Data collection should over the long run be consolidated into a small number of statistical agencies, perhaps two, which would take a leadership role in organizing and collecting statistical data at the federal level.
Upgrading collection efforts in existing agencies and consolidation of data collection are essential tasks for improving the scope and quality of data on illegal drugs. Unless these organizational goals are achieved, the nation will continue to be poorly informed and policies on illegal drugs will be operating largely in the dark. But these steps require determined and effective leadership. Our final recommendation therefore concerns the need for leadership in organizational improvements and reorganization.
The committee recommends that the Office of National Drug Control Policy place organizational improvements for data high on its agenda in the immediate future. If it does not move quickly to implement the changes required to improve statistical data the President and Congress should find other ways to ensure that the substantive and organizational changes are swiftly and effectively achieved.
Executive Office of the President 1999 Report of the Drug Control Research, Data, and Evaluation Committee. Washington, DC: Office of National Drug Control Policy.
National Research Council 1992 Principles and Practices for a Federal Statistical Agency. Committee on National Statistics. Washington, DC: National Academy Press.
2001 Principles and Practices for a Federal Statistical Agency, Second Edition. Committee on National Statistics. Washington, DC: National Academy Press.