BOX 1.5

The Decline in Federal Investment in Software Producibility Research

The extent and structure of federal IT-related R&D investment are documented in a series of reports published by the National Coordination Office (NCO) for Networking and Information Technology Research and Development (NITRD). These reports have been issued annually since the first such report was included in the FY1992 federal budget submission, and they document research sponsorship from a diverse set of federal agencies related to IT and networking. NITRD is a multi-agency coordination activity, which operates under the auspices of the White House Office of Science and Technology Policy (OSTP) and the Office of Management and Budget (OMB), but the funding that is reported is in the individual agencies’ budgets and generally under their control. The reports include extensive narrative descriptions of research accomplishments and plans, as well as a budget matrix that shows investment levels by agency, organized into a set of eight categories.1 The budget matrix shows both proposed amounts for the forthcoming fiscal year and approximate actual amounts for the then-current fiscal year.

There are two categories that relate to software producibility—Software Design and Productivity (SDP) and High Confidence Software and Systems (HCSS). The committee analyzed the trends in these two categories over the past decade and related its findings to the overall NITRD-coordinated budget that totals investment in all eight categories. (Note: The analysis excluded National Institutes of Health (NIH) data. This was done for two reasons: First, NIH changed its reporting methodology in 2010, which creates non-commensurability for a longitudinal analysis. Second, NIH allocations among the NITRD topic categories were determined through the use of an automated text-based pattern-matching algorithm. The committee believes this approach, particularly in topics related to software production generally (rather than, for example, the production of software for particular applications), is likely to lead to significant over-reporting of application software development projects as SDP or HCSS research.)

The principal result of the analysis is that the SDP and HCSS investments, separately and combined and in absolute dollars and as a percentage of the NITRD budget, have dropped considerably in the past 5 years. At the same time, the overall NITRD-coordinated budget has grown. For example, from 2004 to 2010 the combined allocation to SDP and HCSS fell by 45 percent, while the overall NITRD budget more than doubled. On a percentage basis, the combined SDP and HCSS allocation fell by a factor of almost four, from 24.6 percent of the NITRD total in 2004 to just 6.5 percent of the total in 2010.

One of the challenges in this type of budget analysis is the breadth of the categories and the imprecision of category boundaries. This challenge is unavoidable in the analysis of research budgets, but it is particularly difficult in the analysis of NITRD budgets because the different agency staff may apply slightly different criteria when categorizing diverse and innovative research projects. When category labels change, for example, with the introduction of the category of Cybersecurity and Information Assurance (CSIA), it is very likely that some projects in HCSS were relabeled as CSIA. Although it would be desirable to assess each grant for its relevance to the categories or, better, to the particular technical disciplines that support “software producibility,” this would be infeasible because of the large number of research grants and contracts and also because agencies are reluctant to share detailed data regarding awards and category assignments. An analysis of the narrative descriptions associated with the categories in the NITRD report suggests that there is an acceptably close alignment of SDP and HCSS with the overall investment that might directly relate to software producibility. The narrative descriptions do reveal some areas included in SDP or HCSS that might not be included in a “software producibility” category, for example, due to application specificity or other attribute.

Taking all this into consideration, the committee judges that the combination of SDP and HCSS is sufficiently close to a notional category of software producibility that we accept it as a surrogate for overall investment across NITRD agencies (NIH excluded) in research that relates to the present report.


1 The categories have slowly evolved over the years, but categories related to software have remained unchanged for the past decade.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement