Modern science is ever more driven by computations and simulations. In particular, the state of the art in space and Earth science often arises from complex simulations of climate, space weather, and astronomical phenomena. At the same time, scientific work requires data processing, presentation, and analysis through broadly available proprietary and community software.1 Implicitly or explicitly, software is central to science. Scientific discovery, understanding, validation, and interpretation are all enhanced by access to the source code of the software used by scientists.
This report of the Committee on Best Practices for a Future Open Code Policy for NASA Space Science of the National Academies of Sciences, Engineering, and Medicine investigates and recommends options for NASA’s Science Mission Directorate (SMD) as it considers how to establish a policy regarding open source software (OSS) to complement its existing policy on open data. In particular, the report reviews existing data and software policies and the lessons learned from the implementation of those policies, summarizes community perspectives, and presents policy options and recommendations for implementing an OSS policy for NASA SMD.2
The purposes of any open code policies that SMD may develop are to serve the goals of SMD and NASA. Based on NASA’s vision, mission, and mandates; guidance provided to the study committee from NASA representatives; and the general interests of science and society, the committee identified the following seven goals:
- Enhance and enable innovation and discovery.
- Increase the visibility, accessibility, and reuse of NASA-funded code.3
- Facilitate scientific reproducibility.
1Community software, also called community source software, consists of software developed by a group effort, usually in an open source environment. These software packages are often widely reused.
2 This report does not address software related to spacecraft operations, because of safety and national security concerns that would be raised by the publication of such software. Neither does this report deal with software that supports NASA’s administrative functions. Rather, the report focuses on software related to NASA’s core science interests: research and technology development related to basic science and engineering, design concepts, mission support, aeronautic vehicles, and major research and engineering facilities.
3Source code consists of sets of human-readable statements written in a programming language that together compose software. The terms code and source code are often used interchangeably. Software is a general term used for computer programs and applications that provide users with some degree of utility or produce a result or service. Software can be distributed in executable form, as source code or as a service via the Internet.
- Encourage collaboration inside and outside NASA.
- Maximize NASA’s benefit to society.
- Respect the security and privacy of citizens.
- Comply with broader government policies.
A guiding maxim behind all of these goals—and the recommendations in this report—is that software needs to be as open as possible; as closed as necessary.
Past and Current Policies
The committee reviewed data policies at NASA and the U.S. Geological Survey (USGS); data management plans at NASA, the National Science Foundation (NSF), and USGS; software management at a number of federal agencies; federal policy; and large community software projects. Many of the lessons learned can be summarized by the statement “Software is data, but data is not software.” Software is included in the definition of data (Section 2.1), but software is copyrightable, whereas data is not. The ability to claim copyright is an important distinction in software versus data policy implementation. Yet, there are important lessons to be learned from the implementation of open data policies.
Some fields within NASA and elsewhere have already established a culture where openness is expected. An open data access policy for Landsat satellites, which are jointly managed by USGS and NASA, has dramatically increased the economic value and exploitation of the data for research, commercial, and land use management applications. NASA’s efforts to share data have been facilitated by the development of infrastructure, such as formal data archive centers, by SMD. These centers enable responsiveness and timely delivery of data to the public, facilitate data archiving, and provide data access and visualization tools that would be inefficient for individual researchers to create.
The expansion of science enabled by making data open to the public has driven efforts for even greater openness across the federal government (see Section 1.1). Today, federal agencies are required to release at least 20 percent of new software as open source, and NASA encourages vendors to use open source technology.4 Indeed, it is likely that success in the development and implementation of open data plans will continue to create an environment that fosters the development and implementation of open software. While open access policies can dramatically increase the economic value and exploitation of federally funded resources and have unanticipated applications that benefit society, both the policy review and lessons learned from community perspectives highlight the need for a careful and thoughtful process that responds to community feedback during the transition to any new policy. The committee found that it was the changes in agency data policies that prompted changes in community norms such as accepted practices regarding data sharing. Community understanding of new requirements for open data, and prospectively for OSS, has been facilitated by the use of consistent language, clear proposal submission guidelines, and the availability of educational resources.
Software comes in many varieties, and developing open source policies and implementation plans for software is more complex than for data. In some cases, software policy will need to take into account funding sources, the parties involved, the development history, and the size and complexity of the software and computational requirements. For example, a large community-source software project may involve contributions from different institutions, agencies, and countries, each of which may have its own software policies and legal constraints that impede software sharing, export, and licensing practices. Crafting a workable policy in such situations will not be easy. Nonetheless, there are several practices that could generally build support for and facilitate the implementation of OSS policies.
Some scientific communities have already embraced OSS, but others are less familiar with it. Within NASA, program managers understand their research communities best and are in close contact with many of the scientists that new policies would impact. Enacting any new policy that requires a shift in culture also will require community
4 This Office of Management and Budget (OMB) directive was intended to be a 3-year pilot, but it appears to still be in effect and was referenced in the Environmental Protection Agency (EPA) 2018 policy, https://www.epa.gov/open/interim-open-source-software-oss-policy.
support for successful and efficient implementation. Building support for OSS could include pilot studies, gaining the support of program managers, the use of funding as an incentive for researchers, clear reporting guidelines and evaluation criteria, and evaluation of compliance by groups that are independent from a given program—to increase the confidence of the community that open source policies are being fairly implemented. The adoption of open source policies is also being facilitated by journals and publishers who are moving forward in support of a more open science environment, providing both enhanced recognition and access to data and software. An initial and continued assessment of NASA-funded research projects for current OSS practices would be useful for analysis and fine-turning of policy implementation. Understanding community practices sets a baseline for future progress.
The committee examined the perspectives of NASA’s space and Earth science community through the collection of white papers from and conversations with members of the community (see Appendix C). Overall, the community expressed broad support for OSS. Openness and transparency are seen as central to scientific validity and reproducibility, but various challenges occur in the implementation of policy. A majority expressed positive experiences in opening code and described a range of advantages, including efficiency, greater collaboration, reduced duplication, greater use of particular codes, more robust code, and a broadening of the user community. OSS can improve the testing of codes, facilitate the ability of scientists to conduct reproducible research, and enhance the transparency of research.
Many of the white papers, on the other hand, emphasized issues and even pitfalls when trying to regulate the open sourcing of software. Concerns included legal ramifications, institutional barriers, costs, the level of effort required to implement OSS policies, the need for training and education, and other impacts on individual scientists and their careers. Some suggested that an open source policy may not always benefit science, because for researchers, time spent publishing software comes at the expense of time spent doing science. While an open source policy may enhance science for other researchers, it could be at the expense of the original researcher’s scientific output. In addition, there are concerns that researchers may lose motivation to push the boundaries of innovation in their software if they know that they have to immediately release it to the general public instead of having several years to take advantage of the new technology, potentially leading to less innovation in software development. Because doing science and developing OSS are different, but complementary, activities with different motivations and outcomes, OSS policies may be more successful if they clearly identify value in both activities.
Many concerns reflect misunderstandings about open source licensing and processes. Others reveal legitimate legal and institutional barriers. In some cases, it will be necessary to change the culture of institutions as well as scientists. Most unease within the community stems from the culture of how science is currently competed and conducted. For many software projects, open sourcing the code from inception is ideal. For others, a period to verify and validate the code in a research mode may be a better approach. SMD will need to address these concerns as it develops open source policies. In particular, SMD needs to foster a new culture of openness and encourage a social norm of sharing and collaboration, in part by incentivizing the development of OSS in the academic community through the use of targeted grants, fellowships, and prizes. The move toward openness is also facilitated by the establishment and use of open source libraries (code and tools used by programmers when writing software) to collect and disseminate community software. An incremental and flexible approach (as discussed in the policy options below) to OSS software will allow researchers to adjust to new requirements and minimize the impact on their scientific productivity.
Work toward a cultural norm of openness has already begun, with the establishment of open data policies, support, and infrastructure. This progress needs to continue with carefully constructed support for OSS, beyond simple policy development and implementation. A well-defined open source policy, combined with functional processes for review and release of software, can substantially reduce fear, uncertainty, and doubt about making software open source and can be a major enabler of open source development. The finding and recommendations below are based on the committee’s assessment of community perspectives.
Finding: The NASA science community generally recognizes the value of open source software and supports the principles of openness, but concerns prevail on the details of implementation and the impact on science and scientific careers.
Recommendation: NASA Science Mission Directorate should explicitly recognize the scientific value of open source software and incentivize its development and support, with the goal that open source science software becomes routine scientific practice. (Chapter 4)
Recommendation: NASA Science Mission Directorate should initiate and sponsor programs to educate and train researchers in open source best practices. Topics could include, but are not limited to, export controls, licensing and intellectual property, workflows, and software development. These resources could be made available to the community via in-person trainings as well as webpages, screencasts, and webinars. (Chapter 4)
Recommendation: Any open source software policy that NASA Science Mission Directorate develops should not impose an undue burden on researchers; therefore, any policy should be as simple as possible, and any mandates should be fully funded. (Chapter 4)
Recommendation: NASA Science Mission Directorate should support the infrastructure, governance, and maintenance of a healthy open source community, taking advantage of existing community resources to the greatest extent possible. (Chapter 4)
Recommendation: NASA Science Mission Directorate should support open source community-developed libraries that advance NASA science. (Chapter 4)
Recommendation: NASA Science Mission Directorate should foster career credit for scientific software development by encouraging publications, citations, and other recognition of software created as part of NASA-funded research. (Chapter 4)
In this section, the committee outlines a selection of policy options, including both incentives and mandates, for NASA SMD to consider. Based on the charge to the committee and discussion with NASA officials, the committee operated under the assumption that SMD will transition to a greater level of openness in accordance with federal policy. It is important, therefore, that NASA ensures that the transition helps advance science, foster collaboration, and generally advance the goals listed above. The committee believes that the best way to achieve this is to work toward a cultural norm of robust OSS development and maintenance. This will not happen overnight and will require ongoing strategic investment.
The options below can be considered a sort of toolbox to draw from to help move the community toward greater openness while recognizing that different disciplines and code types will have different requirements and transition at different rates. Incentives will help to move the community norms toward greater openness regardless of whether mandates are eventually implemented. Overall, the committee believes that there will need to be a combination of different incentives in place and transition to mandates only as appropriate.
Recommendation: NASA Science Mission Directorate should consider a variety of policy options depending on discipline and software type and transition to greater openness over time. (Chapter 5)
The committee identified the following three OSS policy options:
Policy Option A: Continue Status Quo
Currently, NASA SMD has no division-wide OSS policy regarding software publishing, distribution, or licensing. Option A would continue to allow individual NASA programs to determine whether they and their research communities are interested in moving toward open source. Some programs and modeling centers have already taken steps toward openness. Option A could eventually lead to OSS being required or becoming a de facto norm in some areas, but in others it would remain unusual. Without SMD coordination of an OSS policy, missteps in one division could potentially be repeated in another.
Policy Option B: Incentivize Openness
Option B would preserve community interests while gradually moving to the wide adoption of OSS. The goal of this option is to build trust while working toward making openness a community norm. With mandates absent or delayed, community pressure toward openness would naturally increase as investigators compete for the incentives.
Success with this policy option would depend on the allocation of adequate resources. Incentives within the current budget that lead to reduction of research funds will be less accepted by the community. There may be a delay in the scientific return from research funding. Over the long term, however, moving toward more openness is likely to provide a net benefit to science, as more researchers take advantage of open software. However, because incentives to adopt OSS may be applied at different rates and perhaps be absent altogether at some governmental agencies, some researchers may not participate, possibly gaining a research or career advantage over those who devote time and resources to opening their software.
The committee identified five specific elements, one or more of which could be adopted as part of Option B.
One or more of these elements could be adopted as part of Option B. Each element has its pros and cons, and they are likely to be applied differently for different software types.
Option B1—Funding for Full Open Source Software Proposals
Under Option B1, SMD or its divisions would allocate funding for new proposals addressing an OSS need, such as to open existing software with community reuse potential or replace it with functionally equivalent OSS, to develop new or maintain existing OSS, or to extend community open source libraries and frameworks. Proposals would be required to include a software management plan (SMP) that describes new software produced during a project and how it will be handled during the project and archived afterward. This option allows for a prioritized approach that recognizes the cost of community software development and creates or builds scientific software projects that other researchers can reuse.
This option could delay scientific returns within the programs implementing it, as scientists spend time to gain experience and familiarity releasing software. It would open only some software, and it may provide a disincentive for groups who do not win funding to open their software.
Option B2—Optional Proposal Open Source Add-On
Under Option B2, scientific research proposals submitted to SMD in existing grants programs could optionally include a distinct section to justify additional effort and funding to open software from the project and to provide a software management plan. Unlike B1, the OSS management plan and funding is an augmentation to the scientific proposal. This option is otherwise similar to B1, with the difference that there may be situations where the scientific merit of a proposal is not rated high enough for support, but the open source add-on is seen to have significant value.
Option B3—Pilot Software Management Plans
Under Option B3, specific programs within SMD would begin to require SMPs for scientific proposals containing substantial new software development as part of the proposed research. Requiring an SMP would not mandate software openness, but it could gradually expand existing policy and impose more specific requirements over time. In addition, the requirement for SMPs would be phased in; initially, it would apply only to selected SMD programs. The goal would be to gradually develop an effective policy by identifying different approaches to making software more open and by responding to community feedback. The gradual implementation of requirements for using SMPs would reduce the extent to which such requirements would disrupt SMD programs, and it would allow SMP requirements to be fine-tuned based on the results of the pilot programs.
However, Option B3 imposes an additional requirement that researchers must adhere to and that evaluators must consider. If B3 is implemented rapidly in scientific communities unfamiliar with OSS, innovation and science could suffer due either to inexperience making software open or through the additional burden of time spent by researchers on software. It is unclear what implications successful proposers would face if their stated SMP goals were unmet, except by evaluating previous practices, which would apply only to previous OSS funding from NASA.
Option B4—Support for Open Source Libraries and Infrastructure Software
Under Option B4, SMD would use existing funding mechanisms or allocate SMD employees to support and adopt open-source libraries and infrastructure software that are widely used in NASA-funded research. This option could improve community software quality and generate savings for NASA as a whole. Some software of this type, however, currently exists without dedicated NASA funding.
Option B5—Annual Prizes for the “Advancement of Open Source Software Development and Impact”
Greater recognition for scientists for creating quality OSS would enhance their career advancement. Under Option B5, an SMD award or prize could provide some recognition and visibility for the importance of OSS. This prize would recognize how OSS provides value to NASA. Implementing this option, however, would require resources and take time away from other activities, because SMD needs to create the prize, publicize it, organize a review committee, review applications, and make a selection. Awards and prizes are not nearly as effective at advancing scientific careers as funded proposals unless they are extremely prestigious.
Policy Option C: Mandate Openness
Under Option C, NASA SMD would decide that, by a certain date, software created through NASA SMD funding will be open source, with only a few, strongly justified exceptions. Mitigating the concerns raised in Chapters 3 and 4 would require a period of transitional activities, happening at rates that may vary by program and software type. Moving too rapidly into a mandate would likely be counterproductive and jeopardize future OSS transitioning efforts. The transition would require resources for training, software support and maintenance, and contributions to the overall software infrastructure. A mandate would be the surest and quickest way to increase the transparency of NASA science and to satisfy relevant federal policies; it could potentially enhance NASA’s national and international reputation as a leader in open science; and experience with open data policies suggests
that an OSS mandate could drive other agencies, both nationally and internationally, to enact similar policies, thereby benefiting NASA SMD researchers. A mandate, however, may be the costliest option, requiring major enabling and sustaining infrastructure to enforce the mandate. For some software types, the cost could exceed the benefit. A mandate could flood repository sites with a large variety of software. A mandate might also hinder collaboration with other agencies in creating OSS, notably the Department of Defense (DOD), which would have to provide permission and may have additional security concerns to protect controlled unclassified information and export-controlled information. Mandates are more likely to be effective once incentives have been established and only if mandates are implemented over a carefully planned flexible transition period.
Conclusion: Immediately mandating open source across all software types and in all of SMD could damage the NASA science enterprise.
Conclusion: An incentive-driven transition period is needed before a comprehensive SMD open source software policy. Incentives and timelines will vary by software type and community experience.
Transitioning Toward Openness
A variety of policy options, which will depend on discipline and software type, are warranted and will facilitate the transition to greater openness over time with a clear path toward openness. In Chapter 5 (Section 5.4), the committee describes in detail which policy options may be appropriate for each of seven defined software types and a suggested time frame for moving to mandated openness. The committee considers 3 years to be the minimum transition time, which is applicable to only some software types or communities. Many programs or software types will transition more slowly because of different grant cycles, infrastructure availability, and general community readiness—although they will follow the same general path. There will, however, be limitations—some software cannot legally be open source, some legacy software may simply be too expensive to convert, and different software will have different levels of maintenance (sometimes none). SMD will need to continually balance trade-offs and priorities while continually assessing how policies are meeting their goals. This transition to a desired level of openness requires time and resources for training, software support and maintenance, and contributions to the overall software infrastructure. Introducing OSS requirements without strategic investment in software development and maintenance may not advance innovation and discovery and other goals.
An assessment of how the scientific community uses software before, during, and after implementation of policies could help advance policy goals more efficiently. Implementing a policy that affects how scientists perform their research is a delicate undertaking, and continual assessment will be important to minimize disruptions and capitalize on successes. A measure of how changes in software release policies relate to scientific efficiency and advancement may be difficult to clearly articulate, especially in a short time frame, since many effects related to shifts in policy may develop slowly. Different measures may be needed in different communities. Nevertheless, some attempt at assessment would likely improve policy implementation.
The Copyright Act of 1976 ensures that any original creative work, including computer source code, is automatically protected by copyright once created, except for work created by the federal government, which is excluded by U.S. Code.5 This generally restricts use of software unless the owner has granted a license. A license is considered a public license if the owner grants permissions to the public as a whole to use their software as long as they abide by the license terms.
5 17 U.S. Code § 105 - Subject matter of copyright: United States Government works.
Some licenses are more permissive than others. The most permissive licenses place the least restrictive conditions on use and are very close to dedicating the software to the public domain, putting few restrictions on use or modification of the software for any purpose. This allows users to restrict the redistribution of their own software and contributions by others, even if that software is derived from OSS. In contrast, the most restrictive class of open source licenses, exemplified by GPL,6 requires that derivative works, if released, be released under the same open source license as the original. This ensures improvements and changes to OSS are shared back with the public. To be considered open source, software needs a license that complies with the Open Source Definition. One of the criteria of the definition is that open source licenses “must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software.”7
NASA SMD releases some software created by civil servants under the NASA Open Source Agreement (NOSA 1.3).8 The Open Source Initiative (OSI) approved NOSA, but some controversy about specific provisions in the agreement subsequently ensued, and it was determined to be incompatible with GPL.9 NASA indicated that it has attempted to address some of these compatibility concerns in its latest version, NOSA 2.0, but approval of NOSA 2.0 by OSI remains pending as of the date of this report. NASA will need to consider how best to balance the different goals of enabling innovation, facilitating scientific reproducibility, stimulating the economy, and benefiting society when recommending particular licenses that are as open and permissive as possible and only as closed as necessary.
Recommendation: NASA Science Mission Directorate should encourage the use of standard open source licenses, but not mandate a particular license. Nonstandard licenses should be justified in the software management plan. (Chapter 5)
Software released by NASA employees undergoes a rigorous vetting process to ensure the legality of its release, ensure compliance with software engineering standards, and prevent disclosure of restricted information. The same process applies for all software regardless of the length, topic, or a priori risk of the code. The NASA Technology Transfer Program has recently made major improvements to the process. Nevertheless, NASA’s current internal software release policy procedures can cause undue and potentially harmful delays in the release of low-risk software. The process could also be improved for software in more sensitive areas by identifying the likely risks, working with legal experts in planning software to reduce risk, and expediting the final review by focusing it on the areas of concern.
Recommendation: NASA Science Mission Directorate should develop internal policies and external legal language conducive to the swift release of open source scientific software, and the full participation of NASA employees in internal and external open source projects, without jeopardizing national security or incurring legal liability. (Chapter 5)
Technology changes quickly, and this affects how scientists do research and create software. Any OSS policy based purely on licensing considerations could possibly be circumvented. For example, software as a service (SaaS) is a delivery model where users access software and data through a web interface. Since the software itself is not copied, copyright license terms may not be triggered, particularly for permissive open source licenses. This raises concerns about source code access, reproducibility of science, and long-term sustainability and maintenance,
because the availability of the software may change without notice to the user or disappear entirely during an investigation. So even though SaaS and other computing technologies can be used in a positive way, they can also be used as a mechanism to circumvent policy, and software management plans will need to address the use of SaaS in new software development. However, there is at least one open source license that includes provisions prohibiting use of the licensed code as SaaS.
When it comes to software, openness is not enough to advance science. The basic act of releasing software as open source is not difficult, but it can evoke some complex considerations. To make OSS truly useful and helpful in realizing NASA’s goals requires a coordinated, end-to-end development approach supported by infrastructure, community practices, and education over the long term. Ultimately, OSS will likely advance science, but there will be transition and maintenance costs requiring a careful balance of trade-offs and active engagement by program managers.
The recommendations in this report cover options to increase community education and training and to ease implementation of new policies or requirements. It is important to note that most of the committee’s recommendations apply regardless of whether NASA SMD explicitly requires OSS. Creating a cultural shift toward greater openness will be challenging. Many of the lessons learned from the implementation of open data policies can be applied to the implementation of an OSS policy; however, OSS is more complex than open data. The recommendations allow for an implementation of open source that is carefully planned, gradually implemented, and well-coordinated across NASA SMD.