5

Policy Options and Recommendations

The committee outlines a selection of policy options below, including both incentives and mandates for NASA’s Science Mission Directorate (SMD) to consider. Based on the charge to the committee and discussions with NASA officials, the committee operated under the assumption that SMD will transition to a greater level of openness in accordance with federal policy (see Chapter 1). It is important, therefore, that NASA ensures that the transition helps advance science, foster collaboration, and generally advance NASA goals (see Section 1.2). The committee believes that the best way to achieve this is to work toward a cultural norm of robust, open source software (OSS) development and maintenance. This will not happen overnight and will require ongoing strategic investment.

SMD does not currently have division-wide policy regarding software publication, distribution, or licensing. As described in previous chapters, each software type may have its own legal requirements, and raise different policy issues and concerns from the community on the value and practicality of openness. Correspondingly, existing NASA software varies across a spectrum of openness. Each division may have its own policies, and the chief information officer (CIO) and Technology Transfer Office may differ on matters of OSS release. SMD may choose an overall level of openness for new software produced by and for the directorate in the long term, but individual programs and software types will advance toward greater openness at their own rate and through different means. The options below can be considered a sort of toolbox to draw from and help move the community toward greater openness while recognizing that different disciplines and code types will have different requirements and transition at different rates. Incentives will help to move the community norms toward greater openness regardless of whether mandates are eventually implemented. Overall, the committee believes that there will need to be a combination of different incentives in place and transition to mandates only as appropriate.

Recommendation: NASA Science Mission Directorate should consider a variety of policy options depending on discipline and software type and transition to greater openness over time.

With all that in mind, the committee presents the following three open source policy options at a basic level:

Continuing the status quo (Option A), allows individual SMD programs to determine whether they and their research communities are interested in moving toward open source. Ideally, regardless of what NASA decides, as accepted research metrics begin to routinely include a measure related to software impact, OSS will gradually become more common at least in some disciplines. Option A could eventually lead to OSS being required or a de facto norm in some areas, but in others it would remain unusual. This is the least strategic option and is unlikely to bring much change or fully realize the value of OSS.

SMD could accelerate the move toward openness, especially in programs less inclined to it, by offering specific incentives to investigators (Option B). This could be the long-term policy of SMD or could be a step on the way to some level of SMD-mandated openness (Option C). These incentives can also contribute to fostering an OSS culture and norm of sharing. Section 5.4 describes how incentives and mandates can be combined for different types of software.

5.1 POLICY OPTION A: CONTINUE STATUS QUO

As stated earlier, SMD does not currently have division-wide policy regarding software publishing, distribution, or licensing. Option A continues to allow individual NASA programs to determine whether they and their research communities are interested in moving toward open source. Some programs and modeling centers have taken steps toward openness, but there are no SMD coordinated open-source requirements, education efforts, or trainings. Several examples of existing policies for parts of SMD are discussed in Chapter 3. The community concerns raised in Chapter 4 would mostly remain unaddressed. This option could lead (and may have already led) to inequalities in access to results if some, but not all, programs mandated OSS. Without guidance from NASA, others (including publishers who are increasingly requiring OSS) will determine the course of OSS policies.

Option A also leads to approaches customized to certain communities. For example, some large modeling centers, such as NASA Community Coordinated Modeling Center (CCMC), have what might be called an “open-to-run” approach. They host software packages and run them on in-house hardware, possibly on thousands of CPUs for months at a time. This allows outsiders to run software at the center with cases and configurations of their choice, but under various restrictions. Usually, the software cannot be taken elsewhere. Sometimes the source code is viewable; sometimes not. This ad hoc approach may be a pragmatic compromise, because the software cannot be practically run on other systems. It may also be a way to provide some protection for the investigators developing the software while satisfying some level of research transparency. Investigators may have some ability to reproduce results and learn from the software depending on the restrictions, but this is not OSS.

5.2 POLICY OPTION B: INCENTIVIZE OPENNESS TO ACCELERATE THE CHANGE

Option B uses incentives that preserve community interests while moving to OSS. The goal of this option is to build trust while working toward making openness a community norm. With mandates absent or delayed, community pressure toward openness would naturally increase as investigators compete for the incentives.

As described in Chapters 3 and 4, some policies, such as the compulsory inclusion of a data management plan, was initially not well understood and was seen as a burden to most investigators, even those who do not produce any data. As education efforts and online resources increased, data management plans became a normal section to all proposals. The archive of widely used data at NASA archive centers, rather than individual investigator’s websites, has improved the quality, availability, and usability of NASA investigator-produced data. NASA’s Living with a Star (LWS) program offered researchers funds to make widely used data available to the general community and was better received than unfunded mandates. The implementation of an OSS policy could learn from these experiences, through implementation of options given below.

Success with Option B depends on the allocation of adequate resources. Incentives within the current budget that lead to reduction of research funds will be less accepted by the community. There may be a delay in the scientific return from research funding. Over the long term, however, motivation to move toward more openness is likely to provide a net benefit to science, as more researchers take advantage of the opened software. Careful consideration and guidance to proposal review panels would be required to award incentives to those proposals

with software that is more likely to be reused. This option also recognizes the full cost of community software development.

Because openness incentives may be applied at different rates and perhaps be absent at other governmental agencies, some researchers may not participate, possibly gaining a research or career advantage over those who devote time and resources to opening their software. For example, if one program manager or agency called for OSS and another did not, a researcher might target proposals to programs that allowed them to delay releasing their software.

The five incentives identified by the committee are the following:

One or more of these elements could be adopted as part of Option B. Table 5.1 summarizes how these may apply to the different software types presented in Chapter 2.

5.2.1 Option B1—Funding for Full Open Source Software Proposals

With Option B1, SMD or its divisions allocate funding, either in existing or new grant programs, to open existing software with community reuse potential or replace it with functionally equivalent OSS, to develop new OSS, to maintain existing OSS, or to extend community open source libraries and frameworks. In this case, the proposal provides a software management plan (see Box 5.1) that describes what the software does, its value and user community, the needed software and documentation updates, test suites, licensing, any other legal issues, and a plan for long-term sustainability. The funding could be delivered as contracts or cooperative agreements to allow full oversight with milestones and deliverables.

Similar to the implementation of NASA’s data management plans, a corresponding NASA website could explain what is required for SMPs and provide suggestions and examples to ensure less-experienced researchers are not disadvantaged. Online tools (e.g., https://dmptool.org) could be developed to further help investigators.

Pros: Option B1 provides direct funding for scientists to develop OSS solutions. It allows for a prioritized approach, as evaluation panels of community experts decide which software to support. It recognizes the cost of community software development and it creates or builds scientific software projects that other researchers can reuse, accelerating science.

Cons: Option B1 could delay scientific returns within programs implementing it, as scientists spend time to gain experience and familiarity releasing software. It opens only some software, and it may provide a disincentive for groups who do not win funding to open their software. It may not fully recognize the long-term costs of maintaining the software.

5.2.2 Option B2—Optional Proposal Open Source Add-On

With Option B2, SMD provides an opportunity to add additional pages to scientific research proposals in existing grants programs to justify additional effort and funding to open software from the project and to provide an SMP. The review panel and program manager evaluate add-on proposals and allocate funds to the best of them. Unlike Option B1, the OSS management plan and funding is an augmentation to the scientific proposal.

The pros and cons are similar to those for Option B1, with the difference that there may be situations where the scientific merit of the proposal is not rated high enough for support, but the open source add-on is seen as of significant value.

TABLE 5.1 Policy Recommendation Summary by Software Type

^a Yes, for maintaining current libraries and creating major new libraries.

^b Items with an asterisk (*) indicate that the policy may be applied in some programs before others.

^c Yes, for high priority, general-use legacy pipeline.

NOTE: General approach of moving toward a full SMD-wide OSS policy based on software types described in Chapter 2 (see Table 2.1). The table indicates which policy options may be appropriate for each software type and the recommended time frame for moving to mandated openness (Option C).✓ means that an option can be implemented right now; * means that it might be implemented in some programs before others, and likely with lower priority; and x means that the option is not appropriate for that code type. An x might appear because going straight to a mandate is recommended for that software type (e.g., sensor software), or because a more comprehensive solution involving OSS and other requirements would be necessary (e.g., single use). More detailed descriptions and recommendations are given below.

5.2.3 Option B3—Pilot Software Management Plans

With Option B3, specific programs within SMD begin to require SMPs for scientific proposals containing substantial new software development. Requiring an SMP would not mandate openness, but it could gradually expand existing policy and impose more specific requirements over time. The goal is to gradually develop an effective policy by identifying different approaches to making software more open and responding to community feedback.

Where researchers receive NASA funds to deliver OSS, they also need to provide evidence of OSS practices. The SMP could be added as an evaluation criterion during proposal reviews.

Pros: An SMP would provide an opportunity for the research community to document and be evaluated on their software practices and for NASA to learn from the experience. It will allow reviewers and program managers to understand how knowledgeable and experienced the proposers are at OSS development.

Cons: Option B3 imposes an additional requirement that researchers must adhere to and evaluators must consider. If implemented rapidly in scientific communities unfamiliar with OSS, innovation and science could suffer due to either inexperience making software open or the additional time spent by researchers on software.

It is unclear what implications successful proposers would face if their stated SMP goals were unmet, except by evaluating previous practices, which would only apply to previous OSS funding from NASA.

5.2.4 Option B4—Support Open Source Libraries and Infrastructure Software

With Option B4, SMD uses existing funding mechanisms or allocates SMD employee time to support and adopt open source libraries and infrastructure software that are widely used in NASA-funded research. NASA support of software will demonstrate its commitment and promote the careers of scientists who spend time developing and improving these libraries. Because it is focused on broadly useful software, share-in-savings contracts might be one useful approach in this option.¹

Pros: Option B4 could improve community software quality and generate savings for NASA as a whole. With agency support, community software will increase in visibility and value, which will help the careers of researchers who develop them.

Cons: Some software of this type currently exists without dedicated NASA funding.

5.2.5 Option B5—Create an Annual Prize for the “Advancement of OSS Development and Impact”

With Option B5, greater recognition for scientists who create quality OSS would enhance their career advancement. A NASA SMD award or prize could provide some recognition and visibility for the importance of OSS. This might be similar to NASA’s current “Software of the Year” award,² but the focus would be on open source. The prize would recognize how OSS provides value to NASA. The prize could be judged on a number of criteria including, but not limited to, code contributions, software reuse or extension, training, and advocacy, as documented by application materials or testimonials and letters of nomination. NASA could consider partnering with scientific professional societies to present and administer the prize.

Pros: The prize provides an incentive and recognition for scientists who create quality OSS, while highlighting the importance of NASA SMD OSS resources.

Cons: The prize will require resources and take time away from other activities, as SMD needs to create the prize, publicize it, organize a review committee, review applications, and make a selection. Awards and prizes are not nearly as effective at advancing scientific careers as funded proposals unless they are extremely prestigious.

5.3 POLICY OPTION C: MANDATE OPENNESS

Under Option C, SMD decides that, by a certain date, software created through SMD funding will be open source, with only few, strongly justified exceptions. Mitigating the concerns raised in the prior chapters requires a period of transitional activities that may vary in timing and implementation by program and software type (Table 5.1) and sustained strategic investment.

Pros: An OSS mandate would be the surest and quickest way to increase the transparency of NASA science and to satisfy the recent Office of Management and Budget (OMB) memorandum “Federal Source Code Policy: Achieving Efficiency, Transparency, and Innovation through Reusable and Open Source Software”³ and NASA CIO response to “encourage vendors to use open source technology wherever possible.”⁴ It could potentially enhance NASA’s national and international reputation as a leader in open science. Experience with open data policies suggest than an OSS mandate could drive other agencies, both nationally and internationally, to enact similar policies, resulting in more OSS, thereby benefiting NASA researchers.

___________________

¹ Share-in-savings contracts are where “the Government awards a contract to improve mission-related or administrative processes or to accelerate the achievement of its mission and share with the contractor in savings achieved through contract performance,” https://www.law.cornell.edu/uscode/text/10/2332.

² NASA Software of the Year Award, https://partnerships.gsfc.nasa.gov/internal-inventors/awards/nasa-technology-awards-and-incentives/#softwareoy.

³ See https://sourcecode.cio.gov.

⁴ See https://code.nasa.gov/NASA-M-16-21-OCIO-Memo.pdf.

Allowing flexibility in the timing and implementation of a mandate for different software types or programs could improve buy-in and understanding from the NASA research community, especially if NASA uses the time to implement training programs and some of the incentives under Option B. This flexibility in implementation would also allow costs to be spread over time and offers more flexibility in budgeting.

Cons: A rapid mandate would likely cause backlash from NASA-funded investigators. It would likely put a financial strain on NASA-funded investigators, especially those on soft money and could lead to a drain of people from the field. The response from the community may be to follow the letter but not the spirit of the policy. A mandate may also be the costliest option, requiring major enabling and sustaining infrastructure to enforce the mandate. The cost of implementing mandates could exceed the benefit for some software types. A mandate might also hinder collaboration with other agencies in creating OSS, notably the Department of Defense, which would have to provide permission and may have additional security concerns to protect controlled unclassified information and export-controlled information (WP 31⁵). Last, behavioral research suggests that positive incentives can effectively drive both individual and group behavior, so mandates are more effective once incentives have been established.⁶

Conclusion: Immediately mandating open source across all software types and in all of SMD could damage the NASA science enterprise.

If mandates are implemented, a transition period toward openness is necessary with specific activities to help level the playing field, provide training and resources, and ensure research continuity. In many cases, incentives will need to be in place before firm mandates can be implemented. The transition schedule may need to be modified based on the response to incentives as they are developed. Program managers will play an important role by tailoring their policy execution to their understanding of communities and culture across SMD, to incremental needs depending on software types, and to community experience with open science practices.

Conclusion: An incentive-driven transition period is needed before a comprehensive SMD open source software policy is implemented. Incentives and timelines will vary by software type and community experience.

5.4 TRANSITIONING TOWARD OPENNESS

As mentioned, SMD should consider a variety of policy options depending on discipline and software type and transition to greater openness over time. In Box 5.2, the committee outlines one example of a path to openness for a program or software type that moves to an OSS requirement in 3 years. The committee considers 3 years the minimum transition time applicable to only some software types or communities (e.g., new instrument code or pilot efforts such as NASA’s Earth Science Directorate’s ACCESS). Many programs or software types will transition more slowly because of different grant cycles, infrastructure availability, and general community readiness; but they will follow the same general path. Achieving SMD-wide openness means implementing this path for every program and for all software types. There will, however, be limitations. Some software cannot legally be open source; it may simply be too expensive to convert some legacy software; and different software will have different levels of maintenance (sometimes none). SMD will need to continually balance trade-offs and priorities while continually assessing how policies are meeting their goals. Early assessments are critical in policy implementation and establish a baseline for long-term assessments (e.g., reuse, publications, citations). This transition to a desired level of openness requires time and resources for training, software support and maintenance, and contributions to the overall software infrastructure. Introducing OSS requirements without strategic investment in software development and maintenance may not advance innovation and discovery and other goals.

___________________

⁵ WP is used to reference the white papers submitted to the committee. See Appendix C for a full listing.

⁶ E.E. Smith, S. Nolen-Hoeksema, B.L. Fredrickson, G.L. Loftus, D.J. Bem, and S. Maren, 2003, Atkinson and Hilgard’s Introduction to Psychology, Wadsworth/Thomson Learning, Belmont, CA.

Box 5.2 describes the general path to take, but different software types will have different priorities and requirements in moving toward openness. Table 5.1 lays out how the various policy options described above could be applied and when and if mandates would be applied. Each of the different software types is then discussed in turn. Of course, there will be variance by discipline and community within each software type as well.

5.4.1 Policy Options Applied to Different Software Types

Table 5.1 lays out how the various policy options described above could be applied and when and if mandates would be applied. Each of the different software types is discussed below.

Libraries

Libraries are products that many people depend on, and the user’s ability to verify the software and to adapt it for the user’s own purposes is a key feature of software libraries for science. They would be a high priority to encourage to be OSS, and all the incentives are applicable. Mandating that new libraries and projects adding to existing OSS libraries be open source could happen fairly quickly. Existing closed libraries could be incentivized to become open source. Many libraries exist only because of researchers’ investments of large amounts of time. Funding key developers and covering expenses will ensure the continuing quality and feature sets of existing libraries.

Single-Use Software

Software written for use in unique instances is inherently lower priority for OSS policy. SMD may want to consider some proposal add-ons or pilots to transition single-use software to more broadly useful OSS, but mandates for opening single-use software would be challenging to implement. There may still be interest in opening the code to reproduce results or broaden a study. Journals may require code used in a publication to be available, because reproducing the results requires the inputs and configurations as well as the software that produced the results. The effort to produce reproducible research papers can be manageable if planned from the outset, but it can be impractical if undertaken after a multiyear project. This cultural shift will require broader incentives and commitment than just NASA can provide and will likely be based on the incentives created in scholarly communication that recognize open practitioners.

Analysis Software

Analysis software may involve a few or many investigators and may have specific or broad applications. It would be a high priority to incentivize openness for newly developed and broadly useful software. Legacy software and software created in collaboration with people not supported by NASA would be a lower priority and harder to mandate because of potential licensing challenges. For older software with poor documentation, testing, and so on, rewriting may be the better solution, if the software has broad use and existing contributors have the first opportunity to rewrite.

Modeling and Simulation Software

Modeling and simulation (M&S) software is in broad use and may receive targeted development funds. The diversity of software in this category is wide, and community cultures span the openness spectrum. Research reproducibility, study replication, and extension of results are all served by opening model software, but priorities for opening different models will vary. Policies in the short term may need to adapt to the research community and model scale (e.g., number of investigators involved). With some large software projects, it may be difficult to locate all the contributors and get them to agree to an OSS license. There may also be additional legal concerns that are specific to individual contributors. For commercial models (e.g., chemical kinetics or spectroscopy) that are in broad use, share-in-savings contracts may be possible to fund by open sourcing them. For many established models, the easiest path to openness may be rewriting the software, which could be financially challenging even if the original contributors are involved.

Frameworks

Modeling frameworks glue together different models. They manipulate model quantities, and it is important for researchers to see exactly how, so they can ensure, for example, that the framework follows relevant conservation laws and does not violate downstream assumptions. Open frameworks are a high priority, and all incentives can be applied to motivate the opening or replacement of key existing frameworks. Mandates may be appropriate for new development while recognizing similar concerns about licensing and documentation, as with M&S software.

Sensor Software

Science requires an understanding of the computations and manipulations made to data. Often, a mission’s data center delivers processed data and the processing changes periodically as calibration methods and data improve. It is thus imperative for the software doing these calculations to be available to all investigators, so that investigators can rerun the calculations with parameters and input data of their choosing. For this to happen, the software needs to be open.

All software that manipulates measurements taken by NASA sensors before those measurements are delivered to investigators needs to be open (and ideally, containerized). This may include software running on the spacecraft. As some data-processing centers integrate proprietary database-access software into their calibration process, retroactively applying this recommendation will require negotiating with those centers, and possibly opening only the portions of the software without proprietary storage and retrieval parts. All future contracts for any kind of sensor calibration and data processing software need to specify that the software will be open, documented, and supplied to investigators in a standard form.

Infrastructure Software

Infrastructure software helps manage and store data and helps researchers discover, select, and transfer data. With rapidly growing volumes of data and increasing cloud-based processing, access to the software behind an archive is of increasing interest. Standing up a new data center for a mission is expensive and time consuming. It is in NASA’s best interest to open the software in this category. This will enable principal investigator (PI)-run data centers for small missions to provide modern services and will ensure the best competition for larger data center contracts. The variety of existing infrastructure software, and their current states of documentation and dependence on proprietary components, makes it more practical to implement a future policy than a retroactive one. Openness requirements here are thus best handled on a case-by-case basis for existing software (which may be in use for decades), depending on the challenges and benefits involved. This implies that certain software will need to be maintained for a very long time.

5.5 ASSESSMENT AND FUTURE CONSIDERATIONS

As just discussed, there will be a variety of implementation strategies for the different options, disciplines, and software types. An assessment of community software users before, during, and after implementation of policies could help advance policy goals more efficiently. A Web-based survey of almost 2,000 people, 71 percent with a Ph.D. or equivalent, examined how scientists use software and computers for research. The survey found that scientists use computers differently than professional software developers, scientists work more on their own or in small projects, and they are not as familiar with tools relevant to larger projects (e.g., version control, verification, and testing tools).⁷ It will be critical for NASA to develop an ongoing assessment program to ensure that OSS implementation is advancing science, fostering collaboration, and generally helping to achieve NASA goals (see Chapter 1). The OMB memo on Federal Source Code Policy directed agencies to “collect additional data concerning new custom software to inform metrics to gauge the performance of this pilot.” Any metrics that SMD develops need to be in line with NASA’s goals (e.g., not just reuse, publications, citations, etc.). It will also be important to understand how the community is reacting to and adapting to new policies and incentives. SMD may want to consider funding research to explore that community evolution (e.g., Geiger et al. [2018] studies of the Moore-Sloan Foundation Data Science Initiative⁸) as well as conduct workshops, pilot projects, and other mechanisms to directly solicit community feedback. Formal economic analysis of the impact of OSS, similar to the

___________________

⁷ See https://www.americanscientist.org/article/how-do-scientists-really-use-computers.

⁸ R.S. Geiger, C. Mazel-Cabasse, C.Y. Cullens, L. Norén, B. Fiore-Gartland, D. Das, and H. Brady, 2018, Career Paths and Prospects in Academic Data Science: Report of the Moore-Sloan Data Science Environments Survey, SocArXiv, https://doi.org/10.17605/OSF.IO/XE823;R.S. Geiger, N. Varoquaux, C. Mazel-Cabasse, and C. Holdgraf, 2018, The Types, Roles, and Practices of Documentation in Data Analytics Open Source Software Libraries. Computer Supported Cooperative Work (CSCW), pp. 1-36, https://doi.org/10.1007/s10606-018-9333-1.

analyses done for data and described in Chapter 3, could also be helpful. Early assessments are critical in policy implementation and help establish a baseline for future assessments that might strengthen the science justification for broader open science policies.

Policy options will need to continually adapt to changes in software and technology. At present, the field of machine learning, for example, presents new challenges. A recent paper in Science studied 400 algorithms presented at two top artificial intelligence (AI) conferences and found that only 6 percent shared software and 33 percent shared data.⁹ As stated earlier, releasing data is not sufficient to ensure reproducibility. In machine learning (ML), the way the set is partitioned into training data and testing data may not be documented. Some ML frameworks apportion these subsets automatically and with a degree of randomness (“data shuffling”). The ML community is actively working to address reproducibility concerns (e.g., holding dedicated workshops at the top conferences, such as the International Conference on Machine Learning [ICML] and the Reproducibility Workshop). Research efforts focused on improving ML workflows are currently under way, so this problem may soon be resolved, but NASA will need to assess policy options to require more than source code and input data in order to ensure reproducibility.

Conclusion: SMD will need to assess and adapt policy to new computing technology developments to maintain its intent.

5.6 POLICY IMPLEMENTATION

With policy options for specific software types, NASA SMD also will need to consider implementation details. The committee provided some general recommendations in Chapter 4 that will help establish a general norm of OSS. More specific recommendations on implementing OSS policies, regardless of the options selected for different software types and circumstances, are provided below.

5.6.1 Licensing

Chapter 2 discusses the various legal issues around software release. Legal complexity around software release involves copyright, patents, and legal restrictions on what can be shared. Based on the input received by the committee, understanding of these issues is often lacking, and confusion within the research community and within NASA is common. Applying an appropriate license reduces confusion by clearly indicating the conditions under which software can be reused and redistributed. As defined in Chapter 2, OSS needs to have a standard license (e.g., Open Source Initiative [OSI]-approved), but choosing the right license is still complex. SMD investigators may also contribute code to existing projects, which may bring up issues of license compatibility.

NASA needs to balance the different goals it is trying to achieve with OSS. Industry favors more permissive licenses, whereas some investigators may favor more restrictive licenses or may desire to impose more restrictions until after they publish (see Figure 2.2). License compatibility is a complex yet important issue in ensuring effective software reuse. As noted earlier, the current NOSA license (1.3) is not well respected by the open source community (WP 28).^10,11 Licensing, compatibility, and general legal issues will need to be a major part of the education program recommended in Chapter 4.

Conclusion: NASA will need to balance the goals of enabling innovation, facilitating scientific reproducibility, stimulating the economy, and benefiting society when recommending particular licenses that are as open and permissive as possible and only as restrictive as necessary.

___________________

⁹ See http://www.sciencemag.org/news/2018/02/missing-data-hinder-replication-artificial-intelligence-studies.

¹⁰NASA Open Source Summit, 2011, https://www.nasa.gov/open/source/.

¹¹ See https://docs.google.com/document/d/1TagS_gwDhDfxjr7WpG78_aIcfoPO1tMXBPeCMEE3-Us/edit?hl=en#.

Recommendation: NASA Science Mission Directorate should encourage the use of standard open source licenses, but not mandate a particular license. Nonstandard licenses should be justified in the software management plan.

Standard licenses include, but are not limited to, those considered popular and widely used by OSI.¹² The use of standard licenses is also recommended by the Federal Source Code Policy’s Pilot Program (see Section 3.3.7).

5.6.2 Planning and Facilitating Software Release

Software release is becoming a normal part of the scientific process. The process is often swift, highly collaborative, and exploratory as incomplete solutions are worked out. NASA civil servant scientists need to participate in this process to remain competitive. Currently, software released by NASA employees must undergo a rigorous vetting process before its release, to ensure compliance with software engineering standards (Chapter 3 of NPR 7150)¹³ and to prevent disclosure of restricted information. The same process applies for all software regardless of the scale, topic, or a priori risk of the code. The procedure ignores the different software types and can be unnecessarily burdensome at times (e.g., WP 1). The NASA Technology Transfer Program has made major improvements to the process, recently. The program could improve the process further by enabling expedited review for some software.

Certain software requires rigorous and thorough review (e.g., software near export control boundaries or that includes commercial-off-the-shelf products), but much of the software developed by NASA SMD does not. Creating a streamlined review process for low-risk software would enhance the ability of NASA scientists to work more openly with the rest of the academic community and provide a model for other institutions. For example, Elsevier normally approves OSS release normally within 2 weeks (WP 30).

Expedited review could be based on current definitions of confidential information and the risk that a work could develop new knowledge in areas of security concern. For the vast majority of civil-servant authors, software review could be simple and quick, saving time and money without adding national security or legal risk.

For software in sensitive areas, identifying the likely risks in advance and working with legal experts in planning the software could reduce risk, while expediting the final review by focusing it on the areas of concern. For example, risky code might sometimes be limited to a single module that could be replaced or omitted in public release, while the full code may be made available under appropriate safeguards to trusted citizens.

Conclusion: NASA’s current, internal software release policy can cause undue and potentially harmful delay in the release of low-risk software.

Universities and other research organizations also have policies to control software release. Often these policies are designed to protect intellectual property rights of the institution. Investigators can usually release their software more easily if the open source arrangements were made prior to award of a grant or contract.

Recommendation: NASA Science Mission Directorate should develop internal policies and external legal language conducive to the swift release of open source scientific software, and the full participation of NASA employees in internal and external open source projects, without jeopardizing national security or incurring legal liability.

To implement this recommendation, SMD will need to work in coordination with NASA Technology Transfer Office and export control offices.

___________________

¹² See https://opensource.org/licenses.

¹³ See https://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2B.

5.6.3 Ongoing Compliance

Any OSS policy based purely on licensing considerations could possibly be circumvented. For example, software as a service (SaaS)—a delivery model where users access software and data through a Web interface, like at the CCMC—avoids distributing the code and sidesteps license requirements. Services can be free or require payment. Examples could be as simple as a free Web interface that collocates user-given data points with contemporaneous satellite data and as complex as a paid subscription to high-resolution ocean wave forecasts for shipping. Since the software itself is not copied, copyright license terms may not be triggered. Recent versions of the GNU General Public License (GPL) do require disclosure of software deriving from GPL-licensed work when used in a SaaS environment, but permissive OSS licenses do not. This raises concerns about source code access, reproducibility, and long-term sustainability and maintenance. Some third-party vendors provide useful SaaS, such as machine learning systems and software implementing patented algorithms. Reproducibility of the science may be impacted by SaaS systems, especially generic ones that may change without notice to the user, or that may disappear entirely during an investigation. Software management plans will need to include discussion of use of SaaS in new software development.

Conclusion: SaaS and other computing technologies can be used in a positive way but may also be used as a mechanism to circumvent policy.