Chapter 2

Lessons from Developing a Robust and Flexible Model

“As sometimes happens, the original goals of a Keck award do not come to fruition, but nature reveals new and fascinating paths.”

—2016 W. M. Keck Foundation Annual Report

The National Academies Keck Futures Initiative’s (NAKFI’s) evaluation and environmental scanning sketched a “map” that revealed a number of lessons and working assumptions leading to fascinating and fruitful paths. These discoveries were made over the course of 15 years as the Futures Model (see Chapter 1) developed:

These lessons were a catalyst to achieving the breadth and depth of the program’s scientific, educational, social, and cultural impact, and represent one of NAKFI’s most significant outcomes. They are explored in this and the next chapter through representative examples in the words of NAKFI alumni. Questions are provided at the end of each section for readers to leverage the lessons in their own work and contexts.

For the more than 2,000 alumni, their NAKFI participation created a lasting impression and a desire for more. In keeping with the spirit of the NAKFI Oversight Committee’s desire for future generations to receive the benefits of the Futures Model, Appendix VII provides additional guidance, tips, and tools for “In Practice: How to Think Like NAKFI.”

LESSON 1: SYSTEMIC CHANGE CAN START AT THE GRASSROOTS LEVEL

As observed by the Five-Year Review Panel, for NAKFI to effectively create impact across the myriad of organizations and interconnected systems that comprise the U.S. scientific establishment would have been a tremendous challenge. However, “[t]he nature of NAKFI as an experiment to examine, stimulate, and promote interdisciplinary research in exciting areas is more important than its capacity to drive institutional change” (NAKFI, 2008, p. 11).

Following a shift in evaluation strategy, however, NAKFI discovered that individuals can be powerful drivers of systemic change. Fostering science leadership that changes individual behavior can lead to changes in institutional behavior and drive institutions to adapt. NAKFI has shown there is value in starting at the grassroots level and investing in people who are motivated to change systems—an approach with the potential to ripple through the larger system for decades to come.

Cultivating Science Leadership Creates Lasting Benefits

A common theme in conference evaluations was that one of the most beneficial aspects of the NAKFI experience was the professional and personal growth resulting from it. This was true for established individuals as well as those early in their careers and for scientists and non-scientists alike. Bringing together researchers along the career continuum had significant value and engaging science writing and design students, artists, designers, and scientists created learning on all sides of the exposure.

There are numerous examples of how the NAKFI experience resulted in positive career impact both professionally and personally. Participants have reported making connections with others as either mentors or mentees, being invited to participate or lead collaboration and innovation initiatives at their institutions or with other agencies and professional societies, and expanding their approach to research to include other disciplines or science writers, designers, or young scientists.

A lesson from the NAKFI program is that bringing together an interdisciplinary group of scientists and other creative practitioners, including participants from all career stages, creates the opportunity to cultivate the science leaders of the future.

The Potential for Individuals to Influence the Scientific Endeavor Is Powerful

NAKFI embraced the premise that exposure to a positive experience can change behavior and evolved its model to support creating multiple venues for individual and group interaction from the familiar to the unexpected. It was a welcome surprise to discover through conference surveys, the 10-year review, and anecdotal stories that NAKFI participants were making a difference not only in their research but also in organizations. Alumni have reported a variety of experiences where they have influenced their team, department, institution, or community as a result of their individual efforts to promote collaborative and innovative science.

It would be difficult to carry out a causal study on the NAKFI intervention on larger systemic trends. That said, NAKFI monitored not only its impact on individual participants but also the level of support for interdisciplinary research and interest in the program by stakeholder organizations. Over time, NAKFI noticed an increase in the number of participant organizations promoting the call for Futures Conference applications and recognizing selected participants and grantees through media outreach, annual reports, and other promotional activities.

Leveraging this lesson in new contexts:

• How do we ensure that future science leaders are exposed to and have experience with interdisciplinary, collaborative research?
• How do we support the development of the skills needed by individual scientists and non-scientists to apply the NAKFI model?
• How can institutions and funders cultivate and support science leadership development?
• How do we encourage or incentivize individual scientists to embrace and participate in interdisciplinary, collaborative research?
• How do we help individual scientists overcome the systemic, organizational, and cultural challenges and barriers to interdisciplinary, collaborative research?
• How do we help scientists interested in and committed to interdisciplinary, collaborative research deploy the NAKFI model?
• How can scientists support the use and evolution of the NAKFI model?

LESSON 2: CONTENT + CONTAINER = SERENDIPITY BY DESIGN

NAKFI’s success was supported by efforts to incorporate the “right stuff” into the model. The program was developed with well-established ideas for achieving interdisciplinary collaboration and strengthened through regular and periodic evaluation and process improvement. Once participants experienced NAKFI they appreciated what was unique about the model and lamented the absence of those attributes in the professional opportunities typically available to them. The 10-year review summarized that feedback as a list of desired attributes present in the model:

• There should be more opportunities for cross-fertilization of ideas through short-term visits and mini-sabbaticals, as well as focused in-person experiences to help support interdisciplinary collaborations.
• Collaborative research facilities should include spaces for socializing and brainstorming.
• There should be more opportunities and incentives for researchers to engage with people from diverse backgrounds in terms of ethnic, cultural, gender, and nationality diversity.

• Brainstorming sessions should be done as multi-staged events over time in order to develop increasingly rich interactions and ideas.
• People are creative and innovative when addressing challenges in a face-to-face format.

The model was developed to reflect best practices and to overcome barriers to interdisciplinary and collaborative science. In addition to the early research on facilitating interdisciplinary science and the 5- and 10-year reviews, the program reflected theory and practice supported elsewhere in the literature, including the importance of face-to-face interaction to enhance information exchange and build trust (Duhigg, 2016) and the appropriate use of digital communication to support preparation and follow-up to the onsite experience (NRC, 2015).

Beyond the mechanics of information exchange there is the challenge of creating shared understanding. Communication between specialties and disciplines requires learning the language outside one’s area of expertise. As a participant in the 2008 Complex Systems conference stated, “It is important to learn the language of the other disciplines you want to engage. That requires reading and training so that you can converse with experts in topics that add to your breadth of research.” Learning everything there is to know about another discipline is neither needed nor expected, but having an understanding of another discipline’s basic tenets is important for effective communication and collaboration. NAKFI addressed the challenge of creating shared understanding by providing pre-conference tutorials, activities, and resources that offered information and exposure to the multiple perspectives that would be present at the conference.

NAKFI experimented with a variety of approaches to pre-conference tutorials, including specially made academic style webcasts, conversational podcast interviews, and curating content from other sources. Evaluation survey results revealed that no one approach worked for everyone. It was important to present content for novices and experts and to use a variety of formats. Written materials often supplemented online content. Articles by science journalists had wide appeal and journal articles provided in-depth content for those already familiar with a specific domain. Downloadable podcasts were helpful to attendees who wanted to learn in the gym or during their morning commute. Attendees were encouraged to create their own pre-conference learning from NAKFI’s suggestions depending on their needs, curiosity, and seed group topic.

Creating the NAKFI experience required careful planning at every step: selection of the conference steering committee; research and consultation to select timely and relevant themes; casting a wide net for invitees; and ensuring that the criteria for selection optimally supported the goals and desired outcomes of each gathering. Criteria for participation considered not only expertise and professional accomplishments but also experience with collaborative endeavors and an understanding of group dynamics.

Engaging individuals primed for the NAKFI experience was important but the model did not rely solely on what participants brought with them. The entire event was organized to maximize the experience, from the moment participants applied to the day they left the conference. The schedule created opportunities to socialize, engage in individual and group interactions, expose participants to new and different experiences and ideas, build camaraderie, and just have fun (see Appendix VII). Networking took place during breaks, meals, social events, poster sessions, and bus rides to and from the Beckman Center.

NAKFI provided time and space and also created a unique energy, stimulating commitment and passion from participants. Creating the optimal cohort for any given conference often required difficult decisions—especially during competitive years—and consideration of the group as a whole superseded an individual’s rank or reputation.

Leveraging this lesson in new contexts:

• How can researchers and supporting institutions create experiences that transcend the typical conference format and result in serendipity?
• Who can help design for serendipity? How can the arts, humanities, and other non-STEMM (science, technology, engineering, mathematics, and medicine) disciplines be leveraged to create vibrant knowledge-sharing and problem-solving experiences resulting in serendipity and innovation?
• How can an inclination toward collaboration be fostered in research communities?
• What role can institutions and funders play in supporting collaborative and interdisciplinary readiness?

LESSON 3: SUPPORT FOR VENTURE SCIENCE PROMOTES INNOVATION

Data from Futures Conference participants and grantees and the 5- and 10-year reviews indicate that the Futures Model is effective, robust, and has value to crossing disciplinary boundaries and advancing innovation by supporting what the program has come to call “venture science.”

Futures Conferences provided the time and space necessary for interdisciplinary collaboration, venture science, and innovation and Futures Grants served as an incentive for attendees to further explore ideas generated during the conference. The grant application process was straightforward and reporting requirements were minimal. Principal investigators (PIs) were vetted by the conference steering committees for attendance at the conference, and the grant selection committee looked for projects with the greatest potential to succeed. NAKFI encouraged grantees to “learn as they go” and “to make changes to their research plans as appropriate.” Projects that experienced unexpected delays or needed more time could request a no-cost extension with a simple email explanation. Final reports covered a few key areas of interest to the program and encouraged investigators to reflect on what worked, what did not work, and why.

It is important to understand how the NAKFI model worked and why. Some of the enabling attributes for the success of the program can be gleaned from feedback from conference participants and grantees. First and foremost is that “venture science” supports idea incubation, recognizes

that not all results are immediately apparent, and accepts that there can be great success in failing.

Second, NAKFI provided an opportunity not available elsewhere. Futures Conferences gathered highly accomplished people from multiple fields and organized them to pursue questions together. Most scientific conferences are more narrowly focused by topic or training. Very few such meetings provide the funding necessary to turn a promising encounter into a fruitful collaboration. Additionally, many traditional funding sources and institutions remain risk averse when dealing with interdisciplinary research that is also innovative. Venture science is unpredictable and requires a willingness to fail fast and learn.

NAKFI’s 10-Year Review Reveals Approaches to Support Innovation That Are Highly Important and Not Commonly Practiced

The 10-year review highlighted approaches reported as highly important and not commonly practiced, suggesting that the need for venture science continues to exist, especially among interdisciplinary researchers.

• More scientists should be encouraged to explore problems outside of their disciplinary framework.
• Cross-discipline discussion and working group sessions should be facilitated to address a particular research topic or societal need.
• A portion of all research should be designated for independent and unsupervised research by each member of the research team.
• Failure and negative results should be recognized as learning experiences.
• Creative, big-picture thinking individuals should make decisions about funding.
• Institutions should provide unrestricted funds for investigators to move projects in new, unrealized directions.
• Researchers should be encouraged to study important problems with yet unknown solutions.
• Curiosity-driven research should be incentivized as much as iterative or hypothesis-based research.
• Researchers should learn that there are multiple ways to understand a problem or a phenomenon.
• New mechanisms, models, and incentives are necessary to support partnerships between academia and industry.
• Graduate education, curriculum, and mentoring should encourage students to develop their own creative ideas.
• There should be recognition that innovation occurs in all kinds of science, from basic to applied.
• Early-career scientists should be encouraged to invest time in learning a new area even if it does not immediately result in publications.
• Researchers should be rewarded for being team players rather than sole innovators.

• Scholars should be encouraged to work at the periphery in addition to the core of their primary fields.

As noted during the 10-year review, the importance of risk tolerance is an essential ingredient to the Futures Model. Participants described their experience with the program as the opportunity to “ask questions I could never have dreamed of” and experience “noble failures” where an idea or an approach does not work as planned but leads to important insights nonetheless.

Leveraging this lesson in new contexts:

• How can institutions and funders support venture science?
• How can traditional funding models evolve to embrace risk tolerance and noble failure?
• How can researchers be encouraged to pursue innovation and idea incubation activities?
• How can funding models encourage and support cross-disciplinary and new approaches to research?

LESSON 4: A PLURALISM OF PERSPECTIVES IS MORE IMPORTANT NOW THAN EVER

A pivotal contributor to NAKFI’s success was its ability to encourage inclusive conversations and multiple perspectives by bringing together individuals with keen minds, interdisciplinary inclination, and a high degree of collaborative readiness. Though they often shared an interest in a topic, how they were trained to investigate that topic differed, sometimes greatly. Such an approach was part of the program from the beginning. Aligning with the W. M. Keck Foundation’s mission and embracing the findings of the Facilitating Interdisciplinary Research report, the program reflected the current thinking about interdisciplinary research and the important role of pluralism and inclusion to tackling complex problems and achieving significant innovation.

Fortunately, the National Academies and the W. M. Keck Foundation were committed to a program with an evolving model. In conjunction with astute program design and an approach to evaluation that supported continuous improvement, NAKFI was able to cross boundaries quickly. From inception through its final years the model was adjusted and honed

to support this achievement. The panel that assessed the program’s first 5 years concluded that the program’s success, “has been due in large part to the evolving structure of the conferences, careful organization (e.g., drawing on the experience of previous conferences), pre-conference preparation (e.g., tutorials, reading lists), provocative task group topics, the quality of participants, and the exceptionally strong support and credibility provided by the National Academies” (NAKFI, 2008, p. 1). Incorporating insights from team and group effectiveness research was facilitated by NAKFI’s environmental scanning efforts and involvement in the field of the science of team science. Strategies to improve effectiveness in these areas were outlined in the 2015 National Research Council Consensus Study Report Enhancing the Effectiveness of Team Science.

NAKFI made the most of participants’ inclination toward interdisciplinarity with a vetting process that supported the best fit of participants to the conference topic. As the program evolved, efforts were made to cast as wide a net as possible to identify participants from diverse disciplines and settings who would bring a plurality of perspectives. This diversity was coupled with a self-assessment tool to measure a participant’s inclination toward interdisciplinary work. That score was not used in the acceptance decision. Rather, it allowed the program to check the assumption that NAKFI applicants were already highly engaged in interdisciplinary thinking and activities.

When identifying potential areas for program improvement, the 5-year review noted that NAKFI conferences could benefit from a broader diversity in conference planning and participation. Additionally, the panel recommended that greater attention be paid to the identification, recruitment, and selection of participants, especially regarding the participation of women, under-represented minorities, and younger scientists. In response, greater attention was given to expanding the representation of disciplines beyond science, engineering, and medicine by including social and behavioral scientists (see Table 2-1). Later, artists and designers were also included. The program also began to more closely consider the demographics of applicants and attendees along the dimensions of gender, age, and career stage. Conference topics such as “The Informed Brain in a Digital World” and “Collective Behavior: From Cells to Societies” were more explicitly related to social and behavioral sciences and the last two conferences experimented with the intentional inclusion of artists, designers, and humanists. Methods to find individuals and encourage them to apply and then prepare

TABLE 2-1 The Evolution of Specialties on the Conference Application

NOTES: The number of specialties listed on the conference application expanded over time to accommodate more fields. Applicants could make multiple selections responses and most did. Beginning in 2008, applicants also had the option to select “Other” and provide an open-ended response.

for the NAKFI experience evolved over time, eventually leading to the right mix of people, place, and program.

There is some quantifiable evidence that NAKFI conferences had the potential to change how participants drew on research from other fields in their conference participation. In 2008, an analysis of publications by participants in the 2004 Nanostructures conference revealed that a group of biologists were citing more engineering papers after the meeting than they had previously. The engineers were also citing more biology papers in their writing. Though the analysis was directional and it is not possible to attribute the change directly to the conference, quotes obtained from participants following the conference suggested that it may have caused the change in reference patterns.

When asked to provide the most significant interaction from the conference, an anonymous engineer reported it was learning that “people in the biological research community … are also interested in pursuing interdisciplinary research with people with physical sciences and engineering backgrounds.” This pattern of mutual recognition and exchange of ideas across disciplines is robust across NAKFI conferences. Hundreds of examples exist in survey data and unsolicited testimonial emails sent to program staff.

NAKFI was designed to engage in evaluation that would drive its own evolution. NAKFI learned that tackling the “most important” questions required identifying individuals with deep disciplinary knowledge and the ability to cross boundaries. The model also had to support collaborative interaction if the power of pluralism and interdisciplinarity was to be realized. Collaboration is an arduous activity. It involves more than exchanging information or sharing resources. Successful collaborations, like other types of relationships, are often deep, energetic, and lasting. Since its launch, the Futures Model has fostered numerous interdisciplinary collaborations that would go beyond superficial conference exchanges and support the co-discovery and the creation of imaginative solutions.

Leveraging this key learning in new contexts:

• How do we achieve greater diversity of background and perspectives when we convene people to address a problem, question, or challenge?

• How do we help participants cross disciplines? How can we help bridge understanding of other disciplines’ concepts and language?
• How do we create experiences that allow sharing across disciplines to support innovation?

This page intentionally left blank.