National Academies Press: OpenBook

Best Practices in Assessment of Research and Development Organizations (2012)

Chapter: Appendix F The Army Research Laboratory's Process for Assuring Relevance

« Previous: Appendix E Relevant Statutes and Requirements Documents for U.S. Government Laboratories and U.S. Government Research
Suggested Citation:"Appendix F The Army Research Laboratory's Process for Assuring Relevance." National Research Council. 2012. Best Practices in Assessment of Research and Development Organizations. Washington, DC: The National Academies Press. doi: 10.17226/13529.
×

Appendix F

The Army Research Laboratory’s Process for Ensuring Relevance

For the Army Research Laboratory (ARL), the definition of relevance is how well a research and development (R&D) organization is performing its mission for the benefit of its stakeholders.1,2 Thus, for an organization, or within the organization individual programs or even projects, to be relevant requires that their stakeholders be identified and the requirements or needs or desires of the stakeholders be made known. In the most fundamental sense this requires that the organization take several steps:

•  Identify its stakeholders and customers: Does it know who it is supposed to be supporting?

•  Communicate with the stakeholders and customers: Ask them what they want or need. Negotiate the technological terms to the extent necessary.

•  Direct an appropriate amount of effort toward fulfilling those needs.

•  Continue to communicate with the stakeholders or customers as the work progresses: This communication is needed to ensure that strategies and work are going in the right direction and that requirements have not changed.

•  Follow up after delivery of the final product: This follow-up is needed to ensure that the stakeholders’ or customers’ needs have been fulfilled. If not, an examination is needed of ways to fix problems and develop improved processes for the future.

Based on the work of Edward B. Roberts at the Massachusetts Institute of Technology’s Sloan School of Management, the ARL identified the following stakeholders. The Army’s research, development, and engineering centers (RDECs), in particular, were considered immediate customers, whose missions were, in general terms, to develop materiel solutions to Army problems and to engineer these devices in preparation for production and fielding. The RDECs rely on the ARL to provide them with the basic and/or applied technology from which RDECs can carry out their development and engineering missions. Another significant group of ARL stakeholders consists of the various senior Army management organizations and funding activities. These include ARL’s parent organizations, such as the Research, Development, and Engineering Command (RDECOM) and the Army Materiel Command (AMC), plus the senior Army management—the Army staff. The ARL relevance issues for these organizations revolve around how well the ARL is functioning as a support to the larger Army structure. The other major stakeholder is the end-item user—the soldier in the field, represented by Army program managers and program executive officers as well as the Army Training and Doctrine Command (TRADOC).

____________________________

1 E. Brown, 1997. Measuring Performance at the Army Research Laboratory: The Performance Evaluation Construct. Army Research Laboratory, Adelphi, Md.

2 E. Brown, 1998. Reinventing Government Research and Development: A Status Report on Management Initiatives and Reinvention Efforts at the Army Research Laboratory. Report No. ARL-SR-57. Army Research Laboratory, Adelphi, Md.

Suggested Citation:"Appendix F The Army Research Laboratory's Process for Assuring Relevance." National Research Council. 2012. Best Practices in Assessment of Research and Development Organizations. Washington, DC: The National Academies Press. doi: 10.17226/13529.
×

In addition to these stakeholders identified above are such entities as Department of Defense (DOD) leadership, the Office of Science and Technology Policy (OSTP), the Office of Management and Budget (OMB), the Congress, and eventually the public.

When the ARL was established in 1993, the decision was made to develop processes by which it could show its relevance specifically to its stakeholders or customers and to the senior Army leadership. The chartering document required that several specific actions be taken with regard to its stakeholders and customers, the rationale being that concentrating most of the basic and applied materiel research in one organization with little to no development activity ran a very real risk of decoupling the newly formed corporate laboratory from the rest of the Army—that is, it ran the risk that ARL could lose its relevance. Thus, the ARL was required to direct at least 50 percent of its program resources toward stakeholder or customer requirements. This 50 percent requirement limited discretionary spending ability and thus provided a balance that did not stifle creativity but at the same time prevented an ivory tower effect from taking hold.

Meeting this requirement was accomplished by having the first ARL director meet with his counterparts in the RDECs; a memorandum of understanding (MOU) was executed in which each acknowledged this relationship. Then, for the directed tasks to be undertaken, ARL line managers were required to execute a Technical Program Annex (TPA) to the original MOU; the TPA was a one-page contract signed by the ARL line manager and one of his counterparts from a customer organization specifying the details of the work to be done. These details included a description of the specific task, the application of the end product by the customer, the amount of funds that the ARL would expend on this task, points of contact in both organizations, and any other information deemed necessary. The TPAs had to sum to at least half of ARL’s base funding. This novel approach was deemed to be very unusual since, in effect, the ARL was asking its customers’ permission to spend its own money. At the end of the year, the customer signatory on each TPA was surveyed to determine the level of satisfaction attained. If there were any complaints, or if the customer’s needs were not satisfied, a senior ARL manager was required to contact the customer about how to correct the situation.

Another piece of this customer-relevance construct was a board of directors (BOD) created by that original charter. The BOD members were the directors of the RDECs that constituted ARL’s customer base. The BOD would meet at least once a year at the ARL to review how well the laboratory had been serving their people. They reviewed the results of the year’s TPA process as well as other processes. They then made comments, observations, suggestions, and recommendations as to the state of their relationship with the ARL and how, if necessary, improvements could be made. This entire process ensured that the laboratory would indeed be closely coupled and very relevant to its customer organizations.

In order to reach the senior leadership, the ARL conceived of a device called the Stakeholders’ Advisory Board (SAB). The ARL director approached the commanding general (four-star) of the AMC, ARL’s parent organization, who agreed to chair a group of senior Army staff. By the group’s charter, the purpose of the SAB was to ensure that the ARL was closely coupled to the Army’s vision of how it would fight in the future, ensure that the ARL would be responsive to and support the Army’s senior leadership as it evolved the doctrine and requirements for the Army of the 21st century, and ensure that the ARL would be sustained as a valuable resource for providing the technological edge to shape the future Army.

The SAB’s membership included the senior members of the Army staff—G1 (personnel administration), G2 (intelligence and security), G3 (operations), G4 (logistics), the deputy under secretary, and other members, all at the three-star or equivalent level. Reaching the soldier in the

Suggested Citation:"Appendix F The Army Research Laboratory's Process for Assuring Relevance." National Research Council. 2012. Best Practices in Assessment of Research and Development Organizations. Washington, DC: The National Academies Press. doi: 10.17226/13529.
×

field personally was not possible. However, one member of the SAB was the deputy commander (three-star) of the TRADOC, which describes itself as the user’s representative.

The SAB would meet at the ARL once a year for a half day, during which it would be briefed on ARL’s activities of the past year and its projections and challenges for the upcoming year. The SAB would then be given a few short technology briefings on recent breakthroughs, a summary briefing from both the BOD and the Technical Assessment Board (TAB) of the National Academies’ National Research Council in order to get a complete picture of all facets of the laboratory’s activities, a tour of the facilities with demonstrations of technologies, and a working lunch at which time the chairman (the AMC commander) would solicit from each member opinions, critiques, suggestions, requests, and recommendations. These activities were captured as notes and action items, with the action items being scrupulously responded to over the next few months. Thus, the ARL was able to reach out and communicate its performance and achievements to assure this very senior group of stakeholders of its relevance.

This activity proved to be an enormously powerful management tool for ensuring the laboratory’s relevance and performance as an Army asset. Demonstrating this close relationship with its customers and senior stakeholders served to drive ARL’s performance. However, the SAB concept was abandoned after 2000. With the formation of RDECOM as a new command organization injected between the ARL and the AMC, it was no longer possible from a protocol standpoint to bypass the two-star RDECOM commander to the four-star AMC commander in order to continue the latter’s chairing of the SAB, nor could the RDECOM commander summon all the three-star SAB members to assemble.

Suggested Citation:"Appendix F The Army Research Laboratory's Process for Assuring Relevance." National Research Council. 2012. Best Practices in Assessment of Research and Development Organizations. Washington, DC: The National Academies Press. doi: 10.17226/13529.
×
Page 58
Suggested Citation:"Appendix F The Army Research Laboratory's Process for Assuring Relevance." National Research Council. 2012. Best Practices in Assessment of Research and Development Organizations. Washington, DC: The National Academies Press. doi: 10.17226/13529.
×
Page 59
Suggested Citation:"Appendix F The Army Research Laboratory's Process for Assuring Relevance." National Research Council. 2012. Best Practices in Assessment of Research and Development Organizations. Washington, DC: The National Academies Press. doi: 10.17226/13529.
×
Page 60
Next: Appendix G Examples of Stakeholders »
Best Practices in Assessment of Research and Development Organizations Get This Book
×
Buy Paperback | $37.00 Buy Ebook | $29.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Research and development (R&D) organizations are operated by government, business, academe, and independent institutes. The success of their parent organizations is closely tied to the success of these R&D organizations. In this report, organizations refers to an organization that performs research and/or development activities (often a laboratory), and parent refers to the superordinate organization of which the R&D organization is a part. When the organization under discussion is formally labeled a laboratory, it is referred to as such. The question arises: How does one know whether an organization and its programs are achieving excellence in the best interests of its parent? Does the organization have an appropriate research staff, facilities, and equipment? Is it doing the right things at high levels of quality, relevance, and timeliness? Does it lead to successful new concepts, products, or processes that support the interests of its parent?

This report offers assessment guidelines for senior management of organizations and of their parents. The report lists the major principles of assessment, noting that details will vary from one organization to another. It provides sufficient information to inform the design of assessments, but it does not prescribe precisely how to perform them, because different techniques are needed for different types of organizations.

Best Practices in Assessment of Research and Development Organizations covers three key factors that underpin the success of an R&D organization: (1) the mission of the organization and its alignment with that of the parents; (2) the relevance and impact of the organization's work; and (3) the resources provided to the organization, beginning with a high-quality staff and management.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!