penetration emphasizes widespread service through training or short-term engagements, which can be associated with less revenue generation and lower-levels of impact on clients. Efforts to raise revenue have the potential to be associated with fewer clients served. Efforts to raise impact can also mean larger projects with a smaller set of clients. Maximizing service impact can even effect revenue generation as some types of projects with the potential to yield high impact are stressed over other types of projects that have short-term cost-cutting appeal to potential customers.
Given these potentially diverging objectives, the MEP uses multiple evaluation methods to address them. The core performance measurement activity at MEP is the client survey. This survey, which has been administered since 1996, is conducted approximately one year after an engagement with a client company. The survey draws on a standard nationwide reporting system. Centers populate this system on a quarterly basis with reporting information about center, client, and project and event attributes. The survey is administered by a third party company once a quarter primarily using telephone and web-based methods. The questionnaire has included various questions over the years but the core questions ask clients to report changes in quantitative outcomes such as sales, employment, cost savings, and capital investment. In addition, the current survey asks about factors in using the MEP and strategic challenges the company faces. Prior questionnaires included items about customer satisfaction, but the MEP has reduced these questions, because of lack of variability in overwhelmingly positive responses. The current questionnaire asks clients about the likelihood of recommending the MEP to other companies.
Survey results concerning fiscal year 2010 indicated that the MEP served 34,299 companies. Of the 7,786 participating in the survey (out of 9,654 qualified to do so given their receipt of more intensive services), these clients’ aggregate results attributed to MEP assistance were $8.4 billion in sales, $1.3 billion in cost savings, $1.9 billion investments, and 72,075 jobs created or retained. An economic impact model applied to the results indicated that for every $1 of federal investment, $32 of economic growth is returned (MEP 2011; see also MEP 1994, 1997, 1998). Client responses associated with high impact numbers are authenticated and, if found valid, are profiled as success stories using a mini-logic-based case study method.
The survey is designed to fulfill Government Performance and Assessment Act (GPRA) requirements for metrics on program impacts. Program metrics are reported relative to measurable goals which the program sets as part of GPRA. The survey also forms the basis for tracking center-level performance, since 2003, through a system of Minimally Acceptable Impact Measures (MAIM). 2 MAIM is comprised of five measures: (1) Bottom-line
2NIST MEP Reporting Guidelines, Center Operations, version 6.1. Effective Quarter 2, 2011.