National Academies Press: OpenBook

The Lessons and The Legacy of the Pew Health Policy Program (1997)

Chapter: Appendix A: Telephone Interviews

« Previous: References
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Bill Weissert Thursday, August 17, 1995, 10 a.m.

1a. Based on your experience and familiarity with the fellows and the programs, what did we really accomplish? What were the most important contributions?

We trained a lot of people in research findings from health services research, and we trained the last three or four cohorts pretty well in methods. We helped quite a few careers, and as with any program, we gave some degrees to people who probably should not have them. A program like this is particularly vulnerable because you hope that people will either stay or go into the health policy field instead of becoming academics. This is a real risk when you give a doctoral degree without requiring residence. You wind up giving a degree to people who are not really academics, but who are now qualified to teach. That is a downside of our program and something about which we are constantly vigilant.

The most obvious [contribution] is career development. You can clearly see that and more or less count it. Many alumni were promoted or got better jobs in the policy field as a result.

There is much about the policy process that is still happenstance and convenience—who is around. You can only improve the chances that rationality will play a part in policy by having around a lot of people who are trained rationally. So, the more people you put out there, the better chance you have to formally or informally influence the policy process. We train a lot of people who are interlopers or daily workers in the policy process. Therefore, we increase the chances that what comes out the other end will be better informed than it would have been if people had been shooting from the hip. Yet, it is hard to quantify that.

1b. What is the Pew ''legacy'' in terms of:

  1. health policy?

    We made a lot more people aware of the power of the literature to answer a lot of the questions in a systematic way rather than guessing, as they were doing before. We helped with the diffusion of the health services research findings. I am always struck when I go to meetings and talk to folks who are making major policy decisions or influencing policy decisions without reading the literature. These foundation types and policy types have almost never read the literature. It's shocking. They are trying to move forward the state of

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

the art without having found out what the state of the art is. They are verbal people, and they learn by hearing; they don't read. So, the bottom line: the more people who know about the literature and what it says and can relate to it, the more likely you are to influence the policy process. It may be an inefficient way, but I have not found anything that works better. Certainly, sending policymakers the articles does not work.

  1. education (doctoral, postdoctoral, or midcareer programs)?

  2. your institution?

    For our institution it has led to a new degree program which we will keep. It broadened our presence in training health services researchers at the doctoral level. We had been primarily a master's program with a very small doctoral program; this gave us a whole new program, more bodies, and a slightly different cut of people. People came who were more in the policy process. Pew definitely had an influence on this institution's functioning and its contribution to health services research and health policy. It also made both the program and me more flexible in terms of how policy is defined. Policy is being made even within institutions. For example, I initially opposed a dissertation topic by somebody who was interested in how hospitals allocated the funds they were getting that go to education, how the medical school and hospital fight that out. I had said that wasn't really policy. Well, it is a policy question and now it's the leading policy question.

It has made us more flexible as a program in what we do to support women. When you take women who are in the midcareer process and they come into a long-term program like this, they are going to have babies. You have to be able to cope with that. You have to be prepared occasionally for a baby in the classroom. These are all good things. They make us a better university. I am pleased with that aspect of our program.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

We had an innovation here that dates back 25 years, which is this midcareer weekend program where people come in and take 2 years of the same courses. That was a proven idea that worked and cranked a huge number of people out into the health management community. So, with this program we extended that to the health policy commu-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

nity, and I think it's a good idea. And, it gets better every year. That's how it developed. It was basically an incremental change over a proven program that was and is unique in the country. It was innovative at the time and continues to be innovative.

There is no question that this program will continue. We have expanded it and bought off on it and found that we can sustain it. The big problem is that we will not have funding, and therefore, we will have a limited ability to reach people who need to be here with fellowships. We just don't have the money. To the extent that you're trying to reach people in the policy process, funding is pretty important because it's a field where people are not particularly well paid. Thus, without the proper funding we reduce our likelihood of influencing the policy process. But we will continue to get people who are either able to afford the program because they are MDs or because they are supported by their organization or interest group.

3. What was the need in the health policy community when your program started, and how have those needs changed today? Is the job done?

The need is the same. You have to train a lot of people because the path to influencing policy is very murky and difficult to predict so its better to have a lot of trained people around. Certainly, the job is not done. I continue to be shocked at the poor quality of policy in some areas, although I think it has gotten a lot better in the 20 years since I've been doing this.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

The cohort effect—locking people into being part of a group for 2 years of intensive course work. I think that has a terrifically positive effect, is very successful, and socializes a lot of people. They learn from each other like crazy. The faculty learns as well.

Those things that do not contribute: It's difficult to take people who are midcareer and send them off after 2 years and have them finish their dissertation, although we have worked on that and we are about where we want to be on that. That's been a difficult problem to overcome. Getting back into the work site and not being forced to show up with papers every month. It's difficult not to let job demands take precedence.

There is no solution to this dilemma, however the

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

degree requirements that are imposed upon us by the accrediting organization of the schools of public health do not contribute to the success of the program. To satisfy the public health requirements, we had to find people to teach from other departments. That can lead to courses of variable quality. We amend this by flying people in from Boston and other universities. Every year it's a struggle.

5a. What was the most innovative or unique aspect of:

  1. your program design?

    Lockstep. Everybody has to take the same courses; they come in on weekends, they're midcareer, but the demands on them are no less than those on students in on-campus (OC) programs and may be even greater.

  2. methods of implementation

    One method that works (Leon Wyszewianski's idea, but I implement it) is to give the students the usual statistics courses but have the methods course taught by a practitioner rather than a methodologist. That seems to be one of the most important courses we offer. The idea is that they will know what to do with the knowledge they learn. It's the best idea we have in the program. It takes these people from being theory-trained to appliers.

  3. educational process?

Answered previously.

6. Based on your experience, what lessons were learned about the educational process in terms of:

  1. recruitment?

    We tested the likelihood that we were producing a lot of false negatives. In the past we had been turning down people who might have been able to make it, and this year we let them in. What we found was that with a 10 percent drop in Graduate Record Examinations (GRE) scores we could increase our cohort size by 100 percent. I do not see a substantial difference in performance. We have proportionately more people who shouldn't have been in, but not an increase in the percentage. So, we have the same number of people we wish we had not admitted, but we always have some we wish we had not admitted. Some of those drop out and some don't. It confirms my long-held hypothesis that

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

we only worry about false positives and in the process throw out two or three who could have made it, and this year we included those people and most of them were pretty good choices.

  1. degree requirements?

    We dropped a bunch of prerequisites this year that we thought were pseudo prereqs. We required people to worry and study to pass an exam that everybody passed or everybody flunked. I just threw out everything that wasn't either statistics or economics prerequisites, and it worked. We got rid of some people who just couldn't cut it in terms of the quantitative stuff. And as for the others, we didn't put them through a lot of Mickey Mouse stuff.

  2. curriculum and content?

  3. integration of fellows with other students, the rest of the university, and the program in general?

    It's approaching zero. We have them here all day in class, and they work and party as a cohort. There is almost no interaction with other university students. I have tried to integrate the OC and off-campus programs, but the OC programs are threatened. Someday we'll get more integrated. The greatest positive of integration would be to give a dose of reality to the OC people and perhaps a little more of the role of cognates in research to the off-campus people.

  4. relationship between faculty and students?

    I don't think there are programs in which the relationship is better than in this program. The faculty and students are collegial. The students are experienced and not shy. There is a very good relationship.

  5. completion rates (where applicable)?

    This is something we have worked on constantly. Our penultimate cohort is up to 66 percent with a couple more likely to come through. I don't think doctoral programs ever want to be 100 percent. So I think that's just terrific.

7a. To what extent do you think there were "programmatic" barriers to student completion?

We didn't really have anybody whose job it was to make sure the students were equipped to do a dissertation. They

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

had advisers, but they weren't even seeing their advisers once a month. So, we made an institutional change and added a course where someone actually taught the students how to write, one step at a time.

7b. To what extent do you think the program was beneficial for those who did not finish the program?

Everyone who drops out basically praises what they've learned so far and report that they are immediately applying the stuff. We know from classroom assignments that it is very typical for the students to use what we assign that month or the next in their job. An example would be where we sent them off to do a policy analysis and provide policy options for the Secretary, and the Secretary actually chose one of their policy options. That is not untypical.

7c. How can we measure success for those programs where completion rates do not apply (i.e., postdoctoral programs)?

Look closely at the people who applied and were turned down. Compare their outcomes and influences with those who got in. Where are they with their careers and what kind of impact have they had on the policy process? Do a comparison group study.

8. How does the Pew fellowship approach differ from a traditional fellowship approach? How have the major outcomes differed?

The major differences are within the cohort. We examine the product that these people are getting and make changes in faculty and courses in order to change their total product. So, it is not a course-by-course assessment, which is true for traditional OC programs. We look at the product and say, "Are these people getting enough of this particular thing? If not, where in all of their courses can we add it?" Since there is no discretion, we can focus on things they really need.

Major outcomes do not differ very much. Pew people tend to be somewhat more policy focused and more involved.

9. If you were asked to give advice to another university attempting to initiate a similar program, what would you say?

You need to make sure that the best faculty are willing to teach Saturdays and Sundays. Faculty do not realize that they will be teaching at least 5 of the 7 or 7 of the 9 months on weekends, both Saturday and Sunday. This is a big commitment that needs to be established up front.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Carroll Estes Wednesday, August 23, 1995, 4 p.m.

1a. Based on your experience and familiarity with the fellows and the programs, what did we really accomplish? What were the most important contributions?

I think the Pew program has very successfully seeded the field with competent, well-trained scholars at government levels, nonprofit and foundation levels, and university levels. The flowering and capability of those fellows and their contributions are beginning to be recognized at fairly significant levels. For example, one of our fellows was chair of one of the White House task forces on benefits in the health reform area (Linda Bergthold). The most important contribution of the fellows is a passionate commitment to health policy and health services research that is objective and has an impact and the ability to carry out that work either directly themselves or to stimulate organizations and institutions to do it where they are.

1b. What is the Pew "legacy" in terms of:

  1. health policy?

    There is very specific expertise that is available in the field as a result of the Pew program. I don't think it's enough. I think the program falls short by cutting itself to an end prematurely. I think the Pew Health Policy Fellowship Program needed to be extended a minimum of another 10 years. I look more toward the Robert Wood Johnson Clinical Scholars model. The magnitude of the work and the magnitude of change in the field is such that health policy contributions have just begun to be made, and these scholars will be around as they are becoming policy makers. Nevertheless, there is very important substantive work that still has to be done in health policy, and the training of more leaders continues to be needed.

  2. education (doctoral, postdoctoral, or midcareer programs)?

    There certainly are seminars and courses that exist on our campus that would not have existed on this campus without support from Pew. There are other postdoctoral programs that are multiplier effects from the Agency for Health Care Policy and Research (AHCPR), but now we don't know the extent to which AHCPR is going to be a viable institution to continue what would be complementary

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

programs of training. There is no question that interdisciplinary, scholarly work in health policy has been forwarded as a result of the Pew commitment and faculty commitment to education on a multidisciplinary level has probably grown. In terms of UCSF (University of California, San Francisco) there is no question that the Institute for Health and Aging, which is a major research institute that did not exist prior to the Pew program, probably exists in large part because of the support from Pew for both research and the training program. It does research in education and is a multimillion-dollar-a-year institute that is contributing quite a bit in terms of disability statistics and health and long-term care policy.

  1. your institution?

    Answered above.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

It developed because we had two people who had significant research backgrounds and experience on which a successful training program could be built. Furthermore we had faculty who had many specific methodological substantive skills to offer. Our particular program began with pre-doctoral, management, postdoctoral, and midcareer components. I think we found that the midcareer program was a very good route for certain people who were quite accomplished and had a lot to bring to the classroom, but who needed to acquire the health policy and interdisciplinary skills. These people already had positions of influence into which they could carry this. We obviously had a charismatic leader in Phil Lee. He had great vision of where health policy needed to go, and he served as a catalyst to bring together existing resources and capabilities and make this happen.

To what extent will the program continue now that funding has ceased? I think that there are serious institutional problems at the University of California which have to do with funding problems. There has been a 25 percent cut in educational resources in the last 4 years alone, with no appreciable resource increase. The academic health centers questions with regard to health care restructuring and their competitiveness in the new managed care world have further raised resources questions for major medical centers and health science centers. This just means that the resource base from general-source supports that one had hoped to institutionalize are more difficult than ever to access due to factors

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

that have nothing to do with the success of the program and that are totally exogenous. There are and will continue to be research seminars. There will continue to be the search for postdoctoral training programs—individual and institutional—to keep the concept alive. However, the lack of institutional support or the lack of faculty support and uncertain or declining federal training grant support will impede the magnitude or the size of such programs. It's not news. It has been said at every meeting we've had that Pew should not be cutting off at this point.

3. What was the need in the health policy community when your program started, and how have those needs changed today? Is the job done?

We were very concerned about access, cost, and quality when we started, and we are still concerned with access, cost, and quality. No, the job isn't done. Access is worse than ever. Quality questions are bigger than ever. There are programs that attempt to cap cost but really don't do cost containment on the system level. The work that needs to be done is bigger and more important than ever.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

UCSF has a health policy specialty that it did not have prior to the Pew doctoral program in the Department of Social and Behavioral Sciences. A major success of our program has been the curriculum of the writing seminar and every week a seminar on health policy issues with faculty and fellows presenting on a convening ground. The writing seminar is stunning, and our fellows probably have a much stronger publishing record than they would have had without it. We have been very fortunate to have on our faculty the West Coast Editor of JAMA, Dr. Drummond Rennie, who has really done an incredible job with stimulating and encouraging publications.

5a. What was the most innovative or unique aspect of:

  1. your program design?

    One innovative aspect is the hands-on research with ongoing faculty investigators who have major programs of research and the competency to direct fellows in wide areas of interest, which allowed for special approaches that fellows might have.

    I think the writing seminar was very crucial and very exciting in teaching how to critique and to take critique.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
  1. methods of implementation?

    Implementation was through the colleagueship and mentoring and the coauthorships and separate authorships through these large-scale research projects. The fact the two institutes at UCSF together have about $20 million in funded research every year and probably 40 faculty investigators who are well known in their area has really contributed to implementing very significant training opportunities.

  2. educational process?

    The Pew conferences every year have been important. Networking and socialization to the field, the norms of work, and networking of colleagues have been important parts of the educational process. I have already mentioned the unique element of the writing seminar, which was very challenging or is very challenging to fellows, while also being very productive. Our weekly seminars are valued and the rooms are packed with a lot of other people wanting to join these seminars.

5b. What were the biggest challenges or barriers to overcome of:

  1. your program design?

    Getting doctors to mix with social scientists, to be open to learning, to be able to do what is basically social science-type research. For the social scientists it's similarly the obverse of learning more about how health care institutions actually work from a health professional perspective.

  2. methods of implementation?

  3. educational process?

    The integration of the disciplines really has worked, the method of implementation was constant, and there are continuing opportunities for interchange and various projects and seminars.

6. Based on your experience, what lessons were learned about the educational process in terms of:

  1. recruitment

    We probably send out 200 institutional letters a year. As everything, the word of mouth in the field and the referral from other fellows is a very important source of recruitment. Obviously, the excellence of our faculty and their reputations

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

were the single most important attraction in recruitment. So, I think you need a sterling faculty and a sterling institution for recruitment.

  1. degree requirements?

    We require a doctoral degree or a health professional degree. We only had a predoctoral program for a couple of years. We missed the predoctoral part and were sad that it was cut out. It was not our choice. I think the degree requirements for a postdoc were appropriate, and basically, taking people who already had some health policy experience was far better than having people who had none or who had some health services contact in education. In a short 2-year program people really needed to have another degree or some courses in the area prior to entrance. Otherwise, it is a much bigger learning curve that is harder to deal with. It also holds back others in terms of the complexity of the material.

  2. curriculum and content?

    We made courses required at certain times in the program. We found it was very important to be explicit and to have some details and firm requirements. All students were required to participate in the writing seminar and Pew seminar. But we learned that it was also important to have flexibility for other aspects of the curriculum, allowing people to search out the curriculum opportunities. Nonetheless, we wanted to create a strong methodological component and we wanted to make sure courses in particular areas such as art and science, proposal writing, and other basic course skills were available, and they were essentially required. The proposal writing piece was very important. A course in economics was essentially required, unless people had comparable experience.

  3. integration of fellows with other students, the rest of the university, and the program in general?

    We definitely integrated our fellows with fellows funded from the National Institute on Aging, AHCPR, and other private foundation training programs. This strategy benefited everyone. We opened up our courses to selected other faculty probably more than to other students, although an occasional exceptional student would be permitted (e.g., nurses working on doctorates). But we really tried to keep our courses small and with a select group of fellows, because

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

it needed to have the collectiveness and a community spirit and connection. We also were intent on training and making sure that content was tailored to the interests and skills of our fellows.

  1. relationship between faculty and students?

    The relationship was and is extremely close. The relationships developed through collaborations on research projects and the weekly seminars and occasional social events. I do think the conferences have been important in networking. Faculty and students worked together on those conferences and jointly attended them.

  2. completion rates (where applicable)?

    All our predoctoral students did graduate, and we are very proud of them. In term of our postdoctoral students, this does not really apply.

7a. To what extent do you think there were ''programmatic'' barriers to student completion?

This relates more to predoctoral programs.

7b. To what extent do you think the program was beneficial for those who did not finish the program?

All our fellows finished our program.

8. How does the Pew fellowship approach differ from a traditional fellowship approach? How have the major outcomes differed?

This is much more of an interdisciplinary experience and much less of a lone wolf researcher experience: do it independently on your own and sink or swim on your own. There is much more of a faculty-planned approach and a commitment to annual reviews, mentorship meetings, preceptor meetings, and scheduled reviews of students. There are more groups meeting with faculty members on programs of activity in what they have accomplished and what they want to accomplish and getting mentorship and advice on papers, research proposals, and jobs. I don't think in most other fellowships there exists that kind of tutorage. There is really a community commitment here, a lot of which is possible because of the general support that Pew provided, which allowed for the support of faculty who otherwise would not have support for any teaching and/or mentorship concern. This will be a tremendous loss for the program

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

because there are no replacement funds for that institutionally or in terms of other federal grants. It would have to be a foundation-type program.

The outcomes have differed dramatically from those from the traditional fellowship approach, for example, the productivity of our fellows in terms of research and publications. We would stand them up against any federal postdoc program, and I think they would be probably have twice the rate in terms of explicit measures of productivity. It is that nurturing, collectiveness, orientation, network, and support that the fellows offered each other.

9. If you were asked to give advice to another university attempting to initiate a similar program, what would you say?

There has to be some way of getting the leadership of the university to support faculty time to develop and maintain excellence in teaching and mentoring. I think you need excellent faculty and a strong research program because the training has got to be hands-on with experience and supervision. I would also say how important a writing seminar is, and that there needs to be an explicit allocation of resources to ensure that there is training in writing, proposal development, and publication. It's just essential.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Dennis Beatrice Wednesday, August 16, 1995, 10 a.m.

1a. Based on your experience and familiarity with the fellows and the programs, what did we really accomplish? What were the most important contributions?

We mastered the art of cross-training. We trained our fellows in analysis and policy. This approach is more important now than ever because the fine lines between health policy analysts and health policy makers are becoming blurred.

1b. What is the Pew "legacy" in terms of:

  1. health policy?

    The Pew program enabled people who wanted to become program analysts to obtain both the analytic and policy training needed.

    Pew put policy types in an academic setting. Pew took people from state and federal positions. A new and different cadre of individuals was put through this program.

    The Pew program retooled and juiced up several cohorts of people who would ultimately end up in public policy positions.

  2. education (doctoral, postdoctoral, or midcarrer programs)?

    Pew educated a new kind of PhD, one better prepared and more relevant for today's world.

  3. your institution?

    At Brandeis, Pew succeeded in bringing a different kind of student into the university. The Pew program got Brandeis thinking about ways to accommodate different students in different tracks (practitioner training versus research training).

  4. other?

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

With the help of the Pew program, Brandeis has initiated Health Institute Fellowships. The very best health applicants are recruited. The institute has committed itself to the training and support of these students. This is the way that Brandeis has kept and will continue to keep the Pew spirit alive.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

3. What was the need in the health policy community when your program started, and how have those needs changed today? Is the job done?

The job is not done. PhDs need to change because the environment needs more academic types in government. Government people can no longer shoot from the hip. The stakes are too high. We need to bring those two worlds together. We need to continue to educate, train, and produce a "different" kind of student.

4. What were the biggest challenges or barriers to overcome of:

a) your program design?

How do you create an accelerated program that people can finish without watering down the content? There is tension within this issue. The program works well for some. Some work better at an accelerated pace.

5a. To what extent do you think the program was beneficial for those who did not finish the program?

I did not finish; however, I got a better fix on how one uses analysis and graduate education in nonacademic settings. There is great utility in the Pew training.

5b. How can we measure success for those programs where completion rates do not apply (i.e., postdoctoral programs)?

What happened to the fellows? That is by far the greatest measure of success. Is there a plausible reason to believe that those who did not finish still benefited from their Pew training and achieved a heightened awareness? This may be a soft analysis. But, there is no proxy measure.

6. If you were asked to give advice to another university attempting to initiate a similar program, what would you say?

I would be as explicit as possible. I would emphasize the practical/analytical aspects of the program. Institutions need to be prepared to recruit people who are able to adhere to a very strict, high level of work. Pay close attention to the benefits of "cross-training."

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Hal Luft Thursday, August 17, 1995, 11:30 a.m.

1a. Based on your experience and familiarity with the fellows and the programs, what did we really accomplish? What were the most important contributions?

Among the main things is that we have developed an integrated program across three or four sites that involves a wide range of people from fresh graduate students to fairly senior career people. We have placed those people into various settings ranging from the university to major health policy kinds of settings. The fellows are doing a great job. We have also established three programs that are likely to survive past Pew.

1b. What is the Pew "legacy" in terms of:

  1. health policy?

    It is hard to identify specific health policies that have changed one way or the other as a result of the Pew program per se. I am not clever enough to picture an alternative universe in which there was not a Pew program and what would have been different. But I do have the sense that some of the fellows have been doing very exciting things that are having an impact on the way that the health care system is changing and the way that health policy is being formulated. That probably would not have happened had those people not gone through the Pew program.

  2. education (doctoral, postdoctoral, or midcareer programs)?

    I suspect at UCSF (University of California, San Francisco) that we would have not had a postdoctoral program had there not been the Pew funds to get it started. We would not have had the AHCPR (Agency for Health Care Policy and Research) training grant because that built on our experiences with Pew. And, I suspect that there would not have been as nearly as strong an application from Berkeley and UCSF for the Robert Wood Johnson Health Policy Scholars. I suspect the same thing would have been true for Michigan. All these things can be linked back to the Pew program. It is hard for me to say what would have happened at Brandeis or Michigan without the Pew program, but I can say that at UCSF, without that sort of core training program, our postdoctoral program probably would not have gotten started.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
  1. your institution?

    What has developed here is a clear commitment to postdoctoral training. The training program itself has helped bring together faculty and research projects. We now have quite a few faculty at UCSF who were trained in the Pew program who would not be doing what they are doing at UCSF had they not been Pew fellows. So, in a sense it is being institutionalized through the faculty.

  2. other?

    There is a network of people out there who will probably continue to interact. Something needs to be done to encourage that, to retain that in the future. The Pew network of fellows will continue to draw upon each other in a way that would have not been the case had they just been independent postdocs.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

Phil Lee's commitment to training, to doing health policy and health policy research, and to incorporating a wide range of people with different backgrounds and expertise was crucial. Another aspect to it was that prior to the program, UCSF had gotten a grant from the Mellon Foundation to set up a clinical epidemiology program affiliated with the institute. That served as a basis for people thinking about the development of postdoc training programs. Mellon had a very clear vision of institutionalizing the program by training faculty who would then stay at that campus and become the leaders of the future. Although staying at the site was not a goal of Pew early on, the idea of training people and returning them to the field helped to shape the Pew proposal.

Another aspect was that unlike the other programs, the institute is essentially a freestanding research group, not a department—we don't grant degrees. Our faculty saw the opportunity of developing a postdoctoral training program as a very exciting opportunity to do some teaching, to get involved with students in ways that were not normally available. We're not apt to do a lot of teaching in the medical school or nursing schools. Here was a program where we could really do something.

The third piece was that many of the faculty involved in the institute come out of a multidisciplinary background and believe that good integrative health policy research and policy analysis is something that is valued and is good to do, and

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

this gave us an opportunity, in a sense, to reproduce. There is a certain, almost biological urge, but it made a lot of sense. It allowed us to build a program that fulfilled a need in the field, one that we weren't seeing being filled.

There remains a certain amount of uncertainty surrounding the maintenance of such a program because we don't have any major grants. We are waiting to hear about AHCPR. There are active discussions going on among some leadership people within the university to try to develop a new postdoc program, and so we are working on that and thinking about various ways of financing it. Can I say that we have something in place? No. Are we working on it? Yes, with a strong commitment to make something happen.

3. What was the need in the health policy community when your program started, and how have these needs changed today? Is the job done?

When our program started there were very few integrated postdoc fellowship programs. I think now there are more of them, some funded by AHCPR, Robert Wood Johnson Clinical Scholars, the new Johnson Policy Scholars, etc. So there are definitely more opportunities and programs for postdoc fellows. On the other hand I think the need for training and the demand for well-trained people have been increasing even more rapidly. I still don't see a lot of programs that are really focused on integrating people from multiple disciplines. For instance, the Johnson clinical scholars are for clinicians, the Johnson Policy Scholars are for Political Scientists and Sociologists, and the AHCPR, while broader, tends to have very few people at any one site and is often more predoc focused rather than postdoc focused. Furthermore, there is not much integration across sites and not enough blending.

Bottom line: there is still a real need. I don't think the job is done.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

There are two major things: (1) mentoring. The model that we chose worked well, and that's a model where the fellows basically get involved with one or two faculty members, work on some projects, and get that hands-on experience. It's more of an apprenticeship model than a classroom model. (2) I think over time, admittedly with some prodding from the Pew Charitable Trusts, we developed a more structured set of seminars and courses. I think the health policy

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

seminar had always been successful, but by making it more formal, by structuring sequences within it, I think it has become much better. I think the writing seminar that Drummond Rennie led worked extraordinarily well. It helped the fellows produce publishable papers and learn how to do high-quality work in a supportive but critical environment. My art and science class filled a niche that other people seemed not to get training in.

The curriculum might still have benefited from tighter structure; it is far from rigid. We are tightening up some of the mentoring relationships, doing more reviews, and making sure people are moving along the way they ought to be moving along in terms of productivity. We continue to improve it year by year.

5a. What was the most innovative or unique aspect of:

  1. your program design?

    The apprenticeship model. It is a postdoc program, but its a structured postdoc program with people working on projects with faculty members.

  2. methods of implementation?

    In a sense this is also a response to the kind of situation that we're in where almost all of our faculty are on soft money with research projects. Asking them to teach lots of courses without paying for it can be difficult. Getting fellows involved with them in research projects does work. It's a win-win situation. We have learned how to build a model that works in that kind of environment.

5b. What were the biggest challenges or barriers to overcome of:

  1. your program design?

  2. methods of implementation?

  3. educational process?

In general, one of the biggest challenges early on was space. When we were located in our old site, on the main campus, there wasn't even enough room for faculty. So there was no place for the postdocs. They were hanging out in weird and bizarre places. They were often off site in office space provided, but this hindered interaction. When we moved 5 years ago we provided real office space for all the fellows as part of the unit. They were integrated with the fac-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ulty, with the unit. It works a lot better.

Matching fellows with faculty has always been a bit of a challenge. Because of the nature of the educational model you need to get a good match. We struggled back and forth in terms of how does one develop that match. Sometimes it becomes a problem because the fellows feel like they are in a candy store: there are all these great things to choose from. They sort of freeze. We are now restructuring that a little bit. This year we'll provide a handbook describing each of the faculty and what he or she is doing. We have always had faculty presentations, but it goes by too quickly and is often too intense. Three days of hearing 30 to 40 faculty members speak ... it all blurs together. We are now having present only those faculty seriously interested in having fellows participate.

While I think we have attracted and recruited very good fellows. With one or two exceptions where the fellow had decided to move on to another position early on, I don't think we ever washed anybody out. Sometimes one might question that. It's not like you have a graduate program with exams that people just don't pass. There isn't a clear criterion for dropping somebody after a period of time. It's fuzzy. I'm not sure I would recommend major changes, but I think in the future there might be somewhat more careful review so that people have a more clear sense that the second year is not guaranteed.

6. Based on your experience, what lessons were learned about the educational process in term of:

  1. recruitment?

    Identifying a good match is one of the key things that needs to be done. It's not just saying that this person has the right degree of horsepower and interest, but how is he or she going to fit into the program? Are there a couple of good potential matches with faculty members and their ongoing projects? Working that through is very important.

    At the postdoc level we find ourselves sometimes attracting people who are also looking at starting faculty positions at the same time. The timing of that is not always ideal. They may or may not be getting an offer from somebody, and how are we going to ask them to make a decision by "Friday" when they may or may not have a firm tenure-track offer from a great university in 3 weeks. Are you going to hold them to that or not? What that process then does sometimes is extend the admissions process for those candidates (often the best candidates are the ones in this posi-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

tion), and then if they finally say no, then you may end up going fairly far down your list because other people have already excepted other positions. How to play that through? We need to be working on this.

  1. degree requirements?

    We don't have a degree. In terms of what is expected of the fellows we have been fairly flexible. We have some people who are on trajectories toward faculty positions, so they really need to be publishing and we are encouraging them to do that. There are others who are much more policy oriented, and for these fellows publication is secondary. We encourage research for them as well, but we have them working on other kinds of projects, doing other kinds of reports. This flexibility has worked well. It is not a problem; rather it is a matter of learning how to have everybody understand that to some extent we are willing to tailor the program, but that does not mean that we are willing to be pushovers.

  2. curriculum and content?

    The core seminars are basically the same for everybody, although sometimes we have some flexibility. For example, we encourage fellows to take a research design course, but some fellows who have come in with good research skills ask to take other courses, and we are flexible with that.

  3. integration of fellows with other students, the rest of the university, and the program in general?

    This has worked out very well in terms of our postdocs at the Institute. Part of that has been a conscious strategy of using Pew funds to supplement other fellowship funds so we'll have people who were fully funded by Pew and others who basically got $1,000 all mixed together in the same classes. We did not establish a pecking order. We set a standard for fellowship applicants and say this is the standard all our applicants have to meet, and then once they are in that acceptable pool, who we ill give what kind of dollars to out of what pot partly depends on the constraints of the pots. For example, the AHCPR program really is more interested in people who are going to be academics rather than people going into policy careers. So, we have shifted the AHCPR dollars toward them and the Pew dollars go for the people who are more policy oriented. There are some Pew dollars going to the research people and some AHCPR dollars

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

going to the policy people to bring them up to the same stipend level. Everybody is on equal footing. That integration has worked very well. I frankly cannot remember who of our fellows have been primarily Pew and who have not. We are fully integrated.

In terms of interaction with the rest of the university, that is becoming better. In the last couple of years we have expanded the program in some way. Each year for the last 3 or 4 years we have had at least one clinical fellow spending all of his or her research time with us. They go to our seminars, they get involved with a faculty member research project, and they are basically integrated into the fellowship program, but they are paid for by somebody else entirely. These are people who would have otherwise been doing bench research on pulmonary function, for example. They are now doing work on risk adjustment models and health care costs of AIDS, etc., exactly the kinds of things the Pew fellows do. In this way we are integrating our fellows with other people in the classic fellowship programs on this campus.

  1. relationship between faculty and students?

    The relationship works very well. There have been some situations where the match has not been perfect. In general, people like having fellows. In many instances the faculty see the fellows more as colleagues than as students. We have also had situations where faculty have research assistants who then start functioning as fellows. In other words, what we do is pay them for 70 percent of their time, and the other 30 percent of their time they do fellowship activities. Often we will include them in the Pew program as Pew fellows. This is another way in which we have institutionalized things. My guess is that many of the fellows don't even know whether the person sitting next to them is funded out of fellowship money or project money. We have done that by having a set of criteria where we say this person is eligible for a fellowship. Now, how do we fund him or her?

  2. completion rates (where applicable)?

    Basically everyone has completed the program with the exception of one or two who left early on for wonderful opportunities elsewhere. We have had a few fellows who have taken more than 2 years, and usually it was because they got additional fellowship support for the third year and it made sense for them to stay the third year either for a spousal connection or another reason. I would not call that a lack of completion.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

7a. To what extent do you think there were ''programmatic'' barriers to student completion?

It's more PhD related.

7b. How can we measure success for those programs where completion rates do not apply (i.e., postdoctoral programs)?

I think that this is probably the hardest thing. One way to look at it would be to look through our alumni lists and look at where they are. Do you get a warm, fuzzy feeling? Are they doing the kinds of things that you would like them to be doing? If you look at it from the perspective of the Robert Wood Johnson scholars which basically prepares fellows to be the future chairs of medicine, you probably wouldn't think someone who just went into private practice to be a huge success. On the other hand, I know one person who is in private practice but who continues to be interested in health policy issues and who writes articles on health policy issues for the New England Journal of Medicine. That's still a success. In terms of our postdocs and where they've gone, almost all of them are doing things related to health policy in various kinds of settings; some of them are academic and some of them are not. I don't think there are very many just sort of doing routine kinds of activities. You would not look at these people and say, "Well, they probably would have been doing this anyway." That's probably the best implicit measure of success.

If you really wanted to put a lot of energy and effort into it, what you could do is ask the fellows what they think their career would have been had they not done the postdoc fellowship and what does it now look like? Of course, you get a certain amount of bias in that. Ideally, what you would have had would be the career trajectories that they would have written prior to applying. I can tell you that I went through a postdoc training program, and had I not done that, I would have had a very different career trajectory.

8. How does the Pew fellowship approach differ from a traditional fellowship approach? How have the major outcomes differed?

The Pew program has the policy focus and the integration of different experiences, different disciplines and different programs. You have the network, which is very important, making it bigger than any one program.

Outcomes? Hard to say. Most of the Johnson clinical scholars, the other major program, have done very good stuff, but they are all MDs. Now, some of our fellows are

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

MDs, but most of them are not, so it is hard to separate. Its hard to know how much of that is Pew and how much is MD versus PhD. But I think the Pew program has really been more policy oriented and not wholly academically focused. I think that's a real plus. It also means that some of the academic fellows get to know and rely on the nonacademically focused ones, and vice versa. So, then, people who go and become policy analysts in the government can then call on their colleagues for advice in ways that would not have been done had they not gone through that joint training program.

9. If you were asked to give advice to another university attempting to initiate a similar program, what would you say?

You really need to think through the nature of the faculty, what their incentives are, how they would benefit in the development of a program like this, and how you can bring them along in terms of doing it and structure it very carefully to fit that setting. Our model is very faculty intensive with little in the way of dollars. So, you have to have a whole series of other things that make it work. It's not like we are a PhD program where faculty are going to get teaching credits for doing the teaching as a fairly structured kind of thing where you can say here is the syllabus, go do it. Our program is a little bit like an individually designed house that you need to build onto the wall of a mountain. You really need to understand the local geology. When you do its wonderful. But you can't simply take those blueprints and use them somewhere else.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Doctoral Alumnus John McDonough Tuesday, September 5, 1995, 10 a.m.

1a. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What are the most important contributions?

Personally, the Pew program gave me a lot of skills that I did not have before. It exposed me to folks in the health research community who have been very important and helpful and who I would not have otherwise had the opportunity to meet. It exposed me to an outstanding network of people.

In the aggregate, I think it has done roughly the same for an awful lot of other people from many different walks of life.

1b. What is the Pew "legacy" in terms of:

  1. health policy?

  2. education?

  3. your future?

Not really knowing a large number of other Pew fellows, I think it would be the combined legacy of what people have been able to do with their degrees, and I assume that it's substantial, but I don't know that for sure.

2. What was the most innovative or unique aspect of your program design and implementation?

Figuring out a way to allow people to pursue an advanced degree without having to interrupt their job career.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

The very small student-to-faculty ratio (8:1), the intensive investment by the Michigan faculty, and the multidisciplinary nature of the curriculum.

4. How was the Pew approach different from the traditional teaching approach?

The intensive weekend was unlike anything. The extensive computer linkages among and between students and faculty was terrifically sophisticated.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

5a. How has your professional life changed as a result of the Pew program? Mat value has Pew training added to your life?

I can't say, as is the problem with all these types of programs: How much of it is because of the program and how much would have happened anyway? I don't think you can really know that. When I started the program I was in a minor leadership position in the Massachusetts House of Representatives; since I joined the program I served a period of time as the House Chairman for the Committee on Insurance, where I dealt extensively with health-related manners, and more recently I've become the House Chairman of the Joint Committee on Health Care, where I have very extensive involvement in all health policy. The best contribution of the program to what I do is the giving of more critical policy analysis skills that I didn't have before.

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

I don't know how much is a result of Pew. I think quite honestly I might be here anyway. I feel primarily that what I'm doing now, I will be and am better at because of the program. I can't really say that the program is what put me here. It has, however, made me infinitely better equipped to handle the responsibility.

6. If you did not complete the program, do you plan to? If yes, why? If no, why?

I will definitely complete the program. There is no question. I am right on target with my dissertation, and I should be done next spring. I got a Health Care Financing Administration dissertation grant to help me finish.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Postdoctoral Alumna Lisa Bero Wednesday, August 24, 1995, 2 p.m.

1a. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What are the most important contributions?

It helped me complete a career transition that I had planned from basic science to policy research, and I'm sure that without the fellowship I would not have been able to do that. It also helped me develop a lot of contacts in the health policy world and to learn how to develop policy-relevant research questions.

In general, Pew has succeeded in sending out some highly trained people to a whole variety of positions around the United States to do policy-relevant research. I know one of the goals of the Pew program was to seed the field, and I think it has accomplished that.

1b. What is the Pew "legacy" in terms of:

  1. health policy?

    It established a network of highly trained individuals who will stay in touch forever. That will be really important, as it helps people in different agencies and in different states as they approach similar problems with different perspectives.

  2. education?

    The Pew program has trained fellows in health policy, where there has not been a lot of programs available. From a more personal perspective, the postdoctoral training has really been a lot more valuable because there are just more programs for undergraduate training. That is not to say that we have enough of either. But for the postdoctoral training it filled a very large gap. There just wasn't anything out there, and what that was able to do was to educate people who had fairly advanced training in some specific discipline and then help them target their current training for health policy.

  3. your future?

    The program positioned me for doing a lot of international work. I had my first contact with the World Health Organization while I was a Pew fellow. That rocketed me into doing international health policy work, which was great. I got a real jump start on that through the fellowship.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

It enabled me to make a career transition, as discussed previously.

2. What was the most innovative or unique aspect of your program design and implementation?

The most unique aspect was that it was very tailored to the individual fellow. Every fellow came into the program with different experiences. We had some core courses, but most were tailored to the individual, and the faculty spent a lot of time talking to each fellow to find out specifically what their needs were at the beginning, It was a needs assessment. We were asked: Where do you want to go? What do you need to do that? And then efforts were made to give the fellows what they needed to get to where they wanted to be. This individual attention was by far the strongest point.

You really had to be a certain type of individual. If you needed a lot of direction the implementation didn't work too well. You had to be very self-motivated. It wasn't just like you had to go to class and sit in on lectures and get your grade and come out. You had to decide with your mentor what classes you were going to and what you were going to get out of them. A lot of people audited classes, which gives you the opportunity to sort of blow off the class. You had to be self-motivated in the actual implementation.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

The writing seminar. It was interesting. I was in the cohort where the writing seminar changed directors right in the middle of my fellowship. It was two very different types of writing seminars. They were both great. It helped us learn how to write for a variety of audiences, not just the academic journal audience, and it also helped us learn to be good and fair critics of our colleagues and to offer advice in a constructive way. The writing seminar was great.

The health policy seminar series was great, especially for somebody like me who didn't know a lot about what was happening in health policy. A lot of different speakers, people from government, foundations, and private industry came through, and discussed a whole area that I had never experienced. It was great to listen to these people for an hour and then chat with them informally afterward.

The third thing was the optional part of our curriculum; not everybody had to take this. I took it as part of my tailored curriculum, and that was a research design epidemi-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ology workshop which really helped me as an academic into that sort of thing.

4. How was the Pew approach different from the traditional teaching approach?

The individual curriculum made it different. It was much more interdisciplinary, and that made a huge difference. It was interesting to me because I came from basic science, and I thought that I was coming from such a narrow background that no one would understand me and that I would not understand anybody else. I thought all these sociologists would be much better off than me, but as it turns out we were all in the same boat. I didn't realize that every discipline was so narrow in terms of PhD training. The sociologists and the economists were thinking the same thing. And so we had to learn to understand each other and to really gain an appreciation for all these other disciplines, and I think it was really a win-win situation. We all shared that relief that we were now outside those strict departmental lines. That approach was very different from all the traditional training we had.

5a. How has your professional life changed as a result of the Pew program?

It allowed me to make a career transition. It also allowed me to explore nonacademic job options, which I never did go for, but before I started the Pew program, I would never even have interviewed for nonacademic jobs. I was very interested in being an academic, and after my fellowship I interviewed and actually was offered jobs in nonacademic settings. Still, I wound up staying in academia, and I am very happy here, but on the other hand, I am also in a multidisciplinary setting and I am kind of an odd academic. I do a lot of work on government committees and international committees. Had it not been for the Pew program I would have been a much more typical academic.

What value has Pew training added to your life?

It basically allowed me to do exactly what I wanted to do. I'll never forget when I started the Pew fellowship, the person in charge of the admissions at that time called me up and told me I got the fellowship, and I was working in this lab and I was just so happy. For over a year I had been pondering how I was going to get where I wanted to go. I had this general idea of what I wanted to do, but I had no idea how I was going to get there. I had obsessed on this fellowship to get me there, which it did!

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

As discussed previously, it allowed me to make a career transition from basic sciences to health policy and to consider nonacademic jobs, which I still would not rule out. For example, when I do a sabbatical I'll almost definitely do it in a nonacademic setting. Also, I do a lot of technical assistance and policy work that is kind of off the traditional academic track, which hurts me a little because you don't get a lot of brownie points as an academic for doing that. On the other hand, it makes me feel like what I'm doing is relevant and somebody can use it tomorrow rather than 10 years from now.

6. If you did not complete the program, do you plan to? If yes, why? If no, why?

Not applicable.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Leon Wyszewianski Wednesday, August 16, 1995, 2 p.m.

1a. Based on your experience and familiarity with the fellows and the programs, what did we really accomplish? What were the most important contributions?

We heightened the visibility of health policy. We forced people in academia to address the questions "what is health policy," "what tools do we need to deal with health policy" and "how can we learn to do a better job at formulating and analyzing health policy?"

1b. What is the Pew 'legacy' in terms of:

  1. health policy?

    It is hard to say. It is beyond my knowledge. I can only speak about the Michigan fellows. There definitely was a contribution to the health policy field, and those fellows who wanted to go into health policy were able to go into health policy. Training and/or a degree from Michigan enabled fellows to become health policy people when they were not health policy people before. And then there were those who were already in health policy who I would like to believe are now doing a better job as a result of their training.

    I often see UCLA (University of California, Los Angeles) or Brandeis alumni in the literature. The RAND people are particularly strong in that area. That's another example of contributions to the field. But I'm not in a place to judge whether or not they would have done differently had they not been part of the Pew program.

  2. education (doctoral, postdoctoral, or midcareer programs)?

    Comments about education come up later.

  3. your institution?

    In terms of Michigan, there has been a major contribution in that we now have a program in health policy that we most assuredly would not have if there had not been Pew funding. There are other derivative benefits in having the program here, such as attracting faculty, creating a nucleus of interest around health policy at Michigan, etc.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

It was an interesting case at Michigan. We had been thinking about setting up a program in the OJ/OC (On-Job/On-Campus) weekend mode in the area of health policy at the doctoral level. We had been exploring and thinking about such a program; however, we saw ourselves unable to launch such an effort for lack of financial resources. It was therefore a real godsend when the Pew Charitable Trusts decided to help us set up a program exactly along those lines.

The germ of the idea was already there, but the impetus was certainly the result of the Pew Charitable Trusts. I doubt we would have ever started the program without that.

There is a real commitment and aggressive effort at Michigan to continue the program.

3. What was the need in the health policy community when your program started, and how have those needs changed today? Is the job done?

There was this very large group of people out in the health policy field already, in responsible positions, who felt inadequately prepared to deal with a lot of their tasks. They wanted to do a better job at what they were already doing and were looking toward our PhD program as a place to learn that. And, it was inappropriate. PhD programs are programs to train researchers and faculty, and therefore have a very strong emphasis (as it should) on methodology, research skills, and other scholarly kinds of work, but these people were not looking for scholarly training. More later.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

At the very beginning there was this view that the curriculum would be a division between more purely didactic courses and what were called policy seminars. At the seminars the emphasis would be on discussion and on interaction. It turned out relatively early in the game that the fellows really did not want all this free-form seminar discussion; rather, they wanted discussion in the context of much harder-hitting didactic material. They did not want to just walk in and have some policy questions thrown at them. So, we pared back on those policy seminars.

Another change that I was heavily involved with was beefing up substantially the methodological aspects of the curriculum. We started out with having the students pass a graduate-level statistics exam, and in addition to that, they got one course in methods, and that was it. What they came out with was knowledge that they did not know how to

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

apply to their day-to-day work. So, we instituted entrance exams to show competency up to a certain level in statistics, microeconomics, government, and organizational behavior. This started everyone at the same level. The faculty had been complaining that the students were all at different places. The entrance exams forced the students to bring themselves up to graduate-level knowledge before coming into the program. And then we gave them a statistics course (science, methods, how do you know what you know?, threats to validity, etc.). We then followed that up with two more courses: applied statistics and applied methodology. These courses were coordinated so that the students were doing things in the methods course that was related to the other course. That was the major change. It was a welcomed change. We are not trying to make them into methodologists, we are not even trying to make them into researchers, but we certainly wanted to put them in a position where they could do good-quality analysis and certainly write a good-quality dissertation that had proper methodological components.

We also set up specific dissertation seminars where the students were led into finding a topic and how to go about defining a topic. It provided structure. We initially thought that we would bring in all these very bright successful people who would have no problem writing a dissertation because they would just thirst for learning. Well, even with their thirst for learning they needed structure due to the heavy demands of their full-time careers. They were too busy. These were people who were working 50 to 60 hours a week and trying to go to school full-time. We should have realized the need for structure. We have a lot of experience with this type of student. We somehow forgot that at the beginning of the Pew program, but we soon remembered and then set up the program much as we had done for our master's-level program. This worked much better. We concentrated on the dissertation early on rather than just assuming that somehow the students will know how to find a topic, how to find a committee, and how to move along the path. We learned our lesson.

In the beginning there was a very lenient attitude toward the students and their course work, so we found many students hobbled down with huge numbers of incompletes. This hampered the students' ability to get done because not only were they writing a dissertation but they had stale papers to get done from courses long gone. We soon changed the policy to make it very difficult for the stu-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

dents and faculty to get or give incompletes. What we did was to discourage very strongly the faculty from having a major paper due at the end of the course. You just can't do that with this population. Instead, we encouraged the faculty to have the term paper develop in pieces over the course of the term. The students had deliverables that by the end of the course would add up to the product the faculty were looking for. This worked far better than waiting for it to somehow magically appear in the last flurry of the term. The rate of incompletes dropped dramatically. The students learned that incompletes were given only under the most extreme circumstances (e.g., a death in the family), not because they had a big project at work. They had to make the program top priority and a major presence in their lives.

5a. What was the most innovative or unique aspect of:

a) your program design?

We offered a doctoral program on a nonresidential basis, where students come here once a month for 4 days, 12 times a year. We are still the only game in town. As far as I know we have no competition.

5b. What were the biggest challenges or barriers to overcome of:

a) your program design?

How do you offer a doctoral program to people who are working full-time and who are geographically dispersed throughout the United States?

6. Based on your experience, what lessons were learned about the educational process in terms of.

  1. recruitment?

    This is an academic program. This is not another job. When all is said and done, we offer pretty standard, straightforward doctoral-level courses. The students had to have the academic know-how to make it through those courses no matter what their other qualities were or their other accomplishments may have been. We may have lost sight of that in the beginning. We tried to take in people who were very accomplished in the field but who were not necessarily great achievers in the academic realm. It was a disaster. They just couldn't cope with all this. If you're going to work full-time and go to school, you have to be very good at the school business. You have to already have mastered the craft of

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

being a student. Some of the early cohorts had never mastered that craft. We learned to pay a lot more attention to grades, past academic accomplishment, as well as Graduate Record Examinations (GREs). It turns out that if you take people who have GREs in the 80th and 90th percentiles they do much better. Also, we looked for people who have shown that they can handle work and school at the same time. That is a good predictor of success. This was another lesson learned: no matter how applied this is, and even though we were not looking to create scholars, ultimately they cannot take full advantage of what we have to offer if they are not good students. Many of these students' first inclination is to find someone to whom they can delegate the work, but you can't delegate this. This is one of those essential activities that cannot be delegated.

  1. degree requirements?

    Competency exams were a good screening device. The first day the students arrived they were sat down for 8 hours of exams. That sent a message: we're very serious, this is a serious program, it is not like coming to a conference once a month. It made them have to get back into studying even before they got into school. It was a sort of self-selection process; after the exams some people thought twice about whether or not they were going to do this. This also made the faculty happy because it brought all the students in at the same level. A large set of complaints from the faculty disappeared once the competency exams were instituted.

  2. curriculum and content?

    This was discussed earlier.

  3. integration of fellows with other students, the rest of the university and the program in general?

    It is a concern. The students are sort of isolated, and they tend to gravitate for their dissertation to the faculty that they've had in class. So, one thing that I started, which I believe has been helpful, is to actually bring in faculty every weekend to talk for an hour during lunch about what he or she is doing; in this way some contact is established. Integrating the students into the rest of the university is very difficult. What has been very useful is computer conferencing that all the fellows participate in. That has been a unifying element. There is a lot of conversation going back and forth among the scholars and faculty. Different topics are dis-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

cussed. The fellows also have their own encoded conference that the faculty cannot participate in. It is very important for the fellows to have this opportunity. The only people that understand the kind of hell they're going through are the people going through it with them. They support each other and we foster that.

  1. relationship between faculty and students?

  2. completion rates (where applicable)?

    We learned that they are no different from other doctoral students. The overall completion rate is about 50 percent (statistics on this are not very good), depending on your denominator, and that is about what we have had. Despite all our efforts, we have not been able to improve on that very much, but we are not any worse than other programs, and ours is a tough program to pull off. Given the unique aspects (students holding full-time, highly responsible jobs while taking courses and working on a dissertation) we are doing OK.

7a. To what extent do you think there were ''programmatic'' barriers to student completion?

The main hurdle at Michigan is the fact that the students are working full-time outside the PhD program. That is built in. It may or may not be considered a "programmatic" barrier. Perhaps it is a structural barrier.

7b. To what extent do you think the program was beneficial for those who did not finish the program?

I have spoken with a number of alumni who did not finish but who still believe that they benefited tremendously from the course work. The difference between them and someone who finished is that they did not write or finish a dissertation and the others did. One could argue about how much learning goes into writing a dissertation. We would like to think that everything we teach is useful, but the students on their own account have confirmed the usefulness of the course work.

7c. How can we measure success for those programs where completion rates do not apply (i. e., postdoctoral programs)?

We have some people whose entire direction changed as a result of the Pew program. For instance there is one alumna who entered the program as a practicing pediatric

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

neurosurgeon and the last time I spoke with her she was working as a health aid to one of the representatives on Capitol Hill. She would not be doing that if it were not for the Pew program. And, she has not written her dissertation.

There is another alumnus who is now the Deputy Director for Health in Texas who says that how he approaches problems has been greatly influenced by the 2 years of course work at Michigan.

With regard to measures, did they change direction from non-health policy to health policy? Do they do what they do differently now that they have knowledge that they didn't have previously? Those are basically the two reasons why someone would come here.

8. How does the Pew fellowship approach differ from a traditional fellowship approach?

We are different from everything. We are different from a traditional doctoral program, other Pew programs, etc.

How hare the major outcomes differed?

People who would never have been able to go for their PhD can and have. Precious few would have pursued any kind of doctoral education were it not for this program. A traditional doctoral program would have been way beyond their reach financially and careerwise.

9. If you were asked to give advice to another university attempting to initiate a similar program, what would you say?

If you don't have any experience in dealing with this kind of population, you're in for a very difficult time. There is a substantial investment that must be made up front, and a continuing investment that must be made throughout.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Postdoctoral Alumnus Mark Legnini Thursday, August 24, 1995, 9:30 a.m.

1a. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What are the most important contributions?

It seems to me important to distinguish between the two kinds of Pew programs (postdoc versus predoc). In terms of the UCSF (University of California, San Francisco) postdoc, the biggest contribution is that it helps one to make the transition from training to a professional career. Most people in their doctoral programs are busy learning techniques and how to apply them and such, but that's very different than building a career.

Pew plugged its fellows into a network of people corresponding with their chosen areas and got them involved in things that one would expect to be involved in at the start of a career.

1b. What is the Pew "legacy" in term of:

  1. health policy?

    I don't know if the Pew program has changed the health policy field. I have no idea about that.

  2. education?

    I would imagine that some of the predoc programs are a little different in structure than traditional predoc programs. But I wasn't directly involved in those.

  3. your future?

    It provided me with a good transition from being a doctoral student to being someone in the field.

2. What was the most innovative or unique aspect of your program design and implementation?

The emphasis on networking and getting involved in the politics and the art of doing things rather than just plugging people into existing research projects and making sure they have a job was the most innovative aspect.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

The best thing was that there wasn't a curriculum.

I went there ostensibly to work with a particular person, and when I got there that person decided he was going to take a sabbatical that year. So the flexibility of the program and the fact that people involved in the program could individually work with people to put together something that they needed was to save the day for me, and I think that was a strong point of the program in general.

4. How was the Pew approach different from the traditional teaching approach?

There are all kinds of postdocs. There are postdocs where you don't have to do anything but work on your tennis game. And, then there are postdocs that are really apprenticeships where you go and become the co-Principal Investigator on someone's research grant, and that's all you do. Pew was a little different than both of those models.

5a. How has your professional life changed as a result of the Pew program?

Specifically, the job I got after finishing the program was the result of some of the work I had done during the fellowship, and a result of some of the connections I made and people I met during my time as a fellow.

What value has Pew training added to your life?

It's nice to be employed.

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

That's a tough one to answer. I don't know what I would have done otherwise. I don't know how it would have turned out. I went back to school after working for about a dozen years. After that I went into the Pew program. My path was therefore a little different than that of other people who went from college to graduate school and then the Pew program.

6. If you did not complete the program do you plan to? If yes, why? If no, why?

Not applicable.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone interview with Doctoral Alumna Patricia Butler Thursday, August 24, 1995, 11:00 a.m.

1a. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What are the most important contributions?

I had three objectives when I went into the program: (1) to learn a number of specific technical areas that I had not previously had formal training in; (2) to have a chance to think about the great health policy issues of the day in a more academic setting; and (3) to read the classic literature in the field, because my background is in law and although I've been in health policy for almost all of my career (about 25 years), I had no formal training in it, so I always felt that I might be missing some of the literature that everybody who has formal training had. Of the three objectives, two were very well met and another not so much met given the nonresidential nature of the Michigan program. The courses that I particularly wanted to have training in were epidemiology, research methods, biostatistics, health economics, and, to my surprise, health behavior, which turned out to be one of the most useful and interesting subjects that we studied. I really got my money's worth there. In terms of reading the classics in the field, I now feel that I have that under my belt. In some cases I was pleasantly surprised that I had read a lot of the material but I need a confirmation of that.

Because the program was nonresidential, we went to classes for a long weekend every month. That meant 30 hours of classes virtually nonstop. Unlike a residential program, where you might have a couple of hours of classes a day and then you could perhaps go off to have a beer or coffee with your classmates and maybe even your faculty members in the great old tradition of graduate school, that opportunity for us was rare. People were so busy, and by the time we were done with class at 5:30 p.m. on a Friday or Saturday night, the last thing we wanted to do was spend any more time with one another. Although we always did go out together at least once. I had hoped, perhaps unrealistically, that we would have had a chance to process what we had learned together, but we were always too shot.

The fact that we were connected by a computer system did allow us to communicate actively, even though we came from all over the country. We were able to talk about current issues through this medium, and that was helpful. Some people participated more actively than others, and the faculty

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

got involved as well. So, interaction was not completely unattainable, but it was a limitation of the program that Michigan offered.

On the other hand, for most of us who were midcareer and for many of us who live in places where there isn't even a graduate-level health policy program offered, this was the only way we could do this. That's why I am so grateful to the Pew Charitable Trusts for supporting this kind of program. I am disappointed and concerned about what happens when the money goes away, as it has basically begun to do. I think it is a unique and extraordinary opportunity and one that I think should be available to others in the future. I am particularly concerned that as Michigan turns and tries to make its programs self-supporting, as they need to do without an outside source of revenue, then they are going to get mostly people who have deep-pocket employers who can send them, and that will tend to be more people in the private sector, more health services administrators, as opposed to health policy folks from public agencies. Several people from my class were from public agencies, and the public just does not have the ability to do that. Most people cannot afford what is about $50,000 over the course of the 2-plus years. I think that's a shame. The kinds of people who have been trained in the past will change to people who have the means either personally or through their workplace and more residents of Michigan who don't have to pay the out-of-state tuition. I think it's fine that Michigan gets to take advantage of this thing, but it was such a unique national resource that I am sorry to see the dollars go away.

1b. What is the Pew "legacy" in terms of:

I think these are good questions that I hope someone who has a broader perspective can answer. It seems to me that there is great advantage to nonresidential programs. As I mentioned, they are not perfect, but nothing is. The notion of being able to go to a school with the reputation of Michigan when you don't live there is important. There are a lot of programs that are midcareer that are more mail order or you meet less frequently. Many of those are MBA-type programs and others are at the master's level. I don't think there are any in health policy at the doctoral level, if any at all. I would hope that someone would examine whether this is a good model for midcareer education in this field. It seems to me that it is, but I speak for myself and am grateful for the opportunity to do this, which I wouldn't have been able to do otherwise.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
  1. health policy?

    I certainly know the names of some former Pew folks of the various campuses, and I know that they are contributing to the policy world. I haven't been trying to figure that out. In terms of my own personal experience, although I had been working in the field a very long time, I still felt that I got something out of it. People asked me why are you doing that, you could be teaching that program? I guess I always feel a certain humility about that. One can always learn more. I feel that I am a better health policy analyst as a result of the program, even though I came at it with quite a bit of experience. Some of what I got personally was a sense of self-confidence in the field that I actually knew something about, but I wasn't sure I knew as much as I knew. Health economics was a good example of that. I always felt fairly intimidated by the subject, owing to a bad experience in college, and it was demystified for me. It was made very clear. We had a very fine economics teacher, and although I would never claim to be an expert, I now feel that I can talk knowledgeably and with a sense of self-confidence to people who are health economics experts. Some of that was just breaking down some barriers. For the younger people who may not have had the same exposure to some of the subjects, I think it also offered a real opportunity for a lot of new knowledge. My impression is that people who took the program seriously and worked hard at it should have not only learned a lot but should have also been able to be better policy analysts.

  2. education?

    In terms of education, there are probably some lessons to be learned. I personally feel that way, but I also feel that it would be useful for educators to evaluate this and if it's a good model to try to promote it. It's a shame to think that there is only one such location in the country.

  3. your future?

    Discussed above.

2. What was the most innovative or unique aspect of your program design and implementation?

It was the nonresidential aspect.

3. What was it about the curriculum that contributed or did not contribute to the proaram's success?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

I personally have a strong philosophical orientation toward public health. To me it was very valuable that we were housed in a school of public health. Some people who are more interested in health services research or health systems management may not care as much about that. To me, public health is a fundamental set of disciplines whose influence is often lacking in our current national debate. The neat thing about public health is that it is so interdisciplinary. Our degree was a DrPH, not a PhD, and we met requirements from the School of Public Health specifically, rather than just the graduate school at Michigan, and that meant that we got courses that even the master's students in our department don't get, like epidemiology and health behavior. Those were extremely valuable because they are or should be critical aspects of health policy in general. They are fundamental to an understanding to public health. When we talk about health policy we often think of financing and delivery systems, but at the core should be some of the public health disciplines. Michigan offered that interdisciplinary focus, and I think that's unusual at this point among the other programs. I found that very valuable.

4. How was the Pew approach different from the traditional teaching approach?

It was different at Michigan because of the location in the school of public health. It was also very small; our class had 9 people, 1 of whom dropped out, and I think other classes have had up to 12. The current class, because they are trying to figure out ways to finance it and break even, started with 18 people and still has 16. And, that's still pretty small. It is very intimate. We all got to be extremely close friends. We really needed one another for moral support. Perhaps this is more true here because of the nonresidential aspect. I was surprised at how important that is, because I am self-employed and very self-motivated. I work quite independently without needing to be around people at all, and yet the pressures were so extreme that we often turned to each other for support on both personal and school matters. We will all remain good friends.

Something else that the Pew funds offered Michigan, even though they have an excellent faculty, was the ability to bring in some of the best people nationwide to teach a course if they didn't feel that they had the capacity on their faculty. So we had the Dean of Public Health at the University of North Carolina, Michel Ibrahim, who is an MD/MPH to teach our epidemiology course. Rather than

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

teaching it from a mathematical perspective, he taught more from a policy standpoint, drawing the principles from epidemiology but not getting hung up on how you estimate odds ratios. Michel was just exceptional. He is an internationally recognized expert on all sorts of things. We were taught how to use the epidemiological approach in health policy. I assume that's fairly unusual. We also had an exceptional methods professor who had been with the school, but then left and with Pew funds was brought back to teach. We also had a political science professor from Johns Hopkins. That was very valuable. We weren't constrained by the faculty and interests of people on the campus. That's something that a program that has to sustain itself internally cannot do.

By definition, the Michigan program drew from people from all over the country, and therefore, the diverse experience of the student body was very interesting to me. There were a couple of folks from state government, we had a state legislator, we had someone who worked in the foundation community, a hospital administrator, and a person in the insurance industry who also had physician assistant training: very interesting people who contributed to the very rich discussions that we would have. We weren't all of one mind. As long as you can keep an open mind, you can learn a lot from people like that. A few were people I might have encountered in my normal work with state governments, but at least half of them were not. I guess any midcareer program would attract diverse groups of people, but my sense is that at Michigan we were probably even more diverse, and that's a great benefit.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

I didn't expect to change my career from the work I had been doing and I still don't. The one area substantively that I find very interesting was opened up to me during our research and methods class. I now know that if I were 25 again and starting a career I would be very suited for an applied methods career where I did evaluations. It is my hope that once I finish the degree to look around for those kinds of opportunities and see if I can do any of that kind of work.

In a way the value may not be measurable. The program has given me more self-confidence in a few areas, and it has made me more aware of the diverse sets of perspectives that I should bring to the policy work that I do. It's made me a better analyst, but it might be difficult to measure. Peo-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ple who I work with have commented on changes in my methodological approach. They have been pleased. There are opportunities for me to integrate what I've learned into my work.

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

Answered above.

6. If you did not complete the program, do you plan to? If yes, why? If no, why?

Because of my age and because of my existing experience I don't happen to think that the degree is that important. But, I do plan to get it because I've come this far and because I feel that it is an indication of successfully completing something. The dissertation is a wholly different experience making me understand just how laborious the research experience is. I'm doing something with original data, and I've been at it for a year and a half. Prior to this I had no true appreciation of what it was like to start from an idea and think about data sources and take it all the way. That understanding of the research project is so valuable to me because I find it interesting. However, I'm pretty sure I won't do anything at quite that level again. Our degree program is not training us to be researchers per se; it's training us to be policy analysts, but we'll need to be able to know how to use secondary data, etc. (Patricia Butler finished her dissertation in February 1996 and graduated in May 1996 with her DrPH.).

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Doctoral Alumna Pamela Paul-Shaheen Thursday, August 31, 1995, 9 a.m.

1a. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What are the most important contributions?

In the aggregate, in terms of the totality of the program, the jury is still out. But clearly the program has put a number of people into key policy positions. You can see the success of the program as you look across the spectrum of alumni and the roles that they are playing across the nation, regardless of whether they completed the program totally or not; many have moved into reasonably influential positions. I think what will happen over the course of the next 10 years is that people participating in this program will become part of the core policy network for states and the federal initiative that probably will be taken (post-Republicanism). In that sense if the intent of the program was to train a group of gifted people and allow them an opportunity to develop and move into positions that have key influence over policy, it has accomplished that.

Clearly, for the individuals involved, Pew has created both learning opportunities and incredible networking opportunities. I use many of the people in the Pew network continually, and I'm always amazed as I go through my professional life the people I run into and find out we share the common framework of having gone through the Pew program.

Creating a group of thoughtful individuals who can influence policy has been one of its most important contributions. The proof of that will be even more noticeable in the next 20 years, because, for the most part, its people in their 40s, 50s, and 60s who end up in policy-elite positions, and clearly, if you start with people in their 20s, 30s, and 40s you've got to give them time to move through the system.

1b. What is the Pew ''legacy'' in term of:

  1. health policy?

    The program has successfully put people in the field who have made a contribution to the literature, who have moved into positions of administration, and who have created a more thoughtful environment in terms of decision making. Pew has, at least, expanded the level of talent that's available with good credentials. Look at someone like Larry Patton who has been through the program and is now one

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

of the key advisers to AHCPR (Agency for Health Care Policy and Research). There is a sprinkling of Pew alumni all over in research consortiums, running hospitals, in consulting firms, etc. Can I tell you directly what it's done, like has it resulted in the expansion of health care or reduced costs? No. I don't think the "legacy" diffuses that way. It diffuses through the individuals who were involved.

  1. education?

    It was a wonderful educational experience. My program was unique, and I would not have even been a participant had it not been for the fact that I could move into an on-job/on-campus (OJ/OC) program. I am a very strong advocate for the approach that Michigan took, because it offered people who were in high-visibility positions the opportunity to complete the program while continuing to manage professional work schedules and personal demands. It would never have been of interest to me to resign my position to go to school because the program would not have benefited me financially. It rounded out my education and gave me a credential I wanted, but I didn't go into the program because I knew it was going to move me up the ladder an additional $20,000 a year in income. From that perspective I feel that the educational experience was extremely unique. I would encourage other institutions to look at the OJ/OC model. If you recruit individuals who are in mid-or upper-level positions and bring them into a university setting, you have a class with a diversity of interest, expertise, and ability—it just creates an incredibly rich and interesting learning environment. Secondly, when you combine that, as the Michigan program did, with people who are actively engaged in policy activities in their work, a symbiotic relationships occurs between problems from work and techniques learned from class, adding an interesting dynamic into the program. The program empowered these individuals in their own workplaces.

  2. your future?

    I wanted the professional credential as a way of rounding out my career. It also has given me access to a vast network of individuals who I have drawn on over and over both for work-related issues, to bounce ideas off of, and to do papers with. That part has been very good.

2. What was the most innovative or unique aspect of your program design and implementation?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

The OJ/OC approach was the most unique. Without it I never would have ended up as a Pew fellow. That's the one overriding feature. I would really like to see in today's demanding world more universities look at developing that kind of approach to attract mid-and upper-level career people back into a university setting. I think it benefits both.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

There are a couple of parallels here that are interesting. On the one hand, one of the things that contributed to the program's success at Michigan was the fact that the administration had experience in dealing with off-campus students. Therefore, the supporting infrastructure was in place, allowing the program to run smoothly. On the other hand, I was in the first Pew class, so it was a situation where we were kind of learning by doing and also they were really creating the program around learning by doing because they had never really had any of these challenges at the doctoral degree level. Since we were the first "guinea pigs" there were more barriers in terms of people not knowing how things were going to operate. There were still faculty discussions of what was going to be demanded of students and were these really doctoral students, etc. By the second and third years of the program, those things were all resolved. The only downside was that we were the first class, there was a high degree of uncertainty about how the program was to be run, and what the protocols would be, but that's just part of getting something off the ground.

4. How was the Pew approach different from the traditional teaching approach?

Most of it was around the OJ/OC approach. I didn't find anything else different. I think we were held to exactly the same standards, if not higher ones! Higher perhaps because there were people coming into the program who were very experienced in some aspect of the health field; thus, the expectations extended even higher.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

The most value has been the networking that has resulted. It has offered me an opportunity to link up, work with, and collaborate with a whole range of people across the nation. That has been invaluable.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

I did not look at the program as a way to make a major life change. I followed the same career trajectory for the last 20 years. My whole focus has been on health policy and analysis, planning, and research. That has not changed one iota. What the program has done for me is that it has given me additional credibility, some additional options for choices of work than I would have had without it, but it has not made a major change in my professional life at all. It has just augmented and supplemented it.

6. If you did not complete the program, do you plan to? If yes, why? If no, why?

Well, I got through finally! This past June I defended my dissertation and now can add the DrPH to my name! I'm done. Why did I finish? First of all because if I start something unless there are things in life that I cannot change I will finish. Also, because people made a commitment to funding my education; had I not finished I would have reneged on a commitment I made. It was important for me to complete it. Also, because I was the first woman to come on board at Michigan. I was the only woman in the class. I said come hell or high water I would finish the program. What I didn't want to do, but what I found I needed to do, was take a long time in order to balance my life. I didn't want to get a divorce or give up raising my child. I said I would finish, but it will be on my own time. I'm very proud of the job I did and how everything turned out.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Face-to-Face Interview with Stuart Altman Friday, September 1, 1995, 10 a.m.

1. What was your role in the development of the Pew Health Policy Program?

If you go back to the very beginning, one of the things that was very impressive and very unique about the original solicitation that I had never seen before and have never seen since was the fact that a small group of advisers to Pew said there are important issues that are likely to develop in health care over the next few years, rather than the foundation saying what they are. Therefore, the small group of institutions that were selected to train people were allowed to (1) indicate what they believe are going to be some of the major health care problems that this country will face over the next decade and (2) how they could design a training program to help individuals meet those needs. The foundation didn't dictate what the problems were or how to do it. I have never seen that before.

Stan Wallack and I put our heads together (I give most of the credit to Stan) and worked on developing an application. We were the little kid on the block as a school. We were up against the biggest and the best. But we really focused on what the issues were and what needed to happen. We put together a program that took individuals from the social sciences and provided a more focused training in health care.

2. How did the Pew Health Policy Program start? Who are the key people involved?

The next question was how we were going to make it work and we had several issues. One was how to divvy things up between Boston University and Brandeis. Even while there were different programs between us and Michigan and between Michigan and UCSF (University of California, San Francisco), there also was a very different orientation between Boston University and Brandeis. Our students were integrated into the traditional PhD program here at Brandeis while taking certain special courses that were provided. We required that every student meet all the requirements. Boston University (BU)created a very special program. The advantages for Boston University was that they could really focus directly in on what they thought was needed, but the disadvantage that the students felt was that it was very unstructured. The students were free to pick and choose; they were sort of left alone. They focused much more on health providers: physicians, etc. Our program was much

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

more social scientists getting more of a background in health care. BU added one dimension which I believe was the most critical component to the program's success and that was a person by the name of Steve Crane. Steve Crane really was the father/mother of the Pew program for many years both at BU and Brandeis. We worked well together. We got a lot of good credits for that. That also had to do with the good relationship I had with Dick Egdahl and other people here who did similar things as Steve but not to the extent that Steve did.

I was the dean through the whole program, so I played a reasonably important role in making sure the resources were there and that the program ran smoothly. There were some tricky spots, for instance, funding issues. There was somewhat a feeling of elitism on the part of the Pew fellows. There was some jealousy around the school that I had to deal with, but it was not overwhelming at all. I think most people, outside health, were very appreciative of having gotten into the school. But there are always, on the margin, individual students who don't get a Pew fellowship. I pretty much left the running of the program to Stan Wallack.

When Steve Crane left BU, we moved the whole program [to Brandeis University]. We were then forced to deal with the fact that we had the whole program and that Steve was gone. Mary Henderson ran it for awhile and then we were very fortunate to get Jon Chilingerian to run it. He added a different, more of an intellectual force.

3. What was the environmental need at the time of the program's inception?

Our main thrust was that we could take individuals who had been working tangentially in the health field (mental health, nursing, child health, etc.) and provide them with a more fundamental understanding of the core problems in our health care system so that they could function as a policy analyst or researcher.

We were also trying to recruit, and we did recruit people who had worked for governments either at the state or federal level, but who had started out without a lot of formal training and needed a deeper understanding.

Has that need been met by the Pew Health Policy Program?

We did attract a number of individuals who had been working in the health policy arena at the government level but who had come in as pretty junior people. We added a conceptual framework to what they were doing. They then

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

reasserted themselves back into the policy process, often at higher levels.

We also had the relationship between what we were doing in the health policy center and the work that was going on here with the Pew program. That was a very valuable synergy. It allowed the students to often participate in direct research and in a policy environment. Most of the research going on here is a very policy-related research that they could be a part of. It helped us too, it gave us the benefit of these individuals, and a lot of them stayed with us and became part of our staff for a period of time.

Will there be a future need?

The answer is definitely yes. There is always a need for individuals who have both a broad sense of what the issues are and do not come from a narrowly based discipline and yet have analytical training that allows them to think clearly and to do research. The problems in health care are truly interdisciplinary with multiple dimensions and require individuals like that. There are relatively few places in the county that provide such training.

4. What is the Pew Legacy?"

It's hard to say. I think it's demonstrated that these programs are important, that they can produce value added. I know that the most difficult aspects of evaluating a program, Pew or any other program, is determining what is the "value added"? What's the value added to the individual, and what's the value added to the system? Now, it's a little presumptuous to think or to attribute to any one program, even one that has trained a couple of hundred people, what impact it has had on a $1 trillion industry that affects every American, that employs one out of seven people; let's face it, we are dealing with a gigantic industry. We have to evaluate its impact on the margin.

Clearly, the Pew program did demonstrate the importance of interdisciplinary research. It was also terribly valuable to the Heller School [at Brandeis].

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Doctoral Alumna Sarita Bhalotra Thursday, August 31, 1995, 1:30 p.m.

1a. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What are the most important contributions?

The most important thing that the Pew program accomplished for me was the opportunity to network with a lot of important players in the health policy field, including the other Pew fellows, as well as the people that they worked with. The Pew conferences were very important; we got to meet other Pew fellows and the senior persons they worked with.

1b. What is the Pew "legacy" in terms of:

  1. health policy?

    The Brandeis program is a good program, built as it were around the Heller School. The PhD could have been a little more in depth as far as health policy is concerned. I felt that I got a lot more familiar with health policy with the work that I've done beyond what the PhD program had to offer.

  2. education?

    In terms of my education, it's been priceless. Frankly, without the Pew stipend I wouldn't have entered this program. It was just enough to get me to stop working. It provided the catalyst.

  3. your future?

    I think it has changed my direction in terms of health services and health policy research. When I entered the program I didn't think it would have had that effect. I thought it would be a stepping stone for health care settings, which I am not ruling out right now, but at this point at least I am more interested in research.

2. What was the most innovative or unique aspect of your program design and implementation?

Jon Chilingerian's seminar, which he implemented the second year that I was a Pew fellow, was not just innovative and unique but was also very influential. It filled a spot that needed to be filled. I'm sorry that we did not have it that

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

first year. He introduced us to different theoretical ways of looking at things and to readings that were important regarding health policy and other more general issues.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

I think the dissertation seminars were good. They have a lot of potential. I think that the way that Jon had structured it was something that I really liked. He provided materials for us to discuss. He provided an opportunity for us to present our material. It was structured just enough to lend a good framework. It was unstructured just enough that students had an opportunity to share, develop, and exchange ideas. When it was too unstructured, frankly it was a waste of time. Sometimes it just became an opportunity for one student to speak his or her mind or his or her pet peeve. That was not a useful exercise. Making it too structured is also too restrictive because it is different from a class.

4. How was the Pew approach different from the traditional teaching approach?

The only thing different about the Pew program was that we did have the dissertation seminar. The first year it was a little scattered and that was partly circumstantial. Then there was a change of guard: Steve Crane left, Mary Henderson left, and there really wasn't anyone to pick up the slack. So, it was a little fragmented. However, the second year was a little more concrete, not just Jon's seminar but even at [Boston University].

5a. How has your professional life changed as a result of the Pew program?

The PhD, this particular PhD program, and the way that the Pew program was structured in a way that I was doing projects that seemed to involve more of the senior people. I'm not sure how it came about, but I think my professional life has changed direction because of the Pew program. I have gotten more interested in health policy and health services research than I was before.

What value has Pew training added to your life?

There really wasn't a separate entity called Pew training that so much influenced me; rather it was the whole PhD program that influenced me. Overall, the program enriched my knowledge.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

I was working in a hospital, so I would have expected that I would have continued to work in health care organizations. I am not ruling that out in the future, but at this point it seems like I'll pretty much stay in health services research for the foreseeable future.

6. If you did not complete the program, do you plan to? If yes, why? If no, why?

I am ABD (all but dissertation). I haven't finished my PhD but yes, I plan to finish it. I got too caught up in working here. The dissertation was put on the back burner. It became so much more interesting to do these research projects and be part of a research team. Thus, the dissertation kept receding further into the background. However, I do intend to finish it.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Stan Wallack Wednesday, August 23, 1995, 10 a.m.

1a. Based on your experience and familiarity with the fellows and the programs, what did we really accomplish? What were the most important contributions?

If we looked at this from the perspective of individual programs, the answers would be very different; after all, they are all very different programs. The orientation, philosophy, and politics at Brandeis are very different from those at the other schools. Michigan was very concerned with public health and methodological issues in terms of evaluating different programs. San Francisco had a very different focus because they were training postdocs who already had good theoretical skills in research. Also, each institution had a very different leadership, with very different people with very different interests. The individuals played very important roles. Looking at each institution individually gives only small parts of the program. It is very important to look at all the programs collectively and say that this was more than any one individual program; it was a set of programs that did different things. The institutions very much dictated how you looked at your program. If you were coming from a medical school, you were looking at the focus from a delivery perspective. Schools of public health look at things from a methodological perspective. We looked at things from policy perspectives. But we were all looking at solutions to problems rather than analyzing problems through our varied focuses.

Most traditional PhD programs are teaching methods and theory. Schools of public health teach a lot of statistics, etc. Health is a problem area. Health policy programs are different in the sense that they focus on the problems trying to find solutions. It is a different niche. The Pew program did do something different in developing people who were in that niche in the 1980s when we all focused on problems and their solutions. It was really appropriate for the time.

1b. What is the Pew ''legacy'' in terms of:

  1. health policy?

    For all these programs the legacy is the people. We have gotten some really committed, interesting people into the program. That's the nice thing about these fellowship programs: people stay around for a long time, and they have real careers and real interests. The issues will change, and the

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

policies will change, certainly they have changed in the last few years with the rapid changes in health care. So, it won't be as much the content as it is the people.

  1. education (doctoral, postdoctoral, or midcareer programs)?

    Each institution probably got very different things out of it. The Michigan model was a very innovative model: to try to take practitioners who are out there doing it and give them skills. I think we are going to find more and more that it is a way of educating people through other kinds of media, probably. It's just the beginning of a way of educating people on the job in effective ways. It will continue to be very exciting into the future. I think in terms of Brandeis, Pew has allowed us to have depth. It has and will continue to affect us on the institutional level. Each institution will have had its own impacts on its individual programs and faculty.

  2. your institution?

    It certainly changed us at Brandeis. We had a very broad curriculum that had a lot of contexts, historical contexts and values, and looked at the vulnerable populations and how they were doing. What it did not have was a deep and sound policy framework. It did not look at public-and private-sector policies that make outcomes possible. So, we have added a real depth and focus on policy, and I think we are still building, but we have filled a missing link in the school. We went from social welfare to social policy.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

We did not have a training program. We didn't start off trying to support something in existence. We were invited to participate and asked, "What would you do with a health policy training program in the future?" We saw a world with a delivery system and a passive payment system driven by the providers. We saw a lot of frustration, particularly with costs going up. We thought there was a need to get payers involved, to have a better balance between payers and providers, so we said we should work with payers, and the initial proposal defined those payers as business groups within the community. And then we asked, "How do we get cities involved?" That was a forerunner to getting many cities involved. We were involved with Denver and Cleveland. We developed very strong coalitions, I think we had something to do with those. We had a major symposium for the major

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

corporate sponsors at Boston University (BU). Business firms could become better purchasers of health care managers. We really had a public policy perspective, in that the financiers in states and governments needed to get more involved in managing and running the system, and clearly, over the past 15 years we have had the government move to being a more aggressive payer and policy maker. So, really it was an attempt to get the payers into the equation of health care. We saw a number of different ways of doing it. One way was to get people into Brandeis and educate them about payment and the effects of payment on cost and the delivery system.

We now have courses and a faculty. We know what we're doing. We have a much better handle on how to develop productive people who are going to be problem solvers. We have already institutionalized the curriculum here and the processes.

3. What was the need in the health policy community when your program started, and how have those needs changed today? Is the job done?

We started off in the 1980s with this notion of payers. Just when the private sector began getting involved, together with the HMOs (health maintenance organizations), moving the delivery system. It was early into the era of managed care. We did a lot of projects in this area. Clearly, this has changed over the last 10 to 12 years. When the program switched from BU and Brandeis to Brandeis alone, we became more concentrated on the vulnerable populations that we are most concerned with at Brandeis. We integrated more with the Heller School. We started to be more focused upon disenfranchised populations. We accepted students who were more interested in broad social change. These people did OK, but we probably didn't do as good a job as we should have in terms of focusing them on the health care of the vulnerable populations. Some of our students did, however. We focused on women, poor kids, and inner-city dwellers. However, we may have lost some of the focus that we had had initially as we moved from health care cost and utilization to vulnerable populations.

One particular area in which I believe we can play a major role in the future is organizational behavior and organizational change. Policy research in this area is just starting. We don't know who will be successful and who will not be successful. We don't know how well disabled people are going to perform in these systems. We've gone from a fragmented delivery system to a horizontally and vertically inte-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

grated system. It's interesting, but what are the implications of all that? I think policy makers don't understand how they have affected this change and I don't think the people running organizations understand how they affect policy and how policy affects them. I think the link between policy and organizations is really key. We have to figure out how to regulate organizations as they continue to grow larger.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

Both the weakness and the strength was being in the Heller School. The health policy curriculum is multidimensional in terms of political science, economics, and sociology. We tried to build in a specialization in health after that, and in that we tried to put different lenses on it: lenses of the economist, the political scientist, the sociologist, and the medical system. We looked at policy from different paradigms and different perspectives. I think that is critical if you want to do "health policy."

5a. What was the most innovative or unique aspect of:

  1. your program design?

    Our initial program was very innovative in terms of bringing in business leaders from the community and really working with them and working with their problems and also working with communities.

  2. methods of implementation?

  3. educational process?

    Educationally, the most innovative was the building of a series of policy courses from different perspectives. Health policy is about problems and solutions. How do you get to the solutions? How do you look at a problem? How do you solve it? It is a very doable kind of thing, driving home the need to link a solution with a problem and its analysis. The dissertations therefore have this mix of looking at a problem from a multidisciplinary perspective, although we always stress having a theoretical perspective as well.

5b. What were the biggest challenges or barriers to overcome of:

  1. your program design?

  2. methods of implementation?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
  1. educational process?

    I think the biggest challenges stemmed from the fact that the Heller School curriculum was so broad. It tried to cover social welfare, different methods, and different theoretical disciplines. How do you then add a specialization on top of all that? It would be a lot easier to just be a health research or health policy school rather then a school of social welfare. The educational curriculum was driving our program, and yet we had students who needed certain skills and needed to take certain courses, and yet there were many other courses to take under our program. So, placing the Pew program within the Heller School's curriculum was a very big challenge.

6. Based on your experience, what lessons were learned about the educational process in terms of:

  1. recruitment?

    One of the reasons that we wanted to do this program is because we saw it as an opportunity to attract students because of the stipend level and the national reputation that Brandeis would now have. We were competing against top schools: Michigan, RAND, California, the Massachusetts Institute of Technology, and Harvard. All had applied. That we were one of the schools that won gave us an opportunity to attract a different quality of student with a different set of interests. That was very important. Having said that and given that we were doing an awful lot in policy research that was being recognized, it still did not mean that the best students naturally came to Brandeis. Harvard still attracts very good students; it wouldn't matter if it had nobody in policy, because it's Harvard. We found ourselves still needing to get out there and do a lot of recruitment. We had one person here at the program who did a great job at recruitment. The program really did very well when he was here. Steve Crane really took this program on as his mission. He was a key force. He tried to work with both institutions (BU and Brandeis), and he took a real interest in getting out there and selling the program. You need a champion. Steve Crane was our champion in terms of recruitment, but I think you need someone who can link up recruitment and content. I think we found ourselves both here and at BU more interested in policy. Steve was more interested in organizations. He may have been right. There was a little conflict there. But the success of this program had a lot to do with Steve running

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

this program. Going out there, really identifying people, working hard, writing to people, and going to the right kinds of meetings were very important for us. I don't know about the other institutions, but we certainly found that we got many more applicants and much higher-quality applicants. That was a big positive for us.

  1. degree requirements?

    I talked a little bit about the breadth of the program. I really think we are trying to get people prepared to have very good methodological and statistical tools because of the quantitative nature of policy work today and to have a solid theoretical background. It is very difficult to do it in three different paradigms. I think one of the things that we have learned along the way is that although people need to be exposed to many disciplines, it is important that to understand that they still need a specialization. It's important to understand, yet they really do have to get this notion of specialization. It's important to have one discipline you can really think in terms of so, regardless of the problem, you have a way of thinking about it. You don't just look at a problem and try to find a solution; you really need to have a theoretical paradigm driving you to understand the problem. That would be something that I think is very important in a PhD program. I think the people at San Francisco brought in people who already had PhDs in disciplines, and I think that is important for the kinds of research and publications we need to do. We need to do a lot in a couple of years. One of the things we did was try to do everything in 2 years. I think for learning just a policy framework that would be fine. But if you want to have your trainees do policy writing and research, you need to give them a stronger base in one or more of the disciplines. We train people for the policy world. But we probably train them better as users than doers. We needed more users out there because the world was changing, but now we need more doers with greater theoretical training.

  2. curriculum and content?

    The other thing that was key in terms of our 2-year curriculum and our very high success rate was the focus on problem solving and learning how to get to an answer. The seminar led by Jon Chilingerian was very critical. We've always had very strong cohorts. Jon got them to see studies of successes. How does someone who is in political science

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

move ahead and complete a project? How does an economist go ahead and do it? Our educational process or approach was pragmatic in that we used the procedures any business or person would adopt in solving a problem. Being exposed to this process from the beginning of the program to its end is very important. It is very difficult, but very important for students to see at the outset how they get from where they are starting to the end. This road map became a backdrop for students—one they could put their hooks into as they moved along.

  1. integration of fellows with other students, the rest of the university, and the program in general?

    I think we had a problem initially because the Pew students identified with the BU Pew students only. This problem became different once all the Pew students came over to Brandeis. I think there was a little bickering about the stipend. I think the Pew students got a little more; a lot of government stipends are about $10,000, and they were getting $12,000. But in those years they were the best students we were getting. I think some of them found the work less challenging than other students in the school, but I think over time, as we developed our own program, that concern dwindled. Also, we opened the Pew program to all the Heller School students by not making our new courses restricted to Pew fellows. We started off with just having Pew fellows in the courses, but we realized that while the Pew students were very good, there were some other students who were very good who didn't get a Pew stipend. So we broadened these classes, and I think that by changing the educational process we helped to break down those barriers that we had in effect created by saying it was such a special program. So, the Pew program came in and we developed a new curriculum. When we opened up the program it integrated students. Currently, the seminars we have are open to everybody.

  2. relationship between faculty and students?

    It's probably a little closer for those in the Institute for Health Policy. The institute has housed the Pew program, and therefore, we have really developed strong relationships. To the extent that students wanted to identify with one or more researchers, it has worked out quite well. When Pew fellows went outside the institute faculty, some developed strong relationships as well. Debra Stone is a good example

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

of someone outside the institute faculty but with whom some of our students developed a relationship, and they kept working with her, particularly the lawyers we've had in this program. Students basically found their way. The institute has some excellent economists, political scientists, and policy analysts, but outside the institute there are other interesting faculty with whom the students may choose to work.

  1. completion rates (where applicable)?

    I think our high success rate is pretty amazing given the brief time that people are here at Brandeis. They have done very well. We have had them start thinking about their dissertations right away so as not to make it an overwhelming task. They try to spend that first summer developing a topic. These people are highly motivated. We have been successful because by the time the students leave here they are pretty far along knowing what they're doing. We can help students choose doable studies and make sure their approach has both a sound basis and a sound method so that they can get it done. Doctoral students can get lost in a place like this or in any place. Being in a training program ensures that they don't get lost. The greater intensity of the Pew program probably helped as well. The focus, the intensity, and the caring all worked together.

7a. To what extent do you think there were "programmatic" barriers to student completion?

Not having student support for that third year was a barrier. I think some people needed it. I think we assumed going in that people came knowing a lot about health care, some came with backgrounds in various disciplines, and some had very strong analytical skills. To the extent that someone comes here with no analytical skills and hasn't done quantitative work can be a real problem. I guess they could go off and do a qualitative thesis, but they really should have a quantitative understanding both to get through the program and to move on in their careers. So, the time they may have needed to make up for their deficiencies may be considered a programmatic barrier. We really were not prepared for people who did not have some sound quantitative and analytical skills.

Another thing, related to recruitment, is that there were individuals out there who, when we first started the program, were ideally suited. They were seasoned people who were working in the real world for a long time. They

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

didn't want to spend a lot of years in graduate school. I certainly didn't when I went to graduate school. Enough is enough. Ready to get out because the work is to be done afterward. They just wanted to pick up some tools. The 2-year program was appealing to some students, even if they struggled a little when they got here. But it is unrealistic to expect students to have completed all requirements in 2 years. We expect them to be done with their course work in 2 years and to have a dissertation subject and be pretty well along when they leave. Then they could be expected to finish their dissertation within the third year. We wanted to make it a very fast-paced program that would distinguish us. And some students have come pretty close to completing everything in 2 years. But then looking back to those who have, I think they paid a price. They had such a drive to get out of here that they didn't do enough reflection. Where they had a weakness, they skirted around it rather than dealing with it.

7b. To what extent do you think the program was beneficial for those who did not finish the program?

Either way the program is pretty important. After all, we are getting people to take a problem and solve it. Still, getting through the entire process is important. The courses themselves help, but I think doing a dissertation really forces people to take a problem and analyze it.

7c. How can we measure success for those programs where completion rates do not apply (i.e. postdoctoral programs)?

You can look at the positions people are holding, what roles they're filling, what they are writing. There are a lot of different measures; probably in the postdoc programs they were trying to get people in health and redirect them out of the traditional academic fields. Therefore, the areas they are working in, the problems they are working on, and the positions they are holding would be key indicators. They had a real emphasis on producing research and writing articles. That is an important part of postdoc programs: getting people to hone their skills and start writing articles.

8. How does the Pew fellowship approach differ from a traditional fellowship approach? How have the major outcomes differed?

It's the policy focus. It's the solution focus. Students are focused on critiquing and understanding an issue. We are really focused on solving problems. We are focused on look

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ing at a problem and figuring out how to solve it. We go beyond teaching understanding; we evaluate and teach how to formulate solutions. And the compromises one has to make are very different. We are not looking for the ideal solution; we are looking to make things better.

Major outcomes? The people we put out. The places they've gone, the things they've done. How they are taking on and dealing with real-world problems. I think the real world is finished analyzing the health care problem; we have pretty much analyzed the health care problem as much as we can. Now the world is looking for solutions. Learning by doing. I think our people probably find themselves in a lot of those kinds of positions. Hopefully, they will be able to contribute.

9. If you were asked to give advice to another university attempting to initiate a similar program, what would you say?

So much of what was successful in this program, in terms of a policy curriculum, was really responding to what was needed in the last 10 to 15 years. I don't think starting that same program today would work. It would have to look different. But, let's say you wanted to put together a new, innovative program within a university. Critical mass is important. You have to really identify with a program. You have to have real expectations. You have to have faculty who are committed to it. I think you really need someone who spends a lot of time doing it. One of the problems that we have here is that once Steve left we had people committed, but we didn't have anybody whose major focus was the program. A lot of us teach in the program but have many other things going on. The other thing is that you need time to work out what the right courses are and what the right procedures are. Another thing is that if you are training people to do policy in the real world, you must be flexible.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Marion Ein Lewin Tuesday, November 7, 1995, 10 a.m.

1a. Explain the role of the Institute of Medicine (IOM) in administering the Pew Health Policy Fellowship Programs.

This program was brought to the IOM from the American Enterprise Institute (AEI) (1987). The interesting footnote about that move is that when the Pew Health Policy Program was first thought about at the Pew Charitable Trusts and they were looking for an organization to direct it, IOM had responded to the proposal. IOM didn't win the competition; AEI won. IOM at a very early stage had an interest in this program. Until 1988, IOM was not even called officially the program office because most of the national programs until that time were directed out of Philadelphia. However, at the recommendation of Bill Richardson, after he did an evaluation of the program prior to the last refunding, he recommended that IOM officially be the program office.

Since the beginning, the role of our office has been to develop the joint activities of all the programs: to plan the annual meetings, to plan the meetings each year for new fellows, to develop the annual directory and the semiannual newsletters, and to be responsible for the network and really be responsible for the family of Pew fellows over and above their connection to the individual programs and institutions. Also, aside from the whole idea of the networking and the overarching activities, another role that we've had all along was to provide the interface of the programs to the Washington health policy scene, and the meetings each year for new fellows were specially directed to acquainting fellows with what's happening in Washington. Aside from that, we have a role in monitoring the budgets and making sure that all the programs are performing according to the requirements of the program.

1b. How did program variation between sites affect your administration?

I think one of the really wonderful aspects of the program has been the learning curve at every stage. Another important aspect was the evolution of four different programs and the IOM program thinking of themselves mostly as individual fiefdoms all a little bit in competition with each other to a point where everyone now is working toward a common goal. The program has many more synergisms and mutual interests than issues of conflict and divisiveness. In

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

the beginning everyone was trying to understand what the program was all about, what kind of training program they wanted to pursue, what kind of fellows they wanted to have, etc. So, everybody was very much involved in deciding what they really wanted their programs to represent. In the beginning, when I was still at AEI, we very often felt like we were the umpires or facilitators because the programs were all looking over each other's shoulders and being a little judgmental about other programs. RAND thought that they were much more substantive and analytic than the Brandeis program, and the Michigan people felt that they were much more real world than the people at UCSF or RAND and thought the Brandeis people were touchy-feely community organizer types. So, in the first few years we spent a lot of time just promoting a mutual understanding and linkages across the programs, and I think what started off as a challenge became really very much a reason for the success of this program. The programs now have common objectives, and there is much more commonality among the fellows than there was in the beginning. They are all working toward mutual goals. From my perspective, Pew fellows are not differentiated by where they come from; I am just concerned with making sure that the network continues, that the interface with Washington continues, and that the legacy and the next generation of this program are given every opportunity to succeed.

2a. Based on your experience and familiarity with the fellows and the programs, what did the Pew Health Policy Fellowship Program accomplish?

Above all I think the fellowships enriched in a remarkable and significant way the field of health policy research both in the quality and in the level of people that they were able to attract. This program came along at a time when health policy research was becoming increasingly recognized as an important discipline, and health care programs and budgets were becoming such a significant aspect of public, federal, state, community, and private-sector budgets. Until the Pew Health Policy Program came along there were not many people who were good at policy but who also had the analytic skills to really understand what these dynamics were all about, how to develop a changed agenda. My feeling is that this program really enriched the field. To the degree that it was multidisciplinary and to the degree that there were individual programs, these were also very significant and valuable accomplishments. Even though everyone who was a Pew Health Policy Program fellow has a grounding in

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

the statistics and the analytic skills and a knowledge base in policy research, they all have different expertise to bear on an issue, and so I think that was certainly another wonderful idea that the originators of this program may not have even really thought about. What resulted was that each program has specific expertise, but they all complement one another. The feeling is that the Pew Health Policy Program really enriched the field of health policy research and informed the debate. When you look through the literature at how the Pew fellows are represented, in what areas, and with what activities they've been active, it reflects the world of health policy and health care reform in the last dozen years.

2b. What are the most important contributions? What is the Pew ''legacy?''

The legacy is really even more valuable today than it was several years ago, because funding for these types of activities are going to be ever scarcer, and even the recognition of solid, objective, nonpartisan health policy research is going to be more difficult to come by. The legacy is that these programs have developed the infrastructure, they have developed the institutional memory, and they have developed the potential mentors who are not only the incredible people who led the various programs but also all the alumni. Now all these things are in place, and these people are at the peaks of their careers. The state of health policy research, I think, is not as endangered today, even given the budget constraints and political constraints. These are all issues we need to be concerned about; however, our people are off and running and are ready to do the job. We are much better prepared; it's like saving for a rainy day. We're better prepared to face a future where the funding of these efforts or the capacity building may not be as readily available.

3. What was the need in the health policy community when the programs began, and how have those needs changed today? Is the job done?

When you look in the original brochures that announced the program there were two issues that were emphasized; one was that health care was becoming ever more complex, ever more prominent in the social, physical, and economic fabric of America and did we really have the leadership and the caliber of people who could steer this ever larger and significant shift? One major purpose was to enhance and improve the state of the art in health policy research. Also important was developing a multidisciplinary program and looking at different fields that affect health policy issues: economics, political science,

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

and sociology, not to mention research methods and all the analytic tools. This program was dedicated to training people who could look at health care in this multidisciplinary way, which was very important.

Have those needs changed today? I don't think so, not at all. They may be even more important. When you look at the demographics and social fabric of this country, when you look at the determinants of health, we realize now that health care encompasses so much more than just purely medical care or health care services. The need is there and may even be greater.

Is the job done? Clearly, it is not done. Health care now is moving from the federal level to the states, the private sector, and to local communities, representing an even bigger challenge because these people, to a more significant degree, haven't had the resources to build that infrastructure of expertise. I don't think the job is done, but what I think the Pew program has done, as I said before, is that it has developed an infrastructure for this type of training. It has people who are now in the field who can be applied to these new challenges and who are extremely well trained and can serve as mentors to people who follow after them.

4. Based on your experience, what lessons were learned about the educational process in terms of:

a) recruitment?

There has been tremendous improvements in terms of thinking about the kinds of people you want to recruit to these programs, the kind of people who could succeed in these kinds of programs. It couldn't be only the desire to get a doctoral degree or a postdoctoral degree and an interest in health policy. You really needed people who had the motivation, organizational skills, and discipline to do this particular kind of fast-track program. Everyone realized after the first few years that this was an exceedingly demanding task to do these fast-track programs. In the beginning, at schools like Michigan, more than half of the people they had recruited were not able to complete the course of studies, not because these people weren't smart, but because it took a certain type of person to really succeed in this kind of program.

b) curriculum and content?

Even though the schools had different emphases (i.e., RAND was more analytic; Brandeis was more involved with community change agents and looking at the broader issues

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

like public health and environmental health; UCSF became a postdoc program; and Michigan is a weekend program where people stayed in their jobs), all the programs realized that everyone needed to have a confident knowledge of basic skills of analysis, statistics, evaluation, etc. All of these skills are so important if you're going to stay in this field. For some people, the technical aspects of the health policy research was going to be the mainstay of their program, but even at schools where that was not the mainstay, people realized that the fellows still needed to have rigorous training in basic skills development. Also, the schools realized that the people who were in these programs were very much experts in and of themselves. These were people who were not the traditional students who came in with a blank slate. In many respects these people could teach the course. So, over the years there was a much greater respect and recognition that you had to develop a curriculum that was responsive to people who knew a lot about various aspects of this field and tailor some of the curriculum so that it would still be challenging for them and to make it really nonacademic but rather relevant to today's issues and today's policies. That was another uniform advancement of the curriculum and the content. The programs contributed a great deal in their recognition that you wanted to pay attention to what was important to people who were going to be on the front lines as decision makers and that you also wanted to pay some attention to how to write for decision makers, the politics of research, how to communicate research, and how to interact with politicians. How do you marry the worlds of health policy and health services research with decision makers who have very different requirements and timelines? So it was not only being very aware of the quality content of the curriculum but also being very much aware of how this content needed to be applicable to people who were not going to wind up in an ivory tower but who were going to be on the front lines of change.

c) integration of fellows with other students, the rest of the university and the program in general?

When I did my first site visits to the schools early on, all the schools (with the exception maybe of UCSF) were struggling about how they were going to be accepted in the larger university. In the beginning these programs were looked at with a lot of mixed feelings and suspicions and almost like this was a second-class degree, questioning this method of training academics. So, another very large challenge for all

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

these programs was that they had to prove themselves, and they were in many ways the pioneers of making health policy research more applicable and more relevant to the health policy debate. The schools felt very confident that this was needed, and the Pew Health Policy Program funders felt very confident that this was needed, but it was an uphill battle: the blending of the old world and the new world. Because this program was multidisciplinary there was the question of whether the fellows would gain any thorough expertise in any one discipline as one does when pursuing a traditional PhD. That's not what this program required, and I don't think anyone would change that, but in the university this was sacrosanct and something that raised a lot of skeptical eyebrows. So, I give the programs a lot of credit for holding together all the aspects of putting together and developing the content and curriculum, working on recruitment, and all the while fighting this other larger battle about how to gain respect. It's like primary care physicians in an academic health center. So, they were the vanguard of change, and I think that's always difficult.

One of the immediate proofs of success of this program is how these programs now are valued at their universities, how they are now an established part of their universities, and how the fellows and the alumni of these programs are every bit as prominent as other more traditional alumni of these universities.

Another interesting issue for me was, in the beginning, when we used to have these annual meetings, that no one ever thought of themselves as Pew fellows. People thought, "I am getting a degree at the University of Michigan or at Brandeis or UCSF or at UCLA." Everyone at these different programs was so intense and wrapped up at developing and establishing the individual programs at the individual universities that they looked at the Pew Charitable Trusts as only a financier and that's all we have to do with them. I remember what was for me one of the low moments but also one of the high moments of this program was when we had our annual meeting in Texas (this was the first annual meeting of the Pew program that I ran out of IOM). At that time there were a lot of fellows who were community activists and who wanted to be change agents but who were change agents in a way that said, "We have our values, we have our agenda, we know what we want to accomplish, and we don't care what the rest of the world says. We don't have to communicate with the rest of the world." I remember the year before, Reagan had just won his first election and we had a reception

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

for the fellows where we were lucky to get the new hierarchy at the U.S. Department of Health and Human Services, the Undersecretary, the Assistant Secretary for Planning and Evaluation, etc., and the Pew fellows refused to even talk to these people. The fellows said, "We don't understand their policies, we don't agree with them, we don't need to talk to them." I remember that I felt that we really had our job cut out for us. If you're going to be a change agent, I think the first lesson is that you have to able to dialogue with the people who are in power, even if you disagree with them. If you're just going to wall yourself up as the strong opposition and you're not going to understand where these other people are coming from, you're never going to be an effective change agent. And so at the next meeting at Houston we really carried that out. Some of the fellows didn't like some of the panelists because they didn't share their views, and I remember that some of the program directors felt that it was really time to call the fellows together to talk about this hostility that was being expressed either explicitly or implicitly through sign language, etc. Leon Wyszewianski from the Michigan program got up and said that the purpose of being a Pew fellow is that you gain an understanding about how to be an effective change agent in the world of health policy and that the purpose of the Pew Health Policy Fellowship Program is for people to become good communicators and to work in the real world, not in some ideological world. Several people got up and said, what's a Pew fellow? They said all they knew was that these people paid their checks so that they could get an education. It was really an epiphany for this program, because for the first time we had a very interesting discussion about what it meant to be Pew Health Policy Program fellow; that there is something over and above people paying for you to get your degree, and that the program had a purpose and wanted people to be able to work effectively in the real world. You couldn't work effectively in the world of health policy unless you were willing to listen to people who had other perspectives, and only after listening could you respond in a professional, thoughtful way. There were certain people who were so key in trying to imbue that culture: Steve Crane at Brandeis/Boston was very interesting, as were Leon Wyszewianski at Michigan and Phil Lee and Hal Luft (later on). People really took this upon themselves as something that was very important to do. and it was really only after that meeting that there was a new identity of what it was to be a Pew fellow and a much greater tolerance. The interesting thing is that the recruit-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ment process has changed. We no longer get people who just want to be ideologues. We get people who certainly have strong passions, strong feelings, and strong beliefs, but who also want to think about some of these things rationally.

The whole integration among the fellows across programs and integration with the mission of the Pew Health Policy Program were other very important contributions, as was evolution, which I think is really to be applauded. The credit for this goes to alot of the people at the various universities. They took this challenge of "what is a Pew fellow?" very seriously and wanted to develop a cadre of people who had some important commonalities.

When the program office was at AEI and I was brought in to cool some of those waters, early on there was a lot of hostility. If you looked at the fellows across the programs, you would have to describe them as liberal people who thought more expansively and positively about the role of the federal government, and the program office was run out of what was then perceived as a very conservative think tank. The marriage was not a great marriage, even though people like Jack Meyer were very thoughtful and made enormous contributions. Still, there was this gut reaction. In retrospect that hampered the feeling of commonality with the program office. It was almost a little bit of them versus us. I was there in the early years; I came aboard a year after the program started at AEI. I remember that time after time we would ask the programs to give us topics and ideas for the annual meetings and nobody would respond. Finally, we developed our own agenda, and of course people didn't like it. But that was all part of the growing process. People were so involved in the beginning. Very few people were completing the program. The programs were just getting started, so that people didn't think of themselves as a "family" of Pew fellows. They just thought of themselves very much more as individuals and one school as opposed to another school.

d) relationship between faculty and students?

Although my knowledge is secondhand, my feeling all along has been that one of the stellar jewels of this program has been the quality of the mentors. The Phil Lees, the Stuart Altmans, the Steve Cranes, the Hal Lufts, the people at Michigan, and the people at RAND really were terrific role models and mentors. I don't think you could have selected a lot of other universities where you would have had people who were so nationally recognized and still willing to be role models and mentors. The faculty was one of the aspects of

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

this program that was responsible for so much of its success. And, even though students would sometimes gripe that the faculty were never around or didn't have time for them, others would say if Phil Lee spent 3 minutes with you it was worth 3 hours of someone else's time. And even for these people to tell the students to call up someone and tell them I sent you, that was already an advantage.

The universities where the programs were based were already the leaders in the field of health policy. I thought about whether we would have had more "value added" if we started this program at schools that were still a little bit behind but with some resources that could get them into the first rank. I think it is a shame that we were never able to get some funding to expand these programs to schools (e.g., the LBJ School) that are almost right up there but not quite there. The Pew program and its financial support maybe would have allowed them to have entered this exalted field of top health policy research training institutions. But, I do believe that the quality of the mentors and the fact that they were national leaders was a great bone to the students. You wouldn't be a student unless you had some gripes, and they felt that this was a very demanding program. I've heard students say that sometimes faculty were not that readily available and also that in a program like this what you really need are faculty who really are both facilitators and mentors to the students. But that doesn't get you tenure. It doesn't get you published in the literature. It's not the kind of things that are awarded in an academic institution and my feeling is that the schools were lucky that they always had someone who, just because this was important to them, played that role. But in these kinds of training programs some of the things that are most important on the part of faculty are not awarded in the general structure of academia. Generally, however, the students have really respected the faculty, and both have been willing to learn.

In the beginning, when I went out to Michigan for a site visit, it was clear that the faculty were initially resentful: who are these fellows; are they going to measure up to the other people we have at this school; do I want to teach Friday nights or Saturday morning or Sunday morning; do I want to change my lecture to fit these real-world people? There were these tensions, but what is remarkable is that both sides were able and really wanted, in the end, to roll up their sleeves and do the best that they could do. There was very much an interest and commitment by both the students and the faculty to try to accommodate one another, and in

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

general people are pretty happy with those relationships. There may be some more specific stories either on the pro or con side that I don't know about.

e) completion rates (where applicable)?

We were always concerned about completion rates in the beginning. People were distressed that people didn't complete the program in the time that was originally estimated. I think people were overoptimistic in terms of the ability of the fellows to complete this kind of program considering that students had families and outside responsibilities, etc. But it has improved, and it is all very much a part of the improvement in recruitment and the improvement in curriculum and content. For example, in the beginning at Brandeis and Michigan there was a recognition that you have to start people in a very productive way looking at the degree requirements. So, as Jon Chilingerian indicated, the completion rates are admirable and compare very favorably to the completion rates for other training programs.

5. What are the critical success factors?

The schools learned to recruit people who had the initiative, discipline, and interest to pursue this kind of program and who had what it would take to be a successful Pew fellow once they completed the program. My feeling is that the recruitment really was very much improved to ensure the success of the alumni.

There was a tremendous learning curve. The people from various schools devoted a lot of time and effort to changing the curriculum and the content to make it relevant, to make it constructive, and to make it so that when people finished this course of study they were very much prepared to take on very senior responsibilities in the world of health policy research or health policy administration.

I think the mentorship, hopefully from the program office, but certainly from the individual schools, was a very important component.

I think, not blowing the horn of the national program office, that the network has contributed very much to the identity of the Pew Health Policy Program fellows as a group, and the vanguard of change certainly enhanced the visibility of the program in the larger world of health policy and in Washington. It always amazes me when I go on the Hill and I say that I run the Robert Wood Johnson Health Policy Fellowships Program and also the Pew program that

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

everyone knows the Pew program. It's quite remarkable. The individual Pew programs are not in Washington, but there are a lot of alumni. These people now are at the vanguard. If anyone is looking for a job, all they have to do is get the list of Washington-based Pew fellows and they are in very good shape.

Another critical success factor is the leadership of the program. One has to give credit to the Pew Charitable Trusts because this did not end up being one of their major interests, but they continued to have faith in the program, they continued to fund the program, and they were open-minded about it. People like Carolyn Asbury had an uphill battle because foundations don't inherently like to fund programs for the long term. Their willingness to just hang in there was very important. Among the early board members, certainly Bill Richardson was a guiding light of this program from the beginning and until recently. The people who were the original thinkers behind this program—Robert Blendon, Walter McNearny, and Bill Richardson—should be given alot of credit.

6. To what extent do you think the program was beneficial for those who did not finish the program?

My feeling is that programs like this have a halo effect. Even if you didn't finish you participated at least for a time in a very exciting, learning activity, you met the people, and you got alot of value even if you didn't actually get the degree. Those who went through the whole 2 years or so of course work and then didn't get their degree are still considered Pew fellows in discussion and in the directory. So, my feeling is that for the most part there isn't that distinction, even though clearly the purpose of the program is for people to get their degree and to complete the program. Still, it was beneficial for those who didn't complete the program. If you look through the list in an analytic way, there are a lot of people who haven't gotten their degrees who are playing very important roles. We accept them in the "family."

7a. How does the Pew fellowship approach differ from a traditional fellowship approach? How have the major outcomes differed?

My feeling was that this program had all the ingredients to be uniquely successful because it had the rigorous academic training. That was one important element. It had the very exceptional faculty and mentors, and that was another element. It had a built-in network and family and these com-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

mon interests. Everybody wanted to get advanced training in health policy research, but there was a partnership with other programs that had different levels of expertise and other people who had different career objectives. It was the enrichment of a broader environment and of other programs that added another layer of enormous value. Also, it had the exposure to Washington, which at least until recently was the central focus of some of the major activities in the national health policy debate and in the financing and organization of health care delivery. So, it really was a comprehensive attempt at leadership training. It didn't miss a beat at any score. You look at other fellowships where you get a wonderful training and you see a great curriculum, but where is the network, where are you counterparts in the rest of the country? You may know, them but do you really have a chance to interact with them several times a year through newsletters, through directories, and through meetings? Such an important cadre of alumni are now in Washington at the vanguard of change. They are also at the state and foundation levels. So, what was unique about this program was just that it had a critical mass. You weren't talking about four or five graduates from prestigious organizations; you were talking about hundreds of people, and that makes a difference.

Major outcomes were clearly that these people were trained to be effective change agents in the real world, and to that degree I think that this fellowship was so much ahead of its time. I don't think that when the Pew Charitable Trusts thought about this program many years ago that they even realized how on target this program would be for the world today. You just can't train people in an ivory tower environment and have them be effective change agents in the real world where decisions have to be made under pressure and, within budget constraints, where you have to dialogue with people with very different policy and political outlooks and perspectives. So much of the purpose of this program was to train people to play a leadership role. You can see that now from the number of people who, after they go through this program, are working in the private sector and for some of the major think tanks and consulting firms, whereas in the early years a lot of these people went into academia and into the foundation world. As the responsibility for health policy research falls more to the private sector and to states and localities, the Pew fellows are a wonderful match and are a wonderful addition to the field at every level.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with John Griffith Short version phone interview, June 27, 1996

  1. Based on your experience and familiarity with the fellows and the programs, what did the Pew Health Policy Program accomplish?

    The Pew Health Policy Program raised the sensitivity to policy issues and the ability to handle policy issues. Everyone who entered and completed the course work (not just those who completed the degree) were trained to handle policy issues.

    Those fellows that finished the degree at Michigan all went on to positions with considerable influence. (many fellows came into the program in positions that already had considerable influence).

  2. What are the lessons learned?

    There is a market for the kind of a training program that Michigan developed, even with limited support. There is a very responsive market for this kind of a training program with adequate support.

    It is possible and productive to explore policy issues in a seminar-style course with nontraditional students.

    It is important to remember that Michigan had already experimented with a nontraditional training program; the challenge was to transfer it to a doctoral level.

  3. What is the Pew legacy?

    The health policy community is so big and so diverse that it is difficult to speculate about the value added by training a small cohort of people. The community is many times bigger than the output stream.

    Nonetheless, since we started the program approximately 15 years ago, there has been a noticeable increase in willingness to use factual analysis and careful empirical data as a basis for conclusions.

  4. If you were going to give advice to another university attempting to initiate a similar program, what would you say?

    There is a small market for this type of training. It will never be a large market. There is an immense burden placed upon students in this type of program.

    One needs to look for benefits in the context where a few can make a difference.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Dan Rubin Tuesday July 2, 1996 4:30 pm

1. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What were the most important contributions?

At the Pew program breakfast during the Association for Health Services Research Conference, I thought about the Pew program as a whole. I hadn't thought about that for awhile; it was sort of fun. I think probably that the contribution to interdisciplinary doctoral education in the health policy field rather than the specific training of some number of people is the biggest accomplishment. Because of the prestige of the schools that were involved, it speeded up and added prestige to the field of health policy education. At the breakfast I realized that I had probably underestimated how big a change it was for doctoral health policy studies to move toward a policy analytic framework. My own background is in policy analysis. I have a master's from Berkeley in public policy from 1976 and that was always very interdisciplinary. At that point it was very new to have professionally excellent policy analysis training at the graduate level. I think the first programs started in the late 1960s and Berkeley was one of the first. Frankly, I think there is more interdisciplinary work at the master's level than at the doctoral level. I knew that the Berkeley program went on to doctoral studies. At the time I was there there was no health concentration. Later on a dual program with the school of public health was developed. I probably would have continued if it had existed when I was there. When I was there, there was a joint program with the law school, but that was intensifying the legal tools, not focusing on the substantive areas like health. The core curriculum was philosophically very similar to what you or I had, in that it took some rigor in thinking about major political events. Thus, I kind of have a blind spot as to whether or not this was innovative in the Pew program. I heard about the Pew program in the mid-1980s and seriously considered the Brandeis program but I wound up not being able to apply. Things were happening in my job and I found myself at a point in my career where it seemed less realistic to leave for 2 or more years to get the doctorate.

And then you wound up at Michigan. How?

Several years later I looked again, and the Michigan program at that point looked exciting.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Back to the first two questions. If it's true that the response of prestigious universities vying to offer the Pew programs caused a change in their behavior, then in terms of what's known about how innovations are disseminated, the first people to innovate are often in marginal status positions, so often innovative educational programs have low status. Typically, the next stage in the dissemination of innovation is high-status entities or people pick up the idea, and then the third stage is that, because of their status, what they do is taken seriously by others. It may be, in that dimension, that the Pew money and the high status of the schools that jumped for the opportunity pushed us to a point where that many other schools would want to emulate us. That's a speculation on my part. It will be interesting to hear what people who have been institutionally involved think about that—people who nurtured and developed the programs. This is, however, theory based because I happen to know a bit about the theory of innovations. It has nothing to do with higher education; it has to do with people. So, if that effect is real (i.e., speeding dissemination of interdisciplinary policy analysis into health policy programs), then I think that is the Pew program's biggest impact.

Certainly, churning out a number of graduates is a contribution, but I have some trouble seeing that as the main accomplishment. My sense of realism is that people do end up getting interdisciplinary orientations one way or another, for example, through rubbing shoulders on interdisciplinary teams or because of their own career paths. So, the average doctoral graduates in health services research may not have been as interdisciplinary before the Pew programs. That didn't mean, however, that there were no practitioners who combined fields, whether at the individual level or by being active participants in study centers or on interdisciplinary project teams where there was a good intellectual process. My sense of reality is that the Pew program didn't lead to there being interdisciplinary doctorally trained people for the first time; it's one path to do that. It probably did give those individuals more prestige, whatever that meant. I think there is always a tendency for those who have been through a particular path to exaggerate how special it was and to say that it may have been the only way to reach the same endpoint. We [Pew fellows] may be inclined to believe, because we were chosen to be in the Pew program, that we are completely special and that there would be nobody like us if not for this training. That is clearly exaggerated. This is the way that elitist feelings emerge. There is some truth, and there is some exaggeration.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

I was very pleased to hear at the AHSK (Association for Health Services Research) meeting how the Agency for Health Care Policy and Research (AHCPR) money is stimulating more schools to do the same thing.

There can and should be concerns about how to judge the quality of an interdisciplinary product, but that is nothing new in academic training. The same good and bad stuff about reputation and judging the individuals that is always applied will continue to be applied. We can't expect this program to solve the problems of all higher education. I'm not sure which of these questions this belongs in, but I certainly was aware, at least in the Michigan program, of the difficulty in defining the dissertation. I think it will come up later.

2. What was the most innovative or unique aspect of your particular program design and implementation?

Certainly the on job/on campus (OJ/OC) structure was one of the most unique aspects. It is not innovative in the sense that it wasn't done before, because Michigan had been doing it in the master's program for about 20 years. But, it certainly was innovative compared to any other high-quality doctoral option that I saw. There are other nonresidential doctoral options, but I was unaware of any that was focused and that had the quality of faculty and certainly an actual student body working on similar things. I think the experience level and the caliber of the fellows at Michigan also were exceptional. Just over the years, listening to comments and meeting the fellows from the schools, clearly, there is some distinction of who was in which program.

The Michigan program tended to have a higher percentage of real midcareer, senior-level professionals with considerable responsibilities as opposed to my perception that the Brandeis program tended to have a higher mix of earlier career people who were several years out of a master's program and on a track of considerable responsibility but not yet in senior positions. At Michigan, not everybody was in a truly senior position, but every cohort that I was aware of was peppered with people who had large impacts or major management responsibilities within organizations or major political roles. The intimacy among the fellows at Michigan, combined with that caliber, was very strong. I think a number of people in my cohort felt that the most important thing for them was watching how other students reacted and thought differently about the same topics. There was a very high ability to critique thoughts, based more on practical judgments than on being a master of the disciplines.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Going beyond common sense, though, in my group Pat Butler had done seminal work in her field, had been on Institute of Medicine study groups, and as a lawyer was often called upon to write for government audiences about what the legal framework of ERISA (Employer Retirement Insurance Security Act) means. So, classmates really were in cutting-edge policy roles. There were others, like Bill Lubin, whose experience in senior insurance positions was priceless. How often, in a position like mine, do you get to have extended discussions with somebody who has been living in that corporate environment? It's less true now, but for a long time the exchange between insurance types and clinical or public policy types was very limited.

Another thing that was notable in my cohort, but that wasn't really about program design, was the strength that the external faculty brought in. The way that happened, though, was due to problems finding internal faculty to give the right course at the right time. It was sort of ironic. Michel Ibrahim was there because the epidemiologists in the School of Public Health either were or were perceived to be fairly limited in perspective, most of them doing highly technical as opposed to policy work. Past cohorts had probably found that a typical introductory technical course was very unsatisfactory. With the audience of the Pew fellows, the faculty were challenged to deal with relevance to policy issues. This surprised me. It's hard for me to believe that in truth that there aren't epidemiology faculty who are involved all the time in policy debates. I don't know what to make of it. But that was the story I heard. I don't know how much was a matter of specialization in the epidemiologists, as opposed to the availability of people to work Thursday, Friday, Saturday, and Sunday shifts. Michel liked it; he liked the stimulation. He actually didn't do a very in-depth introductory course at all, but he led us through some very good discussions using epidemiological concerns in policy. We got this benefit because of things that didn't work well in previous cohorts.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

The general fact that the curriculum was interdisciplinary certainly was important. I don't want to dwell on that, but it needs to be said because I would never have applied if it wasn't. I think that the way that the dissertation seminar (and this started with my cohort) was structured was very good. It really focused on producing a draft of a

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

prospectus as the course material. The discipline of doing that step by step was fabulous, integrating things we had thought about before while pushing for production. Those of us who didn't think we were going to go on to do work as principal investigators on major grants thought that the focus may have been too narrow, but the emphasis on getting organized was very helpful. The dissertation was demystified for us.

Some of the lead-in courses that Bill Weissert put in place with policy analysis and quantitative analysis methods were very strong. They built a lot of competence through doing typical steps in a quantitative analytic process. Having senior faculty teaching the courses certainly helped strengthen the curriculum.

One of the things that attracted me to the Michigan program was that it was housed in the School of Public Health. Although I was warned by Leon Wyszewianski that I may not get the rigor in public health policy topics that I wanted, I thought they could have done a better job. The Health Services, Management and Policy Department had very little interest in issues of governmental population-based public health. It was almost entirely health care system. The required courses for a public health degree which the school demanded were not always taken seriously. There are four core requirements to get any public health degree at Michigan: epidemiology, biostatistics, environmental health, and health behavior and psychology. Biostatistics was treated very seriously, that was a core tool whether you put the ''bio'' prefix on or not. Any good program would have intermediate statistical work. The environmental health part was not taken seriously. We had senior faculty doing a class with us, but they never successfully found a way to apply policy thought to environmental health problems. The course was a modification of a survey course on topics in environmental health, and what happened was that the students essentially took control of the course. We were presided over in a very friendly manner by a very nice person, but we did the presentations. It was a great learning experience, though unfocused. The reason we did that was because the structure was failing to give us what we needed. Lectures would go off into digressions about the chemical basis for air pollution. What we got out of the course was valuable, but it did not get into the questions that we had hoped to address, such as: How do you approach environmental policy using tools of policy analysis? How do you interact with scientific issues?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

How do you interact with science politics and the limited ability of governments to act together? That is profound in health policy. Environmental health is a classic example, because the science tends to be hard science, but setting science-based policy continues to be very difficult. I am critical of the fact that after a number of years had gone on the program was unable to find a way to combine environmental health and policy.

Health behavior and psychology was well taught, and that worked as a basic skill, as a discipline that we needed to be aware of. The fourth, epidemiology, we talked about already. Because the methods of epidemiology are extensively used in research, we got the skills that we needed, but if it hadn't been for Michel Ibrahim we might have gotten an inadequate course.

I thought that the program could have done alot better working across disciplinary lines or across interests with people who were involved in public health policy. To my mind the lack of rigor in public health policy is something that needs to be addressed, rather than a reason to continue separating public health from health services research along traditional lines. While my cohort was in progress, the School of Public Health reorganized and a formerly separate department of public health policy was combined with the Department of Health Services Management and Policy (HSMP). The scuttlebutt at the time was how HSMP faculty disliked the combination. I sensed a disrespect of the caliber of work done by most, or some, of the faculty in public health policy, although that was not true across the board. The bottom line was that the chance to work on public health projects with equal rigor was not taken up. In theory, a dissertation could go into those areas. Some dissertations have.

4. How, if at all, was the Pew approach different from the traditional teaching approach?

It's more interdisciplinary and analytic. The students have more to offer; that's my impression. What I heard about the doctoral students that weren't in the Pew program was that there was more of a possibility of drifting and not coming to grips with the need to choose a research topic. Not much of a peer network existed among the non-Pew doctoral students, especially if they came in post-master's. The master's program is more professionally oriented; thus, there is a gap between most master's students and the doctoral students.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

5. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

I'm not sure. I don't have a good sense of this. I think part of that is unique in my life in that I was interrupted in other ways while I was in training—my daughter became sick and ultimately died. I can answer the value to my life. My reason for going into the program was not a clear career goal. I heard that the [Pew Charitable Trusts] wanted to avoid taking students whose career goal was to shift gears toward teaching. I have taught in the past. I would be interested in doing some teaching in the future, and so it's likely that the degree will help me do that. But, I'm not that interested in moving into academia. In that sense what I want is consistent with the Pew program goals. Whether my career speeds up or changes, I think that I can talk the language of health services research better. I know the technical, methodological issues a lot better. That helps my relationship with peer researchers. But I didn't experience a problem in that area before. I simply am sharper. Going back some years, I never had trouble talking to researchers or academics, engaging in dialogue, and talking about both policy issues and technical issues. My master's training sensitized me to the kinds of things that come up, although I learned a great deal technically in the doctoral courses. I knew the generalities but I didn't know that much particularly about statistics. It's too early to say if my career trajectory has changed. I don't know whether it will or not. The more I have to offer, the more value I have. My skills tend to be used in the area of consulting and forming marriages among activities that are more connected than their party's parts may realize. That is a continuation of the kind of role I played in the past. I do think, however, that I do it better now. Right now my work has to do with connecting government roles that relate to health care quality and information use. It calls for judgment about what needs to be related to what, more than judgment about how to assemble the research plan for a study.

Do you think that the people who did not come in with the same interdisciplinary background that you did, can, as a result of the program, better see these "connections" that you're referring to?

Yes.

6. If you did not complete the program, do you plan to? If yes, why? If no, why?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

I do intend to complete the program. I am still predissertation. Life delayed my progress; however, I do intend on completing.

So, you are in a position to do much of what you have always wanted to do and you are called upon to consult, why finish?

The intellectual closure of doing extended work will cause me to finish. One of my personal reasons to do the program was for me to see what it really took for me to do extended work. The bread and butter of government work is not extended, analytic ventures. Sometimes there is extended writing and certainly extended thinking about a topic. But, the kind of thing involved in writing a dissertation is very important for me to learn from and to learn about myself. One of the things I said in my application was that I want to find out whether I would like to shift the mix of work I do into more extended products, and that is still true. I learned a lot through writing the papers in the program, but most of the papers were all short products, more similar to the kind of thing that I've been doing for years.

Once I get the degree it will probably make me a candidate for different things, for good or for bad. I think that perception is very important in who gets screened in and out of job searches. The last few years I haven't been looking to change jobs; I've been looking for stability so that I could be in school and so that my personal life could normalize. I may enter a period in the next few years where I'm beginning to look around more.

Could you speculate about the value of a program like this to people who do not complete it?

If it's a new interdisciplinary perspective, it could be a very high value. In my case, if I decided to stop after the classes, I would have learned a great deal, in an expensive way for the foundation.

7. What is the Pew "legacy" in terms of:

  1. health policy?

    "Impact on health policy" is the impact of what evolves in the health policies of the United States. I think it's unclear, but the network of people trained by Pew is a well-trained and mutually credible group of health policy pros, and they may make a real contribution to good exchange among sectors. There always has been exchange among the

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

university and private research groups and some areas of government. The interchange with the private sector has been less common. One has to ask, though, in how many of the cohorts in all the schools was there a substantial stretch in dealing with people from other perspectives and, particularly, from senior private-sector roles, especially other than hospital administration, which is the area that has had the most interchange traditionally? Mutual credibility at a sophisticated level would be a contribution to policy.

Research analysis being done that affects health policy could have been accelerated by Pew. Yet, I don't think there is anything fundamentally new about interdisciplinary project teams in health policy. Look at RAND studies or General Accounting Office work. This is not unidimensional.

  1. education?

    This I spoke to earlier when I answered the questions of what is the biggest contribution.

  2. your future?

    I've addressed this too.

8. Are there any important issues that this interview does not address? If so, please feel free to add comments and/or concerns.

The issue of the dissertation. What is the dissertation in doctoral health policy studies/policy analysis? The interesting thought that I heard from senior faculty at Michigan was that interdisciplinary things happen through teams of people primarily, not through an individual. So, in the health services research or health policy research side of things, the question is, "Is an interdisciplinary dissertation real, or is interdisciplinarity something that happens in groups of people and a dissertation would become too big if you tried to do it with multiple disciplines to a standard of excellence?" I personally think it is possible to do a multidisciplinary dissertation. Is there such a thing as a policy analysis doctoral dissertation, or is excellence and competence supposed to be judged by the part that really is research? The distinctions aren't complete. In real-life, practical policy analysis, the closest you get to it is probably modeling, where you take a policy problem, model it, deal explicitly with policy options and environmental variables, and get your model to try to speak to various dimensions of outcomes. I think it is very hard to weigh things toward "what is the correct policy?" as opposed to "what is the right answer to a research ques-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

tion?" and come up with a dissertation that you can negotiate with your committee. I've shied away from that because I don't want the hassle. The general question is, "What is meant by a dissertation in this field?" To what extent can the dissertation as well as the instruction be interdisciplinary?

Another thing, to follow up what I said at the AHSR meeting, it might be interesting to look at how people came to the programs. For me, the structure of the Michigan program and being in the School of Public Health were very attractive. The Heller School had other aspects that were attractive.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Doctoral Alumna Joan DaVanzo Wednesday July 17, 1996, 2 p.m.

1. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What were the most important contributions?

I thought there were three very different accomplishments that I saw in a production modality. The program produced doctoral-and midcareer-level students who had a very particular expertise. It produced a network of these folks who were involved in different policy work around the country, and it contributed and expanded the real construct of health policy to sort of produce a certain model of how you think about and how you might do an analysis. I think the most important contribution is that it created this multidisciplinarily trained cohort of individuals who are all working in big ways to influence policy.

Doctoral programs are usually very hard to finish, and there are a whole bunch of ABDs (students who have completed all of their doctoral requirements except for the dissertation) around. At RAND/UCLA (University of California, San Francisco) the motivation to finish was very strong. The Pew programs created a strong motivation to finish, and they did that in a variety of ways. Some of it was networking with alumni. It was seeing the people that you knew in these neat jobs and writing these great papers, and you wanted to finish and join them. My class at RAND had four students, and all finished in a very timely fashion. A lot of that motivation is created by the program. I don't know how much is the selection of people or what you did along the way or being associated with RAND/UCLA, etc. But it seemed to be that the support of the Pew programs distinguished them from other doctoral programs in university settings.

What about your particular program?

Well I have to backtrack and tell you that I started in 1989 at RAND, and then in 1991, when I heard Ron Andersen was coming to UCLA, I switched to UCLA. I don't think I could have had a better program if I set out to make a perfect program. The RAND program was heavily quantitative, I had microeconomics 1 and 2, macroeconomics 1 and 2, microeconomics for regulatory policy, econometrics, and calculus, which I had never even taken before. Then I took four statistics classes to get credit for two and

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

more. The culture in RAND is such that the hierarchy is the economist, the statisticians, the operations research folks, and then the behavioral scientists. You get this sort of skewed perception of the world, where the economics is king, and it sort of shaped your perception and the analysis you do. I'm a psychotherapist, and this route was difficult for me, but I picked it because I knew that's where I needed to go. Then I realized I knew nothing about health, epidemiology, public health, the broader picture. So when I switched to UCLA I got all that. I got the public health stuff, the underserved population stuff; we talked about access and equity and all these important issues that you might have to know about if you wanted to work in an advocacy sense. It was a whole different spin on health policy. So that's the good news and the bad news. Because RAND was so influential in the 1970s and 1980s and had all these great people there, it was good to be exposed to that. However, it was limiting. There was not enough breadth. I got the breadth by going over to UCLA. RAND didn't encourage students to go to UCLA for classes, although they said you could take classes there. I had the best of both worlds. There is a sequence element also. I really had to do the quantitative work up front because then when I went to UCLA I was prepared. I would not only say to someone "do both," I would say "do both in that order." The combination is very valuable.

2. What was the most innovative or unique aspect of your particular program design and implementation?

I would say the flexibility among the programs. I was able to do RAND and then UCLA. Others did RAND and then UCSF. That was a real important aspect of the program.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

What I thought did not work as well was this devaluation of behavioral research, and at UCLA health behavior was really important. Because RAND was so economics focused you did not get a sense that the behavioral aspects were as important as they really are. You also did not get any kind of clinical exposure at RAND. At UCLA there were people doing cancer research and you could go to where the medical students were taught and you could go to seminars where they talked about clinical research, and things were framed clinically, with clinical examples. When they talked

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

about outcomes research there was a clinical realness to it. That wasn't part of the RAND curriculum. RAND was great at what it did; it was fantastic to have had it, and I wouldn't trade it for the world but it was limiting.

What about the dissertation process?

I experienced that at UCLA. My initial idea was to look at depression, autonomy, and Medicare expenditures. The relationship between depression and autonomy was something I was interested in and working with a lot clinically in my private practice (I kept a private practice throughout). Everybody liked my preliminary proposals, but when I got it down to the wire this one professor said I couldn't do depression because it was his research area. I was so steamed, but then another principal investigator said, "Well, you can get mad or you can get a dissertation." So I swallowed it and did autonomy alone and got great results. I never went back to the mental health stuff while I was there. I know everyone has that experience. It seems to be very critical to doctoral programs in general. But I had a fabulous committee. There were six members, and each had their own spin on things. I was lucky because I was working on a project where there were data I could analyze.

What about the Pew conferences?

They were great. The first year was hard because I didn't know what to expect. But I met a lot of people. Then the second one was in Toronto, and that was great. It was all on the Canadian health care systems. Then they had one in Savannah that I didn't go to.

What about the interaction between non-Pew fellows and Pew fellows?

Perhaps there was a tension here as in all other programs, but I think less so at RAND because if you were accepted to RAND, whether or not you were on a Pew fellowship, you had the same project opportunities. We were in lockstep for that first year. We all worked together in the same room, all day, every day just about. There really wasn't that great of a differentiation. The expectations were the same in terms of how people did, the distribution of students, etc. We had very little tension actually. We didn't have any closed seminars. The only thing we did differently was to be able to talk to Kate Korman. That was really it.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

4. How, if at all, was the Pew approach different from the traditional teaching approach?

The work projects were great. You learn about stuff conceptually in the classes and then you do it on the projects. The students have to interview and get hired on the projects. Students have to go around and talk to everybody about their projects; meanwhile, you know nothing and the project directors know that. After you've been there a while and you've developed certain skills and people know you have them, project directors come to you and ask for your participation. It flip flops, but on the front end it's tough. Overall, it was a really great experience. It forces you to learn how to ask the right questions. First you gravitate to what you know, where you strengths are, and then you learn to find projects that will help to strengthen your weaknesses. You learn where your holes are, and you seek out opportunities that will strengthen those holes. Initially, I wanted qualitative because I was getting my head handed to me in quantitative. So, initially I did some ethnographic work, which was so useful, and I did some of that when I got out. I got a job directly from working on the work projects at RAND. Then, at UCLA I was on a Medicare Demonstration Project, and that was working with a super data set. I was forced to learn SAS on a mainframe and learned to do these analyses. It was exactly what I needed. I came out with qualitative and quantitative experience.

The other part is the people. Working on the projects allows you to get to know the professors in a different way. As a student you know a professor is a professor and you are both locked into that role, and then you're thrown on a project with a professor and all of a sudden they are more collegial. You're not just one of the graduate students anymore. You have access to them in a different way. Being able to switch the role is very important when you are a student.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

My professional life has totally changed. I was a health care professional. Now I'm doing research. It gave me skills that have increased my flexibility. There are many things that I now have open to me that I didn't before with just a clinical background. My head has changed. I was told early on that as I learn more about statistics I'd become a better therapist and

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

I didn't believe that. But it was true. It was absolutely true. The process of cognitive development that I went through in this program was probably the most differentiating experience. It's like having a growth spurt in a three-dimensional way, driven by the cognitive development, akin to a baby having a growth spurt that's driven by physical maturation. It was just fabulous, and I would recommend it to anybody simply for the way my thinking changed. Does that happen in all doctoral programs? I just don't know. I don't think so.

6. If you did not complete the program do you plan to? If yes, why? If no, why?

Not applicable.

7. What is the Pew ''legacy'' in terms of:

  1. health policy?

    Pew brought a broader range of individuals to the field, so it's a thing about people. I don't know how much of the curriculum at the schools was the same, but the Pew model seemed to me to provide a pretty heavy dose of economics, a fair amount of stats, and then other stuff. I think we all had a fairly similar analytic strategy, if you will, across the set of programs. At least it seemed that way to me. I don't know the influence of having this bulk of people who have that same view of the world dropped into the health policy arena. I would think that it shapes the construct and enriches it and gives it more depth. On one hand it's a grandiose statement to make; on the other hand it's really not. If you think about the economics of the health policy field, it's really a small field, and if you get that many people (what about 200 or so?) with the same combination of classes yet with varied backgrounds and training, the scene is hit from many different levels.

  2. education?

    Between the network of alumni and the network of the other Pew fellows, it's the people and the stuff together that made the educational experience what it was. The combination of topics from economics, statistics, behavioral, public health, etc., to produce this gestalt is just revolutionary. It was such a great idea.

  3. your future?

    There are two parts to this. The real abstract part is that it has allowed me to synthesize all my experiences and pro-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

vided a vehicle that is useful. I'm still doing work for people that I met in Pew. I moved from San Francisco to Washington, D.C., because I said to myself, "If you want to be in health policy Washington is the place to be." When I landed in Washington I found this whole community of Pew folks that I had access to. It was like being home. Not only people that you know but even people from the Pew Directory, you could just ring them up. Marion Ein Lewin and the Institute of Medicine do a really good job of keeping the network alive with the newsletter and having the functions and the updating of the Pew directory. I know that takes a lot to put that all together, but that's part of the Pew legacy, part of creating an entity and then calling it the Pew legacy. You have to devote time and space. I just think it's been great.

8. Are there any important issues that this interview does not address? If so, please feel free to add comments and/or concerns.

A lot of what I experienced was tied into the fact that I was at both RAND and UCLA. I am a sample of one. I think many people you speak with will have something similar to say. There was something unique in the way that the program unfolds for each and every person. I don't know how you would analyze something like that, but it certainly speaks to the flexibility of the program, and that flexibility is one of its great strengths. Maybe the program instilled this feeling of ownership: every individual had their own piece. Maybe this is part of the Pew mold, the uniqueness, the individuality.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Midcareer Alumnus Terry Hammons Friday July 12, 1996, 8:30 a.m.

1. Based on your experience and familiarity with the Pew program, what did the fellowship really accomplish? What were the most important contributions?

My program's principle accomplishment was that it pulled a number of experienced people, so-called midcareer people, into an environment where they were enabled and helped to become valuable contributors to health care policy, both public and private. They were enabled to do this in a way that grounded policy making in experience and knowledge based on research, which I would say is a very high-quality way to approach policy. That does not mean that policy is not carried out in a political context, which it always is. Yet my experience is that there is a huge advantage in terms of the policy that ends up being made or implemented, grounding it in principle and knowledge, and the political context doesn't just drive it. My program took people who weren't just graduate students and gave them an opportunity to contribute in that way, for example, my participation with the U.S. Congress and also private or quasipublic things like the Kaiser Family Foundation, the University of Virginia, and the Radiological Society of America. A secondary contribution was that it enabled many of us to learn the basics, and in my case more than that because I already had a strong background in health services research, and to do that research in a policy-relevant manner. A lot of health services research that is done is of very little value in developing policy, guiding policy, or guiding action in general. The analog of policy in the private sector is not legislation, regulation, and so forth but how you run an organization and so forth. I think those are the most important contributions of my program. For the program, in general, I think roughly the same things apply. My impression is that the other programs had a lot more junior-level, inexperienced people, and what they turned out were people who were less likely to be able to, at least in the short turn, contribute to policy and more likely to end up in lower-level positions or academic positions. Since they are younger and less experienced you would need to look at longer trajectories. If you look 10 years out you might find that by then they have moved into more senior positions.

2. What was the most innovative or unique aspect of your particular program design and implementation?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Our program was a combination of workshops, which were not just lectures but workshops and seminars that gave us background material, methods, and other knowledge, again, in a context of application. Then the second part of our program was working on real projects, and I thought that, in contrast to my experience in graduate school, for example, these workshops and seminars were more effective than traditional graduate education (I did economics at the Massachusetts Institute of Technology). Most important were the projects. I was fortunate enough to be involved in four and a half really wonderful projects through RAND during the year I was there. They were real projects, such as helping the U.S. Department of Health and Human Services (DHHS) understand what academic medical centers were going through and how to make policy that related to graduate medical education, research and training, and so forth that was appropriate for the nation's goals but that also took into account what academic medical centers could do and were doing. Another one was on the implications of alternative ways to pay physicians on quality and cost of care, and that led to my position with the Congress. A third on the National Institutes of Health decision-making process dealt with evaluations of existing practices and technology, which is a difficult issue. There were others. The projects were just absolutely wonderful.

Can you speculate how many of your classmates or people from the classes before or after you went on to get jobs as a result of these work projects?

My class, so to speak, was three people. One of my classmates modified his professional course to be more involved in policy, the costs of care, and manpower policy in radiology, for example, reimbursement and other regulatory policies for radiological services. The program certainly influenced his career. This person went from a faculty positions and I think assistant chairman to chairman of a department, so he moved up. Whether he would have gotten the same job anyway, I don't know. However, it is clear that it influenced what he does in that job. My other classmate did change his direction and he is now working for the Kaiser Family Foundation, I believe. He has become more involved in policy than he was in the past. For me it is absolutely clear that my career has changed. I went from a fairly typical faculty position at The University of Iowa College of Medicine to deputy director of the Physician Payment Review Commission (PPRC) for Congress. My subsequent positions were then driven out of that. As I focused on internal policy

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

and administrative positions in private organizations and academic medical centers, I am very much involved in changing the way these places work. My expectation is that I will end up more clearly back in the policy world in a clearer public or quasipublic policy role. Right now, in my opinion there is not much going on in health policy that is really going to happen. But, there will be. There is a lot of private health policy changes that I am involved in. That just happens to be the way health policy is changing.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

Going back to what I said before, the seminars and workshops with their mixture of principles, methods and applications were good, and the projects were wonderful. I thought it was a really good curriculum.

Who was in the seminars with you?

There were my two other classmates and then some of the other graduate students at RAND and UCLA (University of California, Los Angeles) that were more like the people in the other Pew programs (less experienced and so forth). So it ranged from some with just the three midcareer people to seminars with four to eight other people. They were all taught by the full-time faculty at RAND and UCLA.

4. How, if at all, was the Pew approach different from the traditional teaching approach?

I haven't really been in a traditional program, but my guess is that Pew is more about application and use and is more targeted at experience through those projects. I'm comparing it, in a sense, to my economics work.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

It completely changed the direction I was going in. It gave me tremendous opportunities that are still playing out and will play out from the rest of my career. It has been just incredibly valuable to me, because even though I had a background in economics and I was viewed within the college of medicine and within the context of academic medicine as pretty well informed about policy things, if you will, I was clearly not capa-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ble of making a major contribution in either policy or health services research that is policy relevant. I just didn't have the critical mass of understanding that you need to do that. When I finished the 1 year at RAND, I did have that. The other thing that was very valuable, and these are intertwined, was the opportunity to get to know a lot of people around the country who were doing that kind of thing at RAND, Brandeis, Harvard or in places like the Congressional Budget Office and DHHS in Washington. That is very much tied into the projects and the applied stuff. That has been wonderful.

Has that network continued?

Yes. Of course, some of the people turn over and some you lose track of, but there are many people I keep in touch with and talk with regularly about what is going on in policy and so forth.

6. If you did not complete the program do you plan to? If yes, why? If no, why?

Not applicable.

7. What is the Pew "legacy?" in terms of:

  1. health policy?

    I assume it's what people did or will do with the training. When we are talking about the younger people we have to keep in mind that it may take longer to see what they will contribute.

  2. education?

    I think there is a real dearth of understanding in medical education of the health care systems and health care policy and so forth, and I've gotten involved with some of that at Case Western and here at Hopkins. But I know there have been others from my program and the other Pew programs that have made huge contributions to education.

    What about how the Pew training has changed, if at all, the method for training health policy makers?

    It has, but it's not the only way. There are obviously many paths by which people end up contributing to health policy. I thought the particular program I went through, the midcareer program, and the way it was designed at RAND was a really effective one, and it was able to be accomplished

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

in only 1 year. I looked at Phil Lee's program, that was a 2-year program at UCSF at the time, and it was between those two. His was a more conventional program, certainly a very good one, but it was very attractive to me that the RAND/UCLA program was 1 year and that it was just packed with experience. I just think there is no way as effective as these projects to learn about policy, how to do policy in a high-quality way, and to be involved in it with people who are the best in the world.

  1. your future?

    The program completely changed my future.

8. Are there any important issues that this interview does not address? If so, please feel free to add comments and/or concerns.

I was angry and disappointed and thought it was a huge mistake for the Pew program to quit funding my particular version of the Pew Health Policy Program in midstream. I have no idea why they did that. I don't know who did it. Perhaps if somebody could explain to me why I would see the logic, but I thought it was really a huge mistake. It seemed to me the most innovative program, and I say that without really fully knowing the rest of them. The RAND program just seemed richest in real experience. Of course, I have a strong bias. I would just really like to know how the decision to end the RAND programs were made.

Paul Ginsburg at the Center for Studying Health Policy Change was one of the lead people at the CBO, then he spent a year at RAND, and it happened to be the same year I was there. Then he went back to Washington and became executive director of the PPRC, and I was his deputy. He is still in Washington. He is a prototypical RAND person. He is a good economist, a superb researcher and thinker, and one of the leading people who could use his academic knowledge to make sound policy.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Doctoral Alumnus Jonathan Howland Tuesday July 16, 1996, 1:30 p.m.

1. Based on your experience and familiarity with the fellows and the programs, what did the program really accomplish? What are the most important contributions?

There are definitely some people who went off into health policy and are doing great things, so the program kind of seeded the field in that way. There are a bunch of us who went into academia and other things, but that too supplements the field. What the program did for a lot of people was to give them an opportunity to get their doctorate under circumstances that if it were not for the program, they would not have been able to have done so. There were two things that the program did. The first was that it bestowed a certain amount of prestige, so it was seen as a career enhancer, and the other thing it did was to provide some funds. So, for example, in my case I had just finished my MPH, and I was in a sense retreating from a previous profession, and I probably would not have been able to get my PhD without the program. So, from my own perspective what the program did was to help us get our doctorate, which was critical to our careers.

The Pew program was supposed to take people out of their careers in health policy, tune them up, give them some skills, and put them back wiser and more skilled. The extent to which it did that I don't really know.

Certainly, one important contribution that it made in a generic sense was that it created a prestigious fellowship for health policy, which in and of itself said something about the importance of health policy as a discipline and that having a doctorate in health policy was a desirable thing. That was important. Some of us did some research that may have been useful. I think the issues that may have been important in health policy 13 years ago have changed so much that we certainly got a grounding and were able to understand what was happening, but the extent to which fellows have gone on and sort of changed the spin of health policy in this country, I don't know.

2. What was the most innovative or unique aspect of your particular program design and implementation?

I was at Boston University (BU) and I thought it was great that we could essentially design our own curriculum and decide what tools we wanted to get. One of the reasons

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

I thought that was so great was that the program enabled me to study epidemiology, which is what I wanted to do, and I was free from taking a whole lot of required courses and able to pursue the courses that were really of interest to me. That was really nice. The other thing that was really nice was that I was able to go through the program in 23 months. I did my course and my dissertation at the same time. There are very few programs that would have allowed me to do that. It was a fast-track program. Now there are good sides and bad sides to that because there were people who went through and should not have gotten the doctorate. But for others, this structure really allowed them to get through in a way that fit with their lives. Very few other programs would have afforded them that opportunity, certainly, very few programs with the kind of prestige that the Pew fellowship brought with it.

Could you talk a bit about the integration between the BU and Brandeis program?

I think it worked pretty well. We felt fairly cohesive as a group. For some people I didn't know what program they were in. The Pew fellows' identity was so transcendent. We felt like Pew fellows, and that had a higher profile than what school we were at. People like Steve Crane were incredibly helpful across the board for all fellows as an advocate, counselor, a mentor. On the other hand, we learned a tremendous amount from people like Stanley Wallack about health policy. I just think that it worked very well. I was very sorry that BU withdrew from the program.

What effect did the tightly structured Brandeis curriculum have, in comparison to the more loosely structured BU program?

The Brandeis students certainly talked about and envied our freedom. On the other hand, there were people who went through the BU program who didn't learn any methodology, who came out of the program not knowing what a regression equation was. That, to me, was really shocking. So, in a way, even though I enjoyed the freedom of being able to do what I wanted, what I wanted out of the program was skill acquisition, and so what BU did was to give me the degrees of freedom to load up on biostatistics courses. And those are the skills that I market right now. What other of my colleagues at BU did was to avoid some of the more rigorous methodological courses, and I think they came out with a shorter skill set and it showed up in their dissertation. If we did this again, I would like to see some

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

sort of mechanism in place that would ensure that people got the necessary skills, yet still be left alone to do it. Otherwise, I would say that it is too risky and that there should be a curriculum that has a minimum skill set requirement that everyone must take. I'm sort of conservative educationally.

When I took over for Steve I inherited a bunch of my classmates as students. It was very interesting because I saw many of my peers struggle with the skills that they should have mastered. So overall I think there was some envy from the Brandeis students over the freedom that we had, and I enjoyed that freedom, but there were some students who didn't make the best choices and would have benefited from being told that they were not leaving the program without understanding what multivariate regression is.

When you have a midcareer program like this, you get two groups of people. You get one group that says, "Thank God for this window of opportunity to learn some stuff," and then you get another group of people who shook off the system for some reason like they weren't too embedded in it. The latter group came in thinking that they knew it all and they left thinking that they knew it all, but they didn't learn anything in between.

3. What was it about the curriculum that contributed or did not contribute to the program's success? Could you also speak a bit about the dissertation process? What was it about the curriculum and/or program that enabled you to get through as quickly as you did. Was it a what, or was it a who?

First of all, the fact that you didn't have to collect your own data and could use an existing data set helped a lot. I did my dissertation using the Framingham Data Set, which was just a beautiful data set. It was well groomed, and the school was crawling with people who knew the variables and how stuff was collected. It was an incredibly supportive environment, and that really made a huge difference. The other thing was that I think I found out in April that I had been accepted into the program, and I started looking for data sets then. I knew I was going to have to do a dissertation, and I finally ended up with a dissertation topic in December of my first year. But I had been working on it since April. So, I say that I did it in 23 months, but the fact of the matter is I gave myself a long lead time. Once I knew what I was going to do I got right to work on it. Now that was my case. A whole bunch of people just went aground on the dissertation topic, and they still haven't done it.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Were there structures built into the curriculum that helped to guide students, like dissertation seminars?

No, I don't think we ever had a dissertation seminar. Steve Crane was a great counselor. Steve and I spent inordinate amounts of time with some people to help with their dissertation and got, what I think, was really high-quality mentoring, yet some still never got it. I don't know how to change this.

What about the curriculum contributed or did not contribute to the success of the program?

Well I think, certainly at the BU side, there should have been some minimal requirements or simple learning objectives: fellows will leave with this basic skill set. People were not served well by not having those learning objectives spelled out and enforced. But for me it was a breeze because I was doing my dissertation as I was picking my courses: harmony of the spheres.

4. How, if at all, was the Pew approach different from the traditional teaching approach?

It's hard for me to say, not having done other doctoral programs. In other doctoral programs there is more of an adversarial relationship between the doctoral candidate and the faculty. You may have your mentor, but as a whole the department's view is that you have to prove that you are worthy of our bestowing this doctorate from our department. There is this kind of onus on individuals to prove themselves. In the Pew program the faculty had a real stake in people getting through; a high attrition rate was not a good thing. There are two sides to this: on one hand you had a very user-friendly faculty; on the other hand it allowed some people to be irresponsible in terms of the course selection, the amount of work that they did, and what learning they did. So, I guess the simplest answer to the question is that the faculty were stakeholders in the success of the program. I think that is different from other places.

You alluded to something earlier that perhaps you could expand upon. You said that there was a strong identification with being a Pew fellow. Do you think that makes the program different than a traditional doctoral programs? And, how do you think that affects the legacy of the program?

It had good and bad effects. In one sense it generated a sense of privilege, and rather than being graceful the Pew fellows were demanding, were often obnoxious, and had an

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

incredible sense of specialness that often manifested itself. That comes with the prestige of the program, plus the whole way in which the Pew fellows were pumped up not only by the faculty but also by the foundation. We needed more discipline and we needed to be reminded that we were really very fortunate to have what had been bestowed upon us, and that made us responsible to behave in a gracious manner. I felt as if, because of the nature of the program and because everybody had such a big stake in our succeeding, that there was no one to come along and tell us to shape up. Once you've been through the selection process and you've been given that $10,000, attrition really isn't an option. Every failure is a failure of the program. Conversely, in a traditional PhD program, the whole process is set up for people to fail. The fact that the students had that sort of unspoken edge over the program, the fact that once they were in it was not in the program's best interest for them to leave, gave a kind of tolerance that I think bred an attitude that was not great.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

Before I started the program I was getting my MPH, and I intended on getting out and getting a job with the Department of Public Health (DPH) and I saw myself as a sort of midlevel bureaucrat. I was kind of hoping to get a job down by the regional office. Today I am celebrating the receipt of my full professorship. So, the Pew program determined the difference between what I had predicted for myself and what happened. I would have gotten a job with the DPH and I would never have gotten my doctorate. It would have been too tough. I would have been too involved and too committed, and I would never have gotten out. The program gave me the opportunity to get the education and set of skills that I just hadn't gotten before. I thought I had lost the opportunity. It was absolutely pivotal for my life and one which I feel extremely grateful for. So when I seem harsh toward the program, sometimes I don't mean to be harsh toward the foundation or the program. They both served me very well. It's just that there are some people I feel didn't get as much out of it as I did or as they could have. In a way, for something that was so good and so important for me, I kind of am always trying to figure out why that couldn't have been for everybody and whose fault that was.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

6. If you did not complete the program do you plan to? If yes, why? If no, why? Let me rephrase this question: Can you speculate about the people who did not finish the program in terms of what value the Pew training had on their career trajectories, and were there people, do you think, who came in with no intentions of completing the program?

I think there may have been a few people like that. I can tell you that I think that is really despicable, because the foundation was paying your tuition and giving you a stipend in exchange for something, and to not take that deal seriously from the word go seems to me unethical. There are people to this day who have blown off their dissertation, but they don't see it in that light. They see it completely from their personal point of view, not from the perspective of a program that invested in them to do something.

Is it a recruitment issue?

I don't know. I run a fellowship program now, and it's really hard to pick who is going to make it and who is not. But, I don't think anyone ever spelled it out for the Pew students. They never said, ''If you are going to take this money you have an obligation to follow through.'' I don't know whether that would have changed the numbers overall. But I think there are people who didn't get their doctorates yet who don't feel at all as if they have failed or haven't fulfilled a responsibility. Also, I think the people who didn't finish were probably the same people who were least aggressive about picking courses that would increase their skill set. I think what they got out of it was 2 years of sabbatical. I'm sure they learned some stuff, but nothing like what they could have learned if they had come in with an aggressive program of skill acquisition and if they had done their dissertation.

7. What is the Pew "legacy" in terms of:

  1. health policy?

    In a way it's a little premature. I think we ought to reask this question in 20 years and see what happened to us. In a way it's a question that seems to me most appropriately addressed to the foundation. It's actually always something I've always wanted to ask to the foundation: "what did you want and did you get what you wanted?" Or even, "Do you know what you got?" They would send evaluators around to see what a wonderful program it as, and the first time I took this very seriously. I think I was just out of the program a few years, and I had very strong feelings about what the program

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

should be, but they weren't negative, just strong. I'd read these reports and they all seemed to be all happy-faced to me.

One of the things that the Pew program really did for a lot of people, even if they didn't get their dissertation, was to add a certain amount of cachet to their resumes.

  1. education and training?

    I don't know whether anything about the program is so unique or so either bad or good as to leave a lasting impression on the educational landscape. There are a lot of fellowship programs. It is interesting that there are certain times in history when there is a demand for people trained in a certain area, and a foundation can come in and sort of create a cadre of those people. I think that is an interesting kind of concept. I am sure that it has happened in the past, and I am sure that it will happen in the future. Sometimes it's the federal government. When I was going to college years ago there was a whole group of people who were subsidized to study Russian or Soviet Affairs. A lot of times the marketplace will do it. I graduated with city planning as my undergraduate at a time when Lyndon Johnson was President and there was the great society, so the marketplace was creating these people. Whether or not the Robert Wood Johnson fellows or the Pew fellows will be seen as seeding the field and changing the spin of health policy, I just don't know.

  2. your future? You've spoken about this already. Is there anything else you'd like to add?

    I will always be grateful. I know I learned to think about health policy in some ways that I just hadn't before. That information will be one of the filters that is part of the way I look at things for the rest of my career.

8. Is there anything else you'd like to add from the perspective of a program director?

I started a fellowship here that takes return Peace Corps people and places them for 2 years as residents of public housing. During that time they get their MPH and they are also AmeriCorps volunteers, so they have to do good work. When I was starting this program I was very enthusiastic about it conceptually, and still am for that matter. We were forming the program and set up a lot of things to make the group of fellows feel privileged, entitled, and special because they were going to five a hard life. Throughout that whole process of early design I kept thinking that we had to be very

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

careful that this doesn't translate into the kinds of sense of entitlement that the Pew program got into, at least at BU and Brandeis. That can get pathological and become unhealthy not only for the program but for the fellow as well. I guess that's one of the things I learned from being both a fellow and filling in for Steve for a while.

Other lessons: In recruitment you need to screen for motivation and completion and, secondly, you have to make it clear that there is an expectation that comes with being accepted to the program that you will finish your doctorate. Of course, things do happen. A lot of people don't finish doctoral programs. But in this case they were getting paid to do it.

In terms of degree requirements, I mentioned before I think a minimum skill set should be defined, and it should be skill oriented and enforced.

In terms of integration with other students and the university, we've spoken at length about it. It's a conundrum. On one hand you want the fellows to be special and to have a sense of pride in their accomplishment at having been accepted to the program, on the other hand you don't want then to go around feeling entitled. Those other students who are trying to prove themselves, these students were supported by the faculty because getting them through was one of the performance measures for the program.

If I were to give advice to another university I would tell them some of the characteristics I would look for in my recruitment screen beyond just how you did in your previous courses. I'd really look at commitment and motivation level. I would establish an independent advisery board to keep an eye on the program. That would allow the faculty people and the administrators to say that they were tempted to go native with the students. It would instill a quality control mechanism. We got in binds sometimes with students and their dissertations because on one hand we clearly had an interest in getting people through; on the other hand we were clearly letting a few people through whose work just was not up to the quality that it should have been. It was not good for us to be conflicted in that way.

I think if I were the Pew Charitable Trusts I would have taken a very close look at the infrastructure of the university, I would try to identify the level of commitment to the program, and I would ask for some prospective plan for institutionalization. I don't think this was the case at BU; rather a lot of the supporting people left.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Kate Korman Wednesday July 3, 1996, 1 p.m.

1. Based on your experience and familiarity with the fellows and the programs, what did the program really accomplish? What are the most important contributions?

The program created a national focus on health policy training which was buttressed by the six institutions in four demographically diverse cities. Each institution brought its particular strengths to the whole of the program and chose fellows whose similar aims were to study health policy. The richness of the program lay in the multitude of choices that these institutions offered. In a sense, program directors recruited fellows for all institutions: if an applicant called the RAND/UCLA (University of California, Los Angeles) office but was looking for a program which allowed him/her to remain at home and employed, the applicant was referred to the Michigan program. Another example: RAND/UCLA offered a midcareer, year-long residential study, but inquiries often came from individuals who had just finished a PhD program or medical residency, so they might be referred to UCSF (University of California, San Francisco) which had established a postdoctoral offering. It was my understanding from people like Steve Crane (Boston University [BU]), David Perlman (Michigan), and Ted Benjamin (UCSF) that they too referred fellows to us or other more appropriate programs.

The most important contributions may be the richness of the health policy community today, which boasts 250 additional minds responsible for health policy decision making in a wide variety of venues: local, state, and national governments; private foundations and institutions; universities; and research institutes. The ripple effect of this knowledge and expertise is found in the lives and future endeavors of those whom past fellows influence. Whether it be a change in policy at the local level or the influence of a teacher for a student to pursue a change in career in health policy, the ramifications are far reaching.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

Al Williams can answer this better than I since he was involved from the very beginning, whereas I was hired after Pew funded RAND and UCLA. My understanding, however, is that there already existed loose ties between the UCLA Department of Medicine, UCLA School of Public Health, and RAND.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Those ties were individual faculty at UCLA who also had appointments in the RAND health sciences program. It must have seemed a natural progression to create programs which formally built on the strengths of each institution. RAND already had the RAND Graduate School of Policy Studies, which offered just a PhD, but no real emphasis on health (although individual students had done their research in health), and UCLA had programs in health services, community health, and epidemiology but no stated emphasis on policy. In addition, the Robert Wood Johnson Clinical Scholars program existed at UCLA, but faculty there were hungry for policy-relevant study for these scholars. From time to time senior scholars from other institutions had come to RAND or UCLA, but no formal program existed for them. Given this background the two programs were developed for the RAND/UCLA Center for Health Policy Study: (1) a residential doctoral program in health policy with students pursuing their degrees either at the UCLA School of Public Health or RAND Graduate School and (2) a residential midcareer program for scholars to steep themselves in health policy for 9 months to a year.

The residential doctoral program continues both at UCLA and at the RAND Graduate School since the courses created for the Pew program are fully vetted at both institutions. However, without outside funding the midcareer program is no longer viable.

Is there any advice that you could give to the programs that are currently being defunded? Is it only funds or is it something more?

I think particularly in the midcareer program you need fellowship monies to come. The curriculum that still exists is for health policy workshops, and they cover a broad range. There are four different courses, and they still exist. The midcareer fellows took these four courses along with others, but they were geared toward people who were not getting a degree to give them the fundamentals.

Midcareer people were leaving their professions for 9 to 12 months. They took a deep-pocket dip, and so the fellowship grant enabled them to come and to take an accelerated and specific course of study. When the money went, that was it for the midcareer program. It just wasn't possible to maintain it. We would need to pay the faculty separately (the courses were already developed) and we really needed the set-aside.

3. What was the need in the health policy community when your program started, and how have those needs changed? To what extent has the Pew program met the changing needs?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Al Williams can better answer this question. I wasn't in health until 1982.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

I think the curriculum was the key to the program's success. It was developed by a cadre of scholars from RAND and UCLA with vital insights and contributions from Allyson Davies who was finishing her PhD in health services at UCLA while working full-time at RAND as a health policy analyst. (She once said to me that she helped create the program she wished she had when she started her studies.) She was a major contributor and actually the associate director in the beginning. Not only did the doctoral students and midcareer fellows take the course work but RWJ Clinical Scholars, nonPew doctoral students at UCLA, and postdoctoral fellows from other departments also contributed to the success and necessity for continuation of the course work even after Pew funding dried up.

I don't think there was anything about the curriculum that didn't contribute or was a problem. We had people eager to teach, and we had experts to come and do specific seminars, for instance, on Medicare. Seminars were often team taught, and people looked forward to them. I think the only possible problem may have been in how often we offered a course. In the beginning we offered the courses every year so that the midcareer fellows would have an opportunity to take them; however, we stopped that later on. We offered one each quarter, and then every other year we offered a fourth one.

So, it was nothing really unique to Pew; rather, it was logistical?

Yes.

5. What was the most innovative or unique aspects of:

  1. your program design?

    The program overall or of the set of Pew programs?

    RAND

    Okay. I think it was the cross-fertilization of UCLA and RAND. I think its a unique situation where you have a think tank where you have the research ongoing and at UCLA were the teaching is such a fertile ground and has so many students available to complement the program.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
  1. methods of implementation?

    A formal agreement was created between RAND and UCLA which had not existed before. There had been this loose connection of faculty, people like Bob Brook and Bob Kane who were actually faculty members at UCLA. They also had appointments at RAND, and they were the ground-breakers. They helped to make this very difficult process happen. They are very different institutions.

  2. educational process?

    Doctoral and midcareer fellows were required to align themselves with a research project or projects immediately upon entering the program. We designated this on-the-job training, and a requirement for doctoral students was 16 to 20 hours a week. This effort produced familiarity with research methods and processes "in the lab" to apply what was being learned in the classroom. As a result, our students progressed rapidly toward the degree, where in the case of midcareer fellows, they often had a publication as a result of work done while in the program.

At the November meeting, someone from RAND mentioned this concept of "work projects." Could you speak a bit more to that?

There weren't formal grades given; however, I would keep tabs of how the fellows were doing in their research project area. We had periodic reports from the team leaders, and in a way they were graded if they were not pulling their weight; they would be nicely asked to look for something else to do. It didn't happen very often, but every once in a while it would. They learned pretty quickly that they couldn't get by without really producing. I made them accountable, as did the project directors. In the very beginning we didn't have this paper accountability.

The research projects were pretty much held with the same importance as the courses, because it was the project work that was supposed to lead to a dissertation. It was a unique opportunity for them, but some of them didn't take these projects quite so seriously. Some didn't realize that at the end there was a real product that was expected.

6. What were the biggest challenges or barriers to overcome in terms of:

  1. your program design?

    The biggest challenge seemed to be not to include the "whole world" of health policy in the curriculum. Initially, the curriculum was pretty enthusiastic, far-reaching, even

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

overreaching. One course which was just too much was eliminated after 2 years. The students couldn't learn everything. This was challenging, because as parts of courses or whole courses were cut out or restructured, some faculty found that their roles changed. However, the program design was much too ambitious in the beginning.

It seems like that would be areal tension in any interdisciplinary program.

It is. We would also have seminars from time to time and invite researchers or policy people to come and speak, and so sometimes the fellows would get a flavor for different kinds of things that had been removed from the original curriculum. That was our way to touch the whole world.

  1. methods of implementation?

    Getting the bureaucracies at RAND and UCLA to accept cross-registration of course work was challenging. It seems simple-minded, but it was actually quite challenging. That went on for at least 3 years, and part of the problem was that we had 15 doctoral student and three midcareer fellows (by the third year). At least 10 of those were taking these workshops, and so UCLA didn't understand that if it was going to be a UCLA course, why couldn't just anybody sign up for it? So we had to deal with these kinds of very political issues. But it all worked out.

  2. educational process?

    I think the biggest challenge was getting students involved right away in ongoing research at the two institutions. We had a potluck, get acquainted session just for the new Pew people, and we had, of course, the Pew people who were already in place, but we also invited a lot of faculty, and they got up and gave a little speech about what kinds of research they were doing. We did that beginning the third year, and it really worked well. It really gave the students a taste. I spent a lot of time taking students around and introducing them to the faculty and explaining the research projects. We arranged lunches for them to talk to the individuals. Some were intimidated, others were great, and yet others would get involved in too many projects. It was sometimes hard to get people to settle down. Students seemed to work better and better as time went on. Students really got right in there. They also knew their funding was for 3 years. You know there is an end to the funding, and if nothing else, that should give you a nice clear incentive.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

7. What lessons were learned about the educational process in terms of:

  1. recruitment?

    There is a variety of needs that exist in the potential student population: postdoctoral, doctoral (both residential and non residential), midcareer. No one program can fulfill all these requirements. I think that made requirement efforts really wonderful for the people. For example, Steve Crane, Ted Benjamin, and I had a booth at the American Public Health Association annual meetings for several years, and we always placed it right next to the Michigan booth because that is where David Perlman was and we had this national program with all these offerings. It was wonderful. This was one of the things that was really interesting. We were, as directors, very eager to have really good students in our program, but if they didn't fit the mold, we were very free to advise them on what the other programs were offering and put them in touch with the other people. Recruitment was a dream. It was great.

  2. degree requirements?

    When implementing a new program within an existing institution, the fellowship requirements must at least meet the needs of the degree grantor, and additional requirements toward that degree may discourage some potential fellows. We put that burden on our students with our curriculum. We required four additional courses. Eventually, both UCLA and RAND accepted one of our courses as a substitute for something that was a requirement, but the other three were really additional. Programs have very little leeway in allowing you electives. There is so much required. Some were discouraged by that, but not too many. It was a burden that most students were willing to bear.

  3. curriculum and content?

    These areas were the great selling points of the RAND/UCLA program, it was unique to require OJT with the classroom study, and potential fellows relished the idea of the meshing of these two areas. That was a real bonus that we didn't even know we had.

  4. integration of follows with other students, the university and the program?

    In the beginning there was some antipathy toward fellows who had lucrative fellowships and seemingly did not

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

have to "work" while going to school. However, this faded over time as nonfellows and faculty began to realize that these were serious, hardworking individuals who had been legitimately honored for their previous scholarship and work. They worked just as hard in the classroom and on their research projects for their dissertations as others. Midcareer fellows were just wonderful. Here were people who came out of senior positions and we were in essence taking them back to the classroom, and they became humble very quickly. It was real interesting. There was a lot of cross-fertilization between the doctoral students and the midcareer students. They often traded notes and gained understanding from each other. It was really nice. We really didn't have any snobs in our group, at least not that I knew of. There may have been experiences that I didn't hear about.

You also had open seminars that probably made a big difference, right?

Yes. They did make a big difference.

  1. relationship between faculty and students?

    Over time very strong bonds were created between faculty and students, since the students were not just faces in the classroom but integral members of research teams with faculty team leaders. That was really great, particularly at RAND. People who are doing research at RAND usually aren't doing just one project. They are usually involved in several and so they might be the leader of the research project in one area and they might be the co-principal investigator on another, or they might just being giving input as a statistician on another. And so they played various roles, and thus, the students were able to see them in different roles, and this was very good. There was a lot of camaraderie, and our faculty showed up for every award ceremony or whatever. Every year we gave a certificate for the midcareer fellows, and the faculty always showed up. There was lots of support.

  2. completion rates (where applicable)?

    We talked about this already. Al Williams will have to speak more specifically to this issues. I no longer have access to this information. My sense is that the doctoral program has a nearly 90 percent completion rate and the midcareer program had no dropouts, which is essentially a 100 percent completion rate. It was a huge success the midcareer program. We even had people who came on their own nickel in the second funding. We funded three, but their were several

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

years when we had four and one year when we had five, which was too many. These were people who said they really wanted to do this, and they didn't care if there was no more Pew money.

8. To what extent do you think the program is beneficial for those who do not finish?

The students that I remember who did not finish actually completed the course work and some research but did not finish the dissertation, which I believe is fairly common. People very rarely do I year and then say that's it. They usually make it to the dissertation stage. At least two of the individuals are in key positions to influence health policy. Al Williams will have to speak to the others. However, these two individuals have revealed to me recently that their experience in the RAND/UCLA program reinforced their work ethic (due to the combined on-the-job training and classroom study experience) and exposed them to a variety of health policy issues so that they have better understandings of the work they are engaged in today. I think that speaks well. One person actually applied for the midcareer program and the PhD program and then chose the PhD program. He did not complete the dissertation, but he made a conscious decision not to finish; it wasn't just that he drifted off. He got an amazing job offer and went with it. So, I don't think it hurt him in his career not to have the PhD, but I think it's obvious from the way he talks that those 2 years were really important to him.

9. How can we measure success for those programs where completion rates do not apply (e.g., postdoctoral programs)?

This is an unusual question. If you mean where no degree is granted, that is one issue, but the postdocs and management fellows at UCSF and the midcareer fellows at RAND/UCLA completed their programs by participation for the duration of the fellowship. However, as a measurement of success, one need only look at where those individuals are today: how they have become engaged in health policy decision making as their careers have progressed.

10. How does the Pew program approach differ from the traditional fellowship approach? How have the major outcomes differed?

Perhaps the greatest difference is the variety within the ''set'' of programs from residential to nonresidential doctoral programs, to midcareer and postdoctoral study at six institutions which offer

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

varied research focuses: social policy, health services research, health policy management. I don't know how the outcomes have differed. One way may be that people in the clinical scholars program become clinical faculty; people who are Kellogg fellows sort of stay where they are but then go off to Turkey and China and different places, but as I see it Kellogg fellows sort of stay in the career they were in, and I think the Health Policy Fellowship Program offered more opportunities for people to either change career or really advance in their career more quickly.

11. If you were going to give advice to another university attempting to initiate a similar program, what would you say?

Get strong commitments from faculty and the administration because not everything works the first time around. Be flexible. Get the best students. Recruitment is key.

12. What is the Pew program "legacy" in terms of.

  1. health policy?

    Former fellows are involved in health policy decision making at all levels, federal, state, and local, as well as in health policy research at universities and public-and private-sector research firms. I really think that is the legacy. What they bring to those areas: they are smarter, they are better informed, and they have a broader viewm and this is true of all the programs, not just RAND.

  2. education and training?

    Former fellows have faculty positions in universities throughout the country, and they are training new health policy analysts based on the Pew training that they themselves received. Additionally, faculty from the six programs have moved from institution to institution and replicated parts of or the whole Health Policy Fellowship Program curriculum and study methods to their new institutions. A prime example is Joe Newhouse at Harvard, who took the RAND program in its entirety, tweaked it some, and put it into place at Harvard. That is seeding the field.

  3. your institution?

    It is my understanding that the RAND Graduate School and UCLA continue to train health policy students using the program set in place by Pew funding.

13. Are there important issues you feel that this interview does not address? If so, please feel free to add comments and/or concerns.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

The pride I feel in the accomplishments not only of "my" Pew fellows but also fellows from the other programs! I think its really a wonderful feeling that I didn't know was going to happen for me. When I go to a meeting, like AHSR, and I see them receiving awards, presenting papers, leading discussions, and being on the planning committee, and then at APHA I see the same kinds of things going on. And now that I'm in Washington I see a lot of them even if they are not in Washington working. They come here, they are leading panels, and it's wonderful. And, of course, the network is phenomenal. The alumni connect with each other. People looking for jobs can just call another Pew fellow, even if they don't personally know them, and more often than not that common bond sets the stage. The whole thing happens. It's grand. Former fellows are very proud to say that they were Pew fellows. Pew ought to be very happy about that. How do you quantify that? You can't.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Steve Crane Tuesday July 23, 1996, 9 a.m.

1. Based on your experience and familiarity with the fellows and the programs, what did the Pew program really accomplish? What are the most important contributions?

The most important contribution has been to create a network of people who continue to interact and who continue to have important positions in the policy system, both public and private policy. These people understand and know each other and are able to get things done. The policy system does not work through formal authority channels but most often through informal channels, and it's not what you know but who you know that counts; the networking was really important. Secondly, I think that the Pew program got some people involved in health policy from perspectives that would not otherwise have been present. This relates a lot to what the Boston University (BU)/Brandeis program specifically intended to do. One of the core concepts for the program was to create a shortened, highly focused program that would reduce the time and cost barriers so that people who were already actively engaged in health care concerns could come back and get that doctoral education and go out and be effective. More than just getting busy people, what we wanted to do was to get people from nontraditional sectors like business to come in and to begin to create a different type of cadre of people who could span both public-and private-sector concerns. At the time that this program was started there was an awakening of the notion that in fact if change was going to occur (and we had no sense that it was going to occur as rapidly or as dramatically as it has), we had to get policy issues out of the public sector and involve the private sector. So, the whole focus, at least of the Boston University side of the BU/Brandeis program, was to try to create links with the business community where attempts were already being made to control costs, improve quality, and gain a handle on what was going on in the health care system. I think that was a pretty significant contribution of the program in total as well as one of the major contributions of at least the BU side of the BU/Brandeis combine.

We can also say that we identified a very bright group of dedicated people who were trained and connected to some of the major policy leaders in the country, and we probably shortened the amount of time that it took them to become effective in the system. The other great thing of the Pew program was the many different opportunities the stu-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

dents had to meet the major leaders in the field, in both the public and private sectors. Those relationships gave the program a weight of influence in the system and gave the students access to those foci that they could use to influence policy more quickly.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

In terms of how our specific program developed, as a result of the competition and as a result of the institute at BU being selected to work with Brandeis, the notion there was really to see if the business community could be involved in this in some way. The particular center at BU that was involved was the Center for Industry and Health Care. A very important part of the history of our program and I think the history of the Pew program is the fact that there were really three programs at BU/Brandeis: the doctoral program, which we always talk about; and the Pew Corporate Fellows program at BU that brought together major industry leaders involved in health care issues to Boston twice a year and created a network in the business community, the results of which are still being seen today in terms of the corporate coalitions that have been created and the analyses that continue to be done on health outcomes. I think these things are attributable to the types of discussions that took place not just at BU. I think we were a catalyst for a lot of that. It is a very important part of our total program. Likewise, Bruce Spitz's Community Program at Brandeis was very important. The notion was that we needed to create change not just at the intellectual level but at the institutional level as well. Bruce's efforts to try to get communities to move and to link together business and the public sector was highly innovative and was a precursor to the coalitions of today and the more population-based research being employed today. That was a very important part of the stimulus, particularly from the BU side.

With respect to the continuation of the program, the program has ceased at BU, and it ceased not too long after I left. The major problem at BU was lack of university institutional support. There was a lot of support at the Center for Industry and Health Care for the program, but what we were trying to do was very difficult. We wanted to create a doctoral program that would get you in and out within 2 years through a challenging process. We wanted to give students a great degree of flexibility in the courses they chose so that they would waste the least amount of time taking cours-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

es that someone thought were important but weren't directly relevant. We wanted to take in students who already had an advanced degree and substantial experience and didn't have to learn what they wanted to do with their lives. We tried to identify and choose people who knew exactly what they wanted to do and who could find what they wanted here at BU, and in that way get them in and out very quickly. We ran up against some big problems in terms of typical university structure: first, where to house this program. Happily, BU had the University Professors Program that allowed for a much more loose set of courses. The problem there was that this program was designed specifically for nonpolicy science types. The program was more in the literature and arts area, so they had a hard time understanding health policy because it wasn't a specific discipline, and they had an even harder time understanding the practical orientation and the need for these people to get out quickly. As Sol Levine, the academic director of the program, put it, the philosophy was that students ought to be able to "wallow" and pursue interests that they wanted to gain the knowledge that they need for a doctoral experience. I certainly subscribed to that in my own doctoral program, but that is not what this program was about. That is not what these students were about. There was a real culture clash between the academic directors of the program where we put the students and the students themselves. That took a lot of work.

Another aspect was that BU wasn't committed in a financial way because there was no school, there was no program, there was no department, and there was no faculty. We borrowed from everyone to put it together. Without that hard, fast focus of a strong department, the program was almost invisible to the university.

Thirdly, the students were nontraditional university types. They weren't planning academic careers; they were planning careers in business and public service. They were interested not so much in peer review but in having an impact on the policy system. I think the university, the traditional academic system, looks somewhat askance at these students at BU. I think the situation was different at Brandeis, where you had a well-established academic center and well-established academics taking all the students under their wing and providing protection. That was very hard in the medical center, and probably the greatest failure of our program was not to get Dick Egdahl more invested in what was going on. But, his interest was much more on the Corporate Fellows Program. He made a great contribution there, and

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Sol [Levine] and I, and later Jon Howland, I think, made a very significant contribution on the doctoral side.

Prior to the Pew program had there been in place a corporate program on which your structure was modeled? Are there now any programs that perhaps drew upon the success of the BU program?

The BU Corporate Fellows program still existed at BU until at least this past year or so. It has transformed itself in various ways, and Dick Egdahl has retired, so I'm not sure of its current status. Certainly at BU that tradition carried on for years after the formal ending of the Pew grant to BU. What happened at other places I really don't know. We certainly promoted this program to Pew. But Pew didn't really promote the program any further than listening to our reports, and because perhaps the centerpiece of the program became the doctoral program, nothing much was done with the Corporate Fellows program. I think it was a lost opportunity. It's ironic because one of the things that Pew really liked in the original proposal was this notion of doing something with the corporate sector, these groups of people who could benefit from the knowledge of health policy research but who generally had little access to it, and likewise to give access inside corporate knowledge to the academics. I think there were some lost opportunities there. I think a lot was accomplished and a lot of networking was done through BU, but being a fairly aggressive person on these things, I think not as much was done as might have been.

How do you think the connection with Brandeis contributed to or hindered the success of the BU program?

My impression of the BU/Brandeis relationship is nothing but positive. I think we worked very well together. The students had access to courses at Brandeis, but I think the greater problem was the Brandeis students who wanted to take more courses at BU but Brandeis said no. From our curriculum's perspective, we wanted our students to have any of the courses they wanted, whether it was at BU or Brandeis. We were able to negotiate this successfully. Brandeis, with its much more structured curriculum, only allowed students to take one or two courses, if that, outside of Brandeis. I think more of the frustration was on the Brandeis side rather than on the BU side. I think the BU students really appreciated the flexibility that we gave them. It was little bit difficult because they would come into the program and sort of have to find their own way, but once they did they appreciated the chance to do what they wanted to do.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

We provided as much advice and guidance as we could; we forced them to take more statistics and research methods classes than they probably wanted to take. They certainly were not bereft of intellectual guidance. As the program progressed we helped students to develop a social science core (sociology, economics, and political science) so that they had a base from which to work, but the whole concept of this program was interdisciplinary. Ironically, the model that we used was Diana Chapman Walsh's. She completed the University Professors Program and then worked very closely with Dick Egdahl. Diana's role in all of this is very important. Interestingly, Allyson Ross Davies, who got the RAND/UCLA program going, was a very close friend of Diana's and modeled some of what was done at RAND and UCLA on what was done at BU with the University Professor Program. Allyson left the program fairly early, and so that legacy was lost. There is a lot of interconnectedness here in terms of the start and operation of the program.

Some people mentioned the tension between having the flexible curriculum and getting the needed skills. Can you speak a bit to that?

There definitely was a tension there. The program wasn't as assertive with the students in saying you must take this and this. We weren't sure what the "musts" should be. As we went along in the program and we saw what students did, we saw then what we needed to do. Toward the end we probably had a much better sense than in the beginning of what we needed to do in terms of requirements. We were learning. Certainly for some students, the lack of our knowledge early on may have created some gaps.

Secondly, we knew that some of our students had gaps. We told them that they had gaps and we told them what they had to do and for whatever reasons some chose not to follow our recommendations. From our perspective they ended up with gaps; however, I'm not sure whether from their perspective they ended up with gaps. That is a faculty issue. Faculty often want to create students in their own image. Yes, there are tensions, but what we broke out of were the traditional doctoral program of 5 years of suffering and 4 years of writing.

3. What was the need in the health policy community when your program started, and how have those needs changed? To what extent has the Pew program meet the changing needs?

I answered that in part by saying that there was a real need for two fundamental things to happen. The first is an

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

interdisciplinary approach. We wanted to get away from the strict departmental lines. The Robert Wood Johnson Scholars program that exists now at Brandeis was really inspired by the Pew program and the emphasis on interdisciplinary training. They have taken a slightly different track by taking people out of disciplines to put them together, and so interdisciplinary is defined as having representatives from each of the different disciplines in the same room. Our approach to being interdisciplinary was that each of these disciplines had to be represented in each student. We felt that the problems in the system were not unique and discipline oriented; they weren't just purely economical or purely political. To have effective people we needed to have people who had true interdisciplinary training. That was point number one. That is why the university program was so wonderful. We were able to allow the students to focus just on the interdisciplinary stuff.

The second need that existed in the system was for the public, private, and not-for-profit sectors to work more closely together. A lot of programs produced people just for the public sector or just for the academic sector, but rarely just for the private sector. What we wanted to do was to have someone who had not only the vocabulary but also the knowledge to cross-walk academics, public service, the private sector, and the nonprofit sector. That is why we tried at BU at least to pull from each of those sectors and to put these people in the same room and to give them the perspectives they needed to go out and in true interdisciplinary style solve some of these very complicated problems.

I think those needs still exist, and most of the academic programs are still turning out discipline-focused individuals. The Heller School at Brandeis is an exception. What we need are people who can understand problems from a multiplicity of viewpoints and who can create solutions that cut across disciplinary and sector perspectives. We are still short on people who can do that.

When the program started there was a real emphasis on training health policy researchers and health policy makers, and it seems that today the programs that are getting funding have changed that emphasis, at least in name, to health services researchers. Why is that? What is the difference between the two terms? Why is it that people seem to flip between the two terms?

That is a good question. There is a real difference. Let me lay out a continuum. On one end of the continuum you have disciplinary research (economics, political science, and

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

sociology). On the other end of the continuum you have policy-related research (defined generally as finding research-oriented solutions to practical problems). Between these two you have health services research. Health services research is, in a way, a systematic and rigorous analysis of how health care is delivered and the consequences of the delivery of health care. That borrows from disciplinary research, because it's often done based on theory. Its purpose is to contribute to a literature, and its main contribution is to help us understand how something works. Policy research, by contrast, takes all that information but then applies it and focuses more on solutions and change as opposed to understanding what has happened in the past. Health services research produces the information. You then take the health policy researchers to figure out where we want to go.

I think the reason why the program has drifted from health policy back to health services research is because health services research has a disciplinary base and traditional academic viewpoints prevail. Health services research is still more accepted than health policy research. Most universities haven't got the foggiest idea what to do with someone who wants to go to a state legislature and work on a bill; that is not considered academic. I think there are institutional forces that sucked those programs back to health services research. Furthermore, there is a tremendous community of support in health services research, and you tend to go where your friends are. Most of the policy-type programs tend not to be oriented toward health care but rather public administration, public policy, and political science, all a different set of institutions than we chose to involve in this program. The field itself is not well defined or well accepted. So, given all that ambiguity and uncertainty I'm not surprised at the shifting back. Again, the standards for the field have become publications and peer review journals. I see nothing wrong with that, but policy makers don't read peer review journals. They read editorials in the New York Times. To borrow a phrase from Bob Blendon, it's more important sometimes for people interested in these problems to publish an Op Ed piece in the New York Times than to publish an article in the New England Journal of Medicine. The academic institutions have slowed us down, again.

How do you think the Pew Health Policy Programs changed this outlook, if at all. Were the programs merely a blip on the screen, or will they be readily institutionalized and replicated?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

I think the programs are a blip on the screen. I could also say that the academics won. The RAND program was always much more academically oriented than any of the other programs. It was a good program. Why the programs were cut off is another study in and of itself. The Michigan program structurally should have been as practically oriented as the BU program, and certainly the students who were at Michigan were closest to the types at BU/Brandeis. The problem with Michigan was that the people teaching the programs (and I was one of them who helped write the Pew grant) were all academics. Very few of them have set foot in the real world, and again, their goals are perfect academic publications. At BU, perhaps because of some radical notions that some of us had, we were willing to push the envelope and say that your dissertation came out and was used by policy makers; this is a triumph. Even the Pew newsletter, with all due respect to Marion Ein Lewin, counts the number of peer-reviewed publications. What message is that sending? We missed the point here. The traditional academics won, and this is not surprising. It was a huge battle, and what we were dealing with were institutions caught up in a lot of tradition. They were structure and process oriented, not outcomes oriented. So, there is a tremendous need for another Pew program, not so much to change health policy, although that is necessary too, but to begin to change academic institutions. We were, in the BU program in particular, a real threat to the academic structure, and there needs to be more of that.

4. What was it about your curriculum that contributed or did not contribute to your program's success?

I spoke a bit about the curriculum already, and the other important part was the weekly seminars. We would come together in what is termed now in the literature as a learning community or learning conversation . We would all try to take a problem and solve it from our different perspectives. People who were into economics would approach the problem from their perspective, and people from political science would approach it from theirs, and we would share views and learn from each other. That was just a tremendously stimulating activity that we could have made even stronger than it was. Nonetheless, it was an important part.

What is an interdisciplinary dissertation? How can you bring someone through an interdisciplinary program and have them produce a product that can be used, as you mentioned above, by policy makers?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

To explain this, one needs to begin with the question, What is a disciplinary dissertation? A disciplinary dissertation is that you start with a problem statement which is, 10 to 12 pages to say what question you're going to address. Then your second part or chapter invariably is a literature review and theory discussion describing how you're going to approach the question. Then you do your analysis and make a contribution to the question and perhaps the theory as well. Because theory is fairly complex, you are encouraged to deal with the most narrow part of a problem or theory.

An interdisciplinary dissertation says that you begin to approach a problem not just from the perspective of one discipline but perhaps from several so that there may be two or three disciplinary frameworks that you would look toward to solve a problem. A simple example: One of the scenarios/ questions that I would ask students when I was interviewing them for the program was to assume that they were the assistant to the mayor of a city and that there is a real problem with garbage collection. Some neighborhoods aren't getting garbage picked up, it's mounting up, and all kinds of issues are arising. How would they undertake analysis of that problem? Invariably even students who come in and say they have an ecumenical view of the world will find a particular cut on that problem that they would use as their lens. They start talking about either resource allocation, cost, the politics, the sociology, or the psychology. I could tell pretty quickly what people's natural disciplinary perspective was based on how they answered that kind of question. In terms of a dissertation it would be the same thing, only in reverse. You would be posing your problem and then discussing the different aspects of that problem and how it could be enlightened by different disciplinary contributions. The other side of it was to have people who on the dissertation committee held divergent views to test, probe, prompt, and push students to think about things in more ways than just one. That was the other part of the dissertation process: having an interdisciplinary committee.

Do you think that faculty in this type of program should be made up of economists, political scientists, and sociologists, or should these positions be filled with professionals who have the type of interdisciplinary degrees that Pew is granting?

That is an interesting question. The kinds of people that the Pew programs were graduating are rare. Universities recruit straight disciplinary people. I, myself, am a product of an interdisciplinary background, and maybe I was using

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

my model as much as anything. My program was essentially a health policy analysis program, and the core discipline, if you could define one, was economics with a lot of overlay of sociology, organization theory, and law. I ended up teaching politics without a graduate course in politics to my name. I wasn't a bad model. Another anecdote: When I went to Michigan in 1970 and 1971 the Institute for Public Policy Studies had just been created out of the old Institute for Public Administration. The whole purpose of that program was to be interdisciplinary. Their definition of interdisciplinary was to get a political scientist, a sociologist, and an economist to have an office beside one another. It just didn't work. Those folks never talked to one another. Eventually, they got it right, but it took a long time to do it. I would like to see good strong disciplinary people around the table, but they have got to be committed to listening and learning from each other. There are some disciplinary people who can do that, and there are others who cannot. A lot of disciplinary people only want to say ''my discipline is right and the only way to go.'' Those are the people I don't want to have at the table. I think it takes a couple of people like myself and the Pew graduates to be able to ask the questions to bring those disciplinary perspectives out, but I don't think it should be all one or all other. But, I do believe that at a minimum there should be straight disciplinary people there.

5. What was the most innovative or unique aspects of:

  1. your program design?

  2. methods of implementation?

  3. educational process?

I've already spoken about the innovations.

6. What were the biggest challenges or barriers to overcome in terms of:

  1. your program design?

  2. methods of implementation?

  3. educational process?

The biggest challenge was how to get students started on a dissertation in the first week of their program when they haven't the foggiest idea of what a dissertation is. This has to be successful if you're going to get people turned

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

out in 2 years. We succeeded in some cases but not in others. I think we solved the problem related to trying to figure out what to do for a dissertation, because we tried to screen people as best we could before that. We really wanted to get people who knew what they wanted to do. Fortunately or unfortunately, some people came in and wallowed until they found other things to write dissertations on. We also celebrate that. That was the toughest thing: how you formulate a question. One of the other big intellectual challenges that I found myself addressing all the time was teaching the students that they came to a doctoral program not to learn what they were to know but to learn what they did not know. That was, hopefully, one of the big contributions I made to the intellectual development of the students: to teach them that it's not what you know, it's what you don't know that is important. And, then I wanted to teach them how they knew or didn't know something, which was the application of research methods and analytical thinking. The other thing, our students averaged 12 to 14 years of experience when they came into the program. They had pretty well-established beliefs about the world. The first few seminars were always really tough because people who had worked up a particular view or perspective would come in and would express it and expect the rest of the world to believe it just because they said it and believed it. In the academic world, that is not good enough. You have to explain not only what you're saying but why you're saying it. To get students to understand that they have to leave their cherished beliefs behind or to keep only those that have some type of empirical or rigorous proof was really tough. It was tough because in traditional programs you have people who are younger, who have less experience, and who are less set in their ways. These were savvy, smart people, and once they got it, they got it and they were great, but getting them to that point was tough. We essentially said that maybe what they believe in wasn't right. I'm convinced that one of the other things this program did was to value the mature learner. We proved that there is a place in an academic institution for the mature learner. The discussions, the group, and the experiences that we had were just tremendous. That maturity factor was important. We as faculty sat around the table as much as the students, and the students who were there were as much faculty as we were. It was a true community of learners. Of course, there was a hierarchy, because at the end of the semester we were able to tell them whether they were right or wrong. One of the major

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

problems the students had was fitting into the role of a student, putting themselves in a totally dependent, subservient role. For many this was very difficult. Many were highly accomplished people with big titles and far bigger salaries then we who were faculty. And, to come in and be talked to, lectured at, evaluated, measured, probed, etc., was a very uncomfortable experience. Some people adapted to it and others fought it. The ones who fought it had the toughest time.

7. What lessons were learned about the educational process in terms of?

  1. recruitment?

  2. degree requirements?

  3. curriculum and content?

  4. integration of fellows with other students, the university, and the program?

  5. relationship between faculty and students?

  6. completion rates (where applicable)?

I think the recruitment really is the critical issue. We were blessed because we worked very hard at it and because we had some very good applicants. The choices were always exceedingly difficult. I think that it has been said often that the quality of a program is presumably measured by what you contribute to a student. Actually it's very easy, because if you take a good student in you'll turn a good student out. I'm not sure we transformed many students. That would have taken a lot longer than a 2-year program.

A couple of lessons: (1) Two years probably was ambitious; 3 years is probably more realistic. But I would keep a 2- to 3-year time frame because if you start a 3-year program it will become a 4-year program. I think that was a lesson. (2) I would now give a much greater emphasis to research methods and slightly more attention to quantitative methods. I think overall we did OK there. There are two or three things that you get out of a doctoral program. One is analytic thinking, and that is where the research methods come in. We really need to do a lot more on research methods, and that means Campbell and Stanley stuff and not just statistics. Second, we need to help students develop that core disciplinary perspective, not so they become a disciplinarian but so that they have a good strong starting point from which to

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

observe other disciplines and other problems. Third, I think we need (we should have) developed more of a core faculty for the program. There was a structural failure, and it was probably on the part of BU to ask for this. Dollars weren't available to pay faculty to become involved in this type of program. It's one thing to provide dollars for students. I think you have to buy people off in an academic setting. There are too many competing demands, and some dollars for faculty would have been very helpful as an inducement, which is sad but perfectly understandable. Fourth, I think that in the end Brandeis probably had a better approach to tying people to specific research projects so that the students didn't have to do primary data collection and they could work very well with a mentor and other researchers. People who worked on those kinds of projects did very well, and I think for a quick degree program that's what you want to do. If you want to learn how to collect data, then I think you ought to go to a traditional doctoral program and take longer to do because it takes longer to collect data. I think a lesson was learned there. I would give as much if not greater emphasis in the future to the policy seminars, to talking with one another. That community of learners is a critical part of all of this. There are some things you have to do on your own, but it's the group process that makes a big difference. Also, we thought that $10,000 was a generous stipend, and as one of my favorite students said when he came into the program early on, "You know we priced the program just right. It was too little to live on but too much to turn down." I always appreciated that phrase, and I've used it a lot, but I think we did a bit of disservice to our students by forcing otherwise senior people to live a student's life. You get what you pay for. I'd put more money into a stipend and help these students, particularly those with families. It was very hard for the people with families. It's hard enough to do a doctoral program without our making it more difficult. That's the monastic view of academics. I also think that we should have tried to bring in more people from the outside than we did. I think Marion's contributions to the program and the national meetings that we had were immense. We used to have two or three, and then we reduced it down to two and ultimately to one. I think that perhaps one of the most important things that we did was to have those joint national meetings where people from the different programs came together and met with outsiders. I knew it was an expensive process, but, boy, did that add a lot for all of us.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Was that unique to Pew or was that something that you had heard of before?

I think we did it more than some others, but the Robert Wood Johnson Clinical Scholars program did it, and that is something that is done because you have people you want to get together, and all the fellowship programs I've been involved with subsequently have had that as an essential part, but typically only once a year. I think that people should come together at a minimum twice a year to get to know one another and to become comfortable. The problems now are not institutionally oriented, they are not state oriented, they are nationally oriented. The problems cut across all the political boundaries. You need to have people who can take that view come together, and the more we spend time just in our own little cells sitting around with faculty from one institution, we are missing the fact that the problem is interdisciplinary, intersector, and national, not local, in scope.

Kate Korman spoke about the joint recruitment effort all the programs had. Can you speak a bit to that?

We pioneered that with our efforts to recruit people at the American Public Health Association's (APHA) annual meetings. We understood that each of the programs had a unique approach. What we wanted to do was to go out and try to get people who would be good for the program in general and then get them slotted or placed in the program that would be most appropriate for their interests. For the younger students, the ones more interested in a traditional academic career, RAND/UCLA (University of California, Los Angeles) was better for them. For the people who were more business oriented or problem oriented, the BU/Brandeis program was more appropriate. On the other side, the Michigan program was an on-job/on-campus program where people who couldn't get away from their work or didn't want to could get their degree. We all had something to contribute. We were competitive only in that we all wanted to be seen and do well by Pew and to stand in the best light. And that is very unique because most doctoral programs don't do that.

Do you want to say anything else about the "set" of programs, how they complemented each other yet were very different? And how that benefited the health policy "scene/field"?

I think one of the great strengths of the program as a whole were the individual faculty members involved. Can

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

you imagine sitting down in a room with Phil Lee, Stu Altman, Stan Wallack, Dick Egdahl, Al Williams, Bob Brook—I could go on and list dozens of names—and be able to go up to them and talk with them, think and discuss with them for a day or two, and then go back to your institutions and do that on an ongoing basis, or if you want, call someone up in another institution: Phil Lee, Assistant Secretary for Health; Stuart Altman, Chairman of ProPAC (Prospective Payment Assessment Commission); and Joe Newhouse at Harvard, to speak with the people who are writing the articles, who are changing the world? Incredible. It was just incredible. So often doctoral students get stuck with assistant professors who are more interested in fighting for tenure and their own existence. The Pew program on the other hand was just unbelievably rich, and the commitment that those senior people had to this group of students was extraordinary. That is something else that could be replicated by another program. There is still a strong need to get people from different institutions together. That phrase "inter" is really so important, not "intra" as doctoral programs traditionally are. They only look within themselves. This was ''inter'' in every respect of the word.

Almost without exception, every program director has mentioned that to have a successful program you need to have someone like Steve Crane. How does an institution go about finding that commitment and dedication?

It's a flattering question, and I wish I had a good answer for it. A couple of things: Maybe one of the great contributions of BU was to have had a culture that would allow someone like myself to come in and gave me free reign to do what was necessary. I particularly appreciated the freedom and the tremendous resources that BU offered. This really made a big difference in what could be done. I do think it takes an individual or a group of individuals who believe very strongly in something to move a system, particularly one that is as rigid as an academic institution. There were lots of times when the answer came back, "No, that's not the way we do things," and we did it anyway. Perhaps because structurally I wasn't in a tenure track, my incentive was to keep the program going because that is where my pay was coming from. That may be a little bit of it, without demeaning it to that level. Certainly not having someone holding a tenure over my head made a huge difference. I think it takes someone who can see the possibilities and who works well with other people. I don't think there was any-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

thing unique about me, but it does very much help to have someone who has the vision and, number two, has the time to spend on this. There are a lot of people like that, I found them all over the place. The trouble is they were more constrained than I was in terms of their institutional setting or structure.

8. To what extent do you think the program is beneficial for those who do not finish?

This is a good question. I would like to think that people who didn't end up with a PhD degree still got a tremendous amount out of the program. We often said, and I don't know if this was right or not, in the end what is important is not whether or not you get a degree but what you learned. Nice idealized statement. We could say it though, because our goal was not to have someone go on and get a tenured track position in a university. If that was your goal and you don't get your PhD, then you have failed miserably. Our goal, however, was to produce highly capable people, people capable of effecting great change. I think we did that. I am sure that there is great personal frustration and disappointment at not being able to complete a dissertation. But in reality, while I'm sure that the academic program bears some responsibility for that result, a lot of responsibility remains with the individual. Either they got sucked back into their professional lives too quickly, they unfortunately couldn't create the time to do it, they refused to undertake the discipline to sitting down and organizing their lives so that it could get done, they couldn't quite get the notion of a single question to answer, they ran into tremendous problems with data, or they wanted to prove something that they couldn't prove and didn't want to do it if they couldn't prove what they wanted to prove. There are a whole variety of reasons for not finishing, and that's frustrating. But every one of those people, whether they finished or not, ended up as an equal part of the club. I. don't think anyone in the program discriminates based on whether you finished or not. We revel and celebrate completion, but no one is considered a second-class citizen for not having finished. That's pretty amazing, because again, I think out focus was more on what you're going to do, not who you're going to be.

Do you think the PhD is an appropriate degree?

Ah, that's the big question. We ran into a lot of problems because it was a PhD as opposed to a DrPH or as

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

opposed to a super master's degree. What was important about the level of education that we were talking about were a couple of things: (1) the emphasis on research methods and being not only a consumer of research, which you become with a master's degree, but a producer of research, which is signified by a PhD. People needed to have that knowledge whether or not they needed the degree for their careers. For some people the PhD was important because when you're out there battling in the world of health care that is so degree conscious, you really need to have the highest degree possible. So, from a marketing point of view, the PhD was really important. I think Michigan folks will feel a little badly about the DrPH because its considered to be a professional degree, which in the end is probably a more appropriate degree for this program, but it doesn't have the same cachet, doesn't give you the same entree or access to academic faculty positions, and doesn't give you the same identification when applying for research grants. So, on balance, I think the PhD has real value, signifying both the level at which you are working and your capability for independent research, and it puts you squarely in a position where you can look a doctor in the eye and not blink. Every other degree less than that is not as strong in that dimension. Whether you absolutely needed a PhD to be successful I would say it should be more competency based than degree based.

9. How can we measure success for those programs where completion rates do not apply (e.g., postdoctoral programs)?

First of all, I think the presumption of this question is all wrong. If you are defining success implicitly as being a degree, I don't think that is the right measure of success. I think again we are slipping back into the same academic mentality, like asking how many peer-reviewed publications do you have. I object to the question, but the answer to both a degree and a nondegree program is the same. It's not a degree, it's not a peer-reviewed publication, it's what you are contributing to the field. That is a soft measure, but one of the things that works very well in tenure review is when you take the number of references made to your work in the field. You're not measuring the quality of the work directly, but you're measuring it indirectly by how it is used. To me that is a tremendous measure. To the extent that the people we produce are in demand in the system, I think that's a measure. Even more important is the extent to which people have assumed leadership positions, and I don't mean to count titles, and are actively striving to make a difference, being innovative, and coming

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

up with new ideas. All of that needs to be looked at to measure success. This program was all about change. It was not about training policy makers but was about training change makers. Change takes interdisciplinary people who can talk to one another. That was what this program was all about. One of the defining purposes of the Pew program, and one that set us apart from other PhD programs was that while we were striving to create people who could produce policy, our emphasis was more on creating people who could use policy. In essence we were creating change agents. Certainly at BU and Brandeis this was true. We focused on the intersection of the three worlds of the public sector, the private sector, and academia. We trained people to straddle the fence. Change comes from the interaction of these three sectors, and we knew that and we emphasized that. Focusing on that intersection effects lasting change. We are seeing this borne out today in the health care sector.

We have spoken about how the Pew HPFP goes against the very nature of a traditional PhD program, and one of the ways it did so was that the faculty were eager and had incentives to see that the fellows got their degree fast and got out. In traditional programs its almost a challenge for the students to prove that they deserve the degree. How do we address that issue?

Again, that is partly faculty recruitment. You don't want to have involved in the program a faculty member who believes that no one is good enough to get the degree or that they should suffer for 10 years before getting the degree. There are a lot of those people out there. At the same time there are different kinds of students. There are students who say they want a quick degree just for the knowledge, to get in and get out and not to have their dissertation be their life's work. Those are appropriate students for a short-term program. We had some other people come into our program who said they wanted a fast track but who were real academics and who really wanted or needed to take the time. We couldn't track them into something faster. For those students it ended up being a mismatch. It is a recruitment issue on both the faculty and student sides, and I think the best approach is truth in advertising. The upside and the downside are to do some heavy-duty counseling in advance and monitoring as you go along.

10. How does the Pew HPFP approach differ from the traditional fellowship approach? How have the major outcomes differed?

I think we said enough about this.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

11. If you were going to give advice to another university attempting to initiate a similar program, what would you say?

We've already discussed this.

12. What is the Pew HPFP "legacy" in terms of:

  1. health policy?

    I'll just use one example: John Dopkeen, a student at BU. He came from Boston City Hospital, knew the importance of incorporating the corporate sector with the government sectors and academia to understand health care policy problems. He made a tremendous contribution to the field in this way. The RAND/UCLA program did very significant work with satisfaction, the Brandeis program contributed at a more micro level, but still a very significant level, to issues of Medicaid, reimbursement, and prospective payment. Overall the Pew programs made lots of contributions on many different levels. The key there is not so much relating to a particular policy finding but rather the community of people who now contribute to ongoing change in policy.

  2. education and training?

    We have already identified a whole bunch of legacies in education and training: (1) the quick PhD that we pioneered, and we probably found as many problems as successes but we learned a lot; (2) the interdisciplinary approach; (3) the focus on change; (4) the intersector approach (the public, private, academic, and not-for-profit sectors); (5) the importance of discussion; (6) the need for emphasis on research methods and analytic thinking; and (7) the ability to take mature learners into the system and have them succeed. These are all very important contributions that we made to health policy education and training.

  3. your institution?

    I think the Center for Industry and Health Care probably is the last bit of legacy for BU. With Dick Egdahl retiring now, I'm not sure what will be left at BU. I think there is a tremendous legacy at Brandeis, but I assume you've gotten that from interviews.

The Pew legacy is really trite but true, and you've heard it before: the people who were trained. Even though we took in more mature students, some of whom are now in their 50s, there is still a huge crop of people coming along who are

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

going to make a huge difference. I'm not sure that we've seen yet the contributions to health policy that the Pew students are going to make. It takes awhile. But as those folks come up and go through leadership positions, I think we'll see more and more. That emphasizes the important of trying to keep this group together as well as having them interact with some of the other program fellowship people who are around: those from the Johnson program and the Kellogg programs. All these people are unique. They are leaders, and they are going to make a contribution. The group effort is only going to enhance their already evolving skills.

It was a great time. It was Camelot.

The Pew Charitable Trusts really need to understand what positive feelings everyone has toward these programs. They have been very supportive. They are our colleagues in all of this. They hung with us when things were rough. They celebrated our victories. The Trusts deserve a lot of commendation for doing this and sticking with it. There are as many lessons to be learned on their side about how you deal with fellowship programs. This is a good point to make for other foundations. If this model is to be replicated, what the foundations are going to need to know about is not how it works at our end but how it works at their end. I could suggest a whole variety of lessons that I think foundations should have learned from this, and we can discuss this further at another time.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Al Williams Monday July 8, 1996, 1 p.m.

1. Based on your experience and familiarity with the fellows and the programs, what did the Pew program really accomplish? What are the most important contributions?

We've produced a large number of successful PhD grads and a smaller of number of midcareer folks who made substantial shifts in their professional interest and orientation to health policy. In addition, we built a strong curriculum structure which is continued.

2. How and why did your specific program develop? To what extent will your program continue now that Pew funding has ceased?

The program was born out of a long term relationship between UCLA (University of California, Los Angeles) and RAND. When Pew sent the letters way back, they sent a letter to both UCLA and RAND and basically said that we should get together and write a proposal. So that's what we did, and that proposal was obviously successful. And, yes, it continues in the sense that we have joint courses taught by faculty at both the RAND Graduate School and the UCLA School of Public Health. What doesn't continue in any consistent way is a rich fellowship. We do have fellowships, but they are not as reliable and not as rich as they were in the Pew days. I think we tend to get more MDs who have the means to support themselves taking the programs. Thus, in recent years we've produced more MDs/PhDs, some of whom have had the RWJ Clinical Scholars support.

3. What was the need in the health policy community when your program started, and how have those needs changed? To what extent has the Pew program met the changing needs?

The need was clearly expressed in terms of inadequate numbers of broadly trained health policy people. Most of that need and demand was in the public sector, and over time the public sector, if it has not shrunk, has certainly not expanded, at least on the federal side. More people are going into private activities in some degree in the state government. Our folks tend to be quite strongly trained health policy researchers, and there are as many positions as there were before. We clearly have done well in meeting those needs in the sense that our fellows have gotten good positions. But there just aren't as many positions except in the private sector now. There is clearly a movement toward private-sector jobs.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

4. What was it about your curriculum that contributed or did not contribute to your program's success?

The main thing was this broad integrated policy research that was going on at RAND and, at the time, to a somewhat lesser extent at UCLA. That continues and has grown so that is a base on which to build a strong program. We also had in the RAND Graduate School a structure of policy workshops that was well suited to the kind of people we attracted. It was more collegial and less academic in terms of the culture as opposed to the content, and so I think the fellows did really well in that environment. But the strongest thing was the long-term relationship between RAND and UCLA and the substantial body of work.

5. What was the most innovative or unique aspects of.

  1. your program design?

    Probably what we call workshops or seminars on broad policy domains with a set of modules dealing with particular things were the most unique. For example, there is one on technology, regulation, innovation, and diffusion, and we deal with everything from the National Institutes of Health process of awarding grants to drug regulation. The one on health care financing has modules that deal with the hospital side, the physician side, and now the managed care side. That has worked well. Another thing that was unique (and one never knows just how unique one's own program is, as you only hear about the others) was that the exercises were in the form of short policy exercises: a short policy memo, a presentation, etc., all oriented toward current problems which captures the fellows.

    The workshop model was a modification of part of the curriculum at the RAND Graduate School. It's oriented toward health and has become more stylized. That basic model was there, and what we tried to do was to apply it to health policy problems.

  2. methods of implementation and

  3. educational process?

    This is a health policy emphasis grafted on two separate programs. One is a broad public policy PhD (at the RAND Graduate School), and the other is mainly a health services PhD at the UCLA School of Public Health. There is a heavy dose of required seminars that take the place of electives that

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

existed in the program before. The fellows can cross-register at each place, but there is a common core of seminars that the folks take.

6. What were the biggest challenges or barriers to overcome in terms of:

a) your program design, b) methods of implementation, and c) educational process?

I think that there was a short-term challenge in the form of leadership problems at UCLA for about 2 years that made things more difficult in terms of coordination and getting good people at UCLA. And there were the minor logistical problems that every program has with courses and instructors.

7. What lessons were learned about the educational process in terms of:

  1. recruitment?

    We had a very low attrition rate, so I guess we did well in recruitment. We had basically two components. One was the PhD and the other was the midcareer component. I think except for some of the usual misjudging of how committed somebody is to doing something, we were pretty successful and didn't necessarily learn anything in particular. I guess we learned that we were doing things pretty well.

    How about the midcareer program? That was a unique and innovative program. How did you go about recruiting these people?

    Basically, we sent out an announcement, and there was a fair amount of word of mouth. Over a fairly long period of time we have had tailored sabbaticals for people coming here. The midcareer program did not continue after the Pew funding ceased simply because we did not have the funding. We offer them an intensive, didactic, almost tutorial with one teacher and no more than three to five participants. That has ended, but we have continued to have people come when they come with their own money or own support from someplace. It pretty much recruited itself. We sent out formal announcement, but it was the word of mouth really.

  2. degree requirements?

    Our fellows for the most part came in with a master's. We were looking for people who had already done work that required, in the case of the PhD students, their commitment to health, and in the case of the midcareer folks we basically

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

took in people who had to convince us that they were committed. In most cases, the midcareer fellows had demonstrated substantial capability in a domain other than policy. We took people from a fairly broad range into the midcareer program. We have a fairly strong quantitative program. The program at UCLA is somewhat less oriented toward the quantitative. In both cases we basically looked for people who had strong backgrounds in this area.

What about the dissertation process?

The RAND Graduate Program has another unique feature that I didn't even mention because it has been around so long, and that is the on-the-job training component that's been there from the beginning. The fellows spend about half their time on projects. What we try to do is get them on a project that will lead toward their dissertation. It doesn't always work. That is not a requirement of the project time, but that is a component of guidance. That tends to get them started, and probably half of them tend to follow that. There are a lot of mentors around because we have a large research program, and so if it's not clicking with one person they move and eventually find someone with whom they click. It works. Obviously, some take longer than others. I could only count two that I would call dropouts (dropped out before completing the course requirements or exams) and only three who I don't think have completed the PhD.

  1. curriculum and content?

    I think it works well. The particular kind of workshop has been one that has been copied elsewhere. To some substantial degree Joe Newhouse's program at Harvard follows it. Imitation is a form of endorsement. Joe taught here until he moved there. I guess the other thing that worked better than I thought it would was that when we had the workshops we expected the midcareer fellows to take the workshops too. That turned out to be a leavening experience both ways. They learned from the more academically and methodologically oriented doctoral students, and the doctoral students learned from the real-world experience of the midcareer fellows.

  2. integration of fellows with other students, the university, and the program?

    Basically we opened up the workshops to people outside the fellowship program, and we had a fair number of

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

takers and we still have a fair number of takers. That form of integration occurred. The RAND Graduate School program is by nature a small program, and so unless the class doesn't bond (which is very rare) there is almost forced integration. The workshops are still going. I just finished teaching a module on drug regulation. Only about half were committed health policy people from the beginning; others were taking the course because they wanted to increase their understanding of regulatory policies or the state of health technology. That is an example of integration.

There was perhaps a little bit of jealously in the early on. The Pew folks were looked at as being particularly well off and particularly free of having to worrying about getting projects. We changed that over time.

  1. relationship between faculty and students?

    It's by nature close. I mentioned earlier that it is more of a collegial atmosphere.

8. To what extent do you think the program is beneficial for those who do not finish?

For those who drop out before completing the courses and exams I don't think much. I've lost track of one of them. Two people up front looked like risks and the program did really work for them, so I wouldn't want to take any credit for how they've done. For the three who haven't finished and are at least in theory in process, I think unequivocally for one the program very strongly affected his career and he has had a major role in the private sector. About the other two I'm a little less clear, but they are both doing health work, last I heard. I think in that sense they have been using their training. The answer is that those who get to the point of passing the general exams stay in health and do good work, yet it's not the same as a PhD. In the private sector, of course, they don't care so much.

9. How can we measure success for those programs where completion rates do not apply (e.g., postdoctoral programs)?

We had essentially no one who didn't complete the midcareer program, but the question is how to measure success when completion rates do not apply. I think you simply look at their career and you have the testimonial there.

10. Haw does the Pew program approach differ from the traditional fellowship approach? How have the major outcomes differed?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

From the beginning we tried to [attract the] cream (of the crop). We intentionally made it a rich program because we wanted to get the best students. Most programs cannot do that, and we can't do it now in the same way. The other thing was the UCLA and RAND relationship and the availability of cross-registering. For instance, the RAND Graduate School doesn't teach epidemiology but they have an epidemiology program at UCLA and a business school at UCLA. At both sides there is a lot of going back and forth across campuses. I don't think that is that common. My sense is, however, that BU and Brandeis did not integrate to the extent that we did.

I think that as a whole the Pew fellows have gone toward policy in a more focused way than people in the past. It did emphasize policy. The emphasis was different across the set of programs, but the policy side was more focused than I think is common. When Paul Ginsburg was running the Physician Payment Review Commission he hired several of our graduates and specifically noted that he preferred our graduates because while he needed people with strong analytic skills, he also needed people who had a background in the health care system and health policy. We created that mix of skills that set our fellows apart. The term management was in the title; however, we took that to mean policy management.

Maybe you could just say a bit more about the common theme among the ''set of programs?''

I think they focused on what is a policy and what happens when you have one. That was a theme that I think existed across all programs: the declaration of a policy doesn't mean you have one. We certainly probed that, and I think it was probed at the annual meetings and at the other schools.

11. If you were going to give advice to another university attempting to initiate a similar program, what would you say?

I can't think of any other place, other than Joe New-house's at Harvard, that has concentrated their focus on getting the best people, as we did. Clearly, though trite, money helps. RAND and UCLA have been doing things together for a long time, but the impetus for sustained cooperation that was in the educational program wasn't there in the absence of funding. In particular, the existence of the Pew program brought the two programs together, particularly on the educational side. Money makes the difference. We have a unique situation here because we had existing programs

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

and we didn't have to hire people to fill new positions. I would think it would be harder to start from scratch.

Thinking back about other programs, my sense is that Minnesota has developed a stronger policy focus over time. The program wasn't established before. The same is true with the program at Penn's Wharton School.

We were a test. The program was defunded and it's continuing. I don't think it would continue in the absence of funding if not for the strong research ties between the two institutions. The people who teach jointly for the most part are people who are also involved in joint research. My guess is that that is an uncommon situation. We clearly have a legacy of continuing joint courses between the two institutions and a fair amount of movement back and forth.

12. What is the Pew HPFP "legacy" in terms of a) health policy, b) education and training, and c) your institution?

The legacy is a continued collaboration between the two institutions; there is also the legacy of continuing collaboration with former Pew fellows. Many of the fellows go through a transition period after they complete their degree. They continue to consult on the projects they were involved in for a while after.

13. Are there important issues you feel this interview does not address? If so, please feel free to add comments and/or concerns.

There is a criticism. Pew became a pretty bumpy place, at least toward the end of our relationship, especially in the health area. The turnover has been great.

Is it reasonable for a foundation to start a program with the idea at the beginning that somehow it will become self-sufficient? I think we have done pretty well, but I don't think the premise for that is strongly based in history. I am sure that were it not for the strong research program here, it would not have continued. The current health program at RAND is about $16 million, and that provides a base for supporting students to some degree, but that is getting harder and harder to do with budgets getting cut, etc. It is very hard to maintain a program in the absence of a strong research base and strong research funding.

Kate and I will try to dig up some statistics and any other information you may find helpful to represent the RAND/UCLA program.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone interview with Midcareer Alumna Kathleen Eyre Monday July 8, 1996, 11 a.m.

1a. Based on your experience and familiarity with the Pew program, what did the program really accomplish? What are the most important contributions?

I did the midcareer program, that unusual 1-year fellowship that brought professionals into the health policy realm. I would have to say that the greatest accomplishment was introducing a paradigm or a way of thinking to existing professionals. We all came out of professional positions. I had a physician and another lawyer in my class. So, we came in with our own professional set of skills and analytic tools, and what we were introduced to for the first time was a rigorous policy analysis paradigm and way of thinking, the kinds of questions to ask, and the different disciplines that influence policy (it's very varied, obviously). I in our brief year we were exposed to all kinds of different academic settings and disciplines. The greatest accomplishment, therefore, for the midcareer fellows was the introduction of a paradigm, of an analytic set of tools that was an overlay to an existing set of professional skills and that was extremely useful in going forward. Other kinds of things would be that professionals were introduced into the academic world, giving us an understanding how academics think and work, and also vice versa, exposing academics to other professionals in terms of "in the real world we would think this way." There was probably pretty positive cross-fertilization both ways. Then, of course, as you hear from everybody the great network that was developed. I just can't underscore that enough. It's been such a huge advantage for me in the policy world, particularly since I've been back in Washington, just to call people up and say "what about doing this project," and there is a lot of good examples of that which I'll talk about later.

In terms of the overall accomplishments, the Pew programs as a set of programs set up a curriculum for the training of health policy analysts and professionals. It really established a model for training in a lot of different settings that will continue in those particular settings and in many others, I'm sure.

2. What was the most innovative or unique aspect of your program design and implementation?

It was the opportunity of stepping out of a successful professional career for a year. It's really unusual to be able to

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

do that. The timing, the setting, the stipend, all of those things contributed to the ability of existing professionals to step out and add that layer of analytic skills to their existing policy skills without investing a huge amount of time or a huge amount of financial sacrifice. It's that incredible unusual opportunity of being able to go back to school, having intensive training in policy analysis, and taking that out into the world.

How did you come to make this obviously difficult decision to leave work? What brought you to the RAND program?

I started as a federal health care antitrust prosecutor. That was my first job out of law school: competition issues for the U.S. Department of Justice, hospital mergers, and physician boycotts. I did some policy-related work, commented on federal regulations and Health Care Financing Administration policies, and the like and really enjoyed that, but decided I wanted to be exposed to the private sector so I went to the law firm, which had its own rewards, but what I really missed from my prior experience was the policy stuff. The decisions we were making then were going to influence an entire industry and the big picture, global movement of the world thing. I really missed those, and I didn't find those desires satisfied in the private sector. So, I left private practice, a very lucrative career, to do my Pew fellowship. However, being able to add that layer of very specific policy analytic skills and move my career in that direction was just tremendous. It was a tremendous opportunity, and I feel very privileged to have had that opportunity. It really made a big difference in my career path.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

Just the basics. Having the opportunity to understand the economic paradigm and the sociological and behavioral sciences. Getting exposed to the social science ways of thinking were the greatest contributions of the curriculum. We also had very specific sets of policy analysis courses, and we had quality of care with Bob Brook and all these absolutely fabulous introductory courses to health policy issues. Then we were able to take that a step further with project work. I think Arlene Leibowitz's class was probably the best that we had in the curriculum. It was competition, regulation, and the notion of understanding markets in an economic point of view and she was just terrific. That was the most enlight-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

ening class of the entire spectrum. And then applying those introductory skills to specific sets of projects and being able to work on projects. I worked on a big MEWA (multiple employer welfare arrangements) project and small employer insurance options project. I worked on some tax-exempt hospital issues. It was very fun to have the ability to apply the sets of skills that we gained into specific projects. I think those were successes in the curriculum.

I think the thing that we missed in my program was that we didn't do a lot of general discussions on the major health policy issues. We didn't have a roundtable on reform, for example. Those were all touched upon, of course, in the course work that we took, but we didn't have a regular, organized, high-level discussion on, for instance, health care reform, which was the hot topic at that time (1989–1990). I missed that. I know that this type of thing happened at other campuses. I know UCSF (University of California, San Francisco) has a regular brown bag session on different topics of fairly global interest. We just didn't have opportunities to take cuts at the big picture. That's the major criticism that I would have of the midcareer program. I don't know if it was because of the time constraints. I think some of it was that, but some of it was that the focus was on giving you your basic skills. That was the most important part of the program. I think being able to put that in context would have been useful as well. Maybe it was a general philosophy of the institution. RAND isn't as liberal leaning as some of the other campuses. It was more concerned about the nuts and bolts of research and getting answers to important research questions. Some of that is just the nature of RAND. They do like to really be careful of any perception that they lean one way or another politically. They work for many different nonprofit clients, and so maybe there was an intentional effort to avoid any overt political discussion.

4. How was the Pew approach different from the traditional teaching approach?

The difference was the integration of the different social sciences and the specific focus on handling problems with a health policy framework. It's hard to understand what exactly "the Pew approach" means. I knew what my program was. I guess it is an interdisciplinary approach across the board, a rigorous exercise in disciplined thinking.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

5b. Has your career trajectory changed as a result of yaur time spent in the Pew program? If yes, how?

It was a complete change for me, I got out of the law entirely. I left the program, consulted at RAND for a year, and then became director of health policy and advocacy for Blue Cross of California, so I worked on the private side in a health plan to help them understand policy issues and develop their own positions. That was really fun. I worked for Leonard Schaffer, who continues to be very interested in the national policy scene and a very thoughtful and controversial commentator on health markets and changes. I wouldn't have gotten the job, nor would I have been ready for the job without having had the policy training. It was really fabulous and wild. I directly link that switch in my career path to being able to do the midcareer program for the year. It was terrific. I then left (about 2 and a half years ago) to come to this institute which is a small nonprofit funded by 10 plans and have focused on managed care best practices. Obviously, it is a very hot topic: how do you integrate public programs into managed care settings, quality issues, and all those important issues? Again, this move was a direct result of the ability to have had the policy training. I can't say enough about how great it was for me. It allowed me to make a major career shift, and I am very happy with the decision that I made. The law was just too narrow, too contentious, and not aimed at thinking and solving the bigger picture issues. The work I do now, I feel, is contributing to society much more directly than the legal work I ever did. People feel differently.

Can your speculate about your fellow classmates?

Yes, I know the two folks in my cohort; Grant Bagley is now at the Health Care Financing Administration, he is an OB-GYN, a lawyer, and more. He is with the bureau of policy development working on things like coverage decision making, how you pay for particular procedures. He is most definitely using his Pew experience. He was at the Food and Drug Administration before that, so he is definitely using his policy spin. Lucy Eisenberg was my other classmate. I have not been in touch with her; however, as I understand it she went back to the law firm she had come from. I presume she is using her policy skills in her health practice, but I don't know for sure. I know all the PhDs I went to school with are doing incredible, fun, and interesting things.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

How was the interaction between the midcarrer fellows and the doctoral students?

It was very good, very healthy. Of course, they had much more strenuous courses, and dealing with the first year of a doctoral program is always hard, but the interaction was very often and very good. We worked together in courses and on projects. Part of the benefit of the program was for us to be able to be working with professional students like them.

6. If you did not complete the program do you plan to? If yes, why? If no, Why?

Not applicable.

7. What is the Pew "legacy" in terms of:

  1. health policy?

    A cadre of incredible well-trained people now exists. They are out there doing it in a lot of different sectors, both private and public sectors. It's an amorphous thing. You've got well-trained people working in the field in a variety of settings making changes. That's a huge legacy. Furthermore, they are talking to each other, which is part of the legacy, having developed that network.

  2. education?

    The development of an established curriculum that I presume will be disseminated and used all over, again is invaluable. It is a terrific investment for the foundation folks.

  3. your future?

    I'm going to continue in the policy world. It may not be in this same position. I'm going to be moving to California in the fall. My husband has a job at Berkeley, so I'll be making some changes, but I will continue to do policy-related work. I will always be grateful for the training, the exposure, the connection, the network, and the exposure to people at RAND. It has been invaluable and will continue to be invaluable for me. I can't say enough positive things about how great the program was. It has had real, genuine, important effects. I'd like to thank the Pew Charitable Trusts for creating the program and including me as a participant.

8. Are there any important issues that this interview does not address? If so, please feel free to add comments and/or concerns.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Some of this is selective memory, of course, but overall I had really positive feelings. There were some criticisms at the time, the major one being that lack of the big picture that I already discussed.

Other than that I think this interview does a great job at getting at the issues. I'm thrilled that you called. We did feel slighted at the last couple of meetings, and I understand that the information just wasn't there, but in terms of presentation, we need some acknowledgment because its produced some really fine researchers who, in fact, in some respects are probably better trained than those from some of the other programs on the quantitative side.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Telephone Interview with Doctoral Alumnus Leighton Ku Tuesday July 9, 1996, 10 a.m.

1a. Based on your experience and familiarity with the Pew program, what did the program really accomplish? What are the most important contributions?

I'm not sure I can differentiate between the contributions and the accomplishments. However, on a personal level, it helped me to get my PhD, which changed my career orientation. In addition, it put me in contact with a network of people who also were involved in health policy issues. At a broader level, I think that the important contributions or accomplishments of the program were that it helped establish health policy as an area of study, which had not been much studied at the point when the program began, and certainly took it out of somewhat, the traditional areas in which health policy was usually associated with, which was either as an adjunct to hospital administration or health economics. The Pew program brought health policy out from there.

There have been great changes in other health policy programs, and I think part of that is a result of what the Pew programs accomplished.

Can you name any particular programs?

There are certainly many more health policy programs than there used to be. For example Harvard has one, and Hopkins now has a fairly large and well-established health policy program. I don't think Hopkins had much of one all that long ago. So, here you have two of the biggest, most prestigious programs around, neither of which got any direct funding from Pew for their health policy programs yet, nonetheless, I think in part modeled some of Pew's activities. I remember back in November at the networking dinner that Kate Korman mentioned that Joe Newhouse took the Pew program to Harvard with him. The other place where Pew may have had an influence (and I'm not sure exactly what they are doing now) is at Berkeley. Helen Schauffler was a Pew fellow and is now one of the health policy professors at the School of Public Health. My impression has been that she sort of carried over some of what she studied.

Can you speak a bit to the overall accomplishments of the Pew programs as a network or set of programs?

The most direct benefit was that they trained a relatively large cadre of researchers and then, for the UCSF (Uni-

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

versity of California, San Francisco) program, program administrators who were able to do a continuing amount of work in health policy at the state and local levels and at the federal level at a time in which the issue suddenly became much more nationally prominent. I think there was definitely an element of fortuitous timing there.

2. What was the most innovative or unique aspect of your program design and implementation?

The most unique aspect of the Boston University (BU)/Brandeis program was the attempt to have a very accelerated program that provided a fair amount of program flexibility. I think the other thing that made it unique was, shall we not say a structural aspect, but it was something that was brought at the personal level, and that was that a lot of the faculty treated students really as friends and as peers. I think that was actually quite important in the program. I think that is one of the things that happened, certainly at our program and at the Michigan program, which may have been related to the fact that a lot of the students were older students who had experience in the field.

What can you say about the integration of BU and Brandeis?

Basically it worked fairly well, insofar as it really let students access facilities, faculty, and such at either institution. For example, I was normally at BU, but in fact, the dissertation that I did was mostly with someone who was at Brandeis. And that is just really unheard of. It was not a perfect marriage. But, in general, it worked about as well at it could have. Right now it's Brandeis only. As much as anything else what happened was that the people at BU left for greener pastures. In some respects the institution was there but the people were not. The real difference between BU and Brandeis was that BU had fewer requirements than Brandeis did and BU had the disadvantage that, whereas at Brandeis you were fully part of the Heller School and so had a place to function from, at BU you weren't part of the School of Public Health or the School of Management, you were part of this University Professors Program. So on one hand you had the flexibility to go anywhere, so some people did stuff with the School of Management, others did things with the Public Health School, and others did things that related to neither. This made it a bit more difficult because you felt like you didn't have a place to sit. You didn't really belong to a particular institution or a larger entity.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

3. What was it about the curriculum that contributed or did not contribute to the program's success?

I have mixed feelings about the curriculum. On one hand the freedom of the curriculum was one of the things that the students liked a lot; on the other hand I think that the BU/Brandeis program didn't give people a good enough methodological training. It was not rigorous enough in methods.

What can you say about the dissertation process?

The process for me was relatively painless. I had a topic that worked fairly well and a dissertation committee that was nice, compliant, and responsive. They would read my things and generally not take me to task too badly. It went well for me.

Were there structures built into the curriculum that kept the process going or at least helped to keep it at that fast pace?

There were definitely efforts to try to start it up. The major problem was that the hardest thing is to find the topic, find the niche, and find some way to deal with that, and then different students have different strategies on how they build their committees. I prefer to find my own project and then pick a committee that will give me support in the areas that I need and not give me a lot of hassles. Other students think they have to pick out the most prestigious names and the biggest experts in a given area to help guide them. That had its virtues too.

4. How, if at all, was the Pew approach different from the traditional teaching approach?

The Pew approach was somewhat different because there was a given set of classes and a slightly different program philosophy. I'm not sure when all was said and done that there was a massive approach difference, but maybe there was something that was a little different: My impression was that at Brandeis, perhaps there was somewhat more of an emphasis to think that either I'm a political scientist, an economist, or a sociologist, whereas at BU there was not an emphasis to identify with one or another social science discipline.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

It certainly changed my professional life somewhat. I was doing policy work and policy research in the federal government beforehand. On the other hand, the area that I was working in, food and nutrition and welfare policy, was fairly different from what I do now, so it permitted me to shift areas into a somewhat broader area and get some extra training. That was very useful and very helpful to me.

6. If you did not complete the program do you plan to? If yes, why? If no, why? Obviously you completed the program. Do you have any ideas how valuable the program was for those who did not complete?

There are two things. The first is that they all got some additional training, regardless of whether or not they finished. I think the other thing is that a fairly high proportion of those who did not complete the program (meaning that they did not get their PhD) still harbor a hope that at some point they will complete it. I know this because when I periodically talk to some of the people who didn't complete the program in the first 2 years, they say that they still plan on getting around to the dissertation any day now. At the least they harbor some hope of completing it. I think that in any PhD program there are those who after the first few years don't complete the program, and then there is a small percentage of the noncompleters who then do finally complete it.

Would you guess that an accelerated program produced more or fewer completers?

I think that it produced a fair number of completers in the time frame. It may well be that one of the things that happened, because of the emphasis on a quick time frame, is that if you didn't complete the program within the first few years, then you sort of had to go out and find a real job and perhaps move away from the Boston area, and then there was a greater likelihood that you wouldn't complete it. There is a different philosophy than that perhaps at Michigan, where you knew you were in it for the long haul and you didn't feel the same pressure to complete the program quickly. I will say that my recollection from the statistics that you showed in Washington was that I think fellows at BU/Brandeis had among the lowest completion rates. However, by the time I completed the program, BU/Brandeis had lots of completers and Michigan had maybe two. It was an issue of different timing.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

7. What is the Pew ''legacy'' in terms of:

  1. health policy?

    It would be difficult to say that there is a clear and unique Pew legacy in terms of health policy. Actually, just recently I saw the video of Mr. Holland's Opus. Remember at the end when the people say, "Gee, Mr. Holland, you didn't do your opus, but we are your opus." That's sort of what the Pew legacy is. Is there a grand work or something that Pew can say is their legacy? I'm not sure that there is something sitting out there brassy and shiny, but there are lots of people that it affected both from the students' perspective and I think also from the program directors' and faculty's perspective. It encouraged faculty to do teaching and furthered them as well.

  2. education?

    One of the things that Pew did was that it helped make more formal the idea that health policy was an area of interest, and it is slowly gaining in acceptance. I do a lot of work for the Association of Health Policy and Management, and the health policy group there seems to keep getting a bit stronger and bigger. Still, that was well reflected in the journal, but partly they are correct because there are lots of health journals but there are not that many management and welfare policy journals so they want to emphasize that. I still wish I'd see a few more health articles in the journal though.

  3. your future?

    I'm still doing the sorts of things that I studied. It had a long-term effect on my past and I assume it will affect my future in the same way.

At the November meeting you made reference to the fact that you are no longer in government and yet you have more influence on policy in your current position than before. The context of this statement was that we were talking about what Pew wanted to produce. Can you speak a bit to that?

Partly it is because when you're in government, at least when I was in government, and for what I did dealing with the program, I was sort of the major policy and budget analyst for a long time, but you are always sort of fitting in and just trying to keep up with a broader administrative agenda for the Administration and/or for Congress. You can shape things, but you're shaping things within that context. If

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

you're on the outside doing policy research, then you can try to shape the agenda somewhat. Whereas when you're working in the government, the way the government is set up it's very hierarchical. Basically speaking, most government people work in relative anonymity, and information goes up the administrative hierarchy and then occasionally goes across into other policy circles, whereas if you're on the outside at a place like the Urban Institute, you can deal on lots of levels with policy makers in government, in the executive branch, in Congress, at state levels, and with some of the other associations. In the past few months I've given talks to the national council and state legislators, and the American Medical Association, and it is the sort of thing that would be unusual to do if I were still in the government.

8. Are there any important issues that this interview does not address? If so, please feel free to add comments and/or concerns.

Not that I can think of at the moment. Is Pew at this point having any clear thoughts about what it wants to do in the future in this area?

As far as I know they are not thinking about health policy. They did start up another program that is more clinically based.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Face-to-Face Interview with Doctoral Alumna Linda Simoni-Wastila Wednesday July 3, 1996, 10 a.m.

1. Based on your experience and familiarity with the Pew program, what did the program really accomplish? What are the most important contributions?

I think the best thing that the Pew program did was that it took people, mostly from different disciplines, and gave them very similar training. There were four programs and each one had different goals but essentially it developed this cohort of people who could do health services research, who could do health policy, and who could talk the language and walk the walk and put them out there all over the place. Pew developed this huge network. Not only did it help those who went through the program but it also helped shape health care and health policy. I think that's one of the most important contributions.

I can go anywhere, to any meeting, and start talking, and someone will say, "What's your background?" I'll happen to mention that I'm a Pew health policy fellow and they'll say, "Oh yeah? I was a health policy fellow too!" I think the other thing for me, on a personal level, is that I really felt nurtured through the program by the staff and faculty, particularly Steve Crane (even though he was at BU; we had the joint program then). He was a real mentor. I also got some good mentoring here, and we nurtured each other. My class was particularly tight. We nurtured each other and sort of developed a need to mentor other people. That was nice; I enjoyed that. Now I find myself in a mentoring position for students and even for some colleagues, and I am prepared. The Pew program sort of facilitated that.

Network building began in the first year. In the first year, all the first-year fellows go to Washington, D.C. You were introduced right away to the Washington scene and the hot issues of the moment, the real cutting-edge issues. If you wanted to know about health care reform you got the latest and the greatest right there. It got students really excited. And, it helped to bridge the programs, to develop that commonality and similar approach to health and health policy. The same thing was true of the annual meetings. The great thing about the Pew program is that it got people excited about the issues and exposed them to the experts, and I'm sad to see that ending. It was almost as if you too became a cutting-edge person. The conversations are incredibly intellectual yet practical at the same time. You felt like, "Wow, we can change the

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

world!" You just don't get that a lot in typical academic programs. You know, even for me, I'm very specialized, I'm not out there doing health care reform, but I was able to apply my interests and expertise. I would love to have a weekend retreat like that every year. I know the AHCPR (Agency for Health Care Policy and Research) fellowships are happening, but I don't think they have the same sort of bringing together of people from across the country and exposing them to some of the leading minds and practitioners. That was one of the biggest benefits of the Pew program.

2. What was the most innovative or unique aspect of your particular program design and implementation?

The most useful and very unique aspect when I was here (the program was a joint BU/Brandeis program) was the dissertation seminar where we were exposed to the second-and third-year students. It really was a dissertation seminar. Every other week there would be a presentation of different research methods and designs. We would spend a whole 2 to 3 hours discussing internal and external validity. The issues discussed were very relevant for the dissertation proposal work. One could take the issues beyond the dissertation process per se; however, they were specific enough to apply to whatever you were working on. It was very useful. It was basic yet thorough: Epidemiology, research methods, and some biostatistics thrown in. The discussions were very cogent. What was even more useful was that we were all, the first-and second-year students, told that we had to be present. This was not something that was taken lightly. Our sessions were 2 to 3 hours; they were long and everyone participated. You didn't get credit, but everyone came. The second-year students and even some of the third-year students came and were helpful in getting us younger folks to start formalizing our ideas. At the same time we were exposed to different processes and different levels of the process. For example, we might have someone come in and say that they were defending their proposal next week and would like to have a dry run, so we'd see what a formal dissertation proposal would look like, and we were then able to provide feedback and hear feedback that was given by other students and/or professors. We had a very good idea what to expect before we actually went through the process ourselves. We even had dry runs for people who were defending their entire dissertations, so by the end of my first year, I was no longer afraid of the dissertation. It was no longer that big huge "D." It was no longer the big black hole that so many

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

students talk about. They don't see that there are steps to it and that there is a sequential process and that it happens over time. I was able to see very clearly after 1 year what the process was, what the steps were, and what sort of things I should consider. And these principles are not just useful for the dissertation process; they are principles that any researcher or user of research will apply throughout his or her career.

Another thing that I liked about the program at Brandeis was that every year we had a seminar series that focused on an area. Some of them were very good, and some weren't so good. We had one that Stan Wallack did on health care financing that was extremely tedious, I guess, but we were exposed to the latest economics in health care in terms of financing and reimbursement and it was excellent, very in-depth. It provided us with an evaluation of the literature. There was another seminar on people's dissertations that were made into books. They were all related to health care. We were thus able to discuss these issues and at the same time examine other people's research. Our focus was mostly qualitative, which is important because I think most research programs don't have enough qualitative courses. One year we had one just on AIDS, which was fantastic. Jonathan Mann came and spoke to us.

Overall, the dissertation seminars were the best. We really learned from each other and developed connections that still exist today. I still call on people, and vice versa. Going through the process together bonded us personally and professionally.

3. What was it about the curriculum that contributed or did not contribute to the program's success?

From the Brandeis side—and it's not a fault of the Pew program, rather, it is a fault of the Heller School—most of the curriculum offered at the Heller School is just not rigorous enough. We had to go off campus to get more economics, econometrics, or statistics. Bill Crown was great. When you get into the Pew program you've already come from a background that's pretty rigorous, like the Agency for Health Care Policy and Research (AHCPR), and so for the most part we really didn't need all this introductory stuff that you get here. That's my one complaint. Part of it may have been that I was younger than most people, and the program was really geared toward people who had been out of school for awhile. I came out of a fantastic master's program and was very well prepared. So indirectly, the Heller School curriculum did not

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

do much to increase the Pew program's success. (I should note that The Heller School has since addressed this concern by overhauling its PhD program curriculum.)

4. How, if at all, was the Pew approach different from the traditional teaching approach?

It was different by the fact that there were smaller seminars, more one on one. There was a lot more mentoring. There was a lot of collegiality between faculty and students in my master's program, which is really unusual for a master's program. So, I thought that would continue in my PhD program. I dreamed of working alongside some great experts and solving the problems of the world. Yet, when I came here I thought this was the most closed-door place I had ever been. At first I thought it was just this place, but then I talked with my friends at Harvard and Johns Hopkins and they said it was the same everywhere. But, the Pew program helped immensely in that area. Some key faculty were available and eager to mentor the Pew fellows. We had ready access. It was built in. We had Stan Wallack, Steve Crane, Jon Chilingerian later, and many others, mostly BU people, which is interesting. Having that available, as well as a really good group of folks, made it different. But it is interesting: a lot of other students resented that; they hated the Pew fellows. Part of it may have been that the seminars were closed off to non-Pew fellows. And so during my last semester we opened it up but only one other person came. But one of the best things about the seminars was that they were so small. There was a real sense of camaraderie. We didn't really want it to be opened up. On the other hand, it was a great opportunity, and all doctoral students should have the same opportunity. So there was a real conflict in my mind about how I felt about the setup. There was also a lot of fuss made about the Pew fellows which I didn't think was appropriate.

5a. How has your professional life changed as a result of the Pew program? What value has Pew training added to your life?

I don't think I would be where I am now if I had just gone through the Heller School PhD program and not specifically the Pew program. I think the Pew program just gave it that extra boost. I don't even know if I would have finished my PhD otherwise. Yes, the funding facilitated it, but the pressure and support to get done also facilitated it. I almost quit the program after 1 year. I thought I might follow another interest of mine, art, but the reason I didn't was

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

because I kept on thinking about what Marion Ein Lewin said at the meetings, "OK, you guys, you can all get done, you can all do it." She would always talk about how important is was to finish. And, then I said to myself that I've got to get done, I've got to do this, I'm not a quitter. I'm really glad I did it. That kind of support motivated me to want to contribute. And, I'm very pleased with my position. My experience with Pew continues to motivate me. When I get the Pew newsletter and I see where everyone is and what they are doing I think, wow, that's great. It makes me want to go out there and publish more, contribute more, and do more. In that way Pew has touched me professionally. It has made me more enthusiastic about what I do, not that I wasn't before, but it has given me that extra boost. It made the Heller program, which was pretty good, excellent.

5b. Has your career trajectory changed as a result of your time spent in the Pew program? If yes, how?

When I finished my master's program I had applied for the Presidential Management Internship but didn't get it. I thought I had wanted to work in Washington. I have many friends who did that and are now "Beltway Bandits." I thought I wanted to be down there doing policy. I thought I wanted to be a policy wonk. But, the Pew program made me realize (even though it is a policy program) that I was not a policy wonk. It didn't make me dislike policy. I actually believe that my research is policy relevant. But, it made me realize that I have a personality that doesn't fit that role. And that's just fine. Yet, it reiterated to me the importance of policy and to study things that have some sort of relevance to a policy, political, or social issue, to strive to deliver as much truth as possible to that particular policy area. In that regard, yes, Pew changed my career trajectory: I realized I was a research nerd. But I do research with practical applications.

6. If you did not complete the program do you plan to? If yes, why? If no, why?

I did complete the program. Just going through the course work is the easy part. All it is is foundation. The hardest part is doing the dissertation. The comprehensive exams don't pull all our knowledge together. Nothing but the dissertation can do that. That is the big challenge. People who don't finish haven't met that challenge. I feel sorry for them. I really think they've missed out on the best part of the whole program. I don't know why they haven't finished.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

Maybe people think they can come in here and pull it off while keeping their full-time jobs, running a family with two kids, and all that going on. Some people can actually do all that. But, you have to really make a commitment to completing the program, and the dissertation needs to be a real priority. I teach a dissertation seminar, and I tell people this: "Don't plan on remodeling your house in the next 2 years. Try to limit everything else. Focus on your dissertation. It's only 1, 2, maybe 3 years of your life." And people get tied up with thinking they need to have blocks of time. You don't need that. You just need a few hours a day or even an hour a day. People just don't see that. I think one of the reasons the Pew program wasn't refunded was because there was a low completion rate. I get a little angry about that. I think it's been denied to other people, this great opportunity to get exposed and get the funding, etc. Of course they still made great contacts. Unfortunately, I think some people come into this just to make the contacts and they sort of leapfrog over people.

7. What is the Pew "legacy" in terms of:

  1. health policy?

    What it has really done is to put people out there who have superb training and an interesting way of attacking problems. These people are now in high-profile positions. They are able to make a real contribution to the field. I think that's real important. I also think that below that there is a level of people, like myself, who are more background people, who do the research. And I think we are real important too, because what we are doing is building a foundation of knowledge that has practical applications, and I hope is done as objectively as possible and as rigorously as possible. But, there are definitely two layers. Policy isn't just those people working for Rockefeller or Kennedy or the bureaucrats in the U.S. Department of Health and Human Services. It's also about the people who do the background work. Pew has done an excellent job of supplying these kinds of people.

  2. education?

    There are two levels here as well. I do some education too. I think it's real important that I return the mentoring that I received, and I think a lot of people came out with that attitude. There are now incredible role models out there for people who are starting out. There is a high profile of Pew

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×

people, and that is a real legacy. To have those type of role models out there and for them to be visible.

It seems to me that the Pew programs provided some consistency to the whole arena of health policy in terms of the quality and expertise of people that may not have been there before. That was achieved through the educational mission and policy focus.

What Pew had also done was to make sure that programs like those at Brandeis, UCLA, and UCSF have the programs even without funding. They all have a real strong interest in continuing programs like this through other funding sources, like AHCPR. They all continue to build this health policy area, broadly speaking. Pew really enabled a few programs to develop educational programs in health policy and gave people fantastic skills. And they are now ongoing. It was like seed money. It's really important. Everyone else can't find jobs, but this is one area where there will always be jobs. Health policy keeps on growing. Pew fostered that, and that's the legacy.

  1. your future?

    Pew gave me a head start. It allowed me to evolve my ideas in a less pressured environment. I didn't have to worry about where my next paycheck was coming from. I didn't have to worry about not having colleagues and peers to support me and mentor me. It gave me a real boost. I'm really happy for that. I'm grateful for that. I know a lot of graduate students who feel like they are in perpetual alienation. Pew prevented that from happening to me.

8. Are there any important issues that this interview does not address? If so, please feel free to add comments and/or concerns.

We have to address the gaps that will now exist without Pew funding. For instance, if AHCPR does go under or if other foundations fail to kick in, hopefully, Pew will reconsider this sort of program, again, but maybe in a different way or with different institutions. We need to always be thinking about training health policy professionals at the pre-and postdoctoral levels. We need to keep an eye on the field and seeing where and what the gaps are in health and medical training.

Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 137
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 138
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 139
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 140
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 141
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 142
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 143
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 144
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 145
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 146
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 147
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 148
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 149
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 150
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 151
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 152
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 153
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 154
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 155
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 156
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 157
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 158
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 159
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 160
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 161
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 162
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 163
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 164
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 165
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 166
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 167
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 168
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 169
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 170
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 171
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 172
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 173
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 174
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 175
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 176
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 177
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 178
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 179
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 180
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 181
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 182
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 183
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 184
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 185
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 186
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 187
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 188
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 189
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 190
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 191
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 192
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 193
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 194
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 195
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 196
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 197
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 198
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 199
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 200
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 201
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 202
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 203
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 204
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 205
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 206
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 207
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 208
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 209
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 210
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 211
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 212
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 213
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 214
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 215
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 216
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 217
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 218
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 219
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 220
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 221
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 222
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 223
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 224
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 225
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 226
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 227
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 228
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 229
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 230
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 231
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 232
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 233
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 234
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 235
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 236
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 237
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 238
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 239
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 240
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 241
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 242
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 243
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 244
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 245
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 246
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 247
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 248
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 249
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 250
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 251
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 252
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 253
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 254
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 255
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 256
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 257
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 258
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 259
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 260
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 261
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 262
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 263
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 264
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 265
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 266
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 267
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 268
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 269
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 270
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 271
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 272
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 273
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 274
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 275
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 276
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 277
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 278
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 279
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 280
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 281
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 282
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 283
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 284
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 285
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 286
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 287
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 288
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 289
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 290
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 291
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 292
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 293
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 294
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 295
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 296
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 297
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 298
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 299
Suggested Citation:"Appendix A: Telephone Interviews." Institute of Medicine. 1997. The Lessons and The Legacy of the Pew Health Policy Program. Washington, DC: The National Academies Press. doi: 10.17226/5821.
×
Page 300
Next: Appendix B: Curriculum and Course Offerings »
The Lessons and The Legacy of the Pew Health Policy Program Get This Book
×
Buy Paperback | $80.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!