Respondent
Ronald A. Thisted
University of Chicago
For the last day and a half we have been talking about change. Change involves risk, and risk involves the possibility of failure. For those reasons, it is uncomfortable to contemplate change. The possibility of failure makes us more comfortable with what we already do, and it makes us worry: if we do effect change, will we change in the right direction and with the intended effect?
My job as last speaker is not to be a discussant but to be a respondent, and so what you will get for the next few minutes are my responses — my knee-jerk responses, if you like — to what has gone on for the last day and a half.
Jean Thiebaux's argument that statistics students should be grounded in philosophy is exactly right. Although she did not follow this idea very far, I think that philosophy has much to offer statistics. In particular, the area of metaphysics is important in philosophy. Metaphysics is the critical study of assumptions, of first principles, of where we start from and how that starting point affects where we can go.
This symposium has been exploring a particular model for change, a model for reform of statistical education, and I hope it is not too much to suggest what George Box constantly reminds us of: ''All models are wrong" (Box, 1976) — not necessarily very wrong, but wrong just the same. And so I will devote some of this response to checking the model of education that we have been discussing — to explore, if you will, some of the possible sources of bias in the conclusions we are drawing that depend in this metaphysical way on the way we framed the question.
In statistics, what we both learn and teach is that assumptions can be violated. A metaphysician would say that our assumptions can limit our thinking, and that being critical about our assumptions can sharpen our thinking. What follows are some of my reactions about the assumptions I see as being thus far implicit in our deliberations.
This symposium is based on the assumption that there is something wrong or, perhaps more charitably, that something is missing in statistical training programs, namely, this interdisciplinary component. What makes John Bailar's comments so disquieting is the suggestion that this assumption, which at the base is a very comfortable assumption for statisticians, is wrong, and that interdisciplinarity is not missing in our training programs but rather is missing in ourselves. Why is "interdisciplinary work" seen as bringing salvation; why is it viewed as a good thing?
I do not believe the real reason is because there are clients whose needs would be better served if statisticians did more interdisciplinary things. Although it is certainly true that statisticians have deficiencies related to failings in interdisciplinary work, fundamentally statisticians have an image problem. There is also a market share problem: the economists have already passed us, while at the same time we genuinely feel that statisticians have something to contribute that is real and important.
The heroes of statistics have been very broad in their approaches to the discipline. Steve Fienberg listed some of the heros; my list includes David Cox, F. N. David, R. A. Fisher, Frederick Mosteller, Jerzy Neyman, and John Tukey. All of these individuals are or were in their time taken seriously by non-statisticians in a way that I am afraid most of us today are not. But these are the data-savvy statisticians; these are the holistic statisticians, and, in truth, none of them are products of a modern statistics PhD program. So, perhaps, if the modern statistics PhD program is changed, more of these heroes of statistics or, as Brad Efron would prefer to call them, the wizards of science, can be produced.
They represent to varying degrees not so much what we as statisticians want to do but what it is we want to be. And this dissonance between what we want to be and what we now do is a proper note on which to return to consideration of some other assumptions. I next list some assumptions that have underlain the symposium discussions, not to suggest that these assumptions are wrong, but that they might be different.
One assumption is that the purpose of statistics training is to produce the right product for the right client. This business metaphor is useful. One can talk about the declining demand for the product, the increasing production costs and — what I believe is the greatest fear of directors of statistics programs — the possibility of excess inventory at the end of the year. But there are other potentially useful metaphors that should be considered. There is the Darwinian metaphor, for example, in which the purpose is not to produce a product but to replicate the species, and wherein the idea of incorporating interdisciplinarity is a way of evolving so as to avoid extinction. I think each of these metaphors puts a different emphasis on what our purpose ought to be. I like the business metaphor better than the evolutionary one, as Ed Rothman very clearly did in his presentation. The very first thing he listed for his curriculum was purpose; everything follows from some set purpose. This is also related to John Bailar's comments that perhaps our purpose and goals are wrong, and a clear vision is needed of what is the purpose of statistical training.
I am involved in creating a new department that I hope represents an interdiscipline in our medical school. The goal of this department is improving the health of the U.S. population. Maybe that it a little ambitious, but we statisticians have perhaps as good a shot at doing it as does Hillary Clinton, albeit in a different way. This is very much related to Jean Thiebaux's question, Who is the real client? Is it the future of science? Is it the future of our nation or our future global economy? I think all of those are important purposes and should be incorporated into our visions of who we as statisticians are and what we do; as Steve Fienberg has just suggested, at the very least we should maintain that statisticians should be contributing to other fields and by those interactions should be bringing something back to and informing the broad field of statistics.
A second assumption is that whatever is done in the statistics training program needs to be done in the context of a four-year PhD program. Twenty-five years ago the modal PhD program was a three-year program, at least in the rhetoric. Now the modal program is advertised to take four years, although the median time to degree in most programs is probably at least a year beyond what the literature says. "Four years" is an assumption. Perhaps making five-year programs a matter of course should be considered.
We can also look at other training models. The four-year program or the three-year program is not what is done in law, medicine, or business — or in physics or biochemistry, for
that matter. In physics and chemistry, new recipients of the PhD go through postdoctoral training where they actually "learn their stuff." Once one gets a law degree, six months are spent cramming for the bar during which local law is learned, as is the material that was not taught in law school; one then joins a law firm as an associate for several years, and only after all of this is the person really considered a lawyer.
A graduate from four years of medical school does not go out and open a practice; instead, at least a three-year residency is taken. A graduate from medical school is not considered fit to practice medicine. The assumption that a new PhD should be fully trained, when nowhere else is that the case, seems a perhaps unrealistic one, and one for which the clients should take some responsibility. The clients have gotten off the hook a bit here by claiming that they no longer have the resources to provide on-the-job training. Who does? Where else will new scientists get on-the-job training?
To some extent the demands of business and government have simply shifted a burden onto the academic suppliers. We need to question whether that shifting is correct and appropriate. (Actually, one could take the perspective — again by changing one's metaphysical point of view — that the system could be serving industry and government very well right now, if only they would recognize it. Although it may be true that new PhDs do not provide what industry and government need, the individuals who have been denied tenure after six years of collaborative work and excellence in communication could do very well in those other environments, and their training has been subsidized by academia already!)
A third assumption that is not universally held at this symposium, but that I believe is more generally held at large, is that the right set of courses can in large measure produce the right product. Furthermore, it is in the contexts of workshops and mentoring, and Joan Garfield's constructive learning, that the interdisciplinary aspects that we have been talking about are most effectively absorbed, confronted, struggled with, and taught. Perhaps the assumption regarding what we do in courses that are very finely structured both in time and space, if not in format, does not provide the best route to achieve interdisciplinary results.
A fourth assumption, which we might prefer to shuffle to the side, is that any student in a statistics program can be molded into the right product. I believe this is simply not correct. There are students who do not have an interest in collaboration; there are students who do not have interest or ability in teaching; there are students who are whizzes at mathematics and are tremendously strong at proving theorems. Perhaps we should not suggest that they should not be PhDs in statistics because they cannot do everything that might be important. Consequently, I feel we have to consider different flavors of statisticians, different tracks, and whether we value them equally.
These assumptions are comforting, but they are not necessary to achieving results. By changing our perspective a little and challenging our assumptions we can sometimes make progress in ways that we had not previously been able to consider.
Joan Garfield summarized the early presentations on client needs by listing three qualities needed in statisticians: they should have teamwork and collaboration skills; they should have communications skills, both oral and written; and they must possess ability to solve real problems. As Duane Meeter commented, these skills, interests, and abilities often describe the complement of those who apply to graduate programs in statistics — and that is a real problem.
Just as journals are at the mercy of those who submit papers to them, so also are statistics programs at the mercy of those who apply to them.
We would like to admit to our programs people who already have an interest in and some initial abilities in these areas, and we would like to make them better. That is what we need, and that is why I am so encouraged by the work of Ed Rothman and of Laurie Snell, which in both cases really reaches out at a very early stage to those people who might otherwise go into psychology or economics or law or business, and captures some fraction of them for the exciting things to be done in statistics.
Moving to another consideration, John Bailar noted, "No one has said, 'I am going to change my life because of what has happened here.'" So let me tell you something about my life, and how what is going on here relates to what I can see myself doing.
Over the next year I will teach three courses and conduct a workshop. Two of the courses are undergraduate course, one is a course in a professional school, and the workshop in some sense is a graduate course for statistics students.
The autumn course is an undergraduate statistical methods course using Moore and McCabe (1989). As to John's question, "Will it change as a result of what is going on here?," the answer is "yes." I am tremendously taken by Joan Garfield's comments on constructive learning and collaborative involvement in teaching. I have taught this course 15 to 20 times over the past years, and her approach seems to me an answer for which I have been searching. I will try to do something with it.
Is this an interdisciplinary course? In a manner of speaking it is — it uses only real data. It focuses on the real questions as opposed to the statistical questions — that is, one is not done after getting the P-value, and the class often draws on collaborations that have been engaged in with other individuals. But it is also not interdisciplinary; in fact, a section of the course has just been split off for economists so that they will not have to listen to examples drawn from biology and medicine. So although I have changed a little, it is perhaps not enough.
The other courses I teach are more directly relevant to what has been considered in this symposium. Interdisciplinary work is inherently cross-cultural. To work with people in teams requires one to acquire another culture. My winter undergraduate course is on quantitative reasoning. It was designed for humanities majors and is taken largely by people in the humanities, social sciences, and so forth. My purpose in that course is for the students to consider what it means to present an argument, what the nature of evidence is, what constitutes reasoning, and what counts for proof — in mathematics, in statistics, in epidemiology — and to compare those, say, statistical arguments to what constitutes an argument or proof in history, or in literature, or in sociology.
Every discipline has its standards for evidence and its own modes of argument. Those for mathematics are very different from those for statistics, which in turn are somewhat different from those for epidemiology. All these things are worth knowing, at least in comparison to one another, in a liberal-arts education. I have these students, mostly freshmen and sophomores, actually read papers from the New England Journal of Medicine, which they find intimidating at first and then kind of fun. I have them read a paper from Biometrika — after a fashion. This is a paper on estimating authors' vocabularies and on deciding authorship. I happen to know this paper fairly well, and in particular I know where all the skeletons are buried. I try to help them understand the structure of the argument in that paper, to understand that an argument is
there, that there are rhetorical devices being used, and to help them understand at whom the exposition is being aimed; what the authors want those people to take away when they have finished the paper; what the authors want the readers to assent to; and, to some extent, how and why the authors "swept some things under the rug."
The class reads The Mismeasure of Man by Stephen J. Gould (1983), partly for its argument and partly to try to get students to understand that some of the biases discussed in the first half of the book are employed, apparently without recognition, in the second half. But as Jean Thiebaux would have it, this is really a philosophy course focusing on evidence and knowledge. Is it interdisciplinary? It is more cross-cultural, and it is a cross-culturalism that cuts in the other direction; this is cross-culturalism for the non-statisticians in Jon Kettenring's or in Phil Ross's group. I hope that this course will train, or at least excite, some individuals who will do a better job of communicating in those teams with the statisticians on them, and I think it is also important for the discipline of statistics as well.
By implication, training statistics students entails a responsibility for those students to become versed in another culture, a process that takes time and commitment. It is a task that is never completed, and that suggests that perhaps more specialization rather than less, in terms of being interdisciplinary early on, should be the order of the day. If interdisciplinary work is introduced into statistics graduate programs, perhaps individuals should be encouraged to become immersed in just one other disciplinary culture and save the others for later. This immersion involves more, I believe, than merely acquiring interpersonal skills; it requires understanding of how other people argue and how they think.
My spring course is a required course for medical students that they take toward the end of their second year. This course has slowly changed over the last four to five years. Five years ago, fewer than 50% of the students came to more than 20% of the lectures; medical students are very good at voting with their feet. The course has since been renamed, modified, and refocused; it is now called "Epidemiology and Clinical Investigation." The focus has changed from our (statistical) culture to one of their (medical) culture, asking for each lecture, "How will the content of this lecture improve these individuals' ability to practice medicine?" Taking that question seriously changes a lot of what one does, and it seems to have been successful — at least in the sense that now over 90% of the students come to more than 80% of the lectures. The class reads current papers from the medical literature. The emphasis is on critical appraisal of the literature, understanding the strengths and limits of different study designs and providing a framework for what is coming to be called evidence-based medicine — in effect, inference.
These last two courses show, at different levels, that if statisticians take the initiative, very good strides can be made at communicating with other individuals in other disciplines and in other cultures. How do we teach our own statistics students, then, to do this? This is asking how holistic, data-savvy statisticians can be created. The workshop to which I alluded earlier is part of what I consider to be a successful model and has much in common with the Carnegie Mellon approach. That successful model is one based on cross-appointments; but I believe that, while helpful, joint appointments are not enough. John Lehoczky talked about Joel Greenhouse's involvement with psychological statistics at Carnegie Mellon, and Joel has many collaborators; but it is clear that those collaborations spill over onto the students. Through my involvement
in the medical school in Chicago, I too have many collaborators that I also "hook up" with students.
How is that done? Merely holding a joint appointment does not do it. Someone in this symposium mentioned the need to have an entrepreneurial spirit. One has to go out and talk to them. When I joined the department of anesthesia, the chairman and I mapped out the high-traffic areas of the department. The highest-traffic place was the so-called coordinator's office where the surgical schedule was made out each day for the next day's surgery. That was a great place because the official schedules were there; when there was a change in Operating Room assignments or someone wanted to know who was in a particular O.R., they came into the coordinator's office. Coordinators used the office for about an hour and a half each day. So we said, "Great! We'll put Thisted there!" So I sit in a tiny room, about half as big as this table, whenever I am in the anesthesia department. It is wonderful because everybody comes by — not to see me, but to check the schedule. And then while they are checking the schedule or looking up something else, they say, "Oh, by the way, I have been working on this problem. Could we get together sometime and talk for a little while?" An appointment is set, and that is when the real work gets done. But in some sense, the real work got done earlier by sitting in that place that is both visible and is clearly part of the department. I have lots of collaborators, and the reason is reflected in what they say when they tell me, "You're the first statistician who talks my language. You really understand what I'm trying to do." Well, in a sense doing that is not too hard, because I always ask them, ''What are you trying to do?" And they are very happy to tell me. It is not in me, it is in them, and they see themselves reflected in me, and everybody likes to look at himself. They also say this about my students, though; one of the great quotes goes something like, "I had to make her second author because she contributed so much." That is simply a wonderful thing to say; it makes me beam with pride. The statistics department has lots of students appearing as second and third authors on papers, and it is because we sort of hammer into them that you have to take it on yourself to learn the other culture. If you are working with somebody to distinguish prerenal azotemia from acute tubular necrosis, you had better know what those things are. It is not enough to know only how to pronounce them.
To work on learning the other culture, several years go we started a workshop in biostatistics modeled on the one example I knew then — Bill Brown's at Stanford. Jim Landwehr reminded me that Bill Kruskal also did this years ago at Chicago, and it must be done in many other places. The object is to bring together statistics graduate students, physicians, investigators, clinical scientists, and administrators to talk through problems, to discuss work in progress, and to talk about plans for studies. In the ideal session, a statistics student and a medical investigator who have been working together on a project lead the discussion, with the investigator giving the medical background for the non-physicians and the student discussing the methodological approaches to the questions under study. We have 25 to 30 people who come every Friday afternoon from 3:30 to 5, not generally considered an advantageous time slot.
The results of this workshop have been phenomenal, including many collaborations between students and investigators and several joint publications that do not have me as coauthor but that do have students from the department of statistics. These articles appear in journals such as Pediatrics, the Journal of Neuroscience, and Anesthesia and Analgesia, and our first New England Journal of Medicine paper appears this month. These students are mostly from
the statistics department. They do not get any course credit for coming to this workshop, nor for working with these investigators. The workshop fulfills no requirements for them of any sort. Our goal is to excite students about the contributions that they can make to real science, and I think it has had some effect in doing that.
What do they get for coming every Friday afternoon from 3:30 to 5? I will tell you my secret: they get cookies. But that is probably not enough. They also get thesis topics, they have gotten jobs, and I believe most importantly, they have gotten tremendous satisfaction from the work they have done.
So to conclude, I believe that a major key to fostering interdisciplinary work is to make it exciting and accessible and real, to do it in the context of a statistics department or any other unit that in some sense is a center of intellectual ferment where exciting and important work is being engaged in. I believe that the group should have as the assumption underlying what it does that interdisciplinary work is not what you do, but rather is really a part of who you are. And I believe that that is what is needed to attract the kind of student who will satisfy the clients and will satisfy themselves. I am convinced that it is worth taking the risks to do that, to make those changes. I am also convinced that "if you build it, they will come."
References
Box, G. E. P. 1976. Science and statistics. J. Am. Stat. Assoc. 71:791-799.
Gould, S. J. 1983. The Mismeasure of Man. New York: Norton. 352 pp.
Moore, D. S., and G. P. McCabe. 1989. Introduction to the Practice of Statistics. New York: W. H. Freeman.
