Click for next page ( 176


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 175
Chapter 7 MEDICAL SCHOOL ADMISSIONS Mary A. Fruen The medical school selection process is the major gateway to the medical profession in the United States. Since approximately 95 percent of entering medical students subsequently graduate, the admissions process largely determines the physician manpower supply.1 This chapter discusses the selection process and describes applicant assessment procedures. In addition, to the extent possible with existing data, it describes characteristics of entering medical s tudents . During the 1960s the emphasis in medical school admissions was on academic aptitude and achievement .2 In the past several years attention has shifted to the personal qualities of applicants.3 Individuals are selected more for medical practice than f or medical school survival. Also, wi th increasing concern for social needs, more emphasis has been placed on selecting applicants who appear to have the greatest potential for meeting society's needs for improved and more accessible medical services. Those individuals more likely to enter primary care specialties and to practice in areas with physician shortages have been given some preference in selection. During the 1970s concerted ef forts were made to broaden the socioeconomic diversity of entering medical classes and to expand the educational opportunities for racial/ethnic groups underrepresented in medicine. Thus, selection factors include both academic measures--college grades and admissions test scores--and information on the personal characteristics of applicants. However, recently declining numbers of college graduates, increasing medical school tuitions, and other factors are contributing to a declining medical school applicant pool. Therefore, the emphasis in selection may again be changing. A new factor has been added to the admissions process in some of the more expensive medical schools, that being the ability to finance medical education. Many medical schools are no longer able to provide financially for their students because the schools themselves are facing serious financial problems. Costs are increasing while the ability to draw on new resources such as patient revenues and practice plans are becoming limited.4 Institutions are forced to increase tuitions as student financial aid declines and low-interest loans nearly disappear. It is projected that medical students will greatly increase their borrowing, do it at substantially higher interest rates, and run a greater risk of having cash-flow problems in residency and starting practice. It is anticipated that higher costs of medical education will adversely affect the medical school application rate over the long run. Potential applicants from lower socioeconomic classes may be discouraged by the prospects of high costs and large debts. They 175

OCR for page 175
would face a very demanding medical education and residency, averaging more than nine years, followed by a less certain future in medical practice. Currently the ef fects of these f inancial factors on application and entry into medical school are only speculative, but admissions officers and others have expressed concern about maintenance of socioeconomic diversity in entering classes (see Appendix C ~ . A related concern is whether special programs for academically disadvantaged applicants and medical students can be maintained as resources shrink. Their loss also could jeopardize the mix of students. Financial difficulties could indirectly affect medical students' performance and retention.4 For example, a student with financial problems may work, worry, skip meals, or have an unpleasant study environment, which could affect academic performance. Some medical school administrators are concerned about the repercussions of f inancial dif ficulties, but only isolated instances of this problem have been reported . One probable indicator of f inancial stress among medical students is the recent increase in transfers from expensive private schools as places open in less expensive schools. Applicat ion Ac t ivity Over the past 20 years the number of applicants accepted by medical schools doubled, rising from 8, 682 in 1961/62 to 17, 286 in 1981/82 (Table 1~. During this period the number of medical school applicants rose even more dramatically, peaking in 1974/75; it has declined 15 percent since that peak year. The proportion of applicants accepted fell from 60.4 percent in 1961/62 to 35.3 percent in 1974/75, and rising to 47.5 percent in 1980/81. The disparity between the number of applicants and the number of entering places in medical school resulted in considerable competition among premedical students and ma jar problems for rejected applicants, many of whom had not seriously considered other careers.5 The decline in the number of medical school applicants is expected to continue for more than a decade, because numbers of college graduates are expected to decrease, given the age distribution of the population. Additional factors may discourage medical school application. For example, tuitions are rising while the availability of loans and scholarships is diminishing. Also, the projected surplus of Physicians and concerns about career possibilities may discourage some propsective applicants. Therefore, some admissions committees are becoming concerned about the likely effects of the diminishing applicant pool on the caliber, and eventually size, of their entering classes. Maintenance of class size is a major concern among admissions officers.4 As the popular ion of college graduates declines, some medical school administrators will eventually be faced with the choice 176

OCR for page 175
between cutting class size or selecting increasing numbers of higher-risk applicants. A few schools are considering cuts in class size already. TABLE 1 Medical School Applicant Data, 1961/62 to 1980/81 First-year Number of Accepted Percentage of Total Class Applicants Applicants Applicants Accepted 1961/62 14,381 8,682 60.4 1962/63 15,847 8,959 56.6 1963/64 17,668 9,063 51.3 1964/65 19,168 9,043 47.2 1965/66 18,703 9,012 48.2 1966/67 18,250 9,123 50.0 1967/68 18,724 9~702 51.8 1968/69 21,118 10,092 47.8 1969/70 24,465 10,547 43.1 1970/71 24,987 11,500 46.0 1971/72 29,172 12,335 42.3 1972/73 36,135 13,757 38.1 1973/74 40,506 14,335 35.4 1974/75 42,624 15,066 35.3 1975/76 42,303 15,365 36.3 - 1976/77 42,155 15,774 37.4 1977/78 40,569 15,977 39.4 1978/79 36,636 16,527 45.1 1979/80 36,141 16,886 46.7 1980/81 36,100 17,146 47.5 1981/82 36,727 11, 286 47.1 SOURCES: Medical education in the United States, 1980-1981 and 1981-82. Journal of the American Medical Association 246:2893-3044, 1981 and 248:3223-3328, 1982. Inc reasi ng Minority and Women Enrol lments Social changes since the late 1960s fostered the rapid rise in numbers of minorities and women entering medical schools. The proportion of underrepresented minority entrants has tripled since 1968/69, rising from 3.0 percent to 9.4 percent in 1982/83 (Table 2~. During the same period, the proportion of women entrants more than tripled, rising from 9.0 percent to 31.7 percent. Acceptance rates vary somewhat among racial/ethnic groups. In 1980/81, an average of 43.2 percent of underrepresented minority applicants were accepted into medical school as compared with an average of 47.9 percent for all others. Among underrepresented minority applicants, the averages by racial/ethnic group were as 177

OCR for page 175
follows: black Americans, 40.7 percent; American Indians, 42.2 percent; Mexican Americans, 53.5 percent; and mainland Puerto Ricans, 53.4 percent. TABLE 2 Medical School First-Year Enrollment Data, 1968/69 to 1982/83 Under- First-year First-year represented Class Enrollment Minoritiesa Women 1968/69 9,863 292 (3.0) 887 (9.0) 1969/70 10,401 501 (4.8) 948 (9~1) 1970/71 11,348 808 (7.1) 1,256 (11.1) 1971/72 12,361 1,063 (8.6) 1,693 (13.7) 1972/73 13,726 1,172 (8.5) 2,315 (16.9) 1973/74 14,185 1,301 (9.2) 2,743 (19.6) 1974/75 14,963 1,473 (9.8) 3,260 (22.3) 1975/76 15,351 1,391b (9 1) 3,656 (23.8) 1976/77 15,667 1,400 (8.9) 3,876 (24.7) 1977/78 16,134 1,450 (9.0) 4,149 (25.7) 1978/19 16,620 1,443 (8.9) 4,184 (25.2) 1979/80 17,014 1,547 (9~1) 4,748 (27.9) 1980/81 17,204 1,548 (9.0) 4,970 (28.9) 1981/82 17,268 1,671 (9.7) 5,317 (30.8) 1982/83 17,254 1,626 (9.4) 5,462 (31.7) NOTE: Figures in parentheses are percentages. aIncludes blacks, American Indians, Mexican Americans, and mainland Ptlerto Ricans; includes 11 to 15 percent repeating f irst-year class. bApproximately 200 of these students were enrolled at the two traditionally black medical schools, Meharry and Howard. SOURCES: Medical education in the United States, 1980-81. JAMA 246: 2893-3044, 1981. AAlIC Office of Minority Affairs. College Ma jars of Medical School Applicants Any college major is generally appropriate preparation for medical school, and no special preference is given to science ma jors.6 Nonetheless, a disproportionate share of applicants ma jar in science, particularly biological science: 47 percent major in biological sciences, 23 percent in physical sciences, and a significant percentage in interdisciplinary majors that include sciences. This compares with 9 percent in Conscience subjects and 4 percent who are health professionals (e.g., nurses). Acceptance rates range from 45 percent in biology to 48 percent in Conscience and 55 percent in physical sciences. There is an apparent trend among admissions of f icers to encourage potential applicants to medical school to consider Conscience ma jars during their college years. 178

OCR for page 175
Academic Performance and Test Scores of Entering Medical Students Few national aggregate data on the characteristics of entering medical students have been published. In fact, the only available data beyond race , sex , and the ma jor data presented above, are premedical college grade point averages (GPA~) and New Medical College Admission Test (MCAT) scores. Table 3 shows trends in the percentage of entering medical students in three categories of premedical college GPAs. The proportion of entering medical students with A averages ~ - - ~ 4.0) rose from 33.9 percent in 1973 to 50.4 percent in (GPA of 3.6 to ~ . 1977, and thereafter gradually dropped to 47.5 percent in 1980. (Some of the increase may have been due to grade inflation. ~ Since average New MCAT scores were initially reported for the medical school class entering in 1978/79, it is premature to consider trends in scores. However, projections indicate that the number of applicants will continue diminishing ~ ~ ~ ~ while entering class size will increase slightly, so that average GPAs and New MCAT scores will likely decline.1 TABLE 3 Premedical Grade Point Average Distribution Among First-year Medical Students, 1966/68 to 1981/82 Academic Year Aa B ~ OtherC 1966/67 13.6% 77.8% 8.6% 1967/68 14.1 76.8 9.1 1968/69 16.8 75.9 7.3 1969/70 17.9 76.6 5.5 1970/71 19.7 73.3 7.0 1971/72 24.0 70.0 6.0 1972/73 28.9 60.1 11.0 1973/74 33.9 54.7 11.4 1974/75 39.3 50.8 9.9 1975/76 44.2 47.4 8.4 1976/77 46.0 47.3 6.7 1977/78 50.4 43.0 6.6 1978/79 49.6 46.9 3.5 1979/80 49.2 47.2 3.6 1980/81 47.S 48.5 4.0 1981/82 45.8 50.2 4.0 l a3.6-4.0 GPA on a four-point scale b2.6-3.5 GPA CLe ss than 2.6 GPA or unknown SOURCES: Medical education in the United States. Journal of the American Medical Association 236:2692, 1976; 246:2919, 1981; 248:3251, 1982. 179

OCR for page 175
Selection Factors Medical School Admissions Requirements, published annually by the Association of American Medical Colleges (A.AMC), describes the specific admissions requirements for each American and Canadian medical school .6 Though admissions policies and procedures, including the role of various selection factors, are locally determined by each medical school, a few generalizations can be made. This section discusses f ive general selection factors, and the next outlines the selection process. The first factor relates to academic preparation, which is particularly important in attempting to gain admittance to medical schools. Although any college major is generally appropriate preparation for medical school, four prerequisite courses are required for entry into most American medical schools. The courses include one year each of biology, physics, general and organic chemistry.6 Additional course requirements for entering medical students vary widely across institutions. A bachelor's degree is usually required, but on occasion, people with as little as two years of college have begun medical school. Also, some high school graduates enter into a combined six-year program leading deco both bachelor 's and M. D. degrees. A second factor in selection decisions involves traditional indicators of academic competence--previous grades and New MCAT scores are relied upon heavily in medical school admissions decisions. The ability of these measures to predict medical school success has been studied extensively. Third, demographic and biographic car background characteristics are of ten used in selection. In-state residents are frequently given preference, particularly in public medical schools. Racial, economic, or community background is of ten considered, although use of these factors involves complex legal and ethical issues beyond the scope of this chapter. A fourth major area in selection decisions involves assessment of personal qualities. Attributes that may be critical in completing a demanding medical education program and successfully pursuing a medical career include motivation, integrity, diligence, and interpersonal skills. These characteristics are assessed by admissions interviews, letters of evaluation by premedical faculty and others, and occasionally by personality tests. Fifth, many medical schools take into account evidence of nonacademic achievements, on the grounds that they are seeking well-rounded students and diversity within classes. An admissions committee may give some weight to special accomplishments such as extracurricular involvements, and may assess the possible effects of extensive activity on the level of academic achievement. Activities indicating motivation for medicine or concern for others may be viewed especially favorably. 180

OCR for page 175
The Selection Process Three major phases typically comprise the medical school admissions process. They are (1) preliminary screening, (2) interviewing, and (3) final selection of applicanes.7 In the first phase, an applirant's suitability for medical school is determined, thus screening out those who appear unsuitable for further, more thorough consideration. At this state academic qualifications are assessed mainly on the basis of previous academic records and New MCAT scores. The academic record is appraised in terms of grades earned and such factors as the institution attended and the program of study. Other information on the application that is likely to be taken into account is state of residence, work in health or medical research, extracurricular activities, and perhaps racial/ethnic, socioeconomic, or community-size background. The second phase of the admissions process at most medical schools is that of interviewing applicants who appear to be the most serious contenders for medical school entry. The applicant has one or more interviews with admissions committee members, faculty, and often medical students or alumni. The primary purpose of these interviews is to evaluate noncognitive factors such as personality traits and motivation. Phase three, the final decision on the selection of a particular applicant, is usually carried out by the admissions committee, which meets to consider all available information about the individual. Admissions Committees Some information on admissions committee characteristics was obtained in a- case study of the admissions process at eight medical schools.7 Of these eight schools, one had an admissions committee of 70 members, with four subcommittees making independent selections, and the other seven had committees ranging from 11 to 17 members, with an average of 15. Four medical schools drew all committee members from among their own faculty and students, whereas about one-third of the members on committees of the other four schools were outsiders-- e.g., other university faculty , nonfaculty physicians, and community members. Six committees had medical student representatives. Most (nearly 70 percent) of the medical school faculty on selection committees of all eight schools were from clinical departments. Six schools had an M. D. f ram a clinical department as chair; in five of those six schools, that person devoted roughly half time to admissions dut ies . Unfortunately, little has been published regarding admissions committee procedures in making selections.3 Reports on the decision processes and specific criteria at this stage are particularly elusive, though available evidence indicates that the particular process seems to vary greatly among admissions committees. However, in making final selection decisions, most committees appear to use 181

OCR for page 175
"clinical judgment rather than a statistically weighted combination of cri teria. Therefore, the decision process would likely be complex and somewhat unsystematic. The actual role of admissions measures in final selection decisions has seldom been documented, but the characteristics, appropriate uses, and limitations of each type of measure are discussed in the next section. Evaluation of Admissions Measures Technical Considerations in Interpreting Measures If it is to be useful in selection, an admissions measure should provide an accurate assessment of a pertinent trait or quality. Evidence on three technical questions indicates the value of an admissions measure in decision making. First, does it consistently assess what is being measured--is it "reliable"? Second, is it measuring what it should be, and is information interpreted and used correctly--is its use "valid"? Third, is the measure "biased"? Reliability indicates the consistency of a measure. If a measure was perfectly reliable, an individual would earn the same score when being assessed repeatedly (assuming that practice made no difference). When numerical scores are assigned, a reliability estimate can be calculated. Similarly, the agreement between raters in the interview or letters of evaluation can be calculated. All admissions measures are imperfect, and therefore unreliable to some degree. Numerical scores may imply a level of precision that is inaccurate, so the potential exists for o~rer-reliance on them. New MCAT score reports include the reliability and standard error of measurement, a statistically derived index of the variability expected in the scores as a result of measurement error. Validity is defined as the "appropriateness of inferences from test scores".8 Scores on a valid test, or other admissions measure, accurately represent a specified domain of skills or knowledge, or predict other relevant variables. It should predict success in medical school or practice. An admissions measure and its use should also lack bias; that is, it should be fair to all applicants. Accordingly, it should not have an adverse impact on the likelihood of selecting a student as a result of factors unrelated to the demands of the medical school program (or practice ~ . The results of a measure should be equally valid and interpre table in the same way for all individuals and groups . Further, content should be balanced and demonstrably appropriate for medical school admissions decisions. And f inally, in an unbiased selection process, the information about each applicant is used as uniformly as possible in decision making. Not only must applicant assessment measures be of high 182

OCR for page 175
quality, but they should be used appropriately in admissions decisions. The decision process and weighting of criteria are important, as well as selection criteria per se. New Medical College Admission Test The New MCAT is used as an admissions decision-making tool in all medical schools in the United States.6 She New MCAT provides information regarding the applicant's academic readiness for medical school, and supplements college grade records. This exam was designed to show evidence of an applicant's overall academic aptitude, comprehension of the knowledge base, and reasoning ability required for successful medical school performance. New MC AT scores are reported in six areas: (1) Biology, (2) Chemistry, (3) Physics, (4) Science Problems, (5) Skills Analysis: Reading, and (6) Skills Analysis: Quantitative.* The first three science scores are based on questions in science knowledge and science problem solving. Areas of knowledge tested have been rated by medical school faculty as important prerequisites for entering medical students.9 Scores in the latter three categories (Science Problems, Reading, and Quantitative) "indicate skills in analyzing and using inf ormation including an assessment of problem solving ability.''] New MCAT results are particularly useful in selection decisions because all applicants can be compared on the same measure of achievement . Individuals f ram widely dif f Brent backgrounds and little-known colleges can be considered on the basis of the same tests, which can be advantageous to those who might not otherwise be coupe t i Live . Reliability of the New MCAT is a major advantage of tests scores over other admissions measures . (The probability is high that an applicant would receive nearly the same score on taking an equivalent test.) The major weakness of the New MCAT, as with other admissions tests, is the limitation in what is being measured. The New MCAT purports to assess factual recall and problem-solving ability, i.e., the cognitive dimension of the applicant Is readiness for medical school. Academic Perf ormance Previous academic performance in college and perhaps graduate school is also a clearly relevant criterion for medical school admissions, since grades reflect the level of performance in an *The New MCAT replaced the MCAT used through 1977, which was composed of four sections: (1) Science, (2) Quantitative, (3) Verbal, and (4) General Information. The science subtest functioned best as a predictor of medical school performance, particularly in the basic sciences portion of the curriculum. 183

OCR for page 175
academic setting somewhat similar to that of medical school, and they also indicate a combination of academic ability and motivation over a long period of time. The college grade point average (GPA) and science grade average appear to be the best selection criteria, but they have several serious weaknesses from the point of view of comparing applicants. The numerical GPA is not the uniform measure across applicants that it might appear to be. There are differences among institutions in standards of performance and grading systems. Even within a particular college, the course program of two applicants and the grading standards of individual professors are likely to differ. The grade inflation of the past few years also limits the usefulness of GPAs. In a review of literature on medical school admissions, Cuca et al. found that premedical grades predicted medical school grades somewhat bet ter than did MCAT scores . 3 As one might expect, the relationship of MCAT scores and GPAs to the Basic Sciences (Part I) exam of the National Board of Medical Examiners has generally been positive, though these results were not entirely consistent. However, the MCAT teas been revamped, so it usefulness as a predictor is uncertaine Also, both grades and MCAT were generally better predictors of basic science pert ormance in the f irst two years of medical school than of subsequent clinical pert ormance . Part of the dif ficulty in assessing predictors of medical school performance is the uncertain reliability and validity of the measures of that performance. Medical school grades are based on locally developed criteria in each medical school.ll However, evaluation in the basic sciences is primarily based on written examinations of knowledge, and sometimes on reasoning and logical analysis. Clinical clerkship evaluations are based primarily on observation and oral questioning. These measures have problems similar to those of admissions measures: the cognitive tests in basic sciences likely vary in content and quality, and the clinical evaluations are based in part on the faculty' s and house of f icers ' judgments, which have uncertain reliability. Many medical schools also use the National Board of Medical Examiners, Part I-Basic Sciences and Part [I-Clinical, which are multiple choice, as external reference points. Numerous studies have been done using admissions measures to predict medical school grades, National Board scores, and other factors.3 Yet, it is uncertain whether individual medical schools have adjusted admissions criteria on the basis of such evidence. The most reliable of medical school admissions measures are New MCAT scores, probably followed by grade point averages. The reli- ability of measures of noncognitive characteristics is generally considerably lower. Assessment of Personal Characteristics Admissions committees are concerned not only that entering students be able to suceed academically, but also that they have the personal 184

OCR for page 175
qualities that will be required of them, such as the ability to relate well to patients. Personal characteristics are typically considered in selecting applicants f or medical school admissions because these noncognitive traits are so important to performance in medical education and practice. Medical School Admissions Requirements 1983-84 describes these characteristics as follows. There is a universal feeling that medicine demands superior personal attributes of its students and prac- titioners. Integrity and responsibility assume major importance in the research laboratory and classroom as well as in relationships with patients and colleagues. Medical schools also look for evidence of other traits such as leadership, social maturity, purpose, motivation, initiative, curosity, common sense, perseverance, and breadth of interests. . . . Anyone who is considering a career as a physician must be able to develop the ability to relate to people effec- ~ ively. A new dimension of the need of this ability has developed with the increasing emphasis on a team approach to medical care. Because of the demanding nature of both the training for and the practice of medicine, motivation is perhaps the most salient nonintellectual trait sought by most admissions committees. . . . Despite the importance of such personal characteristics, the approaches to collecting relevant data are generally judged to be inadequate.12 These sources of information for selection decisions tend to be less reliable than are indicators of academic performance, because interpretation of interviews, letters of evaluation, and statements by applicants is inherently subjective. Not surprisingly, these selection criteria have generally been poorer in predicting medical school success than have MCAT scores and GPAs. 3 One criterion for the utility of an admissions measure is its contribution to improving a selection decision. That is, the addition of a noncognitive measure in combination with academic measures should better predict success in medical education and in practice than do academic measures used alone. Following is a discussion of the limited available research on the efficacy of each type of measure for selection purposes. The Applicant Interview The applicant interview is reported to be an integral part of the admissions process in 121 of the 122 U.S. medical schools.6 The importance of the interview in admissions was confirmed by a national survey in which 72 percent rated it ve ry important and 26 percent rated i t important.~3 Its ranked importance was second after GPA 185

OCR for page 175
Unfavorable information was most influential on ratings and selection decisions, and among raters there was more agreement about negative information than about positive. Wagner argued that, on the basis of the studies reviewed at that time, intelligence of the interviewee is the only characteristic that can be judged with satisfactory reliability.16 According to Ulrich and Trumbo, subsequent studies of the 1950s and 1960s rather consistently indicated two additional areas that also particularly contribute to interviewer decision and show greatest evidence of validity.17 These areas were roughly described as personal relations and motivation to work, the latter being akin to "motivation for medicine" in the selection of medical school applicants. The research cited above suggests clues for improving reliability and validity of the use of interview data, but further evidence of the utility of these data in the medical school admissions context is needed. Study is needed of both the interview process and the use of results in selection decisions. Reliability can only be established empirically, and carefully controlled validation studies are required to ascertain whether the use of interview results contributes to the selection decision. If interviewers do not agree on their evaluations of the same applicant, a random element is introduced into admissions decisions. Its efficacy has not been sufficiently demonstrated. Letters of Evaluation and Applicant's Statements Most medical school admissions commi ttees request that the applicant have letters of evaluation sent by the premedical adviser, undergraduate college faculty, and sometimes others. These letters typically amplify the academic record, discuss extracurricular activities, and comment on the applicant ' s personal qualities. The premedical adviser can probably assess personal qualities better than the interviewer can, because the adviser has often known the applicant over several years of undergraduate college. However, little research has been published on the reliability of these ratings. Anecdotal evidence suggests that, like the interview, categories lack definition, and ratings tend to be high. Thus, negative ratings have more effect on selection decisions than do the nearly universal positive ratings. Though some committees reportedly consider the letters of evaluation as important information for decision making, no published data have demonstrated their efficacy in improving admissions decisions. Even less information is available on the use of applicants' statements by admissions committees that require them. However, it is likely that evaluations of these statements would suffer from problems of low inter-rater reliability similar to those of the interview, as precise criteria for their use have not been established. 187

OCR for page 175
Personality Tests According to Medical School Admissions Requirements 1983-84, "few schools employ interest and personality inventories.' Use of a test to assess personal characteristics of applicants may seem appealing, but is problematic in the selection of individuals. Personality tests are fairly primitive in comparison with academic achievement measures.20 Personality tests tend to be relatively unreliable, and validity for medical school selection purposes would have to be established. A major problem alleged for personality tests is that they can be faked; that is, there is a tendency to give the apparently preferred response. In one experiment, scores changed significantly on three personality tests taken by medical school applicants and repeated following admission. 21 Even though applicants had been assured that personality test results would not be used in admissions decisions, they appeared to tailor their responses prior to admission. Overall, on the basis of available research, personality tests do not appear promising for use in selection decisions. Messick, in a review of several types of personality measures, concluded that none was psychometrically adequate for selection purposes. 20 Cough reached a similar conclusion in his appraisal of personality test use in the context of medical school admissions decisions. 22 Hutchins23 summarized the state of the art on personality tests as medical school selection instruments as follows: Three decades of research on noncognitive attributes using standardized personality instruments for the prediction of medical student outcomes has generated a quagmire of of ten conflicting, low yield results, based too often on methodo- logically faulty procedure, or where sound, limited in external validity by cons traints on the investigator to study subjects in a single school or a limited set of schools. Each with its own devoted group of adherents, all of the standardized personality instruments remain controversial. Other Considerations in Selection Decisions Medical school admissions committees often take into account other factors in selection decisions to meet particular goals of the school or society. For example, an attempt may be made to select individuals likely to both provide quality care, and to practice in underserved geographic areas and in primary care specialties. Individuals from a small town or rural area, for example, might be given some preference in selection, because there is some evidence that physicians coming from such areas are somewhat more likely to practice in similar areas, which tend to be medically underserved.24 Further, attempts are made to broaden the socioeconomic diversity in entering medical classes and to increase educational opportunities for underrepresented racial/ethnic groups. The rationale for 188

OCR for page 175
affirmative action programs for minorities is to compensate for past lack of equal opportunity and to make medical care for underserved populations better and more accessible. This necessitates both special consideration in selection and provision of educational enrichment programs for the educationally disadvantaged. Minorities on the average perform poorer on New MCATs and GPAs than do nonminorities.25 In order to select a greater proportion of minority applicants than would be selected on the basis of academic indicators alone, admissions committees have considered noncognitive criteria in their evaluation, taking into account the effects of external factors on performance. Sedlecek's research identified several characteristics that indicated better prognosis for minority students' perseverance and success.26 They were (1) positive self-concept, (2) understanding and dealing with racism, (3) realist), self-appraisal, (4) preference for long-range goals, (5) availability of strong support system, (6) successful leadership experience, (7) demonstrated community service, and (~) demonstrated medical interests. On the basis of this research, the AAMC designed the Simulated Minority Admissions Exercise. It provides training and experience for admissions committee members in judging those noncognitive characteristics likely to predict medical school success for blacks and Hispanics. In a survey of medical schools, three-quarters of respondents reported that by 1972, they had modif fed admissions criteria to select more minority applicants. Recruitment and Educational Programs for Minorities and Disadvantaged Several types of programs have been established in attempt to increase the number of underrepresented minorities who enter and successfully complete medical school.27 Programs range from academic preparation to counseling and exposure to information about medical careers. These programs are described below, although few evaluative data are available. Recruitment programs for high school students, such as those provided by the University of Arizona College of Medicine and the Mount Sinai Medical School, offer exposure to the medical profession for promising minority and other disadvantaged high school students. Counseling on career opportunities is provided; remedial assistance is sometimes available. Numerous undergraduate colleges and 44 medical schools sponsor special summer programs for aspiring medical students . 27, 28 The Harvard Health Summer Program, begun in 1969, was the first large, national, slimmer program established to prepare "disadvantaged" college students for medical or dental school. The program offered a combination of college courses, particularly in science, exposure to clinical medicine, and specific preparation for the MCAT. Other programs, such as the University of Kentucky Medical School program, place more emphasis on learning skills.27 189

OCR for page 175
Forty-nine medical schools offer summer programs for students accepted to their medical schools.28 These courses are designed to compensate for educational deficiencies and to prepare entering s tudents f or the rigorous medical school curriculum. They cover areas similar to those of the Harvard program, including study skills, science preparation, and of ten exposure to clinical practice or re sea rch . Data on Predictors of Selection Versus Rejection A study of 1976 applicants attempted to determine factors that did ferentiated accepted from rejected applicants.7 Regression analysis indicated that one-third to one-half of the differences between the two groups were explained by academic credentials (GPA and MCAT scores), and on related factors including age, educational level, etc. The following variables accounted for most of the explained variance: (1) undergraduate college selectivity, (2) GPA, and (3) MC AT scores. This study confirmed reports that academic aptitude and achievement are necessary, but not sufficient, conditions for admission to and success in medical school. Racial/ethnic identity also contributed to the differentiation. However, since so small a proportion of the variance was explained, other factors must also be very important in acceptance decisions; perhaps they include the personal characteristics data not collected nationally and therefore excluded from this study. Variables that failed to distinguish between the two groups nationally were socioeconomic background and plans for career-type, specialization, and practice location. It was hypothesized that admissions committees consider such plans premature and uninformed, and therefore too unstable for use as a selection factor. However, when factors related to selection were analyzed by individual medical schools, the majority of medical schools actually underselected the following: (1) those planning general practice as opposed to teaching or research, and (2) those planning general practice or other primary care specialties rather than nonprimary care specialties. The selection of a smaller proportion of applicants who prefer primary care can be explained by their lower GPAs and MCAT scores. Predicting the "Good" Physician An admissions measure is used primarily to select those applicants who are most likely to complete medical school successfully and to become good physicians. Its utility as a predictive tool can be evaluated by a longitudinal research study. That is, the relation- ships would be analyzed between scores on an admissions measure and diverse performance criteria throughout medical education and into practice. One question posed for the present study is as follows: "Can physician performance be predicted?" The question has several components: (1) What are the characteristics of a "good" physician? 190

OCR for page 175
(2) Can these characteristics be sufficiently described, precisely defined, and adequately measured?, and (3) Can comparable characteristics be precisely defined and adequately measured at the applicant level as well? A number of the complex problems related to these questions are discussed below. First, what is a "good" physician? This question has long been illusory.29 Price surveyed many providers and patients in attempt to list those qualit ies commonly believed to character) ze the good physician. 30 Wingard and Williamson, in summarizing the literature on predicting physician performance by medical school grades, pointed out that "there was no general agreement in the studies reviewed as to what constituted quality career performance. Some investigators emphasized the dimensions of performance and others stressed performance standards, while some concentrated on instruments of measurement. Few, if any focused on performance outcomes. 31 It would likely be difficult to specify precise definitions of a good physician that would be agreed upon. Some general categories such as motivation, ability to relate to patients, and problem-solving ability may be common to most definitions. Yet, it is also likely that the personality characteristics one would use to describe the prototypical, good physician would vary by specialty. For example, a primary care physician might appropriately take risks in making certain types of decisions, while a pathologist should generally be more cautious. Furthermore, descriptions of the attributes of a good physician are generally vague and nonquantifiable. Once the desired components of good clinical performance were established, measures would have to be developed for a complex variety of requisite knowledge, skills, and attitudes. In assessing physician performance, one would face measurement problems similar to those discussed above with regard to admissions measures . However, measures of physician competence are generally even more problematic than are New MEAT tests of science knowledge and problem solving, because diagnosis and treatment of a patient's problem are complex, involving data gathering, data interpretation, and problem solving. Barro's review article on physician performance measurement evaluates the range of approaches to clinical competence assessment, discussing the measurement problems associated with each. 32 Two of the largest-scale studies of physician performance exemplify one approach--direct observation of a physician in actual general practice . 33, 34 Observers rated general practitioners on six categories of pefo.=ance : (1) history, (2) physical exam, (3) laboratory work, (4) therapy, (5) preventive medicine, and (6) clinical records. All categories except history were further defined and weights were assigned. Barro indicated that three aspects "largely determine the validity and usefulness of the measurement method" : (1~ what performance dimensions are included, (2) how "good" and "bad" performance along each dimension are defined, and (3) the weight s assigned to each category . The observational approach may be highly valid. Yet, there are trade-of f s. In order to evaluate physicians during the natural course 191

OCR for page 175
of interaction with patients and thereby to gain validity, some reliability is sacrificed. The assessment situation lacks standardization in that each physician would encounter different patients, who would probably present different problems, so that comparisons among physicians would be difficult. This would affect reliability adversely, as would lack of precise evaluation standards and differences among raters. Other assessment approaches involve simulation of the physician- pat lent encounter . These "patient-management problems " describe the patient and the clinical problem, and ask for the physician's diagnostic or therapeutic decisions at various stages of managing the problem. Scoring of such problems can be standardized for reliability of results.32,35 However, validity is questionable in that the correlation is uncertain between a physician's performance on a simulated patient problem and an actual patient encounter. Even less research has been done evaluating the outcomes of medical treatment than in evaluating the process. Only a few physician encounters result in a measurable change in health status, and most indicators of outcome are not well developed. Interpersonal skills are important throughout patient management for purposes of (1) obtaining information for the history, (2) gaining the patient's compliance in carrying out the treatment regimen, and (3) treating nondisease problems. However, assessment of this aspect of physician performance is both difficult and little studied.32 Even if other personal qualities of a good physician could be enumerated and specifically defined, they would have to be translated into the experience level of the applicant, their stability over time would need to be ascertained, and more reliable measures developed. Assessment of such noncognitive factors is generally unreliable, and perhaps invalid, as discussed in the context of admissions measures. The ability to assess personal characteristics reliably for selection decisions is a long way off, and much farther off is the ability to use these data to predict who will become a good physician. Even though evaluation of a physician' s clinical competence is complicated, existing techniques could be refined and evaluations carried out. But prospects are slight for prediction of physician competencies on the basis of admissions data. Numerous methodological problems include attrition of part of the sample, inability to define intervening variables (e.g., medical training) and to isolate their effects, and interaction among variables. tone clue to the difficulty involved is the lack of relationship found in studies that have attempted to predict physicians' performance using medical school grades.29 The reviewers of these studies offered possible explanations for the low correlations: (1) inadequate selection procedures, (2) deficiencies in the grading system, and (3) the effects of intervening experiences. A further hypothesis is that the physician is basically a problem- solver, whereas grades may measure factual recall, though many 192

OCR for page 175
clinical clerkship curricula claim otherwise. Nonetheless, it seems likely that cognitive admissions measures would fare worse than would medical school grader in predicting physician performance, because the latter are closer both in time and clinical content to practice performance. The New MCAT problem-sol~ring section in science is designed to better predict clinical problem solving, but predic- tive validation data are not yet available nationally. Predicting Clinical Perf ormance The prediction of clinical performance in medical school gives some indication of the potential of admissions measures for predicting physician performance. The interview and academic criteria were studied at two medical schools. The usefulness as predictors of success in each of four years of medical school at Tulane was studied, using the following admissions criteria: overall interview rating; MCAT scores in Science, Modern Society, Verbal Ability, and Quantitative Ability; overall grade average and age.36 The science grade average and SCAT Science score best predicted basic science achievement in the f irst two years of medical school, but the predictive ability of overall grade average and the interview were greater ~ though low) f or success in clinical training during the f inal two years of medical school. Clinical success as measured by internship letters was predicted using admissions interview data, MCAT scores, and GPAs for five classes at the University of Missouri (Columbia) Medical School.37 Those students judged by interviewers to have high levels of maturity, nonacademic achievement, motivation, or rapport were two to three times more likely than others to receive outstanding internship recommendations. No significant relationship with internship rating was found for GPA, MCAT scores, or age. However, the interview results may have been somewhat influenced by other factors, because interviewers reviewed the applicant's entire file, including letters of reference and application materials. Summary In summery, premedical grades and New MCAT scores predict grades in the basic science portion of medical school. However, they are generally less successful at predicting performance in the clinical portion of the curriculum, and are not likely to predict physician competence. Measures of the personal characteristics of applicants are Less reliable, thus limiting their ability to predict comparable aspects of physician performance. Use of these measures could be refined to increase reliability, but the predictive validity would remain to be established, and its prognosis would still not be forecast to be high, given the inherent measurement problems. Validation data are needed on admissions measures, including the interview and New MCAT. Also, more follow-up data are needed on minority attrition and career patterns. 193

OCR for page 175
A mayor concern of medical school administrators is financial, and tuitions are rising rapidly to meet costs. While student debts are rising and expected to be much larger soon, the ability to pay is only beginning to intrude into admissions considerations in some schools. The effects of rising medical school costs on class mix should be monitored, and financing alternatives considered. Another concern is the effect of the declining numbers of college graduates on the applicant pool and entering medical school classes. The ef fects on class size, class mix, and quality should be determined . 194

OCR for page 175
REFERENCES 1. Medical Education in the United States, 1980-1981. Journal of American Medical Association 246:2893-3044, 1981. - 2. Erdman, J. B., Mattson, D. E., Hutton, J. G., and Wallace, W. F. Medical College Admission Test: Past, present, and future. Journal of Medical Education 46:937-946, 1971. - 3. 4. Cuca, J. M., Sakakenny, L. A., and Johnson, D. G., The Medical School Admissions Process: A Review of the Literature 1955-76. Washington, D.C.: Association of American Medical Colleges, 1976. Personal communication. Robert J. Boerner, Director, Division of Student Programs, Association of American Medical Colleges. . Levine, D. M., Weisman, C. S., and Seidel, H. M. Career Decisions of Unaccepted Applicants to Medical School: A Case Study of Reactions to a Blocked Career Pathway. Baltimore: Johns Hopkins University, 1974. 6. Medical School Admissions Requirements 1983-84 . Washington, D. C.: Association of American Medical Colleges, 1982. 7. Cuca, J. M., An Analysis of the Admissions Process to U.S. Medical Schools, 1973 and 1976. Washington, D.C.: Association of American Medical Schools, 1978. 8. Standards for Educational and Psychological Tests. Washington, D.C.: American Psychological Association, 1974. 9. New MCAT-Interpretive Manual. Washington, D.C.: Association of American Medical Colleges, 1977. 10. New MCAT Student Manual. Washington, D.C.: Association of American Medical Colleges, 1977. Hellinkoff, S. M., and Arthur, R. J., Medical education institutions. In Samph, T. and Templeton, B., eds. Evaluation in Medical Education. Cambridge, Mass.: Ballinger Publishing Company, 1979. 12. Willingham, W. W., and Breland, H. M., Selective Admissions in Higher Education. San Francisco: Carnegie Council on Policy Studies in Higher Education, 1977. 13. Puryear,.J. B., and Lewis, L. A. Description of the interview process in selecting students for admission to U. S. medical schools . Journal of Medical Education 56: BB1-~85, 1981. 14. Char, W. F., McDermott, Je Ge ~ Jre ~ Haning, W. F. III, and Hanson, M. J.. Interviewing, motivation, and clinical judgement. Journal of Medical Education 50:190-194, 1975. 195

OCR for page 175
15 . Kelly, E. L. A critique of the interview. Journal of Medica Education 32: 78-84, 1957. 16. Wagner, R. The employment interview: A critical summary. Personnel Psychology 2 :17-46, 1949. 11. Ulrich, L., and Trumbo, D. The selection interview since 1949. Psychological Bulletin 65 :199-210, 1965. 18. Wright, O. R. Jr. Summary of research on the selection interview since 1964. Personnel Psychology 22:391-413, 1969. 19. Clowers, M. R., and Fraser, R. T. Employment interview literature: A perspective for the counselor. Vocational Guidance Quarterly 26:13-26, 1977. 20. Messick, S. Personality measurement and college performance. Proceedings of the 1963 Invitational Conference on Testing Problems. Princeton: Educational Testing Service, 1964. 21. Rothman, A. I., Byrne, N., and Parlow, J. Longitudinal study of personality traits from application to graduation. British Journal of Medical Education 7: 225-229, 1973. 22. Cough, H. G. The recruitment and selection of medical students. In Coombs, R. B. and Vincent, C.E., eds. Psychosocial Aspects of Medical Training. Springfield, Ill.: Charles C. Thomas, 1971. 23. Hutchins, E. B. The Selection of Medical Students in Relation to Health Care Needs. Philadelphia: University of Pennsylvania School of Medicine, 1977. 24. Eisenberg, B. S., and Cantwell, J. R. Policies to influence the spatial distribution of physicians: A conceptual review of selected programs and empirical evidence. Medical Care 14:455-468, 1976. ~5. Waldman, B. Economic and racial disadvantage as ref. lected in traditional medical school selection factors. Journal of Medical Education 52:961-970, 1977. 26. Odegaard, C. E. Minorities in Medicine: From Receptive Passivity to Positive Action, 1966-76. New York: Josiah Macy, Jr. l Foundation, 1977 . 27~. Cadbury, W. E. Jr., Cadbury, C. M., Epps, A. C., and Pisano, J. C., eds. Medical Education Responses deco a Challenge. Mount Kisco, N.Y.: Futura Publishing Company, 1979. 28. 1982-83 Minority Student Opportunities in United States Medical Schools. Washington, D.C.: Association of American Medical Colleges ~ 1982 ~ 196

OCR for page 175
29. Gottheil, E., and Michael, C. Il. Predictor variables employed in research on the selection of medical Students. Journal of Medical Education 32 :131-147, 1950. 30. Price, P. B., et al . Performance Measures of Physicians . Salt Lake City: University of Utah, 1963. 31. Wingard, J. R., and Williamson, J. W. physicians career performance: Grades as predictors of An evaluative literature review. Journal of Medical Education 48: 311-322, 1973. . 32. Barro, A. R. Survey and evaluation of approaches to physician pert ordnance measurement . Journal of Medical Education 48: 1051-1093, 1973. 33. Peterson, 0. L., Andrews, L. P., Spain, R. S., and Greenberg, B. G. An analytical study of North Carolina general practice. Journal of Medical Education 31 :1-165, 1956. 34. Clute, K. F. The General Practitioner: A Study of Medical Education and Practice in Ontario and Nova Scotia. Toronto: . . University of Toronto Press, 1963. 35. Center for Educational Development. A Summary of Evidence Regarding the Technical Characteristics of Patient Management Problems. Chicago: 1967. University of Illinois College of Medicine, 36. Hoffman, E. L., Wing, C. W., and Lief, H. I. Short and long tell predictions about medical students. Journal of Medical Education 38:852-857, 1963. 37. Murden, R., Galloway, G. M., Reid, J. C., and Colwill, J. M. Academic and personal predictors of clinical success in medical school. Journal of Medical Education 53: 711-719, 1978. 197