Rethinking the Admissions Process: Evaluation Techniques That Promote Inclusiveness in Admissions Decisions

Filo Maldonado

Texas A&M Medical School

This paper discusses current debate regarding the medical school admissions process, in light of attacks on affirmative action and the subsequent exclusion, in many jurisdictions, of race or ethnicity as factors in admissions decisions. Within the work of admissions committees, some perspectives can be narrow with regard to the consideration of students with broad educational backgrounds, varied interests, and substantial accomplishments, particularly among medical schools. Hence, a discussion is timely and necessary as to the efforts and practices exercised by some to venture creatively and methodically beyond prevailing criteria and processes. These new approaches should be be characterized as concerted efforts to responsibly adapt to the mandates of law, as well as to effectively respond to the changing demographic trends, diversity, population increases, and health needs of the citizens of burgeoning minority groups.

The dismantling of affirmative action in Texas1 has compelled its public institutions to rethink the process of selecting applicants. It has also evoked new and creative perspectives that can be used to examine unique attributes of applicants and to better understand how applicants’ unusual or special life circumstances effect admissions decisions. The loss of affirmative action has also forced Texas institutions to explore how they can best select individuals who will become competent, humanitarian health care providers and who will most likely serve the health care needs of the communities from which they come, and provide direction and inducement for recruitment and retention strategies.

1  

Hopwood v. University of Texas, 78 F.3d 932 (5th Cir., 1996), cert denied, 116 S.Ct. 2580 (1996).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions Rethinking the Admissions Process: Evaluation Techniques That Promote Inclusiveness in Admissions Decisions Filo Maldonado Texas A&M Medical School This paper discusses current debate regarding the medical school admissions process, in light of attacks on affirmative action and the subsequent exclusion, in many jurisdictions, of race or ethnicity as factors in admissions decisions. Within the work of admissions committees, some perspectives can be narrow with regard to the consideration of students with broad educational backgrounds, varied interests, and substantial accomplishments, particularly among medical schools. Hence, a discussion is timely and necessary as to the efforts and practices exercised by some to venture creatively and methodically beyond prevailing criteria and processes. These new approaches should be be characterized as concerted efforts to responsibly adapt to the mandates of law, as well as to effectively respond to the changing demographic trends, diversity, population increases, and health needs of the citizens of burgeoning minority groups. The dismantling of affirmative action in Texas1 has compelled its public institutions to rethink the process of selecting applicants. It has also evoked new and creative perspectives that can be used to examine unique attributes of applicants and to better understand how applicants’ unusual or special life circumstances effect admissions decisions. The loss of affirmative action has also forced Texas institutions to explore how they can best select individuals who will become competent, humanitarian health care providers and who will most likely serve the health care needs of the communities from which they come, and provide direction and inducement for recruitment and retention strategies. 1   Hopwood v. University of Texas, 78 F.3d 932 (5th Cir., 1996), cert denied, 116 S.Ct. 2580 (1996).

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions The common criteria among Texas’ health-professions institutions in identifying and selecting underrepresented or disadvantaged applicants are: Economic, social, and/or educational background; Relationship to school alumni; Parents’ level of education; Whether the applicant would be the first generation of the applicant’s family to attend or graduate from an institution of higher education; Cultural factors such as bilingual or multilingual proficiency; Employment or other responsibilities, such as assisting in the raising of children or being legally responsible for other people as a parent or guardian, while attending school; Region of residence; Whether the applicant is a resident of a rural, underserved, or health professions shortage area of the state; Performance on standardized tests in comparison with the performance of other students from similar socioeconomic backgrounds; Personal interview; and Admission to a comparable accredited out-of-state institution My goal in this discussion is to address the question of how we proceed in identifying and selecting applicants who have the potential of becoming positive health professionals in light of the attacks on affirmative action and the use of race-neutral processes. I have provided an overview of some of the literature that indicates the importance of studying the relationships between traditional cognitive determinants and non-cognitive factors in selecting applicants and in predicting success in the health professions education environment. It is apparent among the authors cited that success in a health professions school is dependent not only on varied and balanced factors—some with stronger correlations than others—but also on an institution’s commitment to advise prospective applicants and provide support services for students perhaps considered vulnerable. My discussion centers on the methods used and issues faced by health professions schools in developing race-neutral admissions processes. I believe that there can be merit in such approaches, provided that there is a change in criteria and in the process of evaluation, as well as an institutional culture that places value on the uniqueness of an applicant. BACKGROUND What can be said about the relevance of non-cognitive variables in identifying and selecting underrepresented minority and disadvantaged applicants? Many medical school admissions committees take the position that an applicant who is motivated and has knowledge of the profession through experience; the skill to lead; resilience, especially in the face of adversity; and a propensity for

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions community service will most likely succeed as a medical student and medical professional.2,3,4,5,6,7,8,9 Most medical schools attach significant value to such traits when making selection decisions.10 According to the studies of Mavis and Doig,8 Tekian,9 Elam, Andrykowski, and Johnson,10 and Elam, Studts, and Johnson,12 the relationship of cognitive and non-cognitive variables to performance in medical school is significant. However, it remains difficult to determine which of the many noncognitive variables to measure and, then, how to measure or draw parallels from these attributes.8,9,10,12,13 Although noncognitive factors are important, it has become apparent that applicants admitted to some medical schools with strong noncognitive features may possess modest grade point averages (GPAs) and Medical College Admission Test (MCAT) scores.8,9 This has compelled many medical schools to track how these students perform in medical school and whether they are succeeding. Madison’s14 research on medical school admission and generalist physicians suggests that it is useful to look back at these admitted students and examine the outcomes in order to determine the relationship of cognitive and noncognitive factors considered in the admissions process. Elam et al.11 conducted this type of retrospective study to determine which cognitive and noncognitive variables among admitted at-risk applicants at the University of Kentucky College of Medicine contributed to their success in the first two years of medical school. After an extensive review of the summary 2   Mitchell, K.J. Traditional predictors of performance in medical school. Acad Med. 1990; 65:149–158. 3   Shen, H., Comrey, A.L. Predicting medical students’ academic performance by their cognitive abilities and personality characteristics. Acad Med. 1997; 72:781–6. 4   Sedlacek WE, Prieto DO. Predicting minority students’ success in medical school. Acad Med. 1990; 65:161–6. 5   McGahie WC. Qualitative variables in medical school admission. Acad Med. 1990; 65:145–9. 6   Walton HJ. Personality assessment of future doctors: Discussion paper. J R Soc Med. 1987; 80:27–30. 7   Mavis B, Doig K. The value of noncognitive factors in predicting students’ first-year academic probation. Acad Med. 1998; 73:201–3. 8   Tekian A. Attrition rates of underrepresented minority students at the University of Illinois at Chicago College of Medicine, 1993–97. Acad. Med. 1998; 73:336–8. 9   Elam CL, Andrykowski MA, Johnson MMS. Assessing motivation to learn: A comparison of ratings assigned by premedical advisors and medical school interviewers. The Advisor 1993; 14(1):2–6. 10   Elam CL, Wilson JF, Johnson R, Wiggs JS, Goodman N. A retrospective review of medical school admission files of academically at-risk matriculants. Acad Med. 1999, October; 74(10 Suppl):S58–S61. 12   Elam CL, Studts JL, Johnson MMS. Prediction of medical school performance: Use of admission interview narratives. Teach Learn Med. 1997; 99:181–5. 13   Tekian A, Mrtek R, Syftestad P, Roley R, Sandlow LJ. Baseline longitudinal data of undergraduate medical students at risk. Acad Med. 1996; 71(10 Suppl):S86–S87. 14   Madison DL. Medical school admission and generalist physicians: A study of the class of 1985. Acad Med. 1994; 69:825–31.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions statements made by admissions committee members regarding the students’ strengths and the concerns about the students’ potential performance, they concluded that the successful at-risk students (those who passed both the first and second years of medical school and Step 1 of the United Stated Medical Licensing Examination [USMLE]) were identified as 1) being scant on service, (2) more likely to be focused on their academics, and (3) less likely to scatter their attention while in medical school. On the other hand, the unsuccessful at-risk students (those who attained GPAs below 2.75 in the first and second years of medical school and repeated either the first year or the first and second years or failed Step 1 of the USMLE) were more likely to have had cognitive weaknesses, stronger personal characteristics, and fitting exposure to the profession. Table 1 presents the findings with regard to the frequency and consistency data for the summary statements regarding concerns about students’ projected performances. However, Elam et al.11 noted two critical limitations to the current study. First, the study was limited by the eccentricities of the students at one institution with a small number of minority students, and second, the study confined itself to the scrutiny of only preclinical variables. In addition, other perplexing factors such as the students’ perceptions of the learning environment and their efforts to obtain tutoring or other forms of academic support were not measured. In view of these findings and limitations, Elam et al.11 encouraged further examination of how “too much service in medical school” (p. S60) functions as a distraction and a predictor of poor performance among at-risk students. Since the findings showed that MCAT scores were predictive of success for at-risk white students but not for at-risk African-American students, it was recommended that this finding be replicated and that the dynamics of the relationship between MCAT scores and noncognitive variables be explored. These findings provide very useful information about the relative contributions of noncognitive variables and academic abilities in predicting academic success, and medical school admissions committees should use these findings to develop or augment programs to advise prospective applicants and provide intervention or support services for students considered at risk. A study from the Texas A&M College of Medicine extended the concept of the relevance of noncognitive variables in the selection of underrepresented or disadvantaged applicants and evaluated the effects on the acceptance or rejection of these applicants when giving different weighting to academic and interview scores. The study was an attempt at examining another means of achieving the goal “to produce doctors who will practice where they are needed most,”15 15   Edward JC, Maldonado FG, Calvin JA. The effects of differently weighting interview scores on the admission of underrepresented minority medical students. Acad Med. 1999, January; 74(1);59–61

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions TABLE 1 Summary Statements of Concerns Related to Performances of At-Risk Students at the University of Kentucky College of Medicine (1991–1994) Frequency*   Consistency† Concern Successful Unsuccessful p Value Successful Unsuccessful p Value Lack of service and leadership 73.5 23.5 .001 1.20 .47 .014 Lack of motivation or purpose 67.6 52.9 NS‡ 1.00 .59 NS Lack of exposure to medicine 50.0 23.6 NS .79 .24 .008 Negative personal characteristics 50.0 11.8 .008 .56 .12 .008 Cognitive weaknesses 76.4 94.1 NS 1.23 1.95 .017 Lack of adversity in life 20.5 23.5 NS .25 .25 NS Presence of distractions 8.8 35.2 .019 .15 .35 NS *Percentage of students who had this concern listed by at least one file reviewer (out of a total of 3). †Average number of file reviewers who listed this concern per student (range = 0–3). ‡NS=not significant; p _.05 which in this case were the underserved areas of Texas. Although the study concluded that increasing the weights on the interview scores consistently produced positive changes in the number of accepted underrepresented minority applicants, the numbers were nonetheless small. The study also set the stage for exploring whether different weighting of the interview score had an effect on the admission of problem students (those students who may require a disproportionate amount of faculty time and effort).15 Using the different weighting on the interview score to compare the numbers and different types of problem students may provide a glimpse at the issues facing problem students and the means of rendering support services for these students in crisis. Table 2 depicts the results, assuming the 70%/30% formula, which weighted the interview score 70% and the academic score 30%.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions TABLE 2 Demographics and Academic Information About 439 Applicants to Texas A&M University College of Medicine Who Would Have Been Affected by a Change in the Admission Process Weighting Formula, 1996–1997.*   Those Who Would Have Gained Acceptance Those Who Would Have Lost Acceptance Underrepresented minorities 3 0 Asian Americans 6 4 Caucasians 9 10 Disadvantaged students† 3 0 Average MCAT scores 9.67 10.81 Average interview scores 8.31 5.20 Average number of acceptances to other medical schools 1.22 1.29 *Applicants were offered or denied admission based on a score that weighted their academic and interview scores at 50% each. The changes shown on this table would have occurred if the formula had instead weighted the academic scores at 30% and the interview scores at 70%. †Two of the three disadvantaged applicants were underrepresented. Since admissions committees do not consider MCAT scores in isolation, it is difficult to predict the power of MCAT scores in conjunction with preadmission criteria. There are a number of studies that have examined the predictive relationships of MCAT scores across different variables and different populations. Wiley16 found a comparability of relationships between MCAT and the USMLE Step 1 scores for students who repeated the MCAT. His results showed no differences in predictive validity associated with use of the first set of scores as the predictor versus use of retest scores as the predictor. Koenig et al.17 studied the relationship between MCAT scores and basic science grades and USMLE Step 1 scores, which resulted in no difference for men versus women, and slight tendencies for over-prediction of performance for African Americans, Hispanics, and Asians. After investigating the predictive relationships for students taking a commercial review course against those who did not, Huff and 16   Wiley A. Predicting Medical School Performance with the MCAT: Does Retesting Make a Difference? Paper presented at the Annual Meeting of the American Educational Research Association, New York, NY, 1996. 17   Koenig JA, Sireci SG, Wiley A. Evaluating the predictive validity of MCAT scores across diverse applicant groups. Acad Med. 1998; 73:1095–106.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions Koenig18 found no difference among examiners. Similarly, Koenig and Swanson19 found no difference when comparing the predictive relationships between medical schools implementing problem-based curricula and those using traditional curricula and modes of instruction. More recently, Huff et al.20 investigated the predictive power of MCAT scores with performance in third-year clerkships, as well as the extent to which MCAT scores predict differences in performance among racial and ethnic groups. The results proved the usefulness of the MCAT in predicting clerkship grades, which accounted for at least 1/5 of the variance in clerkship grades, and demonstrated a reasonable predictive consistency across the racial and ethnic groups studied. When MCAT scores were considered with preadmission data, particularly science GPA, prediction was improved, especially among African Americans and Hispanics. All in all, results from the above studies punctuate the significance of using multiple factors in making admissions decisions and the stability of the predictive relationships across different population groups. The observations stemming from these studies indicate a need for future research into the use of additional variables that may predict success in medical school, such as diligence, motivation, communication skills, study habits, and other relevant characteristics.20 HOW THE USE OF RACE-NEUTRAL PRINCIPLES ACHIEVES DIVERSITY Why is it so compelling for us in medicine to achieve a diverse student body? Medical education has an obligation and a social contract to better prepare physicians to deal with the issues of disease and disability afflicting many Americans.21,22 According to Cohen,21 medical schools in the past have not adequately focused their attention on matters of health promotion and disease prevention. He pointed out that this shortcoming, along with the scant attention paid to behavioral determinants of ill health in medical school curricula and the lack of a research agenda that emphasizes ways to modify unhealthy human behaviors 18   Huff KL, Koenig JA. The impact of commercial review courses on the validity of MCAT scores for predicting performance on USMLE Step 1 (work in progress). 19   Koenig JA, Swanson DB. Examination of MCAT’s Ability to Predict USMLE Step 1 Scores and Students Trained with Problem-Based Instructional Programs. Paper presented at the 1996 Annual Meeting of the American Educational Research Association, New York, NY. 20   Huff KL, Koenig JA, Treptau MM, Sireci SG. Validity of MCAT scores for predicting clerkship performance of medical students grouped by sex and ethnicity. Acad Med. 1999, October; 74(10):S41–S44. 21   Cohen JJ. Missions of a medical school: North American perspective. Acad Med. 1999, August; 74(8):S27–S30. 22   Schroeder SA. Doctors and diversity: Improving the health of poor and minority people. The Chronicle of Higher Education. 1996 Nov 1;B5.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions might all contribute substantially to improving the lifestyle choices and health habits of Americans. Cohen21 believed that this situation can be augmented by “producing a greater stress on population-based health care and strengthening emphasis on disease prevention, health promotion, and the competency of the physician to treat patients from various cultural backgrounds” (p. S29). Another alarming problem in U.S. medicine is inadequate access to health care across population groups. According to the National Center for Policy Analysis,23 the problem of inadequate access stems largely from the fact that 41.4 million Americans, which by Census Bureau statistics comprises 17.7% of the non-elderly population, have no health insurance. Just as worrisome is the additional problem that many Americans who live in inner-city or remote rural areas have no access to health care, despite being insured. One compelling reason for this phenomenon may be an erosion of trust between patient and doctor. The development of commercialized approaches to managing health care in the country seems to be evolving into a system characterized as impersonal and untrustworthy.24 According to Cohen,21 the fact that physicians are encouraged to become more cost-effective in the practice of medicine is a principal reason threatening the fundamental ethic of medicine.21 He describes the fundamental ethic of medicine as one centering on altruism— “holding paramount the best interest of one’s patients, not oneself (p. S30). How can medical education scale back the erosion of trust in the doctor? One important part of the solution is that medical schools must try and select students into the study of medicine who possess the essential traits of altruism, compassion, honesty, and integrity.21 Cohen21 also believed that medical education cannot single-handedly solve the dismal problem of lack of access, but is nevertheless “responsible for addressing those aspects of the problem that it can influence” (p. S29). One issue high on medicine’s agenda is making certain that the medical profession mirrors the growing diversity of the American population. The evidence clearly shows that the numbers of underrepresented minority medical students (African Americans, Mexican Americans, Native Americans and Mainland Puerto Ricans) continue to be disproportionately low.25,26 Despite sufficient data proving that physicians from underrepresented minority groups tend to practice in medically underserved communities, white males continue to make 23   National Center for Policy Analysis. Explaining the Growing Number of Uninsured. Brief Analysis No. 251. 1998 January 12. 24   Mechanic D, Schlesinger M. The impact of managed care on patients’ trust in medical care and their physicians. JAMA. 1996; 275:1693–7. 25   Nickens H., Ready T, Petersdorf R. Project 3000 by 2000: Racial and ethnic diversity in U.S. medical schools. N Engl J Med. 1994; 331:472–6. 26   Petersdorf R, Turner K, Nickens H, Ready T. Minorities in medicine: Past, present and future. Acad Med. 1990; 65:633–70.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions up a high ratio of practicing physicians in this country.21,22,27,28,29 Therefore, to ensure that the healthcare needs of minority and indigent communities are met, physicians from underrepresented minority groups should be in these communities practicing medicine. Unfortunately, physicians from underrepresented minority groups continue to be scarce in the medical profession. In 1996, for example, Schroeder22 reported that African Americans and Hispanics represented only 4% and 5%, respectively, of the nation’s physicians. Another important piece of evidence supporting the need for more physicians from underrepresented minority groups is the extent to which physicians from specific racial and ethnic groups practice in communities with high proportions of black and Hispanic residents and uninsured patients. Studies by Komaromy et al.26 and Cantor et al.27 revealed that black and Hispanic physicians were much more likely to care for patients who were not only from their respective minority population groups, but who were uninsured or on Medicaid and that physicians who treated many minority, poor, and Medicaid patients were motivated to do so because they valued helping the disadvantaged more than making lucrative salaries or using the latest medical technology. In fact, black and Hispanic physicians derived about 25% of their salaries from Medicaid fees in comparison with 15% for white physicians. Another finding showed that black and Hispanic physicians were more likely to treat patients of underserved groups, particularly if the physicians’ parents had low incomes or less than a high school education. Therefore, the need is great. Medical schools, according to Schroeder,22 are producing a national resource. By enrolling more qualified underrepresented and disadvantaged applicants, medical schools have the opportunity to promote better access to health care—and in all probability, improved health—and to help fulfill medicine’s obligation to serve society’s needs. Can We Achieve a Diverse Student Body Using Race-Neutral Principles? It is obvious that American society has become more multicultural and racially and ethnically diverse. The evidence shows that the need to close the diversity gap in the medical profession has become increasingly urgent. For approximately 30 years, many medical schools have undertaken efforts to increase opportunities in medical education for qualified members of underrepresented groups.25 However, the current pressure to eradicate affirmative action policies 27   Komaromy M, Grumbach K, Drake M, et al. The role of black and Hispanic physicians in providing health care for underserved populations. N Engl J Med. 1996; 334:1305–10. 28   Cantor J, Miles E., Baker L, Baker D. Physician service to the underserved: Affirmative action in medical education. Inquiry 1996; 33:167–80. 29   Minority Students in Medical Education: Facts and Figures. 10th ed. Washington, DC: Association of American Medical Colleges, 1997.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions in conjunction with recent litigation has prompted many medical schools, particularly in Texas and California, to rethink how they select applicants from racial and ethnic population groups. Over the last five years, Texas A&M College of Medicine—along with the other seven Texas medical schools* and three dental schools†—has taken significant steps to develop an admissions process that is race-neutral in nature. This action was in response to the decision by the Fifth Circuit Court of Appeals in the Hopwood v. State of Texas case in 1996. In sum, the Fifth Circuit Court2 ruled: The use of race to achieve a diverse student body whether as a proxy for permissible characteristics, simply cannot be a state interest compelling enough to meet the steep standard of strict scrutiny. These latter factors may, in fact, turn out to be substantially correlated with race, but the key is that race itself not be taken into account. Thus, that portion of the district court’s opinion upholding the diversity rationale is flawed. The Hopwood decision has posed a challenge to the College of Medicine’s efforts to enroll significantly more underrepresented minority students. Although the admissions committee attempted to modify its use of algorithms throughout the process of evaluation and selection, a new plan went into effect in 1998 that altogether eliminated the algorithm at the time of selection. The new plan called for: a mindfulness of the vision and mission of the institution in assessing and selecting students; a more inclusive approach in assessing cognitive abilities; a broad-minded scrutiny of applicants’ non-cognitive characteristics at the pre-interview and interview phases of the evaluation process; enhanced interview techniques; improved protocol for admissions committee deliberations; and frequent self-monitoring. The admissions committee embarked on its new plan by first taking into consideration the Health Science Center’s (HSC) and the College of Medicine’s (COM) mission and institutional goals when assessing the relative importance of applicants’ MCAT scores, academic records, and various personal and experiential qualities. This mission, along with the institution’s goals, became the philosophy by which the admissions committee guided and directed the admissions process. An important factor comprising this transition included the formation of three highly focused admissions committee task force groups to study criteria for interviewing applicants, the interview protocol, and admissions committee deliberations. With this focus, the admissions committee then im- *   Texas Medical Schools: Texas A&M, Texas Tech, UT-Southwestern, UTMB-Galveston, UT-Houston, UT-San Antonio, Baylor, U North Texas-Texas College of Osteopathic Medicine †   Texas Dental Schools: TAMUS Baylor-Dallas, UT-Houston, UT-San Antonio

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions plemented a system by which to weigh MCAT scores and grade point averages (GPA) and to sort applicants in a way that would jeopardize neither the integrity of academic criteria nor the breadth of choice of qualified applicants. The equal weightings given to the MCAT (50%) and the overall GPA (50%) produced an “Academic Score,” which was based on the admissions committee’s judgment as to how well the knowledge and skills tested by the exam corresponded to the requirements of the medical school curriculum. This method was adopted in large part because of the findings of the AAMC’s Predictive Validity Research Study,20 which utilized performance data of the 1992 and 1993 entering classes from 14 participating medical schools. The outcomes showed a strong correlation between undergraduate GPA and performance in medical school and between MCAT scores and performance in medical school. For example, the study showed that 34% of the variation in cumulative medical school GPA could be explained by medical students’ undergraduate GPAs; 41% of the variation in cumulative medical school GPA could be explained by medical students’ MCAT scores; and that 58% of the variation in cumulative medical school GPA could be explained by MCAT scores and undergraduate GPA. The study also revealed that 42% of the variation in medical school grades was not explained by students’ MCAT scores and undergraduate GPA. In fact, some of the variability in medical school performance might be attributed to or explained by other academic or experiential factors, such as: major grades in honors or graduate courses or additional degree (graduate or baccalaureate); extraordinary educational achievements or experiences; community service activities; leadership experiences; and character and motivation of the applicant. Mindful of the importance of non-cognitive factors in 1) facilitating the selection of students, 2) remedying the serious lack of diversity in the student body, and 3) abiding by the Hopwood Decision, the admissions committee implemented some extraordinary changes. First, the committee made the decision to widen the range of applicants considered for interview via the academic score and to screen a sizable number (900–1,000) of applicants. This decision was based largely on careful consideration of the distribution of academic scores for the 1997–1998 application years and the need to make better-informed decisions based on information provided by a new screening instrument rather than using solely an algorithm. For example, academic scores (raw scores from 1–1,000 points) within the range of 800–1,000 showed that approximately 20% of applicants (286 from a pool of 1,419) had GPAs and MCATs averaging 3.81 and 27, respectively. The committee, therefore, decided to give these applicants an automatic or high-priority interview (without screening), ensuring that the best of the applicants,

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions Avg. MCAT scores for HA applicants: VR–9 PS–9 BS–9 MCAT score range for HA applicants: VR 6–13 PS 7–13 BS 8–14 Average U-GPA for HA applicants::   3.81   GPA range for HA applicants:   3.10–4.00   Avg. Academic Score-HA applicants:   836   TABLE 4 1997–1998 Application Year (1,419 Total Applicants) Middle of the Pool Applicants—959 (Academic Scores of 799 to 651) Applicants by Race & Ethnicity # Middle of Pool Applicants % of Total Applicants   # Middle of Pool Disadvantaged   % of Total Applicants American Indian/ Alaskan Native 8 0.56%   0   0 Asian Americans 207 14.60% 61 4.3% African Americans 27 1.90% 2 <1% Hispanic 72 5.10% 12 0.85% White/Caucasians 567 40.00% 87 6.1% Other 78 5.50% 4 <1% Total Middle of Pool (MP) Applicants 959 68.00% 166 11.7% Avg. MCAT scores for MP applicants:   VR–9 PS–9   BS–8   MCAT score range for MP applicants:   VR 3–13 PS 4–13   BS 4–13   Average U-GPA for MP applicants::   3.45   GPA range for MP applicants:   2.39–4.00   Avg. Academic Score-MP applicants:   733  

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions TABLE 5 1997–1998 Application Year (1,419 Total Applicants) All Other Applicants—174 (Academic Scores of 650 and below) Applicants by Race & Ethnicity # All Other Applicants % All Other Applicants # All Other Disadvantaged % All Other Applicants American Indian/ Alaskan Native 0 0 0 0 Asian Americans 51 3.6% 6 <1% African Americans 4 <1% 2 <1% Hispanic 18 1.3% 11 0.78% White/Caucasians 84 5.9% 18 1.3% Other 17 1.2% 2 <1% Total All Other (AO) Applicants: 174 12.3% 39 2.7% Avg. MCAT scores for AO applicants: VR–6 PS–6 BS–7 MCAT score range for AO applicants: VR 2–11 PS 3–11 BS 2–11 Average U-GPA for AO applicants:   3.05   GPA range for AO applicants:   1.80–3.95   Avg. Academic Score-AO applicants:   598   These simple data illustrated a consistent contrast in how the applicants were compartmentalized and revealed the complex workload to be undertaken in the screening process. After analyzing the academic scores and post-interview committee scores, it became apparent that we were correct in expanding our hunch to expand our efforts to the middle of the pool. Our analyses also showed that while a substantial number of our middle-of-the-pool applicants enrolled in the medical school, at most, only 19% of our high-academic applicants did so. We found this compelling, and, hence, increased our efforts to give close attention to the middle range of applicants. We were also encouraged by the high

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions number of qualified underrepresented and disadvantaged applicants (nearly 12%) in the middle-of-the-pool group compared with the other two groups. The data provided a number of compelling reasons to scrutinize and screen in greater depth the middle-of-the-pool group and to interview more of these applicants. One of the most pivotal reasons was the increased likelihood of interviewing more underrepresented and disadvantaged applicants. The screening instrument became the key element in our endeavor not only to interview more qualified and potentially acceptable applicants but also to interview more qualified and potentially acceptable underrepresented and disadvantaged applicants without factoring in race or ethnicity. All members of the committee agreed that screening, although time-consuming, was a fairer way to judge our applicants than simply using the quantitative algorithm. The next challenge was to create a screening instrument that would widen our field of vision even further. A brief study was conducted by three people on the committee (the associate dean for student affairs and admissions, the assistant dean for admissions, and the chair of the admissions committee) to examine existing and new screening criteria as well as the process used to evaluate applicants of the 1997–1998 cycle.30 The results of the study were presented to a subcommittee of the admissions committee, who in turn discussed all facets of the instrument and made some revisions. The instrument was subsequently sent to full committee and was adopted unanimously as the screening instrument for upcoming application cycles. The final version of the screening instrument had four categories (with space for comments) whose scores totaled to a maximum of 100 points. Table 6 depicts the M.D. Admissions Screening Evaluation Form. 30   Edward JC, Maldonado FG, Engelgau, GR. Beyond affirmative action: One school’s experiences with a race-neutral admission process. Acad Med. 2000, August; 75(8);806–815.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions TABLE 6 Screening Evaluation Form–2000 Applicant ______________ SSN ______________ Date Screener ____________________________ Signature Score each factor between the range of points given, then provide a total not exceeding the maximum score. ACADEMIC PERFORMANCE AND INTELLECTUAL CAPACITY Score Between 0–10 Points Quality of Educational Institution Most Competitive–10; Highly Competitive–9; Very Competitive–8; Competitive–7; Less Competitive Texas–6; Less Competitive Out of State–5; Non-Competitive Texas–4; Non-Competitive Out of State-3 or less. (See Barron.)   Academic Achievements Overall & Science GPAs, GPA for last 45–60 credit hours, Rigor of Major, Graduate courses or degree, Grades in premedical course requirements, MCAT.   Extraordinary Educational Achievements or Experiences Academic scholarship(s), Academic recognition awards, Research fellowship(s), competitive internships, EMT certification, Paramedic.   TOTAL SCORE (MAXIMUM 30 POINTS) =   COMMENTS: HUMANISM, DEDICATION TO SERVICE, AND CAPACITY FOR EFFECTIVE INTERACTIONS Score Between 0–8 Points Participation in Community Human Service Activities or Al truism Active in improving the community or society in which he/she lives.   Leadership in School Organizations or Projects Holding positions of leadership or taking a lead in projects in school or community.   Clinical or Health Care-Related Experiences Volunteer or job-related experiences-Shadowing, providing ancillary support as a staff employee or volunteer of a hospital, clinic, mission outreach program, etc.   Quality of Personal Statement Genuine, original, well organized, coherent, and substantive.   Motivation for Medicine as a Career Focus for the most part is on the humanistic concern for others.   TOTAL SCORE (MAXIMUM 40 POINTS) =   COMMENTS:

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions SPECIAL LIFE EXPERIENCES Score Between 0–5 Points Circumstances Indicative of Some Hardship or Adversity Financial difficulties; A death in the immediate family; Personal illness; Educational disadvantage; Family illness; Disability; A medical condition; Other condition of suffering or extraordinary responsibility* *As a student, applicant was responsible for other people as a parent or guardian.   Need to Work, and Impact on Life or Educational Circum stances Steady job(s) while attending school necessitating a consistent 20 or more hours per week to meet financial obligations or to alleviate financial burden.   First-Generation College Student and Educational Level of Parents First-generation college student and parents have a high school or less education.   TOTAL SCORE (MAXIMUM 15 POINTS) =   COMMENTS: OTHER COMPELLING FACTORS Score As Shown Below Supportive Letters of Evaluation from (primarily) Professors and Others Health professions committee or individual letters from professors and others: Supportive; Strongly supportive; Problematic "6 points Areas of Interest in Medicine (Refer to Secondary Application, Item 9) Evidence of interest in a practice in: Primary Care; Rural Medicine; Underserved Area "3 points Area in Which Applicant Lives (Refer to Secondary Application, Items 3 and 3B): Rural; Medically Underserved; Low-Income "3 points Awareness and Knowledge of Cultural Factors as They Impact on Health Care Evidence of experiences or skills (e.g., foreign language) sensitizing applicant to other cultures and the human condition. "3 points TOTAL SCORE (MAXIMUM 15 POINTS) =   COMMENTS: GRAND TOTAL (Maximum 100 Points) =   "Second Screen Required because score is less than a total of 55 points. "Second Screen Requested for other compelling reasons (Comments Required) COMMENTS:

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions At the end of his or her evaluation, the evaluator (one of 19 members of the admissions committee or the assistant dean for admissions) could recommend an interview or decide not to interview an applicant, regardless of the number of points given. The number of points set for granting an applicant an interview was 55. This number was determined by one standard deviation (9.8) below the mean of the distribution of screen scores (64.3). The assistant dean for admissions and chair of the admissions committee could slide this score up or down depending upon the distribution of screen scores and the standard deviation during the screening process, but they rarely exercised this option. Instead, they opted to make any adjustment to the screen score cutoff point only after studying the results of the previous year(s), then applying it to the next application season, if necessary. Although a screen score cutoff point was set, an evaluator could exercise his or her discretion to recommend that an applicant not be interviewed and have the application screened by a second person. Applications with scores in the range of two standard deviations below the mean would automatically be screened by a second person. This broad-minded scrutiny of applicants’ academic and non-cognitive characteristics at the pre-interview level allowed us to: identify a broad range of acceptable applicants regardless of race or ethnicity; make judgments about the suitability of each applicant to carry out the mission of the medical school; and evaluate more deeply character, motivation, and life circumstances. If we are to select applicants who will fulfill the complex health care needs of our society and reach out to the wider human community, we must make significant investment of both our emotional and our cognitive selves. This investment extends itself well into the interview process and committee deliberations. There is, therefore, a substantive process in interviewing and deliberating about applicants. In fact, applicants often leave the interview encounter knowing that they have been closely observed and that we have formulated any number of assumptions about them. Our perception of them often depends on how they measured up as people and how they might measure up as professionals. One way we felt we could improve the admissions committee’s ability to broaden its selection of qualified applicants was to change how we deliberated about post-interview applicants during committee meetings and how we ranked applicants. For example, in committee deliberations, the admissions committee was principally charged with evaluating applicants selected for interview and making recommendations for admission to the dean of the College of Medicine. Based on the interview scores (using a scale of 1–10) the committee was called by the chair to exercise nine steps in decision making: Vote as a committee (with no discussion) to accept applicants whose interview scores average 9.0 or better.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions Vote as a committee (with no discussion) to reject applicants whose interview scores average 6.9 or less. Discuss and rate applicants who received interview scores that average in the 7–8.9 range. (Committee members rate individually the applicants based on the information provided during discussion, which includes the interviewers’ evaluation, record of college work, MCAT performance[s], and other pertinent details.) Discuss and rate applicants who have received interviewers’ scores with a discrepancy of three points or more. (Again, committee members rate individually the applicants based on the information provided during discussion.) Vote as a committee (with some discussion) to recommend applicants for the M.D./Ph.D. program whose interview scores average 8.0 or better. Vote as a committee (with some discussion) to reject M.D./Ph.D. applicants whose interview scores average 6.9 or less. Present applicants using the following protocol: Avoid superfluous information in individual presentations such as hometown, school, major, GPA, and MCAT scores. (This information is already provided in the Biographical and Academic Profile Sheet included in the committee packet.) Present only the most salient points or observations derived from the interview, outlining briefly the strengths and/or weaknesses within each category of the Interview Evaluation Form. Keep the total presentation to approximately three minutes. Score the applicant (using the 1–10 scale) on a ballot sheet (which is considered secret and confidential). Rate applicants on the ballot sheet as each applicant is presented and discussed. Submit ballots to the assistant dean for admissions for computer tabulation and ranking of applicants. This approach allowed for greater focus and discussion on the applicants with acceptable interview scores in the range of 7–8.9, which was the range of scores received by the majority of those interviewed. The committee also gave itself the flexibility to present and discuss any applicant who was accepted or rejected by vote. Each week, committee scores for each applicant interviewed were calculated by simply determining the average from the ballots submitted by each committee member. Using the committee score in this way gave greater weight to the interview evaluations and committee deliberations. As each week of interviews elapsed, applicants were rank ordered based on the committee scores. Applicants with committee scores were further distinguished by ranking them on three standards in the following sequence: (1) committee score, (2) overall GPA,

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions and (3) total MCAT score. All in all, changing from a process that was largely quantitative to one involving more extensive evaluation and deliberation of noncognitive factors accomplished two important goals: 1) the criteria as set preserved a sound basis of academic qualifications, and 2) the measures as set allowed for broad-minded scrutiny of applicants’ non-cognitive characteristics at the pre-interview and interview phases of the evaluation process. Table 7 presents some descriptive data about the applicant pool and the enrolled entering classes from 1995 through 2000.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions TABLE 7 Texas A&M University System Health Science Center College of Medicine—Entering Classes of 1995–2000 Progression Through the Admissions Process 1995 1996 1997* 1998 1999 2000 No. % No. % No. % No. % No. % No. % All Applicants 1518 100 1576 100 1412 100 1419 100 1442 100 1782 100 URM† Applicants 154 10 156 10 140 10 141 10 151 11 265 15 All Interviewed 486 32 401 26 438 31 480 34 553 38 486 27 URM† Interviewed 67 14 63 15 25 6 32 7 42 8 44 9 All Accepted 165 34 200 50 208 47 192 40 208 38 168 35 URM† Accepted 31 19 30 15 9 4 18 6 19 9 23 14 All Matriculated 64 100 64 100 64 100 64 100 78 100 64 100 URM† Matriculated 6 9.4 10 15.6 5 8 4 6.3 6 7.7 2 3 *First class affected by the Hopwood decision. †Underrepresented minorities.

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions The data clearly show the effect the Hopwood decision has had on the College of Medicine’s enrollment of underrepresented minority applicants. One can assume that the decrease in the numbers of underrepresented minority applicants interviewed and enrolled in 1997 were affected by the Hopwood decision. Since the admissions committee increased its efforts to commit more time and to contemplate more deeply the admissions process for the entering classes of 1998, 1999, and 2000, more underrepresented minority applicants are being interviewed and have been accepted. However, the number of underrepresented minority applicants who enroll, despite the substantive changes in the evaluation process and the increased efforts to select applicants who fit the mission and ethos of the College of Medicine, still remains alarmingly low. Although it is encouraging to see the admissions committee making such strides, the trend is far below the rate necessary to produce physicians who will meet the needs of the burgeoning minority population in the state of Texas.30 Like Steven Schroeder, President of the Robert Wood Johnson Foundation, we hope that efforts like making substantive changes in the admissions criteria and process, coupled with more efforts to recruit applicants from disadvantaged backgrounds, will result in achieving greater racial and ethnic diversity in the medical profession. And as we approach the fruition of this goal, we will simultaneously improve access to health care—and, in all probability, achieve better health—for a large number of Americans.22 WHAT HAVE WE LEARNED? Developing and implementing a race-neutral admissions process can indeed challenge an institution’s comfort with rigid algorithms, unquestioned admissions criteria and processes, and aloofness from both public and legislative scrutiny. In fact, under these circumstances, the institution may have to engage in practices it deems uncommon and uncomfortable, including 1) frequent self-monitoring, 2) a more inclusive approach in assessing cognitive abilities; 3) a broad-minded perspective of applicants’ non-cognitive characteristics; 4) increased workload; 5) a constant mindfulness of the mission of the institution; 6) accountability; and 7) learning to act in concert to develop legal and effective criteria and processes for selecting applicants who will fulfill the complex needs and issues of the future. It is painstaking work to change a medical school perspective from one that is insular and narrow in seeking students with broad educations, varied interests, and substantial accomplishments to one that examines unique attributes of applicants in new, creative, and effective ways. The work is often characterized by frustration and discouraging results. However, successful agents for change at a medical school usually connect or partner with forces outside of the institution, such as high schools, colleges and universities, other medical schools or health

OCR for page 305
The Right Thing to do, The Smart Thing to do Enhancing Diversity in the Health Professions professions, community agencies, and legislators—all of which can spur action and facilitate institutional change. The paramount challenge to any medical school committed to achieving diversity is to honestly assess itself and address the modifiable causes of rigid, laissez-faire practices. Because those underlying causes are passive and narrow, such a medical school must invest in redesigning its admissions processes and recruitment of members of minority groups to select applicants who fit the mission and ethos of the institution and who will likely care for patients from rapidly growing minority groups.