An Assessment of Research-Doctorate Programs in the United States Engineering (1982) / Chapter Skim
Currently Skimming:
VII. Summary and Discussion
VII. Summary and Discussion
Pages 109-142
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 109... ...
of the reputational survey ratings and an examination of some factors that might nossiblv have influenced the ~ ~..
|
From page 110... ...
In all four disciplines the mean rating of scholarly quality of program faculty (measure 08) 1 Means for measure 16, ~influence" of publication, are omitted since arbitrary scaling of this measure prevents meaningful comparisons across disciplines.
|
From page 111... ...
Program Size 01 12 20 23 20 02 18 22 32 21 03 24 35 49 29 Program Graduates 04 .25 .13 .19 .21 05 5.9 6.9 6.7 7.0 06 .78 .69 .70 .71 07 .15 .19 .17 .17 Survey Results 08 2.7 2.7 2.6 2.7 09 1.6 1.6 1.6 1.7 10 1.1 1.0 1.1 I.0 11 · 9 University Librar`~ 12 Research Support 13 .37 .20 .27 .22 14 7819 7998 7679 7893 Publication Recoros 15 lu '~ zz ' t Total Programs 79 74 91 82
|
From page 112... ...
Research Support (Measures 13-14~. Measure 13, the proportion of program faculty who had received NSF, NIH, or ADAMHA4 research grant awards during the FY1978-80 period, has mean values ranging from as high as .37 in chemical engineering to .20 in civil engineering.
|
From page 113... ...
Of greater interest are the strong positive correlations between measure 02 and measures derived from either reputational survey ratings or publication records. The coefficients describing the relationship of measure 02 with measures 15 and 16 are greater than .60 in all disciplines except mechanical engineering.
|
From page 114... ...
Engin. Program Size 01 .53 .83 .78 .66 03 .82 .71 .82 .78 Program Graduates 04 .00 .01 -.09 -.11 05 .32 .16 .18 .17 06 .22 -.11 .10 -.01 07 .14 .05 .06 .12 Survey Results ~ 08 .83 .72 .76 .67 09 .83 .73 .75 .68 10 .07 .18 .11 .17 11 .79 .75 .81 .70 University Library 12 .40 .39 .47 .32 Research Suppor 13 .45 .39 .39 .33 14 .42 .51 .58 .58 Publication Records 15 .66 .73 .84 .69 16 .69 .65 .85 .52
|
From page 115... ...
In attempting such an adjustment on the basis of the regression of survey ratings on measures of program size, it was found that some exceptionally large programs appeared to be unfairly penalized and that some very small programs received unjustifiably high adjusted scores. Measure 02 also has positive correlations with measure 12, an index of university library size, and with measures 13 and 14, which pertain to the level of support for research in a program.
|
From page 116... ...
Engin. Program Graduates 04 .20 .18 -.03 .08 05 .43 .25 .21 .37 06 .27 .05 .13 .03 - 07 .25 .21 .12 .19 Survey Results 09 .99 .98 .98 .97 10 .31 .35 .23 .14 11 .96 .94 .94 .95 University Library 12 .41 .54 .56 .52 Research Support 13 .62 .62 .56 .52 14 .42 .57 .59 .52 Publication Records 15 .65 .62 .78 .70 16 .65 .57 .80 .57
|
From page 117... ...
The pattern of relations is quite similar for programs in all four engineering disciplines: moderately high correlations with measures of program size and reputational survey results (except measure 10) , and slightly lower correlations with publication measures.
|
From page 118... ...
Engin. Program Size 01 .48 .56 .63 .60 02 .42 .51 .58 .58 03 .50 .38 .62 .46 Program Graduates 04 -.02 .16 .20 .04 05 .00 -.02 .12 .19 06 .07 -.02 .08 .03 07 .02 .21 .30 ~ .13 Survey Results 08 .42 .57 .59 .52 09 .41 .55 .57 .52 10 .18 .04 .09 .07 11 .41 .58 .62 .61 University Library 12 .21 .26 .21 .20 Research Suppor t 13 .09 .29 .21 .16 Publication Records 15 16 .39 .44 .61 .49 .35 .36 .65 .42
|
From page 119... ...
Of particular interest are the moderately high correlations with all three measures of program size and with the reputational survey results (excluding measure 10~. Most of those coefficients exceed .60 and are generally somewhat larger for programs in electrical engineering.
|
From page 120... ...
Engin. Program Size 01 .46 .69 .83 .59 02 .69 .65 .85 .52 03 .65 .52 .81 .45 Program Graduates 04 .07 .17 .01 .02 05 .30 -.05 .10 .18 06 .23 .10 .19 .10 - 07 .14 .05 .14 .20 Survey Results 08 .65 .57 .80 .57 09 .63 .57 .78 .61 10 .34 .35 .17 .24 11 .68 .58 .84 .60 University Library 12 .12 .32 .51 .36 Research Support 13 .35 .26 .52 14 .35 .36 .65 Publication Records 15 .96 .95 .98 .88
|
From page 121... ...
For example, for engineering programs that received mean ratings of 4.0 or higher on measure 08, almost 95 percent of the evaluations requested on measure 08 were pro
|
From page 122... ...
08 SCHOLARLY QUALITY OF PROGRAM FACULTY Distinguished 6.5 7.8 6.5 5.4 5.9 Strong 14.0 14.5 13.8 13.4 14.3 Good 20.5 24.0 19.2 19.2 19.1 Adequate 16.7 20.5 14.5 15.3 15.6 Marginal 7.3 8.6 6.8 7.6 5.8 Not Sufficient for Doctoral Education 1.8 1.9 2.2 1.8 1.4 Don't Know Well Enough to Evaluate 33.3 22.6 37.1 37.4 38.0 TOTAL 100.0 100.0 100.0 100.0 100.0 O9 EFFECTIVENESS OF PROGRAM IN EDUCATING SCIENTISTS Extremely Effective 7.1 8.6 7.0 5.9 6.6 Reasonably Effective 31.6 34.3 28.8 30.0 32.6 Minimally Effective 16.9 20.4 16.6 15.9 14.1 Not Effective 2.8 3.8 3.3 2.4 1.7 Don't Know Well Enough to Evaluate 41.6 32.8 44.3 45.8 45.0 TOTAL 100.0 100.0 100.0 100.0 100.0 10 CHANGE IN PROGRAM QUALITY IN LAST FIVE YEARS Better 10.6 15.2 8.8 9.3 8.0 Little or No Change 30.3 36.5 26.5 28.1 28.7 Poorer 7.4 9.0 7.2 6.7 6.4 Don't Know Well Enough to Evaluate 51.8 39.2 57.5 55.8 57.0 TOTAL 100.0 100.0 100.0 100.0 100.0 11 FAMILIARITY WITH WORK OF PROGRAM FACULTY Considerable 16.8 22.8 16.3 14.4 12.9 Some 42.0 44.7 40.6 40.4 41.7 Little or None 39.6 31.1 41.5 43.1 44.3 No Response 1.6 1.5 1.6 2.1 1.1 TOTAL 100.0 100.0 100.0 100.0 100.0 NOTE: For survey measures 08, 09, 10 the "don't know" category includes a small number of cases for which the respondents provided no response to the survey item.
|
From page 123... ...
08 SCHOLARLY QUALITY OF PROGRAM FACULTY Mean Rating on Measure 08 4.0 or Higher 94.5 98.5 91.7 96.3 89.3 3.0 - 3.9 80.9 92.1 78.0 77.7 76.6 2.0 - 2.9 62.9 76.2 59.3 57.9 57.4 Less than 2.0 49.0 59.3 43.8 45.0 42.1 09 EFFECTIVENESS OF PROGRAM IN EDUCATING SCIENTISTS Mean Rating on Measure 08 4.0 or Higher 90.4 94.3 88.4 91.4 85.1 3.0 - 3.9 72.8 83.5 71.7 67.9 69.2 2.0 - 2.9 53.5 64.0 50.3 49.3 49.7 Less than 2.0 41.1 48.7 37.9 37.6 36.4 10 CHANGE IN PROGRAM QUALITY IN LAST FIVE YEARS Mean Rating on Measure 0E 4.0 or Higher 80.2 90.8 75.7 76.2 70.2 3.0 - 3.9 61.0 77.8 55.4 55.7 55.7 2.0 - 2.9 43.9 58.2 37.3 40.7 38.6 Less than 2.0 31.4 39.4 27.3 29.0 24.9
|
From page 124... ...
For programs with higher mean ratings the estimated errors associated with these means are generally smaller -- a finding consistent with the fact that survey respondents were more likely to furnish evaluations for programs with high reputational standing. The "split-half" correlations presented in Table 7.8 give an indication of the overall reliability of the survey results in each discipline and for each measure.
|
From page 125... ...
2.62 2.59 .86 .89 90 .95 Mechanical Engin. 2.75 2.73 .79 .80 81 .96 MEASURE 09: EFFECTIVENESS OF PROGRAM IN EDUCATING SCHOLARS Discipline Mean Rating Std.
|
From page 126... ...
126 TABLE 7.9 Comparison of Mean Ratings for 11 Mathematics Programs Included in Two Separate Survey Administrations N Survey All Evaluators Measure First X Second N X Evaluators Rating the Same Program in Roth Rllrv~vc First N X Second N X 08 100 4.9 114 4.9 50 4.9 50 4.9 09 90 2.7 100 2.8 42 2.7 43 2.7 10 74 1.2 83 1.2 38 1.1 34 1.2 11 100 1.6 115 1.6 50 1.5 50 1.6 Program B 08 94 4.6 115 4.6 48 4.6 50 4.5 09 81 2.6 91 2.5 40 2.6 39 2.5 10 69 1.0 82 1.0 37 1.0 36 0.9 11 98 1.4 116 1.4 50 1.5 50 1.5 Program C 08 86 3.4 103 3.6 42 3.4 44 3.5 09 56 2.0 66 2.1 28 2.1 29 2.0 10 55 1.1 62 1.3 30 1.2 27 1.4 11 99 1.0 116 1.1 50 1.1 50 1.0 Program D 08 74 3.0 93 3.0 37 2.8 38 2.9 09 50 1.8 48 1.6 27 1.7 16 1.6 10 46 1.4 52 1.5 24 1.4 23 1.5 11 90 1.0 113 0.9 46 1.0 46 0.9 Program E 08 69 3.0 95 3.1 39 3.0 46 3.1 09 40 1.8 60 1.9 25 1.8 30 1.8 10 36 0.8 58 0.9 24 0.8 29 0.9 11 96 0.8 115 0.9 52 0.9 52 1.0 08 63 2.9 90 3.0 26 3.0 32 3.1 09 35 1.8 46 1.7 10 1.6 13 1.8 10 32 1.1 43 1.1 11 1.3 12 1.2 11 95 0.7 115 0.8 43 0.7 44 0.7 Program G 08 69 2.7 92 2.8 39 2.7 39 3.0 09 35 1.7 45 1.6 17 1.7 19 1.7 10 36 1.1 43 1.2 17 1.1 19 1.2 11 85 0.9 116 0.8 46 0.9 46 0.9 Program H 08 58 2.2 73 2.5 36 2.2 37 2.4 09 32 1.3 43 1.3 22 1.2 19 1.3 10 30 1.5 39 1.5 20 1.7 17 1.4 11 90 0.7 116 0.6 51 0.7 52 0.6 Program I 08 55 2.0 74 1.9 30 1.9 30 2.0 09 33 1.0 41 0.9 19 1.0 18 0.8 10 27 1.2 31 1.1 15 1.1 13 1.2 11 99 0.5 115 0.5 50 0.5 50 0.5 Program J 08 51 1.5 67 1.5 26 1.4 28 1.4 09 31 0.8 36 0.7 14 0.6 14 0.7 10 26 1.2 23 1.1 14 1.2 12 1.3 11 96 0.5 113 0.3 49 0.4 48 0.4 Program K 08 33 1.2 48 1.2 17 1.1 21 1.4 09 19 0.8 21 0.5 11 0.6 8 0.4 10 12 0.8 15 0.9 5 1.0 5 0.8 11 99 0.2 114 0.2 48 0.2 47 0.2
|
From page 127... ...
It is evident from the data reported in Table 7.10 that mean ratings of the scholarly quality of program faculty tend to be higher if the evaluator has considerable familiarity with the program. mere is nothing surprising or, for that matter, disconcerting about such an association.
|
From page 128... ...
Since earlier reputational surveys had not provided such information, 10 percent of the sample members, randomly selected, were given forms without faculty names or doctoral data, as a "control group." Although one might expect that those given faculty names would have been more likely than other survey respondents to provide evaluations of the scholarly quality of program faculty, consistently large differences were not found (see Table 7.11) between the two groups in their frequency of response to this survey item.
|
From page 129... ...
ENIALUATOR'S FAMILIARITY WITH PROGRAM Considerable 99~9 99 9 100.0 99.8 99.9 Some 98.2 99.1 96.9 98.5 97.8 Little or None 21.1 31.7 17.1 18.6 18.4 TYPE OF SURVEY FORM Names 66.4 76.4 63.3 61.7 62.0 No Names 70.2 88.6 59.3 71.8 62.0 INSTITUTION OF HIGHEST DEGREE Alumni 98.5 100.0 97.2 98.2 Nonalumni 66.4 77.1 62.6 62.3 EVALUATOR'S PROXIMITY TO PROGRAM Same Region 79.0 85.1 78.3 75.8 75.7 Outside Region 65.0 76.2 61.0 60.7 60.1 NOTE: The item response rate is the percentage of the total ratings requested from survey participants that included a response other than "don't know.~" provision of faculty names apparently had a positive effect on survey sample members' willingness to complete and return their questionnaires in engineering disciplines. The mean ratings provided by the group furnished faculty names are lower than the mean ratings supplied by other respondents in all disciplines but chemical engineering (see Table 7.12~.
|
From page 130... ...
For purposes of analysis, programs TABLE 7.12 Mean Ratings of Scholarly Quality of Program Faculty, by Type of Survey Form Provided to Evaluator MEAN RATINGS CORRELATION Names No Names r N Chemical Engin. 2.68 2.47 .96 79 Civil Engin.
|
From page 131... ...
The findings presented in Table 7.14 support the hypothesis that alumni provided generous ratings -- with differences in the mean ratings (for Resee Appendix I for a list of the states included in each region. Resee Table 7.11.
|
From page 132... ...
m e answer is "very little." As shown in the table, only about one program in every three received ratings from any alumnus.'4 Moreover, the fraction of alumni providing ratings of a program is always quite small and should have had minimal impact on the overall mean rating of any program. To be certain that this was the case, mean ratings of the scholarly quality of faculty were recalculated for every engineering program -- with the evaluations provided by alumni excluded.
|
From page 133... ...
. The results of the analysis, which are presented in the mathematical and physical sciences volume of the committee's report, indicate that there is a high degree of correlation in the mean ratings provided by those in differing specialty fields within these two disciplines.
|
From page 134... ...
At the very least, the survey results provide a snapshot of these impressions from discipline to discipline. Although these impressions may be far from ideally informed, they certainly show a strong degree of consensus within each discipline, and it seems safe to assume that they are more than passingly related to what a majority of keen observers might agree program quality is all about.
|
From page 135... ...
The consequences of these differences in survey coverage are quite apparent: in the committee's survey, evaluations were requested for a total of 326 research-doctorate programs in chemical, civil, electrical, and mechanical engineering compared with 287 programs in the RooseAndersen study. Figures 7.1-7.4 plot the mean ratings of scholarly quality of faculty in programs included in both surveys; sets of ratings are graphed for 61 programs in chemical engineering, 57 in civil engineering, 66 in electrical engineering, and 61 in mechanical engineering.
|
From page 136... ...
FIGURE 7.1 Mean rating of scholarly quality of faculty (measure 08) versus mean rating of faculty in the Roose-Andersen study -- 61 programs in chemical engineering.
|
From page 137... ...
FIGURE 7.2 Mean rating of scholarly quality of faculty (measure 08) versus mean rating of faculty in the Roose-Andersen study -- 57 programs in civil engineering.
|
From page 138... ...
FIGURE 7.3 Mean rating of scholarly quality of faculty (measure 08) versus mean rating of faculty in the Roose-Andersen study -- 66 programs in electrical engineering.
|
From page 139... ...
FIGURE 7.4 Mean rating of scholarly quality of faculty (measure 08) versus mean rating of faculty in tne Roose-Andersen study -- 61 programs in mechanical engineering.
|
From page 140... ...
Nor should the reader place much stock in any small differences in the mean ratings that a particular program may have received in the two surveys. On the other hand, it is of particular interest to note the high correlations between the results of the evaluations.
|
From page 141... ...
What fraction have gone on to become outstanding investigators -- as measured by receipt of major prizes, membership in academies, and other such distinctions? How do program graduates compare with regard to their publication records?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.