Click for next page ( 310


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 309
appendix B The Committee’s Approach to Its Review T his appendix describes the approach taken by the committee in ful- filling its charge to review the effectiveness of the foreign language, area, and international studies programs of the U.S. Department of Education (ED). It describes the activities we undertook and the conceptual model we developed to examine program effectiveness. The appendix in- cludes a number of attachments: (B-1) a summary of studies evaluating the Title VI and Fulbright-Hays (Title VI/FH) programs, (B-2) a list of written comments submitted by people and organizations knowledgeable about the programs, (B-3) agendas of the public meetings held, and (B-4) the guides we developed for the site visit interviews held at eight universities. To begin our study and ensure a complete understanding of the pro- grams’ statutory history and missions, including how their missions relate to those of other similar federal programs, the committee (1) reviewed descriptive information about the Title VI/FH programs and related federal foreign language and international education programs and (2) commis- sioned an analysis of the program’s legislative history. As a result of our review of congressional language and the statutes and the judgment of our individual members, the committee identified other questions in addition to the eight key areas specified by Congress, aimed at assessing the effectiveness of the programs at meeting their statutory missions. We asked: • How responsive are the programs to changing national priorities? What is the role of Title VI programs in relation to other programs in ad- dressing national needs? 0

OCR for page 309
0 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES • To what extent are the programs producing best practices that are used broadly in the international education arena? How are best practices identified and shared? • To what extent does Title VI help channel other university re- sources toward national goals and fund programs or resources that they might not otherwise fund? • How effective are the programs at coordinating with one another within programs, within universities, and across universities? • How are the programs administered? What can and should be done to improve future monitoring of the programs? To guide the review and provide a model for how the programs sup- port the eight key questions and are expected to lead to particular results, we outlined a potential conceptual model. This model, which illustrates the interconnectedness of the programs, is discussed in more detail below. To assess the degree to which the programs meet the combined set of questions, the committee undertook a series of activities: • Reviewed all extant evaluations. The committee conducted a lit- erature review, requested from ED copies of evaluations funded through its International Research and Studies (IRS) Program, and asked experts in the field about available evaluations. The committee reviewed all extant evaluations identified, although very few were available. (See Attachment B-1 for a summary of available evaluations.) • Reviewed program monitoring data, selected grant applications, his- torical financial data, and written comments from experts and officials. The committee requested and received from ED all available program data, including historical funding1 information and program monitoring data collected via the Evaluation of Exchange, Language, International and Area Studies (EELIAS) database. We had hoped that the EELIAS database would be a rich source of information about program performance. Instead, the problems we encountered using EELIAS and the comments from grantees on our site visits led us to make the database itself a focus of our review (see Box B-1 for a summary of database issues). In addition to the EELIAS data, we also reviewed the FY 2006 performance plan for international edu- cation and foreign language studies programs (domestic, overseas, and the Institute for International Public Policy, IIPP) and the Program Assessment 1 Although committee staff worked with ED staff to clarify obvious errors or questions about the data provided, the data were accepted as accurate and not independently verified by the committee.

OCR for page 309
 APPENDIX B BOX B-1 Technical Issues with EELIAS Database The committee encountered multiple technical issues with the EELIAS database that hampered our ability to use the data for analyses: • Lack of internal controls. Although the text of data entry screens indicates that certain fields are required, the system does not always have internal con- trols that enforce that requirement before data are submitted. • Use of open-ended questions. For questions that might require a large num- ber of response options, responses are left as open-ended questions. One example is a recently added question asking whether a standardized proficiency test was given to Foreign Language and Area Studies (FLAS) recipients and, if so, the type of test. This resulted in responses such as “ACTFL,” “ACTF Scale,” “ACTFL OPI,” “Oral Proficiency Interview,” “ACTFL/ILR,” and “ACCTF? I don’t remember exactly,” all of which were probably indicating the same test. More useful data might have been obtained by a series of predefined options for the most common measures, even if it necessitated an additional, “other” category. If the “other” option were supplemented by a question related to the language, it would have provided even more useful information. • Use of “other” category for key questions. In several cases, EELIAS supple- ments a drop-down menu for key data with an “other” option; for example, languages offered and type of language instructor. In the case of languages, responses submitted in the other category included, among others, “Advanced Arabic,” “Intensive Elementary Portuguese,” and “Elementary Persian I,” even though Arabic, Portuguese, and Persian were included in the drop-down menu. • Limited data validation/review. The committee was told that reports are re- viewed by individual project officers and that the information submitted is used to make continuation awards. It was clear, however, that project officers have limited time to review the consistency or reliability of data submitted from indi- vidual grantees. ED has begun efforts with its contractors to review the consis- tency of data across grantees and to address identified data issues. • Inconsistency across programs. The data collected for individual programs vary. Although this might be expected to some extent given the variability in program missions, the differences do not always appear to be purposeful. For example, it is not clear why only the Technological Innovation and Cooperation for Foreign Information Access (TICFIA) Program is required to report on the use of technology despite its recognized importance. Similarly, not all programs are required to submit abstracts. • Contractor coding of some information. The service area descriptor, a po- tentially useful analytic tool added by the department after it assumed manage- ment of the system, has been coded by the contractor based on the project abstract. Given the limited information available in the abstract, this approach has significant limitations. Project staff will be required to enter this information in the redesigned system.

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Rating Tool (PART) performance measurements of the Office of Manage- ment and Budget (OMB) for international education domestic programs.2 The review of extant evaluations and monitoring data showed that there were insufficient current data available to provide meaningful in- sights on five of the eight original questions. Given the time constraints on the review, the committee decided to use commissioned targeted analyses, meetings with stakeholders and experts, public testimony, and site visits to selected universities to address the research questions. • Commissioned papers and targeted analyses. The committee re- viewed or commissioned targeted analyses on increasing representation of minorities,3 technology and instructional materials,4 oral proficiency as- sessment,5 and implementation issues.6 We conducted an analysis of Mod- ern Language Association enrollment data to explore the possible role of National Resource Centers (NRC) in the teaching of less commonly taught languages, and we examined various definitions of “internationalization” of higher education. The committee also obtained preliminary data from ED generated by a contracted study by InfoUse entitled “Study of Graduate Fellowship Programs Participants,” which includes a survey of all FLAS and Doctoral Dissertation Research Abroad (DDRA) Fellowships from 1997 to 1999. • Reviewed written comments and conducted public meetings to get input from experts, officials, and others and met with grantees in Wash- ington and ED officials with responsibilities for these programs. Written comments were submitted in response to a set of questions developed by the committee (see Attachment B-2), and public meetings included presen- 2 PART reviews program evaluation and performance measurement information to assign program ratings. The international education domestic programs were reviewed by OMB in 2004 and assigned a rating of “results not demonstrated.” 3 Joyner, C.C. (2006). Increasing representation of minorities in international service through international education programs. Paper commissioned by the National Research Council Committee to Review the Title VI and Fulbright Hays International Education Programs, Washington, DC. 4 Joyner, C., and Suarez, T. (2006). Technology and instructional materials in Title VI and Fulbright-Hays international education programs. Paper commissioned by the National Research Council Committee to Review the Title VI and Fulbright-Hays International Educa- tion Programs, Washington, DC. 5 Malone, M.E. (2006). The oral proficiency interview approach to foreign language assess- ment. Paper commissioned by the National Research Council Committee to Review the Title VI and Fulbright-Hays International Education Programs, Washington, DC. 6 Ruther, N.L. (2006). Implementation issues and options for the HEA Title VI and Fulbright Hays programs. Paper written for the National Research Council Committee to Review the Title VI and Fulbright-Hays International Education Programs, Washington, DC.

OCR for page 309
 APPENDIX B tations by grantees, federal agency staff, and outside experts (see Attach- ment B-3). We also met with NRC and Language Resource Centers (LRC) grantees when they were in Washington, DC, for an ED-sponsored grantee meeting and with ED officials with responsibilities for these programs. • Conducted site visits to eight universities. To become acquainted with Title VI center programs and gather illustrative examples, committee members and staff conducted site visits to eight universities (Georgetown University, George Washington University, Indiana University, New York University, Ohio State University, San Diego State University, the University of California, Los Angeles, and the University of Wisconsin-Madison) and reviewed selected grant applications from these universities. The universities were chosen to represent a variety of public and private institutions, Title VI funding levels, and geographic locations. Although the site visits do not represent a random sample, they were selected to provide a general picture of the diversity of NRC, Centers for International Business Education and Research (CIBER), and LRC grantees. At each location, we met with uni- versity administrators and staff, faculty, students, and librarians associated with the Title VI programs on their campuses. An interview guide based on the eight key areas specified by Congress guided the interviews at each location (see Attachment B-4). Several factors limited our review. First, in the case of the large center programs (NRC, LRC, CIBER), Title VI typically represents a funding stream rather than a discrete program; that is, Title VI funds combine with other funding sources in order to achieve the desired outcomes, so that it is difficult to attribute specific outcomes to Title VI (U.S. General Accounting Office, June, 1998). Second, we did not have time to conduct a survey of all grantees, so some of our observations and findings are not generalizable; the illustrations based on the site visits, for example, may not be applicable to all grantees. In addition, extant data provided significant information for only three of the eight congressional questions, and the major data system (EELIAS), which theoretically could have provided answers to many ques- tions, has significant problems. CONCEPTUAL MODEL Understanding how a program is expected to lead to particular results can be extremely useful in designing and implementing it and in planning its evaluation. A basic logic model provides a systematic and visual way to present and share an understanding of the relationships among the resources/inputs, activities, outputs, outcomes, and impact expected of

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES TABLE B-1 Categories of Performance Measures Category Definition Examples Impacts The degree to which broad Increased competitiveness, enhanced level of social objectives are achieved education, income of graduates, improved quality of educational institutions Outcomes Accomplishment of program Academic performance improvement, students objectives attributable to accepted at next level of education, graduates program outputs certified as teachers, employer satisfaction Outputs The direct result of program Number of students enrolled, targeted students activities completing training, students applying to next level of education Activities The work performed by the Amount of training given, counseling provided, grantee that directly produces conferences held, reports published the core products and services Inputs Resources consumed by the Generally limited to funds and grant years organization SOURCE: U.S. Department of Education. (1998). Demonstrating results: An introduction to the Government Performance and Results Act. Washington, DC: Author, Office of Postsec- ondary Education. the program.7 Logic models can range from an activities approach model, which describes activities in great detail and is especially useful as a man- agement tool, to a theory approach model, which emphasizes the theory of change that has influenced the design and plan for a program and is especially useful during program planning and design. A third kind of logic model is perhaps more relevant to the committee’s work: an outcome approach model, which shows the causal linkages thought to exist among program components and is especially useful in designing effective evalua- tion and reporting strategies. In considering the programs and the results expected from them, it will be useful to keep in mind the distinctions among several categories of measures established under the Government Performance and Results Act (GPRA) and used by ED in monitoring and reporting on its activities. Table B-1 shows one way of describing those categories.8 7 W.K. Kellogg Foundation (2003). Logic model development guide. Battle Creek, MI: Author. 8 Others might define categories in slightly different ways. In addition, assigning a measure to one or another category (for example, outcome or impact) is often a judgment call on which opinions might differ from one situation to another. Distinctions are sometimes made, as well, between short-term outcomes and longer-term outcomes.

OCR for page 309
 APPENDIX B In requesting this review, Congress asked the committee to address the performance of these programs in eight specific areas, which implies that these areas might be seen as program goals about which informa- tion is needed. Some of those eight areas reflect questions about activities, whereas others address outputs or outcomes. Table B-2 shows these eight areas with comments about the category of performance measure specified or implied. Figures B-1 through B-5 represent a modified “outcome approach” logic model. That is, individual programs are grouped in ways that show how their activities in combination are expected to lead to certain short- term and longer term outcomes. While grouping programs provides a better appreciation for the ways in which they interact, it necessitates some loss of detail in describing the activities and their associated output measures. Figure B-1, which presents an overview of all 14 programs, has the least amount of detail. The schematic in Figure B-1 shows all 14 programs along with their links to the goals identified in the congressional request, other short-term and longer term outcomes that can be inferred from that request, and the legislation and regulations addressing allowable activities in the programs. For the purpose of this report, the goal of “supporting research, educa- tion, and training” is represented at the level of three inferred subgoals: (1) increased research capacity, (2) enhanced body of knowledge about foreign languages and area studies, and (3) increased institutional capac- ity of education and training in K-12 and institutions of higher education (IHE). Reading from left to right, the relationship between short-term and longer term outcomes can be seen. For example, the schematic posits that improving instructional materials is important because of its contribution to improving the institutional capacity of education and training, which in turn contributes to a decrease in the shortage of experts as well as an increase in the global understanding of academics and U.S. educators. The figure also shows that nearly all of the programs are related to multiple outcomes. In Figures B-2 through B-5, programs are grouped by similarity in the way they function or in their purpose. By reducing the number of programs in a single schematic, it is possible to include more detail about the activities that are expected to lead to the outcomes. Figure B-2 shows the six programs that provide funds to individuals: the four Fulbright-Hays programs (DDRA, Faculty Research Abroad or FRA, Group Projects Abroad or GPA, and Seminars Abroad or SA) as well as FLAS and IIPP. Some of these programs could be expected to contribute to a reduction in the shortage of experts, including those who are minori- ties, while others should lead to an enhanced body of knowledge about foreign languages and area studies or improved global understanding. As

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES TABLE B-2 Program Areas to Be Addressed in Responding to Congress Area Comment As stated, output measure would assess the extent to which 1. Supporting research, education and training research, education, and training occurred. One could plausibility link the question to short-term and longer term in foreign language and outcomes of (a) improved institutional research capacity, (b) international studies, including opportunities for improved institutional capacity to provide quality education such research, education, and training in K-12 and institutions of higher education, and training overseas and (c) improved body of knowledge about foreign language and area studies. Outcome measure would be decreased shortage of foreign 2. Reducing shortages of foreign language and area language and area studies experts in public service and experts academia. Short-term outcome measure would be increased infusion 3. Infusing a foreign language and area studies dimension of this dimension into elementary, secondary, and higher throughout the education education, including professional education. One could system and across relevant make an expected causal link between this outcome and a longer term outcome of improved institutional capacity in disciplines, including professional education K-12 and IHE. Short-term outcome measure would be improved 4. Producing relevant instructional materials that instructional materials. That is, one would go beyond the output measure of quantity of materials produced to assess meet accepted scholarly standards that they are “improved” over what previously existed by virtue of meeting accepted scholarly standards. This measure, in turn, would lead to the longer term outcome of improved institutional capacity for education and training. Output measure would describe the ways in which projects 5. Advancing uses of new use technology. Short-term outcome measure would be technology in foreign language and international improved technology to enhance foreign language and studies international studies and its use. This, in turn, would contribute to the longer term outcome of an improved institutional capacity for research, education, and training. A short-term outcome measure would be enhanced 6. Addressing business needs for international materials and activities to meet business needs. knowledge and foreign language skills Outcome measure would be increased representation of 7. Increasing the numbers of underrepresented minorities in international service. Lacking that information, output measure would describe program activities and minorities in international service participants. As stated, output measure would assess the extent to which 8. Conducting public outreach/dissemination these outreach and dissemination activities were conducted. One could plausibly link the question to the short-term to K-12 and higher outcome measure of improved outreach and dissemination education, media, government, business, and of information about other cultures. the public

OCR for page 309
CIBE ↑ Materials & activities to ↑ Global meet business BIE understanding: needs media & general public AORC ↑ Research capacity ↑ Global understanding: NRC government LRC ↑ Global understanding: business ↑ Outreach & dissemination IRS ↑ Global TICFIA ↑ Technology understanding: academia UISFL ↑ Infusion of foreign ↑ Institutional languages & area capacity/quality studies in elem., sec., of education ↑ Global & higher ed, including and training: DDRA understanding: professional education K-12 and IHE U.S. educators ↑ Instructional FRA materials ↓ Shortage of foreign languages GPA & area studies experts in public ↑ Opportunities service & for individuals academia and groups to SA study & do ↑ Educational ↑ Minorities in research activities international targeted to service FLAS IIPP minorities FIGURE B-1 Schematic of all programs, showing short-term and longer term outcomes. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional B-1  request for the study. Landscape view Beneath image is a half inch left for caption, which can be seven inches long

OCR for page 309
 Activities/Outputs Short-Term Outcomes Longer Term Outcomes Graduate DDRA Quality students do research is dissertation research produced on foreign abroad for 6 Enhanced body languages & to 12 months of knowledge area studies about foreign topics Faculty do languages & FRA research area studies abroad Participants’ language competency Teachers, students, &/or Decreased & area faculty go to short-term shortage of studies GPA study or intensive foreign knowledge Program alumni overseas programs languages & improve are employed area studies as experts in experts in public service public service Graduate students use &/or academia & academia academic year & summer programs to study foreign FLAS language &/or area studies Increased representation Increased infusion of of minorities in foreign languages & international area studies in service Minorities are targeted as elementary & undergraduates and secondary education IIPP followed through graduate school with special activities ↑ Institutional capacity/quality of education and training: Alumni gain an improved K-12 and IHE understanding & Improved global K-12 educators do short- Improved outreach & understanding: knowledge of peoples & term study & travel seminars SA U.S. educators dissemination of cultures of other abroad information about countries, which they other cultures share with colleagues upon return FIGURE B-2 Programs that provide funds to individuals. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional request for the study. B-2 Landscape view We have unified some of the boxes without having to recreate the entire figure

OCR for page 309
 APPENDIX B interim outcomes, one would expect to see improvement in individuals’ language competency, production of quality research, and an improved understanding of peoples and cultures of other countries on the part of nonspecialists. In Figure B-3, the two programs specifically intended to support inter- national business education (CIBER and Business and International Educa- tion or BIE) are shown. Meeting business needs is intended to lead to an improved global understanding in the business community, which in turn is expected to enhance U.S. leadership in the global economy. The three programs that function through institutional centers (Ameri- can Overseas Research Centers, AORC, NRCs, and LRCs) are shown in Figure B-4. In a variety of ways, these program activities are expected to improve institutional capacities for research and for education and train- ing. In turn, these institutional changes should lead to an improved body of knowledge about foreign languages and area studies and to both an increase in experts and to enhanced general global understanding. Their activities are expected to focus especially on the less commonly taught languages (LCTLs). The remaining three programs (IRS, Undergraduate International Studies and Foreign Language or UISFL, and Technological Innovation and Cooperation for Foreign Information Access or TICFIA) address the specialist and generalist outcomes through the intermediary outcome of improving the institutional capacity for education and training on foreign languages and area studies in K-12 and IHE. As Figure B-5 shows, their activities increase the infusion of these areas into higher education, improve instructional materials, improve outreach and dissemination, and improve technology, which in turn improves institutional capacities for research and education and training. Not shown in the individual schematics is the overall impact expected of the set of programs and some of the theoretical assumptions that underlie their creation. For example, in its FY 2007 Performance Plan, ED describes the program goal being addressed by all these programs as “to meet the nation’s security and economic needs through the development of a national capacity in foreign languages, and area and international studies.” Given this goal, an impact assessment would attempt to determine whether the nation is more secure and economically sound as a result of the programs. An assessment of this sort is beyond the scope of the committee’s charge. Indeed, although we endeavored to assess outcomes, the available data of- ten restricted our analysis to assessment of inputs, activities, and outputs. In the report, the committee makes recommendations aimed at moving future program monitoring and evaluation efforts toward assessment of outcomes.

OCR for page 309
 APPENDIX B • Linda Lim, Professor of Strategy, Stephen M. Ross School of Business, Director, Center for Southeast Asian Studies • Patrick O’Meara, Professor and Dean, International Programs, Indiana University • Steven M. Poulos, Director, South Asia Language Resource Center, The University of Chicago • Karla Ver Bryck Block, Senior Program Specialist, IEPS, Department of Education 5:00 p.m. Adjourn Fourth Committee Meeting October 5-6, 2006 OPEN SESSION AGENDA The Beckman Center of the National Academies The Board Room Irvine, CA (and via videoteleconference at The National Academy of Sciences 2100 C Street, NW, Room NAS 250, Washington, DC) Thursday, October 5 8:00 a.m. PST Breakfast (for those in Irvine) (11:00 a.m. EST) 8:30 a.m. PST Welcome (11:30 a.m. EST) Janet Norwood, Chair, Committee to Review the Title VI and Fulbright-Hays International Education Programs 8:45 a.m. PST Panel Discussion of Title VI in the 21st Century (11:45 a.m. EST) Al Fishlow, Columbia University (via videoteleconference from Columbia University)

OCR for page 309
0 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Martin Kramer, The Washington Institute for Near East Policy Robert Blake, University of California Language Consortium 10:15 a.m. PST Break (1:15 p.m. EST) 10:30 a.m. PST Recent Developments in IEPS Data Collection and (1:30 p.m. EST) Reporting Karla VerBryck Block, International Education Programs Service 11:15 a.m. PST (2:15 p.m. EST) The Honorable David S.C. Chu, Undersecretary of Defense for Personnel and Readiness (via videoteleconference from the Department of Defense) 12:00 p.m. PST Break (for lunch in Irvine) (3:00 p.m. EST) 1:00 p.m. PST End of Open Session (4:00 p.m. EST) NOTE FOR PUBLIC MEETINGS: This meeting is being held to gather information to help the committee conduct its study. This committee will examine the information and material obtained during this, and other public meetings, in an effort to inform its work. Although opinions may be stated and lively discussion may ensue, no conclusions are being drawn at this time; no recommendations will be made. In fact, the committee will deliberate thoroughly before writing its draft report. Moreover, once the draft report is written, it must go through a rigorous review by experts who are anonymous to the committee, and the committee then must respond to this review with appropriate revisions that adequately satisfy the Academy’s Report Review Committee and the chair of the National Research Council before it is consid- ered a National Research Council report. Therefore, observers who draw conclusions about the committee’s work based on today’s discussions will be doing so prematurely. Furthermore, individual committee members often engage in discussion and questioning for the specific purpose of probing an issue and sharpening an argument. The comments of any given committee member may not necessarily reflect the position he or she may actually hold on the subject under discussion, to say nothing of that person’s future position as it may evolve in the course of the project. Any inferences about an individual’s position regarding findings or recommendations in the final report are therefore also premature.

OCR for page 309
 APPENDIX B ATTACHMENT B-4 Site Visit Interview Guide 8-03-06 Topics to be addressed in every interview: • The 8 areas of review, including how they evaluate their own prog- ress in these areas. • Unique strengths/challenges of their programs. • Impact of Title VI/FH funding (except student interviews). • How can Title VI/FH be improved (except student interviews)? Key to eight areas of review: Q1. Supporting research, education, and training in foreign languages and international studies, including opportunities for such research, education, and training overseas; Q2. Reducing shortages of foreign languages and area experts; Q3. Infusing a foreign language and area studies dimension throughout the educational system and across relevant disciplines including professional education; Q4. Producing relevant instructional materials that meet accepted scholarly standards; Q5. Advancing uses of new technology in foreign language and international studies; Q6. Addressing business needs for international knowledge and foreign language skills; Q7. Increasing the numbers of underrepresented minorities in international service; and Q8. Conducting public outreach/dissemination to K-12 and higher education, media, government, business, and the general public. PROVOST/SENIOR INTERNATIONAL OFFICER Questions related to the eight study areas: 1. Could you give us an overview of your international programs across the university, including among faculty, undergraduates, graduate students, and the professional schools? (Q1, Q3)

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES 2. How do you monitor and evaluate the effectiveness of these pro- grams? What sources of information do you use to assess outcomes and/or impacts? Question about the unique aspects of their program: 3. What do you see as your university’s greatest strength(s) in the field of international education? Its greatest challenge(s)? Questions about the impact of Title VI/FH funding: 4. Please tell me about the impact of Federal Title VI/FH funding on the development and evolution of your university’s international programs. Does the availability of Title VI/FH funding allow you to support programs or activities that you might not be able to oth- erwise? Can you provide some specific activities? (Probes): Languages that would not otherwise be offered? Interdisciplinary work that might not otherwise be car- ried out? Outreach or support to K-12 education? 5. What would happen at your university if Title VI funds were in- creased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 6. Are there ways in which Title VI/FH programs might be changed so that they were more effective? (Probes): More flexibility in the use of funds. Different allocation of funds among the program’s several components and objectives. More funding for professional schools. NATIONAL RESOURCE CENTERS (NRC) DIRECTORS AND DEPUTIES Questions related to the eight study areas: 1. Give us an overview of your center and how it fits in the broader university context. What has been the role of Title VI/FH in the development and evolution of your center? (Q1, Q3) 2. How do you publicize your center and its offerings? How do you recruit students?

OCR for page 309
 APPENDIX B (Probes): Do you think the center is widely known on campus? Are your language classes fully subscribed, or could you accept more students into your classes or into the program as a whole? 3. How and how well are the center’s instructional needs in language being met? (Q1) (Probe): Through language and literature departments and/or other programs? Through collaboration with the LRC? 4. What are the language requirements for the certificate and degree programs, for entering students and for completing students? What standard or standards do you use to measure language proficiency? (Q1) 5. What career paths do students who receive certificates and/or de- grees follow? (Q2) 6. Tell us about your outreach activities. Who is your primary audi- ence? Can you tell us about some of your most effective outreach activities and why you consider them effective? (Q6, Q8) Question about the unique aspects of the program: 7. What do you see as your center’s greatest strength(s)? Your greatest challenge(s)? Questions about the impact of Title VI/FH funding: 8a. (For Centers with continuous funding) Please tell me about the impact of federal Title VI/FH funding on your center’s programs. What activities are supported? (Probes): Is it used to support core instruction? Is it used to support specialized activities like teaching of LCTLs? Does it allow you to support programs that you might otherwise not be able to offer, such as particular lan- guages, interdisciplinary research, or support to the K-12 system?

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES 8b. (For centers that lost funding and then regained it)1 Please tell me about how your center’s programs were affected by the temporary loss of federal funding. (Probes): What activities are you conducting or planning now that you were not able to carry out when you lost fed- eral funding? Can you provide examples? What sources of funding did you turn to when federal funds were gone? What types of activities were you able to maintain? What types of activities were you forced to eliminate? 9. (For centers with continuous funding—do not use for centers that temporarily lost funding) What would happen at your center if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 10. Are there ways in which Title VI/FH programs might be changed so that they were more effective? (Probes): More flexibility in the use of funds. Different allocation of funds among the various pro- grams and objectives. More involvement of professional schools. LANGUAGE RESOURCE CENTERS (LRC) DIRECTORS AND DEPUTIES Questions related to the eight study areas: 1. In what specific ways is your center helping to support research, education, and training in foreign languages across the university (including in the professional schools)? (Q1,Q3) 2. Tell us about the instructional materials you produce. Who is the primary audience for these materials, and how do you ensure that they are of high quality and relevant to the audience? (Q4) (Probe): Are the materials reviewed by other scholars who have 1 San Diego State University: none; George Washington University: none; Georgetown Uni- versity: none; Indiana University: East Asia and West Europe (lost funding 2000-2006); New York University: Latin America (lost funding 2003-2005) and Center for European Studies (lost funding 2000-2002); Ohio State University: East Asia (lost funding 2003-2005); Univer- sity of California, Los Angeles: Latin American Center (lost funding 2000-2003) and Europe/ Russian Center (lost funding 2003-2005).

OCR for page 309
 APPENDIX B language and area expertise? If so, what is the process by which the materials are reviewed? 3. How do you evaluate the use and impact of these instructional materials? (Probe): What types of data do you maintain on the use of your materials? Among various audiences, such as K-12 education, higher education, professional schools, and business? (Q4, Q6, Q8) 4. Do you coordinate or collaborate with other LRCs or NRCs on campus? With other international-focused programs on campus? Do you collaborate with Title VI-funded programs at other universities? Please give examples. Question about the unique aspects of the program: 5. What do you see as your center’s greatest strength(s)? Your greatest challenge(s)? Questions about the impact of Title VI/FH funding: 6. Please tell me about the impact of federal Title VI/FH funding on your LRC. What activities are supported that might not otherwise? (Probes): Is it used to support core language instruction? Is it used to support specialized activities related to LCTLs? Does it allow you to support programs that you might otherwise not be able to offer, such as particular lan- guages, interdisciplinary research, or support to the K-12 system? 7. What would happen to your center if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 8. Are there ways in which Title VI/FH programs might be changed so that they were more effective? (Probes): More flexibility in the use of funds. Different allocation of funds among the various pro- grams and objectives.

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES CIBER DIRECTORS Questions related to the eight study areas: 1. Can you provide some examples of the ways in which your pro- grams address business needs for international knowledge and for- eign language skills? (Q6) 2. Tell us about your outreach activities. Do you target outreach pri- marily to business or to several different audiences? Which of these efforts do you view as most effective and why? (Q3, Q6, Q8) (Probe): In addition to business, do you reach out to other audi- ences, such as K-12 education, other parts of your univer- sity, the general public, the media? 3. Do you coordinate or collaborate with other CIBERs at other uni- versities? With other Title VI-funded programs on campus? Other international-focused programs on campus? Please give examples. Questions about unique aspects of the program: 4. What do you see as your center’s greatest strength(s)? Your greatest challenge? Questions about the impact of Title VI/FH funding: 5. Please tell me about the impact of federal Title VI/FH funding on your CIBER. What activities are supported that might not otherwise? 6. What would happen to your center if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 7. Are there ways in which Title VI/FH programs might be changed to be more effective? (Probes): More flexibility in the use of funds? Different allocation of funds among the various pro- grams and objectives? FACULTY NOTE: Try to meet with faculty alone, without the presence of the NRC director, LRC director, or other administrator.

OCR for page 309
 APPENDIX B Questions related to the eight study areas: General orientation Ask faculty to identify their discipline, home department, particular field of expertise, and center they are associated with. 1. Would you describe the intellectual benefits you gain from partici- pating in the center? (Probe): Do you gain insights from other disciplines? Does the center provide opportunities for interdisciplin- ary collaboration in research, teaching, and/or service? Do you learn about new scholarship? 2. How do you publicize your center and its offerings? How do you recruit students? (Probes): Do you think the center is widely known on campus? Are your language classes fully subscribed, or could you accept more students into your classes or into the program as a whole? 3. (For language instructors) How many levels of instruction are of- fered in the language(s) you teach? (Q1) (Probes): What levels of proficiency are required for students in certificate programs, M.A., Ph.D. programs? How is proficiency measured? 4. Are you or have you been an undergraduate or graduate student advisor? What are the motivations and goals of students pursuing certificate and degree programs in your field? In international/area studies in general? (Q2) (Probe): What career fields do graduates enter (government, busi- ness, and academia)? Question about unique aspects of the program: 5. What do you see as your university’s greatest strength(s) in the area of international education and foreign language instruction? Its greatest challenge? Questions about the impact of Title VI/FH funding: 6. Please tell me about the impact of federal Title VI/FH funding on your teaching, research, and other activities. For what types of ac-

OCR for page 309
 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES tivities have you and/or your students received direct support from Title VI and/or Fulbright-Hays? (Probe): Can you provide any examples of ways in which Title VI/FH funds helped to leverage additional university re- sources for your research and/or teaching? 7. What would happen to your research and teaching if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 8. Are there ways in which Title VI/FH programs could be changed to be more effective? (Probes): More flexibility in the use of funds? Different allocation of funds among the program’s sev- eral components and objectives? More funding for professional schools? STUDENTS General orientation Ask students to identify themselves based on their field of study, whether they are studying a language, and if so, what language, the NRC with which they are associated, and whether they have benefited from direct support from Title VI (i.e., FLAS or DDRA) Questions related to the eight study areas: 1. What motivated you to enroll in language/area studies? (Probes): What did you expect to get from the program (or courses) when you enrolled in it/them? Did you have any ideas about a career in/related to this program? (Q2) 2. Where and how did you first hear about the area or language pro- gram you are enrolled in? (Probes): Do you think other students are aware of these pro- grams? Do you feel your center is isolated from or connected to other departments, divisions, and schools? (Q3) 3. In what subjects or disciplines has your learning experience been the strongest? (Q1)

OCR for page 309
 APPENDIX B (Probe): Have you found that language instruction is integrated within courses in such fields as economics, political sci- ence, religion and culture? 4. (For those who have gone abroad) Was the preparation appropri- ate and adequate for your to take advantage of the cultural and linguistic immersion? What aspects of the learning facilities and/or living conditions enhanced or detracted from your study or research abroad? (Q1) 5. In what ways (if any) do you expect to use your training in area studies, international studies, or foreign languages after graduation or after completing your graduate degree? (Q2) (Probe): Are you considering a job in government, business, aca- demia or a nonprofit? 6. (For graduate students): What aspects of the facilities, that is the library and the technological resources, support or detract from your research? (Q5) 7. In what ways does the faculty integrate technology into instruction in foreign language and area studies? (Q5) 8. Have your studies met or failed to meet your expectations? (Probe): In what ways? Which aspects of your studies? Question about the unique aspects of their program: 9. What do you see as your university’s greatest strength(s) in the field of international education? Its greatest challenge? INTERNATIONAL STUDIES LIBRARIANS From your perspective: 1. In what ways have Title VI funds enabled you to provide greater international studies and language resources to the university community? 2. Are these resources accessed by people outside the university? Who? 3. How do you measure the people who have accessed these resources?