6
Next Steps
This chapter highlights next steps toward realizing the potential of intra- and interpersonal competencies to support students’ persistence and success in higher education. In so doing, it summarizes the committee’s conclusions and highlights its recommendations.
COMPETENCIES FOR COLLEGE SUCCESS
Based on its review of the evidence, the committee concluded that only limited research has been conducted to date on the potential relationships between various intra- and interpersonal competencies and students’ college success.
RECOMMENDATION 1: Federal agencies and foundations should invest in research exploring the possible relationships between various intra- and interpersonal competencies and students’ college success. To address gaps in the research base, these investments should include support for research examining
- how interpersonal competencies may be related to student success in 4-year colleges;
- how intra- and interpersonal competencies may be related to student success in community colleges; and
- how intra- and interpersonal competencies may be related to students’ success in 2- and 4-year science, technology, engineering, and mathematics programs and majors.
Based on the limited intervention studies conducted to date, the committee found promising evidence that the three competencies of a sense of belonging, growth mindset, and utility values are related to college success and malleable in response to interventions. The committee found more modest evidence that five other competencies are similarly related to college success and malleable, yielding the following list:
- behaviors related to conscientiousness,
- sense of belonging,
- academic self-efficacy,
- growth mindset,
- utility goals and values,
- intrinsic goals and interest,
- prosocial goals and values, and
- positive future self.
Although the evidence for sense of belonging, growth mindset, and utility values is encouraging, further research is needed to replicate and extend this evidence and to better understand the potential impact of the other five competencies. The following recommendations are based on the committee’s belief that colleges and universities could play an important role in facilitating this research.
RECOMMENDATION 2: Federal agencies and foundations should invest in intervention research using random assignment and research employing a range of other methods to understand better the competencies identified in this report, their relationship to college success, and the mechanisms through which they operate to improve college success. Research focused on supporting the college success of underrepresented student groups should be a priority.
RECOMMENDATION 3: Colleges and universities should support the intervention research proposed in Recommendation 2 by facilitating the implementation and evaluation of random-assignment interventions, thereby gaining valuable information about their students and building the knowledge base on effective interventions needed to increase student retention and success.
Issues of race, ethnicity, gender, social class, and culture need to be primary considerations when educators, administrators, researchers, and policy makers think about competencies and their contribution to college success. Based on the literature, the committee concluded that certain com-
petencies develop and function differently for different groups and within different cultural and institutional contexts.
RECOMMENDATION 4: To help reduce disparities in college success among student groups, institutions of higher education should evaluate and improve their social and learning environments to support the development of the eight identified competencies, especially among underrepresented student groups.
ASSESSMENT METHODS FOR COLLEGE COMPETENCIES
The committee reviewed the nature and quality of existing assessments of the eight competencies identified above, together with research and professional standards related to the general process of the development, validation, implementation, interpretation, and evaluation of the results of intra- and interpersonal competency assessments. Noting that various professional and legal standards apply to assessing these competencies, the committee concluded that most current assessments of the eight identified competencies are uneven in quality, providing only limited evidence to date that they meet professional standards of reliability, validity, and fairness. Considering potential high-stakes uses of these current assessments or of other assessments of intra- and interpersonal competencies, the committee concluded that the development and validation of such assessments for high-stakes purposes is a rigorous, time-consuming, and expensive process that depends critically on assessment and psychometric expertise. Validity, reliability, and fairness are necessary considerations in evaluating assessment quality.
RECOMMENDATION 5: When developing and validating intra- and interpersonal competency assessments to be used for high-stakes purposes, stakeholders in higher education (e.g., faculty, administrators, student services offices) should comply with professional standards, legal guidelines, and best practices to enable appropriate interpretations of the assessment results for particular uses.
RECOMMENDATION 6: Institutions of higher education should not make high-stakes decisions based solely on current assessments of the eight identified competencies, given the relatively limited research to date demonstrating their validity for predicting college success.
The committee determined that, when used for low-stakes purposes, assessments need not meet the high evidentiary requirements associated with high-stakes individual student assessments. Even when assessments are
not used for high-stakes purposes, however, they need to be sensitive to the competencies they are intended to measure. The committee concluded that even low-stakes uses of intra- and interpersonal competency assessments require attention to validity, reliability, and fairness.
RECOMMENDATION 7: Those who develop, select, or use intra- and interpersonal competency assessments should pay heed to, and collect evidence of, validity, reliability, and fairness as appropriate for the intended high-stakes or low-stakes uses.
This report notes several limitations of self-report measures. Regardless of whether an assessment is high- or low-stakes, for example, respondents may be motivated to present themselves in a favorable light. In recent years, various methods have been shown to mitigate these limitations. Ratings by others, for instance, avoid some of the problems of self-ratings and have been shown to yield more reliable and predictive data in many contexts. Based on its analysis, the committee concluded that most existing assessments of the eight identified competencies, as well as many assessments of other intra- and interpersonal competencies, use self-report measures, whose well-documented limitations may constrain or preclude certain uses of the assessment results, and that many innovative approaches for assessing competencies address these limitations.
RECOMMENDATION 8: Federal agencies and foundations should support additional research, development, and validation of new intra- and interpersonal competency assessments that address the shortcomings of existing measures.
Professional standards make clear that fairness to all individuals for whom an assessment is intended should be a driving concern throughout assessment development, validation, and use. Based on its analysis of research and current practice, the committee concluded that, despite the ever-increasing diversity of undergraduate student populations, attention to fairness for diverse populations often is inadequate in the development, validation, and use of current assessments of the eight identified competencies. Because these fairness standards apply broadly to assessments of all types of competencies, the committee makes the following recommendation.
RECOMMENDATION 9: Researchers and practitioners in higher education should consider evidence on fairness during the development, selection, and validation of intra- and interpersonal competency assessments.
Local norms and contexts may influence self-, peer, or other ratings of an intra- or interpersonal competency and may also mediate or moderate the relationships between these competencies and educational outcomes. The committee concluded that appropriate interpretation of the results of intra- and interpersonal competency assessments requires consideration of such contextual factors as student background, college climate, and department or discipline.
RECOMMENDATION 10: Higher education researchers and assessment experts should incorporate data on context (e.g., culture, climate, discipline) into their analyses and interpretations of the results of intra- and interpersonal competency assessments.
ASSESSMENT USES AND STAKEHOLDERS
A variety of stakeholders, including families, K-12 schools, faculty members, college administrators, accreditors, and state and federal policy makers, use assessment in higher education for four major purposes:
- selection and placement of individual students;
- formative improvement of local educational processes, practices, and programs;
- research and evaluation supporting knowledge generation; and
- accountability.
Based on its analysis of the relevant literature, the committee concluded that in higher education, assessments are more likely to be valuable for supporting student success when their selection, design, analysis, and interpretation are guided by stakeholder information needs, intended uses, and users.
The committee observed that different higher education stakeholders have different needs for assessments of intra- and interpersonal competencies, depending on the immediacy of those needs, the purposes to be served by the data, and stakeholders’ assessment-related knowledge and skills. Having concluded that the selection of an assessment and appropriate use and interpretation of its results vary depending on the intended uses and users, the committee makes the following recommendation.
RECOMMENDATION 11: Leaders in higher education should select, design, analyze, and interpret data from assessments of intra- and interpersonal competencies based on stakeholder information needs, intended uses, and users.
Based on its review of the research on uses of assessment in higher education, the committee concluded further that assessments of intra- and interpersonal competencies are more likely to be implemented and used by stakeholders to improve student success when they are motivated by internal institutional improvement purposes than when they are motivated by accountability purposes.
After reviewing research on the assessment process, the committee concluded that assessments of intra- and interpersonal competencies are more likely to contribute to improved college success among students if efforts to use the results involve stakeholders at multiple levels of the organization, as opposed to individual stakeholders acting alone. In addition, some higher education stakeholders will require additional support and training to develop the knowledge and skills needed to select, use, and interpret assessments of intra- and interpersonal competencies to improve student success in higher education. Considering the available research, moreover, the committee concluded that, relative to the limited research on the relationship between intra- and interpersonal competencies and college success, less evidence is available from an organizational science perspective on how stakeholders in higher education can use data on intra- and interpersonal competencies for improvement and evaluation purposes.
RECOMMENDATION 12: To broaden understanding of how assessments of intra- and interpersonal competencies can lead to greater student retention and success, institutions of higher education should study and report on their use of these assessments for improvement purposes (e.g., enhancing student support services, developing underrepresented students’ sense of belonging, improving courses, identifying effective programs).
INTRA- AND INTERPERSONAL COMPETENCIES AS COLLEGE OUTCOMES
The intra- and interpersonal competencies of ethics, lifelong learning/career orientation, intercultural/diversity competence, civic engagement/citizenship, communication, and teamwork have been identified as valued outcomes of college education. The committee concluded that to date, only limited research has been conducted on these competencies, so that little is known about whether and under what conditions these competencies are related to students’ persistence and success in college.
RECOMMENDATION 13: Federal agencies and foundations should invest in research examining whether, and under what conditions, the intra- and interpersonal competencies identified as outcomes for col-
lege graduates may also be related to students’ persistence and success in college.
CONCLUSION
In conclusion, the committee believes that implementation of its recommendations would provide the research base and practical guidance needed to identify more clearly, develop, and assess those intra- and interpersonal competencies that support student persistence and success in undergraduate education.
This page intentionally left blank.