Evaluating Licensing Tests

Solid technical characteristics and fairness are key to the effective use of tests. The work of measurement specialists, test users, and policy makers suggests criteria for judging the appropriateness and technical quality of initial teacher licensure tests. The committee drew on these to develop criteria it believes users should aspire to in developing and evaluating initial teacher licensure tests. The committee used these evaluation criteria to evaluate a sample of five widely used tests produced by ETS. The tests the committee reviewed met most of its criteria for technical quality, although there were some areas for improvement. The committee also attempted to review a sample of NES tests. Despite concerted and repeated efforts, though, the committee was unable to obtain sufficient information on the technical characteristics of tests produced by NES and thus could draw no conclusions about their technical quality.

Conclusions
  • The committee’s criteria for judging test quality include the following: tests should have a statement of purpose; systematic processes should be used in deciding what to test and in assuring balanced and adequate coverage of these competencies; test materials should be tried out and analyzed before operational decisions are made; test administration and scoring should be uniform and fair; test materials and results should be protected from corruptibility; standard-setting procedures should be systematic and well documented; test results should be consistent across test forms and scorers; information about tests and scoring should be available to candidates; technical documentation should be accessible for public and professional review; validity evidence should be gathered and presented; costs and feasibility should be considered in test development and selection; and the long-term consequences of licensing tests should be monitored and examined.

  • The profession’s standards for educational testing say that information sufficient to evaluate the appropriateness and technical adequacy of tests should be made available to potential test users and other interested parties. The committee considers the lack of sufficient technical information made available by NES and the states to evaluate NES-developed tests to be problematic and a concern. It is also significant because NES-developed tests are administered to very large numbers of teacher candidates.

  • The initial licensure tests currently in use rely almost exclusively on content-related evidence of validity. Few, if any, developers are collecting evidence about how test results relate to other relevant measures of candidates’ knowledge, skills, and abilities.

  • It is important to collect validity data that go beyond content-related



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement