Public Transportation Board Effectiveness: A Self-Assessment Handbook (2004)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

16 SECTION 5 ASSESSMENT INSTRUCTIONS FOR THE ADMINISTRATOR(S) This section contains instructions for the transit board administrator to use in directing the self-assessment process. This set of instructions is designed to serve as a source of briefing material for the self-assessment administrator. The instructions will cover the following topics: (1) the level of assessment to use, (2) categories of assessment, (3) particu- lar assessment subcategories, (4) distribution of assessment tool, (5) responses, (6) compilation of responses, (7) presen- tation of board results, and (8) board actions based on results. Level of assessment to use. The Handbook contains three successive and cumulative levels of assessment that differ with regard to the number of responses required from direc- tors. The three assessment levels are as follows: 1. A Level I assessment uses 13 items to address the most important measures. 2. A Level II assessment includes the next 9 most impor- tant measures, for a total of 22 items. This assessment acquires more detail regarding board dynamics, poli- cies, and procedures. In addition, a goal-setting process is suggested for boards as a means for them to approach future self-assessment based on goals that are adopted. 3. A Level III assessment includes an additional nine per- formance measures that address advanced or more in- depth interactions among board members. This level of assessment typically entails extensive board member participation, discussion, and follow-up, using a total of 31 items. An additional optional behavioral assessment tool is followed by the same goal-setting process used in Level II. The major difference in the assessment levels is in the num- ber of measures used to describe board performance and the resulting time required for board member responses. Boards using the self-assessment instrument for the first time might choose Level I or (if there is an initial interest in more detail) Level II. Level I is regarded as a “bare bones,” minimum level of detail for assessment of board performance effec- tiveness. Level II adds considerably more coverage to the assessment; it is a good choice for first-time users who are interested in more than the minimum (Level I) but who do not desire the amount of detail posed by Level III. Categories of assessment. All three levels of assessment use the same six categories to assess board performance, as follows: 1. Board processes, 2. Strategic planning, 3. Fiduciary and legal responsibilities, 4. Diversity programming and implementation, 5. CEO relations, and 6. Public advocacy. Particular assessment subcategories. The three levels of assessment differ with regard to the detail used to assess board performance for each category. The assessment levels and the coverage of each category of board performance are as follows: Level I has 13 responses that are also included in Level II, along with 9 additional responses; all 22 items are included in Level III, which has 9 additional responses. Over half of the assessment items in Level I pertain to board processes and fiduciary and legal responsibilities, compared with nearly two-thirds of the assessment items for Levels II and III. The other major difference between the levels is the amount of detail devoted to strategic planning in Levels II and III. Level II and Level III also contain a goal-setting process that is highly recommended for boards that are interested in performance evaluation. The adopted goals will act as the per- formance base. The research team recommends that boards set goals that are achievable within 1 year. Goals should not be too easy or too hard to achieve. Appendix A provides a sample of goals that were adopted by the Chittenden County Transporta- tion Authority Board. The level of success for the reassess- ment depends on the number of goals that are achieved. In addition to the goal-setting process, Level III also con- tains an optional organizational behavior assessment tool that helps boards evaluate their group effectiveness. Distribution of assessment tool. The administrator is respon- sible for distributing the assessment tool to each board mem- ber and for collecting and compiling results, as directed by each individual board. The administrator can begin with the purpose and process stated in the Handbook and add addi-

17 tional material if it is appropriate for the specific case. The enclosed tool is not a “one size fits all” tool. Before copying and distributing the forms, the administrator should commu- nicate to the board that it can change the tool in any way that would fit its unique organization. Responses. Directors will respond to each statement about board performance using a 5-point scale that ranges from 1 (“strongly disagree”) to 5 (“strongly agree”). Compilation of responses. The compilation of responses can be qualitative and descriptive, or it can be quantitative and statistical. Informal compilation of responses would reflect a relatively nonquantitative approach, in which direc- tors’ evaluations would be reported for each assessment statement in a summary table. In formal compilation, the responses would be carefully analyzed in terms of degrees. For example, for Question 1, if two out of five respondents chose 1 (on a scale of 1 to 5) and three respondents chose 4, these results might be averaged (1 + 1 + 4 + 4 + 4 =14; 14 ÷ 5 = 2.8). Statistics can be compiled for each statement. These include (1) average score values (sum of all scores divided by the number of responses) for individual statements, for all state- ments for particular board performance categories, or for all performance categories and (2) the ranges between high and low score responses. One of the advantages of typically small transit boards (those with fewer than 10 persons) is that the responses can be compiled relatively easily, without using computers or calculators. However, software can be an effective tool for more formal computations, as well as for building graphical displays of responses. This is particularly useful when there are significant differences between directors or when there are correlations between responses to different assessments that can be shown graphically or statistically. Presentation of board results. The administrator should get directions from the board with regard to how the results should be presented. The board may choose to have the administrator summarize the results orally, or they may prefer a detailed for- mal summary report to be distributed among board members, within the agency, and to the public. The resulting presentation is largely dependent on each board’s preference as to the level of disclosure; this preference will vary from board to board. Board actions based on results. For boards that choose Level II or Level III assessments and therefore go through the goal-setting process, the administrator will compile all sug- gested goals and present the results. The board must collec- tively agree on its adopted goals, either by consensus or by formal board action (resolution). The administrator will then schedule a reassessment time when the board will reevaluate itself against its own adopted goals.