Click for next page ( 162


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 161
8 Effectiveness of Job Placement Programs The final type of program reviewed by the committee attempted to aid youths directly in finding employment. The programs usually offered some services in addition to job referral: workshops on preparing resumes, instruction in appropriate behavior during a job interview, and support groups for job seekers supplemented the more traditional job referral activities. While these programs offered some services that overlapped those we have previously termed labor market preparation, they are distinguished by their very concrete focus on securing employment, within a specified time period, for the youths in the program. In addition, wage subsidies (sometimes to employers and sometimes to the youths themselves) were occasionally used as a transitional device to get youths situated in suitable jobs; the hope was that the job would continue after the subsidy ended. Overall, the evaluation reports in the job placement category were, with one exception, generally weaker in methodological rigor than those addressing other program goals. As a consequence, conclusions about the effectiveness of job placement efforts are at best tentative. PROGRAMS FOR OUT-OF-SCHOOL YOUTHS Among the reports that passed our initial screening were those of four projects that represented job placement efforts serving out-of- school youths: 70001, Job Factory, Job Factory Voucher Program, and Job Track. Table 8.1 details the characteristics of each of these programs; Table 8.2 details the research design and results of the evaluations of the programs. 70001 70001 was a job search program for out-of-school youths aged 16-21. Enrollment was 60 percent female and 87 percent minority; the average participant was 18 years old; only 1 percent held high school degrees. The program consisted of an average of 32 hours of treatment involving job preparation workshops, job search training, and the like. Similar to other job search programs, it attempted to teach 161

OCR for page 161
162 U) 4~ ., cn U] v ._~ U] ._, s o e U] o 4 o o U] 1 o 1 o o U. S~ o a' P4 o . 00 m E o o ,4 Q~ _ s 4 a) ~ U2 ,. P4 ~ ~n O a 4 UD .,' ~ ~ O U) ~ O O ~ ~: 4~ ,~. O ~ O P4 4 :5 O O a) ul Ul ~ ~ ~ ~ V u~ ~ ~ a) ~ ~ ~ o~ 3 0 _' ~ ~ U] C) 4~ ~ ~U. ,1 - U] q~ ~ ~ - O > ~ ~q ~- - a' ~ a' u' ~ ~ ~ - ~ Q - ._' ~ - U] . - ~4 a' u: _ ~ k4 U) U] U] Y Y ~a u, a) 3 s.4 3 :: ~r O C~ ~:> o ~ 0 ~ 11 U] O U] 0) ~, 3 3 0 O S >1 L' S n5 a o 1 C~ U] 3 U] Ul U]U] s s ~Ss JJ ~ ~3 3 U13 O U] :>, O. U] :^ O u] 41) ~O J ~ V J~ 4 ~ J~ ~J ~:^ ~ 3 -1 U] 3 -1 3 tO - rl _] O ~ O ~ 0 3 ~ ~ 0 o4 a) 0 0 S Q' O ~0 Q. ~ O 01 0 0 a, O O ~ ~O ~ O ~ ~ O O ~ ~ ~O S ~ ~ . - S ' - ~ ~ ~ S ~ ~ . - ~S . - ~ V ~ ~ ~ s ~ ~ ~ v ~ ~ ~ ~v ~ 6 1 dP dR dP 1 dP dP otP d~ 1 d~ dP dR dP 1 dR dP q~ ~ 0 ~q~ 0 u~ ~ ~ ~q~ 0 c~ 0 a~ ~ ~0 ~ ~ ~ u, 0 ~ ~ u~ ~O a' 1 1 1 1 JU ~ e ~ ~e ~ ~e 3 3 ~3 O O O O 1 (1) 3 O 1 ~ ~O 0 4J 3 U] h S q ~L~ - ~:~ 0 v 0 ns u 3 ~ ~Ll ' tn C: ~ ~ ~ ~ ~ 0) ~ 0 ~4 0 Q 0 U] - 0) . - 3 -~ ~ 4 ~5- 4~ e U] _ J-) Q Cl Q' Q4 ~ U] ~ ~ O ~ ~ ~ ~ O Ll - ~ ~ ~S - ~ ~ ~ ~V O4 ' 4~ 0 00 )~ O ~ 3 ~ a) U] ~q V Ul ~ ~ ~ ,- ~ ~ ~ a) a' ~ ul u' pJ O U] - ~1 U] ~ 0) 3 Q S Ul Q, 4 ~ ~1 ~1 3 Q U) tt' 3 V2 Q ~ ~ ~ U] O O 1 a ~ a S ~ V ~ a' ' - 3 -I 3 3 ~ O ~ ~ 4~ V ~ O Q ~0~ 3 4~ ~ U] VV V O ~ ~ O ~ V V s `:s ~s ~ s 4J a) Ll V ~V ~V ~ ~ ~ O S 4.Jk4 J ~k4 J~ rl ~ ~ U] J~ (U 4~ u' ~ o~ 3 - s u' u, a' -~ - a)~ - u' ~ 4~ u~ ~ t.q 0101 U] Ul U] ~ Q ~ 3 Q Q U' U]U ~U) ~: 3 ~ O O O U] Q ~Q RS .Q (TS ~ tO ~ ~ ~ ~ tO OO O t~h: 4J ~ ~ U2 ~ ~ U] ~S O ~ U] O Ll ~ - - U] ~ U] V .~ 0) ~ ~~ ~ ~ Q) ~v 3 v ~ a' v s ~ ~ a) ~o a, V - ~ ~ o ~ -- ~ CD E~ >. u 0 ~, ~ ~0 ~ )- ~ ~ ~ ~1 a 0 Q m :D c ~ ~ ~ m ~ co ~ Q 0 O O O O [~

OCR for page 161
163 U] ~5 U] ~r: .,, U] s U] a, ~; e U] s o o o cn 1 o 1 o o U] o P~ a) v o . 0 m U] o 4 ~; Q. a 1 U1 O O UD 0 a' ~r; U) 4~ v q~ q4 o O Ul ~ O O e 0 0 s C) a o ~ Q4 4~ 3 ~ O O C) C' a) N U] U] O 4~ o Q. -,' .,' S~ P4 U) ~ ~Q. O ~ S U) 1 tlD u~ ~i ~ ~ ~a, 0 S ~ ~S ~ -~ t\5 ~ ~ ~ ~ u~ s ~4 ~a) 0 d >1 41) ~ ~ ~Q ~ ~ ~ ~ ~ 0 ~a 3 q~ ~0 ~ ~ ~ O0 1 ~ 1q~ ' - ~Y ~A ~. - 3 ~ 3 0 a~ ~ 0 0 ~5 ~ ~ ~ ~ 4~ s ~ ~ u' ~4~ ~ ~ Q - ~ ~ r~ o U] 0 ~ cr ~ ~ ~ q~ U2 ~ ~ . - -. - 0 a' ~v ~ ~ u~ s ~ ~ ~ ~ s ~ ~ ~ U) ~q~ ~ ~ ::5 ~ U] ~n ~ u, ~ ~ O a) ~ ~ ~ ~ ~ ~ O ~ s Q ~ ~ ~ ~ ~ a ~a ~ C4 U) U) dP oo 1 ~1 ~Q 11 0 d~ d~ O ,~ o o 1 1 ~a~ kD ~Q 1 ~ ~Y tn 4~u' u2 4 ~ ~11 11 S dP UldP dP O dP dP o ~ ~ ~ o Q~ ~S ~0 3 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~O O ~U] ~U' ~ ~ ~ ~ O 11 ~ ~ 11 11 Y ~ 11 11 Y ~ S S cn Y ~ ~ ~ ~ 0 ~ ~ a~ ~4 0 0 ~a) 0 ~ ~ a) 0 ~ ~ a' s~ ~ ~ ~ ~ ~ ~ ~r ~ ~a' ~3 ~3 C O O O ~ Q 3 ~Q~a, ~ ~C) ~O tD O C ~ ~C ~kD dP U] dP d~ a ~oo ~ ~ ., ~U] ~ 3 :~ dP ~ U] ~ s ^ ~U] o U] i, a' ~ 4J Y ~ cO dP a) 0 ~ + ~0 a ~ Q - dP - 3 ~ ~Q u~ u~ 0 a~ d~ O a) 0 ~ dP M0 U] ~) C: U] ~ -i ~ ~ Q ~ ~1= dP ~ ~ kD + -- s a) JJ ~ tD ~ ~1 :^ ~ ~ ns ~ ~ a) 0 u' 0 ~1 ~O U] AS ~l ~ ~ _I _I U] Y ~1 c: ~: dP _] Ul ~ 0 0 ~ ~ ~ Y ~P4 0 ~ c ~ Y a' ~q ~ ~ ~ ~ ~ ~ ~ . ~ ~ ~ t~ ~ O a) ~ 3 a, ~ )" ~a) ~' 3 o ~ 3 0) 4, 0 3 a) ~ ~r ~ ~ ~ S S e k4 3 -~ ~ $-~ ~r ,~ ~ u ~n 3 ~ ~ ~u~ ~ ~ ~ ~ ~ ~ ~ ~ ~ O O ~ O H _I ~ ~0) td ~0) ~> {d C) 0 Q5 ~g + ~Q (V C) e ~ ~U1 ~ H H H U2 4 ~ ~ ~ a) ~:, OJ ~E: 4~ ~ O V U2 ~ ~ O ~r4 O S ~' - O ~ ~ ~U] U] ~ ~ O U] (n ~. 0 G5 RS (1) 0 0? S ~ ns ~ ~ , - V 0 L4 ~ ~ ~ ~O O ~ ~ Q] ~ a) s ~ 0 0 Q. 8 ~ o o U~ ~D U) ~ ~o U ~C~ oo o Q ~ o ~ ~ ~o ~:^ O O JJ ~ a' S ~ ~ O E~ o O ~ :, O o QQ ~ ~Q O ~o OO O o Q. s o a) Q O - - :>, 3 - O ~ a, ~s s C) ~ :' ~ O O C' ~ :> ~Q ~

OCR for page 161
164 youths what employers expected, to teach them job search skills, and to motivate them. Unlike some other programs, the evaluation report indicates that the staff did some (unspecified) amount of follow-up with the youths after they had found a job. The research design by the Corporation for Public/Private Ventures (CPPV) is a matched comparison of youths in five program cities; all of the evaluations were operated by the central organization. The sample size was approximately 500 participants (all program entrants in the five cities between January 1979 and April 1980) and 400 comparisons drawn from a variety of sources, including Employment Service registers, school dropout lists, and other sources. The report notes that an earlier evaluation with a shorter follow-up period found significant initial gains followed by equally large decay effects for a similar job search program (Jobs For Youth), while the initial gains for 70001 did not appear to decay. An important question was whether this effect was an artifact of the sampling procedure (program termination dates are uncertain and the follow-up may work to keep youths in jobs) or whether the effects persisted. The placement rate was 50 percent, and the cost per enrollee (in 1979) was $1,351. The initial difference in earnings between participants and comparisons ($12 per week more or 35 percent higher) is statistically significant, but by 24 to 40 months after starting the program there is no significant difference between the earnings of participants and comparisons. The evidence thus suggests that the program may provide a brokering or screening function in helping youths to obtain job placements and that this effect decays with time as nonparticipants' earnings eventually reach parity. The decay effect persists in multiple regression analyses controlling for various individual characteristics and does not change for various age, sex, or race groups. With respect to job quality, at the 24- to 40-month follow-up, 25 percent of the male participants and 5 percent of the female partici- pants held skilled jobs. For the comparison group, the figures were 21 percent and 19 percent. The authors attribute the female pattern to a higher rate of childbearing by the participants, but even when the analysis is limited to women with no children, the comparison group does as well as the participants. The program stressed completion of the General Equivalency Diploma (GED), and there appears to be a significant long-term impact: 31 percent of the participants received a GED compared with 12 percent of the comparison group. However, only 3 percent of the participants received a regular high school degree compared with 9 percent of the comparison group. Hence, the comparative results for educational attainment are slightly lower than the GED results imply, but still favor the participants by a statistically significant amount. There are no other noticeable effects with respect to training, military service, crime reduction, or the like. The report cautions that the evaluation sites were known to perform better in terms of job placement than sites not chosen. Thus, the sample may not be representative of all 70001 programs operating during the 1979-1980 period. The response rate at 24-40 weeks postprogram was

OCR for page 161
165 87 percent, and there appears to be no substantial attrition bias (efforts were made to control for attrition bias). Participants and comparisons appeared to be matched closely in terms of most demographic characteristics with the exception that at entry female participants had significantly fewer dependents than comparison group females. The difference was no longer significant at the 24- to 40-month follow-up, apparently because of higher rates of childbearing among participants. Female participants were also significantly younger than nonparticipants. Overall, we believe that the results of the evaluation of the 70001 sites studied are reliable, but the results may not be generalizable to all sites because the sample sites were known to be better than average prior to selection. Job Factory Brandeis University evaluated job search assistance programs that operated in Cambridge, Massachusetts, and Wilkes-Barre, Pennsylvania, from 1979 to 1980. However, because of severe implementation diffi- culties with the program in Wilkes-Barre, we disregarded the portion of the report dealing with that program. The Cambridge program, the Job Factory, enrolled 50 youths in each of five cycles. The first and last cycles were for graduating high school seniors and began the first week in June of 1979 and 1980, respectively. The middle cycles were for dropouts. Youths in the first cycle and one of the middle ones received stipends of $3.10 per hour while the others did not. Each cycle lasted four weeks and youths received an average of 83 hours of motivation, job search preparation, and role-playing. The research design used random assignment. The sample size was approximately 203 participants and 165 control individuals. Data were collected on a variety of outcomes including the job finding rate, job characteristics, job-finding methods used, and results of various tests in YEDPA's Standardized Assessment System (SAS) battery. Data from the first follow-up interview at 6 weeks postprogram indicated that, overall, participants were about one-third more likely to be employed than controls (64 percent and 48 percent). By the time of the final follow-up, at 36 weeks postprogram, the job finding rates was slightly higher for the controls (82 percent were employed compared with 79 percent of participants). The quality of the first postprogram job did not differ a great deal between the two groups. A somewhat larger proportion of partici- pants were employed full time (67 percent compared with 53 percent). Differences in hourly wages appeared to favor participants slightly ($3.50 compared with $3.40 per hour). Although the participants' jobs appear to be slightly better, this may be understood by the slightly larger percentage of participants in jobs that were subsidized by public funds (20 percent and 16 percent). Of the seven psychometric scale items in the SAS battery only three were statistically sig- nificant at the .05 level and none was correlated with outcomes.

OCR for page 161
166 The results for the seniors in the first cycle who received stipends exhibited the same time pattern as did the results for dropouts. A slightly higher percentage of the seniors held jobs over the course of the follow-up, but the differences were not great: e.g., at the final follow-up, job holding for seniors was 83.3 percent for participants and 84.2 percent for controls; for dropouts, the numbers were 73.9 percent and 78.6 percent, respectively. Various cost calculations indicate that the average cost per par- ticipant was $989, the cost per employed youths was $1,441, and the net short-run cost per new job (i.e., jobs that would not have otherwise been found within the first 6 weeks postprogram) was $4,468. Attrition in the analysis sample was substantial. Responses to the interview at 6 weeks postprogram were obtained from 64 percent of the participants and 52 percent of the controls; at 20 weeks postprogram response rates were 41 percent and 34 percent, respectively; by 36 weeks postprogram the response rates were 26 percent and 20 percent, respectively, with a total of 53 observations for participants and 33 for controls. Significance tests of the differences in the mean characteristics of jobs between controls and participants were not presented in the evaluation report. Given the small sample size, it is unlikely that the reported differences are statistically significant at conventional levels, and we therefore cannot be very confident about program effects, especially after 6 weeks. Job Factory Voucher Program The Job Factory Voucher Program was a variation of the Job Factory model in which youths received a wage subsidy if they found employment quickly. The supplement was $1.50 an hour for the first 2 weeks of work and $1 an hour extra for weeks 3-12. Youths were recruited for each of six 4-week cycles of the program between November 1980 and December 1981 and randomly assigned to one of three treatments: Job Factory plus voucher, voucher only, or no treatment. The results show a peculiar pattern of effects. At 4 weeks post- program the full-treatment group does better than the voucher-only group and the control (no-treatment) group. By 12 weeks the full-treatment and voucher-only groups are equal, and by 20 weeks the voucher-only group does better than the full-treatment group. Fifty- eight percent of the full-treatment group worked between the second and third follow-up compared with 70 percent of the voucher-only group and 51 percent of the control group. HA Wilkes-Barre program that provided subsidies to employers was also evaluated, (Rivera-Casale et al., 1982) but because it experienced severe implementation problems and did not use a comparison group of youths not participating in the program, the committee decided that the results did not provide reliable evidence.

OCR for page 161
167 Attrition and resulting small sample sizes may account for some of the findings. Between the initial data collection and 20 weeks after enrollment, attrition among participants in the full-treatment group was 20 percent, among the voucher-only group it was 82 percent, and among the control group it was 38 percent. Final analyses are based on observations for 60 controls, 23 participants in the voucher-only group, and 128 in the full-treatment group. A number of other methodological problems in the evaluation are cause for concern: administrative implementation difficulties, changing characteristics of the youths over the course of the program, and interaction among youths in the different treatment groups. Consequently, we are not confident in the results of the Job Factory Voucher Program evaluation. Job Track Job Track was a job search assistance program that offered 2 days of job search training followed by 3 days of support services to out-of-school youths who applied to local Employment Service offices. Participants were 16- to 21-year-old, out-of-school youths. The program operated from July to December 1980 in San Francisco. Olympus Research Centers was responsible for both operating the program and doing the evaluation. Participant outcomes were compared with those of a matched compari- son group of nonparticipants. The evaluation sample was originally composed of 136 comparison group members and 103 participants, but the analyses were actually based on 88 participants and 76 comparison group youths at the 6-week follow-up and 80 participants and 69 comparison group youths at the 12-week follow-up. The regression-adjusted results at 6 weeks postprogram suggest that participants were more likely than nonparticipants to be employed (46 percent compared with 28 percent). At the 12-week postprogram follow-up, the employment rates of the two groups were 66 percent and 49 percent, but the difference was not statistically significant. There were no apparent differences in job search intensity or the number of methods used. About 50 percent of the youths in the program found employment without going through a formal interview procedure. Comparison group members differed from participants in two major respects that would suggest that comparisons were more employable: 30 percent of comparison group members compared with 18 percent of participants were independent (that is, neither family heads nor family members), and 22 percent of comparison group members and 13 percent of participants had some college training. We have limited confidence in the findings of the effects of the original Job Track program. A modified program, Job Track II, operated for 10 weeks between March and June 1982. The new program offered a stipend of $50, extended the program to 2 weeks, and differed from the earlier version in several other respects. The Job Track II evaluation did not use a comparison group, but compared the outcomes of participants in the new program with those in the earlier one. Therefore, we did not consider the findings of the Job Track II program.

OCR for page 161
168 PROGRAMS FOR IN-SCHOOL YOUTH S The committee reviewed reports on three programs that provided job placement services for in-school youths: Jobs for Delaware Graduates, Jobs for America's Graduates, and Project Best. Table 8.3 details the characteristics of each of these programs; Table 8~4 details the research design and results of the evaluations of the programs. Jobs for Delaware Graduates Jobs for Delaware Graduates is a school-to-work transition program for high school seniors. Begun in Delaware, it is currently being replicated throughout the country by a central organization, Jobs for America's Graduates. We reviewed two evaluations: one of the original Delaware program done by Temple University (discussed in this section) and one based on four sites done by Northeastern University (discussed in the next section). In the Jobs for Delaware Graduates (and Jobs for America's Graduates) program, high schools first develop lists of seniors who are in the bottom of their class and who are eligible for the program. The seniors participate in as many as three rounds of interviews and then are selected to enter the program (34 percent of those interviewed were selected in the programs that Temple University examined). The program consists of job preparation workshops (e.g., resume writing and interview techniques), a support club, assistance in job finding, and follow-up after job finding by program counselors. The Temple evaluation for 1980 Delaware graduates used comparison groups drawn from other Delaware high schools that were considered comparable but did not have the program. By 1981 too many high schools in Delaware had the program, so for the evaluation of that year Temple examined the estimated changes in program effects from 1 year to the next. This latter evaluation was of limited usefulness both because further changes may have occurred and because this methodology cannot eliminate effects of changes in the economy between the two periods. Hence our analysis focused only on the study of 1980 graduates. The researchers conducted follow-up interviews 3 and 8 months after graduation, but the program was still in effect even after 8 months because the counselors maintained some follow-up contact. Also, given the findings from other projects concerning decay, 8 months may not be a long enough follow-up period to assess postprogram effects. The differences in outcomes between participants and comparison group members at the time of the 3-month interview are all significant at the 1 percent level or better. The results indicate that partici- pants were more likely to be employed full time at the time of the interview (56 percent compared with 36 percent), more likely to have held a full-time job (75 percent compared with 49 percent), and more likely to have been employed since graduation, (84 percent compared with 73 percent). The results at 8 months postprogram indicate that participants still fared significantly better than nonparticipants in terms of employment, though the difference was smaller than at 3 months

OCR for page 161
169 U] .,, U] U] ,, U] .,' t) s C) o . U2 s ~5 o o o s CO 1 H o U] o a' P4 Q o . a) m ~: E~ O C O . - JJ O . ~ S V 4~ ~ a' P4 U] v .,, JJ 3 u, O - c' a 4 a ~ S E~ C) U] ~ a V ._, ~ O a U) Pi S V ~ O O h Q. ~ ~: 4 a) O h ~ O S 4 :5 O ~1 ce ~ C a' 0 ~C O ' C O 4 ~ N U] ~C S . - J~ ~ ~ C ~ V U] ~ L. ~ `15 tl] C ~R ~ C ~ U) ' ~ S '7 c 0 v c :3 ~ a) s cr. ~IS t ~ -1 S ~ U] ~ ~ a~ c JU v ::~ U] ~I s U] ~ U] ~ O a~ u~ u~ c 3 ~ ~ ~ ~ ~ C 0 ~ ~ ~ a) ~ ~ ~l u, V ~ U] ~ ~ E ~ V S P~ ~4 ~ a) a) a {r 0 a' ~ 0 O ~ O S ~S V ~V U] tq V ~Q U] s~ ~ s~ O ~U] S ~S 0 4J O O Z oo U] C O o ,, U) o ~S ~o 4U _ U2 >1 0 ~ ~a, O ~U] V 4~ tt ~V 11 ~ OS O ~ O V ' V C V ' - ~ ~Q ' - U] ~ ~ ~ ~O O U] O ~ ~ O ~ ~V S ~ C ~ ~ ~ ~ ~ ~S ~C ~ ~ a~ q~ v ~ ~ ~Y ~ O V ~ ~ ~ ~ C s ~U] O V O S Q) ~ ~ ~ - O ~ O ~ O V 4~ 0 V ~ C ~ ~ ~_ ~ C ~ . - U) d~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ O ~ Q In ~ q~ ~ V Ul ~ ~ ~ V ~ ~ ~ ~ O O S C~ ~ Q ~ ~ O ~ C ~ S ~ ~ O ~ 1 ~ dP :5 ~1 ~I dF dR 0 4~ -1 ~ d~ dP V V O dP s ~ 0 ~ 0 Q~ Q) v l~ ~ s ~ Q ~ 1- U) 0~ V S c~ U] c ~Ul ~ ~ V ~ ~ X u' - - C ~u' ~ O S a' I e e ~ e O e O e e I _ V e e e ~ U] ee C C ' - H 1 1 ~ S ~ S ~ O ~ O V O V O .- C 3 ~ ~3 ~ ~Q ~' aJ ~ C ~ C c al ~c ~ ~ ~ c a o U] o U] tO mI V . - ~. - ~ ~, Q a' ~ Q a' O a' a' O V ~ O V ~ u' .~ ~ ~L~ '= C ~> `4 ~ ~tO I aS ~ I tu ~ Q Q' '~ 3 ~ '= 3 ~ ~ O O a~ u~ u~ 0 ~ UD u~ O ~ a~ v ~ ~ ~ ~ ~ ~ ~ Y Q4 0 U) ~1 O4 0 U] ~ ~ Q ~ S U] O S Ul O O n, O C Q Ul ~ ~Q u' ~ q ~S ~ -m O O ra ~ 0 s" 0 ~0 ~ 1 0 ~ 1 0 ~ 1 0 0 a' s 0 ~ s 0 aJ s ~ ~ ~s~ ~ V ~ ~ V Y ~ ~ u' ~ ~ a' u' Y C:4 ~ cn Ll ~ ~ ~ O ~ ~ S O ~ ~ S O ~ ~ S 3 0 0 ~3 0 0 (J) 3 0 0 1 ~ V ~1 ~ V ~1 ~ V ~ O ~ C S 01 0 ~ ~ S U) 0 4J C S (Q - - O ~ -^ O ~ ~ O s~ 1 U) C ~ 0 1 U] ~ ~ 0 1 ~ ~O C C ~ C C ~ C C 0 15 ~ ~ C O RS ~ ::' C O ~ L~ 0 ~ 0 0 ~0 ~ 0 0 ~0 ~ O O a S ~ ~I Q u' S 4~ q~ D cn S ~ q~ Q U] V V V cn u: u~ U] C ~- - >1 a) >, E ~ Y ~:>1 u ~n u ~4J ~ u ~C u~ ~4~ , - _ ~U) ~ ~ ~' - u ~ ~ ~ u' m h ~ S C ~4 ~ ~ ~ a' ~L' ~ ~ ~ ~ ~ a' ~ {J) ,4 a) ~ 0 3 :5 _ ~ a) O ~ :~ -S a' 4~ ~ O ~ c: ~ a' , - , ~ s~ ~ ~ 4J > V ~ ~ O ~ Q :> ~ ~5 Q ~ ~ n~ F:: ~ ~ 0~ 0~ Q ~ ~ E: ~ u~ a~ ~ ~ a~ c: U? ~ ~ h) O C r~ a~ E: s ~ a~ c Q Q tS ~ ~ ~Q <: C~ ~ Z 3 O ~ ;~ O O h:

OCR for page 161
170 U] o C) U] ::s U] a ~; ._' U] a' s v 0 ~; e 0 s o o o v U] H o 0 o a' a O U] Q o d4 0D ~; E~ ~; a 1 U] 3 ~ O O Ql U. O ~ U] aJ o s~ P4 O ~n - O O ~ O O s 4 ~: o L~ Ql :5 ~ O O C) O 4J O N ,4 ~ U] QJ ,t (1) ~ .,' Ql ~ U] PM S z5 0 o Ll :>d ~ o ~ L4 ~1 O ~ O o U] S Ul a~ 4 ~L4 q~ U] ' O q~ - - ~ a' ,4 i,, ~ ~ - ~ C a) a) c u, Y O - ~ ~a, ~ ~ E ~ O ~ ~ ~ ~ U) ~ ~ ~ ~ _ U] 115 U) a 0 ~a' 3 o O U~ tU C) U] - l ~, - ~ ~ ~ , ~a~ 4 ~ 0 0 L4 ~ :5 a' ~ ~ ~Y ~ ~ a' o ~ c: 1 Ir) J~ N U] ~ S O O ~ ~ JU C: O O U] S ~d d ~P ~ dP de ~ u, ~ ~u2 ~o ~u, c~ ~ ~ ~r 0 ut S ~ ~S r~ (D S ~ ~ a' ~S O ~4 ~ ~4 ~ ~C 11 11 ~11 11 ~11 O ~O O S q ~O L~ O c ~3 oP ~ ' o + u ~ u ~a ~n dP L Q - t7 ~ 00 ~ ~3 0 ~O _ + v a' ~ ~ ~ d~ `:~ ~ S ~ ~ ~V 0 3 ~ ~ o + ~ dP~ O G) U] ~3 ~ Q' ~ ~! ~I~,~ a~ u~ 0~ ~_ ~ _ ~ 3 JJ F~ ~ ~ s O ~ ~ ~:> ~0 C4 14 + U] ~ ~ U] - ~ ~ 4~ ~ ~ Ul Y ~ ~ ~ ~ ~ O u' a, ~ ~U2 a' a' 0 ~ ~ ~ ~ 0 ~ ~ ~ ~ O s :^ ~ ~s ~ ~ ~ O ~ ~ u' ~ u' ~ u' S ~ 3 O O ~JJ O 0 4 ~ CD ~ + ' - O S" O `: ~ ~ ~ ~ ~ ~ ~ _ 0 ~ ~ a' 0 ~ ~ ~ Ul ~ .- ~ ~ ~ ~ ~ ~ ~ ~ O ES ~ ~ V ~ ~ [; ~q~ ~ ~ ~ ~ ~ Ei ~ ~ co ~ u' J ~ ~ ~ ~ h ~ ~ ~ ~ ~ ~ ~ a, ~ RS ~ ~ O O ~ ~ ~ ~Q e e ~3 ::' ~ s ~ e [~ e IS) e ~g e + (J e ,~,, ~O Z Z ~U] Q ~ ma O O ~s u, ~ ~O ~ ~ ~ s 3 ~ s ~ ~o ~ ~. - . - CD ~ ~ ~ s O 3 ~ O ~ ~O O U] ~O O ~ P ~V 3 0 a.' s s 0 s o ~ J ~ ~ S~ S O ~: c ~0 ~ 0 oo o Q tD ~ O o o C~ Ul E~ - u ~U~ U] cn a' a) - a) ~ ~ ~ ~m O 3 ~ - O ~ ~ _ 4 t ~ ~ ~a) ~ ~ QO t' QO ~ ~O ~ O O O O S S U] U] O ~ OD CO a~ a 1 1 O 0o

OCR for page 161
171 attitudes, or self-esteem measure of iob-seekinc' sk postprogram (e.g., 58 percent of participants and 48 percent of nonpar- ticipants were employed full time at the 8-month interview).2 The jobs held by participants and nonparticipants were similar in terms of hours per week, skill level, and tenure. However, at 3 months postprogram participant wages on the most recent job were insignifi- cantly below those of the comparison group, but at 8 months the participants had an edge of 38 cents an hour (participants received $3.90 an hour in the late fall of 1980), and this was statistically significant at the 10 percent level. Unfortunately, this is the only instance of a statistically significant difference in the nature of jobs held and hence should be viewed cautiously. There were no differences between the two groups in terms of the Educational Testing Service's (ETS) measures of job knowledge, work There was a significant difference in a ~ _ _ ills at 8 months postprogram that favored participants. The job-holding gains of the participants were offset by the nominally higher but statistically insignificant postgraduation school attendance rates of the comparison group. Participants in the Jobs for Delaware Graduates programs differed in important respects from those in other programs whose evaluations we reviewed. All were high school seniors, so no dropouts were included, and potential dropouts may have been screened out. Only 20 to 25 percent of participants were economically disadvantaged. The program involved an extensive preselection process that may lead to creaming, i.e., selection of those applicants who might be easiest to place. At the same time, the youths in these programs were not in college prepara- tory or vocational programs and were at the bottom of the class ranking in the general curriculum. The sample of participants was 25 percent economically disadvan- taged, 37 percent minority, and 56 percent female; 25 percent had pre- viously had a skilled or semiskilled job, and 72 percent had previously worked in a job paying at least $3.17 an hour. Participants' scores for the SAS reading test indicated a reading level of eighth grade or higher. The comparison group had a higher percentage of minority youths (47) and a higher percentage of limited English speakers (6.3 compared with 2.6), both of which might bias the results in favor of the program, but the comparison group had a better work history (81 percent had previously worked and 48 percent had held skilled or semi-skilled jobs). While the results were less favorable at 8 months than at 3 months, attrition may play an important role. Attrition among participants was 24 percent 3 months after graduation and 28 percent 8 months after graduation; among comparison group youths attrition was 31 percent at 3 2 The summary of the published report (Eleey and Leone, 1982) states that some key findings are insignificant while the body of the text states that they are significant (Eleey and Leone, 1982~; these incon- sistencies were corrected in an errata sheet from the authors that says the findings are statistically significant.

OCR for page 161
172 months and 33 percent at 8 months after graduation. There is no indica- tion that attempts were made to adjust for possible attrition bias. Thus, differences in attrition rates may contribute to the measured differences in postgraduation experiences between participants and nonparticipants. Another potentially troubling issue on which the report is silent is the treatment of dropouts. Since the participant sample includes only graduates, the appropriate comparison would be nonparticipant graduates. Inclusion of nongraduates in the comparison sample would probably tend to overstate program effects on employment. The report presents what we found to be convincing evidence of short-term, i.e., 3-month, postprogram effects in increasing employment. The 8-month findings indicate a smaller effect and are less convincing. Because of sample attrition it is possible that what appears as a decay effect is due at least in part to attrition. Jobs for America's Graduates The Northeastern University study of Jobs for America's Graduates was sponsored by Jobs for America's Graduates and was a 6-month follow-up of spring 1982 graduates in four states: Arizona, Massachu- setts, Missouri, and Tennessee. The study used a matched comparison group methodology with a sample of 1,106 participants and 410 compari- sons. The total sample was 53 percent female, average family income was $11,000, and 95 percent of the sample were high school graduates. Data on participants were collected by program counselors during the 9 months of postgraduation follow-up visits. The results indicate that during fall 1982 participants fared sig- nificantly better than nonparticipants with respect to the probability of being employed, weeks employed, hourly wage rates, and weekly earn- ings. No analysis of intersite differences is provided, but based on other studies there is reason to believe that there would be substantial variation across sites. Several considerations make us skeptical of accepting the results of the evaluation of Jobs for America's Graduates. First, the attrition rate at the 6-month follow-up was 6 percent for participants but 40 percent for the comparison group; no adjustments were made for possible attrition bias, and no data are presented that allow examination of the effect of attrition on the match between participants and nonpartici- pants. Even if the results reported were robust, studies of similar programs indicate that a 6-month follow-up period is too short to allow valid inferences to be drawn about long-term program effects, which are susceptible to decay. Finally, the extensive preselection procedures used may have produced a participant group that is not generally representative of non-college-bound high school seniors. Project BEST The Better Employment through Skills Training Project (Project BEST) involved 1 hour per day of labor-market oriented classroom

OCR for page 161
173 training in conjunction with counseling and "job shadowing n for disad- vantaged minority high school seniors in an inner-city Philadelphia high school. The Temple University study of the project is of interest because it did not find employment gains for participants relative to the comparison group at 3 and 11 months postprogram. The program operated during the 1979-1980 and 1980-1981 academic years and served about 350 students. While the study suggests that the project's job placement strategy was ineffective, its findings cannot be taken as conclusive, inasmuch as the comparison group was not ran- domly selected (they were students at other Philadelphia high schools) and program participants were self-selected. While the evaluators acknowledged the selectivity bias inherent in this approach, they made no explicit correction for it. SUMMARY While most of the evaluations of programs offering job placement services to youths found the programs to be effective in securing employment for participants, most of the evaluations had serious methodological flaws and therefore do not provide reliable evidence on the question of effectiveness. Consequently, we do not believe one can draw strong inferences about program effects on the basis of these studies. Of all the evaluations of YEDPA job placement programs serving out-of-school youths, the CPPV study of 70001 comes closest to providing trustworthy evidence of program effectiveness. The program reported a 50 percent success rate in placing participants in jobs, and program costs averaged $1,351 per enrollee. For the demographic characteristics reported in the evaluation, there is a reasonably close match between the comparison and the participant groups. The only major difference between groups was that female participants in the 70001 program had fewer dependents than the comparison group. Even for this study, however' we have questions. Besides the con- cerns generated by the use of a constructed comparison (rather than a randomly assigned control group), the selection of sites in the 70001 evaluation is a cause for concern. The chosen sites were known to be better performers in terms of job placement. The resultant evaluation data may thus provide an upper-bound estimate of what the 70001 programs achieved. While the design of the 70001 evaluation is somewhat problematic, the execution and reporting of the research were rigorous (see CPPV, 1983~. In contrast to most of the studies we reviewed, the CPPV evaluators and their subcontractor, Institute for Survey Research, Temple University, obtained an 86 percent response rate at 24+ months postprogram. Moreover, the report is appropriately candid about the design problems of the study, and it presents detailed calculations and discussions of the potential effects of selection and attrition bias. Nine months after completion of 70001, participating youths earned an average of $12 per week more than the comparison group. This difference in earnings arose from increased employment rather than

OCR for page 161
174 differences in wage rates: 41 percent of 70001 youths and 29 percent of the comparison group were employed. A subsequent follow-up conducted 24-40 months after participation in 70001 found that this program effect had decayed entirely: employment rates were 38 percent for 70001 participants and 42 percent for the comparison group. On the basis of the 70001 study and other evaluations, there is evidence that the effects of job placement programs decay over time, so that after 24 months there is no discernible difference between participants and nonparticipants on most outcomes. Two highly regarded programs for in-school youths, Jobs for Delaware Graduates and Jobs for America's Graduates, served a segment of the youth population that was least in need of assistance in locating suitable employment--high school graduates, 75 percent of whom came from families that were not economically disadvantaged. While we found the evidence of short-term program effects convincing, any inference that results from such a program could be realized with economically disadvantaged populations or with school dropouts is highly speculative.