Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 203
Comprehensive Family Service Programs: Special Features and Associated Measurement Problems Kathryn Hewett, with the assistance ofDennis Deloria INTRODUCTION A few years ago a team from the Denver, Colorado, U.S. General Accounting Office (GAO) visited a child develop- ment demonstration program in Gering, Nebraska, as part of a routine review of such federally funded programs in the region. At this Child and Family Resource Program, the GAO team was surprised and impressed with the many types of services provided and with the program's approach to planning and delivering services to each family. Intrigued, they looked at 3 more of the 11 demonstration programs in other areas and subsequently produced a glowing report on the comprehensive family service model as implemented by the Child and Family Resource Program. Their report called attention to several aspects of the model that distinguished it from most traditional child development programs: the emphasis on families rather than children, the approach of developing with parents a distinct plan of service and activities for each family, and the program's role as a coordinator of services in the community for each family. So enthusiastic was the GAO team about the model that they recommended it to Congress as a prototype for future child development program legislation (GAO, 1979). The Child and Family Resource Program is one of several research and demonstration programs developed by federal and private researchers that attempt to influence the development of children working with families. A number of demonstration programs funded by the Head Start Bureau of the Administration for Children, Youth, and Families (now part of the U.S. Department of Health and Human Services) have been delivering comprehensive family services for nearly a decade (15th Anniversary Head Start 203

OCR for page 203
204 Committee, 1980). Among these programs are the Child and Family Resource Program, Home Start, the Parent Child Centers, and the Parent Child Development Centers. Most of these Programs had roots in child development inter . ventlon programs developed by researchers such as Levenstein, Weikart, Gordon, Klaus and Gray, Lally and Honig, White and Watts, and many others. Initiated as experiments in providing basic Head Start services, these programs showed the influence not only of the child development philosophies of Head Start but also of philosophies about parent involvement and community political action that characterized Head Start in the late 1960s and early 1970s. At the same time other researchers were developing family oriented programs with the aim of enhancing child development. Such programs as the Brookline Early Education Project, the Syracuse Families Project, and the Family Networks Project at the Merrill-Palmer Institute resulted from these efforts. In general these programs combine early childhood intervention and family support in various degrees, providing them directly by program staff and indirectly through coordination of other service agencies in the community. It is the combined emphasis on child and family and the broad array of available services that makes them "comprehensive." The federal demonstration programs, and some of the private ones as well, provide a full range of health, nutritional, and educational services similar to those available to children in Head Start, along with broad social and educational activities for parents, such as job counseling, child care assist- ance, health screening, housing improvement, and legal and other services. They go beyond Head Start and other early intervention programs in enabling greater parent participation in determining the direction of the program, _ In emphasizing both child development and family services, . . in assuming a greater role in coordinating services In the community for participating families, and in improving family services in general for the community. This paper emphasizes the federal demonstration program models, particularly the Child and Family Resource Program and Home Start. Of course, this paper represents only one approach to issues affecting families. It does not survey the range of policies and programs that cur- rently affect family life. not focus I_ _ _ which legislates broad services for low-income families on entitlement Programs, such as.Title XX, For example, this paper does

OCR for page 203
205 (e.g., day care, homemaker service, family planning, etc.) Nor does this paper consider policies and programs affecting family life in countries other than the United States. Some of the provocative implications of policies in Europe and Asia can be found in the writings of Kahn and Kamerman (1975). There are also important single service programs (such as those for day care or health) that influence the quality of family life (see Travers et al. and Levine and Palfrey, in this volume). The set of models discussed in this chapter were selected to illustrate important issues in measuring the effects of family programs. . First, these models have been implemented in a variety of urban and rural settings nationwide and have served many different subcultural groups of families: Appalachian, urban white, black, Hispanics (both Puerto Rican and Chicano), native Americans, and many other regional groups. Second they have attempted to implement a personal iced, direct approach to delivering services that involves the establishment of relationships with families. Compared with an entitlement program that theoretically serves all eligible families, these programs have a more intensive approach, serving fewer families in greater depth. Their approach is similar in some important ways to a clinical community mental health model of family service and in many other important respects is different, too. Finally, all of the federal and private family service programs mentioned were conducted simultaneously with a research program; thus, these programs and the evaluations associated with them represent the most recent attempts to grapple with the research problems inherent in family service programs. Thus far the research suggests that however promising the comprehensive family service models - - may be, they pose conceptual and practical problems for research that are not yet solved. In this paper the problems posed by family service programs and the efforts to grapple with these problems are organized around three features common to most of the programs mentioned. These features, in turn, have major implications for measurement: · the mix of support and intervention; · the emphasis on individualized treatment; and · the role of integrating community services for families.

OCR for page 203
206 Throughout this paper the discussion of programs and measurement issues rests on a broad general assumption that it is necessary to consider not only measurement but also the social context in which programs operate and the values of different groups who may have a stake in the program. Several researchers have observed that research questions and measures concerning child development have historically reflected the values of individuals and of society at any given time (Kagan et al., 1978; Kessen, 1979). While this observation may apply to all research about children and families, it is particularly true of research concerning social programs that have diverse constituencies. This is not merely a philosophical stance. Because of the comprehensive and individualized nature of the programs, a great many domains of family life might be affected by program participation. The challenge to the researcher is to adopt a framework for research that helps to make explicit the different values, adopt or impose some priority among the effects of interest, and select measures capable of both detecting effects of primary interest and describing other possible effects of interest to other constituencies. OVERVIEW OF FAMILY POLICY RESEARCH Trends in Family Research Family research has roots in many disciplines: rural and urban sociology, economics, and many areas of psychology--developmental, educational, and clinical. Until recently, research in these areas proceeded in parallel, with little exchange across disciplines. Generally, policy research concentrated on outcomes for children and on defining program treatments that children received, but it largely ignored outcomes for family members other than the mother, for families as a whole or for social groups, institutions, or communities. A review of family and family-related policy research published in 1978 concluded that although more than 26,000 studies of the family had been conducted since 1974, the questions posed by family-oriented research represented "new questions for policy, and for behavioral science as well" (Newbrough et al., 1978:85). The "new" questions were concerned with the reciprocal influences among family members, the relationships between families and other social organizations or institutions (extended families,

OCR for page 203
207 schools, and social programs) over childrearing, and the intended and unintended effects of intervention programs. Overall, the questions signal a broadening of concern, from the child as primary focus of research and policy to the family and the social groups and institutions that are intertwined with family life. Because this broadening of concern is reflected in federal and privately sponsored research, it is useful to summarize the more general views that are often associated with it. · Families remain important units for the socializa- tion of children, and outcomes for families as well as children must be the concern of social programs and policies. · Families are characterized by a dynamic process of development as a group that is similar to but not the same as the development of its individual members. In addition, certain states in the normal course of family development are likely to be sources of stress, as are the extraordinary events of family life (Hill and Mattessich, 1977). · Families are both social and psychological units with various sizes, memberships, and values, which reflect subcultures, regions, and ethnic and religious backgrounds (Hill, 1971; Nobles, 1976). · There is an "ecology of human development" (Bronfenbrenner, 1977) in which the family is an important one of many interrelated social groups and institutions which overlap in the care and socialization of children (Hertz, 1976). The Newbrough report summarizes the work and findings of many researchers when it suggests moving federal policies away from programs of categorical services (i.e., programs that provide only health or only child development services) toward providing support in many forms to families based on varying needs from family to family. Furthermore, the report suggests that such programs of support should include mediating structures in social environments (churches, kinship networks, and others). Optimal development of children depends on an ever- broadening circle of influences. These influences begin with the child's individual ability and temperament and move to the immediate family and household, to the extended family, its history and social place, and beyond, to the informal groups and formal institutions of society.

OCR for page 203
208 It is clear that such views about the interdependence of families and social institutions imply a complex concep- tualization of family policy that must be reflected in research. Assumptions About Family Policy The foregoing summary of family research implies that measures of program effects are influenced by assumptions concerning values and social and personal responsibility. Such issues will provide discussion and disagreement among policy makers and researchers. ~ ~ ~ Neither pO1 Icy nor research, However, can oe made without recognizing the importance of such values. Consequently, we present some of the assumptions we make about current family policy. First, we assume that a climate of belief persists concerning the value of providing some types of service or support to American families, although models and systems of service delivery may vary from agency to agency. Second, equity among different income and class groups will continue to be important in determining policy, although the political and economic mood of the country in mid-1981 raises questions about the differ- ential effects of budget cuts on poor, working-class, and middle-class families. While the limitations of program eligibility for low-income families have both desirable and undesirable consequences, those families will continue to receive some types of federal support. Third, we assume that the federal government will maintain some distance from direct intervention in family affairs but will continue to exert indirect influence on families through the types of financial and service structures adopted (services, programs, and stipends) and the types of coordination required between agencies, schools, courts, etc. Fourth, we assume that there is some awareness and an acceptance of the diversity of family types and life- styles in this country. If we accept the integrity of various family types and ways of functioning, we can also assume that it is difficult to agree on a single treat- ment, attitude, or behavior that is the "best" way to enhance child and family development e What is ideal for children and families may vary by region, community, subculture, or developmental stage. Finally, we assume that most policy decisions are political and may not directly reflect the use of

OCR for page 203
209 research. Traditional outcome research is only one of several types of information that policy makers use, depending on the timing and political issues surrounding the decision. However, it is the responsibility of researchers to be aware of the assumptions and implications of the research questions asked and the methods used (Weiss, 1976). COMPREHENSIVE FAMILY PROGRAMS Program Descriptions In 1973 the Administration for Children, Youth, and Families (formerly the Office of Child Development) initiated the Child and Family Resource Program as part of the Head Start Bureau's research and demonstration program. There are 11 of these programs across the country, 1 in each of the 10 regions and 1 representing the Indian and Migrant Division. Each program receives approximately $130,000 per year to serve a minimum of 80 families; some programs serve as many as 350 families. The Child and Family Resource Program is a family oriented child development program, providing support services to families with children from the prenatal period through age eight. It meets children's needs by working through the family as a unit and provides continuity in serving children during the major stages of their early development. This is accomplished through three program components: (1) an infant-toddler component serving parents and their children in the prenatal period through age three; (2) Head Start, for families with children ages three to five; and (3) a preschool-school linkage component, to ensure a smooth transition from preschool into the early elementary school grades. Another distinctive feature of the program is its emphasis on a comprehensive assessment of each family's strengths and needs and the development with the family of an individualized plan for services. There is much variation in the structure and staff rules from program to program. Some programs have separate staff providing child development services, social services, and health services to families; in others a single person has primary contact with the family, although this person is backed up by specialists in the three areas. All programs provide a mix of home visits and center activities for mothers and children. All have a variety of educational

OCR for page 203
210 and recreational activities for parents in addition to child development education. Recent information and evaluations of the program can be found in Nauta (1981); Nauta and Johnson (1981), and Johnson et al. (1980). Home Start operated actively as a demonstration program from 1972 through 197S; it was designed to demonstrate methods of delivering comprehensive Head Start-type services to children and parents for whom a center-based program was not feasible. Much of the program--parent education, social services, and child activities--took place in the home. All of the 16 programs also had center meetings, where children and parents got together two or three times a month. Many of the programs were in rural areas; guidelines were explicit in the intention to provide services as families identified them. In 1975 six Home Start programs became regional training centers for providing assistance to Head Start programs that wanted to adopt a home-based component. Like the Child and Family Resource Program, Home Start emphasized the whole family and not just children; unlike the Child and Family Resource Program, Home Start served only families with Preschool-age children and enrolled them for two years at most. Information about Home Start can be found in Hewett (1978) and Love et al. (1976). The Brookline Early Education Project is a privately developed and funded experimental program in a single service site, Brookline and Boston, Massachusetts. Initially based on the child development philosophy and research of Burton White and the work of White and Watts at the Harvard Preschool Project, the program sought to provide support and parent education to mothers of chil- dren from the prenatal period to age three. The program provided a physical center and educational materials on a lending basis; families were encouraged to use the resource center and to take material home with them. Each family also had a visitor who helped the family acquire educational information and practical experience in playing with and teaching their own children. The visitors also provided assistance to mothers in need for coping with everyday family problems. The program also included comprehensive health screening and treatment referral services. Both low-income and middle-class children were enrolled in the program. Three levels of service were offered, and families were randomly assigned to the levels for purposes of research. (Information about the program and its evaluation can be found in the paper by Levine and Palfrey in this volume.)

OCR for page 203
211 Program Features Three characteristics common to family service program models were selected for discussion in this paper: the mix of support and intervention in the structure and content of family service programs, the use of individual. ized treatment for families, and the program's role as coordinator of services for families in the local community. These features were selected because they illustrate important aspects of family service programs that distinguish them from the more traditional child development intervention programs. These same features also constitute difficult problems for measurement of program outcomes. We noted earlier that the broad goals of comprehensive service programs and their individualized treatment of families create a long list of possible outcomes of potential interest in evaluating the programs. Effects may be anticipated in health and in educational, psycho- logical, and social functioning for children, parents, and other family members. There may also be effects on the ways in which families and individuals in other social groups or institutions interact as well as effects for the services generally available to families in the site of program operation. Possible effects can be organized into three broad groups. The first set of effects are those for individuals within the family household itself. These may include effects on child development, adult development, parent education, parent-child interaction, and intrafamily relationships (e.g., marital or sibling). Child development intervention programs have traditionally concentrated on studying effects on the cognitive and ~- 'Throughout the paper "model" is used to refer to a specific program that was created by federal or private research with a particular set of guidelines and goals (e.g., the Child and Family Resource Program, Home Start, the Brookline Early Education Project). The term "program" refers to the local project(s) implemented under those guidelines according to a specific model. There are ll programs following the Child and Family Resource Program model; 16 programs following the Home Start model. For the privately developed models, one project represented the program model.

OCR for page 203
212 (especially mother) physical development of children and on parent-child ~~ interaction. Another set of effects are those concerning the relationships between families and the informal organizations (social groups, extended families) and formal institutions (schools, agencies) of society. Effects of this sort might be observed in the role of parent involvement in the child's public schooling or the ability of parents to obtain regular income for the family. A third set of effects are those for the service delivery community at the site of program implementa- tion. Since some program models have as their goal improvement of services for families in the community, examples of these effects are improved prenatal care or new links between agencies serving families in the community. The researcher must decide which set of effects is most important and, within each set, which particular The different groups outcomes are of greatest interest. interested in family programs have different views of the priority among these effects, which is suggested by many aspects of a program's philosophy and practice. Each of the three features and the measurement problems associated with them exert influence on the type of effects selected for study and the methods selected for measuring them. In the discussion that follows the potential effects and values of different constituencies, which are mentioned hut not emphasized. are provided primarily as a context ~ , . ~ for the discussion of measurement problems. The Mix of Support and Intervention The family programs described in this paper have been influenced by many social programs and lines of social and psychological research. Inherent in many of the programs and research is a dynamic tension between intervention in the lives of children and families and support of their strengths and capabilities. The two views have been characterized by some researchers as the "deficit" and the "strength" models of family functioning. Both support and intervention are implied by the very broad guidelines that defined the Child and Family Resource Program, Home Start, and other programs funded by the Administration for Children, Youth, and Families. Support was implied by the wording of the Child and

OCR for page 203
213 Family Resource Program guidelines to build on the existing strengths of families and to enhance the total development of children, by working through the family and by offering diverse social and psychological services as needed by each family. Intervention was also implied by the emphasis on educational activities for children and parents and by the guidelines that limited eligibility to low-income families. Theoretically it is possible to see support and intervention as two different, essentially philosophical approaches to the operation of programs--philosophies that influence program structure, content, and the nature of the relationships between staff and families. It can also be argued, however, that the two are inseparable-- that providing support to families can serve as an intervention and that change, the primary goal of intervention, is more likely to occur in an atmosphere of support for parents. Because support and intervention are intertwined in most programs, it is difficult to distinguish them from each other. The list below comprises somewhat stereotypical attributes of the two approaches, as a way of contrasting the hypothetical extreme for each approach: I NTERVENTION Change desired in children, mother Change expected during or immediately after treatment Treatment provided for one period of enrollment, though this may be intense for one to two years Standard treatment defined by professionals outside the family SUPPORT Change in children, parents, and other family members, social institutions Change may be short term but more likely to take place episodically over years Treatment may be intensive at first but intermittent over years, as needed or desired by family Treatment individ- ualized and deter- mined by family and staff; emphasis on parent initative

OCR for page 203
243 across agencies, neighborhoods, and service domains. Single-service agencies usually have some leeway to shift the emphasis of their services, and the comprehensive service program staff are usually well located to spot such needs and bring them to an agency's attention. Effects on State or National Service Programs Sometimes a comprehensive service program can influence the larger state or federal systems that support the networks of local single-service agencies. Such effects might include the following: · Better coordination between services at the state and federal levels (by standardizing eligibility require- ments, application forms, referral networks, and funding procedures). · Increased resources to serve more families, perhaps in more states and counties, or to provide a higher level of services to families now served. ~ Pooling staff or facilities to provide services jointly that existing state or federal programs could not provide alone. These and other related effects tend to happen only occasionally, but they do happen and may have substantial impact. For example, family enrollment procedures became simpler when Head Start managers ruled that AFDC or other public assistance eligibility was sufficient evidence for Head Start eligibility. Also, many states have adopted the Head Start Performance Standards for Title XX Day Care Programs, which opens the way for shared facilities and shared child-recruiting procedures. In yet another example, Head Start and the U.S. Department of Agriculture have agreed that every Head Start center is eligible for food program funds. This decision removes the need to screen every Head Start program for eligibility (thereby saving much time and effort) and frees Head Start funds for other family needs (such as child dental care, for which there are few funds). Comprehensive service programs can sometimes help increase federal and state resources. For example, when a state either lacks a Title XX plan or has a weak plan, local Head Start staff may use their knowledge of family needs and federal law, and state officials may encourage the appropriate state agency to prepare an expanded Title

OCR for page 203
244 XX plan (thereby qualifying the state for increased federal funds). Comprehensive service programs can also help match complementary services. An example of pooling staff resources occurred when national Head Start managers and managers of the Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) program agreed to operate a national experimental program that used local Head Start staff to find and recruit children eligible for these health services. This sharing enabled both programs to serve low-income families better: Head Start by freeing health funds for other family needs, and EPSDT by reaching families that it could not otherwise find without Head Start's established community recruiting network. Evaluation Problems and Approaches Evaluating Effects on Families By measuring the first four family effects in sequence we can determine the success of a comprehensive service program for improving families' circumstances, perhaps its most important goal. But if a Program has little _ ~. ~_ effect on these circumstances, an examination or ones sequence of four effects will also indicate where the program is falling short. Such information is useful to program managers in upgrading deficient local projects. In measuring the four family effects, the sequence must be examined for each family individually. For example, it must be known that EPSDT child health services are not available in a family's community so that a comprehensive services program is not erroneously deemed ineffective because it did not inform the family about EPSDT or enroll it. Likewise, if EPSDT is available but a family does not meet eligibility requirements, the program cannot be held responsible. Or if the family does not need EPSDT services, as determined by their needs assessment, the program would not be expected to affect the family's use of EPSDT. These and other conditions must be examined individually, for each family, to preserve interpretability in an evaluation. Multiple data sources are needed to determine a comprehensive service program's success at each stage. First, program staff should be asked about the avail- ability of community services; for thoroughness, their responses should be independently cross-checked against

OCR for page 203
245 other community information sources to minimize inaccuracies and omissions. Many questions requiring answers are so detailed that only the single-service providers can accurately answer them: How many services do they provide? What kinds? What areas do they serve? What are their eligibility requirements? Are they . capable of serving additional families? Then, following the sequence, we need to determine on a family-by-family basis if each family is aware of the single-service agency, if the family needs the service (according to the individual family's needs assessment), and if the family meets eligibility requirements. Families can be asked directly if Shiv are swarm of the services, but other people will usually have to be consulted to determine whether a family is eligible. One of the first activities the staff of the Child and Family Resource Program and Head Start undertake each year is determining which of their families are eligible for food stamps, EPSDT benefits, and welfare assistance, so the families can be enrolled immediately. A comprehensive service program's apparent effectiveness for enrolling families in services is limited, of course, by the number of eligible but unenrolled families it reaping i n Ph" first place. whether they are eligible, the evaluators have to approach other people for this information. _ ~. _ ~ Since the families themselves seldom know Moreover, eligibility mu ~ es are usually complex and vary by community or state, ~ van Lor many reaera~y evade services. The evalu- ators themselves may have to perform calculations of ~ _ _ c _ _ ~ . ~, . . income eligibility for each family and match them ~ . . ~ ~ . . . Anus vlauallv to the reauirem~ntr: for each .~"r`ri"" hm determine the potential new enrollments possible. Increases in program enrollment can be measured without matching the effects for individual families, but the maximum percentage enrollable cannot; it is a vital policy statistic because it reveals the overall need for a service and the current degree of success reached in meeting the need. The services that families actually use can often be determined by asking families to name them. However it is usually necessary to determine the precise name of a service program, its funding source, or its legislative authority--details families seldom know. This information must typically be laboriously gathered by directly contacting the sources of service cited by parents, although at times linkage program staff know. Rough estimates of the extent to which families use a service

OCR for page 203
246 can often be obtained directly from them, but precise levels of use are difficult to determine. Improved family circumstances can sometimes be measured by equating them to service use (as, for example, when food stamps can be given a dollar value and counted as an increase in family income), but usually indirect means are needed to determine the improvement. When a family change is drastic, such as finding a substantial job for a previously unemployed father, little risk is faced in inferring improved family circumstances. In searching for subtler effects, the problems grow more difficult. The effects of preschool on children are usually less clear, although many people seem to agree that favorable results from selected child tests often imply an improve- ment in family circumstances. Families can ultimately be asked directly if they feel their circumstances have improved or if they experience less family stress. We can always assume (after the fashion of pollsters) that if families say things are better (or worse) then they are better (or worse), even though appearances may suggest otherwise. In areas such as preventive health services, improved status can be determined only by using compli- cated medical procedures that are simply not feasible in most evaluations. Thus, apart from a few notable exceP- tions, the ultimate goal of improved family circumstances is the most difficult aspect of program success to assess. The effect of increased family consumer awareness and advocacy usually results from rather obvious involvement of the families and can frequently be determined by direct questions to the families or program staff. Typical questions ask about parent participation on policy councils, school boards, task forces, political organizations, or ad boa groups. Evaluating Effects of Local, State, and National Programs Few individual families are aware of changes in the service providers over time (because, for example, a family usually applies only once for a service and has no way of knowing previous or subsequent application procedures). Long-term staff employed by the linkage program are often aware of changes and trends in the providers they work with. Since they typically go through service application procedures with family after family, linkage program staff can frequently identify changes

OCR for page 203
247 that are subtle--all the more so if they actively advocated the changes. Most often, however, changes in the single-service agencies can only be identified with any certainty by direct longitudinal review of the agencies themselves. This may involve a study at the community level or it may involve a study of an entire national service delivery network. Because the designs required for this type of evaluation are different from ones for assessing family impacts, it often becomes difficult to carry them out concurrently with evaluations of family effects. Approximate descriptions of changes in service providers can be obtained by gathering anecdotes from appropriate community residents about the kinds of changes they remember seeing. This kind of study need not be longitudinal, thereby considerably reducing the cost. Lazar (1970) conducted such a study on the community impact of Head Start. Changes in service programs at the state and national levels can be identified through changes in regulations, funding levels, legislation, reorganization, staff time allocation, and so on; these can be "measured" by examin- ing official program documents; observing operating practices; and interviewing federal, regional, or state policy makers and managers. Once measurements are taken at the state or federal level there are so many prevailing influences that it becomes almost impossible to attribute any changes found back to a particular comprehensive service program. This does not diminish the policy benefits of straightforwardly describing the changes, however, even without scientific support for the causes of the changes. Evaluators should also be alert for unintended conse- quences of comprehensive service program activities. For example, outreach and referral activities may be so effec- tive that the single-service providers are inundated beyond their capacities, eroding service quality and exceeding available funds or staff capabilities. This may create new problems at the state and federal levels as well as at the local level. CONCLUSION Our discussion has focused on two of the comprehensive family service programs developed as demonstrations under the auspices of Head Start and the Administration for

OCR for page 203
248 Children, Youth, and Families. Three essential features of these programs were described: the mix of support and intervention, the emphasis on individualized program treatment, and the role of the program in coordinating community services. Each of the features was shown to have particular implications for the kinds of evaluation questions posed and the methods used to answer them. The mix of support and intervention was seen as a natural result of the programs' broad goals and of their acceptance of the strengths and different values of families and communities. Individualized treatment also was an expression of the broad goals. Treatment was varied by family along many dimensions, such as the types of activities and services emphasized, the role of parents in the program, and the length and intensity of participation expected. Third, the programs' role as ~brokers" of services and advocates for families implies both support or change for families as consumers of service and change in the services or policies concerning families at the local, state, and even federal levels. Thus, the programs are designed to support and bolster the development of the children, the functioning within the family or household unit, and the ways in which families and social institutions interact. This broad concept of program action necessitates a broadening of methods and perspectives in program evaluation. One area of evaluation to be expanded is the use of process studies to describe and characterize what actually happens in the program between families and staff, among staff, and between staff and community institutions. Different types of process studies can and should be used to identify and clarify goals and constituencies for family programs, to describe what the programs do, and to help relate what goes on in the programs (the process and/or treatment) to what results from them (the out- comes). Process studies also help raise questions and generate hypotheses about unanticipated outcomes and also look at questions about the relationships among different family and child programs at a federal administrative and policy level. We make several recommendations concerning the study of outcomes in order to address the problems posed by individualized treatments for families in several domains of family life over time. One is to retain the quasi- experimental designs currently in use but to supplement them with additional, smaller studies or analyses to enhance their interpretability. Exploration of methods

OCR for page 203
249 used in clinical work or educational psychology but less commonly used in program evaluation is urged in order to give a whole picture of outcomes in several domains for families or groups of families over time. Increased use of atheoretical indicators, ethnography, and descriptive analyses for synthesizing different types of data also is discussed. Overall, while the programs and the methodological problems associated with evaluating them are complex, the endeavor is not overwhelming. It requires careful speci- fication and rating of the questions that are to be answered at any one time for any particular program. It also requires the recognition that many questions concerning the evaluation of family support programs are ultimately questions of values and social policy. Perhaps most important, the problems, as we have described them, derive from the richness and complexity of the programs, their goals, and practices and of the families themselves. It would be a mistake for evaluators merely to complain about such complexity or to adopt new methods or perspectives that would eliminate important variety for the sake of precision and manageability. At this point in the development of family programs and of research about families, it is important for evaluators to try to capture this richness and variety in different ways, using multiple methods and perspective. Under- standing outcomes at the individual, family, and social levels demands that we consider outcome to be (like families and programs) multidimensional, multiply determined, occurring within a particular context, and changing over tome. REFERENCES Bayley, N. (1969) Bayley Scales of Infant Development. New York: Psychological Corporation. Bell, R. Q., and Harper, L. V. (1977) Child Effects on Adults. Lincoln, Neb.: University of Nebraska Press. Belle, D., et al, (1980) Lives in Stress: A Context for Depression. Cambridge, Mass.: Harvard University Press. Bronfenbrenner, U. (1977) Toward an experimental ecology of human development. American Psychologist 32:513-531.

OCR for page 203
250 Burr, W. R., Hill, R., Nye, F. I., and Reiss, I. L., eds. (1979) Contemporary Theories About the Family. Vol. I and II. New York: The Free Press. Campbell, D. T., and Boruch, Re (1975) Making the case for randomized assignment to treatment by considering the alternatives: six ways in which quasi-experimental evaluations in compensatory education tend to underestimate effects. In C. A. Bennett and A. A. Lumsdaine, eds., Evaluation and - Experiment. New York: Academic Press Cook, T. D., and Campbell, D. T. (1979) Quasi-Experimentation. Rand-McNally. Duvall, E. (1977) Family Development. Lippincott Co. . Chicago, Ill.: Philadelphia, Pa.: J. B. General Accounting Office (1979) Early Childhood and Family Development Pro rams Improve nualitv of Life for Low g ~ ~ Income Families. Washington, D.C.: General Accounting Office. February. Goodson, B. D., and Hess, R. (1978) The effects of parent training programs on child performance and behavior. In B. Brown, ea., Found: Lonq-Term Gains From Early Intervention. Boulder, Colo.: Westview Press. Goodwin, W. L., and Driscoll, L. A. (1980) Handbook for Measurement and Evaluation in l Early Childhood Education. San Francisco, Calif.: Jossey-Bass, Inc., Publishers. Gurman, A. S., and Kniskern, D. P. (1978) Research on marital and family therapy: progress, perspective, and prospect. In S. L. Garfield and A. E. Bergin, eds., Handbook of Psychotherapy and Behavior Change. 2nd ed. New York: John Wiley & Sons, Inc. 15th Anniversary Head Start Committee (1980) Head Start in the 1980's: Review and Recommendations. Report by the 15th Anniversary Head Start Committee. Edward Zigler, Chairman. September. Herson, D., and Barlow, R. (1976) Single Case Experimental Design. New York: Pergamon Press.

OCR for page 203
251 Hertz, T. W (1976) The Ecology of Child Development: Research Issues. The Child/The FamilY/Environment/ Services. A Report for Planners of Research and Development. Prepared for the Interagency Panel on Early Childhood Research and Develop- ment. Washington, D.C.: The George Washington University. December. Hewett, K. H. (1978) Partners With Parents: The Home Start Experience With Children and Their Families. Office of Human Development Services. DREW Publication No. 78-31106. Washington, D.C.: U.S. Department of Health, Education, and Welfare. Hewett, K., Connell, D., Affholter, D., and Weiss, S. (1979) Evaluation of the Child and Family Resource Program: Volume I: Design Report Draft. AAI No. 78-111. Cambridge, Mass.: Abt Associates, Inc. Hill, R. (1971) Strengths of Black Families. Washington, D.C.: National Urban League. Hill, R., and Mattessich, P. (1977) Reconstruction of Family Development Theories: A Progress Report. Paper presented at the meeting of the National Council on Family Relations. San Diego, October. Jacob, T. (1975) Family interaction in disturbed and normal families: a methodological and substantive review. Psychological Bulletin 82:33-65. Johnson, L., Nauta, M. J., and Hewett, K. D. (1980) Evaluation of the Child and Family Resource Program: Phase III Program Study Report. AAI No. 80-91. Cambridge, Mass.: Abt Associates, Inc. Johnson, L. (1981) Methodology for the Ethnographic Study: The Child and Family Resource Program Evaluation. Cambridge, Mass.: Abt Associates, Inc. Kagan, J., Kearsley, R. B., and Zelazo, P. R. (with the assistance of C. Minton) (1978) Infancy: Its Place in Human Development. Cambridge, Mass.: Harvard University Press. Kamerman, S. B., and Kahn, A. J. (1978) Family Policy: Government and Families in

OCR for page 203
252 Fourteen Countries. University Press. New York: Columbia Kazdin, A. (1977) Methodological issues in single case analysis. Journal of Clinical and Consulting Psychology 34. Kessen, W. (1979) The American child and other cultural interven- tions. American Psychologist 34:915-920. Kiresuk, T. J., and Sherman, K. E. (1968) Goal attainment scaling: a general method for evaluating community mental health programs. Community Mental Health Journal 4(6):443-453. Kiresuk, T. J., Calsyn, R. J., and Davidson, W. S. (1978) A critique of goal attainment scaling. In T. Cook, ea., Evaluation Studies Review Annual 3:700-715. Lazar, I., Hubbell, V. R., Murray, H., Roche, M., and Royce, J. (1977) Persistence of Presabool Effects: Final Report. Grant No. 18-76-07843, Administration for Children, Youth, and Families. Washington, D.C.: U.S. Department of Health, Education, and Welfare. Office of Human Development Services. Love, J. M., Nauta, M. J., Coelen, C. G., Hewett, K. D., and Ruopp, R. R. (1976) Home Start Evaluation Study. High/Scope Educational Research Foundation and Abt Associates, Inc. Cambridge, Mass.: Abt Associates, Inc. Nauta, M., and Johnson, L. (1981) Evaluation of the Child and Family Resource Program: Phase III Research Report. Cambridge, Mass.: Abt Associates, Inc. Nauta, M. (1981) Evaluation of the Child and Family Resource Program: Phase III Executive Summary. Cambridge, Mass.: Abt Associates, Inc. Newbrough, J. R., Dokecki, P. R., Dunlop, K. H., Hogge, J. H., and Simpkins, C. G. (1978) Families and FamilY-Institution Transactions in Child Development. Final report. Contract No. 105-77-1045, Administration for Children, Youth, and Families, Office of Human Development Services. Washington, D.C.: U.S Department of Health, Education, and Welfare.

OCR for page 203
253 Nobles, W. W. (1976) A Formulative and Empirical Study of Black Families: Final Report. San Francisco, Calif.: Westside Community Mental Health Center, Inc. Rossi, P. H., Freeman, H. E., and Wright, S. R. (1979) Evaluation: A Systematic Approach. Beverly Hills, Calif.: Sage Publications. Stake, R. (1978) The case study method in social inquiry. Educational Researcher 7(2):5-8. Travers, J., Johnson, L., Lynnell, J., and Irwin, N. (1981) An Ethnographic Perspective on the Child and Family Resource Program . Abt Associates, Inc. Weiss, C. Cambridge, Mass.: (1972) Evaluation Research: Methods for Assessing Program Effectiveness. Englewood Cliffs, N.J.: Prentice-Hall. Weiss, R. S., and Rein, M. (1969) The evaluation of broad-aim programs: a cautionary case and a moral. Annals of the American Academy of Political and Social Science 385:133-142. Zigler, E., and Trickett, P. (1978) IQ, social competence and evaluation of early childhood intervention programs. American Psychologist 1:789-798.