Quality of Care, Organizational Variables, and Nurse Staffing
Joyce A. Verran, Ph.D., R.N., F.A.A.N.
Introduction: The State Of The Science
Nurses in acute health care settings are convinced that there is a link between organization variables, including the numbers and types of nursing staff available to provide care, and the quality of nursing care that patients receive. It is plausible to assume that nursing organizational variables interact with clinical treatments to influence patient care outcomes. However, the data to support such an assumption are generally unavailable.
Most health outcomes research to date has focused on treatment effects during a single episode of care and the specific individual patient outcomes that can be related directly to that treatment. An examination of the effect of organizational variables on outcomes requires a different kind of outcome measure that can be attributed to all patients without regard to medical diagnosis. Research that includes these broad measures of outcome (e.g., mortality) invariably focuses on broad structural and organizational variables such as ownership and reimbursement type. Only tangentially have most studies included nursing organizational variables (e.g., number of nursing staff, number of registered nurses, nursing practice pattern). Research that does include nursing level organizational variables most likely have insufficient sample sizes to have adequate statistical power to detect difference, or use inappropriate level of analysis.
Even though empirical data to support the link among nursing organizational
Dr. Verran is a professor in the College of Nursing, University of Arizona, Tucson.
variables and patient outcomes are limited, such links continue to be assumed in the profession and its literature. The American Nurses Association (ANA, 1995) nursing care report card for the acute care setting proposes six broad areas of outcomes to be examined. These areas are mortality rate, length of stay, adverse incidents, complications, patient and family satisfaction with nursing care, and patient adherence to discharge plan. The ANA also attests to the lack of conclusive research supporting the inclusion of these outcomes that are reflective of quality nursing care.
The purpose of this manuscript is to summarize the state of the science on the relationship among nursing organizational variables and patient outcomes. Because a number of recent research projects have included broad patient outcomes as part of their projects, several approaches to the examination of the science are used. First, information is presented from a special conference convened by the American Academy of Nursing. Participants at this conference included researchers with current projects that include elements of nursing care and patient outcomes. Much of the information from these studies is yet to be published. Second, published research related to selected quality-of-care variables is reviewed. In addition, methodological issues related to the research in this area is discussed and gaps in the research base are identified.
American Academy Of Nursing Invitational Conference
In October of 1994, the American Academy of Nursing convened a special conference for researchers who were involved with projects examining the relationship among organizational variables and quality of care. The purposes for this conference were to (1) define linkages between organizational variables, staffing variables, and patient care outcomes; and (2) discuss the current state of knowledge, identify promising trends in preliminary analysis of ongoing projects, and define the problematic issues in this type of research.
Those individuals invited to participate in the conference represented six recently completed or ongoing studies examining patient outcomes, experts in the area of research methodology, and representatives of funding agencies and national foundations or organizations interested in quality-of-care measurement. A more detailed review of the six studies presented in this conference follows.
Current and Recently Completed Research Projects
Six research projects were represented at the special conference. Three of these projects were combined demonstration and research studies, while three were descriptive research that examined existing nursing care delivery and the impact of this delivery on a variety of outcomes.
The three demonstration projects began in 1988–1989 as a response to the nursing shortage. The purpose of all three projects was to create and evaluate the effect of innovative nursing practice models in hospitals with the intention of improving retention. All three studies, which were unit-based models, also examined the costs of care delivery and the effect of the delivery systems on a variety of client outcomes.
Study 1: Baltimore The first project took place at Johns Hopkins Hospital in Baltimore (Gordon et al., 1989). Twenty-four nursing units eventually would implement the "Professional Practice Model," which is a unit-level self-management model including participant decision making, use of primary nursing, peer review, and a salaried status for registered nurse (RN) staff. For the purpose of the research component of the project, eight practice model units and eight comparison units were included in the analysis, which examined three sets of outcomes. Nursing outcomes included work satisfaction and retention; cost outcomes included personnel expenditures, recruitment costs, and orientation costs; patient outcomes included satisfaction, health status, and perceived functional status at discharge and two weeks after discharge. In addition, information was gathered on unanticipated health plan usage. Results of this study have been reported in two articles (Weisman et al., 1993; Wong et al., 1993) and two others are in press and expected to be published in 1995. A full description of the practice model has also been published (Rose and DiPasquale, 1990).
Study 2: Rochester The second project took place at the University of Rochester (Ingersoll et al., 1988). In this project, an "Enhanced Professional Practice Model" consisting of five interdependent conceptual elements was introduced and tested in five experimental units and contrasted with five comparable units. Urban, community, and rural settings were observed. Model components included: control over practice; continuing education responsive to staff nurse need; continuity of care delivery; collaborative practice; and professional compensation reflective of education and experience. The goal of the Rochester project was to invest nurse managers and staff with increased control over practice and decision making authority to produce higher levels of work satisfaction, decreased staff nurse turnover, improved patient outcomes, and increased cost effectiveness of care delivery. Specific nurse outcomes measured were job satisfaction, unscheduled absences, peer group relationships, professional identity, intent to leave, autonomy, leadership responsiveness, perceived RN workload, and advancement opportunity. Cost data included both direct and indirect nursing costs. Patient outcome information included satisfaction, morbidity, mortality, length of stay, untoward hospital incidents, and infections. Three pa-
pers discussing the practice model for this research and methodological problems have been published (Ingersoll et al., 1990, 1991, 1993).
Study 3: Arizona The third demonstration project was located at the University of Arizona (Verran et al., 1988). The practice model for this research was entitled the "Differentiated Group Professional Practice" (DGPP) Model, which was implemented in 4 demonstration hospitals for a total of 10 nursing units. In September 1991, one rural hospital dropped from the project leaving a total of nine units (three intensive care units, two telemetry units, and four general medical-surgical units) in one rural, one community, and one tertiary hospital. For the first 2 years of the project 10 units in 3 other hospitals were included for comparison purposes. The initial plan for the project included the elimination of comparison hospital data collection after 2 years. Since three of these units were in the same hospital as three demonstration units, it was believed that diffusion of innovation could only be controlled for this 2-year period. The original data analysis plan included procedures to project comparison unit data to the last two data collection times. Components of the DGPP Model include group governance (participative unit management, shared decision making by staff bylaws, peer review and professional salary structure), differentiated care delivery (differentiated RN practice, use of nurse extenders, primary case management), and shared values in a culture of excellence (quality of care, support for intrapreneurship, internal and external recognition). The research goals of the Arizona project were to test the effectiveness of the unit-based DGPP Model on (1) professional practice as measured by group cohesion, control over practice, autonomy, and organization commitment, (2) nurse satisfaction as measured by job satisfaction subscales of professional status, task requirements, organizational policies, pay, interaction with nurses, interaction with physicians, and autonomy, (3) nurse resources as measured by rates of turnover, vacancy, stability, activity, positive activity, and negative activity, (4) quality outcomes as measured by rates of intravenous fluid errors, skin injury, patient falls, nosocomial infections, nursing documentation, and (5) fiscal outcomes as measured by operating costs, personnel costs, RN costs, RN agency costs, RN overtime costs, and RN absence costs. An article describing the innovative practice model on this project has been published (Milton et al., 1992) and other methodological and measurement papers are also in print (Milton et al., 1990, 1995; Verran et al., 1995). In addition, a partial report of data for one hospital has been published (Verran et al., 1994).
The three descriptive studies vary considerably from each other in their initial impetus and their approach. The first of these examines the environment in which nurses practice in the critical care setting and is like the previous intervention studies, in that it was an attempt to examine factors that would influence
nurse retention. The second study also has an organizational base but is attempting to identify factors associated with patient-centered care. The third study is not nursing-unit-based and is directed at the quality of care delivered by nurses to patients with the specific condition of acquired immunodeficiency syndrome (AIDS).
Study 4: Seattle The first descriptive study took place around Seattle in 17 hospitals and involved 25 critical care units. The research, entitled "Critical Care Nursing Systems, Retention and Patient Outcomes" (Mitchell et al., 1991), was a correlational study to examine the predictors of patient care outcomes related to the organization vectors in critical care nursing. The study was built on a demonstration project from the American Association of Critical Care Nurses (Mitchell et al., 1989). The underlying framework of the current research is taken from industrial organizations and proposes that high outcomes will result when there is a match between the pattern of structures and processes and the work being done. For nursing, technology includes elements of a professional practice model as described in some of the previous intervention research projects. The model for critical care indicates that the more the actual practice environment approaches the ideal, then the greater will be the organizational and patient care outcomes. Specific hypotheses indicated that the closer a unit was to the idealized score on variables such as expertise, discretion, and standardization, the higher would be the organizational variables such as job satisfaction, propensity to stay on the job, retention of nursing expertise, and documented quality of care, and the lower would be the severity-adjusted mortality rate and length of stay.
Study 5: New York The second descriptive study took place in 17 hospitals in New York with 116 to 119 adult, nonpsychiatric, nonobstetric nursing units. The project, entitled "Improving Patient Centered Care Through Initiatives in Nursing" (Minnick, 1991) examined the variables that contribute to reports of patient-centered care in nursing. A set of hypotheses were developed that examined (1) the effect of labor and capital on outcomes; (2) whether satisfied (contented) employees provide patient-centered care; and, (3) the impact of patient characteristics on patient-centered care. Data have been analyzed from this research and articles are in review for publication. Findings were not shared at the invitational conference, however methodological issues from the study are included. Three articles from this study have been accepted for publication with anticipated publication dates in 1995–1996 (Minnick et al., in press[a,b,c]).
Study 6: San Francisco The final research study presented at the conference was a prospective research project entitled "Quality of Nursing Care for People with AIDS" (Holzemer and Henry, 1989). This descriptive study is examining the quality of care for clients with AIDS across the trajectory of health care from the inpatient hospital setting, home care, and skilled nursing facilities. Data are
collected from nurses about specific patient problems in nursing care activities and from the patients about their perceptions of their problems. Extensive audits of care plans, progress records, and shift reports are also conducted. Unlike the other studies presented at the conference, this research proceeds from a microlevel to examine the relationship between patient problems, nursing care activities, and outcomes. Structural or organizational characteristics are not included. Currently data have been entered on approximately 300 subjects representing over 15,000 patient problems. A number of microlevel findings related to patient problems and nursing interventions are available from this research. In addition, methodological issues related to care planning and measurement have been identified. Published articles from the research include Janson-Bjerklie and colleagues (1992), Holzemer and colleagues (1993), and Henry and coworkers (1994a,b).
Variables Addressed in Current Research
This section will summarize the independent variables and the outcome variables examined in the current research projects. Outcome variables may be organized under three classifications: staff outcomes, organizational outcomes, and client outcomes. None of the studies described included a direct measurement of staffing mix and staff numbers as one of the independent or outcome variables.
The independent variables for the three demonstration projects and two of the descriptive studies involved the environment of the hospital unit within which nurses practice. The intent of demonstration models was to increase the degree of control over nursing practice through a variety of mechanisms and to establish professional practices consistent with previous research.
Although the models differed on specific components, most included some form of self-governance to increase control and could be said to be based upon the characteristics associated with magnet hospitals (McClure et al., 1983; Kramer and Schmalenberg, 1988a,b; Kramer, 1990). This environment includes the following characteristics: (1) high status of nurses within the organization; (2) autonomy in the area of clinical decisions related to nursing practice; (3) decentralized decision making at the unit level; (4) assignment patterns that allow continuity and accountability; and (5) an organizational culture that values excellence and provides rewards commensurate with excellent practice. For the purposes of this paper, the environment just described has been labeled a professional practice environment.
In two descriptive studies, the existing environment of the nursing unit was examined as the independent variable. Specific aspects of the environment stud-
ied included some of the variables associated with professional practice. Individual patient problems were the independent variables identified in the final descriptive study. These problems were identified by nurses and patients and have been categorized with a variety of classification schemes, primarily nursing diagnosis as developed by the North American Nursing Diagnosis Association. One finding of particular interest from this last study is the lack of agreement on the identified problems that was found between patients and nurses. This suggests that patients' perceptions of their problems need to be captured to enhance care and provide data for the analysis of outcomes. The study also confirmed the impression that the chart does not capture well the data required to measure the quality of nursing care.
Staff outcomes were viewed in most of the studies as intervening variables influenced by the independent variable and influencing other outcomes. The primary staff variable included in four of the studies was job satisfaction. In addition, some measure of autonomy was also included. In two of these studies this concept was divided into a measure of control over practice and control over job. Various other measures that represent some aspect of a professional practice environment were also included in one or more of the studies. These variables included organizational commitment, work group cohesion, and work group culture.
In the microlevel research project with patients with AIDS, the staff outcome was nursing care activities. These activities were thought to be influenced by the independent variable, patient problems, and to influence client outcomes.
Two types of organizational outcomes were included in the macrolevel research projects. The first included various retention measures of actual turnover rates or intent-to-leave measures. The second type of organizational variable involved costs of delivering care on a nursing unit. Cost measures were primarily related to various aspects of personnel costs. In none of the studies were charges to the patient or third-party payer included as variables. All costs were those associated with the organization. In the San Francisco microlevel study, work is currently in progress to link quality and cost data across the continuum of care.
Organizational variables were considered to be both intervening and final outcome variables in the studies and were influenced by the practice environment of the nursing unit. Retention outcomes were primarily considered to be intervening variables affecting both quality and cost of service. Organizational costs were considered final outcomes.
For all the studies described, client outcomes as indicators of quality were the final variable examined. These outcomes were thought to be influenced by a combination of the independent variable, staff outcomes, and organizational outcomes related to retention.
Two of the research projects, the Rochester demonstration study and the Seattle critical care study, included severity-adjusted mortality rates as an outcome. The San Francisco microlevel study also includes the measurement of mortality, although results on this variable are not available for report.
Other common client outcome measures included some form of client satisfaction for two of the demonstration and one of the descriptive studies. Two of the intervention studies used negative indicators of client outcomes in the form of untoward hospital incidents and nosocomial infection.
A variety of other client outcome measures were used for individual studies, including unanticipated health care service utilization, length of stay, health status and functional status. In the San Francisco study, the investigators developed a 10-item, 3 factor (self-care, ambulation, and psychological distress) instrument that evidences excellent psychometric properties including predictive validity of mortality at 3 and 6 months (Holzemer et al., 1993).
Findings Related to the Independent Variable
Studies that implemented models that were designed to create a professional practice environment on nursing units had variable success with that implementation. These projects cited the need for strong administrative support and a committed cohort of staff in order for full implementation to take place. Variable levels of implementation led to a methodological recommendation that will not be detailed further in this paper. That recommendation involved the need for the systematic evaluation of the level of professional practice model implementation on the nursing unit in order to assess strength of the intervention. Both the Rochester and Arizona studies developed scales to assess the level of model implementation.
Findings Related to Staff and Organizational Outcomes
For the studies examining the effect of a professional practice environment, findings consistently confirm an increase in job satisfaction when this environment is in place to a high degree. In addition, measures such as autonomy or control over practice tend to increase. One of the projects (Rochester) found limited differences between their comparison units and demonstration units in terms of job satisfaction, however the evidence from this research, in general, tends to support previous studies' findings that a professional practice environ-
ment will increase job satisfaction. In terms of the autonomy variable, for those studies that included both a measure of control over practice and a measure of control over job, control over practice had a greater predictive value on job satisfaction than did control over aspects of the job.
The staff variable for the microlevel study in San Francisco involved the identification of nursing care activities. Over 80 percent of the activities identified could be classified into the 2 categories of monitoring the patient's condition and medication administration. The nursing intervention classification system from the University of Iowa has been adopted for activity classification. For some identified patient problems, no nursing care activities were identified. In general, nurses also did not document individualized problems or interventions for their patients. Standardized care plans and nursing activity lists are usually organized by body systems. Twenty-two percent of patients studied had no care plan under any system (manually generated, computer supported, or standardized). There were no differences on selected outcome variables for patients with or without a care plan.
The results from the macrolevel research also support the belief that professional practice environments will increase retention of nurses on the unit and in the hospital. All of the studies that examined the cost of care delivery found that the professional practice models were cost neutral. In other words, there were no significant increases or decreases in costs under these practice frameworks.
Findings Related to Client Outcomes
In general, there were no relationships among any of the independent variables, staff or organizational outcomes, and the client outcomes examined in the studies reported at the invitational conference. In neither of the research projects that examined mortality were any differences noted. In none of the demonstration or descriptive macrolevel studies were there positive differences in other quality indicators across time or comparison units. The microlevel San Francisco study did not report the examination of nursing care on client outcomes.
Discussion of Findings
The findings of the current macrolevel research projects are consistent with the literature in terms of staff outcomes and organizational outcomes, and support the contention that a professional practice environment will improve these variables. However, there were no significant findings associated with quality measures. There are a variety of methodological issues that help to explain these results, that will be addressed at the end of this paper.
A review of published studies in the area of client outcomes and organizational variables will be presented next. Only research published in the last 10 years was examined. Earlier research tended to examine, primarily, the effect of new nursing care delivery models such as primary vs. team care. In the majority of these studies staff outcomes, including the process of care, were considered to be of primary significance and client outcomes were seldom addressed. More recent studies, like those at the invitational conference, were conducted due to the nursing shortage and attempted to increase retention of nurses. Relatively few national studies with large samples have been reported.
As with treatment effectiveness research, there exists a body of literature related to specific nursing interventions and the effect of these interventions on individual client outcomes. Most of these studies are directed at patients with specific diagnoses and will not be reviewed in this paper. However, five meta-analyses of many of these studies have been conducted and may be of interest (Mumford et al., 1982; Devine and Cook, 1983; Smith and Naftel, 1984; Hathaway, 1986; Heater et al., 1988).
The following sections report on the results of published research that is related to more generalized client outcomes. The primary outcomes that have been examined in the literature are mortality, nosocomial complications, adverse incidents, service utilization, and patient satisfaction. Although there is a beginning attempt to examine health status on discharge and postdischarge, this research is not summarized in this paper for several reasons. First, there are few studies that include nursing variables as predictors to changes in health status. Second, health status is conceptualized in a variety of different ways, from the very abstract to very specific measures related to specific diagnoses. Third, even when the same variable is conceptualized (e.g., functional status) it is measured in such a variety of ways that comparisons are difficult.
The literature on the variables influencing severity-adjusted mortality rates is extensive. When considering qualities of nursing services that influence this variable, two areas are pertinent. Studies have consistently found that the proportion of RNs on the nursing staff or the total number of RNs will have a positive influence on mortality rates. Prescott (1993) has done a complete review of this research base and it will not be detailed further.
More pertinent to the research reported at the invitational conference is the second aspect of nursing that has been shown to have some effect on mortality. These studies have taken place in the critical care setting and indicate that the level of interdisciplinary collaboration has a positive influence on mortality. This finding was originally reported by Knaus and colleagues (1986) and was sup-
ported by the original Critical Care Nurses Association Demonstration Project (Mitchell et al., 1989). More recent findings by the same team as the original research (Zimmerman et al., 1993, 1994) failed to support the original result. The Seattle critical care study described at the invitational conference also failed to find a relationship between units that were closer to the ideal level of practice and adjusted mortality rates.
In an extensive investigation, Aiken and colleagues (1994) reported a strong relationship between the nursing organization found in magnet hospitals and lower adjusted Medicare mortality rates. The hypothesis was also tested that decreased mortality rates were a result of staffing mix, which is traditionally richer in magnet hospitals. No evidence was found to support the contention that skill mix or the proportion of RNs was the key variable in affecting mortality. Instead the authors conclude that the mortality decrease (5 deaths per 1,000 Medicare discharges) stems from ''… the greater status, autonomy and control afforded nurses in the magnet hospitals, and their resulting impact on nurses' behaviors on behalf of patients.—i.e., this is not simply an issue of the number of nurses, or their mix of credentials." (Aiken et al., 1994, p. 783). This finding was not supported in the studies that examined mortality and were reported at the invitational conference. In these cases, there were no difference in mortality rates on the high professional practice units and nonprofessional practice units.
Flood and Diers (1988) examined two general medical units with differing staffing levels to determine the effect of staffing on patient complications. The most frequently occurring complications were infections, heart conditions, and gastrointestinal disorders. They found that the mean number of complications per patient was higher on the "short-staffed" unit than on the unit with adequate staffing. Both generalized infections and urinary tract infections evidenced the greatest difference between units, with rates being almost double those of other complications. For this research, staffing was determined by an index of required staff hours (based on patient acuity) to actual staff hours. Data confirmed that one unit in the study was consistently below the staffing level required while the other was at or above that level. No data were found that indicated that patients were different in terms of age, gender, and diagnosis-related group (DRG) between the two units, although some evidence was found that acuity on the short-staffed unit was slightly higher. This higher rate could have been due to either the increased level of complications or simply the type of patient.
More recently, Taunton and colleagues (1994) reported on research that examined the effect of three organizational variables (absenteeism, unit separation, and work load) on nosocomial infections. This study took place in 4 large Midwestern acute care urban hospitals for a total of 65 patient care units in the sample. Specific units within the sample included 15 critical care units, 5 telem-
etry units, 22 medical-surgical units, 6 pediatric units, 6 obstetric-gynecological units, 4 long-term care/rehabilitation units, and 7 other (type not specified) units. Findings supported a relationship between patient infections and staff RN absenteeism. No other organizational variable evidenced a significant relationship with the outcome variable.
Both of these studies indicate that nurse staffing has some effect on the incidence of nosocomial infection rates. These increased rates were also associated with increased length of hospital stay in the Flood and Diers (1988) research. Disruption in the continuity of care due to absences and inadequate staffing were cited as reasons for the increased incidents of infections.
Other research has examined the effect of practice models on complications. Mitchell and colleagues (1989) found that complications related to infections, immobility, and fluid balance represented nonresolution of problems on admission to the intensive care unit rather than new problems. Brett and Tonges (1990) in a one-unit pilot evaluation of the ProACTTM Model at the Robert Wood Johnson University Hospital found no increase in nosocomial infections despite the planned decrease in the number of RNs.
Two of the intervention studies reported at the invitational conference also included nosocomial infections as part of their research. Neither study found a significant relationship between infection rates and any other included study variable.
Adverse incidents that occur during hospitalization include errors in medication delivery (wrong patient, wrong drug, wrong dose, wrong route, wrong time), patient falls, treatment errors, and skin injury or breakdown. Again, two of the demonstration projects included these items in their research model with no significant findings. The study by Taunton and colleagues (1994) cited earlier also examined the effect of organizational variables on patient falls and medication errors. No significant associations were discovered.
A comprehensive investigation by Wan and Shukla (1987) examined the relationship of contextual and organizational variables with the quality of nursing care. Contextual variables were considered to be attributes of the hospital and region that are beyond control of the hospital. Organizational variables include structural and design variables. Only the design variables are amenable to change. These variables include the nursing care delivery model, staff skill mix, and staffing levels. The patient incidents included in the study were rates of medication errors, patient falls, patient injuries, and testing or treatment errors. Forty-five community acute care hospitals were included in the study. Results indicate that the independent variables did not account for a large portion of the variation in incident rates. Nursing skill mix, nursing model, and nursing resource consumption were not significantly related to any of the dependent variables. The
authors suggest these findings support an earlier study that found that the most significant nursing variable affecting the quality of nursing care was nursing competence. This research1 was unavailable for review. A further explanation may involve the interrelatedness of patient factors, hospital support systems, and nursing variables. Even though a support system index was included in this research, the interaction effect with nursing variables was not examined. Of significance, however, is the fact that patient age had the greatest impact on falls and that age, along with acuity, were the two significant variables in the regression of patient injury on contextual and organizational variables. This suggests that patient characteristics are the most influencing factor for the occurrence of adverse incidents and any research examining these factors should control for the risk of injury.
Currently, there is no evidence to support the belief that nursing variables are directly associated with adverse hospital incidents. There is some suggestion that nursing competence has an effect on quality.
Primarily, the research on the relationship of nursing to service utilization has concentrated on types of care delivery systems rather than staffing variables. An exception to this is the article by Flood and Diers (1988) that found that although total length of stay adjusted by DRG was similar between the two study units, two specific DRGs showed differences in length of stay. Patients within these DRGs (GI hemorrhage  and CVA ) had longer length of stays on the short-staffed unit and, in addition, developed more complications during their hospital stay.
Brooten and colleagues (1986) designed a clinical trial to examine the effect of nurse specialist care on early discharge of low-birthweight infants on selected patient outcomes. The experimental intervention included early discharge of low-birthweight infants meeting specific criteria, to be followed by master's prepared nurse specialists for 18 months after hospitalization. Random assignment resulted in 39 infants in the experimental group and 40 infants in the control group. Groups were equivalent on a number of demographic family variables and infant treatment variables. Findings showed that the experimental group was discharged a mean of 11.2 days earlier that the controls and that there were no differences in rehospitalizations, acute-care visits, failure to thrive, child abuse, foster placement, or developmental quotient of infants. There were also no differences in outcomes for the mothers.
Research on the case management system at Carondolet St. Mary's Medical Center also has found that the content of nursing care has an impact on length of stay. The nursing delivery system at this facility involves a network of home care, hospice, community wellness centers, and hospital nursing with a nurse case manager bridging community and in-hospital care. Case managers, prepared at least at the bachelor's level, are the hub of the health care delivery system. Ethridge and Lamb (1989) report on the effects of the nursing case management intervention with patients who received total hip replacements and those with respiratory disease. For the first group, length of stay was reduced by 2.1 days while it was reduced by 3.5 days for the second group. The authors hypothesize that the reduction for acute illness (hip replacement) occurs at the end of hospitalization and that case management allowed earlier discharge of these patients. For the chronically ill (respiratory disease), length of stay appeared to be reduced at the beginning of the hospitalization. Those patients who were case-managed prior to hospitalization entered the hospital at lower acuity levels and shortened their length of stay by seeking care before illness severity reached a level that would require longer hospitalization.
A further report by Ethridge (1991) examines more than 700 case-managed patients enrolled in a health maintenance organization's senior plan. The data for the case managed clients were compared with national and state statistics for Medicare patients and health maintenance organization service use statistics. Results indicate that the case-managed patients had 53 fewer annualized hospital admissions, 895 fewer bed-days, and an average length of stay 1.73 days lower than other Medicare patients in the state.
Two reports by Naylor and colleagues (Naylor, 1990; Naylor et al., 1994) examine the effects of gerontological nurse specialist care. This care primarily involved a discharge planning protocol for hospitalized elders that was implemented by clinical nurse specialists while the client was hospitalized and 2 weeks after discharge. The earlier manuscript reports on the pilot study for the full project and included 40 hospitalized patients, age 70 and older, who were randomly assigned to the discharge planning or control group. No statistical differences were found in initial hospital length of stay or posthospital infection rate. However, there were significant differences in the number of rehospitalizations during the 12 weeks after discharge. In the larger, second study, 276 patients were included. Findings indicated that patients in the medical intervention group had fewer readmissions and fewer total days rehospitalized than the control group. No differences were found between the surgical intervention group and the control. Again, there were no differences in initial hospital length of stay.
Studies such as those described above indicate that nursing care does have an impact on the utilization variables of length of hospital stay and readmission rates. However, the research has more to do with the practice pattern of nurses than with the mix of staff or staffing ratios. These studies also indicate the advantages of nurses in specialized advanced practice. Unfortunately, no re-
search has been reported that describes such practice patterns with less well educated staff to determine whether results would be similar.
Research on the utilization of health care services has included primarily the variables of readmission and length of hospital stay. Continued use of these variables, particularly hospital length of stay, is problematic due to increased implementation of critical paths and prospective payment that essentially predetermines length of hospitalization. What appears to be of greater significance in the future is a measure of episode of illness that would include both length of stay and readmission.
Many studies that examine models of nursing practice also investigate the effect of these models on patient and family satisfaction. Both Burnes-Bolton and colleagues (1990) and Brett and Tonges (1990) report the results from one nursing unit. The first report notes increases in satisfaction, while the second indicates that two measures of satisfaction remained stable. This latter finding has some significance since the practice model implemented included a decrease in the number of RNs and an increase in unlicensed personnel. However, the data presented from both studies are considered pilot work from only one involved nursing unit.
Lamb and Huggins (1990) in another report of the Carondolet nurse case management system also reported increase in patient satisfaction over time. The study by Mitchell and colleagues (1989) previously mentioned under the heading of "Mortality" also noted higher levels of patient satisfaction in the ideal critical care environment.
In a 1985 publication, Koerner and colleagues (1985) report the results of the implementation of a system of professional nursing practice through collaboration with physicians. The system contained the five interrelated parts as specified by the National Joint Practice Commission in 1977: a joint practice committee of physicians and nurses; primary nursing; nursing clinical decision making within the scope of nursing practice as defined by the joint practice committee; integrated patient records; and joint practice review. Again this study contained only one demonstration unit and one control unit. The sample included 280 patients (100 from the control unit and 180 from the demonstration unit). The researchers evaluated patient satisfaction with a self-developed instrument designed to measure patient-provider interaction, quality of care, health education, knowledge of practitioners, and environment of the unit. Patients from the collaborative practice unit reported significantly greater patient-provider interaction, provider knowledge, health education, and respectful treatment. No differences were
noted in patient satisfaction with the physical environment or expectations of care. The authors note a limitation to the study in that the satisfaction instrument was designed with an emphasis on items pertinent to a collaborative practice environment and results may be biased toward the positive evaluation of the experimental treatment.
These research studies tend to indicate an increase in patient satisfaction with the implementation of a more professional model of practice, which is consistent with that previously described as part of the magnet hospital studies. However, there is also some indication that satisfaction remains stable even when the mix of nursing staff is altered to fewer professional nurses and more unlicensed personnel.
The popularity of patient satisfaction as an outcome measure is reflected in the number of instruments that have been developed for evaluation and research purposes (McDaniel and Nash, 1990). While there are many ways to measure patient satisfaction, not all are accurate or reliable. Specifically, the construct validity of many instruments designed for individual studies is questionable. Often, it appears that patients are being asked to evaluate quality of care rather than their satisfaction with that care. A further limitation of patient satisfaction with nursing care as an outcome measure is that patients traditionally report high levels of satisfaction, thus decreasing the variability of responses that are needed for analysis (LaMonica et al., 1986). Measures of satisfaction that are clinically feasible for use because of their parsimony often lack the sensitivity required to tap fine differences in patient perception.
Relationship of Staff Satisfaction to Client Outcomes
Much of the research investigating factors to increase staff nurse satisfaction makes the assumption that increased levels of satisfied staff will directly lead to increased quality of care. Two projects have examined this relationship as their primary hypothesis and will be described here, even though one of the studies occurred prior to the 1985 review date.
Holland and associates (1981) examined resident mental health patients and staff from 22 units in 3 psychiatric hospitals. Of the eligible staff, 98 percent (N = 297) were included in the analysis. However, exact titles and preparation were not delineated in the report. Sixty-eight percent (N = 249) of the total patients were selected with stratified random sampling to be included in the study. The focal unit for this research was the nursing unit and when individual data were aggregated to that level an analysis sample of 22 resulted. The outcome examined was potential posthospital adjustment as measured at discharge by a standardized scale. Improvement in resident functioning was moderately associated
with staff satisfaction. Greater staff participation in resident treatment affected the outcome only indirectly through staff satisfaction. The total effect of staff satisfaction, using path analysis, was .55 (.38 direct and .17 indirect).
A 1985 publication by Weisman and Nathanson (1985) reported on a study of teenage clients who attended 1 of 77 family planning clinics. Although this research does not involve the acute care setting, findings may still be pertinent. The 77 clinics involved in the research represented data from 344 nurses and 2,900 clients. Outcomes investigated were client satisfaction and rate of compliance with contraceptive prescriptions. Using path analysis, a significant direct effect (.32) of job satisfaction on client satisfaction was discovered. There was no direct effect of this variable on compliance rates. However, there was a significant indirect effect (.08) of job satisfaction on compliance through client satisfaction. Hays and White (1987) reanalyzed the Weisman and Nathanson (1985) data using structural equation modeling with LISREL. They supported the model proposed in the original study and proposed an alternative model that also fits the data. Both models supported the significant relationship between job satisfaction and client outcomes.
Of the four studies at the invitational conference that were also looking at the relationship between staff satisfaction and outcomes, none reported significant direct or indirect relationships. However, final data analyses were not completed for all of the studies.
Summary: Linkages And Gaps
Although some of the findings from investigations are limited by the number of studies supporting the relationship, small sample sizes, or both, some tentative conclusions can be drawn from the research presented. Research, either previously published or reported through the invitational conference, has supported the following linkages:
There is a relationship between a professional practice environment and perceptions of control over practice and autonomy.
There is a relationship between a professional practice environment and job satisfaction.
A professional practice environment and job satisfaction improve retention of staff.
The implementation of a professional practice model is cost neutral.
The proportion of RNs on a nursing staff has positive influence on severity-adjusted Medicare mortality rates.
A professional practice environment has a positive influence on severity-adjusted Medicare mortality rates, over and above the influence of staffing mix.
There is inconsistent evidence of the effect of nursing-staff-related
- variables on the development of nosocomial infection rates. It is likely, however, that when continuity of care is assured through various staffing mechanisms infection rates are lower.
Despite the existence of some research, there is currently no evidence that nurse staffing variables have an effect on adverse patient incidents. There is some indication, however, that nurse competence may have a positive effect on these rates.
There is sufficient evidence to suggest that innovative nursing delivery patterns (e.g., case management; transitional models of hospital to home care) will reduce utilization of health care services through either decreased length of hospital stay or reduced readmissions.
Findings on the effect of nursing on patient satisfaction are inconsistent. Published studies indicate that satisfaction is stable or increased with professional practice environments even when staffing levels are reduced; current unpublished research as presented at the invitational conference does not support this conclusion.
In outpatient and inpatient psychiatric settings there is a relationship between job satisfaction and client satisfaction; this result has not been supported for acute care hospitals and does not hold for outcomes other than client satisfaction.
In addition to needing more research to further support or clarify some of the tentative findings indicated above, research is needed to fill the gaps in the state of the science. Some of the gaps in our current empirical knowledge include:
Whether satisfied staff in acute care give better care, resulting in improved client outcomes.
Whether there is a link between professional practice and staffing needs.
Whether there is a relationship between professional practice environments and patient outcomes other than mortality.
Whether there is a connection between staffing mix and client outcomes of health status.
What the interaction is between productivity and quality care and the main effect and interactive effect of these variables on client outcomes.
What the influence of leadership is on the productivity and quality of nursing care.
A number of methodological problems and issues in research involving the investigation of nursing's effect on client outcomes were identified at the invitational conference. Only those that directly relate to the impact of nursing on
client outcomes and the measurement of those outcomes are addressed in this paper. These issues tend to categorize into three areas: sampling, sensitivity of measures, and consistency of measures.
The first issue under the area of sampling involves the sample size of research projects that examine nursing interventions that are unit based. These interventions include practice patterns, staffing mix, collaboration, and a number of other aspects of the work environment. Since the environment in which nurses deliver care is often determined by the unit or the hospital in which they practice, the unit of analysis for most of these studies is not at the individual level. In general, the cost to implement a practice model requires a smaller sample size than would normally be desired when the sampling unit is the work group. A further cost factor has to do with the expense of collecting data in a number of sites or on a number of units. Since some data must be collected at the individual level and aggregated to the unit (e.g., staff satisfaction) researchers generally expend scarce resources to collect the individual staff data and identify already existing measures of outcomes that can be considered unit or hospital based. Such information is normally collected and reported by hospitals. This approach results in outcome measures with a high degree of variability that are unit based from a limited number of units and a subsequent reduction in power. Under such conditions, it is extremely difficult to find statistical significance. It is quite likely that this effect explains the inconsistent findings in the research on nursing's impact on mortality rates. Those studies with larger sample sizes yield significant findings; those with smaller (N ≦ 30–40) sample sizes have no significant results. The same may be true for other measures of client outcomes such as untoward hospital incidents and nosocomial infections. The good news about these measures is they are rare events; the bad news is that nursing's effect cannot be shown unless, like mortality data, the information is included in large databases to which researchers have access.
A further result of small sample sizes is the resultant inability to use sophisticated statistical procedures that consider the multivariate nature of nursing practice and the complexity of factors that lead to positive client outcomes. In the Arizona study we were limited, primarily, to the most basic inferential statistics. When we were able to use individual data and, thus, increased our sample size, we found highly significant results. Also, when we examined patterns of response we had important findings—they may not be statistically significant but they are clear patterns that result from the use of more complex statistical techniques.
A second issue with sampling has to do with the number of patients who are ineligible to participate in the research for a variety of reasons. This problem is exemplified in the descriptive New York study (Study 5). For this project,
individual patients were interviewed. The potential sample was approximately 4,600. Of these, 956 subjects refused to be part of the study and 577 were ineligible. These may be the patients who are most sensitive to quantity and quality of nursing care. However, when individual patient responses are required, these data are never collected (e.g., patient satisfaction). In addition, patients are lost to studies for a variety of reasons and those reasons may, again, be critical in determining outcomes.
The need to identify nurse-sensitive patient outcomes has been discussed in a number of settings with both clinicians, managers, and researchers. Although this need is critical, there remains a concurrent need to continue to use more traditional outcome measures (e.g., mortality, morbidity, length of stay). These client outcomes are recognized by consumers and other providers as being important in examining patient welfare. It must be recognized, however, that large sample sizes will be needed to increase the sensitivity of these measures and the measures must be readily available in large data sets to which researchers have access.
When nurse-sensitive outcome measures are discussed, usually they are considered to be those that relate to specific patient problems or conditions. For macrolevel research as described above, other outcomes must be identified. These outcomes need to be applicable across settings and conditions, and they must be reflective of the pattern of nursing care delivered. In order to be useful such outcomes must also be easy to measure on large numbers of patients, and when aggregated to the unit or hospital level they must have validity and reliability at that level. The issue of aggregation has been examined recently and criteria for examining reliability and validity have been described (Verran et al., 1992, 1995). Existing and ongoing work, such as that by the researchers in the San Francisco study, may provide a model for the development of such outcome measures. Their 10-item instrument is clinically feasible, reliable, and valid with predictive power. We need more of these measures that are applicable to a wide variety of clients, including the family of a client, and that can be used across units, hospitals, and settings. Such measures could eventually be included in databases and if used in a number of research projects will allow comparability. Currently, an expert panel of the American Academy of Nursing is beginning work on the identification of a set of these outcomes. What may be needed is to expand this work into a nationally supported nursing Patient Outcomes Research Team similar to those established for specific patient conditions through the Agency for Health Care Policy and Research.
A further issue in the area of sensitivity has to do with the timing of outcome measurement. Research that examines outcomes at one point in time, such as discharge, will probably not see significant results. In today's health care envi-
ronment, positive outcomes usually occur in the community a significant period of time after the client leaves the hospital. In order to show the effect of nursing on outcomes, longitudinal studies need to be conducted. Hospital effects probably don't occur while the client is in the hospital. It is after they are home for a while that they realize they don't know how to care for themselves or that they know a variety of techniques to relieve pain or discomfort. In association with this timing issue is the need to examine timing expectancies. There is an ideal outcome level to be achieved and there are benchmarks along the trajectory toward that achievement that need to be identified.
Consistency of Measures
The third area of concern with researching the relationship of nursing to patient outcomes has to do with the consistency of outcomes measurement. Definitions of terms need to be very specific in order to compare findings across studies. With the complexity of nursing processes and small sample sizes, the only way firm conclusions can be drawn is by synthesizing the results from several research projects. One way to establish this specificity is to establish standardized nursing vocabularies for outcomes. These are not medical vocabularies—they are nursing oriented. We have some consistent work on a vocabulary for nursing problems and one for nursing interventions. A group of researchers at the University of Iowa is also working on a taxonomy for outcomes.
These taxonomies or categorization schemes are less meaningful, however, if the variables are not included in national databases. Nursing information on practice, staffing patterns, problems, interventions, and outcomes need to be incorporated into health care systems information. A national effort needs to be mounted to establish standardized vocabularies, incorporate them into databases, and encourage research using such systems.
This paper has been presented in four sections. The first reported on six current research projects examining the interrelationship of staff, organizational, and client outcome variables. Second, a review of literature from 1985 to 1995 on the effect of nursing care on general client outcomes was presented. From these two approaches, the linkages and gaps in our current knowledge about the effects of nursing care on client outcomes on a macrolevel were identified. Finally, methodological issues with research in this field were discussed. These issues were organized into the three categories of sampling, sensitivity of measures, and consistency of measures.
Aiken, L.H., Smith, H.L., and Lake, E.T. Lower Medicare Mortality Among a Set of Hospitals Known for Good Nursing Care. Medical Care 32:771–787, 1994.
ANA (American Nurses Association). Nursing Care Report Card for Acute Care. Washington, D.C.: American Nurses Publishing, 1995.
Brett, J.L.L., and Tonges, M.C. Restructured Patient Care Delivery: Evaluation of the ProACT™ Model. Nursing Economics 8:36–44, 1990.
Brooten, D., Kuman, S., Brown, L.P., et al. A Randomized Clinical Trial of Early Hospital Discharge and Home Follow-up of Very Low Birthweight Infants. New England Journal of Medicine 315:934–939, 1986.
Burnes-Bolton, L., Daviver, M.A., Voxburgh, M.M., et al. A Cost Containment Model of Primary Nursing at Cedars-Sinai Medical Center. Pp. 129–149 in: G.G. Mayer, M.J. Madden, and E. Lawrenz, eds. Patient Care Delivery Models. Rockville, Md.: Aspen, 1990.
Devine, E.C., and Cook, T.D. A Meta-Analytic Analysis of Psychoeducational Interventions on Length of Postsurgical Hospital Stay. Nursing Research 32:267–274, 1983.
Ethridge, P. A Nursing HMO: Carondelet St. Mary's Experience. Nursing Management 22:22–27, 1991.
Ethridge, P., and Lamb, G.S. Professional Nursing Case Management Improves Quality, Access and Costs. Nursing Management 20:30–35, 1989.
Flood, S.D., and Diers, D. Nurse Staffing, Patient Outcome And Cost. Nursing Management 19:34–43, 1988.
Gordon, D., Weisman, C., Bergner, M., and Wong, R. A Model for Reorganizing Nursing Resources. Funded by the National Institute of Nursing Research, National Institutes of Health (NR02091), 1989.
Hathaway, D. Effect of Preoperative Instruction on Postoperative Outcomes: A Meta-Analysis. Nursing Research 35:269–275, 1986.
Hays, R.D., and White, K. Professional Satisfaction and Client Outcomes. Medical Care 25:259–262, 1987.
Heater, B.S., Becker, A.M., and Olson, R.K. Nursing Interventions and Patient Outcomes: A Meta-Analysis of Studies. Nursing Research 37:303–307, 1988.
Henry, S.B., Holzemer, W.L., and Reilly, C.A. The Relationship Between Type of Care Planning System and Patient Outcomes in Hospitalized AIDS Patients. Journal of Advanced Nursing 19:691–698, 1994a.
Henry, S.B., Holzemer, W.L., Reilly, C.A., and Campbell, K.E. Terms Used By Nurses To Describe Patient Problems: Can SNOMED III Represent Nursing Concepts in the Patient Record? Journal of the American Medical Informatics Association 1:61–74, 1994b.
Holland, T.P., Konick, A., Buffum, W., et al. Institutional Structure and Resident Outcomes. Journal of Health and Social Behavior 22:433–444, 1981.
Holzemer, W.L., and Henry, S.B. ''Quality of Nursing Care for People with AIDS." Funded by the National Institute of Nursing Research, National Institutes of Health (R01 NR02215), 1989.
Holzemer, W.L., Henry, S.B., Stewart, A., and Janson-Bjerklie, S. The HIV Quality Audit Marker (HIV–QAM): An Outcome Measure for Hospitalized AIDS Patients. Quality of Life Research 3:99–107, 1993.
Ingersoll, G.M., Schultz, A.W., Ryan, S.A., et al. "The Enhanced Professional Practice Model." Cooperative agreement award jointly funded by the National Institute of Nursing Research, National Institutes of Health and the Division of Nursing, Public Health Service (U01 NR02156), 1988.
Ingersoll, G.M., Ryan, S.A., and Schultz, A.W. Enhanced Professional Practice Model for Nursing. Research Review: Studies for Nursing Practice 7:3, 1990.
Ingersoll, G.M., Ryan, S.A., and Schultz, A.W. Evaluating the Impact of Enhanced Professional
Practice on Patient Outcome. Pp. 301–313 in: I.E. Goertzer, ed. Differentiating Nursing Practice into the Twenty-First Century. Kansas City, Mo.: American Academy of Nursing, 1991.
Ingersoll, G.M., Bazar, M., and Zentner, J. Use of a Process Evaluation Map to Measure Program Implementation Progress. Nursing Economics 11:137–142, 1993.
Janson-Bjerklie, S., Holzemer, W.L., and Henry, S.B. Patients' Perceptions of Pulmonary Problems and Nursing Interventions During Hospitalization for Pneumocystis carinii Pneumonia. American Journal of Critical Care 1:114–121, 1992.
Knaus, W.A., Draper, E.A., Wagner, D.P., and Zimmerman, J.E. An Evaluation of Outcome from Intensive Care in Major Medical Centers. Annals of Internal Medicine 104:410–418, 1986.
Koerner, B.L., Cohen, J.R., and Armstrong, D.M. Collaborative Practice and Patient Satisfaction. Evaluation and the Health Professions 8:299–321, 1985.
Kramer, M. The Magnet Hospitals: Excellence Revisited. Journal of Nursing Administration 18:20–25, 1990.
Kramer, M., and Schmalenberg, C. Magnet Hospitals: Institutions of Excellence, Part I. Journal of Nursing Administration 18(1):13–24, 1988a.
Kramer, M., and Schmalenberg, C. Magnet Hospitals: Institutions of Excellence, Part II. Journal of Nursing Administration 18(2):11–19, 1988b.
Lamb, G.S., and Huggins, D. The Professional Nursing Network. Pp. 169–184 in: G.G. Mayer, M.J. Madden, and E. Lawrenz, eds. Patient Care Delivery Models. Rockville, Md.: Aspen, 1990.
LaMonica, E.L., Oberst, M.T., Madea, A.R., and Wolf, R.M. Development of a Patient Satisfaction Scale. Research in Nursing and Health 9:43–50, 1986.
McClure, M., Poulin, M., Sovie, M.D. et al. Magnet Hospitals: Attraction and Retention of Professional Nurses. Kansas City, Mo.: American Academy of Nursing, 1983.
McDaniel, C., and Nash, J.G. Compendium of Instruments Measuring Patient Satisfaction with Nursing Care. QRB (Quality Review Bulletin) 16:182–188, 1990.
Milton, D., Verran, J.A., Murdaugh, C., and Gerber, R. Implementing Differentiated Practice as Part of a Professional Practice Model. Pp. 279–285 in: I.E. Goertzer, ed. Differentiating Nursing Practice into the Twenty-First Century. Kansas City, Mo.: American Academy of Nursing, 1990.
Milton, D., Verran, J.A., Murdaugh, C., and Gerber, R. Differentiated Group Professional Practice in Nursing: A Demonstration Model. Nursing Clinics of North America 27:23–30, 1992.
Milton, D.A., Verran, J.A., Gerber, R.M., and Fleury, J. Tools To Evaluate Reengineering Progress. Pp. 195–202 in: S.S. Blancett and D.L. Flarey, eds. Reengineering Nursing and Health Care. Gaithersburg, Md.: Aspen, 1995.
Minnick, A. Improving Patient Centered Care Through Initiatives in Nursing. Funded by the Picker/Commonwealth Fund, 1991.
Minnick, A. Kleinpell, R., Micek, W., and Dudley, D. The Management of a Multi-Site Study. Journal of Professional Nursing, in press(a).
Minnick, A., Roberts, M.J., Young, W., et al. Telephone Survey Response Rates and the Measurement of Patient Centered Care. Nursing Research, in press(b).
Minnick, A., Young, W., and Roberts, M.J. Reports of Service, Ratings of Quality and Intent to Recommend: 2000 Patients Relate Their Hospital Experiences. Nursing Management, in press(c).
Mitchell, P.H., Armstrong, S., Simpson, T.F., and Lentz, M. American Association of Critical-Care Nurses Demonstration Project: Profile of Excellence in Critical Care Nursing. Heart and Lung 18:219–237, 1989.
Mitchell, P., Hegevary, S., and Secrest, K. "Critical Care Nursing Systems, Retention and Patient Outcomes." Funded by the National Institute of Nursing Research, National Institutes of Health (R01 NR02343), 1991.
Mumford, E., Schlesinger, H.J., and Glass, G.V. The Effects of Psychoeducational Intervention on Recovery from Surgery and Heart Attacks: An Analysis of the Literature. American Journal of Public Health 72:141–151, 1982.
Naylor, M.D. Comprehensive Discharge Planning for Hospitalized Elderly: A Pilot Study. Nursing Research 39:156–161, 1990.
Naylor, M., Brooten, D., Jones, R., et al. Comprehensive Discharge Planning for the Hospitalized Elderly. Annals of Internal Medicine 120:999–1006, 1994.
Prescott, P.A. Nursing: An Important Component of Hospital Survival Under a Reformed Health Care System. Nursing Economics 11:192–199, 1993.
Rose, M., and DiPasquale, B. The Johns Hopkins Professional Practice Model. Pp. 85–97 in: G.G Mayer, M.J. Madden, and E. Lawrenz, eds. Patient Care Delivery Models. Rockville, Md.: Aspen, 1990.
Smith, M.C., and Naftel, D.C. Meta-Analysis: A Perspective for Research Synthesis. Image: The Journal of Nursing Scholarship 16:9–13, 1984.
Taunton, R.L., Kleinbeck, S.V.M., Stafford, R., et al. Patient Outcomes. Are They Linked to Registered Nursing Absenteeism, Separation or Work Load? Journal of Nursing Administration 24:48–55, 1994.
Verran, J.A., Murdaugh, C., Gerber, R., and Milton, D. Differentiated Group Professional Practice in Nursing. Cooperative agreement award jointly funded by the National Institute of Nursing Research, National Institutes of Health, and the Division of Nursing, Public Health Service (U01 NR02153), 1988.
Verran, J.A., Mark, B.A., and Lamb, G. Psychometric Examination of Instruments Using Aggregated Data. Research in Nursing and Health 15:237–240, 1992.
Verran, J.A., Gerber, R., Milton, D., and Murdaugh, C. Differentiated Group Professional Practice in Nursing Project: Pathfinding for Patient Care Restructuring. Pp. 26–42 in: M.L. Parsons and C.L. Murdaugh, eds. Patient-Centered Care: A Model for Restructuring. Gaithersburg, Md.: Aspen, 1994.
Verran, J.A., Gerber, R., and Milton, D. Data Aggregation: Criteria for Psychometric Evaluation. Research in Nursing and Health 18:77–80, 1995.
Wan, T.T., and Shukla, R.K. Contextual and Organizational Correlates of the Quality of Hospital Nursing Care. QRB (Quality Review Bulletin) 13:61–65, 1987.
Weisman, C.S., and Nathanson, C.A. Professional Satisfaction and Client Outcomes. A Comparative Organizational Analysis. Medical Care 23:1179–1192, 1985.
Weisman, C.S., Gordon, D.L., Cassard, S.D., et al. The Effects of Unit Self-Management on Hospital Nurses' Work Process, Work Satisfaction, and Retention. Medical Care 31:381–393, 1993.
Wong, R., Gordon, D.L., Cassard, S.D., et al. A Cost Analysis of a Professional Practice Model for Nursing. Nursing Economics 11:292–297, 323, 1993.
Zimmerman, J.E., Shortell, S.M., Rousseau, D.M., et al. Improving Intensive Care: Observations Based on Organizational Case Studies in Nine Intensive Care Units: A Prospective, Multicenter Study. Critical Care Medicine 21:1443–1451, 1993.
Zimmerman, J.E., Rousseau, D.M., Duffy, J., et al. Intensive Care at Two Teaching Hospitals: An Organizational Case Study. American Journal of Critical Care 3(2):129–138, 1994.
Attendees at the Invitational Workshop "Quality of Care:
Examining the Influences of Nursing Resources"
October 23–24, 1994
The workshop was sponsored and organized by the American Academy of Nursing.
Ada Sue Hinshaw
Jan Heinrich, American Academy of Nursing
Pat Moritz, National Institute of Nursing Research
Gooloo Wunderlich, Institute of Medicine
Angela McBride, American Academy of Nursing
Barbara Donohoe, American Academy of Nursing and the Robert Wood Johnson and Pew Foundations
Marjorie Beyers, American Organization of Nurse Executives
Deborah Nansom, Joint Commission on Accreditation of Healthcare Organizations
Marilyn Chow, American Nurses Association