**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

*Organ Procurement and Transplantation: Assessing Current Policies and the Potential Impact of the DHHS Final Rule*. Washington, DC: The National Academies Press. doi: 10.17226/9628.

**A**

**Data Sources and Methods**

In an effort to be comprehensive in addressing the task of reviewing the current policies of the Organ Procurement and Transplantation Network (OPTN) and the potential impact of the Final Rule, the committee explored various data sources in a concerted effort to cast a broad net for the collection and assessment of information. These sources included public input and testimony from federal agencies, professional societies, organizations, and individuals; a review of recent scientific literature; and statistical analyses of over 68,000 records of patient listings for liver transplantation.

In addition to these fairly traditional sources of data, expert liaisons were assembled for the committee to consult with throughout the project (see Box A1). The expert liaisons are people with recognized experience and expertise on the issues before the committee. They provided technical advice and guidance in framing the issues, identifying important sources of information, and ensuring a comprehensive analysis. A summary description of the committee's evidence-gathering method follows.

**TESTIMONY AND PUBLIC INPUT**

Over the course of the study, the committee requested and received written responses and presentations from organizations and individuals representing many perspectives of organ procurement and transplantation. The committee felt it was important to receive as much input as possible from public groups involved with or seeking involvement in the organ allocation process, as well as from health professional and other organizations. To accomplish this, the committee convened public meetings on March 11 and April 16, 1999, to gather information and hear from groups and individuals. The committee made every effort to include as many groups as possible, given the short time available.

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

*Organ Procurement and Transplantation: Assessing Current Policies and the Potential Impact of the DHHS Final Rule*. Washington, DC: The National Academies Press. doi: 10.17226/9628.

Committee members heard presentations and asked questions to explore the particular issues and unique perspectives that each organization represented. In particular, the committee was interested in hearing of the potential impact of the Final Rule on these respective parties. The organizations and individuals that addressed the committee are listed in Box A-2.

Vicki Crosier, National Kidney Foundation Donor Family Council Charlie Fiske, National Transplant Action Committee Pushkal Gang, Johns Hopkins University Robert J. Kelly, Recipient Family Member George Walton, Donor Family Member Bruce Weir, Transplant Recipient International Organization
Ronald W. Busuttil, University of California at Los Angeles Clive Callender, Howard University Hospital Anthony D'Alessandro, University of Wisconsin Hospital and Clinics Arnold Diethelm, University of Alabama, Birmingham Ronald M. Ferguson, Ohio State University John Fung, University of Pittsburgh William E. Harmon, Children's Hospital, Boston John F. Neylan, Emory University
Carol Beasley, Partnership for Organ Donation James Childress, University of Virginia Rudolph C. Morgan, Organ and Tissue Acquisition Center, San Diego, Calif. Howard Nathan, Gift of Life Transplant Program Robert M. Sade, Medical University of South Carolina Rodney Taylor, National Minority Organ Tissue and Transplant Education Program Charles Thomas, Samaritan Transplant Services, Phoenix, Ariz. Kathy Witmer, University of Washington |

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

*Organ Procurement and Transplantation: Assessing Current Policies and the Potential Impact of the DHHS Final Rule*. Washington, DC: The National Academies Press. doi: 10.17226/9628.

Milton Benjamin, American Society of Transplant Surgeons Vicki Crosier, National Kidney Foundation Donor Family Council Marcia Crosse, U.S. General Accounting Office Beverly Dennis, U.S. Department of Health and Human Services Mike Hall, American Liver Foundation William Harmon, American Society of Transplantation Craig Irwin, National Transplant Action Committee Richard Luskin, Association of Organ Procurement Organizations Robert Merion, Patient Access to Transplantation Coalition William W. Pfaff, United Network for Organ Sharing Bruce Weir, Transplant Recipient International Organization Andrea Zachary, American Society of Histocompatibility and Immunogenetics
Ronald W. Busuttil, University of California at Los Angeles Clive Callender, Howard University Hospital Ronald M. Ferguson, Ohio State University Jameson Forster, University of Kansas Doug Hanto, University of Cincinnati Robert Higgins, Henry Ford Hospital Mark Joensen, CONSAD Research Corporation Goran Bo Gustaf Klintmalm, Baylor University Medical Center Patrick McCarthy, Kaufman Center for Heart Failure, Cleveland Robert Metzger, Translife, Orlando, Fla. William Minogue, Suburban Hospital, Bethesda, Md. Paulita Narag, Hendrick Medical Center, Abilene, Texas Howard Nathan, Delaware Valley Transplant Program Mary Ann Palumbi, North American Transplant Coordinators Organization William W. Pfaff, United Network for Organ Sharing Timothy L. Pruett, University of Virginia Byers Shaw, University of Nebraska Medical Center Kevin Stump, Mississippi Organ Recover Agency Carlton Young, University of Alabama, Birmingham |

In addition to the participants listed in Box A-2, many other individuals attended and participated in the public meetings, and/or provided written information to the committee. These individuals are listed below:

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

**Other Participants and Contributors**

Patricia Adams Bowman Gray School of Medicine

Mike Adcock Patient Access to Transplantation Coalition

Jason Altmire UPMC Health Systems

Denise Alveranga Lifelink Transplant Institute

Bill Applegate American Society of Transplantation

David Benor Department of Health and Human Services

Audrey Bohnengel Ohio Solid Organ Transplantation Consortium

Jodi Chappell American Society of Transplantation

Dolph Chianchiano National Kidney Foundation

Karen Chiccehitto United Network for Organ Sharing

Coralyn Colladay Department of Health and Human Services

Pat Daily United Network for Organ Sharing

Todd Dickerson University of Cincinnati

Isabel Dunst Hogan and Hartson Washington, D.C.

Gail Durant American Society of Transplant Surgeons

Erick Edwards United Network for Organ Sharing

Jon Eiche The Living Bank International

Mary Ellison United Network for Organ Sharing

Lorraine Fishback Department of Health and Human Services

John Ford U.S. House of Representatives Committee on Commerce

Walton Francis Department of Health and Human Services

Robert Goldstein Juvenile Diabetes Foundation

Walter Graham United Network for Organ Sharing

Carol Green U.S. Senate Committee on Health Education, Labor, and Pensions

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

Pamela Guarrera Transplantation Institute

Ann Harper United Network for Organ Sharing

Baxter Harrington American Society of Minority Health and Transplant Professionals

Russell Hereford Office of Evaluation and Inspections

Roy Hogberg General Accounting Office

Lesly Hollman Bureau of National Affairs

A. J. Hostetler Richmond *Times-Dispatch*

Melody Hughson Hoffman-LaRoche

Kent Jenkins United Network for Organ Sharing

Linda Jones Lifeline of Ohio

Karen Kennedy Transplant Resource Center of Maryland

Jerry Klepner United Network for Organ Sharing

Lisa Kory Transplant Recipient International Organization

Evan Krisely Patient Access to Transplantation Coalition

Eugene Laska Nathan Kline Institute

Judy LaSov Maryland Patient Advocacy Group

William Lawrence United Network for Organ Sharing

Sue Leffell American Society of Histocompatibility and Immunogenetics

Becky Levin Renal Physicians Association

Pearl Lewis Maryland Patient Advocacy Group

Chris Lu U.S. House of Representatives Government Reform Committee

Michael Manley Alaska Regional Organ Recovery Agency

Mark Marin University of Cincinnati

Mary Mazanec Senator William Frist's Office

Patrick McCarthy Kaufman Center for Heart Failure

Eileen Meier North American Transplant Coordinators Organization

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

Laura Melkler Associated Press

Behn Miller General Accounting Office

Joshua Miller American Society of Transplant Surgeons

Marlene Mitman American Society of Transplant Surgeons

Joseph Morton Maryland Patient Advocacy Group

Elizabeth Neus Gannett News Service

Jill Nusbaum National Kidney Foundation

Joseph O'Donnell Transplant Resource Center of Maryland

Lazar Palnick University of Pittsburgh

Matthew Piron Transplant Recipient International Organization

Dave Ress Richmond *Times-Dispatch*

Lisa Rossi University of Pittsburgh

Paul Schwab Association of Organ Procurement Organizations

Timothy Shaver INOVA Fairfax Hospital

Haimi Shiferaw *The Blue Sheet*

Bernice Steinhardt General Accounting Office

S. John Swanson, III Organ Transplant Service and Consultant to Army Surgeon General for Transplantation

Alice Thurston American Association of Kidney Patients

Sibyl Tilson Congressional Research Service

Jennifer Van Horn U.S. Senate Committee on Health, Education, Labor, and Pensions Subcommittee on Public Health

Cliff VanMeter United Network of Organ Sharing

Angela Vincent National Medical Association

Jim Warren *Journal of Transplant News*

Lynn Wegman Department of Health and Human Services

Marc Wheat U.S. House of Representatives Committee on Commerce

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

J. White Department of Health and Human Services

Marlene Whiteman Strategic Alliance Management

Donna Henry Wright United Network for Organ Sharing

Elaine Young Juvenile Diabetes Foundation

Troy Zimmerman National Kidney Foundation

To gain the perspective of people who could not attend the public meetings, a notice was mailed to more than 1,000 professional societies, organizations, and interest groups. The mailing included a one-page description of the study, the committee roster, and a cover letter explaining the committee's purpose for requesting the information. The letter asked those interested to send or fax comments pertinent to the committee's five tasks. The information submitted supplemented the materials obtained by the committee through the literature review, public meetings, and data analyses.

All written materials presented to the committee were reviewed and considered with respect to the five tasks. This material can be examined by the public. The public access files are maintained by the National Research Council Library at 2001 Wisconsin Avenue, N.W., Harris Building, Room HA 152, Washington, DC 20007; tel: (202) 334-3543.

**LITERATURE REVIEW**

The committee conducted numerous literature searches as part of its effort to be comprehensive. Search terms used included organ donation policy, ethics, organ donation, organ procurement, organ preservation, ischemic time, costs of transplantation, and secondary analyses of existing databases. In addition, many transplant professionals and the expert liaisons provided literature to the committee for review and consideration.

**STATISTICAL ANALYSIS**

At the committee's request, the United Network for Organ Sharing (UNOS) provided a large amount of data regarding organ-specific allocation policies; waiting list mortality rates; waiting lists from multiple organ procurement organizations (OPOs); citizenship of patients recently added to the waiting lists; survival rates and transplant rates by OPO population size; OPO death rates on the liver waiting list by initial status and status at death; algorithms; and audits regarding classification of recipients.

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

**Analysis of Waiting Time**

The statistical development of the model used in this analysis is described by Hedeker and Gibbons (1994). Note that as previously described, the unit of analysis is the patient-day and not the patient. Following Efron (1988) we assume that days within patients are conditionally independent on the prior days as long as the competing risk outcomes of interest (i.e., death or mortality) can only occur on the final day for each subject. Using the terminology of multilevel analysis (Goldstein, 1995) let *i* denote the level-2 units (OPOs) and let *j* denote the level-1 units (patient-days within OPOs). Assume that there are *i* = 1, . . . , *N* level-2 units (i.e., OPOs) and *j* = 1, . . . , *n*_{i} level-1 patient-days nested within each OPO. The *n*_{i} patient-day measurements include the set of all available measurement days for all patients in OPO *i* (i.e., *n*_{i} is the total number of daily measurements in OPO *i*). Let *y*_{ij} be the value of the nominal variable associated with level-2 unit *i* and level-1 unit *j*. In our case, these represent transplant, death, and other and we code the *K* + 1 response categories as 0, 1, 2.

Adding random effects to the multinomial logistic regression model of Bock (1970), Nerlove and Press (1973), and others, we get that the probability, for a given OPO *i*, and patient-day *j, Y*_{ij} = *k* (a response occurs in category *k*), conditional on *β* and *α*, is:

where *z*_{ijk} = *χ′* _{ij}*β*_{ik} + *w′* _{ij}*α*_{k}*.* Here, *w*_{ij} is the *p* × 1 covariate vector and *χ*_{ij} is the design vector for the *r* random effects, both vectors being for the *j*th patient-day nested within OPO *i*. Correspondingly, *α*_{k} is a *p* × 1 vector of unknown fixed regression parameters, and *β*_{ik} is a *r* × 1 vector of unknown random effects for OPO *i*. The distribution of the random effects is assumed to be multivariate normal with mean vector *μ*_{k} and covariance matrix Σ_{k}. Notice that the regression coefficient vectors *β* and *α* carry the *k* subscript. Thus, for each of the *p* covariates and *r* random effects, there will be *K* parameters to be estimated. Additionally, the random effect variance-covariance matrix Σ_{k} is allowed to vary with *k*.

It is convenient to standardize the random effects by letting *β*_{ik} = *Τ*_{k}*θ*_{i} + *μ*_{k}, where *Τ*_{k}*Τ*_{k}=*Σ*_{k} is the Cholesky decomposition of *Σ*_{k}. The model is now given as

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

In this form, it is clear that this generalizes Bock's (1972) model for educational test data by including covariates *w*_{ij}, and by allowing a general random-effects design vector *x*_{ij} including the possibility of multiple random effects *θ*_{i}.

**Parameter Estimation**

Let *y*_{i} denote the vector of nominal responses from OPO *i* all *n*_{i} patient-day measurements nested within. Then the probability of any *y*_{i}, conditional on the random effects θ and given *α*_{k}, *μ*_{k}, and *Τ*_{k}, is equal to the product of the probabilities of the patient-day responses:

where *d*_{ijk} = 1 if *y*_{ij} = *k*, and 0 otherwise. Thus, associated with the response from a particular patient-day, *d*_{ijk} = 1 for only one of the *K* + 1 categories and zero for all others. The marginal density of the response vector *y*_{i} in the population is expressed as the following integral of the likelihood, *l*(·), weighted by the prior density *g*(·):

where *g*(*θ*) represents the population distribution of the random effects.

For parameter estimation, the marginal log-likelihood from the *N* OPOs can be written as: log *L* = Σ_{i}^{N} log *h*(*y*_{i}). Then, using *η*_{k} to represent an arbitrary parameter vector,

where

J_{r} is a transformation matrix eliminating elements above the main diagonal (see Magnus, 1988), and *ν*(*Τ*_{k}) is the vector containing the unique elements of the Cholesky factor *Τ*_{k}. If *Τ*_{k} is a *r* x 1 vector of independent random effect variance terms, then *z*_{ijk} /*Τ*_{k} = *x*_{ij}*θ* in the equation above.

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

Fisher's method of scoring can be used to provide the solution to these likelihood equations. For this, provisional estimates for the vector of parameters Θ, on iteration *ι* are improved by

where the empirical information matrix is given by:

In general, the total number of parameters equals the *K* x *p* fixed regression coefficients (*α*_{k}; *k* = 1, . . . , *K*), plus the *K* x *r* means of the random effects (*μ*_{k}; *k* = 1, . . ., *K*), and the *K* x *r* x (*r* -1)/2 random effect variance-covariance terms (v[*Τ*_{k}]; *k* = 1, . . ., *K*). Notice that the parameter vector v(*Τ*_{k}), which indicates the degree of OPO population variance, is what distinguishes the mixed-effects model from the ordinary fixed-effects multinomial logistic regression model.

At convergence, the MML estimates and their accompanying standard errors can be used to construct asymptotic z-statistics by dividing the parameter estimate by its standard error (Wald, 1943). The computed z-statistic can then be compared with the standard normal table to test whether the parameter is significantly different from zero. While this use of the standard errors to perform hypothesis tests (and construct confidence intervals) for the fixed effects *μ*_{k} and *α*_{k} is generally reasonable, for the variance and covariance components v(*Τ*_{k}) this practice is problematic (see Bryk and Raudenbush, 1992, p. 55).

**Numerical Quadrature**

In order to solve the above likelihood equations, numerical integration on the transformed *θ* space can be performed. If the assumed random-effect distribution is normal, Gauss-Hermite quadrature can be used to approximate the above integrals to any practical degree of accuracy. In Gauss-Hermite quadrature, the integration is approximated by a summation on a specified number of quadrature points *Q* for each dimension of the integration; thus, for the transformed *θ* space, the summation goes over *Q*^{r} points. For the standard normal univariate density, optimal points and weights (denoted *B*_{q} and *A*(*B*_{q}), respectively) are given in Stroud and Sechrest (1966). For the multivariate density, the *r*-dimensional vector of quadrature points is denoted by **B**_{q}´ = (*B*_{q1}, *B*_{q2}, . . ., *B*_{qr}), with its associated (scalar) weight given by the product of the corresponding univariate weights,

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

If another distribution is assumed, other points may be chosen and density weights substituted for *A*(*B*_{q}) or *A*(*B*_{qh}) above (note, the weights must be normalized to sum to unity). For example, if a rectangular or uniform distribution is assumed, then *Q* points may be set at equal intervals over an appropriate range (for each dimension) and the quadrature weights are then set equal to 1/*Q*. Other distributions are possible; Bock and Aitkin (1981) discussed the possibility of empirically estimating the random-effect distribution.

For models with few random effects the quadrature solution is relatively fast and computationally tractable. In particular, if there is only one random effect in the model (as in the present case), there is only one additional summation over *Q* points relative to the fixed effects solution. As the number of random effects *r* is increased, the terms in the summation (*Q*^{r}) increase exponentially in the quadrature solution. Fortunately, as is noted by Bock, et al., (1988) in the context of a dichotomous factor analysis model, the number of points in each dimension can be reduced as the dimensionality is increased without impairing the accuracy of the approximations; they indicated that for a five-dimensional solution as few as three points per dimension were sufficient to obtain adequate accuracy. In general, specifying between 10 to 20 quadrature points for a unidimensional solution and 7 to 10 points for a two-dimensional solution is usually reasonable.

**Hazard Rates and Cumulative Survival**

For a model with one random-effect and three categories, we can estimate the probability of each outcome conditional on a particular covariate vector as

These are referred to as ''subject-specific" probabilities because they indicate response probabilities for particular values of the random subject effect *θ*_{i} (Neuhaus et al., 1991, Zeger et al., 1988). Replacing the parameters with their estimates and denoting the resulting subject-specific probabilities as , marginal probabilities are then obtained by integrating over the random-effect distribution, namely . Numerical quadrature can be used for this integration as well. These marginal probabilities represent the hazard rate for a particular competing risk of interest (i.e., transplant, mortality or other)

**Suggested Citation:**"Appendix A: Data Sources and Methods." Institute of Medicine. 1999.

expressed as a daily rate for status 1 or monthly rate for status 2B and 3 patients. The cumulative survival rate is then computed by summing the daily risk for status 1, or monthly risk in the case of status 2B and 3, over time adjusting for the number of subjects remaining on the list at that time point (i.e., adjusted for the competing risk).

All computations were performed using the MIXNO program developed under a grant from the National Institute of Mental Health and available at no charge at http://www.uic.edu/labs/biostat/.

**ANALYSIS OF COSTS**

The General Accounting Office (GAO) provided the committee with data that were instrumental in analyzing the potential effects of the Final Rule on transplantation costs. These included data on costs of solid organ transplantation, transportation costs, and costs of assembling a transplantation team. Roger Evans assisted Institute of Medicine staff and the committee in the analysis of these cost issues.