EXHIBIT C-1 Quality Assurance Protocols and Quality Control Activities

RTI Quality Assurance Protocol

Quality Control Activities

Analysts and programmers provide sufficient documentation so that their work can be audited if needed.

IHS reviewed the programming code and documentation prepared by RTI. The contents of Appendix A combined with the internal documentation appears sufficient that one can understand the data and methods used to complete the analyses and, if needed, replicate RTI’s payment simulations.

Analysts and programmers check and compare outputs to available benchmarks.

While no benchmarks exist for which to compare the payment simulations, substantial efforts were made by RTI, the IOM committee members, and IHS to compare the payment simulation results to CMS’s current geographic adjustments factors and to understand the drivers of differences between the various geographic indices.

Report tables identify the source computer run so that table contents can be verified against the program(s) that generated them.

The IOM report and Appendix A identify those tables where the table content was generated from an RTI simulation. RTI’s internal documentation identifies which computer program produced each table. Computer logs of the program code were reviewed by IHS.

Key study findings are sufficiently documented to allow for replication by other researchers.

This QA protocol is similar to a requirement of peer-reviewed journal articles—that sufficient information is provided to allow for replication of study findings. While RTI’s documentation does allow for researchers to understand the data and methods, some key data sources are not publically available. Consequently, replication of study findings would require access to special data tabulations prepared by various government agencies and purchase of nonpublic data.

All data tables shared with the IOM committee or included in the IOM should be closely reviewed by the project director.

Discussions with the RTI project team indicates that the RTI project codirectors were intimately involved with all aspects of this analysis—from reviewing programmer code to reviewing all data tables provided to the IOM committee and/or included in the IOM report.

Verify the correctness of the original and constructed variables before beginning the analysis.

In line with industry best practices, RTI produced frequency distributions and summary statistics (e.g., mean, minimum, maximum values) for both the original data and the variables created. This helped identify outliers or data anomalies that were then checked for accuracy.

NOTES: CMS = Centers for Medicare & Medicaid Services; IOM = Institute of Medicine; QA = quality assurance.

geographic reimbursement might impact access to services. However, the approach does not fully address the question. The payment simulations show the impact on redistribution of Medicare dollars by geographic area and shortage designation, and by different characteristics of facilities providing care to Medicare patients. The report notes the paucity of extant research to describe how changes in Medicare payments might affect the propensity of individual providers and facilities to serve Medicare patients. While one would expect that lower Medicare payments to a particular geographic area will have a detrimental effect on provider supply in that area (thus reducing patient access to care), additional research is required to fully understand the implications on patient access and quality of care. The data and methods used to quantify the financial implications of changes to Medicare’s geographic adjustment appear appropriate.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement