National Academies Press: OpenBook

Improving Breast Imaging Quality Standards (2005)

Chapter: Appendix A: ACR Survey Methods and Analysis

« Previous: 5 Beyond MQSA
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

Appendix A
ACR SURVEY METHODS AND ANALYSIS

The 2003 Survey was similar to its predecessor, the American College of Radiology’s (ACR’s) 1995 Survey of Radiologists and Radiation Oncologists (Deitch et al., 1997), but incorporated important improvements throughout the survey process. These ranged from more thorough canvassing of all ACR leadership in order to identify issues of importance and ascertain priorities among them, through use of a multifaceted “tailored design method” (Dillman, 2000) to maximize the response rate, to use of an expanded and more intensive array of steps to improve data quality.

The questionnaire for the 2003 Survey consisted of 36 items; many items in turn consisted of multiple subitems. Questionnaire items and topics were elicited from two rounds of canvassing ACR physician leaders and staff leaders, winnowed according to priorities indicated by top leadership, and pretested in two large pretests conducted in autumn 2002, with refinements made after each pretest.

The survey sample, a stratified random sample composed of four strata, was taken primarily from the American Medical Association’s (AMA’s) Physician Masterfile, a reasonably complete listing of all allopathic physicians in the United States, whether or not AMA members. The sample from the Masterfile consisted of a 16 percent sample of all those self-designated in the Masterfile as vascular/interventional radiologists, an 8 percent sample of all other radiologists, and an 8 percent sample of nuclear medicine specialists. The sample included residents, fellows, and retirees, not merely posttraining professionally active physicians, and it included physicians whether or not the Masterfile had usable addresses for them. The Masterfile sample was obtained from Medical Marketing Service, Inc. (Wood Dale, IL), the commercial firm designated by the AMA to provide Masterfile data, in January 2003. In addition, the sample included 92 osteopathic radiologists, selected at random by the American Osteopathic College of Radiology (AOCR) from its membership. Based on information supplied by the AOCR, this was an approximately 6.7 percent sample of all osteopathic radiologists in the United States, including non-AOCR members.

In March 2003, the ACR contractor, the Center for Survey Research (CSR) of the University of Virginia, mailed the survey. Nonrespondents were sent up to four remailings as necessary, at approximately monthly intervals. In addition, to boost the response rate: first-class stamps (not metered postage) were used on all outgoing and return envelopes; the survey was publicized in ACR hard-copy and electronic newsletters and those of other radiology organizations; the third remailing was conducted by U.S. Postal Service Priority Mail, which uses a large, attention-getting red, white, and blue envelope; nonrespondents for whom we had telephone numbers were telephoned after the third remailing (with a message left if not reachable after two calls) to urge them to complete the survey; and the third and fourth remailing had a handwritten note urging completion of the survey. The last remailing took place in mid-July; acceptance of responses ended a month later.

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

As in previous ACR surveys, among nuclear medicine specialists, the ACR was interested only in those who had major ties to radiology; this concept of a major tie to radiology was operationalized as holding American Board of Radiology (ABR) certification and/or being a member of the ACR (Sunshine et al., 2002). On this basis, approximately two-thirds of the original sample of nuclear medicine specialists were omitted from consideration.

The total sample of interest, which was composed of the four strata of interventionalists, all other allopathic radiologists, osteopathic radiologists, and nuclear medicine specialists of interest, consisted of 3,090 physicians. From these, 1,924 usably complete responses were received. In addition, not in the form of completed questionnaires, the ACR received information that 21 addressees were deceased, 6 were no longer practicing in the United States, and 6 were not radiologists. The response rate was thus (1,924+6)/(3,090−21−6)=63 percent.

Responses were weighted so that the weighted statistics would be representative of the answers that would have been received if all physicians in the United States in the four strata had been surveyed and had responded. The weighting process has been described previously (Sunshine et al., 2002). To begin, logistic regression analysis was employed to determine how many different sets of weights were to be used in each of the four strata. For the 2,743 physicians in the “all other allopathic radiologists” stratum, the analysis showed that ACR membership and age had statistically significant effects on the response rate, while sex, geographic region, and listing in the Masterfile as a “radiologist,” “diagnostic radiologist,” or “radiology subspecialist” did not. Accordingly, 10 weighting categories, based on whether or not a physician was an ACR member and his/her age, were used, and responses in each category were weighted by the reciprocal of the category’s response rate. A similar logistic analysis of the 202 interventionalists in the sample resulted in two weighting categories, based on whether or not the physician was an ACR member. Because logistic regression showed no statistically significant effect, only one weighting category was used for the nuclear medicine specialists of interest and one for the osteopathic radiologists. After all responses in each weighting category were given a weight equal to the reciprocal of the response rate for that category, these weights were multiplied by the reciprocal of the sampling rate to complete the process of making responses representative of the entire U.S. population of radiologists. For example, if a weighting category had a response rate of 65 percent and it was part of a stratum that had been sampled at the general 8 percent sampling rate, then all responses in that weighting category were given a weight of (1/0.65)×(1/0.08)=19.23.

Data Quality Improvement

Every survey has some deficient data—that is, missing items, responses not in accordance with directions given by the questionnaire, and responses that are inconsistent or have other problems. The leading tool to minimize data deficiencies in this survey was the designation of the 12 items on the questionnaire judged most crucial as “core questions.” When questionnaires were returned, CSR checked that these 12 items were indeed answered, and made three designated consistency checks involving them. If there were any problems with the core items, CSR telephoned the respondent to obtain the missing response(s) and/or resolve the consistency problems.

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

During the data entry process, CSR spot checked entries against the paper questionnaires and found an error rate of less than 0.1 percent. Judging this error rate satisfactory, the data were not double entered.

Data used in this report have been additionally cleaned and edited to further minimize deficiencies. An example of items with relatively extensive cleaning and editing is as follows: For two questions about how radiologists spend their time, answers to subparts were supposed to total to 100 percent. Actual totals were computed, and it was found that in the vast majority of cases in which the entries did not total to 100 percent, the total was slightly below 100 percent. Consequently, if the recorded percentages totaled 95 to 99, all recorded percentages were checked against the paper questionnaire and any errors corrected. The data for all respondents were then edited using an algorithm the ACR has long used with items that are supposed to sum to 100 percent: recorded percentages are summed. If the sum is 80 percent to 125 percent, each percentage is divided by the sum, which makes the revised percentages total to 100 percent. If the sum is <80 percent or >125 percent, the responses are deemed too deficient to use and all responses are set to missing.

REFERENCES

Deitch CH, Chan WC, Sunshine JH, Shaffer KA. 1997. Profile of U.S. radiologists at Middecade: Overview of findings from the 1995 survey of radiologists. Radiology 202(1):69–77.

Dillman DA. 2000. Mail and Internet Surveys: The Tailored Design Method. 2nd ed. New York: Wiley. Pp. 150–153.


Sunshine JH, Cypel YS, Schepps B. 2002. Diagnostic radiologists in 2000: Basic characteristics, practices, and issues related to the radiologist shortage. American Journal of Roentgenology 178(2):291–301.

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

ANALYSES AND REPORTS ON RADIOLOGISTS PERFORMING MAMMOGRAPHY

The American College of Radiology provided the following tables to the Institute of Medicine Committee on Improving Mammography Quality Standards. The following list of column and row headings for each table reflects the type of information provided to the Committee. Actual data is omitted, but is available on request.


TABLE I Number of Radiologists with Various Breast-Imaging-Related Characteristics


Rows:

  1. All posttraining professionally active radiologists

  2. Radiologists who interpret any mammograms (number of mammograms >0)

  3. Radiologists with a fellowship in breast imaging

  4. ACR’s 2003 Survey of Radiologists)

  5. Radiologists who designated breast imaging as their secondary subspecialty

  6. Radiologists who spend more than 30 percent of time in breast imaging

  7. Radiologists who spend more than 50 percent of time in breast imaging

  8. Radiologists who interpret less than 480 mammograms per year

  9. Radiologists who interpret at least 480 mammograms per year

  10. Radiologists who interpret at least 1,000 mammograms per year

  11. Radiologists who interpret at least 2,000 mammograms per year

  12. Radiologists who interpret at least 5,000 mammograms per year

  13. Radiologists who do any nonmammo breast imaging (ultrasound biopsy, etc.)

  14. Radiologists who do any other breast imaging, but no mammograms

Columns:

  1. Unweighted number of responses

  2. Weighted number=number of radiologists in the United States who meet the definition, with standard deviation

  3. Weighted percentage of all radiologists, with standard error

TABLE II Combinations of Breast-Imaging-Related Characteristics


Rows: Same as Table I


Columns: Same as B through N


Note: Each cell indicates the percentage (and standard error) of those in the row who also meet the column definition. For example, this table will give the percentage of those who say breast imaging is their primary specialty who interpreted 2,000 mammograms a year, the percentage of those who did a breast imaging fellowship that now spend at least 30 percent of their clinical work time doing breast imaging, etc.

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

TABLE III Further Information About the Breast Imaging Activity of Those in Each Category, A Through N


TABLE IIIa Further Information on Mammography Activity and Other Breast Imaging Activity, by Radiologists’ Breast-Imaging-Related Characteristics


Rows: Same as Table I, definitions A through N


Columns:

  1. Unweighted number of responses (Repeat from Table I)

  2. Weighted number of radiologists (Repeat from Table I)

  3. Weighted percentage of all radiologists (Repeat from Table I)

  4. Percentage who interpret any mammograms

  5. Weighted average number of mammograms for those who interpret any

  6. 25th percentile, 50th percentile, and 75th percentile of number of mammograms for those who interpret any

  7. Overall average number of mammograms (not only for those who interpret any)

TABLE IIIb Further Information on Nonmammography Breast Imaging Activity, by Radiologists’ Breast-Imaging-Related Characteristics


Rows: Same as Table I, definitions A through N


Columns:

  1. Percentage doing any nonmammography breast imaging

  2. Average number of types (of the ones listed below) of nonmammography breast imaging done

  3. Percentage who do each of the following types of nonmammography breast imaging, with standard error

    1. ultrasound-guided breast biopsy

    2. stereotactic breast biopsy

    3. localizations for surgical breast biopsy

    4. fine needle aspiration (FNAC)

    5. computer aided detection (CAD)

    6. full-field digital mammography

    7. breast magnetic resonance imaging (MRI)

TABLE IV Geographic Variation


Rows:

National total

4 Census regions

1=Northeast

2=Midwest

3=South

4=West

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

9 Census divisions

1=New England

2=Mid-Atlantic

3=East North Central

4=West North Central

5=South Atlantic

6=East South Central

7=West South Central

8=Mountain

9=Pacific

Columns:

  1. Percentage of all radiologists in the area who interpret any mammograms, with standard error

  2. For those who interpret any mammograms, average number of mammograms

  3. 25th, 50th, and 75th percentiles of number of mammograms, for those who interpret any

  4. Overall average number of mammograms (not only for those who interpret any)

  5. Number of radiologists interpreting mammograms per 10,000 women age 40 and older in area

  6. Percentage of all radiologists in the area who do any nonmammography breast imaging

TABLE V Information by Degree of Urbanness of Location


Rows: Same as Table I, definitions A through N


Columns:

For each of the following degrees of urbanness:

  • All locations

  • Large metro main city

  • Large metro suburb

  • Small metro main city

  • Small metro suburb

  • Nonmetro

Each of the following columns:

  1. Percentage of radiologists who interpret any mammograms in each location type, with standard error

  2. Average number of mammograms for those who interpret any mammograms, with standard error

  3. Number of radiologists who interpret mammograms, per 10,000 women age 40 or older

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

TABLE VI Age Distribution


TABLE VIa Information by Age


Rows: Same as Table I, definitions A through N


Columns:

For each of the following age categories:

  • All ages

  • Ages <45

  • Ages 45–54

  • Ages 55–64

  • Ages 65+

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

  3. Average number of mammograms for those who interpret any mammograms, with standard error

TABLE VIb Number and Percentage of Radiologists Who Interpret Mammograms and Mammography Volume, by Age


Rows:

Each of the following age categories:

  • All ages

  • Ages <45

  • Ages 45–54

  • Ages 55–64

  • Ages 65+

Columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

  3. Average mammograms by those who interpret any mammograms, with standard error

  4. 25th, 50th, and 75th percentile of number of mammograms, for those who interpret any

TABLE VII Demographics


TABLE VIIa Number and Percentage of Radiologists Who Interpret Mammograms, by Practice Type


Rows: Same as Table I, definitions A through N

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

Columns:

For each practice type:

  • All practice types

  • Academic practice

  • Nonacademic multispecialty practice

  • Nonacademic private radiology practice

  • Solo practice

  • Nonacademic government practice

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

TABLE VIIb Number and Percentage of Radiologists Who Interpret Mammograms, by Site(s) Served by Practice


Rows: Same as Table I, definitions A through N


Columns:

For each type of practice setting in which radiologist works, or types of settings the radiologist’s practice serves:

  • All settings

  • Hospitals only

  • Nonhospital sites only

  • Both

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

TABLE VIIc Number and Percentage of Radiologists Who Interpret Mammograms Overall, for Females, for Those Who Work Full-Time, and for Those Who Are Board Certified


Rows: Same as Table I, definitions A through N


Columns:

For each of the following:

  • All radiologists

  • Male versus female radiologists

  • Full-time versus part-time radiologists

  • Radiologist board certified or not

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
  1. Percentage of radiologists who interpret any mammograms, with standard error

TABLE VIId Number and Percentage of Radiologists Interpreting Mammograms, by Practice Size


Rows: Same as Table I, definitions A through N


Columns:

For each of the following practice size categories:

  • All sizes

  • 2 to 4

  • 5 to 7

  • 8 to 10

  • 11 to 14

  • 15 to 29

  • 30 and more

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

TABLE VIIe Number and Percentage of Radiologists Interpreting Mammograms, by Self-Reported Enjoyment of Working as a Radiologist


Rows: Same as Table I, definitions A through N


Columns:

  1. Average (mean) enjoyment score (Scores are: enjoy very much=2; enjoy somewhat=1; etc.), with standard error

For each of the following enjoyment scores in Q9:

  • All scores

  • Enjoy very much

  • Enjoy somewhat

  • Neither like nor dislike

  • Dislike somewhat or very much

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

TABLE VIII Percentage Performing Mammograms and Number of Mammograms Performed, by Gender


Rows: Same as Table I, definitions A through N

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

Columns:

For each of the following categories:

  • All

  • Male

  • Female

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

  3. Average mammograms by those who interpret any mammograms, with standard error

  4. Estimated total number of mammograms

TABLE IX Number and Percentage of Radiologists Who Want More or Fewer Hours of Work and Amount of Increase/Decrease in Hours Desired


Rows: Same as Table I, definitions A through N


Columns:

For each of the following categories:

  • Those who want their work and income increased

  • Those who want their work and income decreased

Each of the following columns:

  1. Number of radiologists who interpret any mammograms

  2. Percentage of radiologists who interpret any mammograms, with standard error

  3. Average desired percentage change in workload

TABLE X Work Status by Gender and 5-Year Age Group


Rows:

Each of the following age categories:

  • <35

  • 35–39

  • 40–44

  • 45–49

  • 50–54

  • 55–59

  • 60–64

  • 65–69

  • 70–74

  • 75+

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×

Columns:

For each gender category:

  • All

  • Male

  • Female

And for each of the following work status categories:

  • In residency training

  • In fellowship training

  • Working full-time in radiology

  • Working part-time in radiology

  • Not working in radiology

The following column:

  1. Estimated number (weighted count) of U.S. radiologists in this category

NOTE: This is the only table that uses all survey responses, including trainees and retirees.

Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 189
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 190
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 191
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 192
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 193
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 194
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 195
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 196
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 197
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 198
Suggested Citation:"Appendix A: ACR Survey Methods and Analysis." Institute of Medicine and National Research Council. 2005. Improving Breast Imaging Quality Standards. Washington, DC: The National Academies Press. doi: 10.17226/11308.
×
Page 199
Next: Appendix B: Society of Breast Imaging Survey »
Improving Breast Imaging Quality Standards Get This Book
×
Buy Paperback | $45.00 Buy Ebook | $35.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Mammography is an important tool for detecting breast cancer at an early stage. When coupled with appropriate treatment, early detection can reduce breast cancer mortality. At the request of Congress, the Food and Drug Administration (FDA) commissioned a study to examine the current practice of mammography and breast cancer detection, with a focus on the FDA’s oversight via the Mammography Quality Standards Act (MQSA), to identify areas in need of improvement. Enacted in 1993, MQSA provides a general framework for ensuring national quality standards in facilities performing screening mammography, requires that each mammography facility be accredited and certified, and mandates that facilities will undergo annual inspections. This book recommends strategies for achieving continued progress in assuring mammography quality, including changes to MQSA regulation, as well as approaches that do not fall within the purview of MQSA. Specifically, this book provides recommendations aimed at improving mammography interpretation; revising MQSA regulations, inspections, and enforcement; ensuring an adequate workforce for breast cancer screening and diagnosis; and improving breast imaging quality beyond mammography.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!