Background: Although systems strategies are effective in improving health care delivery, little is known about their use for cancer screening in U.S. primary care practice.

Methods: We assessed primary care physicians' (N = 2,475) use of systems strategies for breast, cervical, and colorectal cancer (CRC) screening in a national survey conducted in 2007. Systems strategies included patient and physician screening reminders, performance reports of screening rates, electronic medical records, implementation of in-practice guidelines, and use of nurse practitioners/physician assistants. We evaluated use of both patient and physician screening reminders with other strategies in separate models by screening type, adjusted for the effects of physician and practice characteristics with multivariate logistic regression.

Results: Fewer than 10% of physicians used a comprehensive set of systems strategies to support cancer screening; use was greater for mammography and Pap testing than for CRC screening. In adjusted analyses, performance reports of cancer screening rates, medical record type, and in-practice guidelines were associated with use of both patient and physician screening reminders for mammography, Pap testing, and CRC screening (P < 0.05).

Conclusion: Despite evidence supporting use of systems strategies in primary care, few physicians report using a comprehensive set of strategies to support cancer screening.

Impact: Current health policy initiatives underscore the importance of increased implementation of systems strategies in primary care to improve the use and quality of cancer screening in the United States. Cancer Epidemiol Biomarkers Prev; 20(12); 2471–9. ©2011 AACR.

This article is featured in Highlights of This Issue, p. 2469

Screening for breast, cervical, and colorectal cancer (CRC) is widely recommended in clinical practice guidelines in the United States (1–7), although uptake varies by screening type (7). In 2008, more than 75% of eligible women were up to date with recommended mammography (8) or Pap testing (7). However, only 55% of eligible women and men were up to date with any of the recommended CRC screening modalities (9), including fecal occult blood testing, flexible sigmoidoscopy, or colonoscopy. Physician recommendation is one of the strongest predictors of patient receipt of screening (10), in part, because many screening tests require a referral. By recommending cancer screening to their patients, primary care physicians (PCP) play a central role in implementing the screening guidelines of major professional organizations.

The effectiveness of systems strategies, including clinical information systems, delivery system design innovations, and decision support strategies, in improving primary care delivery is well established (11–15). For example, use of patient and physician reminders has long and consistently been associated with increased uptake of cancer screening in meta-analyses and systematic reviews (16–20). Reminder systems, electronic medical records (EMR), decision support, and other systems strategies are widely used within some primary care settings, such as the Veterans Health Administration (21) and in many managed care organizations (22), but have not been widely adopted across the United States (23–25). A recent study reported that EMRs with clinical decision support were used in only 17% of U.S. patient visits (23) The extent to which reminders and other systems strategies have been adopted for cancer screening by primary care practices in the United States is largely unknown.

In this study, we used data from a national survey of PCPs to describe and explore the following: (i) the adoption of multiple systems strategies which may improve cancer screening performance and (ii) whether the use of systems strategies varies for breast, cervical, and CRC screening. Because of the strength of evidence about the importance of patient and physician reminders for improving screening (16–20), we also explored the relationship of other systems strategies with the adoption of reminder systems.

Data source

We used data from the 2006–2007 National Survey of Primary Care Physicians' Recommendations and Practices for Breast, Cervical, Colorectal, and Lung Cancer Screening (26). The survey used a split-sample design, in which one half of the sample was randomly assigned a questionnaire covering breast and cervical cancer screening, and the other half, a questionnaire covering CRC and lung cancer screening. The survey used the American Medical Association's Physician Masterfile as the sampling frame to identify a nationally representative sample of PCPs. Physicians who were younger than 76 years old, held an active license, and listed patient care as their major professional activity were eligible for the survey. We used a stratified random sample with 4 primary care specialties as sampling strata (i.e., general practice, family medicine, general internal medicine, and obstetrics/gynecology). Selection was proportional to the specialty's representation in the U.S. physician population.

Survey items were developed with a panel of PCPs, and the survey was cognitively tested with a convenience sample of 18 practicing PCPs to ask about understanding and interpretation of survey questions to ensure validity of their responses. The questionnaires were revised on the basis of the cognitive testing findings. A total of 2,478 physicians completed the survey with an American Association for Public Opinion Research response rate 3 (which includes physicians determined to be ineligible to participate in the survey in the denominator; ref. 27) of 68.4% and cooperation rate 3 (which excludes ineligible physicians from the denominator) of 74.2%. More information about the survey, including access to its questionnaires, is available at the National Cancer Institute website (26). Because we were interested in systems strategies to support cancer screening tests routinely endorsed in clinical practice guidelines (1–7, 28), we restricted the sample to the 99% of physicians who reported recommending Pap smear, mammography, or CRC screening (N = 2,475). We report findings separately for physicians who reported ordering or performing at least one Pap test (n = 1,111), mammogram (n = 1,209), or CRC screening test (n = 1,264) in a typical month.

Guiding framework and measures

The Chronic Care Model guided the choice of measures (11, 12). The model is a framework for patient-centered evidence-based care delivery in practice. It includes the domains of community resources, the health care organization, decision support, clinical information systems, delivery system design, and patient self-management support. Interventions within these domains are hypothesized to result in prepared and proactive practice teams and informed and activated patients, ultimately improving the productivity of patient health care teams and leading to improved quality of care and better patient outcomes. Our analysis focused on systems strategies, including clinical information systems, delivery systems design, and decision support.

Clinical information systems strategy measures included the use and type of patient and physician reminders that a patient is due for mammography, Pap test, and CRC screening. Reminder use was classified as patient reminders only, physician reminders only, reminders to both patients and physicians, and neither type of reminder. In addition, we measured physician receipt of performance reports of their rates of cancer screening, whether such reports compared their performance with other practitioners, and whether their compensation was affected by screening performance.

Delivery system design strategy measures in the main primary care practice included the type of medical record system (paper charts, partial EMRs, in transition from paper to full EMRs, full EMRs) and the number of nurse practitioners and/or physician assistants (NP/PA; 0, 1, and ≥2). Use of NP/PAs reflects the decision to hire multiple clinician disciplines to provide care. Medical record system and NP/PAs do not refer to systems introduced specifically for cancer screening, per se, but reflect generic systems adopted by the practice. These measures were included because of the evidence that EMR technology (29) and a team approach in which responsibility for screening activities is shared among other members of the practice, including NP/PAs (30–32), can improve screening practice.

Decision support strategy measures included primary care practice implementation of in-practice guidelines for cancer screening and whether these guidelines were available in electronic format at the point of care or at a desk or workstation.

Physician characteristics measured included age, gender, race, board certification, and medical school affiliation. Physician specialties were general practice/family medicine, internal medicine, and obstetrics/gynecology.

Practice characteristics measured were region of the country, practice size (solo practice, 2–5, 6–15, or ≥16 physicians), practice specialty composition (single specialty, multispecialty group, other), main primary care practice setting (physician-owned practice, large medical group, university hospital or clinic, hospital or clinic not affiliated with university, other), and percentage of patients insured by Medicaid (<5%, 6%–25%, 26%–50%, ≥51%).

Analyses

Physician and practice characteristics were assessed with descriptive statistics. Use of systems strategies was stratified by screening type (i.e., mammography, Pap test, CRC screening). Because of the strong evidence for the effectiveness of reminders in increasing cancer screening uptake (16–20, 33), we also evaluated the use of both patient and physician reminders with other systems strategies (i.e., medical record system, number of NP/PAs, in-practice guidelines for screening, and performance reports of cancer screening rates) by screening type. Use of both patient and physician screening reminders was the outcome in 12 separate multivariate logistic regression models that adjusted for the effects of physician age, specialty, geographic region, practice size, practice type, and percentage of patients insured by Medicaid. This approach allowed us to evaluate use of multiple evidence-based systems strategies in primary care practice while controlling for the effects of physician and practice characteristics previously reported to be associated with cancer screening (34, 35).

Results from the regression analyses are presented as predicted marginals. The predictive margins method (36) directly standardizes the outcome of each group to the covariate distribution of the population. Standardized results from these logistic models can be compared as percentages. A sample weight that accounts for the probability of selection as well as an adjustment for nonresponse by sample strata was assigned to each survey respondent. For questions that were identical in the 2 samples, we used the combined survey weights. For questions that were unique in one of the samples, we used the specific survey sample weights. The statistical software SUDAAN (37) was used to apply the sampling weights and incorporate the stratified survey design in the calculation of descriptive statistics and logistic regression results.

Physicians were mostly male, white, non-Hispanic, and board certified (Table 1). Most worked in relatively small (≤5 physicians) physician-owned single specialty practices. Most reported that less than one fourth of their patients were insured by Medicaid.

Clinical information systems

The use of clinical information systems strategies varied by screening type (Table 2). The majority of physicians reported using at least one type of reminder (i.e., patient or physician) for mammography and Pap testing, whereas most physicians reported that they do not use reminders for CRC screening. Physicians reported significantly higher use of both patient and physician reminders for mammography [27.2%; 95% confidence interval (CI), 24.6%–30.0%] and Pap testing (26.6%; 95% CI, 24.0%–29.3%) than for CRC screening (8.9%; 95% CI, 7.3%–10.9%).

Among physicians who reported using patient reminders, mailed (68.0%, 72.2%, and 56.1% for mammography, Pap test, and CRC screening, respectively) and telephone (38.6%, 41.8%, and 39.8% for mammography, Pap test, and CRC screening, respectively) reminders were the most commonly used. Among physicians reporting use of physician reminders, chart (54.5%, 53.0%, and 48.0% for mammography, Pap test, and CRC screening, respectively) and computerized reminders (48.0%, 51.1%, and 44.4% for mammography, Pap test, and CRC screening, respectively) were the most common types.

Use of performance reports of cancer screening rates also varied by screening type (Table 2), with significantly more physicians reporting these for mammography and Pap test screening (approximately 30%) than for CRC screening (12%). Among physicians in practices with reports about screening rates, those that allowed performance comparisons with other practitioners were more common for CRC screening than for mammography or Pap testing (data not shown). Use of performance reports of screening rates to affect compensation was rare for all screening types (<15%).

Delivery system design

The use of medical record systems was similar across types of cancer screening (Table 2). Most physicians used paper charts (approximately 60%), and fewer than 20% had full EMRs. Approximately 16% reported being in transition from paper to full EMR and the remaining 11% reported partial EMRs. Use of NP/PAs was also similar across screening type, and approximately half of PCPs reported that they worked with at least one NP/PA in their practice (Table 2).

Decision support

More physicians reported in-practice guidelines for CRC screening (61.5%) than for mammography (48.9%) or Pap testing (48.0%; Table 2). Of physicians with in-practice guidelines, approximately 60% had guidelines in an electronic format at their desk or workstation for all screening types. Less than one third reported guidelines in electronic format at the point of care (data not shown).

Overall, fewer than 10% of physicians reported using all systems strategies in their primary care practice (i.e., both patient and physician reminders, performance reports of cancer screening rates, EMRs, NP/PAs, and in-practice guidelines), ranging from 8.5% (95% CI, 7.2%–10.1%) for mammography to 7.7% (95% CI, 6.4%–9.3%) for Pap testing to 1.7% (95% CI, 1.1%–2.6%) for CRC screening.

Multiple systems strategies

Figures 1 to 3 illustrate the adjusted associations between individual systems strategies and use of both patient and physician reminders, by screening type. Physicians in practices with performance reports of screening rates were more likely to use both patient and physician reminders than were physicians in practices without reports, for all screening types (P < 0.05; Fig. 1). The type of medical record system was associated with use of both patient and physician reminders in adjusted analyses, with greater use among physicians in practices with full EMRs, followed by those in transition to EMR or with partial EMRs (Fig. 2). Physicians using paper charts were least likely to report using both patient and physician reminders (P < 0.05 for all screening types).

Physicians in practices that had implemented in-practice guidelines for screening were also significantly more likely to report using both patient and physician reminders, for all screening types (P < 0.05; Fig. 3). Use of both patient and physician reminders was not associated with the number of NP/PAs in the multivariate models.

Despite extensive evidence supporting the effectiveness of clinical information, delivery system design, and decision support strategies to improve patient care (11–15), we found that use of these strategies for breast, cervical, and CRC screening in U.S. primary care practices was relatively low. In our study, fewer than 10% of physicians reported using the comprehensive set of the systems strategies to support cancer screening, although more than half reported using at least one systems strategy. Because these systems strategies have been shown to improve delivery of other evidence-based preventive services, such as smoking cessation and chronic disease care (16), our findings are broadly applicable. Efforts to better understand the dissemination and implementation of effective systems strategies to improve health care delivery are needed (38, 39), particularly within the context of diverse and complex health care systems (40, 41).

We found that the use of clinical information system strategies, including patient and physician reminders and performance reports of cancer screening rates, varied by screening type. Although these strategies were used infrequently for all types of screening, they were more commonly used for mammography and Pap test screening than for CRC screening, as has been reported elsewhere (42). Screening for breast, cervical, and CRC in asymptomatic adults has been widely recommended in practice guidelines for more than a decade (1–7), yet uptake of CRC screening is much lower than for mammography or Pap testing (7–9). Several factors may contribute to the differences we observed in use of systems strategies and with cancer screening uptake more broadly. Multiple CRC screening modalities are currently recommended in major practice guidelines, whereas mammography and Pap testing have been the primary screening modalities recommended for breast and cervical cancer screening, respectively, for decades. Furthermore, each CRC screening modality has a different recommended screening interval. Finally, CRC screening has a shorter guideline history than breast and cervical cancer screening. Further development of clinical information systems strategies could aid efforts to increase CRC screening to the levels of other cancer screening tests, particularly because physician recommendation plays a central role in implementation of screening (10).

We found partial use of delivery system design strategies—NP/PAs and medical records systems-in U.S. primary care practices. About half of physicians work in practices with at least one NP/PA, consistent with other reports (43). Although our survey question was not explicit about the role of NP/PAs and teams in cancer screening, they have still been shown elsewhere to impact screening behavior (30, 44). Focused evaluation of the role of NP/PAs in cancer screening will be an important area for future research.

Adoption of EMRs is another general practice strategy for improving screening; physicians have different perceptions of what constitutes an EMR, but experts consider a fully functional EMR to include not only health information and test results but also capabilities for order-entry management and decision support (24). Prior systematic review has suggested that health information technology can improve quality of care, at least at benchmark institutions (45). A fully functional EMR can be used to identify patients eligible for cancer screening based on age and prior screening history as outlined in evidence-based guidelines, document discussions about screening between patients, physicians, and other providers (e.g., NP/PAs), and implement patient and physician reminders (46). The EMR could then be used to document screening results and ensure that patients are notified of findings. For patients with abnormal results, EMRs could be used to ensure that follow-up testing is scheduled, completed, and test results documented (15). The EMR could also generate reports comparing screening practices. However, in our study, approximately 60% of physicians worked in practices with paper charts, with the remainder with full EMR (14%–18%), in transition from paper to full EMR (16%), or with partial EMR (10%–11%). These estimates are consistent with other national surveys of physicians, suggesting that many physicians lacked fully functional EMRs during this period (24, 47). Fully functional EMR use in primary care practice has increased over time (23, 48), although small practices may continue to face challenges to their implementation (49).

Increasingly, government and other health care payers are recognizing the potential impact of systems strategies in improving medical practice. As part of the 2009 American Recovery and Reinvestment Act (ARRA), Medicare and Medicaid will offer incentive payments to hospitals and physicians for meaningful use of EMRs (50). Ongoing evaluation of EMR implementation and use will be important for understanding facilitators and barriers to the delivery of high-quality cancer screening and other care across the cancer control continuum, particularly in the synergistic context of national attention to establish patient-centered medical homes (51). Screening is necessary, but not sufficient alone, for early detection of cancer and improved patient outcomes. For patients with abnormal screening results, systems strategies to improve cancer screening may also facilitate efficient interaction among PCPs and specialists including gastroenterologists, radiologists, surgeons, and medical oncologists. Tracking patient care transitions and team communication have also been highlighted as a key issue in the development of meaningful use criteria for EMRs (51, 52). Because many patients in the United States receive screening services from multiple providers, tracking transitions and communication will remain a challenge, even with improved EMR capability in individual practices (53).

We also found that use of systems strategies was highly correlated, even after controlling for the effects of physician and practice characteristics, such as specialty, age, and practice size in multivariate analyses. Physicians in practices that generated performance reports of cancer screening rates or who had implemented in-practice guidelines for cancer screening were significantly more likely to use both patient and physician reminders for screening than were physicians in practices without these systems strategies. Again, this is consistent with other reports of the importance of multiple systems strategies to enable physician behaviors (54).

Despite the strengths of having data from a large nationally representative survey of PCPs with a high response rate, our study has some limitations. We did not have information about community resources, patient self-management support, organization of care, or patient activation, and as a result, could not examine all components of the Chronic Care Model. We could not assess whether physicians' actual screening recommendations and patient receipt of screening tests were guideline consistent and influenced by the presence of system strategies in the practice. Our survey cannot reflect the diversity and complexity of all primary care practices in the United States. Nonetheless, national surveys such as this one are important tools for summarizing key characteristics of primary care practice and identifying areas for more in-depth research, using rigorous application of both qualitative and quantitative methods (55, 56) Finally, our survey data were collected in 2007, and some aspects of primary care practice may have evolved since that time. However, to our knowledge, these are the only data which address systems strategies in relation to breast, cervical, and CRC screening in a large national sample of physicians and serve as a baseline for evaluating future efforts. In summary, despite evidence showing the effectiveness of systems strategies in improving cancer screening uptake, few PCPs are using them in their practices. Increased implementation of systems strategies in primary care is needed and will be important for improving cancer screening in the United States. The impact of national initiatives to promote the widespread implementation of information technology (HITECH; ref. 57) and patient-centered medical homes should be assessed in relation to their effect upon population health activities, such as cancer screening.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the National Cancer Institute. No potential conflicts of interest were disclosed.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
U.S. Preventive Services Task Force
. 
Screening for cervical cancer: recommendations and rationale
.
Am J Nurs
2003
;
103
:
101
9
.
2.
U.S. Preventive Services Task Force
. 
Screening for colorectal cancer: recommendation and rationale
.
Ann Intern Med
2002
;
137
:
129
31
.
3.
Pignone
M
,
Rich
M
,
Teutsch
SM
,
Berg
AO
,
Lohr
KN
. 
Screening for colorectal cancer in adults at average risk: a summary of the evidence for the US Preventive Services Task Force
.
Ann Intern Med
2002
;
137
:
132
41
.
4.
Smith
RA
,
Cokkinides
V
,
Eyre
HJ
.
American Cancer Society
American Cancer Society guidelines for the early detection of cancer, 2004
.
CA Cancer J Clin
2004
;
54
:
41
52
.
5.
U.S. Preventive Services Task Force
. 
Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement
.
Ann Intern Med
2009
;
151
:
716
26
.
6.
U.S. Preventive Services Task Force
. 
Screening recommendations for colorectal cancer: U.S. Preventive Services Task Force recommendation statement
.
Ann Intern Med
2008
;
149
:
680
2
.
7.
Smith
RA
,
Cokkinides
V
,
Brooks
D
,
Saslow
D
,
Brawley
OW
. 
Cancer screening in the United States, 2010: a review of current American Cancer Society guidelines and issues in cancer screening
.
CA Cancer J Clin
2010
;
60
:
99
119
.
8.
Centers for Disease Control and Prevention. Vital signs: breast cancer screening among women aged 50–74 years–United States, 2008
.
MMWR Morb Mortal Wkly Rep
2010
;
59
:
813
6
.
9.
Klanbunde
CN
. 
Trends in the Use and Quality of Colorectal Cancer Screening in the United States
.
In
:
NIH State-of-the-Science Conference: Enhancing Use and Quality of Colorectal Cancer Screening, February 2–4, 2010
,
Bethesda, Maryland
, pp.
35
9
. http://consensus.nih.gov/2010/colorectalabstracts.htm.
Accessed October 17
, 
2011
.
10.
Klabunde
CN
,
Schenck
AP
,
Davis
WW
. 
Barriers to colorectal cancer screening among Medicare consumers
.
Am J Prev Med
2006
;
30
:
313
9
.
11.
Glasgow
RE
,
Orleans
TC
,
Wagner
EH
,
Curry
SJ
,
Solberg
LI
. 
Does the Chronic Care Model serve also as a template for prevention?
Milbank Q
2001
;
79
:
579
612
.
12.
Coleman
K
,
Mattke
S
,
Perrault
PJ
,
Wagner
EH
. 
Untangling practice redesign from disease management: how do we best care for the chronically ill?
Annu Rev Public Health
2009
;
30
:
385
408
.
13.
Garg
AX
,
Adhikari
NKJ
,
McDonald
H
,
Rosas-Arellano
MP
,
Devereaux
PJ
,
Beyene
J
, et al
Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review
.
JAMA
2005
;
293
:
1223
38
.
14.
Anhang
Price R
,
Zapka
J
,
Edwards
H
,
Taplin
SH
. 
Organizational factors and the cancer screening process
.
J Natl Cancer Inst Monogr
2010
;
40
:
38
57
.
15.
Zapka
J
,
Taplin
SH
,
Anhang
Price R
,
Cranos
C
,
Yabroff
R
. 
Factors in quality care - the case of follow-up to abnormal cancer screening tests-problems in the steps and interfaces of care
.
J Natl Cancer Inst Monogr
2010
;
40
:
58
71
.
16.
Stone
EG
,
Morton
SC
,
Hulscher
ME
,
Maglione
MA
,
Roth
EA
,
Grimshaw
JM
, et al
Interventions that increase use of adult immunization and cancer screening services: a meta-analysis
.
Ann Intern Med
2002
;
136
:
641
51
.
17.
Mandelblatt
J
,
Yabroff
KR
. 
Effectiveness of interventions designed to increase mammography use: a meta-analysis of provider-targeted strategies
.
Cancer Epidemiol Biomarkers Prev
1999
;
8
:
759
67
.
18.
Yabroff
KR
,
Mandelblatt
JS
. 
Interventions targeted to patients to increase mammography use
.
Cancer Epidemiol Biomarkers Prev
1999
;
8
:
749
57
.
19.
Baron
RC
,
Rimer
BK
,
Breslow
RA
,
Coates
RJ
,
Kerner
J
,
Melillo
S
, et al
Client-directed interventions to increase community demand for breast, cervical, and colorectal cancer screening: a systematic review
.
Am J Prev Med
2008
;
35
:
S34
55
.
20.
Baron
RC
,
Melillo
S
,
Rimer
BK
,
Coates
RJ
,
Kerner
J
,
Habarta
N
, et al
Intervention to increase recommendation and delivery of screening for breast, cervical, and colorectal cancers by healthcare providers: a systematic review of provider reminders
.
Am J Prev Med
2008
;
38
:
110
7
.
21.
Department of Veterans Affairs. Veterans Health Information Systems and Technology
Architecture (VISTA). VistA-HealtheVet Monograph. July 2008;
http://www.va.gov/vista_monograph/.
Accessed October 17
, 
2011
.
22.
The HMO Cancer Research Network (CRN): Capacity, Collaboration, and Investigation; 2010
.
[cited 2011 Sep 12]. Available from:
http://www.crn.cancer.gov/.
23.
Romano
MJ
,
Stafford
RS
. 
Electronic health records and clinical decision support systems: impact on national ambulatory care quality
.
Arch Intern Med
2011
;
171
:
897
903
.
24.
DesRoches
CM
,
Campbell
EG
,
Rao
SR
,
Donelan
K
,
Ferris
TG
,
Jha
A
, et al
Electronic health records in ambulatory care: a national survey of physicians
.
N Engl J Med
2008
;
359
:
50
60
.
25.
Bodenheimer
T
,
Grumbach
K
. 
Electronic technology: a spark to revitalize primary care?
JAMA
2003
;
290
:
259
64
.
26.
National Survey of Primary Care Physicians' recommendations & practice for breast, cervical, colorectal, & lung cancer screening; 2011
. [cited 2011 Sep 4]. Available from: http://www.healthservices.cancer.gov/surveys/screening_rp/.
27.
The American Association for Public Opinion Research (AAPOR)
. 
Standard definitions: final dispositions of case codes and outcome rates for surveys
.
5th ed
.
Lenexa, KS
:
AAPOR
; 
2008
.
28.
Humphrey
LL
,
Helfand
M
,
Chan
BK
,
Woolf
SH
. 
Breast cancer screening: a summary of evidence for the U.S. Preventive Services Task Force
.
Ann Intern Med
2002
;
137
:
347
60
.
29.
Friedberg
MW
,
Coltin
KL
,
Safran
DG
,
Dresser
M
,
Zaslavsky
AM
,
Schneider
EC
. 
Associations between structural capabilities of primary care practices and performance on selected quality measures
.
Ann Intern Med
2009
;
151
:
456
63
.
30.
Klabunde
CN
,
Lanier
D
,
Breslau
ES
,
Zapka
JG
,
Fletcher
RH
,
Ransohoff
DF
, et al
Improving colorectal cancer screening in primary care practice: innovative strategies and future directions
.
J Gen Intern Med
2007
;
22
:
1195
205
.
31.
Naylor
MD
,
Kurtzman
ET
. 
The role of nurse practitioners in reinventing primary care
.
Health Aff
2010
;
29
:
893
9
.
32.
Pohl
JM
,
Hanson
C
,
Newland
JA
,
Cronenwett
L
. 
Analysis & commentary. Unleashing nurse practitioners' potential to deliver primary care and lead teams
.
Health Aff
2010
;
29
:
900
5
.
33.
Sequist
TD
,
Zaslavsky
AM
,
Marshall
R
,
Fletcher
RH
,
Ayanian
JZ
. 
Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial
.
Arch Intern Med
2009
;
169
:
364
71
.
34.
Klabunde
CN
,
Frame
PS
,
Meadow
A
,
Jones
E
,
Nadel
M
,
Vernon
SW
. 
A national survey of primary care physicians' colorectal cancer screening recommendations and practices
.
Prev Med
2003
;
36
:
352
62
.
35.
Yabroff
KR
,
Klabunde
CN
,
Yuan
G
,
McNeel
TS
,
Brown
ML
,
Casciotti
D
, et al
Are physicians' recommendations for colorectal cancer screening guideline-consistent?
J Gen Intern Med
2011
;
26
:
177
84
.
36.
Graubard
BI
,
Korn
EL
. 
Predictive margins with survey data
.
Biometrics
1999
;
55
:
652
9
.
37.
Research Triangle Institute
. 
SUDAAN language manual: release 9.0
.
Research Triangle Park, NC
:
Research Triangle Institute
; 
2004
.
38.
Glasgow
RE
,
Marcus
AC
,
Bull
SS
,
Wilson
KM
. 
Disseminating effective cancer screening interventions
.
Cancer
2004
;
101
:
1239
50
.
39.
Green
LW
,
Glasgow
RE
. 
Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology
.
Eval Health Prof
2006
;
29
:
126
53
.
40.
Weiner
BJ
,
Amick
H
,
Lee
SY
. 
Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields
.
Med Care Res Rev
2008
;
65
:
379
436
.
41.
Kitson
A
,
Harvey
G
,
McCormack
B
. 
Enabling the implementation of evidence based practice: a conceptual framework
.
Qual Health Care
1998
;
7
:
149
58
.
42.
Malin
JL
,
Kahn
K
,
Dulai
G
,
Farmer
MM
,
Rideout
J
,
Simon
LP
, et al
Organizational systems used by California capitated medical groups and independent practice associations to increase cancer screening
.
Cancer
2000
;
88
:
2824
31
.
43.
Oliveria
SA
,
Altman
JF
,
Christos
PJ
,
Halpern
AC
. 
Use of nonphysician health care providers for skin cancer screening in the primary care setting
.
Prev Med
2002
;
34
:
374
9
.
44.
Hudson
SV
,
Ohman-Strickland
P
,
Cunningham
R
,
Ferrante
JM
,
Hahn
K
,
Crabtree
BF
. 
The effects of teamwork and system support on colorectal cancer screening in primary care practices
.
Cancer Detect Prev
2007
;
31
:
417
23
.
45.
Chaudhry
B
,
Want
J
,
Wu
S
,
Maglione
M
,
Mojica
W
,
Roth
E
, et al
Systematic review: impact of health information technology on quality, efficiency, and costs of medical care
.
Ann Intern Med
2006
;
144
:
742
52
.
46.
Taplin
SH
,
Rollason
D
,
Camp
A
,
diDonato
K
,
Maggenheimer
E
. 
Imagining an electronic medical record for turning cancer screening knowledge into practice
.
Am J Prev Med
2010
;
38
:
89
97
.
47.
Dentzer
S
. 
One year after the stimulus, will we get health IT right?
Health Aff
2010
;
29
:
582
.
48.
Linder
JA
,
Ma
J
,
Bates
DW
,
Middleton
B
,
Stafford
RS
. 
Electronic health record use and the quality of ambulatory care in the United States
.
Arch Intern Med
2007
;
167
:
1400
5
.
49.
Rao
SR
,
Desroches
CM
,
Donelan
K
,
Campbell
EG
,
Miralles
PD
,
Jha
AK
. 
Electronic health records in small physician practices: availability, use, and perceived benefits
.
J Am Med Inform Assoc
2011
;
18
:
271
5
.
50.
Halamka
JD
. 
Making the most of federal health information technology regulations
.
Health Aff
2010
;
29
:
596
600
.
51.
Bates
DW
,
Bitton
A
. 
The future of health information technology in the patient-centered medical home
.
Health Aff
2010
;
29
:
614
21
.
52.
Ralston
JD
,
Coleman
K
,
Reid
RJ
,
Handley
MR
,
Larson
EB
. 
Patient experience should be part of meaningful-use criteria
.
Health Aff
2010
;
29
:
607
13
.
53.
Hesse
BW
,
Hanna
C
,
Massett
HA
,
Hesse
NK
. 
Outside the box: will information technology be a viable intervention to improve the quality of cancer care?
J Natl Cancer Inst Monogr
2010
;
40
:
81
9
.
54.
Holden
DJ
,
Jonas
DE
,
Porterfield
DS
,
Reuland
D
,
Harris
R
. 
Systematic review: enhancing the use and quality of colorectal cancer screening
.
Ann Intern Med
2010
;
152
:
668
76
.
55.
Nemeth
LS
,
Nietert
PJ
,
Ornstein
SM
. 
High performance in screening for colorectal cancer: a Practice Partner Research Network (PPRNet) case study
.
J Am Board Fam Pract
2009
;
22
:
141
6
.
56.
Creswell
JW
,
Klassen
AC
,
Plano
Clark VL
,
Smith
KC
,
for the Office of Behavioral and Social Sciences Research
. 
Best practices for mixed methods research in the health sciences
.
Bethesda (MD)
:
NIH
[cited 2011 Aug]. Available from
: http://www.obssr.od.nih.gov/mixed_methods_research.
57.
De Moor
G
,
O'Brien
J
,
Fridsma
D
,
Bean
C
,
Devlies
J
,
Cusack
CM
, et al
Policy brief on the current status of certification of electronic health records in the US and Europe
.
Stud Health Technol Inform
2011
;
170
:
83
106
.

Supplementary data