Abstract
Psychosocial constructs have been used to predict colorectal cancer screening and are frequently targeted as intermediate outcomes in behavioral intervention studies. Few studies have conducted analyses to adequately test construct validity. The psychometric analyses undertaken with U.S. populations of 16 theory-based, colorectal cancer screening items designed to measure five factors (salience-coherence, cancer worries, perceived susceptibility, response efficacy, and social influence) are an exception. The current investigation replicates previous work by examining factor validity and invariance in a random sample of Ontario, Canada residents. A survey instrument was administered to 1,013 Ontario male (49%) and female (51%) residents randomly selected by the Canada Survey Sample. Single-group confirmatory factor analyses (CFA) assessed data fit to the proposed five-factor model for males and females separately, and then a multigroup CFA evaluated if the factor structure was invariant for men and women. The five-factor model provided good fit for both males and females. Tests for factorial invariance between sexes, however, found mixed results. χ2 difference test was significant (P = 0.025); however, ΔRMSEA = 0.0001. Factor loadings were similar by sex except for two social influence items, with item frequency distributions suggesting an extreme response style, in females, on these items. Overall, the single-group and multigroup CFA results support factorial validity and partial invariance of the five-factor model first identified in the U.S. populations. The items can be used to evaluate and compare psychosocial correlates across U.S. and Canadian samples. Additional research is needed to show invariance for other ethnocultural and national subgroups. (Cancer Epidemiol Biomarkers Prev 2008;17(11):3279–83)
Introduction
Fecal occult blood testing (FOBT) and flexible sigmoidoscopy reduce colorectal cancer (CRC) morbidity and mortality in proportion to use (1-3). Despite this evidence and screening guidelines established in the United States since 1997 and in Canada since 2001, CRC screening (CRCS) prevalence is lower than desirable (4-6). Based on current U.S. estimates, 46.5% of males and 43.1% of females reported CRC testing with one of the three recommended tests (FOBT, flexible sigmoidoscopy, or colonoscopy) within recommended time intervals (7). CRCS prevalence is also modest in Ontario, Canada where recent estimates indicate that FOBT screening uptake was ∼9% (excluding individuals with a history of CRC, colonic polyps, or inflammatory bowel disease; ref. 8). Although this may be a status quo underestimation, it is considerably lower than current U.S. population estimates, indicating that 16.1% of males and 15.3% of females reported a home-based FOBT in the past year (7). Efforts are now under way to rapidly increase Canadian CRCS rates through population-based screening recruitment in multiple provinces. Survey data about the attitudes and norms of the Canadian population regarding screening are useful for designing the required recruitment interventions and examining whether associations between CRCS and psychosocial variables replicate in U.S. and Canadian populations. However, correlate patterns can only be compared if the factor structure of the survey items used remains invariant across groups.
Thus, empirical evidence regarding the factor structure and invariance of the survey items used is critical for establishing psychometric validity. If the factor structure is not equivalent across the groups assessed, findings regarding group differences will be biased due to measurement error. In addition, behavioral interventions based on survey data with poor construct validity may be irrelevant and ineffectual when used with target populations. To date, only three studies have examined the reliability and validity of scales designed to measure psychosocial factors associated with CRCS (9-11). Two of these studies had limited generalizability due to homogenous sample characteristics (9, 10). The third had a more diverse sample and replicated the factor structure established by Vernon and colleagues, showing invariance across African-American and Caucasian groups. Because the sample only consisted of U.S. residents, it remains unclear whether these scales, assessing psychosocial correlates of CRCS, can be validly used with non-U.S. populations who receive care in different health care contexts. According to best practices, the validity of instruments should be investigated for every new population (12).
In this study, we add to the validity literature by reexamining the factorial invariance of the 16 survey items developed by Vernon and colleagues, by assessing Canadian attitudes and norms toward CRCS. Whereas previous studies examined these survey items in a U.S. population of male automotive workers and in a primary care clinic population of African-Americans and Caucasians, the current study examines the factor structure in a randomly selected population of CRCS-eligible Ontario, Canada residents (N = 1,013; 50 years and above). We focused on replicating previous survey validation work (9, 11) in two ways: (a) by determining if the previously identified five-factor model for the 16 CRCS items fits data from a different national population and (b) by examining whether the five-factor model was equivalent across sex in this different population.
Materials and Methods
The Preventive Health Model
The survey investigated is based on five psychosocial constructs: salience-coherence, perceived susceptibility, response efficacy, cancer worries, and social influence. These constructs were, in turn, drawn from the Preventive Health Model, a self-regulation model shown to predict intention and behavior for CRCS (13, 14).
Setting
A standardized scripted protocol was administered to Ontario residents, aged 50 y and above, via random digit dialing based on sampling methodology of the Canada Survey Sample, a selection engine that generates random samples of residential telephone numbers (15). The Canada Survey Sample maintains a comprehensive list of all populated exchanges across Canada (and Ontario) and is updated regularly at 3-mo intervals. Detail on the random digit dialing and Canada Survey Sample methodology is presented elsewhere (15, 16). Altogether, 1,558 individuals, 50 y and above, were phone contacted to obtain a sample of 1,013 for a response rate of 69%, following a contact rate of 31%. The survey originated from Cancer Care Ontario, a planning and research organization that advises the provincial government on cancer care and informs health care providers. Ethical approval of the study was obtained from the Sunnybrook Health Sciences Centre and the University of Toronto.
Measures
Details about the 16 items measuring five constructs (salience and coherence, perceived susceptibility, response efficacy, cancer worries, and social influence) are described by Tiro and colleagues (11). Item wording, response options, and frequency distributions are available as Supplementary Data.
Data Analysis
χ2 tests assessed potential male-female differences for marital status and education. Cronbach's α was computed for the five scales. With respect to the following procedures, we replicated the analytic processes used by Tiro et al. (11). Single-group confirmatory factor analyses (CFA) assessed data fit to the proposed five-factor model for males and females separately. Several absolute and incremental fit indices were used to assess suitability of the proposed model. The χ2 test statistic indicates absolute model fit by comparing the observed covariance matrix with a covariance matrix from a perfectly fit model. The χ2 test is sensitive to sample size and requires normally distributed data for validity.
The 16 individual survey items were nonnormally distributed; therefore, in addition to the traditional χ2 statistic, we included the Satorra-Bentler χ2 test statistic because it includes a scaling correction for nonnormal sampling distributions (17). The ratio of the χ2 test statistic to the degrees of freedom is suggested as an alternative to the χ2 test, with values less than or equal to two indicating acceptable fit (18). The root mean square error of approximation (RMSEA), another measure of absolute model fit, measured the closeness of fit between an a priori model and the observed data. RMSEA values of <0.05 indicate good fit (19, 20) and 90% confidence intervals for the RMSEA were also calculated. The comparative fit index (CFI), an incremental fit measure, was also used to compare a target model with a more restricted baseline model, adjusted for sample size (21). CFI values of >0.90 indicate acceptable fit according to Kline (22), whereas Hu and Bentler suggest that values of ≥0.95 indicate good fit (19).
After finding acceptable fit of the five-factor model, two-group CFA was done to determine if the model was similar for men and women. For this objective, two models were created. The first model was unconstrained whereby all factor loadings, error terms, and variances were allowed to vary between male and female samples. The second model was constrained whereby all factor loadings were equal between the two groups. We calculated the χ2 difference between the unconstrained and constrained models to evaluate equality between the covariance structures. The difference test statistic follows a χ2 distribution and a nonsignificant test statistic indicates invariance across models (i.e., the survey items measure similar latent constructs across groups).
Results
More men than women reported being married or living as married and having completed a 4-year university education (Table 1). Cronbach's α coefficients were largely similar between men and women for all of the constructs except for social influence (Table 2).
Marital status and education for the total sample and stratified by sex from the Cancer Care Ontario–FOBT survey
. | Total (N = 1,013), % (n) . | All males (n = 498), % (n) . | All females (n = 515), % (n) . | χ2 (df), P . | ||||
---|---|---|---|---|---|---|---|---|
Marital status | ||||||||
Married/living as married | 65.0 (659) | 69.1 (344) | 61.2 (315) | χ2(2) = 7.82 | ||||
Divorced/separated/widowed/never married | 30.6 (310) | 26.5 (132) | 34.6 (178) | P = 0.0201 | ||||
Refused/not sure | 4.3 (44) | 4.4 (22) | 4.2 (22) | |||||
Education level | ||||||||
<High school | 14.5 (147) | 12.5 (62) | 16.5 (85) | χ2(4) = 10.37 | ||||
High school degree | 30.7 (311) | 28.3 (141) | 33.0 (170) | P = 0.0346 | ||||
Some college/university | 5.3 (54) | 4.8 (24) | 5.8 (30) | |||||
≥4-y university degree | 46.1 (467) | 50.4 (251) | 42.0 (216) | |||||
Refused/not sure | 3.4 (34) | 4. 0 (20) | 2.7 (14) |
. | Total (N = 1,013), % (n) . | All males (n = 498), % (n) . | All females (n = 515), % (n) . | χ2 (df), P . | ||||
---|---|---|---|---|---|---|---|---|
Marital status | ||||||||
Married/living as married | 65.0 (659) | 69.1 (344) | 61.2 (315) | χ2(2) = 7.82 | ||||
Divorced/separated/widowed/never married | 30.6 (310) | 26.5 (132) | 34.6 (178) | P = 0.0201 | ||||
Refused/not sure | 4.3 (44) | 4.4 (22) | 4.2 (22) | |||||
Education level | ||||||||
<High school | 14.5 (147) | 12.5 (62) | 16.5 (85) | χ2(4) = 10.37 | ||||
High school degree | 30.7 (311) | 28.3 (141) | 33.0 (170) | P = 0.0346 | ||||
Some college/university | 5.3 (54) | 4.8 (24) | 5.8 (30) | |||||
≥4-y university degree | 46.1 (467) | 50.4 (251) | 42.0 (216) | |||||
Refused/not sure | 3.4 (34) | 4. 0 (20) | 2.7 (14) |
Abbreviation: df, degree of freedom.
Coefficient α values found for each survey subscale in the Ontario sample
Psychosocial constructs . | Total (N = 1,013) . | All males (n = 498) . | All females (n = 515) . |
---|---|---|---|
Salience and coherence | 0.724 | 0.718 | 0.728 |
Cancer worries | 0.604 | 0.632 | 0.575 |
Perceived susceptibility | 0.666 | 0.682 | 0.648 |
Response efficacy | 0.598 | 0.563 | 0.628 |
Social influence | 0.548 | 0.606 | 0.487 |
Psychosocial constructs . | Total (N = 1,013) . | All males (n = 498) . | All females (n = 515) . |
---|---|---|---|
Salience and coherence | 0.724 | 0.718 | 0.728 |
Cancer worries | 0.604 | 0.632 | 0.575 |
Perceived susceptibility | 0.666 | 0.682 | 0.648 |
Response efficacy | 0.598 | 0.563 | 0.628 |
Social influence | 0.548 | 0.606 | 0.487 |
CFI fit indices indicate adequate fit for both men and women (0.931 and 0.930, respectively). RMSEA values and the associated 90% confidence intervals support good fit (Table 3). Examination of the LaGrange multiplier tests suggests including two error covariance terms ([cc_drthk: I want to do what my doctor or health professional thinks I should do about colorectal cancer screening and cc_famthk: Members of my immediate family think I should have colorectal cancer screening]; [cc_drdo: My doctor or health professional thinks I should have CRC screening and cc_famdo: I want to do what members of my immediate family think I should do about colorectal cancer screening]; refs. 23, 24). After including these two error covariances, the model fit improves in both sex subgroups with all CFI values >0.95 and RMSEA values <0.05. Adding shared error variance between survey items measuring the same construct (in this case social influence) is a common finding among scales measuring attitudes and beliefs (21, 24).
Single-group and multigroup CFA results of the 16 CRCS items for males and females (N = 1,013) from a population-based random sample of individuals residing in Ontario, Canada
Model . | n . | χ2 . | Scaling correction factor . | Mean-adjusted χ2 . | df . | P . | CFI . | RMSEA . | (90% CI) . | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Five-factor | ||||||||||||||||||
Males | 498 | 210.567 | 189.816 | 94 | 0.0001 | 0.931 | 0.045 | (0.036-0.054) | ||||||||||
Females | 515 | 212.037 | 187.906 | 94 | 0.0001 | 0.930 | 0.044 | (0.035-0.053) | ||||||||||
Five-factor and two correlated errors (famthink, drthink) and (famdo, drdo) two modifications: one following recommended in original article, the other one (Dr. Do Dr. Thk) | ||||||||||||||||||
Males | 272 | 170.926 | 154.955 | 92 | 0.016 | 0.955 | 0.037 | (0.027-0.047) | ||||||||||
Females | 291 | 179.811 | 160.493 | 92 | 0.021 | 0.949 | 0.038 | (0.028-0.048) | ||||||||||
1. Unconstrained | 1,013 | 350.738 | 1.11 | 315.489 | 184 | 0.0001 | 0.952 | 0.027 | (0.021-0.031) | |||||||||
2. Equal factor loadings | 1,013 | 394.074 | 1.13 | 349.191 | 195 | 0.0001 | 0.944 | 0.028 | (0.023-0.033) | |||||||||
3. Equal factor loadings with drdo constraint released | 1,013 | 381.670 | 1.13 | 338.848 | 194 | 0.0001 | 0.947 | 0.027 | (0.022-0.032) | |||||||||
4. Equal factor loadings with drdo and dr constraints released | 1,013 | 378.563 | 1.13 | 335.564 | 193 | 0.0001 | 0.948 | 0.027 | (0.022-0.032) | |||||||||
Model comparisons | N | χ2diff | Δdf | Δtest scaling correction factor | Mean-adjusted χ2diff | P | ΔRMSEA | |||||||||||
1 vs 2 | 1,013 | 43.336 | 11 | 1.41 | 30.742 | 0.001 | 0.001 | |||||||||||
1 vs 3 | 1,013 | 30.932 | 10 | 1.40 | 22.159 | 0.014 | 0.000 | |||||||||||
1 vs 4 | 1,013 | 27.825 | 9 | 1.46 | 19.011 | 0.025 | 0.000 |
Model . | n . | χ2 . | Scaling correction factor . | Mean-adjusted χ2 . | df . | P . | CFI . | RMSEA . | (90% CI) . | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Five-factor | ||||||||||||||||||
Males | 498 | 210.567 | 189.816 | 94 | 0.0001 | 0.931 | 0.045 | (0.036-0.054) | ||||||||||
Females | 515 | 212.037 | 187.906 | 94 | 0.0001 | 0.930 | 0.044 | (0.035-0.053) | ||||||||||
Five-factor and two correlated errors (famthink, drthink) and (famdo, drdo) two modifications: one following recommended in original article, the other one (Dr. Do Dr. Thk) | ||||||||||||||||||
Males | 272 | 170.926 | 154.955 | 92 | 0.016 | 0.955 | 0.037 | (0.027-0.047) | ||||||||||
Females | 291 | 179.811 | 160.493 | 92 | 0.021 | 0.949 | 0.038 | (0.028-0.048) | ||||||||||
1. Unconstrained | 1,013 | 350.738 | 1.11 | 315.489 | 184 | 0.0001 | 0.952 | 0.027 | (0.021-0.031) | |||||||||
2. Equal factor loadings | 1,013 | 394.074 | 1.13 | 349.191 | 195 | 0.0001 | 0.944 | 0.028 | (0.023-0.033) | |||||||||
3. Equal factor loadings with drdo constraint released | 1,013 | 381.670 | 1.13 | 338.848 | 194 | 0.0001 | 0.947 | 0.027 | (0.022-0.032) | |||||||||
4. Equal factor loadings with drdo and dr constraints released | 1,013 | 378.563 | 1.13 | 335.564 | 193 | 0.0001 | 0.948 | 0.027 | (0.022-0.032) | |||||||||
Model comparisons | N | χ2diff | Δdf | Δtest scaling correction factor | Mean-adjusted χ2diff | P | ΔRMSEA | |||||||||||
1 vs 2 | 1,013 | 43.336 | 11 | 1.41 | 30.742 | 0.001 | 0.001 | |||||||||||
1 vs 3 | 1,013 | 30.932 | 10 | 1.40 | 22.159 | 0.014 | 0.000 | |||||||||||
1 vs 4 | 1,013 | 27.825 | 9 | 1.46 | 19.011 | 0.025 | 0.000 |
Abbreviation: 90% CI, 90% confidence interval.
The multigroup CFA comparing fit between the constrained and unconstrained models found a significant test statistic (mean-adjusted χ2difference = 30.742; P = 0.001). Modification indices recommend releasing constraints for the social influence item (cc_drdo: My doctor or health professional thinks I should have CRC screening) and the salience-coherence item (cc_import: Having colorectal cancer screening is an important thing for me to do), although making these changes did not result in a nonsignificant test statistic (see Table 3). However, the difference in RMSEA values is ≤0.001 when comparing the constrained and unconstrained models. Further, comparison of the factor loadings across sex found a similar pattern of direction and magnitude. Examination of the item frequencies across males and females found significant differences in the response distributions with females more likely using the “strongly disagree” and “strongly agree” options of the Likert scale.
Discussion
Our findings from single-group CFA support the application of a five-factor model for the 16 items measuring psychosocial correlates of CRCS to a randomly sampled population of Canadian men and women. Our data provide evidence for partial factorial invariance across men and women with a lack of equivalence shown for the social influence items. Collectively, these findings suggest that salience and coherence, cancer worries, perceived susceptibility, and response efficacy are interpreted similarly by Canadian men and women. Thus, our data suggest that these items can be used to evaluate and compare psychosocial correlates across U.S. and Canadian samples.
Although the factor loadings for the social influence items were statistically different between men and women, the difference may not be clinically meaningful. According to Gregorich (25), our results may have two interpretations, notably (a) the social influence items are being interpreted differently by sex (i.e., not equivalent factor loadings) and (b) the factor loading estimates are biased due to extreme response style. The latter is suggested if one group favors using the extremes of the Likert scale (e.g., strongly agree/strongly disagree versus agree/don't know/disagree). The significant differences found in the empirical distribution of the CRCS items by sex, the small difference in RMSEA values across models, and the similar pattern but slightly different magnitude of factor loadings by sex support the latter hypothesis. Future research could use qualitative methods such as cognitive interviewing to uncover the reasons for differential response styles among men and women and determine whether the social influence items have different meanings by sex. Such a study is now being undertaken by several authors of this article. There is also a need for quantitative research to directly test the hypothesis that women show more extreme responses than men and, if so, to what degree in which subpopulations. Given the long history of breast and cervical cancer screening, it is possible that females exhibit a more extreme response style than males as, over time, they have processed more communication messages about cancer screening and prevention and thus may be more inclined to state strong agreement or disagreement. This process of generalization, from breast and cervical cancer screening to CRCS, may be especially true in populations where current levels of CRCS awareness are relatively modest.
In terms of reliability, four scales (salience and coherence, cancer worries, perceived susceptibility, and response efficacy) had internal consistency estimates above or nearly equal to the typical 0.70 acceptability criteria (26). As Cronbach's α is reactive to the number of items used per subscale, reliability could be increased if homogenous items were added, particularly because two subscales contain only two items (cancer worries and response efficacy; ref. 24). On the other hand, a countervailing value favors survey brevity as fewer items are typically associated with higher response rates in sampled populations. Regarding the social influence scale, which showed a Cronbach's α of 0.548, one possible reason for the lower value is that the scale represents a composite reflection of two subconstructs: motivations to comply with key referents and key referents observed as supportive or unsupportive of CRC screening. If efforts were directed at elevating the Cronbach's α of this scale, one strategy might be adding homogenous items to each subconstruct. From an intervention perspective, this would only be valuable if researchers could identify additional referents who influence screening behavior. Regardless of adding items, Cronbach's α for a social influence scale will mathematically be lower than other scales of comparable length because α is based on item-to-scale correlations and any sampled population will include normatively and nonnormatively driven individuals (e.g., people who respond versus do not respond to external pressure regarding performance of health behaviors and key referents who are versus who are not supportive of CRCS).
In conclusion, there are reasons, other than national boundaries, to hypothesize that a random Canadian CRC screening eligible sample might differ in attitudes toward CRC screening from U.S. samples. There are differences in the health care system, likely differences in individual knowledge and awareness, and in communication messages, emphasizing the importance of cancer screening and early detection. Thus, it cannot be taken for granted that items assessing attitudes toward CRCS, with evidential support of factorial validity and invariance across race and sex in a U.S. population, would show a similarly invariant factor structure in a Canadian population. Our findings are thus worth noting as they lay the foundation for using this survey with multinational populations to determine if factors influencing CRCS are similar for individuals living in different health care contexts.
Disclosure of Potential Conflicts of Interest
No potential conflicts of interest were disclosed.
Grant support: Canadian Institute for Health Research Operating Grant 107413 (P. Ritvo, R. Myers, M.L. Del Giudice, L. Pazsat, and L. Rabeneck).
Note: Supplementary data for this article are available at Cancer Epidemiology, Biomarkers & Prevention Online (http://cebp.aacrjournals.org/).
Acknowledgments
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.