Factorial Validity and Invariance of a Survey Measuring Psychosocial Correlates of Colorectal Cancer Screening among African Americans and Caucasians

Background: Psychosocial constructs are widely used to predict colorectal cancer screening and are targeted as intermediate outcomes in behavioral intervention studies. Reliable and valid instruments for measuring general colorectal cancer screening psychosocial constructs are needed; yet, few studies have conducted psychometric analyses. This study replicated a five-factor structure for 16 theory-based, general colorectal cancer screening items measuring salience and coherence, cancer worries, perceived susceptibility, response efficacy, and social influence. In addition, we examined factorial invariance across race and sex.

Methods: African American and Caucasian patients (n = 1,413) attending an urban, primary care clinic were included in this study. These individuals completed a baseline survey as part of a colorectal cancer screening intervention trial. Single and multigroup confirmatory factor analyses using maximum-likelihood estimation were done.

Results: The five-factor general colorectal cancer screening model provided excellent fit and was invariant across race-sex subgroups.

Conclusions: The findings of invariance across sex and race subgroups support the use of these scales to measure group differences. (Cancer Epidemiol Biomarkers Prev 2005;14(12):2855-61)

Randomized controlled trials have shown that screening for colorectal cancer with fecal occult blood testing and/or flexible sigmoidoscopy can reduce morbidity and mortality (1-5). Since 1997, when screening guidelines were first established, the prevalence of colorectal cancer screening has not increased substantially and remains underused in the general population (6, 7). Based on the 2000 National Health Interview Survey, 17.1% of respondents reported having an fecal occult blood testing in the past year, 33.9% reported undergoing an endoscopy (flexible sigmoidoscopy or colonoscopy) in the past 10 years, and 42.5% reported either test within the recommended time interval (7). Whereas use of fecal occult blood testing was similar for both men and women, use of endoscopy or either test was lower among women than men (31.1% versus 37.4%; ref. 7). African Americans reported lower screening rates of fecal occult blood testing, endoscopy, and either test within the recommended time interval than Caucasians (14.6% versus 17.6%, 29.7% versus 35.0%, and 37.8% versus 43.6%, respectively; ref. 7).

To adequately design behavioral interventions that promote the use of colorectal cancer screening, researchers need to identify perceptions and barriers to screening that are amenable to change. Sallis et al. (8) found that the development and validation of psychosocial measures used to predict many behavioral outcomes has received scant attention in the behavioral science literature. There is substantial variability in the operationalization of colorectal cancer screening psychosocial constructs and this variability may contribute to inconsistent patterns of association found between those constructs and colorectal cancer screening (9, 10). For example, nine studies found a positive association between perceived risk and colorectal cancer screening (11-19), and six studies found no association (20-25). Janz et al. (15) investigated two different measures of perceived risk (susceptibility to develop colorectal cancer and comparative risk) and found positive, null, and negative associations depending on the risk measure and the type of screening test. Vernon et al. (9) hypothesized that conflicting results in the literature were due to lack of standardization of measures across studies and recommended that researchers provide psychometric data on their measures.

To date, only two studies have examined the reliability and validity of scales designed to measure psychosocial factors associated with colorectal cancer screening (26, 27). Both studies may have limited generalizability due to the characteristics of the samples. The study populations were predominantly Caucasian, male, reported higher incomes, and had health insurance. We do not know whether the scales used in these studies show factorial invariance (i.e., whether scale scores of the factors are comparable when administered to different groups). Establishing factorial invariance or equivalence is necessary to accurately investigate group differences in mean scores and patterns of association with other variables. If scales are not equivalent, then findings about group differences or correlates are potentially biased and may mislead intervention researchers (28).

Colorectal cancer screening is the first cancer screening behavior recommended to both sexes and, thus, presents the first opportunity to look at sex differences in colorectal cancer screening psychosocial measures and screening use. To explore attitudinal differences between men and women, psychosocial measures that are equivalent across the sexes are needed. The growing diversity in culture, race, and ethnicity in the United States also highlights the need to investigate whether data from surveys are comparable across subgroups (29). Johnson et al. (30) found that questions asked on national health surveys, such as global health rating, disease labeling, health care access, and physical activity were interpreted differently by African Americans, Hispanics, and Caucasians and that the differences remained after controlling for gender, age, education, and income. To date, only Pasick et al. (31) and Champion et al. (32, 33) have investigated the extent to which survey items about mammography screening are reliable and valid across ethnic and cultural groups.

The primary goal of the current study is to examine the factor structure and invariance across race and sex subgroups using data from a survey administered to a diverse urban, primary care population participating in a behavioral intervention trial promoting colorectal cancer screening. The aims of this report are to extend research by Myers, Vernon, and colleagues (26, 34) in two ways:

1. Replicate the factor structure identified by the Vernon et al.(26) study for survey items referring to the group of colorectal cancer screening behaviors (hereafter called the general colorectal cancer screening measurement model) within each race-sex subgroup.

2. Examine whether the general colorectal cancer screening measurement model is invariant across race and sex subgroups.

Psychosocial constructs examined in this study were based on the Preventive Health Model, a self-regulation model that has been shown to predict intention and behavior for colorectal cancer screening (35, 36). The Preventive Health Model is a framework that draws from several theoretical models, including the Health Belief Model (37, 38), Social Cognitive Theory (39, 40), the Theory of Reasoned Action (41, 42), and Antonovsky's research on the sense of coherence in health behavior (43, 44). This study examined the following Preventive Health Model constructs in relation to general colorectal cancer screening: salience and coherence, perceived susceptibility, response efficacy, cancer worries, and social influence.

This research was undertaken as part of a grant from the National Cancer Institute (grant CA084140, Principal Investigator: R.E. Myers) to conduct a 4-year randomized behavioral intervention trial to increase colorectal cancer screening in a diverse, urban population attending a primary care clinic. The grant was approved by the institutional review board at Thomas Jefferson University in Philadelphia, Pennsylvania. For this psychometric analysis, we used baseline survey data from the intervention trial.

Male and female patients (n = 6,682) ages 50 to 74 years who visited an urban primary care clinic between January 1999 and July 2002 were initially identified. Patients who had died, had a history of colorectal neoplasms, had a colonoscopy in the 10 years before July 2002, received a flexible sigmoidoscopy test between January 1999 and July 2002, or had incomplete address information were excluded from the study (n = 838). The remaining participants (n = 5,844) were sent an introductory mailing that described the study and provided a form and postage-paid envelope allowing recipients to indicate their interest or refuse participation in the study. Of those, 10.7% refused to be contacted, 4.3% were found to be ineligible, and 2.4% could not be located. Eleven individuals who took part in a focus group that pretested messages for the intervention trial were also excluded from the study. Mathematica Policy Research, Inc., attempted to call the remaining 4,814 patients to complete a baseline telephone survey and discovered an additional 1,432 were ineligible and 803 could not be located. Overall, 1,589 patients of the remaining 2,579 individuals (61.6%) completed the entire telephone survey, 19 completed a brief mail survey, and 971 did not complete either survey. Among those who completed a telephone survey, 62 were dropped from the study due to contamination of the randomization process; therefore, the study sample for this report consisted of the remaining 1,546 respondents to the telephone survey.

The baseline survey was administered by telephone and included four questions to measure demographic characteristics as well as 16 items measuring five general colorectal cancer screening psychosocial constructs (salience and coherence, cancer worries, perceived susceptibility, response efficacy, and social influence). The general colorectal cancer screening constructs were initially developed in 1997 and involved extensive content analysis by cancer prevention and control experts at the University of California, Los Angeles, and at Fox Chase Cancer Center. Vernon et al. (26) evaluated the validity of the five general colorectal cancer screening constructs as well as self-efficacy and intention in a predominantly Caucasian random sample of 2,244 male automobile workers who were participating in The Next Step Trial, a randomized controlled intervention trial designed to increase colorectal cancer screening and change eating behaviors. The data set was randomly divided into two subsamples. Exploratory factor analysis was conducted on the first sample to identify the factor structure and confirmatory factor analysis was used to cross-validate the hypothesized measurement model. These analyses supported an adequately fitting, independent seven-construct model [comparative fit index (CFI) = 0.95]. The items measuring self-efficacy and intention were not included in the current study because items were modified to refer to specific colorectal cancer screening tests.

The 16 general colorectal cancer screening psychosocial items were measured on a five-point Likert scale ranging from “strongly disagree” (1) to “strongly agree” (5). Appendix 1 lists the survey items and variable names.

Salience and Coherence was defined as the perception that performing a health behavior is consistent with beliefs about how to protect and maintain health. Participants were asked three items about whether colorectal cancer screening made sense, was important, and would help them protect their health. A fourth item asked whether one would be healthy if he or she avoided colorectal cancer screening and was reverse coded. Cronbach's α for a six-item scale of this construct in The Next Step Trial was 0.91 (26).

Cancer Worries was defined as concerns about negative consequences of completing a behavior and included two items: “I am afraid of having an abnormal colorectal cancer screening test result” and “I am worried that colorectal cancer screening will show that I have colorectal cancer or polyps.” Cronbach's α in The Next Step Trial was 0.64 (26).

Perceived Susceptibility was defined as subjective personal risk for developing colorectal cancer or polyps and was measured by four items that asked whether an individual's chances of developing colorectal cancer or colorectal polyps was high, whether he or she was very likely to develop colorectal cancer or polyps, and whether he or she was at lower risk for colorectal cancer compared with other persons his/her age. Cronbach's α was 0.79 in The Next Step Trial (26).

Response Efficacy has been defined by Rogers (45) as beliefs that adopting a behavior will be effective in reducing disease threat. Two items were included: “When colorectal polyps are found and removed, colorectal cancer can be prevented” and “When colorectal cancer is found early, it can be cured.”

Social Influence was defined as perceived beliefs about and desire to comply with key references' attitudes toward colorectal cancer screening. Four items were included that addressed two key references—family and doctor or health professional. For the current study, we substituted health professional for friends [used in the Vernon et al. study (26)] as a key referent because several studies have shown physician recommendation to be one of the most important predictors of colorectal cancer screening adherence (13, 18, 22, 24, 46-54) and because we wanted to show that the influence of the family and health professional forms one factor as hypothesized by Ajzen and Fishbein (42, 55) and Cialdini and Trost (56).

Baseline data were collected on 1,546 study participants. Participants who reported Hispanic, Asian, Native American, “other,” or were missing data for their ethnicity (n = 133) were not included in this analysis. Analysis of item response patterns did not find any study participants who were missing responses to more than half of the general colorectal cancer screening items. For participants who were missing <50% of the items, mean values for each scale were calculated and imputed for the missing items. The remaining study participants (n = 1,413) were divided into four race-sex subgroups: Caucasian males (n = 274), Caucasian females (n = 291), African American males (n = 195), and African American females (n = 653). To examine whether respondents within each race-sex subgroup differed with respect to other demographic variables, we computed ANOVA to compare the four groups on age and χ² test statistics to compare them on marital status and education. To determine whether internal consistency reliability of the five scales differed across race and sex, Cronbach's α was computed.

Univariate skew and kurtosis as well as Mardia's normalized estimate of multivariate kurtosis were done to evaluate univariate and multivariate normality assumptions in the data. Single-group confirmatory factor analyses using EQS for Windows 6.1 (57) were done on each of the race-sex subgroups for the five-factor general colorectal cancer screening model to assess fit (aim 1; Fig. 1). Several tests were used to evaluate the fit of the proposed measurement models. The χ² test statistic was used to evaluate whether the covariance matrix created by the hypothesized model was significantly different from the actual or observed covariance matrix. Because the χ² test statistic is sensitive to large sample sizes, we also examined incremental and absolute fit indices as recommended by Hu and Bentler (58). Incremental fit indices, like the CFI, measure the proportionate improvement in fit by comparing a target model with a more restricted baseline model. Absolute fit indices like root mean square error of approximation assess how well an a priori model reproduces the sample data. According to Hu and Bentler (58), acceptable fit is indicated when the CFI value is ≥0.95 and the root mean square error of approximation value is <0.07.

If fit was acceptable, then multigroup confirmatory factor analysis was done to determine if the five-factor model was invariant across the four race-sex groups (aim 2). Two models were created. The first model was unconstrained and allowed all factor loadings, variances, and error variances to vary between groups. The second model set all factor loadings to be equal across the four groups. We conducted a global test of equality of covariance structures that computed the χ² difference value versus the degrees of freedom (59, 60). This difference test statistic follows a χ² distribution. If the difference in the χ² test statistics between the two models was not significant (P > 0.05), we could conclude that invariance was supported (i.e., the survey items measured latent constructs similarly across groups).

The race-sex subgroups differed with respect to marital status and educational attainment (Table 1). Specifically, African American females were less likely to be married than the other groups. A higher percentage of Caucasians of both sexes reported greater than a high school education than their African American counterparts.

Cronbach's α coefficients were fairly consistent across subgroups for salience and coherence (Table 2). However, African Americans responded to the cancer worry and perceived susceptibility items less consistently (i.e., the scales had lower inter-item correlations) than Caucasians, whereas the opposite was true for the response efficacy and social influence items.

For the 16 items, univariate skew estimates ranged from −6.47 to 0.96 with an average value of −1.11. The univariate kurtosis estimates ranged from −1.77 to 53.17 with an average value of 3.53. Mardia's normalized estimate of multivariate kurtosis ranged from 25.57 for White males to 59.61 for Black females. Based on these estimates, we report the Satorra-Bentler χ² statistic and the traditional χ² statistic because the former incorporates a scaling correction for nonnormal sampling distributions (61).

Overall, the five-factor general colorectal cancer screening model provided excellent fit to the data for African American males (Table 3).⁵

Less ideal fit was found initially for the other race-sex subgroups. Although the Satorra-Bentler χ² statistics were expected to be significant due to the large sample size among the subgroups (62), the CFI values were <0.95, indicating less than ideal fit for some of the variables. Therefore, we examined the modification indices across subgroups to see how fit may be improved and found large χ² values for two error covariances ([cc_famthk, cc_famdo] and [cc_famthk, cc_drthk]). This result indicated shared error variance among scales measuring the same construct that is a common finding among scales measuring attitudes and beliefs (60). We respecified the five-factor model by adding two error covariances and found improved fit to the data for all race-sex subgroups (Table 3). All of the CFIs were 0.95 or higher and the root mean square error of approximation values ranged from 0.000 to 0.051.

The multigroup confirmatory factor analysis of the five-factor general colorectal cancer screening measurement model showed invariance of the factor structure across the four race-sex subgroups when the constraint for the perceived susceptibility item, rcc_low, was released (Table 3). The χ² difference statistic was not significant between the factor loading constrained and the unconstrained models (χ² difference = 40.42, P = 0.097; ref. 30).

Findings reported here for the general colorectal cancer screening survey items support the use of Preventive Health Model–based colorectal cancer screening measures. Further, the confirmatory factor analyses provide strong evidence that the underlying latent constructs are similar for African Americans and Caucasians of both sexes. Only one perceived susceptibility item (rcc_low) seemed to load differently across the groups. This item had much lower factor loadings among African Americans than Caucasians. This finding was corroborated by the observed lower Cronbach's α scores of the perceived susceptibility scale for African American males and females. The Preventive Health Model–based measures did well among all the race-sex subgroups although the groups were significantly different with respect to demographic characteristics. Thus, the psychometric analyses we conducted support using these general colorectal cancer screening survey items to identify group differences. Further, the strong factor loadings of physician influence items support that it is important to measure the effect health care providers may have on patient's decision making about colorectal cancer screening.

Preliminary research by Janz et al. (15) suggested that African Americans have lower awareness about colorectal cancer screening, and, as a result, may respond differently to questions about attitudes, feelings, and intentions regarding colorectal cancer screening. Our findings of invariance do not support Janz et al.'s hypothesis. Using these validated measures of salience and coherence, cancer worries, perceived susceptibility, response efficacy, and social influence may help researchers identify predictors of colorectal cancer screening and resolve the inconsistent patterns of association described in past studies (9, 10). The measures may also be used to evaluate the effectiveness of behavioral interventions aimed at influencing these psychosocial constructs. Further, these measures can be used to examine whether or not there are differential patterns of association and intervention efficacy across race and sex subgroups.

It is important to develop and validate psychosocial measures that are invariant because it is the only way to accurately investigate group differences in correlate patterns and intervention effectiveness. If interventions are shown to be equally effective in influencing key psychosocial constructs associated with colorectal cancer screening for different races and sexes, then such interventions could be used to help promote colorectal cancer screening in a wider audience. Additional research on survey items that refer to specific colorectal cancer screening tests (e.g., fecal occult blood testing or flexible sigmoidoscopy) is needed because it may affect how researchers design intervention messages. For example, if items measuring perceived barriers do not produce separate test-specific factors (i.e., persons embarrassed about fecal occult blood testing are also likely to report embarrassment about sigmoidoscopy), then health educators would not need to send separate messages to overcome those barriers (e.g., how the benefits of both tests outweigh temporarily feeling awkward).

A limitation of this study is the small number of variables (e.g., two to four items) that were used to operationalize each psychosocial construct. This is probably why the reliability of the five confirmed scales was lower than the 0.7 level recommended in the psychology literature (63). This circumstance did not hinder our ability, however, to achieve acceptable fit for the confirmatory factor analysis on general colorectal cancer screening items. In addition, although our sample was large (n = 1,413), it was not equally distributed among the race-sex subgroups. Relative to the African American female group, all of the other race-sex groups were substantially smaller. Two Monte Carlo studies have shown that when there are three or more indicators per factor and the sample sizes are >200, the frequency of an improper solution is rare (64, 65). We also do not know how declining to participate in the study may have affected our results.

In summary, our study is an important contribution to the scant literature on the validation of psychosocial constructs that are hypothesized to be associated with cancer screening behaviors. We replicated the factor structure of five psychosocial factors regarding general colorectal cancer screening (salience and coherence, cancer worries, perceived susceptibility, response efficacy, and social influence) first developed by Vernon, Myers, and colleagues (26, 34). Our findings support that these scales are invariant for African Americans and Caucasians. Future research should examine whether the general colorectal cancer screening survey items are also invariant among ethnic groups whose first language is not English (e.g., Hispanics and Asians) as well as studies examining whether the patterns of association between colorectal cancer screening and these constructs are similar across race and sex.