An Exploration of Changes in the Measurement of Mammography in the National Health Interview Survey

Population surveys are an important means of monitoring the health of a population (1). Data obtained from regularly administered population surveys can be used to determine whether health behaviors and outcomes are improving, worsening, or staying the same over time, thus providing useful information to develop public health priorities, policies, and programs (2). However, the quality of data obtained from population surveys can be compromised by both survey developers who change questions and survey respondents who provide inaccurate information.

Previous research has shown that changes to question wording can result in differences in the responses obtained (3–5). For example, changes in the assessment of physical activity in the National Health and Nutrition Examination Survey resulted in a 20-percentage point increase in the proportion of the population reporting no leisure-time physical activity (6). Similarly, a decrease of 3.5 percentage points in the proportion of women reporting lifetime mammography in the Behavioral Risk Factor Surveillance Survey was observed when the definition of mammography provided before the question was changed (7).

Response biases exhibited by survey respondents have been thoroughly documented. It is now widely accepted that completion of several mental tasks is required to provide an answer to a survey question (8). The predominant model of the question response process posits that individuals engage in four types of mental tasks: question comprehension/interpretation, memory retrieval, judgment formation, and response editing (3, 8). Cognitive and motivational factors may impact the question response process and result in biased responses (9). Remembering an event to have occurred more recently than it did (telescoping) during the memory retrieval phase is an example of a cognitive factor contributing to biased responses (10). Providing an untrue or incorrect response to maintain a positive self-presentation (social desirability) is an example of a motivational factor (9). There is some literature to suggest that response biases are more likely to occur among members of racial/ethnic populations (9).

In the United States, self-reported data from the National Health Interview Survey (NHIS), an annual household interview survey assessing the health of the civilian, noninstitutionalized population, is used to monitor progress toward national Healthy People objectives for cancer (https://www.healthypeople.gov/2020/topics-objectives/topic/cancer/objectives). Every 2 to 3 years, the NHIS includes a cancer screening supplement (11). In 2011, a one-part mammography history question was introduced: “Have you had a mammogram in the past 12 months?” (12). Before 2011, women were first asked whether they had ever had a mammogram, and if they responded yes, were asked to provide the date of their most recent mammogram. This change in question wording from a two-part mammography question to a one-part question roughly coincided with the issuance of age-specific recommendations for breast cancer screening from the United States Preventive Services Task Force (USPSTF; ref. 13; Fig. 1).

At least two studies [by Pace and colleagues (ref. 14) and Fedewa and colleagues (ref. 15)] used data from the NHIS to explore the impact of the 2009 guideline change on mammography among women ages 40 and older but came to somewhat different conclusions. Pace and colleagues (14) compared reported mammography use from the two-part question that was administered in 2008 and the one-part question that was administered in 2011. They found a significant 2.0% increase in mammography among women ages 40 and older, but 0.8%, 2.2%, and 1.1% increases within specific age groups (40–49, 50–74, and 75+, respectively) were not significant. Fedewa and colleagues (15) compared responses with the same two-part mammography history question that was administered in the 2008 and 2013 NHIS. They found no changes overall in any age group, and nonsignificant changes for women ages 40–49, 50–64, and 75+ indicated decreases rather than increases (−1.9%, −3.3%, and −1.7%, respectively).

Inconsistencies in the direction of the nonsignificant results of these two studies motivated our primary research question that women may be responding differently to the two types of mammography questions. Availability of both questions in the 2013 NHIS enables an exploration of whether the changes in mammography reported by Pace and colleagues (14) are due to differential responses to the two questions rather than true differences in screening behaviors. There are two key objectives for this analysis: To investigate whether question wording leads to different responses in past-year mammography and to discuss implications for estimating trends and disparities research. Using multiple years of the NHIS, we first attempt to replicate the findings of Pace and colleagues (14) and Fedewa and colleagues (15) to confirm that population-level estimates of past-year mammography were higher in years when the one-part question was asked than in years when the two-part question was asked. We then use the 2013 NHIS, which asked women both the one-part and two-part mammography questions, to see whether measurement error explains the patterns observed. Specifically, we examined: (i) whether reports of past-year mammography would be higher on the one-part question as compared with the two-part question when both were asked in the same 2013 survey; (ii) differences in the proportion of women (Hispanic and non-Hispanic Asian, black, and white) ages 40–49 and 50–74 inconsistently reporting past-year mammography; and (iii) whether any racial/ethnic differences observed in unadjusted 2013 comparisons persisted when controlling for other indicators of social disadvantage.

We hypothesized that women would be more likely to report past-year mammography when asked the one-part question than when asked the two-part question (hypothesis 1), possibly as a result of shallower processing of the one-part question and thus a greater tendency to telescope or engage in social desirability. Confirmation of this hypothesis would offer an explanation as to why Pace and colleagues (14) found a significant increase in mammography among women ages 40 and older when comparing responses from the two-part (2008) and one-part (2011) questions, and Fedewa and colleagues (15) did not when comparing responses from the same two-part question (2008 and 2013). We also examined the influence of racial and ethnic group membership on reports of past-year mammography. Neither Pace and colleagues (14) nor Fedewa and colleagues (15) explicitly focused on racial/ethnic differences in changes in mammography after the 2009 USPSTF breast cancer screening recommendation. However, previous studies have indicated that over-reporting of mammography may be more frequent among nonwhite women (16, 17). Health literacy, English proficiency, different cultural understandings of health care and time, and motivations to present oneself in a favorable manner have been offered as explanations for these response patterns (9). On the basis of this literature, we hypothesized that inconsistent responses to the questions administered in 2013 would be more likely among nonwhite women (hypothesis 2), but these racial/ethnic differences would be attenuated after controlling for other indicators of social disadvantage (hypothesis 3).

We analyzed data from breast cancer screening questions asked in the 2008, 2011, and 2013 sample adult core of the NHIS (http://www.cdc.gov/nchs/nhis.htm). The NHIS is the leading household interview survey used to assess the health of the civilian adult noninstitutionalized population in the United States. The nationally representative NHIS sample is drawn through a multistage area probability sampling plan that allows for random selection of households. Within each selected household, one adult per family is selected to complete the Sample Adult Core and relevant supplemental questionnaires. The analytic sample was comprised of 25,588 women (6,189 in 2008; 9,567 in 2011; 9,832 in 2013) between the ages of 40 and 74 who did not report a history of breast cancer. Women who did not provide complete responses to the one-part and two-part mammography questions were excluded from the respective analyses. Pace and colleagues (14) and Fedewa and colleagues (15) took a similar approach to missing data. Slightly more women had missing data on the two-part question (2.98%) than the one-part question (2.23%) in 2013.

As shown in Fig. 1, past-year mammography is measured in two different ways in the NHIS. The two-part question first asks women whether they had ever had a mammogram, and if so, when (“Have you ever had a mammogram”; “When did you have your most recent mammogram?”). For the second item, respondents are provided an open reference period and can answer in a variety of ways [month/year, unit/time (e.g., 18 months ago), selection of a time interval]. Responses are collapsed into a single categorical variable that indicates time since last mammogram (a year ago or less, more than 1 year but not more than 2 years, more than 3 years but not more than 5 years, over 5 years ago). This two-part question is asked every 2 to 3 years in a Cancer Control Supplement cofunded by the National Cancer Institute (NCI) and the Centers for Disease Control and Prevention (CDC; ref. 11). The second mammography history question is a one-part question funded by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) since 2011: “Have you had a mammogram during the past 12 months?” (18). In contrast with the two-part question, the one-part question has a defined reference period of 12 months, and respondents are simply asked to provide a response of yes or no.

To replicate the patterns reported by Pace and colleagues (14) and Fedewa and colleagues (15), we estimated the unadjusted percentage of women reporting mammography by age (40–49 and 50–74) and race/ethnicity (Hispanic, non-Hispanic White, non-Hispanic Black, non-Hispanic Asian, or non-Hispanic other) for the year before (2008) and the years after (2011–2013) the 2009 USPSTF recommendations. Bivariate χ² tests were used to examine temporal differences as a function of question wording. Our hypotheses regarding within-respondent consistency were tested using 2013 data only. To determine the extent of inconsistent responses, we ran crosstabulations to obtain point estimates of past-year mammography from the one-part and two-part questions, and conducted Rao-Scott second-order χ² tests to determine whether there were differences in unadjusted distributions of responses to the two questions overall (hypothesis 1) and by race/ethnicity (hypothesis 2). Finally, logistic regression models were used to determine whether racial/ethnic differences in inconsistent responses to the two questions held when controlling for other sociodemographic characteristics (hypothesis 3). Post hoc multinomial regression models provided additional details about correlates of specific types of inconsistent responses. Covariates in the regression models are indicators of social disadvantage (19) and include health insurance status (public, private/military, or other), education level (less than high school, high school graduate, some college/AA degree, BA or higher), usual source of care (one or more places, none), and survey language (English, non-English). All analyses are stratified by age groups consistent with the USPSTF recommendations (40–49, 50–74), and weighted to generate population-level estimates. SAS-callable SUDAAN was used to account for the complex sampling design of the survey.

As shown in Table 1, reports of past-year mammography were higher in years when the one-part question was asked (in 2011 and 2013) than when the two-part question was asked (in 2008 and 2013). When different questions were used to estimate changes in past-year mammography over time, a small but significant increase was found in past-year mammography among women ages 50–74 between 2008 (58.0%) and 2011 (60.6%, P = 0.04), and between 2008 (58.0%) and 2013 (60.9%, P = 0.03). These results appear to be driven by statistically significant increases among White women (2008 vs. 2011: P = 0.03; 2008 vs. 2013: P = 0.05). On the other hand, results from analyses comparing responses to the same questions in 2008 and 2013 did not show changes in past-year mammography for any individual age or racial/ethnic group.

Unadjusted prevalence estimates indicated that nearly 10% of women in the overall sample provided inconsistent responses to the two past-year mammography history questions asked within the 2013 NHIS. A greater percentage of women ages 50 to 74 (10.7%) provided inconsistent responses as compared with women ages 40–49 (6.5%; P < 0.0001; Table 2). Second-order Rao-Scott χ² tests indicated that responses to the two questions were significantly different among women ages 40–49 (χ² = 4,705.45; P < 0.0001) and among women ages 50–74 (χ² = 4,289.04; P < 0.0001): reports of past-year mammography were significantly higher for the one-part question, compared with those based on the two-part question, for all racial/ethnic groups in both age groups, with the exception of women of an “Other” racial/ethnic background ages 40–49, for whom the direction was reversed (Table 1).

In comparison with White women (9.5%), and without adjusting for population differences in social disadvantage, a significantly greater percentage of Black (14.1%), Hispanic (13.9%), and Asian (13.9%) women ages 50–74 provided inconsistent responses to the two mammography questions (P < 0.01). Although inconsistent responses were also more likely among Black and Hispanic women ages 40–49, these differences were not statistically significant. Across age and racial/ethnic groups, the most common type of inconsistent response occurred when women reported receiving a mammogram in the past 12 months when asked the one-part question and then reported a date beyond the past 12 months when asked the two-part question (Fig. 2).

Multivariable logistic regression models that explored the effects of race/ethnicity after adjusting for indicators of social vulnerability, including education, revealed that Black women ages 50–74 [aOR, 1.50; 95% confidence interval (CI), 1.16–1.93], and women ages 40–49 who reported being in poor health (aOR, 2.22; 95% CI, 1.09–4.52) had significantly higher odds of inconsistent responses (Table 2). Post hoc analyses revealed that these results held only for the reporting pattern that was the reverse of that described previously: They reported no past-year mammography when asked the one-part question but then reported that their last mammogram was within the past 12 months when asked the two-part question (Supplementary Table S1). In addition, women in both age groups without a usual source of care had significantly lower odds of inconsistent responses on the two past-year mammography questions, compared with those women with at least one usual source of care [40–49: aOR, 0.42; 95% CI, 0.21–0.85; 50–74: aOR, 0.42; 95% CI, 0.24–0.74; Table 3]. The post hoc multinomial regression analysis indicated that this pattern persisted only for the initial type of inconsistent response described previously: A response of yes to the one-part question but then reporting that the most recent mammogram occurred more than 12 months ago on the two-part question (Supplementary Table S1).

Population surveys like the NHIS are valuable tools for monitoring health behaviors and outcomes over time. This study demonstrates that the way questions about mammography use are asked impacts estimates derived from population surveys in ways that are explainable, if not fully predictable. Self-reported mammography is sensitive to question wording: Women were more likely to report past-year mammography when asked a one-part question with a 12-month reference period than when asked a two-part question that required generation of a specific date. Our analyses explained how two previously published studies (14, 15) that used NHIS data could report contradictory directions of insignificant results in self-reported mammography after the 2009 USPSTF breast cancer screening recommendation. We also investigated patterns of inconsistent responses to the two types of past-year mammography questions and identified sociodemographic factors associated with inconsistent responses. We discuss each of these in turn and highlight implications for estimating trends and conducting disparities research.

An apparent 3% increase in mammography after the 2009 USPSTF recommendation among White women ages 50–74 was observed when comparing responses with the two-part (2008) and one-part (2011–2013) mammography history questions. This increase did not appear when responses to the two-part question in 2008 and 2013 were compared. Furthermore, over-reports resulting from use of the one-part question in 2011 and 2013 may have concealed decreases in mammography among nonwhite women ages 50–74. Decreases of approximately 4% were observed, but were not significant, when responses to the two-part question in 2008 and 2013 were compared. Similarly, nonsignificant increases in past-year mammography observed among nonwhite women ages 40–49 when comparing responses with the 2011 and 2013 one-part question with responses to the 2008 two-part question were often exaggerated. These patterns highlight the importance of comparing responses to the same questions when examining changes within racial/ethnic groups across time.

Although our study examined consistency in self-reports of mammography, other researchers have looked at the accuracy of such reports. Previous validation studies comparing self-reported mammography with medical records have documented over-reporting of mammography (20), and one meta-analysis found that approximately 20% of self-reported mammograms could not be confirmed in medical records (16). Over-reporting has been found to be higher among nonwhite women (16, 17), which may conceal the magnitude of racial/ethnic disparities in mammography (16, 21). Our results expand upon these findings by showing that over-reports of mammography are likely to be higher when women respond to a one-part question that asks about screening behavior in the past 12 months than when they provide the date of their last mammogram in response to a two-part question. Taken together, the existing literature and the findings presented here suggest that consistent use of the two-part mammography history question is best for accurately tracking trends breast cancer screening over time, although estimates of disparities may still be artificially low due to differences in response patterns across racial/ethnic groups.

Nearly all women, irrespective of age or race/ethnicity, were more likely to report past-year mammography when asked the one-part question. The inconsistent responses observed in the current study could be due to a number of factors affecting the survey response process (9). The most common type of inconsistent response was reporting a mammogram in the past 12 months when asked the one-part question, and then reporting a date prior to the past 12 months for the most recent mammogram when asked the two-part question. This pattern of results was indicative of forward telescoping bias, which occurs when an event is remembered as happening more recently than it did (4). Several studies have pointed to forward telescoping as an explanation for overreports of mammography (c.f. Burgess and colleagues; ref. 9). Our results suggest that telescoping is more likely to occur when answering the one-part question versus the two-part question.

Asking about mammography, a socially desirable health behavior, could have motivated affirmative responses to both the one-part and two-part questions as a means of presenting oneself in a favorable light (20). The simple yes/no response format for the direct question as well as the lack of follow-up questions may have made it easier for women to provide a socially desirable response to this question. However, there is no evidence to suggest that there is a difference in pressure to report that a mammogram occurred within the past 12 months. More likely, the two-part question increases the time available to recall when one's last mammogram occurred, thereby promoting more accurate reporting (10, 22).

The different supplemental definitions of mammography offered to women may provide an additional explanation for respondent differences. If women needed additional information after being asked the two-part question, they were told “A mammogram is an X-ray taken only of the breast by a machine that presses against the breast.” The definition provided after the one-part question was: “A mammogram is an X-ray of each breast to look for breast cancer.” On the basis of the differing definitions, it is not implausible that women responded honestly to what they interpreted as two different questions. This phenomenon would represent an example of an error in question comprehension and interpretation (3). There is some evidence from the Behavioral Risk Factor Surveillance Survey (BRFSS) to support this interpretation (7). Siegel and colleagues found that the percentages of women reporting lifetime mammography in the BRFSS decreased by 3.5 percentage points as a result of changing the wording preceding a question about lifetime mammography from “These next questions are about mammograms, which are X-ray tests of the breast to look for cancer” in 1990–1991 to “I want to ask you a few questions about a medical examination called a mammogram. A mammogram is an X-ray of the breast and involves pressing the breast between two plastic plates” in 1992 (7). The definition of mammography provided in the one-part question is very similar to the question used in the 1990–1991 BRFSS that led to inflated population mammography estimates. Unfortunately, we do not know how often definitions are used in a field survey environment.

Respondent age is another factor that could have contributed to inconsistent responses to the two mammography questions. Age-related cognitive decline has been identified as a concern for the validity of self-reported data (23), and a systematic review that examined utilization of health care services found that age was the demographic characteristic most commonly associated with inaccuracies in self-reported healthcare utilization (24). Across all racial/ethnic groups, we observed inconsistent responses among a larger proportion of women ages 50–74 as compared with women ages 40–49. Thus, it is probable that the older women in our sample were more prone to cognitive errors in question comprehension/interpretation or memory retrieval than the women ages 40–49.

When controlling for indicators of social disadvantage (including education and health insurance), both Black women ages 50 to 74, and women ages 40 to 49 in poor health, had higher odds of providing inconsistent responses, as compared with White women overall and women in excellent health, respectively. Women in these two groups were more likely to provide the less common type of inconsistent response: that is, reporting no mammogram in the past 12 months when asked the one-part question, but then indicating that they had had a mammogram during this time period when asked the two-part question. It has been suggested that cultural perceptions concerning time may differ among racial/ethnic groups (25), but more research is needed to explore how these cultural factors may influence recall processes. Failing to encode one's last mammogram into memory, and thus failing to recall that is occurred during the specified period, is another possible explanation (22). This phenomenon could be particularly likely for women in poor health who presumably are seeking care for numerous conditions, and thus having regular contact with the medical system, making receipt of a mammogram less distinct and thus more difficult to recall (3). Similarly, greater saliency and recall accuracy among women for whom getting a mammogram may require paying out of pocket, travel arrangements, or other hardships (24) may explain why women without a usual source of care had significantly lower odds of inconsistent responses.

Our findings are limited by the inability to compare responses to the mammography history questions with medical records. Record-check studies that compare self-reports with medical records are needed to determine which types of questions most accurately capture the desired data. Furthermore, we were unable to directly examine the cognitive, motivational, or other factors that may account for the inconsistent responses we observed.

This study makes two valuable contributions to the literature. We provide evidence that measurement error due to use of differently worded questions in different survey years can contribute to apparent, but inaccurate, changes in self-reported screening behavior over time. Because self-reports of mammography tend to be higher than documented in medical claims or records (16), the two-part mammography history question may be assumed to be more accurate than the one-part question that yielded higher estimates. There were no appreciable differences in missing data on the one-part and two-part mammography history questions. These findings have been submitted to the National Center for Health Statistics for consideration in the development of the next NHIS questionnaire. We also call attention to racial/ethnic differences in responses to survey questions, something that has been underexamined and underappreciated in comparative survey research (26).

Researchers should use equivalent questions to reduce the impact of measurement error, especially when assessing the impact of policy recommendations over time. Equivalent questions are particularly important for disparities-related research to minimize the risk of inconsistent responses, which are more likely among Black women, women with a usual source of care, and women in poor health. Agencies funding new questions on national surveys, particularly those added to allow for examination of policy or intervention impacts, should evaluate whether existing questions can provide the desired information. Finally, studies are needed to disentangle which combination of question characteristics (one-part vs. two-part, defined vs open reference period, definition mentioning breast cancer vs. not) influence these factors in a way that results in the most accurate responses.

No potential conflicts of interest were disclosed.

Conception and design: F.A. Gonzales, N. Breen, T. Yan, K.A. Cronin, S.H. Taplin, M. Yu

Development of methodology: N. Breen, T. Yan, K.A. Cronin, M. Yu

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): N. Breen

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): F.A. Gonzales, G.B. Willis, N. Breen, T. Yan, K.A. Cronin, S.H. Taplin, M. Yu

Writing, review, and/or revision of the manuscript: F.A. Gonzales, G.B. Willis, N. Breen, T. Yan, K.A. Cronin, S.H. Taplin, M. Yu

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): N. Breen

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.