Reliability and Validity of a Telephone Questionnaire for Estimating Lifetime Personal Sun Exposure in Epidemiologic Studies

Sun exposure is the major environmental risk factor for skin cancer (1). In the 1980s and later, epidemiologic studies improved our understanding of the relationship between personal sun exposure in different periods of life and melanoma and basal and squamous cell carcinomas (BCC and SCC) of the skin (2-6). Because of the evidence that risk of cutaneous melanoma, and probably BCC too, is related to pattern as well as amount of sun exposure (1), these components of exposure need to be reliably distinguished, adding to the complexity of measuring sun exposure. To meet these needs, we developed and used quantitative, questionnaire-based methods that measured individual sun exposure on working and nonworking days separately; simple addition gave an estimate of weekly exposure. Because questions addressed defined periods in life, each measure could be calculated within age intervals or summed over a lifetime (4, 5, 7).

Estimating sun exposure over a lifetime, however, depends on subjects' recall and is made difficult by the generally low salience of sun exposure and its variability in amount and frequency over time. To help reduce measurement error, we used a preinterview calendar for participants to record their personal lifetime residence and work history as a source of autobiographical, contextual memory cues to assist their recall in a face-to-face interview about whole-of-life sun exposure (4). We tested the reliability of this approach: an intraclass correlation coefficient (ICC) of 0.65 [95% confidence interval (95% CI), 0.48-0.78] for total lifetime exposure recalled twice at an interval of 5 years showed it to be reliable (8).

In 1996, we began a case-control study of the relationship between sun exposure and ocular melanoma (9, 10). Because ocular melanoma is rare and needed nationwide ascertainment to accrue sufficient cases in Australia, high costs and complex logistics were a barrier to a face-to-face interview. The solution was a telephone interview, but our questionnaire had to be shortened to retain interest and encourage careful consideration of each question in this setting (11). We report here on construction of the shortened questionnaire and a test of its reliability against the longer, face-to-face version. We added to it parallel measures of sun exposure in occupations and recreational activities; these also provide evidence of the validity of the telephone questionnaire in measuring the components of lifetime sun exposure.

This report is based on measurements of individual sun exposure made in three of four case-control studies of sun exposure done between 1988 and 2001 (Table 1). Data were first collected in a face-to-face interview about sun exposure for a case-control study of BCC in 1988 in Geraldton, Western Australia, and their reproducibility was tested at reinterview 5 years later in 190 subjects in a case-control study of SCC (8). The interview questionnaire sought, among other things, to quantify hours of sun exposure on working days and, separately, nonworking days in typical weeks in the warmer and cooler months of each year of life. Analyses were done, in 1996, of the data from 700 controls in the 1988 Geraldton study to assist in constructing a shortened, telephone questionnaire for an ocular melanoma study in 1997 to 1998 (10). We tested the reliability of this telephone questionnaire by reinterviewing 60 participants in the ocular melanoma study 15 months later, on average, using the 1988 face-to-face interview. Finally, we assessed the agreement of the working and nonworking day exposure measurements in the shortened telephone questionnaire with the parallel measurements of occupational and recreational sun exposure that we had added to it. This was done using data from the 1997 to 1998 ocular melanoma study and, for occupational and working day exposure only, a 2000 to 2001 case-control study of non-Hodgkin's lymphoma done using similar measurement methods (12).

Our original face-to-face questionnaire was used to ask 201 cases with BCC and 700 community controls age between 40 and 64 years to recall details of their lifetime sun exposure, birthplace, and pigmentary characteristics (4). The participants also completed a self-administered preinterview calendar to record their history of residential locations, schools, occupations, and number of working and nonworking days in an average week in each of the warmer and cooler months in each year of life. At interview, they indicated periods of life from age 8 years with reasonably stable levels of outdoor activity and the interviewer asked detailed questions for each period about the usual number of hours they spent outdoors not under any shade between 9 a.m. and 5 p.m. on working and on nonworking days in a typical week in the warmer and cooler months and in their summer holidays. Outdoor hours not under shade between 9 a.m. and 5 p.m. were regarded as equivalent, for practical purposes, to hours of sun exposure. Participants were also asked about hat and sunscreen use and whether a specified body site, the site of skin cancer for cases and a randomly allocated site for controls, was usually exposed when outdoors. The reported hours were assigned to each year in the time period under question and all periods summed for estimates of sun exposure over a lifetime and on working and nonworking days separately and, for each, within specific age periods. Exposure hours from birth to age 8 years were extrapolated from the hours reported at age 8 years when estimating total exposure.

To shorten and simplify the face-to-face questionnaire, we analyzed data from all 700 controls in the Geraldton study of BCC (4) to identify particular years of age in which sun exposure measurements were highly predictive of lifetime sun exposure. Although the oldest participants were ages 64 years, the life years in the analyses were restricted to age 40 years, the age of the youngest participants, so that all participants contributed sun exposure for the same years of age. The restriction to an upper limit of age 40 years did not compromise our ability to test the hypothesis that sun exposure causes ocular melanoma because little UV radiation is transmitted through the lens of the eye after late teens or early adult life (13, 14).

Stepwise linear regression models in SPSS (SPSS, Inc., Chicago, IL) used lifetime (i.e., up to 40-64 years depending on the subject's age) annual mean sun exposure as the dependent variable and hours in individual years of life from birth to age 40 years as the independent variables. The square of the correlation coefficient (R²) from the best-fitting model was calculated as a measure of goodness of fit. We modeled whole-body sun exposure; exposure to the head and neck was also modeled as an approximation to ocular exposure after accounting for hat wearing in men and women separately and together. The best-fitting model of total exposure was identified in an analysis of 355 randomly selected controls and its predictive performance was tested in all 700 controls. The overall models for each sex had 390 men and 310 women, and the models for sun exposure to the head and neck had 209 men and women who were questioned about hat use; they had been randomly allocated the head and neck as the site for specific questions about sun exposure.

The shortened questionnaire asked questions about sun exposure on working and nonworking days only for the first 4 decade years of life (ages 10, 20, 30, and 40 years; see Results for derivation of these years). It was first used in a computer-assisted telephone interview (CATI) in the ocular melanoma study in 1997 to 1998 with 298 cases and 916 controls who also completed a preinterview calendar similar to that used in the original Geraldton study (4) to record all residential locations, schools, and jobs held for ≥1 year. The telephone interviewer read a preamble to each decade year cycle of sun exposure questions using the calendar information as a memory cue to help subjects recall their sun exposure: “Let's talk about the year that you turned <x years of age>. That was in <year> when you were living in <y place> and you were <going to z school/college/university or working at x job>.”

A random sample of participants interviewed in 1997 and living in Sydney were asked, after completing the CATI, if they would agree to be reinterviewed face-to-face with the whole-of-life Geraldton questionnaire. Two experienced interviewers, neither of whom had interviewed subjects with the CATI, followed identical procedures to the Geraldton study (4, 5). Subjects answered sun exposure questions for each period of stable activities, not just for the decade years, and completed a preinterview lifetime calendar similar to that in the Geraldton study (7).

Three sun exposure measures were computed for comparison between the two methods: estimated working day, nonworking day, and total (working day + nonworking day) sun exposure hours to age 40 years. The measures were calculated from the sun exposure hours on working and nonworking days reported at interview and the number of working and nonworking days per week recorded in the calendar. Subjects were ranked on each of the three measures for the CATI and the reinterview because the total exposure from the face-to-face interview was summed across every year from birth to age 40 years, whereas that from the telephone interview was summed across just the 4 decade years. The ICC and 95% CI were calculated using one-way ANOVA in SPSS allowing for subject effects.

Separate questions in the CATI allowed us to calculate lifetime occupational and recreational sun exposure hours as well as sun exposure hours on working and nonworking days in the decade years (10). The occupational questions asked about jobs, paid or unpaid, held for ≥1 year after age 14 years with some outdoor exposure between 9 a.m. and 5 p.m., the usual number of outdoor hours per day (if >1 hour), and job duration in years. For recreational sun exposure, subjects were asked about participation in each of 10 common outdoor recreational activities between 9 a.m. and 5 p.m. on at least 10 days in any year since leaving school and, if they had any, the years they started and stopped and the frequency, usual hours per day, and seasons of each activity. The activities were going to a beach or swimming pool, sailing, windsurfing, water skiing, fishing from a boat, snow skiing, tennis, field sports, golf, lawn bowls, gardening, walking or jogging, and any other nominated regular activity. These parallel measurements of sun exposure were added because the CATI questionnaire was going into the field before testing the reliability of the substantially shortened working and nonworking days approach to estimating lifetime sun exposure.

We used the CATI again to measure sun exposure in a study of non-Hodgkin's lymphoma in 704 cases diagnosed in 2000 to 2001 and 694 population controls, with the decade year questions extended to ages 50 and 60 years and the recreational exposure questions removed from the questionnaire to permit inclusion of questions that addressed other exposures (12).

Spearman correlation coefficients (R) and their 95% CIs were calculated to compare sun exposure estimated from the decade years questions (first 4 decade years for ocular melanoma study and first 6 for non-Hodgkin's lymphoma study) with those from the occupational questions in both the ocular melanoma and non-Hodgkin's lymphoma studies and the recreational questions in the ocular melanoma study. We also compared total sun exposure estimated from the decade years questions and the parallel measurement questions. The subjects who reported no occupational exposure (45% of subjects in the ocular melanoma study and 44% in the non-Hodgkin's lymphoma study) or no recreational exposure (two subjects only) were excluded from calculations of R. The numbers of participants in these studies allowed us to study variation in agreement between the measures between cases and controls and among categories of sex, age, ability to tan, birthplace, and socioeconomic status (SES).

These studies were approved by The Cancer Council New South Wales Ethics Committee.

The regression analyses identified 4 separate years of age that together captured 93% of the variance in average annual sun exposure in the initial subset of 355 controls and then in all 700 controls (Table 2). The years of age that best predicted whole-body sun exposure were similar to those that predicted head and neck exposure (Table 2) and were numerically close to the first 4 decade years of age (10, 20, 30, and 40). We included these 4 decade years in a second set of regression models and the R²s did not change (Table 2).

The yearly hours of sun exposure, estimated from hours reported in a typical week in the warmer and cooler months in controls, were plotted for exposure of the whole body and the head and neck (Fig. 1). Sun exposure hours for the whole body were higher in men than women at all ages and increased in men and fell in women during the teens before stabilizing at about age 30 years. The predictor years identified (10, 20, 30, and 40) were approximately the first and last years of detailed measurements (questions started at age 8 years), the exposure minimum at 18 years in women, and the high point in the late 20s in both sexes for total sun exposure. Sun exposure to the head and neck fell in both sexes in the mid-teen years (from ∼15 years of age in men and 13 in women) to a minimum at age 20 years, rose to a peak at 26 to 30 years and then fell progressively to age 40 years (Fig. 1).

Based on these observations, we decided to ask only about sun exposure in the decade years, ages 10, 20, 30, and 40 years, in the shortened CATI questionnaire for the ocular melanoma study.

Most people interviewed with the shortened CATI questionnaire and selected to take part (60 of 72, 83% overall or 96% of those contactable, 34 men and 26 women) agreed to a whole-of-life, face-to-face reinterview. The one-to-three case-to-control ratio was the same as in all study subjects and the two interviews were done, on average, 15 months apart (range, 9-20 months). Reinterviewed subjects were younger than all participating subjects (30% ages <45 years compared with 18% for all subjects; 40% ages >60 years compared with 51%); the sex ratios were similar. Because the subjects were younger, a smaller proportion (77%) had been asked about sun exposure at age 40 years than had all subjects in the ocular melanoma study (87%). Less reinterviewed subjects had skin that tanned deeply (18% compared with 27% for all subjects), but the distributions across other grades of ability to tan were similar, as were eye color and number of nevi.

The ICC for total sun exposure (hours on working and nonworking days) from the two questionnaires was 0.65 (95% CI, 0.48-0.78) in men and women together (Table 3). Agreement was poorer for working days and total exposure in women than men and in men than women for nonworking days exposure.

Recalled sun exposure on working days in the decade years and time outdoors in outdoor occupations were highly correlated in participants in both the ocular melanoma study and the non-Hodgkin's lymphoma study (R = 0.75; 95% CI, 0.71-0.78 and R = 0.71; 95% CI, 0.67-0.74, respectively; Table 4). These correlations were much stronger in men (R = 0.78 and 0.73) than women (R = 0.54 in both studies) but similar in cases and controls and across categories of age at interview and ability to tan. They seemed stronger in people born in Australia than elsewhere and weaker in people in the highest SES category. Almost all subjects who reported no occupational exposure, except four in the ocular melanoma study, reported some working day exposure. Their total hours of working day exposure [summed over a week in the warmer and a week in the cooler months in 4 (ocular melanoma) or 6 (non-Hodgkin's lymphoma) decade years], however, was much less than in those who had some occupational exposure (means, 56.7 and 128.5 in ocular melanoma study and 71.8 and 137.2 in non-Hodgkin's lymphoma study, respectively).

Correlations between hours of sun exposure on nonworking days and exposure in specific recreational activities, which we could only calculate for the ocular melanoma study, were much weaker than those between working days exposure and exposure in outdoor occupations: R = 0.22; 95% CI, 0.16-0.27, overall (Table 4). Correlations between total working and nonworking days and total recreational and occupational exposure, however, were similar to those for working days with occupational exposure although slightly weaker in some categories (Table 4). In both of these comparisons, correlations were higher in the youngest subjects, people who tanned least well, and those born in Australia.

There was little difference between cases and controls in any of these correlations between parallel measures.

Analyses of data collected in our whole-of-life, face-to-face interview in 1988 (4) showed that recalled sun exposure in the first 4 decade years of life was a strong predictor of total sun exposure recalled over a whole lifetime from age 8 years (Table 2). On this basis, we collected data only for the first 4 decade years in our shortened CATI questionnaire for our ocular melanoma study (10). We found that estimates of total sun exposure in the ocular melanoma study, based on the first 4 decade years, were reliable on retest using the original face-to-face questionnaire (ICC = 0.65), as were those for working and nonworking days exposure, although to a lesser extent. Comparison of data from simultaneous parallel measures of occupational and recreational sun exposure with data on working and nonworking days exposure in the decade years in the CATI questionnaire showed good agreement between the occupational and working days measures (R = 0.71 in ocular melanoma and 0.75 in non-Hodgkin's lymphoma study) and for total sun exposure measures (R = 0.70 in ocular melanoma study only). The agreement between parallel measures of recreational and nonworking days sun exposure was much less.

The methods we used to construct our shortened questionnaire and test its reliability have several strengths. We did not change the content of the original questionnaire but shortened and simplified it after identifying the most informative data points, a practice that has been recommended for abbreviating food frequency questionnaires (15). By allowing subjects to complete the telephone version before asking about a reinterview, we removed any effect on their responses to the CATI of knowing they would be interviewed again and their responses compared (11). In addition, we followed the preferred method in intermethod reliability studies of having the presumed more accurate instrument as the second measure, because it may be less prone to error and therefore less affected by recall of responses from the first method (11).

The main weakness of this intermethod reliability study was the small numbers (n = 60), which prevented us from examining differences in reliability between cases and controls with precision. The time lapse of 15 months, on average, between the two questionnaires was reasonably long, although a few months either way probably has little effect on the accuracy of recall of a lifetime of sun exposure. The time between test and retest would probably be more important for recall of a specific event at a fixed point in time. The younger age of reinterviewed subjects meant that slightly fewer had been asked about sun exposure at age 40 years than had all subjects in the ocular melanoma study, but subjects were otherwise reasonably similar in case-to-control and sex ratios and pigmentary characteristics.

The shortened CATI questionnaire agreed less well with our original face-to-face interview questionnaire than the latter did with itself when readministered after 5 years (8). Its readministration obtained an ICC of 0.77 (95% CI, 0.71-0.83) for total sun exposure, whereas the corresponding figure for the CATI to face-to-face comparison was 0.65 (95% CI, 0.48-0.78). This greater agreement might simply be as expected for readministration of the same questionnaire, or the lesser agreement between the CATI and face-to-face questionnaires may reflect loss of accuracy in the former through shortening and telephone administration. Such a loss of accuracy could be due to the sampling of sun exposure at predetermined ages, the sampling of less years of life (on average, Geraldton subjects were asked about six periods of life), and, perhaps, less contextual cues to recall because the telephone interview restricted subjects to the contexts of sun exposure in the decade years only (10, 12) and not the whole of life as in the face-to-face interview. The agreement between CATI and face-to-face questionnaires in this study, however, was closer to agreement of other sun exposure questionnaires on readministration in other studies: an ICC of 0.68 for lifetime outdoor work exposure measured 22 months apart in the Helios multicenter study of sun exposure and skin cancer in Europe in 1989 and 1992 (16) and weighed κ of 0.73 for current sunbathing habits reported 3 weeks apart in a study of Swedish nurses (17) and 0.60 for diary compared with questionnaire responses about sun-related behavior in Swedish patients with dysplastic nevi (18).

Our capacity to study agreement between parallel measures of sun exposure obtained simultaneously is an additional strength, as is the many people for whom these data were collected. The high correlation between the two measures of sun exposure at work supports their accuracy and this correlation is reflected in the fact that risk of ocular melanoma increased strongly with increasing exposure on working days and in occupations, whereas risk of non-Hodgkin's lymphoma increased with neither (10, 12). The low correlation of the apparently parallel measures of exposure on nonworking days and in specified recreational activities is at odds with the substantially higher correlations between measurements of exposure on nonworking days when compared by readministration of the face-to-face questionnaire or, although less so, by sequential administration of CATI and face-to-face questionnaires. This may indicate that the 10 nominated recreational activities did not adequately account for sun exposure outside of working hours or that quite a lot of the time spent in them was spent on working days. The latter is suggested by the much higher correlation in women between exposure in recreational activities and total sun exposure (R = 0.47; 95% CI, 0.40-0.54) than with nonworking day sun exposure (R = 0.12; 95% CI, 0.04-0.21). Either way, outdoor time on nonworking days and time spent outdoors in a range of recreational activities may not be truly parallel measures of nonoccupational sun exposure.

If exposure on working days and in occupational activities can be taken as truly parallel measures of occupational exposure and have uncorrelated errors, then the coefficient of correlation between them can be taken as a lower limit estimate of their validity as measures of occupational sun exposure (11). That is, their coefficients of validity for occupational sun exposure are not less than 0.71 (non-Hodgkin's lymphoma study) to 0.75 (ocular melanoma study). We have no way of knowing, however, whether their errors are uncorrelated.

Cases and controls had similar levels of reliability assessed from the parallel measures of occupational sun exposure (Table 4), although there was weak evidence that controls recalled nonworking day or recreational exposure better than cases. There was little difference between cases and controls in the ICC for total sun exposure in the Geraldton study (8), but in the Helios study the ICCs were less in controls than cases for work, vacation, and sports exposure (16). Thus, there is no consistent evidence that cases are more or less reliable in reporting their sun exposure than are controls.

The results of the present study indicate that occupational sun exposure is substantially more accurately measured in men than women (Tables 3 and 4). In addition, women have low occupational sun exposure: although women in the ocular melanoma study averaged 76% of the total outdoor hours of men in the decade years, they only averaged 17% of the total outdoor occupational hours. That occupational sun exposure in women is low and measured comparatively inaccurately may prevent finding effects of occupational sun exposure in women in epidemiologic studies. On the other hand, women may recall nonworking day sun exposure more accurately than men (Table 3) and it forms, on average, a higher proportion of their total sun exposure than it does in men. Higher total sun exposure in men than women (Fig. 1) was also reported in the Melbourne Visual Impairment Study (19) and in a 1992 Australian time use survey (an extra 80 minutes a day on average; ref. 20). In addition, substantially lower annual ocular sun exposure in women than men has been reported from the United States (21). Both the Helios study and the Geraldton reliability study reported that they found no difference between men and women in reliability of sun exposure measurements, although neither reported sex-specific ICCs (8, 16). Ways to capture accurately the different amounts and patterns of sun exposure in men and women have not been explored. One might speculate that women's dual family and occupational roles contribute to difficulties in measurement, and use of additional autobiographical cues, such as the number and age of children or other dependents, and provision for stating whether work was full-time or part-time might help improve measurement accuracy.

Our study indicates a high degree of reliability when measuring total lifetime and occupational sun exposure and a moderate level for recreational sun exposure when using an autobiographical approach and memory cues. Studies of reliability of sun exposure measurements in the Helios and Geraldton studies were consistent with this observation, finding the greatest reproducibility for measures of total lifetime sun exposure and the least for more restricted settings, such as exposure of a particular anatomic site, sun exposure on vacations (8), or sports exposure (16). The high correlation of our parallel measures of occupational sun exposure indicated that the decade year and whole-of-life techniques measured work-related sun exposure well. Poorer reliability of women's recall of total and particularly working days exposure suggest that our questionnaire works less well in women than in men. Further research into these methods could help eliminate this difference.

We thank Huu Nguyen for the regression modeling and Dr. Ann Maree Hughes for providing the data from the New South Wales non-Hodgkin's lymphoma study.