Methodologic Considerations in Calculating and Analyzing Proportion of Time Covered as a Measure of Longitudinal Cancer Screening Adherence

To reduce the burden of cancer, cancer screening programs aim to keep patients up-to-date with recommended screening procedures. Measuring screening adherence over time (i.e., longitudinally) is important in evaluating the success of delivering screening interventions or programs.

Proportion of time covered (PTC) is an adherence measure that has been applied to medication use, screening, and other preventive health services. In medication use studies, medication possession ratios (1) measure the number of days a patient has a specific medication in their possession divided by the number of days in the time period of interest. In studies of screening and other preventive services, the denominator of PTC is the observed period of time a person is eligible for a preventive service (i.e., given age and other factors). The numerator is the observed period of time a person has received the service and is not yet due again. PTC, also called “covered time,” was initially proposed by Vogt and colleagues to compute a quality measure (the prevention index) to “evaluate practice variations in the delivery of preventive care” (2). More recently, PTC has been used as an outcome in studies of colorectal cancer screening adherence (3, 4) and liver cancer surveillance (5). PTC can be calculated on the basis of a person's eligibility for screening and when they had various screening and follow-up tests. For example, colorectal cancer screening is recommended between ages 50 and 75 years, and one screening option is colonoscopy every 10 years (6). A 75-year-old person who received only one colonoscopy, at age 50, would have 40% of eligible time covered; a single colonoscopy “covered” them for 10 of 25 years during which screening was recommended. Another person who had screening colonoscopies at ages 50 and 60 would have had 80% covered time (20 of 25 years).

Murphy and colleagues noted advantages to PTC as a measure of screening adherence: “(1) accounting for quality, timing, and results of screening examinations; (2) not penalizing patients for examinations performed just outside the recommended interval; and (3) including all available follow-up time” (3). PTC is particularly useful when different screening modalities have different recommended rescreening intervals. Vogt and colleagues (2, 7) and Murphy and colleagues (3) argue that PTC can avoid some pitfalls of other quality measures like HEDIS (Healthcare Effectiveness Data and Information Set), such as failing to account for follow-up of abnormal results.

Despite these advantages, there are challenges and nuances in computing and interpreting PTC. We describe some desirable properties of PTC measures, challenges in achieving those, and potential solutions. We follow this by describing how to calculate and statistically model PTC measures. Finally, we discuss additional interpretation and implementation challenges. Understanding the relationship between PTC and screening effectiveness is important and presents further methodologic issues but is outside the scope of this manuscript. Our focus is on PTC as an outcome measuring the longitudinal success of an intervention or program in delivering screening as a preventive service.

In this article, screening refers to testing for a disease in the absence of signs or symptoms (8, 9). Unless otherwise noted, we consider screening in a population at average risk for the disease of interest. We use colorectal cancer screening as the application. We call screening in a high-risk population (e.g., after the detection of cancer precursors) surveillance. Diagnostic testing evaluates signs or symptoms of disease. For colonoscopy, we refer to two types of diagnostic tests: follow-up colonoscopies are triggered by positive stool tests and symptomatic colonoscopies are triggered by patient symptoms. Follow-up colonoscopies are part of the screening episode though they themselves are not screening tests.

Table 1 summarizes some desired properties of PTC measures, challenges defining a measure with those properties, and potential solutions. Each is described in more detail below.

Assume screening colonoscopy provides 10 years of coverage. Suppose person “A” is observed from ages 50–52 and screens by colonoscopy at age 51; they are covered for 1 of 2 years observed (50% covered time). Person “B” is observed from ages 50–70 and screens by colonoscopy at age 60; they are covered for 10 of 20 years observed (50% covered time). We argue that these individuals do not give us the same information about a program's success promoting screening adherence. If “A” had been observed for longer, they might have had extremely high adherence (e.g., 24 of 25 years), extremely low adherence (e.g., 1 of 25) years, or anything in between. We do not know as much about the longitudinal adherence for person “A” as we do for person “B.” Thus, person “B's” experience is more informative about the program's success. Unfortunately, percentages alone do not reflect this. One solution, when comparing two different populations, is to describe the total amount of observation time in each population and measures of variability across people (i.e., median, interquartile range, mean, and SD). Doing so can help the reader compare PTCs. In addition, one can quantify the range of possible values we might see had all person-time within the relevant age ranges been observed. For example, person “A” could have been covered anywhere from 1/25 = 4% to 24/25 = 96% of the time, and person “B” could have been covered from 10/25 = 40% (if they screened only at age 60) to 15/25 = 60% of the time (if they screened again at age 70). Another approach weighs individuals according to their total observation time (see Results).

Some screening strategies require multiple steps. For example, the fecal immunochemical test (FIT) requires a follow-up colonoscopy if the FIT is positive. The follow-up colonoscopy earns the patient 10 years of covered time. Without the colonoscopy, a person with a positive FIT has not been adequately screened and should not be considered “covered.” It is challenging to determine how much covered time to give a person who has a positive FIT but does not complete a follow-up colonoscopy. Such a decision may be based on evidence, estimates from microsimulation models (10), or consensus. Murphy and colleagues previously suggested 6 months' time for a positive FIT (vs. 12 months for normal FIT; ref. 3). A similar challenge exists for assigning covered time to a test with suboptimal quality: how much to assign to colonoscopy with inadequate preparation (3)? Such a test is not considered sufficient for screening. Sensitivity analyses could be applied ranging the amount of covered time, especially when incomplete episodes are relatively common.

The standard approach to PTC stops counting covered time in the numerator and denominator once a person is lost to follow-up (e.g., disenrolls from a health system, moves out of a region). When time after loss to follow-up would have been covered, the standard measure underestimates true PTC. This is particularly problematic for screening tests with longer recommended intervals (e.g., colonoscopy every 10 years). Assume we observe person “C” from ages 50–75, who has 40% covered time (10 of 25 years) for a colonoscopy at age 65. Assume we observe person “D” from ages 50–70, who also has a screening colonoscopy at age 65 but then disenrolls from the health system at age 70. If person “D” is censored at disenrollment to calculate PTC, they will have only 25% covered time (5 of 20 years), despite having identical screening behavior to person “C.” This is unsatisfying because the health system was no less successful delivering screening to person “D” as it was for person “C.”

Thus, we propose a complementary modified PTC measure (mPTC) that gives a system or program full credit for delivering a screening test, regardless of what happens to the patient subsequently. The rationale for this approach is that a system's success delivering screening at a point in time should not be influenced by what happens in the future. Once a person is screened, even if they are lost to follow-up, information on their outcome (covered time) is not missing. We know how long they are covered once they receive a screening test, even if we cannot subsequently observe them.

We propose calculating mPTC by adding to both the numerator and denominator the number of years that a test covers someone at the time of the test, regardless of whether or not they were subsequently lost to follow-up. Figure 1 illustrates this approach for person “D.” The mPTC for person “D” is calculated as follows:

Importantly, this is the same value as for person “C,” who had the same screening behavior but was not lost to follow-up.

A similar approach can handle death or cancer diagnosis. The traditional PTC measure excludes time after death or cancer from both the numerator and the denominator. However, it is problematic for a system or program to receive less covered time because the person screened is diagnosed with cancer or later dies. Using the complementary mPTC measure avoids this problem. For example, a person screened with colonoscopy at age 50 will receive 10 years in the numerator and the denominator at that time, regardless of whether they later die. It may be counterintuitive to give 10 years of credit to someone who lives for only 4 years. However, this discomfort may be mitigated by thinking of PTC as a measure of a system's or program's success delivering screening rather than an attribute of a patient.

Patients diagnosed with cancer are not eligible for subsequent screening, and patients diagnosed with an advanced precursor to cancer need surveillance. Thus, diagnoses of cancer or cancer precursors affect patients' subsequent needs, but these diagnoses should not alter assessment of how successful a program or system has been delivering screening up until the time of diagnosis. Two systems that screen 60% of their 50-year-old patients for colorectal cancer by colonoscopy have—as of that point in time—had equal success in delivering screening, regardless of the findings of those screening tests.

If the goal of measuring PTC is to quantify success delivering screening, it is problematic for covered time to depend on screening results. Doing so penalizes systems or programs for finding cancer or its precursors—what screening is designed to do. Therefore, for analyses of adherence to a program (or intervention) among average-risk patients, we recommend mPTC be presented along with the standard PTC measure. While traditional PTC censors at cancer diagnosis, mPTC gives equal credit to systems regardless of test results. It is worth noting that a person who has a positive FIT followed by a diagnostic colonoscopy will receive more covered time than a patient whose FIT is negative. This is not because of the FIT result per se but because of the action (colonoscopy) that was taken during the screening episode; all FIT-positive patients who receive a follow-up colonoscopy would receive the same amount of covered time, regardless of what was found during the colonoscopy.

Sometimes, we may also be interested in successful delivery of surveillance. For example, we might be interested in how well a system or program does at delivering surveillance colonoscopy to patients with high-risk adenomas. One option is to study average-risk screening and surveillance separately. This makes sense when a program is designed to promote screening or surveillance, but not necessarily both. An alternative is to study screening and surveillance together, as described by Murphy and colleagues (3), by crediting patients with covered time until they are next due for screening/surveillance (e.g., 10 years for a normal colonoscopy and 3 years if high-risk adenomas are found). An advantage of the latter approach is evaluating screening and surveillance in the same analysis. However, covered time is credited based on results of the screening test, and thus some of the problems discussed earlier are present.

One of the initial motivations in developing PTC measures was to describe delivery of preventive services. Ideally, only person-time following a screening test should be included in the numerator, and only person-time when a person is symptom free and screen eligible should be in the denominator of the standard PTC measure. In practice, there are at least three approaches to handling symptoms and indication, keeping in mind that, often, symptom and indication data will not be available or complete (8).

This approach is most feasible. National screening programs may not have any data on symptoms or indication. The only option is to consider tests for any reasons in persons with and without symptoms.

In other settings (e.g., health care systems with claims and utilization data), data on some symptoms—particularly severe ones—may be available. If these data are available and of high quality, one can censor patients if they develop symptoms during uncovered time. A person would reenter the analysis when symptoms are gone and when they are due again for screening. If the necessary data are available, this approach is preferred because it is most consistent with focusing on delivery of preventive services. However, this approach requires having symptom information available on testers and nontesters alike. Given that nontesters may not be interacting with the health care system, they are more likely than testers to have missing data on symptoms. A second challenge is identifying when symptoms resolve and thus when a patient can return to screening (i.e., contribute to denominator). So, practically, this approach may be challenging or impossible to implement.

If symptom data are not available on everyone, but test indication data are, one could censor people at the time of a diagnostic exam (e.g., one triggered by symptoms such as rectal bleeding). Subsequent person-time would be excluded from both the numerator and denominator. Consistent with logic described in prior sections, for mPTC, if a diagnostic test occurred during covered time, censoring would occur after that screen-covered period was over. Individuals could begin contributing time once they were due for screening, which would be 10 years after a colonoscopy. This approach is more feasible than the second approach because it does not require symptom information, except as needed to determine test indication.

In the Supplementary Materials and Methods, we use hypothetical examples to compare the three approaches, assuming no misclassification. When symptoms and diagnostic testing are uncommon, the three approaches yield similar results. The more common symptoms are, the greater the difference in approaches. When diagnostic testing among patients with symptoms is high (e.g., 100%), censoring at diagnostic tests is a better approximation of the preferred approach of censoring at symptoms than is ignoring indication. When diagnostic testing among patients with symptoms is moderate (e.g., 50%), ignoring indication is a better approximation of the preferred approach. When diagnostic testing among patients with symptoms is low (e.g., 0%), neither of the other approaches approximates the preferred approach.

In this section, we describe options for summarizing and modeling PTC to compare populations or groups.

An individual i's PTC (PTC_i) is equal to the amount of time they are covered by screening tests divided by their total time eligible for screening:

where CT = covered time and TT = total time. For the standard PTC measure, both CT and TT are truncated at loss to follow-up, death, or cancer. For mPTC, CT and TT are extended to include the full recommended screening interval for the test, regardless of subsequent events during covered time; we denote these as mCT and mTT, respectively.

There are two ways to summarize PTC for a group of n people (e.g., a health system, a country, or an arm in an intervention study). One can take the mean of the individual measures of PTC [we call this average PTC (PTC_A)]. Or, one can take the mean of PTC weighted by each person's total time [we call this overall PTC (PTC_O)]. Table 2 shows these equations and the analogs for mPTC_O and mPTC_A.

PTC_A weights all individuals equally so it is a person-level average estimate, while PTC_O weights by total time, so those individuals with more total time are given more weight than those with less total time. As described in Materials and Methods, we believe that PTC_O (or mPTC_O) will generally be of greater interest than PTC_A (or mPTC_A) because it more heavily weights observations that are more informative (i.e., based on longer observation time).

We illustrate the computation of PTC_A, mPTC_A, PTC_O, and mPTC_O, for the population of 5 people in Table 3.

PTC_A is a simple average of the 5 individuals' PTC_i:

Similarly, mPTC_A is a simple average of the 5 mPTC_i measures:

PTC_O is computed by multiplying each individual's PTC_i by the weight for that person, summing across the population.

Or,

And mPTC_O is:

Or,

To quantify the effect of an intervention on screening adherence, or compare PTC in different groups per populations, it will often be necessary to use statistical models. Using a regression framework further allows one to adjust for any potential confounders. Assuming PTC is the measure of interest (i.e., standard or modified, PTC_A or PTC_O), then conducting the analysis can be done in numerous ways; we will outline a general weighted regression framework. We assume for simplicity that the target estimand of interest is the risk difference (RD) defined as the difference in the average proportion covered time between two populations. We can fit a linear regression model on the outcome PTC_i to calculate PTC_A, which weights each person equally:

|$E({\rm{PT}}{{\rm{C}}_i}) = {\beta _o}\, +\,{\rm{R}}{{\rm{D}}_A}{X_i}\, +\,{{\bm{ \bibeta }}_Z}{{\bi{Z}}_i}$| with no weights

where |${X_i}$| is the indicator of being in a specific population versus another population, |${{\bm{Z}}_i}$| is a vector of potential confounders, and |${\rm{R}}{{\rm{D}}_A}$| is the estimand of interest, the average-risk difference.

Or we can fit a model weighting each person based on their total time, to calculate PTC_O:

|$E({\rm{PT}}{{\rm{C}}_i}) = {\beta _o}\, +\,{\rm{R}}{{\rm{D}}_O}{X_i}\, +\,{{\bm{\bibeta }}_Z}{{\bi{Z}}_i}$| with total time weights

where |${\rm{R}}{{\rm{D}}_O}$| is the estimand of interest, the overall risk difference.

mPTC measures can be modeled similarly to traditional PTC measures except using mPTC_i as the outcome instead of PTC_i.

For mPTC_A: |$E({\rm{mPT}}{{\rm{C}}_i}) = {\beta _o}\, +\,{\rm{R}}{{\rm{D}}_{MA}}{X_i}\, +\,{{\bm{\bibeta }}_Z}{{\bi{Z}}_i}$| with no weights.

For mPTC_O: |$E({\rm{mPT}}{{\rm{C}}_i}) = {\beta _o}\, +\,{\rm{R}}{{\rm{D}}_{MO}}{X_i}\, +\,{{\bm{\bibeta }}_Z}{{\bi{Z}}_i}$| with modified total time weights.

These models can be fit using generalized estimating equations with robust SEs to relax model assumptions including normality of PTC_i.

Regardless of decisions made on topics presented above, PTC measures have important limitations worth noting. First, covered time is just one facet of screening adherence that is important to understand and improve. There are others that may also be important to address depending on the setting, for example, percent of population in need of screening at a given point in time, time to first screening, or percent never screened (11). Any variation of PTC alone will likely not provide a complete picture of screening adherence.

Second, there are multiple ways to arrive at the same PTC estimate. A person can have the same PTC_i measure following different screening behaviors. For example, both of the following scenarios give rise to PTC_i = 0.50 over 20 years.

• Being screened by FIT (recommended annually) every other year (10 years/20 years)

• Being screened by FIT annually for 10 years and then stopping

• Receiving a single screening colonoscopy (10 years/20 years)

Or, on a population level, we could have a population of 100 people observed between ages 50 and 75 (2,500 person-years). PTC_O would be 0.50 in both of the following situations:

• 50 people each covered for 25 years (50 × 25 = 1,250 years covered of 2,500 years)

• 100 people each covered for 12.5 years (100 × 12.5 = 1,250 years covered of 2,500 years)

Do the two scenarios in each example above (individual and population) represent the same quality of longitudinal adherence? In studies of colorectal cancer screening, the average follow-up may be relatively short (<10 years) compared with the recommended interval for screening by colonoscopy (10 years). In such studies, PTC may be more of a measure of screening initiation rather than repeat testing (and thus adherence over time).

These issues suggest a third challenge: it is not clear how different values of PTC are related to outcomes such as cancer incidence and mortality. Future empirical research on this subject is needed.

The discussion of analytic decisions in the previous sections was not exhaustive. Other decisions may include (but are not limited to):

• What should be done when a person screens before they are due? For example, a patient who has a normal colonoscopy and then receives a FIT 2 years later, or a patient who completes two FITs 8 months apart.

• How should screening before cohort entry be accounted for (when it is known)? For example, suppose one is interested in describing the PTC_O of a health care system. Some patients will be known to have screened and be “covered” before enrolling in the system.

• How should gaps in observed time be handled? For example, suppose a patient disenrolls from the health system for years and then reenrolls.

• How should systems account for the choice not to screen after a high-quality shared decision-making process?

• How should we handle changes in recommendations about screening intervals that occur over time while a patient is still “covered?”

There is likely not a single correct approach to any of these questions. Therefore, we recommend analysts clearly describe the decisions that have been made and their rationale.

While PTC is an appealing measure of longitudinal adherence, efforts must be made to ensure that it reflects the underlying concepts a researcher is trying to study. A selection of important decisions to make are summarized in Table 4. mPTC may, for some applications, have more desirable characteristics for describing screening adherence than the standard PTC approach.

Because there are many ways that PTC can be calculated and modeled, it is important for authors to describe and justify their approach. To facilitate comparisons across studies, we recommend multiple PTC measures be presented in secondary analyses and that PTC be accompanied by a suite of other adherence measures (e.g., proportion never screened) and descriptive statistics such as total observation time across groups, rates of cancer and censoring, and information about symptoms and test indication.

These considerations are not unique to PTC or measures of longitudinal adherence. Questions about the availability of symptom data and how to handle diagnostic tests are relevant to other measures of cancer screening, as are issues about how to weight individual observations when estimating population-level summaries. Thus, developing common approaches, terminology, and reporting practices in the context of PTC has the potential to improve the study of screening adherence more broadly. This manuscript focused on longitudinal screening adherence as an outcome. Future studies should investigate modifications or implementation recommendations for using PTC as an exposure in studies of cancer screening effectiveness.

J. Chubak reports receiving a commercial research grant unrelated to cancer screening from Amgen. No potential conflicts of interests were disclosed by the other authors.

Conception and design: J. Chubak, M.L. Anderson, A.J. Cook, C.C. Murphy, M.L. Jackson, B.B. Green

Development of methodology: J. Chubak, M.L. Anderson, A.J. Cook, M.L. Jackson

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): M.L. Anderson, A.J. Cook, M.L. Jackson, B.B. Green

Writing, review, and/or revision of the manuscript: J. Chubak, M.L. Anderson, A.J. Cook, C.C. Murphy, M.L. Jackson, B.B. Green

Study supervision: J. Chubak

The authors thank Dr. V. Paul Doria-Rose and Ms. Ronit Dalmat for helpful comments on the manuscript.

This work was supported by a grant from the NCI of the NIH (R01CA121125, principal investigator: B.B. Green).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.