Comparison of Longitudinal CA125 Algorithms as a First-Line Screen for Ovarian Cancer in the General Population

Ovarian cancer remains the leading cause of death from gynecological cancer among women and accounts for 5% of all female deaths from cancer, corresponding to annual deaths of around 4,100 in the United Kingdom (1), 42,700 in Europe, 22,280 in the United States (2) and 152,000 worldwide (1). Most women are diagnosed in advanced stage (Stage III–IV) with reported 5-year survival rates of 19% (Stage III) and 3% (Stage IV) respectively. The higher survival rates of 70%–90% in earlier stage (Stage I–II) disease has driven international screening efforts to detect the disease earlier (3). To date the large screening trials have used measurement of a tumor marker CA125 (Cancer Antigen 125 protein) in the blood and transvaginal ultrasound to image the ovaries.

In the ovarian component of the Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial (PLCO), an absolute CA125 cutoff value of 35 U/mL and pelvic ultrasound were used as first-line annual tests. There was no stage shift or reduction in ovarian cancer-related deaths between the screen and no screening (control) arms (4). More recently in UKCTOCS, multimodal screening (MMS) resulted in significant detection of earlier-stage disease in women with invasive epithelial ovarian/tubal/peritoneal cancers (iEOC/PPC) compared with the control arm. In a pre-specified subgroup of the primary mortality analysis ROCA reduced mortality in the 20% of cancers where a baseline CA125 was measured, that is, in incident cancers. However, further follow-up is needed to assess whether screening results in a definitive mortality reduction for all ovarian cancers (5). In MMS the annual first-line test was CA125 which was interpreted using the longitudinal Risk of Ovarian Cancer Algorithm (ROCA). Women found to be at increased risk had repeat CA125 and/or transvaginal ultrasound (second-line test).

As a first line test, ROCA had significantly better performance characteristics for detection of invasive epithelial ovarian/tubal cancer (iEOC/PPC) compared with a CA125 cutoff value in the UKCTOCS dataset. Only 48% of the incident cases would have been detected at the last annual incidence screen if a CA125 cutoff value of 35 U/mL had been used (6). The statistical model underlying the ROCA is built on two important assumptions: (i) each woman has her own baseline CA125 level and variation about this level, and (ii) after cancer inception the tumor sheds CA125 into the circulation whereupon serum CA125 increases exponentially reflecting tumor doubling. ROCA best detects cancers where a significant increase above a woman's CA125 baseline occurs; hence, the pre-specified subgroup analysis for cancers where a CA125 baseline was measured during screening. The second assumption corresponds to a change point in the serum CA125 time series as the cancer develops (7–9). The development of new ovarian cancer detection algorithms that further minimize assumptions remains an important scientific goal as does the performance of serial algorithms compared with single thresholds in the context of screening. Parametric empirical Bayes (PEB; ref. 10) is another algorithm that has been described for interpreting serial CA125 data.

In this article, we use the data from 50,083 post-menopausal women who underwent 346,806 annual screens and follow-up in the multimodal (MMS) arm during the course of UKCTOCS to (i) build a new algorithm for longitudinal analysis of cancer biomarkers, “method of mean trends” (MMT), which measures the dynamics of the biomarker over time using multiple trend indices and (ii) investigate the performance of both longitudinal biomarker algorithms (MMT and PEB) and CA125 cutoff as first line tests for ovarian cancer screening.

In UKCTOCS, 202,638 low-risk postmenopausal women ages 50 to 74 were randomized between 2001 and 2005 to one of two screening (ultrasound: USS; multimodal: MMS) arms or a no screening (control: C) arm in a ratio of 1:1:2. Exclusion criteria were self-reported previous bilateral oophorectomy or ovarian malignancy, increased risk of familial ovarian cancer, or active non-ovarian malignancy. Ethical (IRB) approval for UKCTOCS was given by NRES Committee North West, Haydock (Ref: 00/8/034) and NREC Committee London, Harrow (Ref: 05/Q0505/57). The trial was listed on ClinicalTrials.gov (NCT00058032). The study was done in accordance to the ethical guidelines set out by United Kingdom Ethics Committee Authority. Trial design, including details of recruitment and randomization have been described elsewhere (5, 6, 11). All women provided written informed consent.

Women were offered annual screening from randomization to December 31, 2011. The screening protocol and management of screen-detected abnormalities have been previously described (6, 11). In brief, in the MMS group women had an annual serum CA125 that was interpreted using ROCA. If the “risk of ovarian cancer” was normal they were triaged to annual screening; if intermediate they had repeat CA125 in three months and if elevated, they had repeat CA125 and transvaginal scan.

All volunteers were followed using their National Health Service number through data linkage with the appropriate national agencies for cancer registrations and/or deaths as well as by postal questionnaires. Primary cancer site, morphology, stage, and grade were assigned as of December 31, 2014 following review of all medical notes by an independent outcomes review committee (two pathologists and two gynecological oncologists) who were blinded to the randomization group as previously described (5, 6, 11).

The sample set is derived from data on all eligible women randomized to the MMS arm who were included in the mortality analysis and had a CA125 measurement (5). “Cases” were women confirmed at censorship (December 31, 2014) by the outcomes review committee to have iEOC/PPC, borderline epithelial and non-epithelial ovarian cancer. Controls were all women who did not have ovarian/tubal/peritoneal cancer. The dataset of eligible women was randomly divided in a stratified manner, so that both controls and cases were each split in a 1:1 ratio into a training and validation sets.

Because the ROCA was prospectively evaluated in UKCTOCS, following the annual screen, all repeat CA125 tests were triggered by ROCA. Hence, to limit concerns over potential bias, the CA125 data for this analysis were limited to annual measurements. All CA125 values were transformed by taking a logarithm of their values before applying the algorithms because on this scale the distribution was much closer to a Normal distribution than the original scale.

This new method evaluates the dynamics of longitudinal markers by averaging weighted derivatives of marker changes for all intervals of time between measurements. Because the most recent biomarker measurement is more important than all previous ones, weights were proposed in order to take into account the importance of those samples, which were closer to the most recent observation. For each individual woman “i” the whole serial pattern, |${Y_{i,j}},\ j\ = \ 1..T$|⁠, was mapped into a new five-variable space, including its mean derivative, the three indices described below and the most recent measurement |${Y_{i,T}}$|⁠. In this way, instead of the initial collection of T measurements over time for a particular marker, the dimension for the CA125 marker was reduced to 5 variables. For this marker, and each interval between two consecutive measurements, the derivative was approximated using the expression |$\Delta Y_{ij}/\Delta t_{ij}$|⁠, where |$\Delta {Y_{i,j}} = {Y_{i,j+1}}\minus {Y_{i,j}},\Delta {t_{i,j}} = {t_{i,j+1}} \minus {t_{i,j}}$|⁠, then the mean derivative was calculated, giving the most recent measurement higher weight |$\sum_{j=1}^{T\minus1} w_{ij} {\frac {\Delta Y_{i,j}}{\Delta t_{j,j}}}$|⁠, where the weights |${\rm{\ }}{w_{ij}}\ $|were computed for each interval between sequential samples as:

where |${t_{i,T}}$| was the age of the patient at the time of the most recent sample whereas|${\rm{\ }}{t_{i,j}}$| was the age of the patient when the j-th sample was taken. In this way, more recent measurements were provided a higher weighting. Apart from the mean derivative, multiple indices were analyzed, and after using Akaike Information Criterion (AIC) for the model selection, which deals with the trade-off between the goodness of fit of the model and the simplicity of the model, three further indices were used as additional parameters of the MMT:

As a final step, MMT used a logistic regression model based on the weighted average derivatives, the described indices (1) to (3), and the latest currently available measurement taken for each patient. This logistic regression model (12) was then fitted to obtain coefficients, which provide predictions on the probability scale which were the basis of the rule for classification into cases and controls. If there was only a single CA125 value, the threshold at 90% specificity was used for classification. An important advantage of this proposed approach is the ability to use more than one marker simultaneously by adding into the logistic regression model-average derivatives and other indices calculated separately for each of markers |$m\ = \ 1..M$|⁠.

In summary, the MMT algorithm applied for the prediction of disease based on serial measurements is as follows:

• – Step 1: Approximate the time-derivatives of the biomarker series, for each patient and each measurement, as |$\Delta Y_{ij}{/}\Delta t_{ij}$|⁠.

• – Step 2: Calculate the weight for each derivative as |${w_{ij}}{\rm} = {\rm{\ }\frac{1}{{{t_{i,T}}}} - ( {{t_{i,j + 1}}} + {t_{i,j}} )/2}}$|⁠.

• – Step 3: Calculate the weighted mean |$\sum\nolimits_{j = 1}^{T - 1} {{w_{ij}}{\frac{{\Delta {Y_{i,j}}}}{{\Delta {t_{i,j}}}}} $|⁠.

• – Step 4: Calculate the indices A_i, B_i, C_i in expressions (1), (2), and (3).

• – Step 5: Use AIC to select the predictors (out of weighted derivative, indices A_i, B_i, C_i and raw measurement of the biomarker) that best explains the labels of the patients (control = 0, case = 1).

• – Step 6: Fit the logistic regression with the selected predictors and the labels of all patients.

Once the logistic regression is fitted it can be used to predict the risk of the disease for the new patient. If more than 1 biomarker measurement is available for all the patients, the procedure above is repeated calculating the 5 predictors for each biomarker (Step 1–4) and including them all in the AIC variable selection step (Step 5).

This method, described previously (10), allows calculation of a biomarker threshold for each subject based on their previous screening history. The approach requires a serial pattern of markers in healthy women and can be used for analyzing the performance of new individual markers. Suppose a subject with n historical screens with an average marker concentration |$\overline {{y_n}} $| is going to have a new screen and assuming that we operate at level α, the threshold given by the PEB algorithm is:

where μ is the population mean, |${B_n} = {\frac{{{\tau ^2}}}{{{\sigma ^2}/n + {\tau ^2}}}$|⁠; |$V\ = \ Var[ {{Y_{i,j}}} ]$| is the variance of measurements |${Y_{i,j}}$|⁠; |${\sigma ^2}\ {\rm and}\ {\tau ^2}$| are the within-subject and between-subject variances, |${\sigma ^2} = ( {\frac{1}{2}} )Var[ {{Y_{i,2}} - {Y_{i,1}}} ],$||${\tau ^2} = V - {\sigma ^2}$|⁠; |${z_\alpha }$| is the |$\alpha $|-quantile of a standard normal distribution (⁠|${z_\alpha }\ = \ 1.96$| when |$\alpha \ = \ 0.975$|⁠). At the initial screen |${B_0}\ = \ 0$| and so the threshold becomes:

It should be noted here that, because after obtaining all the parameters, including the level α from the training set for each patient the PEB algorithm yields an outcome of 0 (no cancer) or 1 (cancer present), depending on whether the last measurement is higher than the threshold or not, we consider only the value of the sensitivity at a fixed level of specificity in the sequel. The area under the ROC curve cannot be used to analyze performance of the PEB algorithm because in this setting the outcome (0 or 1) does not allow the use of thresholds required for ROC curve construction.

OB was provided all data, including CA125 measurements for each woman, dates of birth, dates when the measurements were taken, case–control status and dates of diagnosis for cancer cases on the training set. The MMT and PEB were developed/trained, respectively, using the training set by OB. The annual CA125 values were used in a sequential manner. At each annual screen, all previous annual measurements were incorporated and a new PEB/MMT classification was determined. The outcome for the PEB was either 0 or 1 for each measurement in a longitudinal time series while that for MMT was continuous result of the logistic regression. At the training stage, a threshold was calculated for the MMT to provide similar specificity to annual ROCA classification of “normal risk” that was 87.6% in UKCTOCS⁶. Because MMT uses trend indices, which cannot be calculated for the first annual measurement (no previous history), the 0.9 quantile was calculated from the control measurements. After that, for every patient, the first annual measurement was compared with this quantile and depending on whether the measurement was higher or not, risk was assigned as abnormal or normal, respectively. Sensitivity, specificity, and AUC were calculated.

The validation set comprised of a set of women with their serial annual CA125 measurements but no outcomes. OB as described above, normalized all measurements by taking a logarithm of their values. The annual CA125 values were used in a sequential manner. At each annual screen, all previous annual measurements were incorporated and a PEB and MMT classifications (and prediction probability) were calculated. The data were then transferred to MB who unblinded the outcome data and compared the performance of the two algorithms and the single CA125 cutoff value.

The primary outcome was iEOC/PPC diagnosed within 1 year of annual CA125 measurement. Women with borderline epithelial or non-epithelial ovarian cancer were excluded from this analysis. Secondary outcome was all primary ovarian/tubal/peritoneal cancers and the whole data set was used for this analysis. When dealing with the determination of outcomes, the last blood sample was considered as a true positive (if within 1 year from diagnosis) and all prior annual samples as true negatives. A subgroup analysis was undertaken which was restricted to cancers diagnosed between 1 and 2 years from last measurement in order to see if there was any difference in lead time between the algorithms. For this analysis, if there was more than one measurement within 1 to 2 years then the closest to 2 years was used as the “last measurement.” As above the last measurement was considered as true positive and prior annual samples considered true negatives. All annual samples beyond the last measurement were discarded. When dealing with controls, all samples were included as true negatives.

The performance characteristics of the two algorithms and CA125 were evaluated and compared in terms of (i) the sensitivity (proportion detected of those with cancer) at a fixed specificity (proportion of controls correctly detected not to have cancer): For PEB the threshold was implicit in its formulation; for MMT the threshold was the value that provided 0.9 specificity in the training set; for CA125 the threshold was the common value of ≥30 U/mL for postmenopausal women and (ii) the area under the Receiver Operating Characteristic (ROC) curve (AUC). Inference for the ROC curves was based on cluster-robust standard errors that accounted for the serially correlated nature of the samples. It was not possible to create AUC for PEB given the outcome was not continuous.

The eligible women comprised all 50,083 of the 50,624 randomized to the MMS group who attended for screening and had an annual serum CA125 level measured. They underwent 347,002 annual screens (median 8, IQR 6–9). Median follow-up was 11.1 years (IQR 10.0–12.0; ref. 5).

During follow-up, 332 developed ovarian/tubal/peritoneal cancer as of December 31, 2014. The training and the validation sets comprised of 25,041 women with 161(139 iEOC/PPC) cases and 25,042 women with 171(143 iEOC/PPC) cases correspondingly (Table 1). Baseline characteristics of women included in the training and validation sets were balanced (Table 2). Morphology of cases together with histological subtype and stage of iEOC/PPC are presented in Table 3. Longitudinal algorithms were applied to the validation set, which contained 174,270 annual CA125 measurements from 25,042 women (Table 1).

Figure 1 shows the ROC curves for MMT and the CA125 threshold rules for detection of iEOC/PPC cases diagnosed within 1 year of last annual sample (primary analysis). MMT provided a higher area under the curve, 0.921 compared with 0.884 for the single threshold rule. The AUC for CA125 single threshold was significantly lower than for MMT (P = 0.01).

At a specificity of 89.5% for PEB (for which it was not possible to compute AUC), sensitivities were 73.1% [95% confidence interval (CI) 63.6–80.8) for the single CA125 threshold, 86.5% (95% CI, 78.4–91.9) for MMT and 88.5% (95% CI, 80.6–93.4) for PEB. In a hypothetical cohort of 100,000 women with an average incidence of about 50 per 100,000 a year this result would imply that MMT would detect about (86.5–71.3) × 50/100 ≈ 7 extra cases and PEB would detect about (88.5–71.3) × 50/100 ≈ 8 extra cases compared to the CA125 cutoff value. To assess the significance of differences in sensitivity at fixed specificity for different algorithms, McNemar's exact test was used. The sensitivity was significantly different compared with the single threshold rule. The longitudinal approaches were not significantly different from each other. 11.5% (12/104) of iEOC/PPC were not detected on the last annual screen by either longitudinal algorithm.

Table 4 shows the sensitivity, specificity and AUC confidence intervals for each of the algorithms and the CA125 single threshold rule in the primary and secondary analyses for the primary and secondary outcomes for both sets. Both longitudinal algorithms provided similar characteristics for both outcomes in the primary and secondary analyses. In all the subgroups of the analysis, PEB and MMT provided higher sensitivity compared with the single CA125 threshold.

The performance for non-epithelial and borderline cancers diagnosed within 1 year of last annual sample is detailed in Supplementary Table S1. Both algorithms performed similarly.

Figure 2 shows the ROC curves for iEOC/PPC diagnosed >1 and ≤2 years from last annual sample (secondary analysis). Here, MMT (0.613) had slightly higher AUC compared to the CA125 (0.598), although the difference was not significant (P = 0.639).

In the largest available serial dataset of CA125 results in the general population comprising of 347,002 serial annual CA125 measurements from 50,083 women who participated in multimodal screening in UKCTOCS with no selection bias, two serial biomarker algorithms had high and comparable performance in the context of a first line screening test for invasive epithelial ovarian/tubal/peritoneal cancer and were significantly superior to a CA125 cutoff value. We have previously reported that the longitudinal algorithm ROCA outperforms CA125 cutoff values (6). We now show that in comparison with thresholds, other longitudinal algorithms have similar superior performance as a first line test as ROCA (sensitivity 87.1%; specificity 87.6%; AUC 0.915) in UKCTOCS (6). The results emphasize the need to incorporate serial change in biomarker levels in the context of screening and early detection of cancer. Screening is not recommended in the general (low-average risk) population as there is no definitive evidence of a mortality benefit (13). However, our findings have immediate implications for high-risk women in countries where CA125 screening is an option (14). The results clearly show that longitudinal approaches are better tools for the early detection of invasive epithelial ovarian cancer than a single threshold rule, which is the current norm.

We compared two serial algorithms, PEB (10) which is the only other reported serial algorithm that has been used for ovarian cancer screening and our newly developed algorithm (MMT) as a first line test for ovarian cancer screening. The MMT evaluates the dynamics of longitudinal markers by analyzing different trend indices whereas the PEB models marker trajectory in healthy individuals over time to generate person-specific positivity thresholds. We developed the MMT algorithm and trained it together with the PEB in a random training set which included half the healthy women and half the ovarian/tubal/PPC cancers that were diagnosed before December 31, 2014 in the MMS arm of UKCTOCS. The ROCA, which is built on a change-point pattern in an individual's CA125 values, was developed on data from previous trials and was prospectively evaluated in UKCTOCS. In future, it will be further refined using the data from the UKCTOCS training set and the refined ROCA will be compared with MMT and PEB performance in the validation test. The advantages of the MMT and PEB algorithms are that they are computationally simpler than ROCA and therefore can be more easily applied to longitudinal analysis of multiple biomarkers. The MMT algorithm is based on the construction of a logistic regression, and therefore for any additional biomarker we only have to calculate trend indices, add them to the logistic regression model and fit it. An advantage of ROCA is that it incorporates tumor doubling into the model, a well-accepted biological dynamic in cancer biology, and is therefore potentially more powerful than algorithms that do not incorporate such biology.

Our results confirm the superiority of serial algorithms for detection of iEOC/PPC diagnosed within one year of the last annual screen. Previous retrospective analysis has involved small sample sets. Drescher and colleagues (15) evaluated PEB in a serial serum CA125 sample set from 44 incident ovarian cancer cases identified from participants in the PLCO (Prostate, Lung, Colorectal, and Ovarian) Cancer Screening Trial Comparison. Application of these new algorithms require incorporation into a multimodal strategy with development of cutoff values so that women can be triaged to repeat CA125 testing and second line tests such as transvaginal ultrasound (11) or other novel tests such as circulating tumor DNA (ctDNA; ref. 16). The latter are essential to increase the specificity of the screening strategy and decrease the number of women referred to surgery.

In our secondary analysis where we determined sensitivity of the serial algorithms for detection of cases diagnosed more than one year but within 2 years after the annual sample, both MMT and PEB detected similar small proportions of cases but it is likely that this would not have led to improved lead time. This suggests that further improvements in sensitivity require inclusion of additional ovarian cancer biomarkers to confirm the CA125 trend detected by the serial algorithms. A highly specific marker such as ctDNA would be ideal, but less specific markers such as HE4 may also contribute to earlier diagnosis. We are evaluating the HE4 in this sample set and will report in the near future.

Key strengths of our analysis are the size of the dataset, use of the entire cohort with minimal selection bias, completeness of data on cancers diagnosed in the cohort ensured by linkage to national cancer/death registries using a unique identifier together with two rounds of postal follow-up and independent blinded outcome review of iEOC/PPC. The test results in the validation set were generated by OB who was blinded to outcomes with the unblinding and statistical analysis done independently by MB. The main limitation is that we are only able to assess the algorithms as first line tests. Hence, it is not possible to assess the true performance characteristics when incorporated into a multimodal strategy.

In conclusion, our analysis provides definitive evidence of the superiority of longitudinal algorithms compared with single-threshold rules, which is the current norm for interpretation of serum CA125 as a first line test in ovarian screening. It is likely that this also applies to other serum markers used in cancer screening. Use of these newer algorithms in ovarian cancer screening requires incorporation into a multimodal strategy and evaluation in clinical trials to assess overall performance.

J.F. Timms is a consultant/advisory board member for CRUK. S.J. Skates reports receiving speakers bureau honoraria from Duke University and the University of Pennsylvania, is a consultant/advisory board member for Abcodia Inc., LUNGevity Foundation and SISCAPA Assay Technologies. I. Jacobs is an employee of and has ownership interests (including patents) at and is a consultant/advisory board member for Abcodia Ltd. U. Menon has ownership interests (including patents) at and reports receiving commercial research grants from Abcodia Pvt. Ltd. No potential conflicts of interest were disclosed by the other authors.

Conception and design: O. Blyuss, A. Gentry-Maharaj, R. Manchanda, M. Parmar, I. Jacobs, A. Zaikin, U. Menon

Development of methodology: O. Blyuss, I.P. Mariño, R. Manchanda, M. Parmar, A. Zaikin, U. Menon

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): A. Ryan, A. Gentry-Maharaj, R. Manchanda, I. Jacobs, U. Menon

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): O. Blyuss, M. Burnell, A. Ryan, A. Gentry-Maharaj, J.F. Timms, A. Zaikin, U. Menon

Writing, review, and/or revision of the manuscript: O. Blyuss, M. Burnell, A. Ryan, A. Gentry-Maharaj, I.P. Mariño, R. Manchanda, J.F. Timms, S.J. Skates, A. Zaikin, U. Menon

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): A. Ryan, J. Kalsi, I. Jacobs, U. Menon

Study supervision: U. Menon

Other (algorithm construction): O. Blyuss

Other (contributions to discussions during algorithm development and project management): J. Kalsi

The analysis is part of PROMISE, which was funded through Cancer Research UK PRC Program Grant A12677 and by The Eve Appeal. It was supported by the National Institute for Health Research (NIHR) University College London Hospitals (UCLH) Biomedical Research Centre. UKCTOCS was core funded by the Medical Research Council, Cancer Research UK, and the Department of Health with additional support from the Eve Appeal, Special Trustees of Bart's and the London, and Special Trustees of UCLH. We thank all the trial participants and all the staff involved in the UKCTOCS trial.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.