Background:

In this study, we evaluated accuracy of HPV testing on self-samples versus clinician-taken samples through the VALHUDES protocol. VALHUDES was designed as a diagnostic test accuracy study, where women referred to colposcopy collected self-samples followed by clinician-taken cervical samples.

Methods:

Four hundred eighty-five women recruited in five colposcopy clinics (median age = 40 years; IQR, 31–49) with valid results for all specimens were included in the main analysis: 230 vaginal self-samples were collected with Evalyn Brush and 255 with Qvintip. Cervical samples were taken by the gynecologist with the Cervex-Brush. HPV testing was performed with BD Onclarity HPV assay (Onclarity). Colposcopy and histology were used as the reference standard for accuracy estimation.

Results:

The sensitivity for CIN2+ on vaginal self-samples overall was not different from cervical samples (ratio = 0.96; 95% CI, 0.90–1.03), whereas specificity was significantly higher (ratio = 1.09; 95% CI, 1.02–1.16). However, the relative accuracy (self- vs. clinician sampling) differed by vaginal collection device: relative sensitivity and specificity ratios of 1.00 (95% CI, 0.94–1.06) and 1.15 (95% CI, 1.05–1.25), respectively for Evalyn-Brush; 0.91 (95% CI, 0.79–1.04) and 1.03 (95% CI, 0.95–1.13), respectively for Qvintip.

Conclusions:

Clinical accuracy of BD Onclarity HPV assay on vaginal self-samples was not different from cervical samples.

Impact:

VALHUDES study showed that HPV testing with Onclarity HPV on vaginal self-samples is similarly sensitive compared with cervical specimens. However, differences in accuracy by self-sampling devices, although not significant, were noted. Onclarity HPV testing on vaginal self-samples following validated collection and handling procedures may be used in primary cervical cancer screening.

Women who do not participate in cervical cancer screening remain at highest risk of developing invasive cervical cancer (ICC; refs. 1, 2). During the last decades, vaginal and urine self-sampling devices have been developed and received international attention of the scientific community and public health authorities. These self-sampling devices can be offered to underscreened women to increase the participation rate and prevent development of ICC (3, 4).

Many countries worldwide have introduced or have been piloting HPV-based cervical cancer screening, which showed high effectiveness in long-term protection against cervical cancer (5–10). In the current outlook of HPV-based cervical cancer screening programs, self-sampling plays an important role in reaching underscreened women (4). Offering self-samples has shown to be more effective in triggering a response than an invitation to visit a physician or nurse for collection of a cervical specimen (4).

More than 200 commercial HPV tests are currently available on the market, of which only few have been validated and approved for cervical cancer screening on clinician collected cervical specimens (11, 12). None of these tests have been approved or validated on self-samples, nevertheless several countries have implemented HPV testing on self-samples within HPV based screening programmes (8–10). Because of lack of data and international consensus on noncervical specimens it is necessary to further study, standardize, optimize, and validate HPV testing using self-sampling devices (13). Therefore, a protocol for validation of human papillomavirus assays and collection devices for HPV testing on self-samples and urine samples (VALHUDES) was established to provide scientific community and stakeholders with comparative accuracy data of HPV tests on noncervical and cervical specimens (14). The first evidence from VALHUDES suggested similar accuracy of the RealTime High-Risk HPV assay (Abbott GmbH) on first-void urine and vaginal self-samples compared with cervical specimens (15).

In this report, we evaluate the clinical performance of BD Onclarity HPV assay (Onclarity HPV; BD Diagnostics) on vaginal self-samples collected either with Evalyn Brush (Rovers Medical Devices) or Qvintip (Aprovix AB) among women referred to colposcopy. Because two self-sampling devices were used in the study, clinical accuracy was demonstrated for both devices separately and combined. In addition, we evaluated analytical performance of Onclarity HPV and compared cycle threshold (Ct) values across the specimens.

Study design

The VALHUDES framework (NCT03064087) was established to evaluate clinical performance of HPV assays on self-collected vaginal samples following a diagnostic test accuracy design (14). The first VALHUDES instalment recruited 523 women [median age = 40; interquartile range (IQR), 31–49 years] referred because of previous HPV infection or cervical abnormality to one of five Belgian colposcopy centers [University Hospitals of Antwerp (UZA), Brussels (UZ Brussels), Ghent (UZ Ghent), Liège (CHU de Liège), and the General Regional Hospital Heilig Hart Tienen (RZ Tienen)] between December 2017 and January 2020 (14, 15). Women were excluded from the study if they were pregnant, had hysterectomy or were incapable to understand and sign inform consent or refused to participate. Women were contacted by phone by the nurse inviting them to participate in the study. Upon arrival at the colposcopy center all study participants confirmed and signed the informed consent form.

At the colposcopy center women were instructed to collect a vaginal self-sample with Evalyn Brush (Rovers Medical Devices) or Qvintip (Aprovix AB) followed by the collection of cervical specimens by a gynecologist using the Cervex-Brush (Rovers Medical Devices) in agreement with European guidelines (16). Colposcopy centers in Antwerp and Ghent started to offer Evalyn Brush devices to women, whereas colposcopy centers in Brussels, Liege, and Tienen started using Qvintip. When about half of the required sample size was reached, sampling devices were switched across the study centers.

Cervical samples were suspended in 20 mL PreservCyt medium (Hologic, Inc.) by the gynecologist after collection. Cervical samples and dry vaginal brushes were stored at the colposcopy clinics at room temperature. Both samples were sent to Algemeen Medisch Laboratorium (AML; Antwerp) for storage and further processing within maximum of 6 days after collection. In the laboratory, the dry vaginal specimens were suspended into 20 mL PreservCyt medium. All cervical and vaginal samples were stored at 4°C for a maximum up to 3 months. Thereafter, samples were vortexed for 15 to 20 seconds, aliquoted into 1 mL aliquots and frozen at  −80°C (BioBank, BB190002).

VALHUDES trial (NCT03064087) was approved by the central Ethics Committee of the University Hospital of Antwerp/University of Antwerp (B300201733869) and the local Ethics Committees of all the other involved centers. The study was conducted in accordance of Declaration of Helsinki. Written informed consent was obtained from all study participants prior to enrolment.

HPV testing

HPV testing was performed using real-time PCR-based BD Onclarity HPV assay, which targets E6 and E7 oncogenes of 14 high-risk HPV (hrHPV) genotypes and human β-globin as an internal control. Onclarity HPV has been clinically validated for cervical cancer screening using clinician-collected cervical specimens transferred in PreservCyt or SurePath (BD Diagnostics) solution (11). The assay consists of three wells with four channels each. Each channel targets three groups of HPV types and β-globin. The assay identifies six individual genotypes (HPV16, 18, 31, 45, 51, and 52) and eight genotypes in three groups (HPV33/58, 35/39/68, and 56/59/66). HPV testing was performed in batches of 30 tests per run on the BD Viper LT system (17). Briefly, 0.5 mL of sample was transferred into an BD Viper LT diluent tube coded with the same unique lab sample code. The sample tube was placed in the BD Viper LT system for fully automated DNA extraction and amplification. Real-time PCR and result interpretation are automatically performed by the BD Viper LT software (17, 18). HPV testing on cervical specimens was considered as the comparator test and on vaginal self-samples as the index test. We used colposcopy and histology as the reference standard. Samples from women without abnormal colposcopy impression and without biopsy outcome were classified as <CIN2. If a particular patient had multiple biopsy outcomes, only the most severe outcome was considered for the analysis. Samples were considered HPV positive if cycle threshold Ct value was ≤38.3 for HPV16 and ≤34.2 for all other types, as defined by manufacturer.

Statistical analysis

Absolute and relative accuracy were estimated for the whole study population and for women of 30 years and older. Sensitivity analysis was performed to evaluate absolute and relative accuracy for women younger than 30. The McNemar test and paired 95% confidence intervals were computed to assess differences in sensitivity and specificity between vaginal and cervical specimens. Differences are not statistically significant when confidence intervals include unity (PMcN > 0.05). When the relative sensitivity and specificity is <1 and confidence intervals exclude unity (PMcN < 0.05), we conclude that Onclarity HPV is less sensitive or specific on vaginal self-samples than on the clinician-taken comparator sample. When the relative sensitivity and specificity is >1 and confidence intervals exclude unity (PMcN < 0.05), we conclude that Onclarity HPV is more sensitive or specific on vaginal self-samples than on clinician-taken sample. The power analysis is described in details in VALHUDES protocol (14).

Cohen κ was used to assess agreement between cervical and vaginal samples (combined and separately for each self-sampling device) as follows: 0.00–0.19 as poor, 0.20–0.39 as fair, 0.40–0.59 as moderate, 0.60–0.79 as good, and 0.80–1.00 as excellent concordance. Agreement and Cohen κ were estimated for overall hrHPV positivity, individual HPV genotypes and groups as follows: HPV16, HPV18, HPV31, HPV45, HPV51, HPV52, HPV33/58, HPV35/39/68, HPV56/59/66, and other hrHPV (other than 16/18). We used the Mann–Whitney test to compare the differences in viral and β-globin median Ct values between matched cervical and vaginal specimens (combined and separately for each self-sampling device). The Wilcoxon signed-rank test was used to assess the difference in viral and β-globin Ct values between Evalyn Brush and Qvintip samples for independent comparisons. We used boxplots indicating median Ct values, interquartile ranges, and extreme values (whiskers) for graphical demonstration of hrHPV and β-globin Ct values. To compare overall hrHPV signal strengths in case of multiple HPV infections, only the lowest Ct value was considered. Similarly, as three channels report β-globin Ct value, only the lowest signal was considered for comparison. Statistical analyses were performed using Stata 14.2 (College Station).

Ethical approval

VALHUDES trial (NCT03064087) was approved by the central Ethics Committee of the University Hospital of Antwerp/University of Antwerp (B300201733869) and the local Ethics Committees of all the other involved centers. The study was conducted in accordance of Declaration of Helsinki. Written informed consent was obtained from all study participants prior to enrolment.

Study characteristics

Overall, of 523 recruited women (median age = 40; IQR, 31–49 years), 24 samples were excluded due to major protocol violations (15, 19). HPV testing was performed on 499 duplets of samples, of which 485 (median age = 40 years; IQR, 31–49) were included in the main analysis. Six cervical and eight vaginal samples were excluded due to a retest failure on the respective specimens (Fig. 1). Median age in women with <CIN2 (41 years, IQR, 32–50) was higher than in women with CIN2+ (35 years, IQR, 29–44; P-value Mann–Whitney test = 0.002). Forty-seven percent (230/485) of samples were collected with Evalyn Brush (median age = 40; IQR, 31–47) and 52% (255/485) with Qvintip (median age = 40; IQR, 30–50). Characteristics of the study population are reported in Table 1.

Figure 1.

Flow chart of samples included in the VALHUDES trial tested with the BD Onclarity HPV assay. Grey boxes represent excluded samples. Detailed exclusions are reported elsewhere (15).

Figure 1.

Flow chart of samples included in the VALHUDES trial tested with the BD Onclarity HPV assay. Grey boxes represent excluded samples. Detailed exclusions are reported elsewhere (15).

Close modal
Table 1.

HPV prevalence and disease outcome by age group and colposcopy centers.

Cervical hrHPVVaginal (E+Q) hrHPVDisease outcome
Age categoryParticipantsPosPos<CIN2CIN2+CIN3+
N (%)N (%)N (%)N (%)N (%)N (%)
<30 96 (19.8) 65 (23.4) 62 (24.0) 72 (18.1) 24 (27.9) 11 (24.4) 
30–34 85 (17.5) 54 (19.4) 47 (18.2) 65 (16.3) 20 (23.3) 10 (22.2) 
35–39 54 (11.1) 24 (8.6) 26 (10.1) 46 (11.5) 8 (9.3) 4 (8.9) 
40–44 69 (14.2) 40 (14.4) 36 (14.0) 53 (13.3) 16 (18.6) 7 (15.6) 
45–49 63 (13.0) 26 (9.4) 26 (10.1) 57 (14.3) 6 (7.0) 3 (6.7) 
50–54 49 (10.1) 24 (8.6) 23 (8.9) 46 (11.5) 3 (3.5) 2 (4.4) 
55–69 43 (8.9) 31 (11.2) 27 (10.5) 37 (9.3) 6 (7.0) 6 (13.3) 
60+ 26 (5.4) 14 (5.0) 11 (4.3) 23 (5.8) 3 (3.5) 2 (4.4) 
Total 485 (100.0) 278 (100.0) 258 (100.0) 399(100.0) 86 (100.0) 45 (100.0) 
Colposcopy Participants Pos Pos <CIN2 CIN2+ CIN3+ 
center N (%) N (%) N (%) N (%) N (%) N (%) 
Antwerp 18 (3.7) 10 (3.6) 6 (2.3) 18 (4.5) 0 (0.0) 0 (0.0) 
Brussels 70 (14.4) 40 (14.4) 40 (15.5) 64 (16.0) 6 (7.0) 6 (13.3) 
Ghent 215 (44.3) 98 (35.3) 93 (36.1) 186 (46.6) 29 (33.7) 11 (24.4) 
Liege 39 (8.0) 33 (11.9) 25 (9.7) 13 (3.3) 26 (30.2) 22 (48.9) 
Tienen 143 (29.5) 97 (34.9) 94 (36.4) 118 (29.6) 25 (29.1) 6 (13.3) 
Cervical hrHPVVaginal (E+Q) hrHPVDisease outcome
Age categoryParticipantsPosPos<CIN2CIN2+CIN3+
N (%)N (%)N (%)N (%)N (%)N (%)
<30 96 (19.8) 65 (23.4) 62 (24.0) 72 (18.1) 24 (27.9) 11 (24.4) 
30–34 85 (17.5) 54 (19.4) 47 (18.2) 65 (16.3) 20 (23.3) 10 (22.2) 
35–39 54 (11.1) 24 (8.6) 26 (10.1) 46 (11.5) 8 (9.3) 4 (8.9) 
40–44 69 (14.2) 40 (14.4) 36 (14.0) 53 (13.3) 16 (18.6) 7 (15.6) 
45–49 63 (13.0) 26 (9.4) 26 (10.1) 57 (14.3) 6 (7.0) 3 (6.7) 
50–54 49 (10.1) 24 (8.6) 23 (8.9) 46 (11.5) 3 (3.5) 2 (4.4) 
55–69 43 (8.9) 31 (11.2) 27 (10.5) 37 (9.3) 6 (7.0) 6 (13.3) 
60+ 26 (5.4) 14 (5.0) 11 (4.3) 23 (5.8) 3 (3.5) 2 (4.4) 
Total 485 (100.0) 278 (100.0) 258 (100.0) 399(100.0) 86 (100.0) 45 (100.0) 
Colposcopy Participants Pos Pos <CIN2 CIN2+ CIN3+ 
center N (%) N (%) N (%) N (%) N (%) N (%) 
Antwerp 18 (3.7) 10 (3.6) 6 (2.3) 18 (4.5) 0 (0.0) 0 (0.0) 
Brussels 70 (14.4) 40 (14.4) 40 (15.5) 64 (16.0) 6 (7.0) 6 (13.3) 
Ghent 215 (44.3) 98 (35.3) 93 (36.1) 186 (46.6) 29 (33.7) 11 (24.4) 
Liege 39 (8.0) 33 (11.9) 25 (9.7) 13 (3.3) 26 (30.2) 22 (48.9) 
Tienen 143 (29.5) 97 (34.9) 94 (36.4) 118 (29.6) 25 (29.1) 6 (13.3) 

Abbreviations: CIN, cervical intraepithelial neoplasia; E+Q, samples collected with Evalyn Brush (E) and Qvintip (Q) combined; N, number, Pos, positive.

Clinical and analytical performance

Onclarity HPV assay on cervical samples was positive in 78 of 86 CIN2+ and 42 of 45 CIN3 cases with a corresponding absolute sensitivity of 91% (95% CI, 83%–96%) and 93% (95% CI, 82%–99%), respectively. Although, on vaginal self-samples 75 of 86 CIN2+ and 40 of 45 CIN3 cases were HPV positive corresponding to 87% (95% CI, 78%–93%) and 89% (95% CI, 76%–96%) absolute sensitivity values, respectively. Of 399 <CIN2 cases, 199 were HPV negative on cervical (specificity 50%; 95% CI, 45%–55%) and 216 were negative on vaginal samples (specificity 54%; 95% CI, 49%–59%; Table 2).

Table 2.

Absolute clinical sensitivity and specificity of the BD Onclarity HPV assay on clinician-collected and self-samples.

SensitivitySensitivitySpecificity
n(95% CI) CIN2+n(95% CI) CIN3n(95% CI) <CIN2
Total study population (n = 485) 
 Cervical 78/86 0.91 (0.83–0.96) 42/45 0.93 (0.82–0.99) 199/399 0.50 (0.45–0.55) 
 Vaginal (E+Q) 75/86 0.87 (0.78–0.93) 40/45 0.89 (0.76–0.96) 216/399 0.54 (0.49–0.59) 
 Evalyn Brush 45/49 0.92 (0.80–0.98) 22/24 0.92 (0.73–0.99) 102/181 0.56 (0.49–0.63) 
 Qvintip 30/37 0.81 (0.65–0.92) 18/21 0.86 (0.64–0.97) 114/218 0.52 (0.45–0.59) 
Women ≥ 30 years old (n = 390) 
 Cervical 59/64 0.92 (0.83–0.97) 32/34 0.94 (0.80–0.99) 175/331 0.53 (0.47–0.58) 
 Vaginal (E+Q) 54/62 0.87 (0.76–0.94) 29/34 0.85 (0.69–0.95) 185/326 0.57 (0.51–0.62) 
 Evalyn Brush 31/33 0.94 (0.80–0.99) 15/17 0.88 (0.64–0.99) 90/156 0.58 (0.50–0.66) 
 Qvintip 23/29 0.79 (0.60–0.92) 14/17 0.82 (0.57–0.96) 95/170 0.52 (0.48–0.63) 
SensitivitySensitivitySpecificity
n(95% CI) CIN2+n(95% CI) CIN3n(95% CI) <CIN2
Total study population (n = 485) 
 Cervical 78/86 0.91 (0.83–0.96) 42/45 0.93 (0.82–0.99) 199/399 0.50 (0.45–0.55) 
 Vaginal (E+Q) 75/86 0.87 (0.78–0.93) 40/45 0.89 (0.76–0.96) 216/399 0.54 (0.49–0.59) 
 Evalyn Brush 45/49 0.92 (0.80–0.98) 22/24 0.92 (0.73–0.99) 102/181 0.56 (0.49–0.63) 
 Qvintip 30/37 0.81 (0.65–0.92) 18/21 0.86 (0.64–0.97) 114/218 0.52 (0.45–0.59) 
Women ≥ 30 years old (n = 390) 
 Cervical 59/64 0.92 (0.83–0.97) 32/34 0.94 (0.80–0.99) 175/331 0.53 (0.47–0.58) 
 Vaginal (E+Q) 54/62 0.87 (0.76–0.94) 29/34 0.85 (0.69–0.95) 185/326 0.57 (0.51–0.62) 
 Evalyn Brush 31/33 0.94 (0.80–0.99) 15/17 0.88 (0.64–0.99) 90/156 0.58 (0.50–0.66) 
 Qvintip 23/29 0.79 (0.60–0.92) 14/17 0.82 (0.57–0.96) 95/170 0.52 (0.48–0.63) 

Abbreviations: CI, confidence interval; CIN, cervical intraepithelial neoplasia; E+Q, samples collected with Evalyn Brush and Qvintip combined; n, number.

Clinical sensitivity and specificity for women ≥ 30 years old are shown in Supplementary Table S1.

Clinical sensitivity for CIN2+ on vaginal samples collected with Evalyn Brush (0.92; 95% CI, 0.80–0.98) was higher than on samples collected with Qvintip (0.81; 95% CI, 0.65–0.92; ratio = 0.91; 95% CI, 0.77–1.08; Table 2).

Clinical sensitivity for CIN2+ on vaginal samples combined was not different from the sensitivity on cervical samples (ratio = 0.96; 95% CI, 0.90–1.03), whereas specificity was significantly higher (ratio = 1.09; 95% CI, 1.02–1.16; Table 3). For Evalyn Brush samples, similar sensitivity for CIN2+ was observed with 95% CI including unity (ratio = 1.00; 95% CI, 0.94–1.06) and higher specificity compared with cervical samples (ratio = 1.15; 95% CI, 1.05–1.25). Although sensitivity for CIN2+ on Qvintip samples versus cervical was 9% lower (ratio = 0.91; 95% CI, 0.79–1.04) with similar specificity (ratio = 1.03; 95% CI, 0.95–1.13), however including unity in the 95% CI (Table 3). Similar accuracy estimates were observed when restricting the analysis to women of 30 years and older compared with the total population (Tables 2 and 3). Absolute and relative accuracy estimates for women below 30 years are reported in Supplementary Tables S1 and S2.

Table 3.

Relative sensitivity and specificity of the BD Onclarity HPV assay on clinician-collected versus self-samples.

Relative sensitivity (95% CI) CIN2+Relative sensitivity (95% CI) CIN3Relative specificity (95% CI) <CIN2
Total study population (n = 485) 
 Vaginal (E+Q) 0.96 (0.90–1.03) 0.95 (0.87–1.05) 1.09 (1.02–1.16) 
 Evalyn Brush 1.00 (0.94–1.06) 1.00 (1.00–1.00) 1.15 (1.05–1.25) 
 Qvintip 0.91 (0.79–1.04) 0.90 (0.73–1.11) 1.03 (0.95–1.13) 
Women ≥ 30 years old (n = 390)   
 Vaginal (E+Q) 0.95 (0.89–1.09) 0.90 (0.80–1.01) 1.08 (1.01–1.16) 
 Evalyn 1.03 (0.97–1.10) 1.00 (1.00–1.00) 1.15 (1.03–1.24) 
 Qvintip 0.84 (0.71–1.00) 0.81 (0.64–1.03) 1.05 (0.95–1.13) 
Relative sensitivity (95% CI) CIN2+Relative sensitivity (95% CI) CIN3Relative specificity (95% CI) <CIN2
Total study population (n = 485) 
 Vaginal (E+Q) 0.96 (0.90–1.03) 0.95 (0.87–1.05) 1.09 (1.02–1.16) 
 Evalyn Brush 1.00 (0.94–1.06) 1.00 (1.00–1.00) 1.15 (1.05–1.25) 
 Qvintip 0.91 (0.79–1.04) 0.90 (0.73–1.11) 1.03 (0.95–1.13) 
Women ≥ 30 years old (n = 390)   
 Vaginal (E+Q) 0.95 (0.89–1.09) 0.90 (0.80–1.01) 1.08 (1.01–1.16) 
 Evalyn 1.03 (0.97–1.10) 1.00 (1.00–1.00) 1.15 (1.03–1.24) 
 Qvintip 0.84 (0.71–1.00) 0.81 (0.64–1.03) 1.05 (0.95–1.13) 

Abbreviations: CI, confidence interval; CIN, cervical intraepithelial neoplasia; E+Q, samples collected with Evalyn Brush and Qvintip combined.

Relative sensitivity and specificity for women ≥ 30 years old are shown in Supplementary Table S2.

Agreement between cervical and vaginal specimens was high with moderate to excellent Kappa values for hrHPV positivity and for individual HPV genotypes or groups of types (Table 4). Fifty percent (242/485) of samples were concordantly hrHPV positive on both specimen types, 7% (36/485) were positive only on cervical and 3% (16/485) only on vaginal specimens (Table 4). HPV test concordance between vaginal Evalyn Brush tended to be higher than the concordance between vaginal Qvintip and cervical samples (Supplementary Table S3).

Table 4.

Type-specific agreement and test concordance between cervical and vaginal (Evalyn Brush + Qvintip) samples.

Type-specific agreement and test concordance between cervical and vaginal (Evalyn Brush + Qvintip) samples.
Type-specific agreement and test concordance between cervical and vaginal (Evalyn Brush + Qvintip) samples.

Median Ct values for HPV and β-globin were lower in cervical compared with vaginal samples overall (Evalyn Brush + Qvintip), indicating higher concentrations of viral and human DNA in cervical specimens (Fig. 2; Supplementary Table S4A). Genotype-specific Ct values were lower in cervical samples for HPV16, HPV31, and HPV52 than in the vaginal samples (Supplementary Table S4). No difference in HPV and β-globin median Ct values was found between Evalyn Brush and Qvintip samples (Supplementary Table S4D). The median viral Ct was lower among women with CIN2+ than in women with <CIN2 in cervical samples, but not in vaginal (P values: 0.007 and 0.179, respectively; Supplementary Fig. S1).

Figure 2.

Signal strength of BD Onclarity HPV assay expressed as cycle threshold (Ct) values in cervical and vaginal samples collected with Evalyn Brush or Qvintip. Boxplots indicate median Ct values, interquartile ranges and extreme values (whiskers) for hrHPV and β-globin. For β-globin and in case of multiple HPV infections the lowest Ct value was considered. Samples with Ct values above zero and beyond the manufacturer's cut-offs were considered for comparison. Individual mean HPV and β-globin Ct values are presented in Supplementary Table S4. Higher concentration corresponds with lower Ct values. Therefore, the direction of the Y-axis is reversed.

Figure 2.

Signal strength of BD Onclarity HPV assay expressed as cycle threshold (Ct) values in cervical and vaginal samples collected with Evalyn Brush or Qvintip. Boxplots indicate median Ct values, interquartile ranges and extreme values (whiskers) for hrHPV and β-globin. For β-globin and in case of multiple HPV infections the lowest Ct value was considered. Samples with Ct values above zero and beyond the manufacturer's cut-offs were considered for comparison. Individual mean HPV and β-globin Ct values are presented in Supplementary Table S4. Higher concentration corresponds with lower Ct values. Therefore, the direction of the Y-axis is reversed.

Close modal

In this study, we evaluated BD Onclarity HPV assay on vaginal-self samples collected with either Evalyn Brush or Qvintip device. Our findings demonstrated similar sensitivity and higher specificity of Onclarity HPV on vaginal versus cervical samples. Interestingly, accuracy of HPV testing on samples collected with Evalyn Brush was better than on Qvintip samples. These findings are in line with our previous report and studies by Cadman and colleagues and Jentschke and colleagues (20, 21). Our previous VALHUDES report evaluated accuracy of Abbott RealTime High-Risk HPV on vaginal samples. Sensitivity of the Abbott HPV assay on vaginal samples to detect CIN2+ was lower than on cervical, although a posterior cut-off optimization resulted in accuracy improvements. We also observed slightly better sensitivity of Abbott HPV assay on samples collected with Evalyn Brush, compared with Qvintip (19). In the Predictors 5.1 trial, four vaginal devices (dry flocked swab, Dacron swab, HerSwab, and Qvintip) were evaluated using Onclarity HPV assay in 600 women (20). Authors reported lower accuracy of Qvintip and HerSwab compared with dry flocked and Dacron swab. Jentschke and colleagues performed a direct comparison of Qvintip and Evalyn Brushes in a small study setting suggesting also slightly better sensitivity with the latter brush. Ørnskov and colleagues evaluated the performance of cobas 4800 HPV assay on vaginal samples collected with Evalyn Brush in a larger referral population with over 300 participants. In agreement with VALHUDES, sensitivity on vaginal samples was not different compared with cervical samples (22). A large randomized-non-inferiority trial also demonstrated a similar accuracy in vaginal versus cervical samples where the Evalyn brush was resuspended in 1.5 mL of PresevCyt media versus 20 mL for the physician-collected sample (23). A recent meta-analysis including 81 studies concluded that PCR-based HPV DNA assays on self-samples are non-inferior to clinician samples to detect CIN2+ or CIN3+ (4). Another meta-analysis showed high agreement of PCR-based HPV assays on self-collected vaginal versus clinician-collected cervical samples suggesting new extension for validation of HPV assays based on the agreement between the specimens (24). This may facilitate validation of HPV assays and devices by evaluating agreement between already validated assay/device with a new one.

Evidence that HPV testing provides better and longer protection against cervical cancer triggered many countries to revise the screening recommendations and switch from cytology to HPV-based cervical cancer screening (5, 6, 8, 9). Nevertheless, coverage of the screening programs remains lower than 70% recommended by WHO (9, 25). A considerable fraction of women does not participate in the screening programs and is at highest risk of developing cervical cancer; therefore, self-sampling strategies could be implemented to reach nonresponders and increase the participation rate. Self-sampling could also be used in resource poor countries, although, before implementation within the screening programmes, it is necessary to generate sufficient scientific evidence of HPV test accuracy on self-samples. Validation of HPV tests on cervical samples are well established. First, in 2009 Meijer guidelines were established outlining requirements of HPV tests to be suitable for primary HPV-based cervical cancer screening. Following criteria were established: sensitivity (≥90%) and specificity (≥98%) for CIN2+ compared with already clinically validated tests, and high inter- and intra-reproducibility (26). In 2012, extended guidelines were formulated for validation of HPV genotyping tests (VALGENT; ref. 27). Four VALGENT frameworks have been designed and more than 50 studies have been published since, to further optimize and provide evidence of HPV tests suitable for cervical cancer screening on cervical samples, but not on self-samples (28). Therefore, new international consensus is necessary to provide evidence and accuracy data of HPV test and devices on self-samples.

In the Netherlands, self-samples are offered to women since 2017, and in 2022 will change from an opt-in to opt-out program. The Dutch HPV-based cervical cancer screening program has published first data including 30,808 vaginal samples collected with Evalyn Brush and 456,207 cervical samples. Dutch colleagues reported slightly lower modelled sensitivity for CIN3+ of HPV testing with cobas 4800 HPV assay on samples collected with Evalyn Brush compared with the clinician-taken samples (9). In line with the Dutch report, the median viral Ct values in both VALHUDES studies were lower in cervical samples, reflecting higher viral load concentration in cervical specimens compared with vaginal (19). In addition, present VALHUDES demonstrated lower β-globin Ct value in cervical versus the vaginal samples, which may have been due to the 20 mL resuspension volume used for both sample types. A Danish self-sampling study showed the opposite trend, β-globin Ct value in vaginal samples taken with the Evalyn brush were approximately 2 Cts lower than corresponding cervical samples, where the self-sample was resuspended in 3 mL versus 10 mL for the physician sample (SurePath preservative media; ref. 29). Interpretation of Ct values data and resuspension volumes of vaginal samples may require further research.

Interestingly, the Onclarity HPV testing on vaginal samples demonstrated better specificity compared with Onclarity testing on cervical samples. Vaginal samples were resuspended in 20 mL PreservCyt solution, which is the standard transport volume for cervical specimens but not for vaginal. In addition, cervical samples were collected by a gynecologist targeting the transformation zone where virus replication occurs (30), whereas vaginal samples were taken by women from the vaginal tract. It is known that the hrHPV viral load is highest in scraped cervical cells and decreases progressively when samples are taken more distally: upper vagina > lower vagina > perineum (31). Accordingly, our data demonstrated higher viral Ct values in vaginal samples indicating lower viral load. The dilution in a quite large volume of transport medium and lower amount of virus in the vagina both may explain lower analytical sensitivity and higher analytical and clinical specificity in vaginal specimens (nondetection of small amounts of virus in women without high-grade lesions).

According to the STARD guidelines the evaluation of an assay should be conducted in the environment where the assay will be used (32). HPV testing on self-samples will have its main application in primary cervical cancer screening. However, VALHUDES is based on the findings of our previous meta-analyses, which demonstrated that the relative sensitivity and relative specificity of HPV testing on self-collected versus clinician-collected samples were similar in primary screening and in follow-up settings (3, 4). There are several advantages of running the study in a colposcopy setting including: efficiency (∼500 patients are sufficient to reach required power), shorter duration of enrolment, and lack of verification bias which is a major issue in primary screening studies. Nevertheless, besides the strengths of the VALHUDES design we recognize that the enrolment in a colposcopy setting as a weakness with respect to the spectrum of disease. Another limitation is that age range of our study population was 19 to 70 years, whereas recommended HPV based screening age is above 29 years. Yet, the clinical performance in screening recommended age group was similar to the total population. Finally, women with normal colposcopy impressions and without reference biopsy were classified as non CIN2+. This classification could have impacted absolute accuracy estimates, as small fraction of women could have been misdiagnosed by colposcopists, yielding a possible overestimation of absolute sensitivity and underestimation of absolute specificity for both samples types. However, such a misclassification bias, if present, occurs in both types of samples (since colposcopists are not aware of HPV test results on either type) and, therefore, influence on relative accuracy would be limited with a possible tendency of relative accuracy estimates towards unity (33).

Our study demonstrated similar accuracy of hrHPV testing with Onclarity HPV assay on self-collected vaginal samples as on clinician-collected cervical specimens, in particular when the self-samples were collected with Evalyn Brush.

A. Latsuzbaia reports grants from Horizon 2020 Framework Programme for Research and Innovation of the EU (grant no. 847845; RISCC Network) and grants and nonfinancial support from VALHUDES during the conduct of the study. S. Van Keer reports grants from University of Antwerp and Research Foundation – Flanders (FWO), and other support from Becton Dickinson during the conduct of the study; grants from Research Foundation – Flanders (FWO), University of Antwerp, and Eurostars (H2020) outside the submitted work; and The University of Antwerp received payment for participation of S Van Keer in an Advisory Board of Novosanis (Subsidiary of OraSure Technologies Inc.). All funds are handled and managed by the University of Antwerp. E. Peeters reports that E. Peeters was supported by the RISCC Network funded by the Horizon 2020 Framework of DGResearch and Innovation, European Commission, Brussels, Belgium (grant no. 847845) at the time of data analysis. A. Vorsters reports other support from Becton Dickinson and grants from University of Antwerp during the conduct of the study; grants from Merck, GSK, Novosanis, Hologic, Abbott outside of the submitted work; also has a patent for liquid collection device, liquid sampler, kit of parts, and method for assembly issued and licensed; and reports employment with Uantwerp spinoff company Novosanis now fully owned by Orasure in which does not have financial interest. M. Arbyn reports grants from Horizon 2020 Framework Programme for Research and Innovation of the EU (grant no. 847845; RISCC Network) and grants and nonfinancial support from VALHUDES during the conduct of the study. No disclosures were reported by the other authors. The VALHUDES project is a researcher-induced study, designed by Sciensano (Principal Investigator; Brussels, Belgium), CEV (University of Antwerp, Antwerp, Belgium), and AML (Antwerp, Belgium). Manufacturers of HPV assays and devices can participate in the VALHUDES framework contributing financial support and equipment for laboratory testing and statistical analysis under the condition of accepting independent publication of results. This research was supported by BD (BD Diagnostics, Sparks, MD, USA), Novosanis NV (Wijnegem, Belgium), University of Antwerp (Antwerp, Belgium). The study group received sample collection devices from Rovers Medical Devices B.V. (Oss, The Netherlands) and Aprovix AB (Uppsala, Sweden).

A. Latsuzbaia: Data curation, formal analysis, visualization, writing–original draft, writing–review and editing. D. Vanden Broeck: Conceptualization, resources, data curation, funding acquisition, validation, methodology, project administration, writing–review and editing. S. Van Keer: Conceptualization, resources, data curation, funding acquisition, validation, methodology, project administration, writing–review and editing. S. Weyers: Investigation, writing–review and editing. G. Donders: Investigation, writing–review and editing. J. Doyen: Investigation, writing–review and editing. W. Tjalma: Investigation, writing–review and editing. P. De Sutter: Investigation, writing–review and editing. E. Peeters: Conceptualization, resources, data curation, funding acquisition, validation, methodology, project administration, writing–review and editing. A. Vorsters: Conceptualization, resources, data curation, funding acquisition, validation, investigation, methodology, writing–review and editing, project administration. M. Arbyn: Conceptualization, resources, data curation, formal analysis, supervision, funding acquisition, validation, investigation, methodology, project administration, writing–review and editing.

We thank the team members of the Gynaecology Departments of Antwerp University Hospital, Ghent University Hospital, General Regional Hospital Heilig Hart Tienen – Femicare vzw, UZ Brussel – VUB and University Hospital Liège for the study recruitment. We thank L De Baere (AML) and D Maes (AML) for the laboratory work. M. Arbyn and A. Latsuzbaia were supported by the RISCC Network (grant no. 8478459) funded by the Horizon 2020 Program for Research and Innovation of the European Commission (Brussels, Belgium). S. Van Keer was supported by a junior postdoctoral fellowship of the Research Foundation – Flanders (1240220N).

The publication costs of this article were defrayed in part by the payment of publication fees. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734.

Note: Supplementary data for this article are available at Cancer Epidemiology, Biomarkers & Prevention Online (http://cebp.aacrjournals.org/).

1.
Landy
R
,
Sasieni
PD
,
Mathews
C
,
Wiggins
CL
,
Robertson
M
,
McDonald
YJ
, et al
.
Impact of screening on cervical cancer incidence: a population-based case-control study in the United States
.
Int J Cancer
2020
;
147
:
887
96
.
2.
Peirson
L
,
Fitzpatrick-Lewis
D
,
Ciliska
D
,
Warren
R
.
Screening for cervical cancer: a systematic review and meta-analysis
.
Syst Rev
2013
;
2
:
35
.
3.
Arbyn
M
,
Verdoodt
F
,
Snijders
PJ
,
Verhoef
VM
,
Suonio
E
,
Dillner
L
, et al
.
Accuracy of human papillomavirus testing on self-collected versus clinician-collected samples: a meta-analysis
.
Lancet Oncol
2014
;
15
:
172
83
.
4.
Arbyn
M
,
Smith
SB
,
Temin
S
,
Sultana
F
,
Castle
P
.
Detecting cervical precancer and reaching underscreened women by using HPV testing on self samples: updated meta-analyses
.
BMJ
2018
;
363
:
k4823
.
5.
Arbyn
M
,
Ronco
G
,
Anttila
A
,
Meijer
CJLM
,
Poljak
M
,
Ogilvie
G
, et al
.
Evidence regarding human papillomavirus testing in secondary prevention of cervical cancer
.
Vaccine
2012
;
30
:
F88
99
.
6.
Maver
PJ
,
Poljak
M
.
Primary HPV-based cervical cancer screening in Europe: implementation status, challenges, and future plans
.
Clin Microbiol Infect
2020
;
26
:
579
83
.
7.
Saville
M
,
Hawkes
D
,
McLachlan
E
,
Anderson
S
,
Arabena
K
.
Self-collection for under-screened women in a National Cervical Screening Program: pilot study
.
Curr Oncol
2018
;
25
:
e27
32
.
8.
Serrano
B
,
Ibáñez
R
,
Robles
C
,
Peremiquel-Trillas
P
,
de Sanjosé
S
,
Bruni
L
.
Worldwide use of HPV self-sampling for cervical cancer screening
.
Prev Med
2022
;
154
:
106900
.
9.
Inturrisi
F
,
Aitken
CA
,
Melchers
WJG
,
van den Brule
AJC
,
Molijn
A
,
Hinrichs
JWJ
, et al
.
Clinical performance of high-risk HPV testing on self-samples versus clinician samples in routine primary HPV screening in the Netherlands: an observational study
.
Lancet Reg Health Eur
2021
;
11
.
10.
Harder
E
,
Thomsen
L
,
Hertzum-Larsen
R
,
Albieri
V
,
Jochumsen
MVH
,
Juul
K
, et al
.
Determinants for participation in human papillomavirus self-sampling among nonattenders to cervical cancer screening in Denmark
.
Cancer Epidemiol Biomarkers Prev
2018
;
27
:
1342
51
.
11.
Arbyn
M
,
Simon
M
,
Peeters
E
,
Xu
L
,
Meijer
C
,
Berkhof
J
, et al
.
2020 list of human papillomavirus assays suitable for primary cervical cancer screening
.
Clin Microbiol Infect
2021
;
27
:
1083
95
.
12.
Poljak
M
,
Valenčak
AO
,
Domjanič
GG
,
Xu
L
,
Arbyn
M
.
Commercially available molecular tests for human papillomaviruses: a global overview
.
Clin Microbiol Infect
2020
;
26
:
1144
50
.
13.
Arbyn
M
,
Latsuzbaia
A
,
Castle
PE
,
Sahasrabuddhe
VV
,
Broeck
DV
.
HPV testing of self-samples: Influence of collection and sample handling procedures on clinical accuracy to detect cervical precancer
.
Lancet Reg Health Eur
2022
;
14
.
14.
Arbyn
M
,
Peeters
E
,
Benoy
I
,
Vanden Broeck
D
,
Bogers
J
,
De Sutter
P
, et al
.
VALHUDES: a protocol for validation of human papillomavirus assays and collection devices for HPV testing on self-samples and urine samples
.
J Clin Virol
2018
;
107
:
52
56
.
15.
Van Keer
S
,
Peeters
E
,
Vanden Broeck
D
,
De Sutter
P
,
Donders
G
,
Doyen
J
, et al
.
Clinical and analytical evaluation of the RealTime High Risk HPV assay in Colli-Pee collected first-void urine using the VALHUDES protocol
.
Gynecol Oncol
2021
;
162
:
575
83
.
16.
Arbyn
M
,
Herbert
A
,
Schenck
U
,
Nieminen
P
,
Jordan
J
,
McGoogan
E
, et al
.
European guidelines for quality assurance in cervical cancer screening: recommendations for collecting samples for conventional and liquid-based cytology
.
Cytopathology
2007
;
18
:
133
9
.
17.
Ejegod
D
,
Serrano
I
,
Cuschieri
K
,
Nussbaumer
W
,
Vaughan
L
,
Ahmad
A
, et al
.
Clinical validation of the BD Onclarity HPV assay using a non-inferiority test
.
J Med Microbiol Diagn S
2013
;
3
:
2161
0703
.
18.
Bonde
JH
,
Pedersen
H
,
Quint
W
,
Xu
L
,
Arbyn
M
,
Ejegod
DM
, et al
.
Clinical and analytical performance of the BD Onclarity HPV assay with SurePath screening samples from the danish cervical screening program using the VALGENT framework
.
J Clin Microbiol
2020
;
58
:
e01518
19
.
19.
Latsuzbaia
A
,
Vanden Broeck
D
,
Van Keer
S
,
Weyers
S
,
Tjalma
WAA
,
Doyen
J
, et al
.
The clinical performance of the RealTime High Risk HPV assay on self-collected vaginal samples within the VALHUDES framework
.
Microbiol Spectr
2022
:
e0163122
.
20.
Cadman
L
,
Reuter
C
,
Jitlal
M
,
Kleeman
M
,
Austin
J
,
Hollingworth
T
, et al
.
A randomized comparison of different vaginal self-sampling devices and urine for human papillomavirus testing-predictors 5.1
.
Cancer Epidemiol Biomarkers Prev
2021
;
30
:
661
68
.
21.
Jentschke
M
,
Chen
K
,
Arbyn
M
,
Hertel
B
,
Noskowicz
M
,
Soergel
P
, et al
.
Direct comparison of two vaginal self-sampling devices for the detection of human papillomavirus infections
.
J Clin Virol
2016
;
82
:
46
50
.
22.
Ørnskov
D
,
Jochumsen
K
,
Steiner
PH
,
Grunnet
IM
,
Lykkebo
AW
,
Waldstrøm
M
.
Clinical performance and acceptability of self-collected vaginal and urine samples compared with clinician-taken cervical samples for HPV testing among women referred for colposcopy. A cross-sectional study
.
BMJ Open
2021
;
11
:
e041512
.
23.
Polman
NJ
,
Ebisch
RMF
,
Heideman
DAM
,
Melchers
WJG
,
Bekkers
RLM
,
Molijn
AC
, et al
.
Performance of human papillomavirus testing on self-collected versus clinician-collected samples for the detection of cervical intraepithelial neoplasia of grade 2 or worse: a randomised, paired screen-positive, non-inferiority trial
.
Lancet Oncol
2019
;
20
:
229
38
.
24.
Arbyn
M
,
Castle
PE
,
Schiffman
M
,
Wentzensen
N
,
Heckman-Stoddard
B
,
Sahasrabuddhe
VV
.
Meta-analysis of agreement/concordance statistics in studies comparing self- vs clinician-collected samples for HPV testing in cervical cancer screening
.
Int J Cancer
2022
;
151
:
308
12
.
25.
WHO
.
Coverage of national cervical cancer screening program
.
Accessed 2022 May 3; Available from
: https://www.who.int/data/gho/indicator-metadata-registry/imr-details/4693.
26.
Meijer
CJLM
,
Berkhof
J
,
Castle
PE
,
Hesselink
AT
,
Franco
EL
,
Ronco
G
, et al
.
Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older
.
Int J Cancer
2009
;
124
:
516
20
.
27.
Arbyn
M
,
Depuydt
C
,
Benoy
I
,
Bogers
J
,
Cuschieri
K
,
Schmitt
M
, et al
.
VALGENT: a protocol for clinical validation of human papillomavirus assays
.
J Clin Virol
2016
;
76
:
S14
s21
.
28.
Bonde
J
,
Ejegod
DM
,
Cuschieri
K
,
Dillner
J
,
Heideman
DAM
,
Quint
W
, et al
.
The Valgent4 protocol: robust analytical and clinical validation of 11 HPV assays with genotyping on cervical samples collected in SurePath medium
.
J Clin Virol
2018
;
108
:
64
71
.
29.
Lam
JUH
,
Rebolj
M
,
Ejegod
DM
,
Pedersen
H
,
Rygaard
C
,
Lynge
E
, et al
.
Prevalence of human papillomavirus in self-taken samples from screening nonattenders
.
J Clin Microbiol
2017
;
55
:
2913
23
.
30.
Doorbar
J
,
Griffin
H
.
Refining our understanding of cervical neoplasia and its cellular origins
.
Papillomavirus Res
2019
;
7
:
176
79
.
31.
Belinson
JL
,
Hu
S
,
Niyazi
M
,
Pretorius
RG
,
Wang
H
,
Wen
C
, et al
.
Prevalence of type-specific human papillomavirus in endocervical, upper and lower vaginal, perineal and vaginal self-collected specimens: implications for vaginal self-collection
.
Int J Cancer
2010
;
127
:
1151
7
.
32.
Cohen
JF
,
Korevaar
DA
,
Altman
DG
,
Bruns
DE
,
Gatsonis
CA
,
Hooft
L
, et al
.
STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration
.
BMJ Open
2016
;
6
:
e012799
.
33.
Arbyn
M
,
Ronco
G
,
Cuzick
J
,
Wentzensen
N
,
Castle
PE
.
How to evaluate emerging technologies in cervical cancer screening?
Int J Cancer
2009
;
125
:
2489
96
.

Supplementary data