A Randomized Comparison of Different Vaginal Self-sampling Devices and Urine for Human Papillomavirus Testing—Predictors 5.1

Self-sampling for high-risk human papillomavirus (hrHPV) has been recognized as an alternative to clinician collected samples for cervical screening. In the United Kingdom, the uptake of cervical screening has been declining in recent years. The most recent quarterly figures (March 31, 2019) show 70.2% of women in England ages 25–49 years were screened adequately within the previous 3.5 years, and 76.4% for women ages 50 to 64 years within the previous 5.5 years (1).

The introduction of primary screening by hrHPV testing presents an opportunity for underscreened women to be offered self-sampling at home as an alternative to a speculum examination in a health care setting (2, 3). To date, self-sampling has been suggested primarily for those who are underscreened or who have not been screened. However, given a preference for self-sampling by most women, accumulating data that it is as sensitive for HPV testing as clinician taken samples and the potential for major savings for health service, self-sampling has the potential to become the first choice for screening in all women (4–6).

An updated meta-analysis using a range of assays in a range of settings, found PCR-based assays in screening populations had similar sensitivities for self-sampling compared with clinically taken samples (ref. 7, Table 1) with sensitivities of about 96% for CIN2+ and CIN3+ for both methods, and specificities for <CIN2 of about 79%. Similar results were seen in a large randomized trial (8). In general, it has been suggested that the specificity of self-collected, vaginal samples, may be lower than for clinician taken cervical samples due to passenger hrHPV found in the vagina (9), but this is less clearly established. Increasing interest in self-sampling for HPV has led to the development of a number of kits, and dozens of devices are currently commercially available (10). These include brush-type devices and different types of swabs. Some use a liquid specimen transport medium and some are transported dry. The latter has obvious advantages in terms of safety for home use, especially when small children are present, and also fewer restrictions for posting. A number of kits for urine collection are also available (https://www.amazon.co.uk/Medical-Specimen-Collection-Containers/b?ie=UTF8&node=6285950031). However, none of these provide an easy way to collect initial steam urine, which has been shown to be important (11), and also do not contain a stabilization medium, which has to be added separately to avoid DNA degradation.

Cost and simplicity of use are also important factors—especially in low and middle income countries, where self-sampling may prove more cost effective and be the only practical option. In this study, we compared simple swabs against more complex and expensive devices, and also those transported dry versus wet in a liquid transport medium. As vaginal collection has an invasive element we also included urine testing, which is noninvasive and may be more acceptable to some women or cultures, but has been less extensively studied. A systematic review of 14 studies found that urine testing had good accuracy for the presence of cervical HPV, and that initial stream urine yielded more accurate results than random or midstream sampling (12). In a recent study in a referral population, we found similar performance of the Trovagene test and five other HPV assays in clinical cervical samples, but a lower sensitivity in a urine sample (13). Three other small studies with matched urine, self-collected vaginal and clinician taken cervical samples in a referral population have been reported. Two (14, 15) found similar sensitivities for CIN2+ between all three samples, but the third found a lower sensitivity for urine (16). All used an Evalyn (or similar Viba) brush for the vaginal samples. Urine was collected in a sterile pot in two of these studies (14, 17) before adding a preservative, and the third used the Colli-Pee device (15). Different HPV assays were used in the different studies. Here we compare the performance of four vaginal sampling devices, including wet and dry transport methods, and an initial stream urine sample in a referral population using the Becton Dickinson Onclarity HPV test. We also conducted additional analyses to examine the effect of adjusting for sample cellularity and the use of different positivity thresholds.

In all cases, an initial stream urine sample was requested as the first sample, using the Colli-Pee collection kit (Novosanis NV), which collects a 20 mL sample of which 7 mL is a prefilled urine conservation medium (UCM). Two vaginal samples were provided subsequently. Women were randomized to use either the dacron-based digene Female Swab Specimen Collection Kit (Qiagen GmbH) placed in 1 mL liquid specimen transport medium (STM; denoted as wet dacron—WD below) and the Copan FLOQswab (Copan Diagnostics Inc) as a dry sample (dry flocked—DF) or the Qvintip Kit (Aprovix AB; QT) and the HerSwab (Eve Medical) both as dry samples (HS; the sampling devices are illustrated in Supplementary Fig. S1). The order of sampling was also randomized, leading to four equal sized groups. Samples were stored locally at 4°C–8°C before being transported to the laboratory within 2 weeks of collection.

Recruitment took place between April 2016 and August 2017. New referrals to the Royal London Hospital colposcopy clinic (London, UK) who had a positive screening result were sent information about the study and an offer to join it with their appointment letter. This was also sent to women under observation for prior abnormalities, provided that their most recent abnormal smear test was within the last 3 years. Pregnancy or ablation/excision treatment to the cervix within 3 years were exclusion criteria. All samples were taken prior to colposcopic examination. Cervical biopsies were only taken where clinically indicated. As well as having a negative biopsy, a negative colposcopy with satisfactory evaluation of the transformation zone was considered to indicate no disease even without a biopsy. Patient outcomes were collected for 1 year after enrollment, and this period would normally include a follow-up visit for women with low-grade disease at first visit. Histology was reported by hospital pathologists and reviewed internally according to NHS Trust quality control practices. Where cytology or histopathology results were regraded following local review the reviewed result was used.

Women were also asked to complete a simple survey evaluating their preference for a vaginal sample versus urine, their preferred vaginal sampling device, discomfort with sample collection, confidence in sampling and ease, or difficulty of each collection method.

The study was reviewed and approved in February 2016 by the London City Road & Hampstead Research Ethics Committee (Reference 15/LO/2134), and was conducted in concordance with the principles sent out in the Declaration of Helsinki guidelines. Written informed consent was obtained from all participating women. The trial was registered with the ISRCTN Registry (ISRCTN68980717).

All samples were tested in the Wolfson Institute of Preventive Medicine laboratory (London, United Kingdom). On receipt, the WD swab samples, which were transported in STM, were frozen at −20°C until further processing. The dry DF, HS, and QT samples were placed into 15 mL tubes, 8 mL of PreservCyt solution (Hologic) was added and the tubes were vortexed thoroughly at maximum speed to resuspend the cells into the solution. These samples were then split into 700 μL aliquots. The WD samples, collected in 1 mL of STM, were split into 100 μL aliquots before freezing. All the aliquots were kept frozen until tested. Urine was refrigerated at 4°C and split into 2 mL vials within 7 days of receipt before being frozen at −20°C.

All samples were tested for hrHPV using the BD Onclarity HPV Assay. This assay is based on simultaneous PCR amplification and detection of target DNA using amplification primers and fluorescent-labeled detector probes. An internal human β-globin (HBB) gene control was used to measure sample cellularity to judge sample adequacy. The assay detects 14 high-risk HPV types overall: six types are individually genotyped (16, 18, 31, 45, 51, and 52) and the other types are reported in groups (33, 58), (56, 59, 66), and (35, 39, 68). The manufacturer's predefined threshold for positivity was a C_t value ≤38.3 for HPV16 and ≤34.2 for all other genotypes and the β-globin internal control. Use of a C_t threshold of either 34.2 or 38.3 for all genotypes was also explored.

For DF, HS, and QT, 500 μL of sample was added to an LBC Diluent tube. A total of 50 μL of the WD sample was added to a tube containing the cervical brush diluent medium. For urine, 2 mL was added to empty LBC Diluent tubes. The Onclarity software produces an “indeterminate” result when no HPV is detected and the HBB internal control did not meet the amplification threshold, indicating insufficient cellularity of the specimen. Samples producing invalid results were repeat tested once. Twenty samples were indeterminate a second time and excluded from the analyses. One urine sample was of insufficient cellularity to provide a valid result.

The C_t values were compared between devices using the unpaired Wilcoxon test. Comparisons of the numbers of women who were hrHPV positive were based on binomial statistics. The Fisher exact test was used when any of the four expected cell values was ≤ 10, but otherwise χ² normal approximations were used.

Agreement in hrHPV positivity between devices was calculated using both an unadjusted approach and also an approach adjusted for cellularity, using the kappa statistic. The direction of discordance (+/− vs. −/+) was calculated by using McNemar OR. Sensitivity for CIN2+ and CIN3+ and specificity for <CIN2 were calculated using standard approaches. C_t values adjusted for cellularity were obtained by first subtracting the internal control C_t value from the raw HPV C_t value. Because of its extensive prior use, WD was selected as the reference device, and a further additive adjustment was made to give the same overall positivity rate as for the raw values for this device at the predefined genotype-specific C_t cutoff (≤38.3 for HPV16 and ≤34.2 for other genotypes) for the primary analysis, and at the 34.3 and 38.3 cutoffs for all genotypes in the secondary analyses. The same additive adjustment was made for all other devices, except for urine, where cellularity was generally higher and a cellularity adjustment was not explored. Bootstrapping was used to estimate the confidence intervals (CI) for the kappa statistic and the difference between the raw and adjusted kappa statistics. A small preliminary study of DNA stability was undertaken using aliquots from 60 women (30 sampled with DF and 30 with HS) comparing baseline levels of DNA and nucleic acid using Nanodrop and RNAse to those of matched aliquots stored at 25°C for 1 or 2 weeks. No significant differences were seen with either assay.

All statistical analyses were performed in R (v3.6.3).

A total of 620 women consented to take part, but 20 were excluded as not eligible and the primary analysis group comprised the remaining 600 women (Fig. 1). Of these, 505 women also provided an initial stream urine sample of between 8 and 20 mL, inclusive of 7 mL UCM preservative.

Median age was 29 years [interquartile range (IQR): 27–33, range 21–63], and 330 (53.2%) were aged less than 30 years, 261 (42.1%) were aged 30–44 years, and 29 (4.4%) were aged 45 or older. Each woman provided two vaginal self-samples (Fig. 1). Age at enrollment, referral cytology reading, time since most recent abnormal cytology and previous treatment were similar across all four randomization groups (Supplementary Table S1).

There were 66 cases of CIN2 and 68 cases of CIN3+ within the 1-year follow-up period. This included two stage 1A1 squamous cell carcinomas (SCC), one stage 1B SCC, and one stage 1A1 adenocarcinoma. A cross-tabulation of histology and cytology results is shown in Supplementary Table S2.

The HPV assay could not be performed on 20 samples (HS = 16; QT = 4) and one urine sample due to low cellularity (Fig. 1). These samples all came from different women. There were no significant differences in hrHPV positivity rates according to collection order for any of the devices (Supplementary Table S3), and results from first and second samples have been combined for all analyses. HPV positivity rates for all hrHPV types combined are shown in Table 1. At the predefined HPV test positivity cutoff of C_t ≤ 38.3 for HPV16 and ≤34.2 for all other genotypes, positivity of 71.7% was seen for WD and DF, with nonsignificantly higher values for urine (76.0%; P = 0.17) and nonsignificantly lower rates for QT (69.3%; P = 0.52) and for HS (65.1%; P = 0.09). Agreement on the same sample between WD and DF was high (kappa = 0.801, 95% CI: 0.777-0.826), but lower between QT and HS (kappa = 0.753, 95% CI: 0.723–0.779) and lower still between urine and the four vaginal samples (kappa ranging from 0.568–0.646), largely due to the higher positivity rate with urine (Supplementary Table S4). Positivity by genotype is shown in Supplementary Table S5. There were no statistically significant or obvious differences in genotype-specific positivity by device, although type 45 was lower for Herswab and Quintip, and the positivity for types 35, 39, and 68 was higher for urine and Quintip using the primary cutoffs without a cellularity adjustment.

The highest sensitivity for CIN3+ was obtained with WD (91.2%). Sensitivities for CIN3+ for the other devices were as follows: urine (89.7%), DF (88.2%), QT (81.8%), and HS (77.4%), none of which were significantly lower than WD (Table 2A; Supplementary Fig. S2). A similar order was seen for CIN2+, and again no differences were significant. Increasing the cutoff to ≤38.3 for all HPV types gave higher sensitivities for all devices (Table 2C). The order of sensitivities between the devices was similar to before, but the differences were smaller, especially for HS and no differences were significant. However, there was a substantial loss of specificity for <CIN2 for all devices, for example, dropping from 34.2% to 26.3% for WD in this referral population. Slightly but not significantly better agreement between the different devices in terms of overall hrHPV positivity was seen at the higher 38.3 cutoff (Supplementary Table S4).

The number of cells collected in vaginal samples by the different devices and the urine sample were compared using the C_t values for the β-globin internal control as a proxy for cell number (Fig. 2). The greatest number of cells was observed for DF [median C_t = 24.3 (IQR:23.4–25.5)], followed by urine [median C_t = 24.6 (IQR:23.4–26.2)], QT [median C_t = 25.3 (IQR:24.0–27.2)], WD [median C_t = 25.7 (IQR:25.2–26.8)], and HS [median C_t = 26.1 (IQR:23.9–30.0)]. DF provided more cells than QT and HS (P < 0.0001 for both tests). A C_t value ≤34.2 for β-globin was used to determine sample adequacy, and samples with values above this level were deemed inadequate unless a positive value was seen for some HPV type. On average HS samples provided the smallest median number of cells and 16 of 300 samples (5.3%) failed to meet this sample adequacy measure. Four QT samples (1.3%) also failed to meet this requirement, as did one urine sample, but all other samples were deemed adequate (Fig. 1). The largest variation in cellular yield based on β-globin C_t values was observed for samples collected with HS and the smallest with WD.

The hrHPV positivity rate was also examined after adjustment for cellularity for the vaginal devices. This was particularly relevant for HS, because of its generally low and variable cellularity. After adjustment its overall positivity rate, sensitivity for CIN2⁺ at the predefined genotype-specific cutoff increased to 85.2%, which was comparable with WD and DF (Table 2A). Supplementary Table S4 indicated agreement between pairs of vaginal devices at the predefined type specific cutoff was nonsignificantly improved after cellular adjustment (WD and DF kappa 0.803 before vs. 0.858 after; QT vs. HS 0.739 before vs. 0.850 after).

Of the 600 women, 504 provided an adequate urine sample and 454 of these replied regarding device preference (see supplement for the questionnaire). Within this group, 46% preferred one of the two vaginal devices, 32% preferred urine, and 22% expressed no preference (Supplementary Table S6, where further details by device are given). In general, the vaginal device used first was preferred to the one used second, with the average preference for the first device being 29% versus 17% for the second sample (P < 0.0001). Preference for the urine sample was similar to that for the first vaginal sample. More women expressed no device preference in the WD/DF group, but Table 3 shows when restricted to women who expressed a preference between vaginal devices, there was a clear preference for DF over WD (71.4%; P = 0.003), but no clear preference between QT and HS (53.4% preferred HS; P = 0.48). Similar vaginal sampling device preferences were seen when the 95 women who did not provide a urine sample and one who had an inadequate sample were included. No correlation was seen between “quite difficult” or “very difficult” ease of use and cellularity ≥ 30 C_t for either QT or HS (P > 0.5 for both). There was no clear preference for device according to age. Neither was time since last urination predictive of preference.

Ease of use was recorded on a five-point scale: very easy, quite easy, neither easy nor difficult, quite difficult, and very difficult. No effect of sample order was seen in this reply and combined results are shown in Table 4A. The HS device emerged as the device least easy to use with 12.6% indicating it was quite difficult or very difficult (P < 0.003 for all comparisons), with no other significant differences.

A major difference in confidence of correct use across devices was seen (Table 4B), with the greatest confidence seen for urine (61.6% as very confident and 4.4% not confident, P < 0.0001 for very confident, all comparisons) and the least confidence for HS (36.4% very confident and 14.2% as not confident, P < 0.03 for not confident for all comparisons except DF). Lower numbers expressing high confidence were also seen for QT than for WD or DF, but these were of marginal significance.

The focus of this study was to compare four different devices for vaginal self-sampling between themselves and also a urine sample, for HPV positivity, user preference, and confidence in collection. We have reported separately results for CIN2+ and CIN3+, because CIN2 is less likely to progress and has greater variability in diagnosis (16, 17) and CIN3 has been shown to have greater reproducibility (18).

Use of the ≤34.2 cutoff, as we previously have done for clinical samples, led to a clinically relevant reduction in sensitivity, but when a higher C_t cutoff was used for HPV16 (≤38.3), as recommended by the manufacturer and used previously (19), sensitivities and were similar to those seen in ours and other previous studies (7, 20–23). However as seen in Table 2, specificity was strongly dependent on the positivity threshold, and large reductions in specificity were seen when a 38.3 threshold was used for all genotypes. Sensitivity can be fairly accurately measured in a referral population, as it focuses on the diseased population. However, specificity is more problematic, as this is highly dependent on the HPV positivity rate, and passenger HPV infections not related to disease and likely to regress naturally are known to be common. While useful information on relative specificity of different tests and devices can be obtained, as was our primary goal here, a study in a screening population is required to assess absolute specificity. We are planning such a study of self versus clinician samples taken at the same visit in a screening population, which has not been previously undertaken on a large scale. In addition to evaluating the sensitivity and specificity in the intended use population, such a study will also examine preference issues in such a population, where the issues could be different than in a referral population, as women are already committed to having screening and other procedures.

High sensitivity is particularly important for self-sampling, especially for women who have declined previous invitations for a clinician taken test, as this may be a rare opportunity to screen them. Thus, technical issues, such as dilution amounts, and less restrictive thresholds for positivity that increase sensitivity, need to be explored. In general, one could imagine three outcomes from a self-sample: (i) HPV negative—return to routine screening, (ii) strongly HPV positive based on higher risk genotypes, methylation profile, viral load and/or other markers identifiable in the self-sample, which would lead to immediate colposcopy, and (iii) weaker positives, based on lower risk genotypes (e.g., 56, 59, 68), lack of methylation or other biomarkers, in which case the next step could be a clinically taken sample where cytology could also be performed, or a repeat self-sample at, say 3–12 months.

When not adjusted for cellularity, the WD swab transported in liquid STM and the dry transported DF gave the highest hrHPV positivity, overall, and for CIN2+ and CIN3+. Women also expressed a preference for the DF swab over the WD device, but a limitation of the study design did not allow for direct comparisons of these devices with QT or HS. Urine samples gave similar sensitivities to WD and DF, and were well received by the women, but specificities were lower. The HS performed least well, apparently largely due to the very large variability in cellularity of the sample between women. For 20 (13%) women, no cells were detected with HS. When adjusted for cellularity, the results were more similar to the other devices. However, women were least confident that they had provided a satisfactory sample with this device. The Evalyn brush has been used in several other studies with good results, but in two previous studies in minority populations (23, 24), we found women were less confident in using it, and so in this study we chose to put more emphasis on simpler devices, which we found the women preferred. Although not included here, we note that the manufacturer of the Evalyn device also supplies a cheaper Viba brush which has the same brush head attached to a simple shaft handle and has reported equivalent performance to clinician-collected samples (25).

At a pragmatic level, the DF swab appeared to be the device of choice—based on high detection rates, more favorable cost, and the important ability to transport the sample dry, which is especially important for home collection where small children may be present and transport via the post, as the STM used for liquid transport contains guanidinium hydrochloride, which is known to be an irritant. We acknowledge a design weakness in that only WD or DF swab versus QT or HS were compared in the same woman. Ideally, we would have randomized all possibilities in which each woman had an equal chance of receiving any two of the four possible vaginal samplers. However, this presented logistic challenges and a danger of errors in a busy clinic and was not implemented, although full randomization of devices between women was achieved.

The urine sample collected using the Colli-Pee device was well received by the women, and gave good results with high sensitivity, but lower specificity. In contrast, a recent study (25), where urine was collected by a different method, lower sensitivity but higher specificity was seen for urine than for self-collected vaginal samples. The liquid UCM preservative medium used here is nontoxic, which makes it suitable for home use, although urine is a body fluid and hence a biohazard.

In summary, there were clear differences in the performance and patient preferences between the five collection methods. Both WD and DF gave good results, and there was very good agreement in HPV positivity. Urine had a similar sensitivity to these two, but its specificity was lower. An important advantage of DF is that it can be transported dry. Sensitivity was very dependent on the positivity threshold and use of a higher C_t cutoff for HPV16 improved results. However, these results need confirmation in a routine screening population, and specificity needs evaluation in that context. The role of triage markers such as genotyping and methylation also require more study.

L. Cadman reports grants from Becton Dickinson, Qiagen, and Genefirst; non-financial support from Novosanis and Eve Medical, during the conduct of the study. L. Ashdown-Barr reports grants from Becton Dickinson, Qiagen, and Genefirst; non-financial support from Novosani, and Eve Medical, during the conduct of the study; J. Cuzick reports personal fees from Hologic, grants from Qiagen, Genera Biosystems, Hologic, GeneFirst and Trovagene; personal fees from Qiagen, Becton Dickinson, Genera Biosystems, and Merck during the conduct of the study. No dislosures were reported by the other authors.

L. Cadman: Resources, supervision, validation, writing–original draft, project administration, writing–review and editing. C. Reuter: Investigation. M. Jitlal: Formal analysis, validation, writing–review and editing. M. Kleeman: Investigation, writing–review and editing. J. Austin: Data curation, writing–review and editing. T. Hollingworth: Supervision, writing–review and editing. A.L. Parberry: Resources. L. Ashdown-Barr: Resources, data curation, writing–review and editing. D. Patel: Resources, data curation, software, writing–review and editing. B. Nedjai: Resources, supervision, validation, investigation, project administration, writing–review and editing. A.T. Lorincz: Conceptualization, resources, supervision, methodology, writing–review and editing. J. Cuzick: Conceptualization, data curation, formal analysis, supervision.

Funding was provided by Cancer Research UK programme grant no. C569/A16891 to J. Cuzick. Assay kits and an additional financial contribution were provided by Becton Dickinson. Colli-Pee urine collection kits and HerSwab kits were donated by Novosanis and Eve Medical, respectively. Commercial companies had no role in study design, data collection, analyses, or writing of this article but were allowed to see the article prior to submission for publication.

We thank the staff of the gynecology outpatients department, The Royal London Hospital, for their invaluable help. We are very grateful to all the women who took part in the study.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.