Low-dose CT (LDCT) is widely accepted as the preferred method for detecting pulmonary nodules. However, the determination of whether a nodule is benign or malignant involves either repeated scans or invasive procedures that sample the lung tissue. Noninvasive methods to assess these nodules are needed to reduce unnecessary invasive tests. In this study, we have developed a pulmonary nodule classifier (PNC) using RNA from whole blood collected in RNA-stabilizing PAXgene tubes that addresses this need. Samples were prospectively collected from high-risk and incidental subjects with a positive lung CT scan. A total of 821 samples from 5 clinical sites were analyzed. Malignant samples were predominantly stage 1 by pathologic diagnosis and 97% of the benign samples were confirmed by 4 years of follow-up. A panel of diagnostic biomarkers was selected from a subset of the samples assayed on Illumina microarrays that achieved a ROC-AUC of 0.847 on independent validation. The microarray data were then used to design a biomarker panel of 559 gene probes to be validated on the clinically tested NanoString nCounter platform. RNA from 583 patients was used to assess and refine the NanoString PNC (nPNC), which was then validated on 158 independent samples (ROC-AUC = 0.825). The nPNC outperformed three clinical algorithms in discriminating malignant from benign pulmonary nodules ranging from 6–20 mm using just 41 diagnostic biomarkers. Overall, this platform provides an accurate, noninvasive method for the diagnosis of pulmonary nodules in patients with non–small cell lung cancer.

Significance:

These findings describe a minimally invasive and clinically practical pulmonary nodule classifier that has good diagnostic ability at distinguishing benign from malignant pulmonary nodules.

With cigarette smoking as the acknowledged root cause, lung cancer remains the primary source of cancer-related deaths worldwide. This is, in part, due to the lack of adequate early detection protocols, and, in part, because early symptoms are so subtle. The demonstration that lung cancer screening by low-dose CT (LDCT) reduces mortality among high-risk current and former smokers (>55 years, >30 pack years; refs 1–3) led to an overall increase in LDCT screening programs (4). Although LDCT does identify significantly smaller nodules than conventional X-rays, this ability comes with the challenge of distinguishing the small percentage of pulmonary nodules that are malignant from the majority of those detected that are benign (5). The National Lung Screening Trial (NLST) detected lung nodules ≥4 mm in diameter in 40% of the patients screened, with 96.4% being false positives over the 3 rounds of screening (6). To reduce this high false positive rate, the recent Lung-RADS classification (7) and new guidelines from the Fleischner group (8) set the detection of nodules ≥6 mm as the positive threshold. However, positive CT scans remain particularly problematic for that class of indeterminate pulmonary nodules (IPN), which range in size from 6 to 20 mm, for which the best course of clinical action is not well specified (6).

Our earlier studies demonstrated that rapidly purified (within 2 hours) peripheral blood mononuclear cells (PBMC) contain gene expression data that can distinguish benign from malignant lung nodules with high accuracy (9). This work established a new paradigm in nodule diagnosis by showing that even an early-stage cancer in the lung affects gene expression in PBMC that is predictive of malignancy. However, this approach was limited by the need to rapidly purify PBMCs from blood samples to maintain sample consistency and RNA integrity. This made it difficult to collect samples in environments where rapid isolation of PBMC was not possible, including most community clinics and physician offices. In addition, the microarrays, which were so useful for diagnostic development, are technically complicated and prone to variabilities associated with reagent batches and enzymatic processes, making them less amenable to clinical applications. The high quality of RNA required for microarray studies is also potentially problematic for studies with patient-derived samples (10). This retrospective/prospective study sought to determine whether accuracies similar to what we achieved in our PBMC studies (9) could be achieved with RNA from whole blood collected in RNA-stabilizing PAXgene tubes. PAXgene RNA is stabilized at the time of collection, immediately fixing the gene expression patterns. The RNA is stable at 15°C–25°C for 5 days and at −20 to −70°C for 8 years. This allows samples to be collected in any clinical setting where blood is drawn without the need for special equipment for storage or for cell purifications (11–13) and allows samples to be transferred to a central facility for testing, as routinely as with other blood tests. In addition, long-term storage with no loss of RNA integrity makes the system well suited for retrospective analyses. We also asked whether a PAXgene signature developed on Illumina microarrays could be transitioned to the NanoString nCounter platform already FDA-approved for the Prosigna Breast Cancer prognosis assay (14) and more recently used to develop a clinical-grade assay that predicts clinical response to PD-1 checkpoint blockade. This PD-1 assay is currently being evaluated in ongoing pembrolizumab clinical trials (15). Because the NanoString assays do not include any enzymatic reactions or amplification steps, the system avoids potential reagent batch effects and PCR biases while decreasing the opportunities for cross contamination by minimizing sample handling. Although we recognized that a gene expression profile from whole blood would be of a greater complexity and could potentially result in a reduction in important diagnostic signals, there was also the prospect that important additional cell types might contribute to the classifier performance.

We now report that the gene expression in whole blood, collected using PAXgene RNA stabilization tubes, can distinguish benign from malignant lung nodules detected by LDCT with high accuracy on independent validation and also report the successful transition of this pulmonary nodule classifier (PNC) from the microarray developmental platform to the NanoString nCounter platform.

Study design

The process of biomarker selection and validation across all studies is summarized in Fig. 1. A total of 821 samples from patients with malignant and benign pulmonary nodules were analyzed across three platforms: Illumina microarrays, the NanoString Pan Cancer Immune (PCI) panel, and finally a custom NanoString custom panel. Microarray data from 264 patient samples (Table 1; Supplementary Table S1) from 4 clinical sites was used for microarray model development. Estimations of performance were based on an independent validation set of 51 samples. In addition, 220 samples, including 201 of the 264 microarray samples, were analyzed on the NanoString PCI platform to select additional biomarkers to be included in the custom NanoString panel. Samples from a fifth collection site not included in the biomarker selection process were analyzed only on the custom NanoString platform. The final NanoString PNC (nPNC) was developed on the data generated from the custom NanoString panel using 583 training samples [included 215 samples used originally in the microarray training set, and 368 samples (70%) never used for the biomarker selection] and validated using a set of 158 independent samples never involved in probe selection. The characteristics of the samples used at the different steps of the classifier development are shown in Supplementary Table S2.

Figure 1.

Study design. A total of 821 unique samples were analyzed in this study. Illumina HT12v4 microarrays and the NanoString PCI panel were used to select candidate biomarker probes using 283 total samples. A total of 264 samples were used for biomarker selection on microarrays and 201 of the 264 + 19 new samples were used to select the biomarkers from the PCI panel. The 51 samples used for validation were not used in any biomarker selection. A total of 559 of the biomarkers selected from the microarray and the PCI panel analyses were successfully designed for the NanoString custom panel. The custom panel was assayed with 237 of the samples used in probe selection, to ensure that the new platform successfully reproduced the microarray results, and an additional 346 independent samples not previously assayed on any platform (total 583). The 583 training samples were used to create a nPNC. An additional 158 samples that were never involved in NanoString probe selection were used for Nanostring custom platform (346 for training and 141 for validation) for a total of 821 independent samples. MN, malignant; BN, benign.

Figure 1.

Study design. A total of 821 unique samples were analyzed in this study. Illumina HT12v4 microarrays and the NanoString PCI panel were used to select candidate biomarker probes using 283 total samples. A total of 264 samples were used for biomarker selection on microarrays and 201 of the 264 + 19 new samples were used to select the biomarkers from the PCI panel. The 51 samples used for validation were not used in any biomarker selection. A total of 559 of the biomarkers selected from the microarray and the PCI panel analyses were successfully designed for the NanoString custom panel. The custom panel was assayed with 237 of the samples used in probe selection, to ensure that the new platform successfully reproduced the microarray results, and an additional 346 independent samples not previously assayed on any platform (total 583). The 583 training samples were used to create a nPNC. An additional 158 samples that were never involved in NanoString probe selection were used for Nanostring custom platform (346 for training and 141 for validation) for a total of 821 independent samples. MN, malignant; BN, benign.

Close modal
Table 1.

Patient demographics for samples in the microarray study

Illumina training setIllumina validation
CategoryMalignant nodulesBenign nodulesPMalignant nodulesBenign nodulesP
Total N 131 133  33 18  
Gender       
 Female 73 (56%) 68 (51%) 0.454 24 (73%) 11 (61%) 0.529 
 Male 58 (44%) 65 (49%)  9 (27%) 7 (39%)  
Age 67 ± 7 65 ± 7 0.029 72 ± 7 64 ± 7 0.0057 
Race       
 Black 19 (15%) 17 (13%) 0.121 3 (9%) 2 (11%)  
 White 107 (82%) 108 (81%)  30 (91%) 15 (83%) 0.375 
 Other 5 (3%) 8 (6%)  1 (6%)  
Smoking status       
 Current 33 (25%) 47 (35%)  8 (24%) 6 (33%)  
 Former 90 (69%) 81 (61%) 0.165 23 (70%) 10 (56%) 0.466 
 Never 8 (6%) 5 (5%)  2 (6%) 1 (5.5%)  
 Unknown 0 (0%) 0 (0%)  0 (0%) 1 (5.5%)  
Pack years 40 ± 21 38 ± 14 0.725 36 ± 19 41 ± 21 0.962 
Lesion size, mm 22 ± 8 8 ± 4 1 × 10−13 17 ± 4 15 ± 4 0.327 
Cancer stage       
 I 87 (66%)   33 (100%)   
 II 23 (18%)   0 (0%)   
 III 7 (5%)   0 (0%)   
 IV 9 (7%)   0 (0%)   
 Unknown 5 (4%)   0 (0%)   
Illumina training setIllumina validation
CategoryMalignant nodulesBenign nodulesPMalignant nodulesBenign nodulesP
Total N 131 133  33 18  
Gender       
 Female 73 (56%) 68 (51%) 0.454 24 (73%) 11 (61%) 0.529 
 Male 58 (44%) 65 (49%)  9 (27%) 7 (39%)  
Age 67 ± 7 65 ± 7 0.029 72 ± 7 64 ± 7 0.0057 
Race       
 Black 19 (15%) 17 (13%) 0.121 3 (9%) 2 (11%)  
 White 107 (82%) 108 (81%)  30 (91%) 15 (83%) 0.375 
 Other 5 (3%) 8 (6%)  1 (6%)  
Smoking status       
 Current 33 (25%) 47 (35%)  8 (24%) 6 (33%)  
 Former 90 (69%) 81 (61%) 0.165 23 (70%) 10 (56%) 0.466 
 Never 8 (6%) 5 (5%)  2 (6%) 1 (5.5%)  
 Unknown 0 (0%) 0 (0%)  0 (0%) 1 (5.5%)  
Pack years 40 ± 21 38 ± 14 0.725 36 ± 19 41 ± 21 0.962 
Lesion size, mm 22 ± 8 8 ± 4 1 × 10−13 17 ± 4 15 ± 4 0.327 
Cancer stage       
 I 87 (66%)   33 (100%)   
 II 23 (18%)   0 (0%)   
 III 7 (5%)   0 (0%)   
 IV 9 (7%)   0 (0%)   
 Unknown 5 (4%)   0 (0%)   

NOTE: Median ± interquartile range is given for continuous values. P values indicate significance of comparison between malignant and benign nodule groups.

Study population

Samples were prospectively collected from incidental subjects with a positive LDCT from 5 clinical sites including Helen F. Graham Cancer Center (Newark, DE), The Hospital of the University of Pennsylvania (Philadelphia, PA), Roswell Park Comprehensive Cancer Center (Buffalo. NY), Temple University Hospital (Philadelphia, PA), and subjects from New York University Langone Medical Center (New York, NY). The NYU subjects included patients recruited as a part of an EDRN lung screening program at NYU. The study was Institutional Review Board–approved at each participating site and conducted according to the principles expressed in the Declaration of Helsinki. All participants signed an informed consent before being enrolled. The study population was primarily smokers and ex-smokers, >50 years of age with >20 pack-years of smoking history, and no previous cancer in the past 5 years (except for nonmelanoma skin cancer). Nodules were confirmed as malignant or benign by repeated imaging or by pathologic diagnosis through bronchoscopy, biopsy, and/or lung resection. In addition, >97% of benign nodules had 4 or more years of follow-up with the remainder having 2 or more years at the time of analysis. Samples associated with MNs were collected within 3 months of definitive diagnosis or prior to any invasive procedure including curative surgery. A small number of participants were found to be never smokers after they had been assayed. The effect on classifier performance of including these samples was assessed. In cases where multiple nodules were present, the diameter of the largest nodule was reported.

RNA purification, quality assessment, and microarrays

Each collection site was provided with a standard protocol for sample collection and storage as specified by Preanalytix [https://www.preanalytix.com/products/blood/rna for the PAXgene Blood RNA Tube (IVD)]. Samples were either stored on site and then bulk transferred overnight on dry ice, or they were transferred to Wistar by courier on the day of collection and stored at −70°C until processing. Total RNA was isolated using the PAXgene miRNA Kit (Qiagen), to capture miRNAs as well as mRNAs. Samples were quantitated with NanoDrop 1000 Spectrophotometer (Thermo Fisher Scientific) and assayed for RNA integrity on the Agilent 2100 BioAnalyzer. Average RNA yields were 3 μg/2.5 mL of blood and on average, RNA integrity numbers (RIN) are >8. Only samples with RIN >7.5 were used for the microarray studies. A constant amount (100 ng) of total RNA was amplified (aRNA) using the Illumina-approved RNA Amplification Kit (Epicenter) and hybridized to the Human-HT12 v4 human whole-genome bead arrays. Microarrays were processed in sets of 48 to minimize potential batch effects.

NanoString assay conditions

The NanoString hybridization was carried out for a constant 19 (within the recommended 12–25 hours) hours at 65°C. Posthybridization processing in the nCounter Prep Station used the standard settings. The cassette scanning parameter was set at high [555 field of view (FOV)]. Supplementary Fig. S1A shows the total normalized counts detected using 100 ng, 200 ng, and 300 ng of total RNA scanned at the low (280 FOV) and the high (555 FOV) settings. The 555 FOV setting significantly increases the overall signal. All assays were carried out using this setting. The standard sample size was 100 ng, an amount we expected to have in all samples. Supplementary Figure S1B shows the stability of the assay across multiple repeats of the Universal Human RNA control sample. Variations less than 5% were observed for the majority of the gene probes with 50 or more detected counts. Supplementary Figure S1C–S1F show that although most sample RIN numbers were above eight, even samples with RINs ≤3 met all four NanoString quality measures, supporting the platforms utility with degraded RNA samples. We found no significant impact on the overall expression profiles for the degraded RNA.

Statistical analysis

Microarrays.

Microarray raw expression data were exported for analysis using Genome Studio software. The raw data was quantile normalized and log2-scaled. Genes with average expression values ≥2× the background levels were used to develop the PNC using support vector machines and recursive feature elimination (SVM-RFE) and the 10-fold 10-resample cross-validation (see details below). The top ranked probes (Borda count) that most accurately distinguished malignant from benign nodules were selected as candidates for the NanoString custom panel. The SVM training set was also stratified into subset A, which contained smaller nodules and stage I and II cancers and subset B, which contained malignant and benign nodules that were balanced for lesion size (Supplementary Table S1). The additional analyses of sets A and B considered either nodule class alone (malignant or benign) or sample class plus collection site as the factors in a linear regression model for each observed gene expression. This resulted in six different regression models and two additional sets of genes were selected from these analyses for inclusion in the NanoString model based on the following parameters: (i) 59 genes with a minimal P value across the comparisons using P < 1 × 10−4 threshold and (ii) 76 genes with a maximum regression coefficient b > log2(1.2) at P < 0.01. Housekeeping (HK) genes were selected from a candidate pool of well-expressed genes (>5× background) with coefficients of variation (CV) for the absolute and log2-scaled expression less than 20% and 2.5%, respectively. Twenty candidate HK genes were ultimately selected: the 12 HK candidate genes with the least CV and 8 candidate genes that overlapped with existing NanoString HK probes.

NanoString.

Background correction was performed on NanoString PanCancer Immune Panel samples by subtracting the geometric mean of the counts of negative controls. The sample counts were normalized by scaling all the values by the ratios of geometric mean of sample controls to the overall geometric mean of control gene counts across all samples. This was done for both spike-in positive controls as well as for HK genes. The NanoString custom panel was quantile normalized and the NanoString Code Set batch differences were corrected using the ratios of expression of samples replicated between code sets, as per NanoString's recommendation. Z-scores were calculated from the final values of custom panel counts and used as inputs for SVM-RFE.

SVM-RFE data analysis.

Supervised classification using a linear kernel SVMs with RFE (16) was used to analyze a z-score–transformed gene expression data set to develop the microarray classifier based on a training set that can distinguish malignant and benign patient classes. A balanced set of cases and controls was used in classifier development as SVM have been shown to require a balanced input for the development of the most accurate classifiers (17). The independent validation, which tests the validity of the classifier developed in the training on a completely new set of samples, is blinded to the identification of samples as either cases or controls. As described previously (9), we employed a 10-fold cross-validation approach with folds resampled 10 times (100 training–testing split models). For each split of the microarray data, the top 1,000 probes ranked by P value (two-tailed t test on 9-folds) were selected and linear kernel SVM was trained on 9-folds and tested on the remaining folds. Each RFE iteration eliminated 10% of the features with the least absolute model weights, in each round as described by Guyon and colleagues (16). A single feature elimination per SVM iteration was used for NanoString data analysis. The final average scores were calculated as follows. The final score for any sample in a training set is calculated as an average among the scores generated for that sample in all testing folds (10 such folds among all 100 splits). The final score for any independent sample in a validation set is calculated as an average among all 100 split models. Each sample is then assigned to a class using the final average scores and a score threshold determined from the training set (0 for unbiased accuracy, or at a fixed threshold corresponding to 90% sensitivity) and sensitivity, specificity and accuracy were calculated. Probes ranking across all 100 splits were combined according to the following procedure based on the Borda count method. In each ranked list n, each gene i was assigned a score: {s_{i,n}} = {\frac{1}{{{r_{i,n}}}}$⁠, where ri,n is the rank of the gene i in list n. A final score FS for each gene i was calculated by taking the sum of the scores of gene i across all 100 lists: F{S_i} = \sum _{n = 1}^{100} {s_{i,n}}$⁠. The resulting final scores for each gene were then used to assign their ranking in the classifier. Final ranking of the probes was produced using all 741 available NanoString samples.

Optimal number of probes for microarray data was determined as the minimum number of probes that maintained an ROC-AUC within 1% of the ROC-AUC achieved by the SVM with top 1,000 gene probes. For the NanoString custom panel, the optimal number of genes was chosen by determining that point where the removal of additional genes/probes resulted in a decline in classification performance. The performance was assessed by determining the ROC-AUC after the removal of each gene using the moving average with a smoothing window size of 5. The probe number at which the ROC-AUC was at maximum was selected as the final optimal classifier.

Data and materials availability

Microarray data have been deposited to NCBI GEO database (https://www.ncbi.nlm.nih.gov/geo) under accession number GSE108375.

Testing for a lung cancer–related gene signature using peripheral blood RNA

The demographic characteristics for the 315 patients used to develop and validate the microarray lung cancer signature are shown in Table 1. The samples used for model building were primarily early stage non–small cell lung cancer with stage I + II cancers comprising 84% of the training set population, and with 100% of the cancers in the independent validation set being stage I.

Gene expression from 264 samples (Table 1, Illumina Training set) was used to select the microarray gene signature that most accurately distinguished malignant from benign lung nodules using SVM-RFE (9, 16). The accuracy of classification was stable across a wide range of probe numbers (Supplementary Fig. S2A) and a panel of the 1,000 highest ranked probes achieved an ROC-AUC of 0.878. As the performance slowly decreased with elimination of lower ranked probes, we selected the smallest number of probes that maintained an ROC-AUC within 1% of the 0.878 achieved by the top 1,000 SVM gene probes. We identified 311 probes that returned an AUC of 0.866 [95% confidence interval (CI): 0.824–0.910] in the training set (sensitivity 77.9%, specificity 74.4%). The performance was well maintained on independent validation (Table 1, Illumina Validation), achieving an AUC of 0.847 (95% CI: 0.742–0.951) (sensitivity 72.7%, specificity 88.9%), similar to the performance of the training set (Supplementary Fig. S2B and S2C). This demonstrated the accuracy of prediction is similar to that of the 29 gene classifier reported from our purified PBMC study (9) and indicates that the presence of cancer in the lung can be also detected in the PAXgene-collected blood RNA with an equal and, in some cases, better performance. Importantly, the performance is maintained as the number of probes is reduced (Supplementary Fig. S2A and S2B), indicating a robust signature is maintained across different numbers of genes.

Transitioning the PNC from microarrays to the NanoString platform

Having verified that mRNA expression from PAXgene samples can distinguish malignant from benign pulmonary nodules, we developed a strategy to transition the microarray-based PNC to the NanoString nCounter platform (14). Because it was difficult to know, a priori, how the microarray expression measurements would replicate on the NanoString platform, we designed the custom panel to contain enough redundancy to mitigate platform differences. We included the top ranked 300 biomarkers from the Illumina gene panel identified by the SVM-RFE analysis. We also included an additional set of 59 markers representing the most significantly differentially expressed probes at P < 1 × 10−4 and 79 probes that exhibited the largest fold change in expression between the malignant and benign groups while maintaining P < 0.01. A set of 20 HK genes with the most consistent expression on the microarrays (see Materials and Methods) was also added.

Because the mRNA samples to be processed on the NanoString platform did not undergo reverse transcription and PCR amplification, we were concerned that some of the microarray probes we selected might be expressed at levels too low for detection without amplification. To establish performance criteria, we analyzed 220 of the mRNA samples with the NanoString PCI panel (catalog no. XT-CSO-HIP1-12; 115 malignant, 105 benign). Although the actual probes were not identical to the Illumina probes, 755 out of 770 genes represented in the PCI panel were also represented on the microarray platform. This study allowed us to correlate the detectable levels of gene expression between the two platforms, providing an estimate of the expression levels that could be robustly detected on both platforms. The results suggested that the probes detected at 2× the background levels on microarrays were robustly detected on the NanoString platform.

The PCI data was also analyzed using SVM-RFE with 10-fold 10-resample cross-validation and although the PCI panel only demonstrated a ROC-AUC = 0.754 compared with the 0.866 achieved with the microarrays (Supplementary Fig. S2D), we selected 106 of the most discriminatory PCI probes for inclusion in our custom panel. An additional 55 probes for genes that were identified as being associated with outcome in our PBMC microarray studies (18, 19) were also added, bringing the number to 619 potential probes for the NanoString custom panel. Supplementary Table S3 summarizes the sources of the final list of the candidate biomarkers selected for the custom NanoString panel. The NanoString probes were then designed to target the same or closely located transcriptome regions as those targeted by the Illumina microarray probes whenever possible. Probes that met the NanoString quality control criteria were successfully designed for 559 of the 619 selected biomarkers (Supplementary File S1).

Developing, refining, and validating the nPNC

We first assessed how well the classification accuracies had been retained between the Illumina and NanoString platforms by reassaying 199 of the samples from the microarray training set. For the comparison of the classification accuracies, we only used the 276 microarray biomarkers that were successfully designed as NanoString probes. We observed a Spearman correlation ρ = 0.73 (P < 1 × 10−12) for the sample classification scores between the two platforms (Supplementary Fig. S3A). The ROC-AUC based on the 276 probes was 0.881 for the microarrays and 0.838 for the NanoString (Supplementary Fig. S3B), indicating that the platform transition was successful.

To carry out an unbiased assessment of the performance of the custom panel, we analyzed a total of 741 patient samples, including samples from a new fifth collection site. The final nPNC training set of 583 samples and the validation of 158 samples had balanced numbers of malignant and benign samples (Table 2) to provide the best conditions for selecting a classifier with good sensitivity as well as specificity (17).

Table 2.

Demographics of samples assayed with NanoString custom panel

NanoString lung nodule classifier training setNanoString lung nodule classifier validation set
CategoryMalignant nodulesBenign nodulesPMalignant nodulesBenign nodulesP
Total N 290 293  74 84  
Gender       
 Female 155 (53%) 145 (49%) 0.2530 45 (61%) 44 (52%) 0.2728 
 Male 135 (47%) 146 (50%)  29 (39%) 38 (45%)  
 Unknown 0 (0%) 2 (1%)  0 (0%) 2 (2%)  
Age 68 ± 6 62 ± 6 3 × 10−10 69 ± 7 65 ± 7 0.0097 
Race       
 Black 34 (12%) 20 (7%) 0.0020 5 (7%) 10 (12%) 0.5134 
 Other 35 (12%) 17 (6%)  8 (11%) 10 (12%)  
 White 221 (76%) 256 (87%)  61 (82%) 64 (76%)  
Smoking status       
 Current 73 (25%) 110 (38%) 0.0124 23 (31%) 22 (26%) 0.8885 
 Former 198 (68%) 170 (58%)  45 (61%) 54 (64%)  
 Never 15 (5%) 11 (4%)  5 (7%) 6 (7%)  
 Unknown 4 (1%) 2 (1%)  1 (1%) 2 (2%)  
Pack years 40 ± 20 38 ± 13 0.6066 42 ± 19 42 ± 15 0.5767 
Lesion size, mm 22 ± 10 8 ± 4 4 × 10−37 18 ± 6 7 ± 4 6 ×10−10 
Cancer stage       
 I 173 (60%)   47 (64%)   
 II 41 (14%)   9 (12%)   
 III 49 (17%)   10 (14%)   
 IV 4 (1%)   1 (1%)   
 Unknown 23 (8%)   7 (9%)   
Sitea       
 HFGCC 146 (50%) 49 (17%) 6 × 10−26 37 (50%) 15 (18%) 2 × 10−6 
 NYU 74 (26%) 198 (68%)  24 (32%) 62 (74%)  
 Roswell-Park 11 (4%) 13 (4%)  7 (9%) 6 (7%)  
 Temple 3 (1%) 11 (4%)  0 (0%) 0 (0%)  
 UPenn 56 (19%) 22 (8%)  6 (8%) 1 (1%)  
NanoString lung nodule classifier training setNanoString lung nodule classifier validation set
CategoryMalignant nodulesBenign nodulesPMalignant nodulesBenign nodulesP
Total N 290 293  74 84  
Gender       
 Female 155 (53%) 145 (49%) 0.2530 45 (61%) 44 (52%) 0.2728 
 Male 135 (47%) 146 (50%)  29 (39%) 38 (45%)  
 Unknown 0 (0%) 2 (1%)  0 (0%) 2 (2%)  
Age 68 ± 6 62 ± 6 3 × 10−10 69 ± 7 65 ± 7 0.0097 
Race       
 Black 34 (12%) 20 (7%) 0.0020 5 (7%) 10 (12%) 0.5134 
 Other 35 (12%) 17 (6%)  8 (11%) 10 (12%)  
 White 221 (76%) 256 (87%)  61 (82%) 64 (76%)  
Smoking status       
 Current 73 (25%) 110 (38%) 0.0124 23 (31%) 22 (26%) 0.8885 
 Former 198 (68%) 170 (58%)  45 (61%) 54 (64%)  
 Never 15 (5%) 11 (4%)  5 (7%) 6 (7%)  
 Unknown 4 (1%) 2 (1%)  1 (1%) 2 (2%)  
Pack years 40 ± 20 38 ± 13 0.6066 42 ± 19 42 ± 15 0.5767 
Lesion size, mm 22 ± 10 8 ± 4 4 × 10−37 18 ± 6 7 ± 4 6 ×10−10 
Cancer stage       
 I 173 (60%)   47 (64%)   
 II 41 (14%)   9 (12%)   
 III 49 (17%)   10 (14%)   
 IV 4 (1%)   1 (1%)   
 Unknown 23 (8%)   7 (9%)   
Sitea       
 HFGCC 146 (50%) 49 (17%) 6 × 10−26 37 (50%) 15 (18%) 2 × 10−6 
 NYU 74 (26%) 198 (68%)  24 (32%) 62 (74%)  
 Roswell-Park 11 (4%) 13 (4%)  7 (9%) 6 (7%)  
 Temple 3 (1%) 11 (4%)  0 (0%) 0 (0%)  
 UPenn 56 (19%) 22 (8%)  6 (8%) 1 (1%)  

NOTE: Median ± interquartile range is given for continuous values. P values indicate significance of comparison between malignant and benign nodule groups.

aHFGCC, Helen F. Graham Cancer Center; NYU, Langone Medical Center; Temple, Temple University Hospital; Roswell, Roswell Park Cancer Center; UPenn, University of Pennsylvania, Perelman School of Medicine.

The classification model using all 559 probes demonstrated an ROC-AUC of 0.833 (95% CI: 0.799–0.864) on training set and ROC-AUC of 0.826 (95% CI: 0.760–0.891) on the independent validation set (Fig. 2A). The training set performance remained stable during the recursive feature elimination process (Supplementary Fig. S4A). Incrementally decreasing sets of probes achieved similar ROC-AUCs (Fig. 2B–D; Supplementary Fig. S4B). Sensitivities, specificities, and positive and negative predictive values (PPV and NPV) are also similar in both training and validation sets (Table 3). The rank of each gene in the classifier based on the training set (Supplementary Fig. S4A) and the combined set (Supplementary Fig. S4C) is shown in Supplementary File S1. Although the 41 probe classifier achieved an AUC of 0.834 (95% CI: 0.800–0.865) for training and 0.825 (95% CI: 0.759–0.890) for the independent validation (Fig. 2C), even using as few as six probes maintained ROC-AUC above 0.8, though there is a slight drop in the validation set performance (Fig. 2D).

Figure 2.

Classification performance of NanoString lung nodule classifier. A–D, Comparison of ROC-AUC in training and validation sets with progressive reduction of the numbers of probes. E, The calculated probability of malignancy for an individual nodule for different classification scores using the 41 probe nPNC.

Figure 2.

Classification performance of NanoString lung nodule classifier. A–D, Comparison of ROC-AUC in training and validation sets with progressive reduction of the numbers of probes. E, The calculated probability of malignancy for an individual nodule for different classification scores using the 41 probe nPNC.

Close modal
Table 3.

Classification performance using different number of probes

N probes
SetPerformance metric559100416
Training Sensitivity 76.5% 73.4% 74.7% 73.7% 
 Specificity 76.6% 73.8% 74.8% 73.4% 
 Accuracy 75.8% 74.6% 74.3% 73.6% 
 ROC-AUC 0.833 0.825 0.834 0.800 
 ROC-AUC 95% CI 0.799–0.864 0.790–0.857 0.800–0.865 0.765–0.836 
 Specificity at 90% sensitivity 53.2% 52.9% 51.9% 45.1% 
 Positive predictive value (PPV)a 0.056 0.049 0.052 0.048 
 Negative predictive value (NPV)a 0.994 0.993 0.994 0.993 
Validation Sensitivity 67.6% 67.6% 68.9% 52.7% 
 Specificity 83.3% 83.3% 82.1% 85.7% 
 Accuracy 75.9% 75.9% 75.9% 70.3% 
 ROC-AUC 0.826 0.817 0.825 0.782 
 ROC-AUC 95% CI 0.760–0.891 0.749–0.885 0.759–0.890 0.709–0.855 
 Specificity at 90% sensitivity 46.4% 36.9% 51.2% 32.1% 
 Positive predictive value (PPV)a 0.069 0.069 0.066 0.063 
 Negative predictive value (NPV)a 0.993 0.993 0.993 0.990 
N probes
SetPerformance metric559100416
Training Sensitivity 76.5% 73.4% 74.7% 73.7% 
 Specificity 76.6% 73.8% 74.8% 73.4% 
 Accuracy 75.8% 74.6% 74.3% 73.6% 
 ROC-AUC 0.833 0.825 0.834 0.800 
 ROC-AUC 95% CI 0.799–0.864 0.790–0.857 0.800–0.865 0.765–0.836 
 Specificity at 90% sensitivity 53.2% 52.9% 51.9% 45.1% 
 Positive predictive value (PPV)a 0.056 0.049 0.052 0.048 
 Negative predictive value (NPV)a 0.994 0.993 0.994 0.993 
Validation Sensitivity 67.6% 67.6% 68.9% 52.7% 
 Specificity 83.3% 83.3% 82.1% 85.7% 
 Accuracy 75.9% 75.9% 75.9% 70.3% 
 ROC-AUC 0.826 0.817 0.825 0.782 
 ROC-AUC 95% CI 0.760–0.891 0.749–0.885 0.759–0.890 0.709–0.855 
 Specificity at 90% sensitivity 46.4% 36.9% 51.2% 32.1% 
 Positive predictive value (PPV)a 0.069 0.069 0.066 0.063 
 Negative predictive value (NPV)a 0.993 0.993 0.993 0.990 

aCalculated using prevalence of 1.8% of lung cancer observed in the National Lung Screening Trial (NLST).

The optimal 41 gene signature had an unbiased sensitivity and specificity of 68.1% and 82.1%, respectively. It achieved a specificity of 51% at a sensitivity of 90% for both training and validation with NPV and PPV values of 0.99 and 0.0066, respectively, for the independent validation. The classifier detected cancers with 64% sensitivity for stage I and a sensitivity of 70% for the late-stage cancers. Probabilities of malignancy across a range of nPNC classification scores are shown in Fig. 2E. It should be noted that a small number of individuals with malignant or benign pulmonary nodules that had no history of smoking were included in the analysis. Removal of these subjects from the study did not change AUC validation performance for the 41 probe classifier (AUC difference of 0.001) or the full 559 probe classifier (AUC difference of 0.010). Adding age, sex, race, and smoking history as additional factors did not have an impact on the classification producing ROC-AUC of 0.837, as compared with 0.840, when only the gene expression was used.

nPNC classifier outperforms existing clinical models

Focusing on the 41 biomarker panel classifier, we compared the performance of the nPNC on all the samples in the difficult to assess 6–20 mm size range with the performance of three clinical algorithms, the Brock University (St. Catharines, Ontario, Canada) developed in a high-risk population (20, 21) and the Mayo Clinic (Rochester, MN; ref. 22) and Veteran's Affairs (VA; ref. 23) models developed using data from a more incidental nodule population. These algorithms assess the cancer risk of a pulmonary nodule based on a variety of demographic and pathologic parameters including nodule size and location (Fig. 3A). The nPNC outperforms all three clinical models on nodules in the 6–20 mm diameter range. Because nodule size is a well-accepted risk factor included in each of the clinical models, we also demonstrate an increased accuracy of the classification as compared with a classification using only size for the samples in the 6–20 mm range (Fig. 3B).

Figure 3.

Performance of the 41 probe nPNC for benign and malignant nodules in the 6–20 mm range. A, Compared with The Brock University, Mayo Clinic, and VA lung cancer risk clinical models. B, Compared with classification by nodule maximum diameter

Figure 3.

Performance of the 41 probe nPNC for benign and malignant nodules in the 6–20 mm range. A, Compared with The Brock University, Mayo Clinic, and VA lung cancer risk clinical models. B, Compared with classification by nodule maximum diameter

Close modal

Classifier performance for different nodules size ranges

Because nodule size is an important risk factor and the definition of IPNs is not very well defined (6) with the changing guidelines, we examined the performance when comparing malignant and benign pulmonary nodules that were similar in size ranges. We calculated the performance of the 41 probe classifier across the various nodule size ranges using baseline positive thresholds of 4 mm from the NLST study (24), 6 mm and 8 mm as discussed in the recent reports from the Fleischner Society (8), and the Lung-RADS (7) as well as a baseline threshold of 10 mm. The ROC-AUCs and the specificities when the sensitivity is held at a performance of 90% were calculated for all possible ranges for the selected thresholds for training and independent validation sets (Fig. 4). Overall, training and validation set performance was highly conserved across all size ranges except where only a few validation set samples fall within a particular size range, as is evident in the smaller validation set. The 41 probe nPNC performs particularly well on independent validation with nodules in the difficult-to-diagnose 8–14 mm range, achieving a 64% specificity at 90% sensitivity, although the specificity drops to 48% in the larger combined dataset. Whether 4 mm or 6 mm is used as the threshold for a positive screen, our classifier demonstrates its utility in classifying IPNs by performing well across all the ranges with a ROC-AUC of 0.83 and 0.81, respectively, in the combined dataset and a threshold of 8 mm only reduces the AUC to 0.80. The specificities at 90% sensitivity are similarly stable and are calculated as 0.50, 0.46, and 0.48 for 4, 6, and 8 mm, respectively.

Figure 4.

Classification performance across different nodule size ranges. Performance values (ROC-AUC, blue; specificity at 90% sensitivity, red) are given for the training (top), validation (middle), and combined data sets (bottom). Each row is labeled on the left side of the figure with the lower nodule size range from minimum (any size) to 10 mm. The column labels across the bottom correspond to the upper nodule size range from 10 mm to maximum (any size). Each square of a panel then shows classification performance in distinguishing benign from malignant nodules that fall in range from lower to upper size in mm along with numbers of nodules being compared for both benign (BN) and malignant (MN) classes. The color intensity is used for visual accent and is proportional to the reported performance values with the color scales shown at the top of the panels. For example, nPNC demonstrated the best ROC-AUC performance of 0.87 and a specificity of 0.64 at 90% sensitivity in distinguishing 8–10 mm nodules (set contained 6 malignant and 14 benign).

Figure 4.

Classification performance across different nodule size ranges. Performance values (ROC-AUC, blue; specificity at 90% sensitivity, red) are given for the training (top), validation (middle), and combined data sets (bottom). Each row is labeled on the left side of the figure with the lower nodule size range from minimum (any size) to 10 mm. The column labels across the bottom correspond to the upper nodule size range from 10 mm to maximum (any size). Each square of a panel then shows classification performance in distinguishing benign from malignant nodules that fall in range from lower to upper size in mm along with numbers of nodules being compared for both benign (BN) and malignant (MN) classes. The color intensity is used for visual accent and is proportional to the reported performance values with the color scales shown at the top of the panels. For example, nPNC demonstrated the best ROC-AUC performance of 0.87 and a specificity of 0.64 at 90% sensitivity in distinguishing 8–10 mm nodules (set contained 6 malignant and 14 benign).

Close modal

The overall benefits of lung cancer screening programs using LDCT are evident in the reported 20% increase in the patient survival. However, this success comes with the associated problem of to how to evaluate the large numbers of primarily benign IPN being detected and the concern for over-management (25). The recent Lung-RADS assessment has also suggested that the implementation of a positive screening threshold of 6–7 mm rather than the 4 mm used in the NLST study may be more appropriate in the management of lung cancer screening results (7) and that this change would reduce the magnitude of the IPN problem with a minimal effect on patient care (8). Even with new guidelines, the potential for over-management of the estimated 1.6 × 106 lung nodules detected each year in the United States remains a significant challenge particularly for nodules ≥6 mm and less than ≤20 mm, where the risk of malignancy can range from approximately 8%–64% (26). The development of alternative noninvasive approaches to assess these IPNs in a clinically meaningful way is an important goal in pulmonary medicine.

Most noninvasive early detection approaches have depended on the identification of tumor-derived nucleic acids, antibodies, or proteins present in blood, plasma, serum, or sputum (27–29), with the caveat that these analytes are frequently rare in the presence of smaller early-stage cancers that are most amenable to curative surgery and which are now more being readily detected by LDCT. Additional studies that have avoided this issue have combined bronchoscopy with gene expression in normal airway epithelial cells or with gene expression associated with nasal brushings. This approach is based on the concept of “field cancerization” whereby the tumor induces gene expression alterations in the uninvolved respiratory tract that differs with the presence of a malignant or benign lung nodule. These approaches work well for nodules likely to be accessed by bronchoscopy (27, 30, 31), but are less effective with smaller IPNs that also represent a major management concern.

We previously showed that a malignant lesion in the lung can extend its influence beyond the pulmonary cancer field to the peripheral blood, as the gene expression in PBMC-derived RNA efficiently distinguishes malignant from benign lung nodules (9). The existence of this extra-pulmonary effect is supported by early reports from mouse models for lung cancer demonstrating that soluble factors produced by premalignant lesions in the lung influenced the expression of specific activation markers in the bone marrow macrophages and that this effect was enhanced with tumor progression (32–34). Although the PBMC studies provided an important proof of principle for extrapulmonary involvement, the need for the rapid purification of the PBMC samples to stabilize the transcriptional profiles was a hindrance to expanding to the collection sites outside of the academic environments and to the development of a robust clinical platform. We have now demonstrated that RNA from whole blood, easily collected in PAXgene RNA stabilization tubes, can also be mined for gene expression information that distinguishes malignant from benign lung nodules. This minimally invasive, 2.5-mL blood collection system allows samples to be collected not only at major medical centers, but wherever blood is routinely drawn. The RNA stability at room temperature for 5 days means that no special storage system is required to maintain sample integrity, thereby facilitating sample collection and subsequent transfer to a central testing facility even from remote locations. The quality of the RNA makes it amenable to analysis on a wide variety of platforms including a variety of sequencing platforms that require high-quality RNA.

We have tested the utility of the PAXgene collection system using samples collected at four academic pulmonary centers and from a community hospital. Samples were collected, stored, and transferred in bulk, or were collected daily and then transferred by courier to our test site for storage and final processing without any detectable effect on platform performance. We built our diagnostic model from global gene expression assayed on Illumina microarrays with cancers that were primarily stage I (69%) and II (17%) and nodules that ranged in size from the 4 mm threshold measurement of the NLST study to 20 mm, spanning the range of malignancy risk from <1% to 64% (8). Importantly, our PAXgene microarray classifier maintained a ROC-AUC of 0.847 (95% CI: 0.742–0.951) on independent validation almost identical to that of the training set used for classifier development. In many studies, validation set accuracy is somewhat diminished, suggesting the model used for classifier development was not large enough to adequately capture the potential subject diversity (9, 35, 36). Moving forward from the microarray developmental platform, we successfully transitioned the nodule classifier to the NanoString nCounter platform. The nCounter platform requires minimal sample handling, is technically simple, and has the ability to evaluate degraded and nondegraded RNA in the same assay. The FDA approval of the NanoString-based Prosigna Breast Cancer Prognostic Assay based on the PAM50 gene signature (14) and the more recent development of a NanoString-based immune signature that predicts the clinical response to PD1 blockade (37) further supports the clinical utility of this platform.

Although the preliminary gene panel for our NanoString-based classifier included 559 biomarkers, that number could be reduced to 41 probes while maintaining the ROC-AUC and, thus, suggesting the potential for simplifying the test platform. In assessing the contributions of the various probes represented in the 41 probes, 46% of the top ranked probes came from the SVM analysis, 29% from PCI panel, and with the fewest candidates being selected by P value. The myeloid-related genes linked to the survival in our PBMC studies (18) were not represented in the 41 probe classifier, but were well represented in the top 100 ranked probes whereas the NK-related probes were mostly in the lower half of the probe set, perhaps because the NK signal is significantly diluted in the PAXgene samples. As patient outcome data is accumulated, we will further assess the utility of the prognostic biomarkers included in our panel that were selected because of an association with recurrence/survival in our previous PBMC studies (18, 19, 38).

Although robust technical performance is important for any clinical platform, the resultant benefit to the patient is primary. The performance of our NanoString custom panel on the 741 samples analyzed on that platform has significant clinical implications with a potential to impact the use of invasive approaches for assessing some classes of difficult-to-diagnose IPN. Our study does not depend on the presence of circulating tumor cells, tumor proteins, or tumor RNA, whose presence is more consistent with more advanced cancers. In this study, we have primarily addressed that class of IPNs that are 6–20 mm in diameter, are of moderate to high risk (39) and frequently not easily accessible by either bronchoscopy or fine needle biopsies and early-stage cancers that are most amenable to surgical approaches. We have also assessed the performance with smaller nodules in the 4–6 mm range where the risk of malignancy is small but whose presence can remain of some concern. Importantly, our nPNC outperformed clinical algorithms presently used to stratify candidates with IPN for treatment or follow-up, including the Brock University (20, 21), Mayo Clinic (22), and VA (23) clinical models in the 6–20 mm range. Although these algorithms work well when applied to datasets that include mostly smaller benign nodules and larger cancers, performance is somewhat diminished when applied only to malignant and benign in the problematic size range. Although the size range of pulmonary nodules we have analyzed is important, there is still a significant difference in the median size between the malignant and the benign in our study. It will be important to address how well biomarkers and clinical algorithms function when benign and malignant nodules are more closely matched in size and where clinical algorithms are likely to perform poorly. We attempted to test this type of comparison as shown in Fig. 4. Although the overall AUCs, sensitivities, and specificities are well conserved whether we use 4, 6, or 8 mm as a positive threshold, as the comparisons get more granular, some comparisons are significantly more accurate than others and this is particularly evident in the validation study where samples number are smaller. We achieved a specificity of 64% at 90% sensitivity for benign and malignant in the 8–14 mm size range dropping to 40% in the 6–14 mm range.

Nodule size is the primary consideration in how IPNs are treated (40). Although this study has interrogated a large number of patient samples and demonstrated a potential utility, a further assessment with a larger number of samples where malignant and benign nodules are more closely related by size will extend that utility as this is the scenario where size is no longer informative. In moving forward, it will be important to more completely address the issue by the comparison of benign and malignant nodules of similar sizes across the range of nodule sizes that remain problematic. The highly simplified and proven method for the acquisition of large numbers of samples of consistent quality from a variety of locations will facilitate this process. Expanded studies will also allow us to address the biological basis for the differences we are detecting between the patient classes and to assess whether those differences may have therapeutic implications.

A.V. Kossenkov has ownership interests (including stock, patients, etc) in patent applications WST155P1 and WST117P. G. Criner has ownership interests (including stock, patients, etc) in HGE Health Care Solutions and is a consultant/advisory board member for AstraZeneca, Boehringer Ingelheim, AVISA, Lungpacer, GlaxoSmithKline, Philips Respironics, NAMDRC, EOLO Medical Inc, Helios Medical, Novartis, Olympus, Spiration, Holaria Inc., Mereo BioPharma, Third Pole, PneumRx, BTG plc, Pearl Therapeutics, Pearl Therapeutics, and Broncus Medical. A. Vachani reports receiving a commercial research grant from Oncocyte, Inc., MagArray, Johnson & Johnson, and Broncus Medical and is a consultant/advisory board member for Oncocyte, Inc. L.C. Showe is a adjunct professor at the University of Pennsylvania and reports receiving a commercial research grant from SRA from OncoCyte and has ownership interests (including stock, patients, etc) as an inventor on pending unpublished provisional patent application(s) assigned to The Wistar Institute that relates to compositions and methods of using a new lung nodule classifier. No potential conflicts of interest were disclosed by the other authors.

Conception and design: A.V. Kossenkov, H. Pass, A. Vachani, B. Nam, W.N. Rom, M.K. Showe, L.C. Showe

Development of methodology: A.V. Kossenkov, R. Qureshi, C. Chang, T. Kumar, L.C. Showe

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): R.S. Majumdar, C. Chang, T. Kumar, E. Kannisto, G. Criner, J.-C.J. Tsay, H. Pass, S. Yendamuri, A. Vachani, T. Bauer, W.N. Rom, L.C. Showe

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): A.V. Kossenkov, R. Qureshi, N.B. Dawany, J. Wickramasinghe, Q. Liu, W.-H. Horng, H. Pass, S. Yendamuri, M.K. Showe, L.C. Showe

Writing, review, and/or revision of the manuscript: A.V. Kossenkov, R. Qureshi, N.B. Dawany, J. Wickramasinghe, Q. Liu, G. Criner, J.-C.J. Tsay, H. Pass, S. Yendamuri, A. Vachani, T. Bauer, W.N. Rom, M.K. Showe, L.C. Showe

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): A.V. Kossenkov, R.S. Majumdar, C. Chang, S. Widura, T. Kumar, H. Pass, L.C. Showe

Study supervision: C. Chang, L.C. Showe

We wish to thank the Wistar Genomics and Bioinformatics facilities for support, Dr. Rachel Locke for editorial assistance, and everyone responsible for recruitment and sample collection. This study was supported by the PA Department of Health grant #4100059200 (Diagnostic Markers for Early-stage Lung Cancer in PAXgene Blood Samples), NCI U01 CA200495-02, and OncoCyte Sponsored Research Agreement to L.C. Showe. NCI U01 CA111295 (to H. Pass), NCI U01 CA086137 (to W.N. Rom), R21 CA156087-02 (to A. Vachani, S. Yendamuri, and L.C. Showe), R50 CA211199-01 (to A.V. Kossenkov), Support for Shared Resources utilized in this study was provided by Cancer Center Support Grant (CCSG) P30 CA010815.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Crowswell
J
,
Baker
S
,
Marcus
P
,
Clapp
J
,
Kramer
B
. 
Incidence of false-positive test results in lung cancer screening
.
Ann Intern Med
2010
;
152
:
505
12
.
2.
Henschke
CI
,
Yankelevitz
DF
,
Altorki
NK
. 
The role of CT screening for lung cancer
.
Thorac Surg Clin
2007
;
17
:
137
42
.
3.
Heuvers
ME
,
Wisnivesky
J
,
Stricker
BH
,
Aerts
JG
. 
Generalizability of results from the National Lung Screening Trial
.
Eur J Epidemiol
2012
;
27
:
669
72
.
4.
Ruparel
M
,
Quaife
SL
,
Navani
N
,
Wardle
J
,
Janes
SM
,
Baldwin
DR
. 
Pulmonary nodules and CT screening: the past, present and future
.
Thorax
2016
;
71
:
367
75
.
5.
Henschke
CI
,
Yankelevitz
DF
. 
CT screening for lung cancer: update 2007
.
Oncologist
2008
;
13
:
65
78
.
6.
Aberle
DR
,
Adams
AM
,
Berg
CD
,
Black
WC
,
Clapp
JD
,
Fagerstrom
RM
, et al
Reduced lung-cancer mortality with low-dose computed tomographic screening
.
N Engl J Med
2011
;
365
:
395
409
.
7.
Martin
MD
,
Kanne
JP
,
Broderick
LS
,
Kazerooni
EA
,
Meyer
CA
. 
Lung-RADS: pushing the limits
.
Radiographics
2017
;
37
:
1975
93
.
8.
Azharuddin
M
,
Adamo
N
,
Malik
A
,
Livornese
DS
. 
Evaluating pulmonary nodules to detect lung cancer: Does Fleischner criteria really work?
J Cancer Res Pract
2018
;
5
:
13
9
.
9.
Showe
MK
,
Vachani
A
,
Kossenkov
AV
,
Yousef
M
,
Nichols
C
,
Nikonova
EV
, et al
Gene expression profiles in peripheral blood mononuclear cells can distinguish patients with non-small cell lung cancer from patients with nonmalignant lung disease
.
Cancer Res
2009
;
69
:
9202
10
.
10.
Draghici
S
,
Khatri
P
,
Eklund
AC
,
Szallasi
Z
. 
Reliability and reproducibility issues in DNA microarray measurements
.
Trends Genet
2006
;
22
:
101
9
.
11.
Debey-Pascher
S
,
Eggle
D
,
Schultze
JL
. 
RNA stabilization of peripheral blood and profiling by bead chip analysis
.
Methods Mol Biol
2009
;
496
:
175
210
.
12.
Fricano
MM
,
Ditewig
AC
,
Jung
PM
,
Liguori
MJ
,
Blomme
EA
,
Yang
Y
. 
Global transcriptomic profiling using small volumes of whole blood: a cost-effective method for translational genomic biomarker identification in small animals
.
Int J Mol Sci
2011
;
12
:
2502
17
.
13.
Kennedy
L
,
Vass
JK
,
Haggart
DR
,
Moore
S
,
Burczynski
ME
,
Crowther
D
, et al
Hematopoietic Lineage Transcriptome Stability and Representation in PAXgene Collected Peripheral Blood Utilising SPIA Single-Stranded cDNA Probes for Microarray
.
Biomark Insights
2008
;
3
:
403
17
.
14.
Wallden
B
,
Storhoff
J
,
Nielsen
T
,
Dowidar
N
,
Schaper
C
,
Ferree
S
, et al
Development and verification of the PAM50-based Prosigna breast cancer gene signature assay
.
BMC Med Genomics
2015
;
8
:
54
.
15.
Ayers
M
,
Lunceford
J
,
Nebozhyn
M
,
Murphy
E
,
Loboda
A
,
Kaufman
DR
, et al
IFN-γ–related mRNA profile predicts clinical response to PD-1 blockade
.
J Clin Invest
2017
;
127
:
2930
40
.
16.
Guyon
I
,
Weston
J
,
Barnhill
S
,
Vapnik
V
. 
Gene selection for cancer classification using support vector machines, machine learning
.
Machine Learning
2002
;
46
:
389
422
.
17.
Wei
Q
,
Dunbrack
RL
 Jr
The role of balanced training and testing data sets for binary classifiers in bioinformatics
.
PLoS One
2013
;
8
:
e67863
.
18.
Kossenkov
AV
,
Dawany
N
,
Evans
TL
,
Kucharczuk
JC
,
Albelda
SM
,
Showe
LC
, et al
Peripheral immune cell gene expression predicts survival of patients with non-small cell lung cancer
.
PLoS One
2012
;
7
:
e34392
.
19.
Kossenkov
AV
,
Vachani
A
,
Chang
C
,
Nichols
C
,
Billouin
S
,
Horng
W
, et al
Resection of non-small cell lung cancers reverses tumor-induced gene expression changes in the peripheral immune system
.
Clin Cancer Res
2011
;
17
:
5867
77
.
20.
Chung
K
,
Mets
OM
,
Gerke
PK
,
Jacobs
C
,
den Harder
AM
,
Scholten
ET
, et al
Brock malignancy risk calculator for pulmonary nodules: validation outside a lung cancer screening population
.
Thorax
2018
;
73
:
857
63
.
21.
McWilliams
A
,
Tammemagi
MC
,
Mayo
JR
,
Roberts
H
,
Liu
G
,
Soghrati
K
, et al
Probability of cancer in pulmonary nodules detected on first screening CT
.
N Engl J Med
2013
;
369
:
910
9
.
22.
Swensen
SJ
,
Silverstein
MD
,
Ilstrup
DM
,
Schleck
CD
,
Edell
ES
. 
The probability of malignancy in solitary pulmonary nodules. Application to small radiologically indeterminate nodules
.
Arch Intern Med
1997
;
157
:
849
55
.
23.
Gould
MK
,
Ananth
L
,
Barnett
PG
,
Veterans Affairs SNAP Cooperative Study Group
. 
A clinical model to estimate the pretest probability of lung cancer in patients with solitary pulmonary nodules
.
Chest
2007
;
131
:
383
8
.
24.
National Lung Screening Trial Research Team
,
Aberle
DR
,
Adams
AM
,
Berg
CD
,
Black
WC
,
Clapp
JD
, et al
Reduced lung-cancer mortality with low-dose computed tomographic screening
.
N Engl J Med
2011
;
365
:
395
409
.
25.
Redberg
RF
,
O'Malley
PG
. 
Important questions about lung cancer screening programs when incidental findings exceed lung cancer nodules by 40 to 1
.
JAMA Intern Med
2017
;
177
:
311
2
.
26.
Massion
PP
,
Walker
RC
. 
Indeterminate pulmonary nodules: risk for having or for developing lung cancer?
Cancer Prev Res
2014
;
7
:
1173
8
.
27.
Kathuria
H
,
Gesthalter
Y
,
Spira
A
,
Brody
JS
,
Steiling
K
. 
Updates and controversies in the rapidly evolving field of lung cancer screening, early detection, and chemoprevention
.
Cancers
2014
;
6
:
1157
79
.
28.
Hoseok
I
,
Cho
JY
. 
Lung cancer biomarkers
.
Adv Clin Chem
2015
;
72
:
107
70
.
29.
Li
CM
,
Chu
WY
,
Wong
DL
,
Tsang
HF
,
Tsui
NB
,
Chan
CM
, et al
Current and future molecular diagnostics in non-small-cell lung cancer
.
Expert Rev Mol Diagn
2015
;
15
:
1061
74
.
30.
AEGIS Study Team
,
Perez-Rogers
JF
,
Gerrein
J
,
Anderlind
C
,
Liu
G
,
Zhang
S
,
Alekseyev
Y
, et al
Shared gene expression alterations in nasal and bronchial epithelium for lung cancer detection
.
J Nat Cancer Inst
2017
;
109
.
doi: 10.1093/jnci/djw327.
31.
Silvestri
GA
,
Vachani
A
,
Whitney
D
,
Elashoff
M
,
Porta Smith
K
,
Ferguson
JS
, et al
A bronchial genomic classifier for the diagnostic evaluation of lung cancer
.
N Engl J Med
2015
;
373
:
243
51
.
32.
Redente
EF
,
Dwyer-Nield
LD
,
Merrick
DT
,
Raina
K
,
Agarwal
R
,
Pao
W
, et al
Tumor progression stage and anatomical site regulate tumor-associated macrophage and bone marrow-derived monocyte polarization
.
Am J Pathol
2010
;
176
:
2972
85
.
33.
Redente
EF
,
Higgins
DM
,
Dwyer-Nield
LD
,
Orme
IM
,
Gonzalez-Juarrero
M
,
Malkinson
AM
. 
Differential polarization of alveolar macrophages and bone marrow-derived monocytes following chemically and pathogen-induced chronic lung inflammation
.
J Leukoc Biol
2010
;
88
:
159
68
.
34.
Redente
EF
,
Orlicky
DJ
,
Bouchard
RJ
,
Malkinson
AM
. 
Tumor signaling to the bone marrow changes the phenotype of monocytes and pulmonary macrophages during urethane-induced primary lung tumorigenesis in A/J mice
.
Am J Pathol
2007
;
170
:
693
708
.
35.
Li
XJ
,
Hayward
C
,
Fong
PY
,
Dominguez
M
,
Hunsucker
SW
,
Lee
LW
, et al
A blood-based proteomic classifier for the molecular characterization of pulmonary nodules
.
Sci Transl Med
2013
;
5
:
207ra142
.
36.
Silvestri
GA
,
Tanner
NT
,
Kearney
P
,
Vachani
A
,
Massion
PP
,
Porter
A
, et al
Assessment of plasma proteomics biomarker's ability to distinguish benign from malignant lung nodules: results of the PANOPTIC (Pulmonary Nodule Plasma Proteomic Classifier) Trial
.
Chest
2018
;
154
:
491
500
.
37.
Ayers
M
,
Lunceford
J
,
Nebozhyn
M
,
Murphy
E
,
Loboda
A
,
Kaufman
DR
, et al
IFN-gamma-related mRNA profile predicts clinical response to PD-1 blockade
.
J Clin Invest
2017
;
127
:
2930
40
.
38.
Showe
MK
,
Kossenkov
AV
,
Showe
LC
. 
The peripheral immune response and lung cancer prognosis
.
Oncoimmunology
2012
;
1
:
1414
6
.
39.
Massion
PP
,
Walker
RC
. 
Indeterminate pulmonary nodules: risk for having or for developing lung cancer?
Cancer Prev Res
2014
;
7
:
1173
8
.
40.
Vachani
A
,
Tanner
NT
,
Aggarwal
J
,
Mathews
C
,
Kearney
P
,
Fang
KC
, et al
Factors that influence physician decision making for indeterminate pulmonary nodules
.
Ann Am Thoracic Soc
2014
;
11
:
1586
91
.