Improving Lung Cancer Diagnosis with CT Radiomics and Serum Histoplasmosis Testing

Approximately 1.5 million indeterminate pulmonary nodules (IPN) are identified each year in the United States (1). While most are benign, they often require costly and invasive testing to rule out malignancy. IPN diagnosis can be especially challenging in regions where both smoking prevalence is high and pulmonary fungal diseases, such as histoplasmosis, induce lung granulomas and mimic lung cancer on imaging (2). We have shown that the specificity of 18F-flouro-deoxyglucose PET (FDG-PET), which is often recommended for intermediate-risk IPNs, is reduced in histoplasmosis endemic geographic regions, further limiting noninvasive diagnostic options (3).

Current management of an IPN depends on its pretest probability of malignancy which can be ascertained through clinical judgment or a validated lung cancer prediction model such as the Mayo Clinic Model (Mayo; ref. 4). Low-risk IPNs can be followed with surveillance imaging, while high-risk nodules are usually referred for surgical biopsy or resection (5, 6). The majority of IPNs fall into the intermediate-risk category which pose the greatest diagnostic challenge and often require costly and invasive procedures to obtain a diagnosis. Current guidelines recommend FDG-PET, biopsy, or surveillance imaging (4, 7). The greatest number of invasive biopsies for benign disease occur within this risk category (8).

Noninvasive biomarkers are needed to improve the evaluation and management of IPNs. Quantitative imaging analysis (radiomics) can serve as a digital biomarker by extracting hundreds of features from CT scans to develop risk models for evaluating pulmonary nodules (9). Radiomic approaches have been shown to enhance lung cancer prediction accuracy (10–14). The enzyme immunoassay (EIA) for Histoplasma IgG and IgM was recently validated for diagnosing benign IPNs in regions endemic for histoplasmosis with a positive predictive value of 100% for patients with both positive IgM and IgG levels (15).

To improve IPN diagnosis, we combined a highly specific fungal biomarker for benign disease, that when positive rules out malignancy. However, as a negative fungal test offers minimal diagnostic information, we combined it with radiomic and clinical risk factors (Mayo) to improve cancer risk estimation. We hypothesized that this combined biomarker approach would improve the diagnostic accuracy of IPN evaluation and meaningfully reclassify intermediate-risk nodules. In addition, we hypothesized that population histoplasmosis and cancer prevalence would impact model performance when fungal and imaging biomarkers were used.

Following a prospective specimen collection, retrospective blinded evaluation study design, serum, and clinical information were identified from patients with newly detected 6–30 mm IPNs (16). The population included in this study was derived from four cohorts and contained patients with incidentally discovered and lung cancer screening detected IPNs. The first cohort consisted of patients from Vanderbilt University Medical Center and the Tennessee Valley VA Healthcare System in Nashville, TN (VUMC, n = 111) consented for research between 2003 and 2017. Approximately 15% of the VUMC cohort consisted of patients from a lung screening population, and 85% were incidentally discovered. The second cohort was derived from the University of Cincinnati in Cincinnati, Ohio (UoC, n = 46) consented for research between 2015 and 2019 from a surgical clinic with incidentally discovered nodules. The third cohort consisted of patients with IPNs from the University of Pittsburgh Medical Center's lung screening program (UPMC, n = 71) consented for research between 2006 and 2015. The fourth cohort consisted of patients from the Detection of Early Cancer Among Military Personnel consortium (DECAMP, n = 99) which includes screening and incidentally discovered nodules from 12 Veterans Affairs sites (Supplementary Fig. S1; Supplementary Table S1). The DECAMP participants were made available as a case–control cohort consented for research between 2013 and 2017 (17). All samples were enrolled under separate Institutional Review Board (IRB) in accordance to Belmont Report or U.S. Common Rule guidelines.

Patients were included in the study if they had an IPN between 6 and 30 mm in largest axial diameter, were 40–90 years old at the time of enrollment, had prospectively collected treatment-naïve serum and CT scans (with slice thickness ≤3 mm), and had a definitive cancer or no-cancer diagnosis. In patients with multiple nodules, the IPN of greatest clinical concern was selected for analysis. Diagnosis was biopsy proven for malignant nodules. Benign nodules were diagnosed through biopsy, at least 2-year longitudinal imaging follow-up from detection time showing no signs of growth (extended past 3 years for subsolid or suspicious nodules), or benign evidence on radiology (e.g., nodule shrinking or benign calcification pattern seen on CT). Exclusion criteria included patients with nonsolid nodules, known metastatic or benign disease at time of enrollment, lung cancer diagnosis within 2 years of enrollment, histology showing non-lung primary malignancy, >90 days between CT and serum collection (>120 days in the VUMC cohort if the medical record confirmed no treatment within this window), and patients with missing radiomics scores, histoplasmosis EIA, or definitive IPN diagnosis. This study was approved by the VUMC IRB #030763 and 000616 and separate IRBs from participating sites. All patients were consented to have biological specimens and clinical data used for future research. Written informed consent was obtained from patients.

Frozen serum was shipped to MiraVista Diagnostics who performed serologic testing. All biological specimen collection, storage, and processing followed existing Early Detection Research Network's protocols (18). Specimens were tested separately for Histoplasma IgG and IgM antibodies by EIA. The methodology for serologic testing has been reported previously (19). MiraVista was blinded to all clinical data.

A previously derived and validated radiomic score from Kammer and colleagues was used in this study (12). Pulmonary nodules were segmented and quantitative features extracted from CT scans using the HealthMyne picture archiving and communication system (Madison, WI). Briefly, morphologic features were extracted, including measurements of nodule size, shape, and location. Textural features, including measurements of heterogeneity and density distribution and wavelet features, including three-dimensional transforms for multi-dimensional spectral content, were extracted for each segmented nodule. These features were collated and used in statistical analysis, such as analyzing variability and building predictive models. A total of 856 quantitative imaging features across the three types were extracted using HealthMyne. The 10 most informative features related to nodule size, shape, and texture were identified and combined into a single lung cancer risk score. This radiomic risk score was obtained using regression shrinkage and subset selection by the Least Absolute Shrinkage and Selection Operator (LASSO) method. The methodology for radiomic score development has been described in more detail elsewhere (12). Investigators performing imaging analysis were blinded to all clinical data other than nodule location and CT-related features.

The Mayo Clinic model is a validated logistic regression lung cancer prediction model, commonly used in clinical practice for predicting risk of IPN malignancy. It includes a combination of patient and nodule characteristics (20). In this study, the Mayo model was calculated for each patient using fixed coefficients for the variables age, smoking history, history of extrathoracic cancer ≥5 years prior, nodule diameter, location, and spiculation.

Patient demographics and nodule characteristics were summarized using descriptive statistics. Analyses were performed using Stata version 16 and R version 3.6.3. P values <0.05 were considered statistically significant.

Multivariable logistic regression models were estimated to predict the presence of lung cancer. The binary outcome for each model was cancer versus benign disease. Predictor variables were continuous and included Mayo score, radiomics score, histoplasmosis IgG EIA, and histoplasmosis IgM EIA levels. Radiomics score and IgM EIA level were assumed to have a linear relationship with the outcome, while Mayo score and IgG EIA level took a flexible functional form via restricted cubic splines to allow for nonlinearity. This selection was based on visual assessment of variable effect plots and of overall model calibration plots. Results presented in this study were derived from models including Mayo score and IgG as restricted cubic splines with three knots.

Only patients with complete data (Mayo score, radiomics risk score, histoplasmosis IgM and IgG EIA levels, and definitive nodule diagnosis) were included in the analysis. Area under the ROC curve (AUC) and calibration plots were used to compare the diagnostic performance of prediction models to assess the combined performance of the radiomic marker and histoplasmosis markers versus their individual improvements upon the Mayo model. This study investigated two prediction modeling approaches outlined in Fig. 1. The initial simplified model (approach 1) demonstrated the need for an interaction term to address the varying effect of histoplasmosis endemicity on predictor variables (Supplementary Table S2; Supplementary Fig. S2). Both the methods and results of this modeling approach can be found with more detail in Supplementary Data S1.

To account for varying fungal prevalence in different populations, and the potential differential effect of radiomics and histoplasmosis antibody levels on predicting cancer, a second modeling approach (approach 2) was investigated. An additional variable was added to the model indicating the endemic presence of histoplasmosis along with interaction terms between all predictor variables except Mayo. These interaction terms were explored in relation to the radiomic risk score, histoplasmosis IgG levels, and histoplasmosis IgM levels. The model was developed and assessed in the combined dataset (n = 327) and allowed for differential effects to be represented by binary interaction terms for the endemic presence of histoplasmosis in the region in which the sample was collected. Model performance was also assessed in the endemic (VUMC and UoC, n = 157) and non-endemic (UPMC and DECAMP, n = 170) subgroups. Mayo and histoplasmosis IgG were included as nonlinear restricted cubic spline terms, based on prior nonlinearity assessments and given that calibration plots demonstrated slightly improved model calibration. Model coefficients were fixed when assessing the apparent AUC in both the combined dataset and in the subgroups. In addition, a bootstrapping approach was used to internally validate the models and provide optimism-corrected assessments of AUC (21, 22).

Reclassification of IPNs out of the intermediate-risk category was calculated using the combined biomarker predictions (Mayo, radiomics, and histoplasmosis EIA) with interaction terms compared with the baseline Mayo prediction. All four cohorts were pooled together, and IPNs separated according to malignant (n = 192) or benign (n = 135) diagnosis. Nodules were classified into low-, intermediate-, or high-risk categories using decision thresholds of 10% and 70% (<10% indicating low risk, 10% to 70% intermediate risk, and >70% high risk) per British Thoracic Society guidelines (23). After grouping nodules by these cutoffs, the bias-corrected cNRI was calculated for cancer and benign disease separately. This method of obtaining a bias-corrected cNRI adjusts for expected reclassification based on random movements alone, and can be used to investigate the clinical utility of a combined biomarker prediction model.

The data generated in this study are not publicly available as no public database with this information exists, but are available upon reasonable request from the corresponding author.

A total of 157 patients with complete data were included in the histoplasmosis-endemic cohort (VUMC and UoC), and 170 patients were included in the non-endemic cohort (UPMC and DECAMP). Population histoplasmosis prevalence was >90% in VUMC, >70% in UoC, and <10% in the non-endemic cohort from UPMC and DECAMP (24). Within our endemic cohort, 22% of patients were histoplasmosis IgG positive (35/157). Of these 35 IgG positive patients, 22 were from VUMC. Likewise, within the endemic cohort, 6% of patients were histoplasmosis IgM positive (10/157). Of these 10 IgM positive patients, 7 were from VUMC. Within our non-endemic cohort, 8% of patients were IgG positive (14/170). Of these 14 IgG positive patients, 8 were from the DECAMP cohort. Furthermore, 4% were IgM positive (7/170). Of these 7 positive patients, 3 were from DECAMP. From the entire cohort, 6 patients were dual IgG and IgM positive. Of these 6 patients 4 were from VUMC, 1 was from UoC, and 1 from DECAMP. Of note, none of these patients who were both IgG and IgM positive had cancer. Cancer prevalence was 75% in the endemic cohort, and 44% in the non-endemic validation set (Table 1). Characteristics of each site can be found in the Supplementary Data (Supplementary Table S1).

The following results are presented in Table 2, Fig. 2, and Supplementary Fig. S3. The AUC for Mayo alone in the histoplasmosis-endemic cohort was 0.71 [95% confidence interval (CI), 0.61–0.80]. In the non-endemic cohort, the AUC for Mayo alone was similar (0.70; 95% CI, 0.62–0.78). When both cohorts were combined, the AUC for Mayo was 0.73 (95%CI, 0.67–0.78).

When histoplasmosis IgG and IgM were added to Mayo including a variable accounting for endemic histoplasmosis (interaction term), the endemic cohort's AUC increased to 0.79 (95% CI, 0.70–0.88; P = 0.05). However, in the non-endemic cohort, the AUC for this model did not significantly improve (0.71; 95% CI, 0.63–0.79; P = 0.46). When the two groups were combined, the AUC was 0.80 (95% CI, 0.75–0.85; P = 0.0006).

When radiomics score was added to Mayo with an interaction term, diagnostic accuracy was not significantly improved in the histoplasmosis-endemic cohort with an AUC of 0.73 (95% CI, 0.64–0.82; P = 0.44). However, in the non-endemic set, diagnostic accuracy of this model did improve with an AUC of 0.81 (95% CI, 0.74–0.87; P = 0.0007). When both groups were combined, the AUC was 0.80 (95% CI, 0.76–0.85; P = 0.0005).

The combined biomarker model included Mayo, radiomics, histoplasmosis IgG and IgM, and interaction terms. This model demonstrated improved diagnostic accuracy over Mayo alone with an AUC of 0.81 (95% CI, 0.73–0.89; P = 0.03) in the endemic group. Likewise, this model exhibited improved performance in the non-endemic group with an AUC of 0.81 (95% CI, 0.75–0.87; P = 0.0007). Diagnostic improvement remained when both groups were combined with an AUC of 0.84 (95% CI, 0.79–0.88; P < 0.0001). This model demonstrated a sensitivity of 90%, specificity of 61%, and positive predictive value of 77% based on Youden index.

Each of the above models was internally validated in the combined population (n = 327) as well as the endemic and non-endemic subgroups. Bootstrap optimism-corrected AUCs are provided in Table 2. Model optimism was minimal across different groupings and models, indicating overestimation of model performance was minimal to none. Calibration plots demonstrated similar findings with improved model calibration when Mayo and histoplasmosis IgG were allowed to have a flexible nonlinear relationship (Supplementary Fig. S4).

Reclassification was calculated using predictions from the combined biomarker model including interaction terms versus the Mayo model. We observed improved reclassification among malignant nodules with the combined biomarker model (cNRI = 0.18). There were no nodules incorrectly reclassified as low risk (false negatives) when the combined model was used. There were 19 benign nodules incorrectly reclassified as high risk (false positives), and we observed no significant difference in the ability of the combined model to reclassify benign nodules compared with the Mayo model (cNRI = 0.009; Fig. 3).

IPNs constitute a diagnostic and financial burden in healthcare. This is especially true in regions endemic for histoplasmosis. Here, the rate of positive lung screening results can be triple the rate in non-endemic regions (25). Although most are benign, many IPNs require expensive and invasive procedures to obtain a diagnosis. This study demonstrated an improvement in the diagnostic accuracy of IPNs when a combined biomarker model including histoplasmosis antibodies and radiomics was used compared with the current standard of risk estimation (Mayo). Furthermore, the combined biomarker model exhibited a strong ability to reclassify malignant nodules from intermediate risk to high risk, illustrating its potential as a clinically useful rule-in test for cancer.

This study also demonstrated the importance of understanding populations when building clinical risk prediction models. Four sites were used to train and validate lung cancer prediction models in this study. VUMC and UoC had high cancer prevalence and were located in histoplasmosis-endemic regions, while UPMC and DECAMP were not. In addition, these sites included both screening and nodule clinic populations. We observed that histoplasmosis and cancer prevalence in a population impact model performance when these biomarkers are used. When the combined biomarker model was created in the VUMC cohort (initial simplified prediction model), it demonstrated variable results in the validation cohorts due to population differences. This showed a single risk model could not perform in different types of populations, perhaps due to varying degrees of cancer and fungal prevalence.

A model including the effect of histoplasmosis endemicity was needed to account for these population differences. When the combined biomarker model including this endemic interaction term was applied to cohorts grouped by fungal prevalence, it highlighted the differential effects seen from radiomics and histoplasmosis. When radiomics was added, there was a significant increase in the AUC within the non-endemic cohort. When histoplasmosis was added, the AUC increased significantly within the endemic cohort. Internal validation supported the robustness of these results. Similar trends were noted in our first simplified modeling approach when VUMC served as a training cohort. While the Mayo model includes certain variables related to nodule features such as diameter, location, and presence of spiculation, the radiomics score includes far more complex image-derived morphologic as well as textural features. When these additional quantitative imaging features were used, there was an improvement in the diagnostic accuracy for IPNs among the combined cohort. However, optimization and validation of the radiomic model is needed for endemic regions.

The combined biomarker model exhibited a strong ability to reclassify malignant nodules out of the intermediate-risk category. Notably, 19 of 99 benign Mayo-calculated intermediate-risk nodules were incorrectly reclassified as high risk, representing this model's limitation in reclassifying benign disease. Because 17 benign nodules were correctly reclassified as low risk, an almost equal number of patients were shifted into these respective categories. Thus, mathematically the cNRI is close to zero (0.009) meaning the model showed no significant difference in the ability to reclassify benign nodules. This is likely attributable to the high cancer prevalence seen in the histoplasmosis endemic populations comprising this study. Even so, these results suggest the potential for a combined biomarker approach to serve as a clinically meaningful rule-in test for cancer.

There are several strengths in this study including the addition of histoplasmosis antibodies to a combined biomarker lung cancer prediction model. While studies have demonstrated an improvement in cancer risk prediction using combinations of biomarkers and clinical risk factors, the addition of a fungal biomarker is a novel approach (12, 26, 27). The histoplasmosis serology testing with EIA used in this study was developed by a company with extensive peer reviewed publication on its use and is a strength of this work (2, 19, 28). MiraVista Diagnostics is a Clinical Laboratory Improvement Amendments- and College of American Pathologists-certified laboratory which conducts diagnostic testing for fungal disease exclusively. This laboratory is accessible to multiple institutions, allows for a variety of sample media (serum, plasma, urine, sputum, and bronchoalveolar lavage) and various settings including clinics, hospitals, universities, and larger laboratories. The test is also highly robust in signal detection in frozen samples irrespective of age of storage, assuming few freeze-thaw cycles. By using this diagnostic company, we attempted to ensure quality control of histoplasmosis EIA testing. Additional strengths in this study include the blinding of specimens and imaging which minimized the chance a diagnosis affected the interpretation of results. Our study population consisted of adults with IPNs, which is the setting for which this combined biomarker model is intended. Both lung screening and surgical clinic (incidentally detected nodule) populations from various regions were included in the study which increases generalizability. With at least 85% of lung cancer associated with smoking behaviors and 100% of the screening population having a significant smoking history, the vast majority of our overall population had some smoking history which would be a potential benefit if used in the screening setting (29–31). Finally, our prediction models accounted for nonlinearity among variables and included a specific variable addressing population differences in histoplasmosis prevalence. Of note, we did not include an interaction term with the Mayo variable as the Mayo score was developed for the general population and thus no differential effect was expected.

This study also has several limitations. Our initial aim was to train and validate a combined biomarker model in various populations (approach 1, initial simplified model). While external validation was performed after the model was trained in VUMC, the sample size of the endemic validation cohort from UoC was small with little benign disease. To develop a more optimized prediction model, we next included the effect of histoplasmosis endemicity (approach 2, full prediction model with interaction term). Including the endemic interactions allowed for differential effects of the newly considered biomarkers in these populations. We did not include the interaction in the well established and validated Mayo model. As such, some of the improvement in discriminative ability of the new models as compared with Mayo may be due to the effect of histoplasmosis endemicity on the predicted cancer risk. The results of Mayo with the endemic main effect term can be found in the supplement (Supplementary Table S3). While the model including endemic interaction terms was internally validated, the bootstrap did not account for all modeling steps (namely, the selection of linear vs. nonlinear terms) which could result in an underestimation of the model optimism. Furthermore, this more advanced model requires external validation. The majority of cohorts were derived from tertiary care referral centers which could contribute to population or selection bias. Thus, the prevalence of cancer seen in this study may not represent that of the general population and therefore model performance may not be as generalizable. In addition, while there was a minimal difference in patient age between benign and malignant disease for the non-endemic cohort, there was a larger difference in age between nodule diagnoses among the endemic cohort. As age is a variable within the Mayo model, this difference is a potential confounder in our study. Our study included the Mayo model as this model is a commonly used lung cancer prediction model in clinical practice. The nodule radiomic features are derived from the segmented nodule only and not from additional features outside the nodule, like nodal or splenic calcifications, which is a limitation to the combined model's radiomic feature set (12). Not all variables for newer risk models such as the Brock model were available. Finally, the prediction models investigated in this study focused on histoplasmosis antibodies as fungal biomarkers. Future directions should include other mycotic diseases and their biomarkers as well.

In conclusion, this study demonstrated an improvement in the diagnostic accuracy of IPNs when a combined biomarker model including histoplasmosis antibodies, radiomics score, and clinical risk factors (Mayo) was used and modeling included interaction terms to account for population differences. While further validation is required, and recalibration and/or optimization of radiomic models is needed in regions endemic for histoplasmosis, integrating a combined biomarker approach into the diagnostic algorithm for IPNs could increase diagnostic accuracy and reclassification and therefore improve clinical management.

H.N. Marmor reports grants from NIH grant T32 CA106183-18 during the conduct of the study. S.A. Deppen reports grants from NCI during the conduct of the study. M.N. Kammer reports other support from Meru Biotechnologies and personal fees from Biodesix outside the submitted work. C.M. Godfrey reports grants from NIH-NCI during the conduct of the study. E.L. Grogan reports grants from NIH during the conduct of the study. No disclosures were reported by the other authors.

H.N. Marmor: Formal analysis, investigation, visualization, writing–original draft, writing–review and editing. S.A. Deppen: Conceptualization, formal analysis, supervision, funding acquisition, investigation, methodology, writing–original draft, writing–review and editing. V. Welty: Formal analysis, investigation, visualization, methodology, writing–original draft, writing–review and editing. M.N. Kammer: Resources, methodology, writing–original draft, writing–review and editing. C.M. Godfrey: Writing–original draft, writing–review and editing. K. Patel: Resources, software, formal analysis, writing–review and editing. F. Maldonado: Resources, writing–original draft, writing–review and editing. H. Chen: Software, formal analysis, methodology, writing–original draft, writing–review and editing. S.L. Starnes: Resources, writing–original draft, writing–review and editing. D.O. Wilson: Resources, writing–original draft, writing–review and editing. E. Billatos: Resources, writing–original draft, writing–review and editing. E.L. Grogan: Conceptualization, resources, supervision, funding acquisition, investigation, methodology, writing–original draft, writing–review and editing.

This work is supported by U01CA152662 (E.L. Grogan and S.A. Deppen) and T32CA106183-18 (H.N. Marmor).

The publication costs of this article were defrayed in part by the payment of publication fees. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734.