Prospective Evaluation of Multimodal Optical Imaging with Automated Image Analysis to Detect Oral Neoplasia In Vivo

The overall 5-year survival rate for oral cancer has remained constant at approximately 64% over the past decade (1). Early detection is associated with a higher 5-year survival rate of 80% (1, 2). Unfortunately, although the oral cavity is readily accessible for clinical examination, only 30% of patients with oral cancer are diagnosed at a localized stage. The 5-year survival rate for patients with advanced disease is only 37% (1). Early diagnosis of oral cancer remains essential for improving treatment outcomes and survival rates.

The current standard of care for oral cancer detection, visual inspection and palpation under white light illumination, is strongly dependent on the experience and expertise of the clinician. Potentially malignant lesions can often appear similar to benign lesions or be missed because of poor visual contrast between the lesion and surrounding normal tissue (2). A definitive diagnosis can be confirmed with a biopsy, but the process is resource intensive, time consuming and requires a trained health care provider. Imaging tools that can rapidly and accurately identify potentially neoplastic oral lesions could improve patient outcomes.

Wide-field autofluorescence imaging (AFI) is a promising method to improve the contrast between normal and neoplastic tissue (3–8). Tissue is illuminated with blue or ultraviolet light, which excites blue–green fluorescence predominantly originating from collagen cross-links in the stroma (9–11). Neoplasia is associated with loss of fluorescence (LOF) likely related to degradation of collagen cross-links (9, 11). Secondary contributors to the LOF include thickening of the epithelial layer, increase of epithelial scattering, lymphocytic infiltration of the lamina propria (subepithelial stroma), and angiogenesis associated with dysplasia (9, 12–14). AFI has been reported to identify neoplastic lesions with high sensitivity; however, it has poor specificity because benign lesions, such as inflammation, can also show LOF (7, 9).

High-resolution imaging modalities, such as confocal microscopy or microendoscopy, have the potential to improve the specificity of AFI (15–23). Topical contrast agents, including acetic acid and proflavine, highlight cell nuclei and enable direct visualization of cellular morphology without removing tissue (20, 24–28). Direct visualization of cell morphology enables real-time assessment of nuclear features, such as nuclear area, nuclear crowding, and nuclear to cytoplasmic area ratio (29–31). One drawback of these high-resolution imaging techniques is their small field of view (FOV), which can potentially miss focal areas of neoplasia.

Multimodal imaging, where wide-field AFI is used to identify suspicious regions of tissue, which are then further interrogated with high-resolution imaging, could enable identification of neoplastic oral lesions with high sensitivity and specificity. Pierce and colleagues performed a 30-patient pilot study to evaluate the accuracy of multimodal imaging using AFI and high-resolution microendoscopy (HRME) to identify oral neoplasia (25). Manual image analysis of the multimodal images correctly classified 98% of nonneoplastic imaging sites and 95% of imaging sites graded as moderate or severe dysplasia or cancer (25). Moreover, multimodal optical imaging with manual image analysis classified as neoplastic 35% of sites graded as mild dysplasia; of these, 80% overexpressed p63. In contrast, only 10% of mild dysplasia sites that were classified as nonneoplastic by multimodal imaging overexpressed p63. Although results of the initial pilot study were promising, the analysis was performed retrospectively in a relatively small dataset. In addition, the image analysis procedure used in the study required several user input steps. Here, we report a prospective evaluation of a fully automated analysis of multimodal optical imaging to identify oral neoplasia. Using data from the 30 patients reported by Pierce and colleagues as a training set, we developed an automated image analysis algorithm. We compare performance of this automated algorithm with the previously reported manual algorithm. An additional 70 patients are then evaluated as a test set.

A total of 100 patients over the age of 18 scheduled for surgical resection of clinically visible oral lesions were recruited for the study and provided written, informed consent prior to participation. This study was approved by the Institutional Review Boards of the University of Texas MD Anderson Cancer Center (Houston, TX) and Rice University (Houston, TX).

Images of the oral mucosa were obtained using two imaging systems: a wide-field AFI system and an HRME. The technical specifications of both systems have been previously described in detail (23, 25). Briefly, the AFI system collects both a 405 nm excited autofluorescence image and a white light reflectance image from a 4.5-cm diameter FOV with a 100 μm lateral resolution to reveal suspicious regions with LOF. The HRME is a fiber optic, fluorescence microscope that collects a 455 nm excited fluorescence image from a 720 μm FOV with a 4.4 μm lateral resolution; topical application of the fluorescence contrast agent proflavine (0.01% in PBS) enables visualization of nuclear morphology to confirm the presence of neoplasia (32).

Proflavine has a long history of safe use as a topical antiseptic and has been used as a contrast agent for confocal endomicroscopy trials (33, 34). In addition, proflavine is a component of triple dye, a topical antiseptic used primarily for umbilical cord care (35). We performed this study with proflavine under IND status with the FDA (IND #108932).

We followed the study protocol described by Pierce and colleagues (25). Briefly, patients were imaged while under general anesthesia immediately before surgery. The oral surgeon first performed a standard white light examination to identify up to four clinically abnormal imaging sites and one clinically normal imaging site. Imaging site locations were selected on the basis of the surgeon's identification of any lesion areas under white light examination; additional sites could be selected on the basis of the signal from the imaging systems. Each imaging site was then described and classified by the surgeon as normal, abnormal low risk, abnormal high risk, or cancer. The surgeon was not blinded to the patient's history. Next, autofluorescence images were acquired from each anatomic area containing an imaging site. After AFI imaging, up to 1 mL of proflavine solution (0.01% w/v) was topically applied with a cotton-tipped applicator to each region of interest (ROI). HRME images were then acquired; the surgeon placed the fiber optic probe in gentle contact at each imaging site. A digital photograph was taken at each location where the fiber probe was placed for later correlation of imaging and biopsy sites. Finally, either a 4-mm punch biopsy was taken at each imaging site location or a surgical specimen was taken and the oral surgeon identified the location of the imaging site on the specimen after resection of the tissue.

Imaging sites corresponding to a 4-mm punch biopsy and imaging sites corresponding to a surgical specimen were evaluated separately. Biopsy and surgical specimens were routinely sectioned and stained with hematoxylin and eosin, then reviewed by the study pathologist who was blinded to the optical imaging results. Sites were graded as normal/hyperkeratosis, mild dysplasia, moderate dysplasia, severe dysplasia, or cancer according to the WHO grading system (36).

A systematic review was also performed for histologic slides of the imaging sites taken from surgical specimens in the test set classified as neoplastic by optics to evaluate their heterogeneity. Each slide was visually evaluated by the study pathologist to measure the overall length of the slide specimen and to map the different histologic diagnoses present in each slide. The fraction of neoplasia observed in each slide was then calculated using the histologic map. Here, fraction of neoplasia is the length of the slide specimen classified as neoplastic by the study pathologist divided by the entire length of the slide specimen.

To explore correlation between optical and molecular markers of progression, IHC staining of molecular markers Ki-67, p63, and PHH3 was performed on sites corresponding to a biopsy diagnosed as normal or dysplastic (any grade) by pathology to aid in delineation of mild dysplasia. These markers were previously selected by Pierce and colleagues because prior studies have shown they are overexpressed in oral neoplasia and associated with the risk of neoplastic progression (25). Ki-67 is widely accepted as a marker of cellular proliferation. Increased Ki-67 expression has been observed in oral tissue diagnosed from normal through cancer. In addition, Ki-67 has shown predictive potential when combined with other markers (37). p63 is part of the p53 tumor suppressor family and has been assessed as a biomarker to predict oral cancer risk in patients with leukoplakia (38). High expression levels were previously suggested to be well associated with positive optical classification using HRME and AFI (25). Finally, phosphohistone-H3 (PHH3) is a core histone protein associated with mitotic figures (39). Evaluation and expression scoring of tissue sections followed the criteria reported by Pierce and colleagues (25). Briefly, IHC staining was performed using standard techniques with the automated BOND MAX immunohistochemistry stainer by Leica Biosystems. After staining, the study pathologist designated a discrete IHC score based on the staining of each marker for each site. Ki-67 staining was reported as staining confined in the basal layer only, lower one third of the epithelium, mid one third of the epithelium, or full thickness of the epithelium. p63 expression was reported as staining confined in the lower one third, two thirds, or full thickness of the epithelium. PHH3 staining was reported by quantifying the number of positively stained nuclei within a single 20× microscope FOV in addition to the epithelial location.

Autofluorescence images were reviewed for quality control using the following criteria: area of interest was in focus and no motion blur was present. For each autofluorescence image passing quality control, the surgeon selected a 31 × 31 pixel ROI that corresponded with the location of an imaging site. This approximately corresponds to the area of a 4-mm punch biopsy. The white light image of the anatomic area and the digital photograph of the fiber probe location were used as a reference to identify the ROI location.

Multiple HRME videos were taken for each imaging site. HRME videos were reviewed from each site to select a single, representative image for each imaging site. A single frame was first selected from every movie acquired using an automated frame selection algorithm described by Ishijima and colleagues (40). Single frames were then reviewed together at each corresponding imaging site. Next, three reviewers blinded to the histologic diagnosis of each biopsy site selected the single image deemed to have the highest image quality by consensus agreement. The reviewers used the following selection criteria: greater than 50% of the imaging FOV was unobscured by debris or keratin, motion blur was not present, and nuclei could be distinguished by visual evaluation. In the case where multiple high-quality images were available, reviewers selected the image that appeared to have the largest nuclei by visual evaluation.

To identify suspicious regions in autofluorescence images, the ratio of red to green autofluorescence intensity was calculated at each pixel in the image. To account for patient to patient variation, this red to green (R:G) intensity ratio map was normalized with respect to the R:G intensity ratio of a normal ROI in the same patient. Selection of the normal ROI was performed by an automated algorithm, which identified the ROI with the lowest R:G intensity ratio in the tissue region visible in the image; a clinician subsequently viewed the corresponding white light image and confirmed that this ROI was indeed clinically normal.

HRME images for each site were analyzed using an automated image analysis procedure (30). Briefly, the algorithm first identifies regions in the HRME image that are free of areas that are too bright or too dim, adjusts the contrast of the image, and converts the image into a binary image to separate the nuclei from the cytoplasm. Morphologic filtering is then applied to the binary image to eliminate regions corresponding to small pieces of debris. After image segmentation, the following metrics were calculated for each HRME image: mean, median, and SD of nuclear area, eccentricity, solidity, and aspect ratio. Additional metrics included ratio of nuclear intensity to cytoplasmic intensity, Weber contrast, Michelson contrast, and nuclear to cytoplasmic area ratio. In addition, each nucleus in the FOV was classified as normal or abnormal using criteria based on nuclear area and eccentricity reported by Quang and colleagues (30). In this case, a nucleus was defined as abnormal if its nuclear area was greater than or equal to 162.9 μm² and nuclear eccentricity was greater than or equal to 0.686. In addition, any nucleus with a nuclear area greater than or equal to 200 μm² was also classified as abnormal regardless of its eccentricity. The fraction of abnormal nuclei and the number of abnormal nuclei per mm² were calculated for each FOV. All analyses were performed in MATLAB (Mathworks).

A two-class linear discriminant algorithm was developed to discriminate between neoplastic (histologically graded as moderate/severe dysplasia or cancer) and nonneoplastic (histologically graded as normal) imaging sites. A training set for the algorithm was generated using data from the first 30 patients previously reported by Pierce (25), and a test set was generated using data from the remaining 70 patients. Imaging sites diagnosed as mild dysplasia were excluded from the initial classification. To select the optimal HRME image metric, ROC curves were generated for each of the 21 image metrics calculated. Diagnostic performance was evaluated by calculating the area under the ROC curve (AUC). The HRME image metric with the largest AUC for the training set was selected as the optimal image metric. Next, the normalized R:G ratio and the best performing HRME feature were used together to develop a two-class linear discriminant analysis algorithm to classify the site as neoplastic or nonneoplastic. The algorithm was developed using images from the training set and evaluated using the test set. Performance was calculated separately for sites where the histologic diagnosis was obtained from a biopsy and where it was obtained from a surgical specimen. The distinction between biopsies and surgical specimens was made because imaging site correlation was performed at different points in time during the study procedure and because of the difference in size between the biopsies and the surgical specimens.

A total of 100 patients were recruited for the study. Images were obtained successfully from 93 patients; 7 were not imaged because of either instrument failure or excess of blood at the lesion area. Images were acquired from 296 sites. AFI and HRME images were both available and passed QC review at 206 sites. Ninety sites were excluded from the multimodal image analysis for the following reasons: missing image data from either AFI or HRME (44 sites) or poor-quality image from either AFI or HRME (46 sites). Sites excluded due to missing data occurred because the AFI image taken did not capture the area where the biopsy was taken. Sites excluded because of poor image quality most commonly occurred because nuclei could not be visualized in the HRME image because of the presence of surface keratin. Table 1 summarizes the histologic diagnosis for all sites included in the analysis. For the first 30 patients, images from 92 sites were used for the training set. Biopsies were obtained for 86 of these sites, while surgical specimens were obtained from six sites. For the next 70 patients, a 4-mm punch biopsy was taken for 56 of the imaged sites, and a surgical specimen was available for 58 of the imaged sites. Images corresponding to biopsies and to surgical specimens were evaluated as two separate test sets.

Figure 1 shows multimodal images analyzed using the automated procedure from a patient with a clinically suspicious lesion on the right posterior tongue. Figure 1A and B show the white light reflectance and autofluorescence images, respectively. The white squares in Fig. 1B represent ROIs corresponding to biopsy site locations. Biopsies were taken at the lesion from an area displaying LOF and at a clinically normal region selected by the surgeon. Figure 1C shows the normalized R:G ratio overlaid atop the white light image. The highlighted areas represent areas with a normalized R:G ratio above a previously established threshold value. Figure 1D–F and 1G – 1I shows the raw and processed HRME images and corresponding histology of sites 1 and 2, respectively. Site 1 was graded as nonneoplastic, and corresponding images showed a low normalized R:G ratio and number of abnormal nuclei (1.26, 99.8). Site 2 was graded as severe dysplasia, and corresponding images showed a high normalized R:G ratio and number of abnormal nuclei per mm² (2.39, 371.1).

Table 2 summarizes the area under the ROC curve calculated for the five best performing HRME image metrics for imaging sites from the training set. Number of abnormal nuclei per mm² was the best performing HRME metric with an AUC of 0.896.

In the original analysis by Pierce and colleagues, 100 imaging sites were originally acquired from 30 patients used in the training set. For the automated analysis, eight of the 100 imaging sites were excluded from analysis because of poor image quality. Figure 2A shows the original multimodal image analysis reported by Pierce and colleagues using nuclear to cytoplasmic (N/C) area ratio and normalized R:G ratio; the linear threshold shown correctly classified 98% of imaging sites histologically graded as nonneoplastic and 95% of histologically graded neoplastic imaging sites. Figure 2B shows multimodal analysis for the training set generated from the same 30 patients using the automated image analysis reported here. Number of abnormal nuclei per mm² was used instead of N/C ratio. The linear threshold for this analysis correctly classified 95% of nonneoplastic sites and 94% of neoplastic sites. Figure 2C and D shows the multimodal analysis for the test set. Results for the 56 imaging sites that were biopsied are shown in Fig. 2C, and results from the 58 sites for which a surgical specimen was obtained are shown in Fig. 2D. For sites that were biopsied, the same linear threshold used in Fig. 2B correctly classified 100% of nonneoplastic sites and 85% of neoplastic sites. For the 58 sites corresponding to a surgical specimen, performance was comparable for nonneoplastic sites (100% correctly classified), but lower for neoplastic sites (only 61% correctly classified).

We hypothesized that the reduced accuracy for identification of neoplasia in sites from which a surgical specimen was obtained was because of challenges identifying the location from which the image was obtained in the relatively large surgical specimen. Such registration errors would not affect performance for nonneoplastic specimens. To test this hypothesis, we assessed the fraction of the epithelium that contained neoplasia for each surgical specimen. We compared this fraction for neoplastic sites that were correctly classified and for those that were incorrectly classified; results are shown in Fig. 3. The mean fraction of neoplasia in the surgical specimen was significantly lower (P < 0.001) in sites that were misclassified as nonneoplastic than in sites that were correctly classified as neoplastic, suggesting that the decrease in performance is related to challenges in identifying the image site on the surgical specimen.

Of the 296 sites imaged, IHC staining for Ki-67, p63, and PHH3 was performed for sites that were histologically graded as normal or dysplastic (any grade). Of these sites, IHC staining was acquired from 102 sites corresponding to a 4-mm punch biopsy. Complete IHC results were available at 97 sites; of these, 48 were graded as normal, 25 as mild dysplasia, 14 as moderate dysplasia, and 10 as severe dysplasia. Figure 4A shows mean IHC scores for each biomarker versus pathology diagnosis; in general, biomarker expression level increased with pathology grade for all three markers.

Complete IHC results and multimodal imaging data were available for 44 of the 48 biopsy sites graded as normal and 42 of the 49 biopsy sites graded as dysplasia (any grade); of the dysplasia sites, 21 were graded as mild dysplasia, 12 as moderate dysplasia, and 9 as severe dysplasia. Figure 4B shows the fraction of imaged sites with a positive IHC score for p63 and for the full panel of markers tested. For sites graded as mild dysplasia, p63 expression was elevated in 27% of sites classified as nonneoplastic by optical assessment, but was elevated in 60% of sites classified as neoplastic by optical assessment. Figure 4C shows a scatter plot of the image metrics for the 21 imaging sites graded as mild dysplasia stratified by p63 expression.

Early detection of oral neoplasia remains the best way to improve long-term patient outcomes. Improved diagnostic tools are needed to detect early-stage disease quickly and accurately. Multimodal imaging using AFI to identify suspicious areas and HRME to investigate those areas further can be a valuable diagnostic adjunct to identify oral neoplasia. The addition of automated image analysis provides quantitative, real-time feedback. We developed a two-parameter classification algorithm using a training set generated from the first 30 patients enrolled in the study and evaluated its performance in an independent test set generated from the remaining 70 patients. In the training set, the automated image analysis achieved comparable classification accuracy to previously reported manual results while following consistent, objective criteria and minimizing reliance on user input. Furthermore, performance of the classification algorithm showed similar performance in the test set for those sites that were biopsied immediately following imaging.

The apparently lower sensitivity in sites corresponding to surgical specimens (61% sensitivity at sites corresponding to surgical specimens compared with 85% sensitivity at sites corresponding to biopsies) is likely due to the challenges in precisely identifying the small location that was imaged in the large surgical specimen. The FOV of the HRME is 720 μm in diameter, an area that is slightly smaller than a punch biopsy, but more than an order of magnitude smaller than a typical surgical specimen. Within a given specimen, histology can vary from normal to high-grade dysplasia and back within a small spatial region. The multifocal nature of oral dysplasia makes precise coregistration between the image site and the histology result more difficult in large specimens. Figure 3 shows that classification errors are more likely in images that were obtained from surgical specimens in which a smaller fraction of the epithelium contains neoplasia, consistent with this hypothesis.

It can be challenging for clinicians to estimate the risk of malignant progression for lesions with mild dysplasia as it can occur temporarily because of inflammation or trauma or it can be the initial stage of transition to high-grade dysplasia (41). We observed high p63 expression in 60% of sites graded as mild dysplasia and classified as neoplastic by optical assessment. In contrast, we observed high p63 expression in only 27% of sites graded as mild dysplasia and classified as nonneoplastic by optical assessment. These results suggest that there is a relationship between high biomarker expression and optical imaging. However, other candidate markers for progression should also be explored in a larger number of sites with mild dysplasia.

Although the work described in this study shows promise, additional work is needed to eventually use multimodal optical imaging practically in a general clinic setting. Future work would start with integrating the image analysis algorithms with the image acquisition and upgrading instrumentation to improve image quality. Currently, the image analysis algorithms have a computation time of approximately 12 to 16 seconds. Porting to a lower level computer language would further reduce this computation time. Instrumentation upgrades would entail reducing the size and cost of the systems and reducing the image acquisition times to improve image quality. In addition, integration into the clinical workflow could be aided by use of tablet-based interfaces, which has been reported in tandem with the HRME (30). Future work will also entail implementing the automated image analysis in real-time to investigate how the availability of real-time diagnostic information will impact the clinician's decision-making process.

Results of this study provide evidence that multimodal optical imaging could be a valuable diagnostic aid to improve identification of oral neoplasia and to assist in determining optimal sites for biopsy. Multimodal imaging provides objective, quantitative results that may be especially useful in situations where an expert clinician with experience evaluating oral lesions is not available.

R.A. Schwarz and R. Richards-Kortum have ownership interest (including patents) in licensing fees for IP licensed from the University of Texas at Austin by Remicalm LLC. No potential conflicts of interest were disclosed by the other authors.

Conception and design: N. Vigneswaran, A.M. Gillenwater, R. Richards-Kortum

Development of methodology: T. Quang, R.A. Schwarz, N. Vigneswaran, R. Richards-Kortum

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): T. Quang, E.Q. Tran, R.A. Schwarz, M.D. Williams, N. Vigneswaran, A.M. Gillenwater

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): T. Quang, E.Q. Tran, R.A. Schwarz, M.D. Williams, A.M. Gillenwater, R. Richards-Kortum

Writing, review, and/or revision of the manuscript: T. Quang, E.Q. Tran, R.A. Schwarz, M.D. Williams, N. Vigneswaran, A.M. Gillenwater, R. Richards-Kortum

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): E.Q. Tran, R.A. Schwarz

Study supervision: R. Richards-Kortum

The authors thank current and former personnel at Rice University and MD Anderson Cancer Center who assisted with data collection: Sharon Mondrik and Travis J. King (Rice University); Jana M. Howe, Vijayashree Bhattar, Jesus A. Blanco, Hawraa Darwiche, and Meiling Zhong (MD Anderson Cancer Center).

This work was supported by NIH grants R01 CA103830 (to R. Richards-Kortum), R01 CA185207 (to R. Richards-Kortum), and by the Cancer Prevention and Research Institute of Texas (CPRIT) grant RP100932 (to R. Richards-Kortum).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.