Abstract
Histopathologic analysis through biopsy has been one of the most useful methods for the assessment of malignant neoplasms. However, some aspects of the analysis such as invasiveness, evaluation range, and turnaround time from biopsy to report could be improved. Here, we report a novel method for visualizing human cervical tissue three-dimensionally, without biopsy, fixation, or staining, and with sufficient quality for histologic diagnosis. Near-infrared excitation and nonlinear optics were employed to visualize unstained human epithelial tissues of the cervix uteri by constructing images with third-harmonic generation (THG) and second-harmonic generation (SHG). THG images enabled evaluation of nuclear morphology in a quantitative manner with six parameters after image analysis using deep learning. It was also possible to quantitatively assess intraepithelial fibrotic changes based on SHG images and another deep learning analysis. Using each analytical procedure alone, normal and cancerous tissue were classified quantitatively with an AUC ≥0.92. Moreover, a combinatory analysis of THG and SHG images with a machine learning algorithm allowed accurate classification of three-dimensional image files of normal tissue, intraepithelial neoplasia, and invasive carcinoma with a weighted kappa coefficient of 0.86. Our method enables real-time noninvasive diagnosis of cervical lesions, thus constituting a potential tool to dramatically change early detection.
This study proposes a novel method for diagnosing cancer using nonlinear optics, which enables visualization of histologic features of living tissues without the need for any biopsy or staining dye.
Introduction
Histopathology has been the most important and definitive method to diagnose malignant neoplasms since Dr. Rudolf Virchow's time. It has not only made prominent contributions to detailed understanding and classification of malignant disease but keeps on providing novel information including even genetic mutations (1). However, the efficiency of histopathologic diagnosis tends to depend on the amount of the tissue collected. A tissue specimen resected by biopsy is too small to reflect the detailed status of the whole lesion. Particularly, it is difficult to provide a diagnosis with sufficient sensitivity and specificity for lesions when it is hard to obtain enough amount of tissue, such as in the lung (2), pancreas (3), and uterus (4, 5). Invasiveness is a contradictory issue in terms of collection volume. Moreover, the turnaround time from biopsy to diagnostic report is another improvable point because multiple biopsies from one lesion are often performed to confirm the final diagnosis instead of spending more time in re-examination. Therefore, it is desirable to develop a novel technique that can provide a diagnosis in real time without sampling the tissue.
Cervical cancer is the fourth most common cancer in women, with approximately 570,000 new cases in 2018, representing 6.9% of all cancers in women (6). The age-specific incidence rate of cervix cancer starts rising after the age of 25 years (6), and it is an important cause of death in the young population, especially in low- and middle-income countries (7). Although histopathology is necessary to confirm the cancer diagnosis, the conventional biopsy method targeting cervical neoplasms shows inadequate sensitivity and specificity (5) for the reasons mentioned above. Furthermore, when patients are suspected to have cervical cancer during pregnancy (8, 9), the invasive procedure of cervical biopsy is not feasible.
To overcome these problems, we elaborated a novel diagnostic system by utilizing the nonlinear optics, including third-harmonic generation (THG) and second-harmonic generation (SHG), to visualize the human body in real time without using any staining dye. In the present study, we indicate the utility of our novel imaging method as well as the quantitative diagnostic approach for the imaging data with machine learning algorithm.
Patients and Methods
Clinical specimens
Samples of cervical neoplastic and normal tissues were collected from 41 patients with cervical neoplasms postoperatively at either Kyushu University Hospital (Fukuoka, Japan) or Osaka University Hospital (Osaka Prefecture, Japan; Table 1). A total of 22 invasive carcinoma tissues (18 squamous cell carcinoma, 3 adenocarcinoma, and 1 small cell carcinoma) and 22 normal tissues were collected from 23 patients at Kyushu University Hospital. Two invasive carcinoma tissues (1 squamous cell carcinoma and 1 adenocarcinoma), 13 tissues of cervical intraepithelial neoplasia (CIN; 3 tissues of CIN2 and 10 tissues of CIN3), and 8 normal tissues were collected from 18 patients at Osaka University Hospital. Hysterectomy, a surgical operation to remove all parts of the uterus, was performed on 23 patients at Kyushu University Hospital and 5 patients at Osaka University Hospital, and samples for imaging were collected from the surface of the tumors or residual normal epithelial tissue surrounding the tumors (cervix or vagina). The collected samples were immediately delivered to the imaging room immersed in PBS with 10% FBS and penicillin/streptomycin. Immediately after imaging, samples were fixed in 10% neutral-buffered formalin (Muto Pure Chemicals) and processed routinely for paraffin embedding. In local excision (13 patients at Osaka University Hospital), imaging was performed without tissue sampling, and specimens were returned to the hospital for pathologic diagnosis immediately after imaging. All patients were diagnosed histologically preoperatively by biopsies. All patients provided written-informed consent, in accordance with the ethics committee requirement of each university and the Declaration of Helsinki. This study was conducted under the supervision of the ethics boards of Kyushu University Graduate School of Medical Sciences or Osaka University Graduate School of Medicine. The Osaka University Graduate School of Medicine Institutional Review Board approved the study protocol on December 17, 2015 (No. 15369). The Kyushu University Graduate School of Medical Sciences Institutional Review Board approved the study protocol on March 17, 2016 (No. 27-383).
Summary of cases participating in this study.
. | . | Number of patients . | . | ||
---|---|---|---|---|---|
. | . | Acquired area by nonlinear optical imaging . | . | ||
. | . | (Normal only)* . | (Normal and lesion) . | (Lesion only) . | Average age (±SEM) . |
Final diagnosis | CIN1 | 1 | 43 | ||
(1) | (0) | (0) | |||
CIN2 | 4 | 50 (±5.0) | |||
(1) | (3) | (0) | |||
CIN3 | 10 | 41 (±4.2) | |||
(0) | (1) | (9) | |||
Carcinoma | 26 | 42 (±2.1) | |||
(2) | (22) | (2) | |||
Total | 41 | 43 (±1.8) | |||
(4) | (26) | (11) |
. | . | Number of patients . | . | ||
---|---|---|---|---|---|
. | . | Acquired area by nonlinear optical imaging . | . | ||
. | . | (Normal only)* . | (Normal and lesion) . | (Lesion only) . | Average age (±SEM) . |
Final diagnosis | CIN1 | 1 | 43 | ||
(1) | (0) | (0) | |||
CIN2 | 4 | 50 (±5.0) | |||
(1) | (3) | (0) | |||
CIN3 | 10 | 41 (±4.2) | |||
(0) | (1) | (9) | |||
Carcinoma | 26 | 42 (±2.1) | |||
(2) | (22) | (2) | |||
Total | 41 | 43 (±1.8) | |||
(4) | (26) | (11) |
Note: Although we attempted the imaging of the lesion in all cases, some cases were used as images of normal tissue because the imaging area did not match the lesion spot determined by histopathologic diagnosis after the imaging (*).
Imaging of human cervical tissues and group separation of data files
The imaging system consisted of an upright microscope (A1RMP+, Nikon) driven by a laser (Chameleon Discovery) tuned to 1,170 nm, and an upright microscope equipped with a 25× water-immersion objective (CFI75 Apo 25 × W MP/NA 1.10, Nikon). The microscope was enclosed in an environmental chamber. This optical equipment had already been installed as shared equipment prior to this study. Tissue samples were positioned with the surface facing upward, and 3% acetic acid was applied onto the epithelial surface of the tissue just before the imaging, without washing afterward. A coverslip and rubber O-ring were put on the tissue to place a drop of water between the sample and objective lens. The imaging position was determined with reference to the preoperative biopsy results. Histopathologic diagnosis, which was completed by experienced pathologists using hematoxylin and eosin (H&E) staining after the imaging procedure, was used as the correct answer label for the algorithm. Multifluorescent images were acquired by direct fluorescence detection using external nondescanned detectors equipped with dichroic and emission filters, including an infrared-cut filter (DM750), dichroic mirrors (DM442 and DM635), and emission filters (400/40 nm for the THG image and 593/40 nm for the SHG image). To acquire a series of image files from one sample, image stacks were collected at 3-μm vertical steps at a depth of more than 120 μm below the sample surface with 1.0× zoom and with 1,024 × 1,024 X–Y resolution (0.50 μm/pixel).
We divided all Z-stack files from the normal tissue and invasive carcinoma into two groups depending on the cases and lesions. Data in Z-stack files of group A (normal tissue from 10 cases and invasive squamous cell carcinoma from 8 cases) were used in the construction of classification models, including training, validation, and feature amount induction. The files in group B [normal tissue from 20 patients and invasive carcinoma from 16 patients (11 with squamous cell carcinoma and 5 with carcinoma of other histological types)] were used in testing classification results. All data from the same lesion in the same case were gathered in the same group. After the grouping, the original images were divided into four Z-stack files (512 × 512 pixels each) before the image analysis described below. For the files of group B, one file from every lesion was used in the analysis of THG/SHG alone, and the other files were used for the combinatory analysis. The original images from CIN were divided into four Z-stack files similarly, and all files were used for the combinatory analysis.
Nuclei segmentation and classification model with THG images
All images were normalized before the analysis so that minimum value was 0 and maximum value was 255. We calibrated the Z-coordinate of each Z-stack files by defining Z = 0 as the first height at which one or more cellular nuclei could be recognized in the THG channel image. The files with less than 41 stack images (from z = 0 to 120 μm) were omitted from analysis. The image data from the files in group A were separated into four data sets: (i) training set for construction of nuclei segmentation model, (ii) validation set for the same, (iii) test set for nuclei segmentation capability, and (iv) data set for construction of the nuclear atypia classification model.
First, the training set (comprised of 220 images) consisted of manually annotated cellular nuclei by researchers other than expert pathologists. Using these images, we constructed our nuclei segmentation model from scratch based on U-Net, a type of deep learning architecture (10, 11). This model consisted of 11 convolution layers and 5 max pooling layers in the encoder section and 11 convolution layers and 5 upsampling layers in the decoder section. Each convolution layer was followed by batch normalization and activation function (leaky ReLU). Adam optimization algorithm was used with a learning rate of 0.0005 with respect to binary cross-entropy loss. In training, data augmentation, such as flip, rotation, and crop, was randomly applied in each epoch.
Second, 76 images were used after training for validation, including the determination of early stopping and threshold. The U-Net code was implemented in Python using TensorFlow and Keras, which are open-source software libraries for machine intelligence.
Third, images in another data set (comprised of 41 images) were examined to evaluate the capability of the nuclei segmentation model. These data consisted of the files obtained from different areas from the files used for training and validation. In addition to the U-Net segmentation, nuclei images of ground truth were created by a pathologist through manual annotation. For both types of images, we performed image postprocessing, such as filling in the blanks (because segmented nuclei are sometimes extracted with the inside vacant), excluding extracted noise by area filtering (to remove extremely small segmented area), and separating adhered multiple nuclei using the watershed algorithm. Thereafter, the two types of images were compared on a pixel basis. Precision was calculated by dividing the numbers of true positive pixels by the sum of true-positive and false-positive pixels. Recall was calculated by dividing the numbers of true-positive pixels by the sum of true-positive and false-negative pixels. We also extracted feature amounts from each segmented nucleus in the images. The area was calculated by multiplying the pixel numbers in the segmented region and the area per pixel (0.25 μm2). Circularity was calculated as 4π (area/perimeter2). The nearest distance was defined as the closest Euclidean distance from the centroid of one label to the centroid of another. Statistical values of these feature amounts, including maximum value, median, and mean absolute deviation (MAD), were calculated from images with three or more segmented nuclei and evaluated using the correlation coefficient.
Finally, segmentation using the constructed model was performed for the final data set from group A files, for the construction of the nuclear atypia classification model. After the image processing and feature amount calculation mentioned above, we constructed nonlinear support vector machine (SVM) as classification algorithm. The radial basis function was used as a kernel function. The soft margin constant C and the kernel function parameter γ were optimized by grid search, and we obtained C = 20 and γ = 0.1. In the preliminary processing, we conducted normalization so that the mean 0 and SD 1 were obtained in all feature amounts. Statistical values of the feature amounts, including maximum value, median, and MAD, were calculated from the images with three or more segmented nuclei ranging from Z = 0 to Z = 120 μm and were used for algorithm construction (2,631 images in total).
In the evaluation of the classification procedure mentioned above, nuclei segmentation was performed in each image (ranging from Z = 0 to Z = 120 μm) in each Z-stack file in group B. After segmentation and image processing described above, classification with nonlinear SVM was conducted in every image with three or more segmented nuclei. The proportion of images classified as “atypical nuclei” among all classified image stacks in one file, termed malignant probability, was calculated in every file analyzed.
Fiber segmentation and classification model with SHG images
Z-coordinate calibration was performed using the same method as that in the analysis of THG images. The segmentation model for fibrous structure in SHG image was established using Pix2Pix, a type of deep learning architecture (12). We performed training with 96 images in the files from group A on the setting of learning late for generator to 0.0005, learning late for discriminator to 0.005 and L1 parameter to 1,000. A total of 32 images from group A were used after training for validation, including the determination of early stopping and threshold. After construction, all SHG images (ranging from Z = 0 to Z = 120 μm) in each Z-stack file from group B were tested for automatic segmentation of fiber structures, and fiber ratio was calculated for each image, defined as segmented pixel numbers per the total pixel numbers in the image area. The sum of fiber ratios of all analyzed image stacks from one file, termed fiber volume, was calculated for each file from group B.
Histology and immunohistochemistry
Paraffin-embedded specimens were cut into 4-μm sections and stained using immunoperoxidase-based procedures. Sections were counterstained with hematoxylin for 1 minute before mounting. The primary antibody for IHC staining was anti-collagen I (ab138492, Abcam). Sections were also stained with H&E using a standard protocol.
Mice
All animal studies were approved by the Osaka University Animal Care and Use Committee. Six- to 8-week-old C57BL/6 female mice were purchased from CLEA Japan, Inc. Multiphoton imaging was performed after hair removal of the trunk and intravenous administration of Hoechst 33342 (DOJINDO) under excitation near-infrared ray tuned to 1,170 nm.
Statistical analysis
For comparisons between groups with non-Gaussian distributions, a Mann–Whitney U test was used to calculate P values. Spearman's correlation coefficient was used for correlative evaluation of statistical values related to the degree of nuclear atypia in the segmentation images. To compare classification accuracies between conventional diagnosis by experienced pathologists with H&E staining and algorithmic classification, the quadratic weighted kappa was calculated as a measure of interobserver agreement (13).
Data availability
All relevant data used in this study are not publicly available and are available from the corresponding author on reasonable request.
Code availability
The code base for the deep learning framework makes use of proprietary components, and we are unable to publicly release the full code base. However, all experiments and implementation details are described in sufficient detail in the Materials and Methods.
Results
Imaging of human fresh cervical tissue with THG and SHG
We performed imaging of the human fresh cervix using the nonlinear optical microscopy system with near-infrared lasers (Fig. 1A). Initially, we focused on THG (14, 15) to identify nuclear morphology. THG imaging of the fresh cervical epithelium was vague without any treatment (Fig. 1B, left). However, the application of 3% acetic acid, widely used in gynecological clinical settings, allowed the visualization of circular structures in epithelial cells (Fig. 1B, middle). These structures well matched the nuclei observed by hematoxylin staining (Fig. 1B, right) and Hoechst 33342 in mouse tissues (Supplementary Fig. S1). These results indicated that nuclei from fresh tissues can be quickly identified by THG imaging with acetic acid. Then, we compared the histologic differences of the normal epithelium and cervical lesions using nonlinear optical imaging. Subsequently, histopathologic diagnosis was made by pathologists. In the normal epithelium, the nuclei were round shaped and uniform in THG images, and nonspecific signals were hardly detectable in SHG images (top row of Fig. 1C, left of Fig. 1D and Supplementary Video S1). In contrast, the nuclei in the carcinoma tissue were irregular in shape with high density, and SHG images showed bright fiber structures in the superficial area (bottom row of Fig. 1C, right of Fig. 1D and Supplementary Video S2). Generally, we could observe apparent differences between normal and carcinoma tissues with our nonlinear optical imaging.
Imaging with nonlinear optics visualized histologic features of human fresh cervical tissue in real time without staining dye. A, Schematic of the nonlinear optical microscopy system. A coverslip and rubber O-ring were placed to keep water between the fresh tissue and objective lens. Excitation near-infrared ray was tuned to 1,170 nm and omitted from femtosecond pulse laser, and fluorescence signals were detected with nondescanned detectors after transmission of dichroic mirrors and emission filters. B, Representative images of THG imaging with or without the application of 3% acetic acid to the human cervical tissue. The H&E-stained image was constructed after the imaging analysis. Bar, 25 μm. C, Representative images of nonlinear optical imaging of human normal or malignant cervical epithelium. Red and green images indicate THG and SHG signals, respectively. Bar, 25 μm. D, Representative images of three-dimensional view constructed from Z-stack files of normal (left) or malignant (right) cervical epithelium.
Imaging with nonlinear optics visualized histologic features of human fresh cervical tissue in real time without staining dye. A, Schematic of the nonlinear optical microscopy system. A coverslip and rubber O-ring were placed to keep water between the fresh tissue and objective lens. Excitation near-infrared ray was tuned to 1,170 nm and omitted from femtosecond pulse laser, and fluorescence signals were detected with nondescanned detectors after transmission of dichroic mirrors and emission filters. B, Representative images of THG imaging with or without the application of 3% acetic acid to the human cervical tissue. The H&E-stained image was constructed after the imaging analysis. Bar, 25 μm. C, Representative images of nonlinear optical imaging of human normal or malignant cervical epithelium. Red and green images indicate THG and SHG signals, respectively. Bar, 25 μm. D, Representative images of three-dimensional view constructed from Z-stack files of normal (left) or malignant (right) cervical epithelium.
Classification for THG images by nuclear atypia
Next, we established the classification algorithm of the two-dimensional image stacks of vertical cervical slices (Z-stack files) obtained from normal or carcinoma tissues. Histopathologic diagnosis of each file was determined by the pathologists after imaging. The algorithm for THG images consisted of two steps: (i) nuclei segmentation with deep learning and (ii) nuclear atypia classification by feature amount extraction of segmented nuclei with machine learning (Fig. 2A and Supplementary Fig. S2). All image files were divided into two groups, A (for algorithm construction) and B (for test), followed by the construction of nuclei segmentation model based on U-Net, a type of deep learning architecture (10, 11), using images from group A. The results from group A files indicated that segmented nuclei of the normal epithelium were significantly small and round shaped with sparse distribution compared with those of carcinoma (Fig. 2B and C), which is consistent with conventional histopathologic findings (16). The comparison between the nuclei images of ground truth, manually annotated by a pathologist, and the output images through our segmentation model indicated precision of 0.85 and recall of 0.65 on a pixel-based evaluation (Supplementary Fig. S3A). Moreover, six statistical values describing the degree of nuclear atypia (shown in Fig. 2C) indicated a correlation coefficient ≥0.70 between these two types of images (Supplementary Fig. S3B and S3C). Therefore, we developed a nuclear atypia classification model based on nonlinear SVM, a type of machine learning model (17), with 6 statistical values (Fig. 2C) to analyze group B files. In group B files, segmented nuclei were smaller and sparser in the normal tissue than those in carcinoma (Supplementary Videos S3 and S4). An analysis using the nuclear atypia classification model was performed for each file from all 36 lesions in group B. The results indicated that 86.5% of normal images were classified as “normal nuclei,” whereas 91.8% of carcinoma images were classified as “atypical nuclei” (Fig. 2D). The proportion of images classified as “atypical nuclei” among all classified image stacks from one file, termed malignant probability, was calculated for all analyzed files, which showed significant difference between the two groups (Fig. 2E). Under a threshold of 0.68 (from the ROC curve in Fig. 2F), all 16 carcinoma files were classified as “malignant,” and 19 of 20 normal files were classified as “non-malignant,” with a sensitivity, specificity, and AUC of 1.0, 0.95, and 0.98, respectively (Fig. 2F). These results indicated that quantitative classification based on nuclear morphology was possible with THG image analyses.
Nuclei segmentation and quantitative classification by nuclear atypia for THG images using machine learning algorithm. A, Simplified flow chart of the classification algorithm for THG images. See Supplementary Fig. S2 for details. B, Representative images of nuclei segmentation results for THG images from group A. Bar, 25 μm. C, Box-and-whisker plots showing statistical values of nuclear feature amount in group A images. The top and bottom of the rectangle indicate the third quartile and the first quartile, respectively. A horizontal line in the rectangle indicates the median. The top and bottom vertical lines indicate the maximum value and the minimum value, respectively. ND, nearest distance. D, Classification results of images from group B files for each Z coordinate. Images classified as “normal nuclei” are indicated in yellow boxes, and images classified as “atypical nuclei” are indicated in red boxes. Images with less than three segmented nuclei were excluded from the classification (50 normal images and 171 carcinoma images; indicated in black boxes). E, Malignant probability of all analyzed files from group B. The probability was calculated as the ratio of red box numbers in D to numbers of yellow and red boxes in each file. The red lines indicate mean ± SEM. F, ROC curve of malignant probability.
Nuclei segmentation and quantitative classification by nuclear atypia for THG images using machine learning algorithm. A, Simplified flow chart of the classification algorithm for THG images. See Supplementary Fig. S2 for details. B, Representative images of nuclei segmentation results for THG images from group A. Bar, 25 μm. C, Box-and-whisker plots showing statistical values of nuclear feature amount in group A images. The top and bottom of the rectangle indicate the third quartile and the first quartile, respectively. A horizontal line in the rectangle indicates the median. The top and bottom vertical lines indicate the maximum value and the minimum value, respectively. ND, nearest distance. D, Classification results of images from group B files for each Z coordinate. Images classified as “normal nuclei” are indicated in yellow boxes, and images classified as “atypical nuclei” are indicated in red boxes. Images with less than three segmented nuclei were excluded from the classification (50 normal images and 171 carcinoma images; indicated in black boxes). E, Malignant probability of all analyzed files from group B. The probability was calculated as the ratio of red box numbers in D to numbers of yellow and red boxes in each file. The red lines indicate mean ± SEM. F, ROC curve of malignant probability.
Classification for SHG images by fiber structures
We developed another classification algorithm with SHG images (Fig. 3A). Previous studies have indicated that collagen type 1 is the strongest candidate producing SHG (18, 19). IHC analysis showed that collagen type 1 was exclusively expressed in the subepithelial interstitial area of the normal tissue (Fig. 3B, left). In contrast, the collagen fibers could be detected among cancer cells in the intraepithelial area in the carcinoma tissue (Fig. 3B, right). These results indicated that quantification of intraepithelial fiber structures was useful in the histologic classification of malignancy. A fiber segmentation model based on Pix2Pix, another type of deep learning architecture (12), was established using the images from group A. A test operation using group B files (the same files analyzed in Fig. 2) extracted fiber structures in the superficial area of the carcinoma, which could not be clearly detected in the normal tissue (Fig. 3C and Supplementary Videos S5 and S6). The fiber ratio, defined as segmented pixel numbers per the total pixel numbers in the image area, showed an obvious increase in carcinoma images (Fig. 3D). The sum of fiber ratios of all analyzed image stacks from one file, termed fiber volume, was calculated for the analyzed files from group B, which showed a significant difference between the two groups (Fig. 3E). Under a threshold of 0.051 (from the ROC curve in Fig. 3F), all 16 carcinoma files were classified as “fibrous,” and 17 of 20 normal files were classified as “non-fibrous,” with a sensitivity, specificity, and AUC of 1.0, 0.85, and 0.92, respectively (Fig. 3F). These results indicated that the detection of intraepithelial fiber structures with SHG images was useful in the quantitative classification of cervical lesions.
Segmentation and quantitative classification of intraepithelial fiber structures for SHG images via deep learning. A, Flow chart of the classification algorithm for SHG images. B, IHC staining of collagen type 1 for tangential sections from the normal epithelium (left) and carcinoma tissue (right). Arrows in the carcinoma image show intraepithelial expression of collagen. Bar, 100 μm. C, Representative images of fiber segmentation results for SHG images from group B files. Bar, 25 μm. D, Calculated fiber ratios of images in group B. Black squares and green dots indicate the mean values of normal and carcinoma images at each Z coordinate, respectively. The black and red lines indicate SEM. E, Fiber volume of all analyzed files from group B. The red lines indicate mean ± SEM. F, ROC curve of fiber volume.
Segmentation and quantitative classification of intraepithelial fiber structures for SHG images via deep learning. A, Flow chart of the classification algorithm for SHG images. B, IHC staining of collagen type 1 for tangential sections from the normal epithelium (left) and carcinoma tissue (right). Arrows in the carcinoma image show intraepithelial expression of collagen. Bar, 100 μm. C, Representative images of fiber segmentation results for SHG images from group B files. Bar, 25 μm. D, Calculated fiber ratios of images in group B. Black squares and green dots indicate the mean values of normal and carcinoma images at each Z coordinate, respectively. The black and red lines indicate SEM. E, Fiber volume of all analyzed files from group B. The red lines indicate mean ± SEM. F, ROC curve of fiber volume.
Diagnosis between the normal tissue, CIN, and invasive carcinoma
We applied these algorithms to CIN, which is the early preinvasive stage of squamous cell carcinoma, the most common histologic type in cervical neoplasms (20). Clinically, general treatment options differ between the disease conditions: local excision for CIN and hysterectomy for invasive carcinoma (21). Therefore, we developed a more useful algorithm that enables differentiation of normal tissue, CIN, and invasive carcinoma by combining the THG and SHG algorithms (Fig. 4A). In addition to all the group B files (except the ones already analyzed in Figs. 2 and 3), Z-stack files from CIN tissues were added in the analysis. Because THG images of CIN indicated irregularly shaped nuclei as well as carcinoma (Fig. 4B and Supplementary Video S7), the files classified as “non-malignant” by nuclear atypia classification were algorithmically diagnosed as normal. Analyses using the nuclear atypia classification model indicated that 70.6% of CIN images were classified as “atypical nuclei” (Fig. 4C). Malignant probability was significantly higher in CIN files than in normal files (Fig. 4D). Under a threshold of 0.323 (from the ROC curve in Fig. 4E), 44 of 52 CIN files were classified as “malignant,” which is consistent with the results from 44 carcinoma files. The sensitivity and specificity of malignant lesions were 0.92 and 0.95, respectively, with AUC of 0.97 (Fig. 4E). Subsequently, analyses using the fiber segmentation model were performed for the files classified as “malignant” (including 3 files from normal tissue, 44 files from CIN, and 44 files from carcinoma). Intraepithelial expression of collagen type 1 was seldom observed in CIN immunohistochemically (Fig. 4F), unlike in carcinoma tissues (Fig. 3B). Thus, we hypothesized that fiber volume quantification was useful in distinguishing CIN and carcinoma. In fact, fewer fiber structures were observed in SHG images from CIN than in those from carcinoma (Figs. 4B and G). Fiber volume was significantly lesser in CIN files than in carcinoma files (Fig. 4H). Under a threshold of 0.076 (from the ROC curve in Fig. 4I), 34 of 44 CIN files were classified as “non-fibrous,” whereas 38 of 44 carcinoma files were classified as “fibrous.” The sensitivity and specificity of the fibrous lesion to differentiate CIN and carcinoma were 0.86 and 0.77, respectively, with an AUC of 0.84 (Fig. 4I). Positive-predictive values of the normal tissue, CIN, and invasive carcinoma using the combinatory algorithm were 0.87, 0.81, and 0.78, respectively (Table 2), which sufficiently distinguishes CIN from carcinoma. The weighted kappa coefficient was 0.86. These results suggested that imaging with nonlinear optics was useful in performing detailed differentiation of CIN and carcinoma.
Combinatory classification algorithm with THG and SHG images enabled detailed diagnosis of the normal epithelium, intraepithelial neoplasia, and invasive carcinoma. A, Flow chart of the classification algorithm. B, Representative images of nonlinear optical imaging of CIN tissue. Bar, 25 μm. C, Classification results of THG images from normal and CIN tissues for each Z coordinate. Images classified as “normal nuclei” are indicated in yellow boxes, and images classified as “atypical nuclei” are indicated in red boxes. Images with less than three segmented nuclei were excluded from the classification (267 CIN images and 142 normal images; indicated in black boxes). D, Box-and-whisker plots showing malignant probability of all analyzed files. E, ROC curve of malignant probability. F, IHC staining of collagen type 1 for tangential section from the CIN tissue. Bar, 100 μm. G, Calculated fiber ratios of images from CIN and carcinoma files classified as malignant by nuclear atypia classification. Cyan triangles and green dots indicate the mean values of CIN and carcinoma images at each Z coordinate, respectively. The black and red lines indicate SEM. H, Box-and-whisker plots showing fiber volume of all analyzed files (3 normal, 44 CIN, and 44 carcinoma files). I, ROC curve of fiber volume.
Combinatory classification algorithm with THG and SHG images enabled detailed diagnosis of the normal epithelium, intraepithelial neoplasia, and invasive carcinoma. A, Flow chart of the classification algorithm. B, Representative images of nonlinear optical imaging of CIN tissue. Bar, 25 μm. C, Classification results of THG images from normal and CIN tissues for each Z coordinate. Images classified as “normal nuclei” are indicated in yellow boxes, and images classified as “atypical nuclei” are indicated in red boxes. Images with less than three segmented nuclei were excluded from the classification (267 CIN images and 142 normal images; indicated in black boxes). D, Box-and-whisker plots showing malignant probability of all analyzed files. E, ROC curve of malignant probability. F, IHC staining of collagen type 1 for tangential section from the CIN tissue. Bar, 100 μm. G, Calculated fiber ratios of images from CIN and carcinoma files classified as malignant by nuclear atypia classification. Cyan triangles and green dots indicate the mean values of CIN and carcinoma images at each Z coordinate, respectively. The black and red lines indicate SEM. H, Box-and-whisker plots showing fiber volume of all analyzed files (3 normal, 44 CIN, and 44 carcinoma files). I, ROC curve of fiber volume.
Diagnostic comparison between the algorithm and conventional histopathologic method by pathologists with H&E staining; each shows the number of analyzed Z-stack files.
. | . | Algorithm . | . | ||
---|---|---|---|---|---|
. | . | Normal . | CIN . | Carcinoma . | Total . |
H&E | Normal | 55 (36%) | 2 (1.2%) | 1 (0.6%) | 58 (38%) |
CIN | 8 (5.2%) | 34 (22%) | 10 (6.5%) | 52 (34%) | |
[CIN2: 7 (4.5%)] | [CIN2: 5 (3.2%)] | [CIN2: 12 (7.8%)] | |||
[CIN3: 1 (0.6%)] | [CIN3: 29 (19%)] | [CIN3: 10 (6.5%)] | [CIN3: 40 (26%)] | ||
Carcinoma | 0 (0%) | 6 (3.9%) | 38 (25%) | 44 (28%) | |
Total | 63 (41%) | 42 (27%) | 49 (32%) | 154 (100%) |
. | . | Algorithm . | . | ||
---|---|---|---|---|---|
. | . | Normal . | CIN . | Carcinoma . | Total . |
H&E | Normal | 55 (36%) | 2 (1.2%) | 1 (0.6%) | 58 (38%) |
CIN | 8 (5.2%) | 34 (22%) | 10 (6.5%) | 52 (34%) | |
[CIN2: 7 (4.5%)] | [CIN2: 5 (3.2%)] | [CIN2: 12 (7.8%)] | |||
[CIN3: 1 (0.6%)] | [CIN3: 29 (19%)] | [CIN3: 10 (6.5%)] | [CIN3: 40 (26%)] | ||
Carcinoma | 0 (0%) | 6 (3.9%) | 38 (25%) | 44 (28%) | |
Total | 63 (41%) | 42 (27%) | 49 (32%) | 154 (100%) |
As shown above, it is possible to classify disease status adequately with nonlinear optical imaging. In addition, our method requires fewer procedures for examination than conventional histopathologic methods such as fixation, dehydration, paraffin embedding, sectioning, and staining. Moreover, it is also possible to obtain digital images from fresh tissue in real time. Therefore, we are free from the image digitization process and can access a quantitative diagnostic system, such as deep learning, fast and easily. These facts indicate that the imaging of fresh tissues with nonlinear optics possesses enough potential to examine the histology in real time and contribute to noninvasive, early, and quantitative diagnosis (Fig. 5).
Schematic flow chart of the clinical utility of the nonlinear optical imaging in comparison with that of conventional histopathologic examination.
Schematic flow chart of the clinical utility of the nonlinear optical imaging in comparison with that of conventional histopathologic examination.
Discussion
In this study, we propose an innovative diagnostic method for cervical carcinoma that is minimally invasive, objective, and time-efficient without the need for biopsy. Generally, various screening methods have been developed for early detection of malignant diseases, including cytology, radiographic examination, and tumor-specific nucleic acid detection. For cervical cancers, molecular tests to detect DNA of human papilloma virus (HPV) have been gaining much attention as a new screening method for neoplasms (22, 23). However, HPV screening tests have higher false-positive rates than conventional cytology because many HPV infections regress without therapy unless they progress to neoplasia (24). These results indicate that histopathologic diagnosis remains indispensable in the next step of the screening procedure. In contrast, the most important limitation of colposcopy combined with biopsy is the suboptimal specificity of 85% (5). Thus, multiple biopsies are often required to confirm diagnosis regardless of invasiveness. Our imaging method is considerably simple because no exogenous dye is required in the imaging and only requires the application of acetic acid, which is commonly used during colposcopy. Moreover, unlike biopsy, it is possible to obtain the histologic images of various regions in real time. Therefore, the technique has sufficient potential to compensate for the limitation of conventional colposcopy in its simplicity, time efficiency, and noninvasiveness; it contributes to the reduction of unnecessary biopsies as well. In additionally, as the required acetic acid concentration of 3% is lower than that in vinegar (about 5% in Japan), it would be possible to apply our imaging method to other areas, including the skin, oral cavity, and upper gastrointestinal tract.
Other approaches to establish alternative colposcopy techniques along with the development of optical technology have been reported (5). Among several new optical devices, multiple studies have reported on the usefulness of confocal microscopic imaging for cervical tissues (25–27). However, there are some decisive differences between our novel method and confocal imaging. First, the observable range is limited to about 50 μm in confocal microscopy (25), whereas it is possible to visualize the deeper area with our imaging method using near-infrared excitation compared with confocal microscopy. Second, the vast majority of confocal microscopic imaging techniques for cervical tissues were performed with exogenous DNA-staining fluorophore molecules, such as acriflavine hydrochloride (25). Although this fluorescent dye has a long history of clinical use and does not penetrate deeply into the stratified squamous epithelium (27), the risk of allergy to the dye must always be considered in preclinical and clinical application. On the contrary, our imaging method does not use any exogenous dye besides acetic acid. Thus, our method may be adapted to many patients without potential problem. Lastly, there are no reports on confocal imaging that provide additional information other than cytological features, such as fibrogenesis. In contrast, our method can provide various information, including intraepithelial fibrosis with SHG signals, which provides valuable suggestion on histologic diagnosis of tumor invasion.
Several recent studies have shown the usefulness of deep learning as an image discrimination tool in clinical medicine (28, 29). It would be probably possible, in our research as well, to perform classification using only deep learning, without segmentation and feature amount calculation. However, one of the most important technical challenges with deep learning analysis is the explainability of analysis results (30) because the imaging characteristics measured by deep learning are highly obscure (29). In contrast, to the best of our knowledge, this is the first report on nonlinear optical imaging of the cervix uteri. Images with THG and SHG are not familiar to almost all clinicians, unlike conventional image inspections, such as H&E staining and computed tomography. Therefore, we considered it might be undesirable, especially for clinicians, to classify unfamiliar tissue images in a way that is difficult to explain the reasons for diagnosis. In contrast, it is possible to classify the disease quantitatively with explainable factors based on the findings of conventional histopathology by our machine learning algorithm using nonlinear SVM. When images with nonlinear optics and deep learning classification will become popular in clinical settings or when the classification by deep learning systems will become sufficiently understandable, another classification method may be more appropriate in the future.
One of the bottlenecks for propagating this methodology seems to be the high initial investment for installing a special laser oscillator in nonlinear optical imaging. Especially, a Ti:Sapphire laser commonly used in our experiments is expensive and relatively unstable and may be unsuitable for medical diagnostic devices at the bedside, although other kinds of laser apparatus with reasonable costs and usability are currently being under developed for clinical application (31). Another important factor in the application of our method in clinical practice is the miniaturization of medical devices targeting the uterine cervix. Importantly, there have been reports on the development of a compact endoscope with near-infrared excitation (32, 33). Therefore, the implementation of noninvasive real-time diagnosis of cervical neoplasms is possible with our novel imaging method.
Therefore, we proposed an innovative diagnostic method of histopathology that is less invasive, objective, and time-efficient without the need for biopsy or staining. Several studies have indicated the importance of detecting malignant neoplasms as early as possible, for not only cervical cancers (23, 34), but also the other tumors. We believe that our method will contribute to effective cancer detection and mortality reduction globally.
Disclosure of Potential Conflicts of Interest
T. Matsui reports grants from Takeda Science Foundation, grants from Osaka Cancer Society, and grants from Japan Society for the Promotion of Science during the conduct of the study. M. Ishii reports grants from Japan Agency for Medical Research and Development during the conduct of the study. No potential conflicts of interest were disclosed by the other authors.
Authors' Contributions
T. Matsui: Conceptualization, data curation, formal analysis, validation, investigation, methodology, writing-original draft, writing-review and editing. R. Tamoto: Software, formal analysis. A. Iwasa: Software, formal analysis. M. Mimura: Software, formal analysis. S. Taniguchi: Validation. T. Hasegawa: Validation, writing-review and editing. T. Sudo: Validation, writing-review and editing. H. Mizuno: Validation. J. Kikuta: Validation. I. Onoyama: Resources. K. Okugawa: Resources. M. Shiomi: Resources. S. Matsuzaki: Resources. E. Morii: Formal analysis. T. Kimura: Conceptualization, resources. K. Kato: Conceptualization, resources. Y. Kiyota: Conceptualization. M. Ishii: Conceptualization, supervision, funding acquisition, project administration, writing-review and editing.
Acknowledgments
We thank Tadasuke Nagatomo, Masaharu Kohara, Masaru Nishino, and Mayumi Kawashima for their kind technical assistance. This work was supported by research grants from the Japan Agency for Medical Research and Development (AMED; JP18he0902001 to M. Ishii), from Takeda Science Foundation (to T. Matsui), and from Osaka Cancer Society (to T. Matsui) and Grant-in-Aid for Early-Career Scientists (JP18K15080 to T. Matsui) from Japan Society for the Promotion of Science.
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
References
Supplementary data
movie file
movie file
movie file
movie file
movie file
movie file
movie file