PurposeTo use deep learning techniques to aid the automation of prostate cancer grading from histologic samples, thereby improving diagnostic accuracy and treatment selection.

IntroductionAmong the most valuable tools in the evaluation of prostate cancer is the Gleason Score (GS) that is assigned by a pathologist to prostatic biopsies. This score represents the aggressiveness of the tumor and ranges from 3 to 5 (least to most aggressive) in a primary and secondary pattern. Here we use deep learning techniques to automate the classification of prostate samples by GS. By automating this process, we are able to reduce the human subjectivity and potential error of Gleason grading and are able to create a pipeline by which we can integrate gene expression to quickly determine cancer progression in individual patients.

Methods733 prostate tissue slide images were downloaded along with clinical information on 500 individuals from the PRAD study contained in The Cancer Genome Atlas (TCGA). We used the Xception network architecture within the Keras software package and TensorFlow backend in Python. We used 120 images that were scored by a board-certified pathologist as training images (with minimal data augmentation) to create a refined deep convolutional neural network to automate Gleason grading to identify cancer regions of interest on test training slides. We then used this model to score new unannotated by a different pathologist blindly to gauge accuracy. Additionally, we have looked at gene expression by combining patients within a specific GS, and compared profiles within each group with gene expression software. From this we were able to select a set of unique markers to identify new patients by GS as well as identify the amount of genetic variance within a given dataset.

Results We compared the model results of the Gleason grading from 10 images with the score that was reported in the TCGA and found that the system was able to correctly identify cancer regions at 93% and severity at 55%. We were able to identify 6,411 differentially expressed genes (DEG) between normal adjacent tissue and GS6, 186 DEGs between GS6 and GS7, 1,855 DEGs between GS7 and GS8, and 603 DEGs between GS8 and GS9.

ConclusionsDeep learning can be applied to determine the histologic severity of tissue images. Given future modifications to improve the accuracy of our current model, we can be hopeful of producing an automated software workflow that will aid in identifying tumor areas, determining their severity, and influencing treatment decisions.

Citation Format: Derek Van Booven, Victor Sandoval, Oleksander Kryvenko, Madhumita Parmar, Andres Briseño, Himanshu Arora. Automated deep-learning system for Gleason grading of prostate cancer using digital pathology and genomic signatures [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 187.