Introduction: Experts at tertiary care centers provide solutions to complex cases not addressed by high quality evidence. They intuitively retrieve patterns from years of experience to make treatment decisions. Short of personal consultations, there is no way to access this vast “experience database.” Experience Engine (XE) is a machine learning solution to structure experiential knowledge relevant for decision making, derive a similarity metric for patients who have received similar treatments, and predict treatment decisions that experts are likely to recommend.

Methods: 277 patient histories relating to 743 breast cancer tumor board decisions at two tertiary care centers were abstracted as the training set for machine learning. 161 distinct histories relating to 496 decisions for a separate expert opinion service at one of the centers was the holdout test set. Data was structured into 690 features based on a novel ontology designed specifically for breast cancer decision making.

To uncover nonlinear similarities, (for example, treatments for younger patients with multiple comorbidities and elderly patients may be similar), treatment decisions were grouped by timing and modality into 13 groups, such as primary surgery, 1st line palliative chemotherapy, etc.

Similarity metric was derived using machine learning on the training set. The target for prediction was the specific treatment decision i.e. TAC or another adjuvant regimen. The primary endpoint was percent accuracy of agreement between XE's predicted decision and experts' actual decision in the holdout test set. Multiple similarity distance metrics including Bhattacharya, Eskin, Goodall, etc., and multiclass classification algorithms such as Extreme Gradient Boosted Trees, Support Vector Machines, etc., were systematically evaluated to arrive at the algorithms that best fit each treatment group.

Results: The winning XE algorithms were 71% to 89% accurate for the various treatment groups, in predicting the actual treatment decisions recommended by the experts. The most frequent treatments recommended across all groups were standard evidence based therapies, as are often recommended by experts. For instance, when XE recommended standard adjuvant therapies for Her2- patients, it was 88% to 97% accurate. When XE recommended nonstandard therapies for the same treatment group, it was 72% to 90% accurate, related to larger number of nonstandard therapies within each treatment group and smaller samples of patients who underwent each type of nonstandard therapy. XE learned to weigh features relating to comorbidities and toxicities when recommending nonstandard therapies.

Conclusion: Machine learning on a structured database of past treatment decisions made by experts, can yield a predicted treatment decision that an expert is likely to recommend for a new patient. By including complex decisions that consider toxicities and morbidities, a rich source of knowledge can be created. Despite the limited dataset, XE learned features that experts strongly consider when making decisions. XE has the potential to analyze variations in decision making at expert practices, assess when to recommend nonstandard therapies, and serve as a training tool for new oncologists to make expert grade treatment decisions.

Citation Format: Ramarajan N, Gupta S, Perry P, Srivastava G, Kumbla A, Miller J, Feldman N, Nair N, Badwe RA. Building an experience engine to make cancer treatment decisions using machine learning [abstract]. In: Proceedings of the 2016 San Antonio Breast Cancer Symposium; 2016 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2017;77(4 Suppl):Abstract nr P1-14-01.