Background: Most endometrial carcinoma patients are diagnosed at an early stage with a good prognosis. However, a relatively low fraction with lethal disease constitutes a substantial number of patients due to the high incidence rate. Preoperative identification of patients with high risk and low risk for poor outcome is necessary to tailor treatment. Nucleotyping refers to characterization of cell nuclei by image cytometry, including the assessment of chromatin structure by nuclear texture analysis. This method is a strong prognostic marker in many cancers but has not been evaluated in preoperative curettage specimens from endometrial carcinoma.

Methods: The prognostic impact of changes in chromatin structure quantified with Nucleotyping was evaluated in preoperative curettage specimens from 791 endometrial carcinoma patients prospectively included in the MoMaTEC multicenter trial.

Results: Nucleotyping was an independent prognostic marker of disease-specific survival in preoperative curettage specimens among patients with Federation Internationale des Gynaecologistes et Obstetristes (FIGO) stage I–II disease (HR=2.9; 95% CI, 1.2–6.5; P = 0.013) and significantly associated with age, FIGO stage, histologic type, histologic grade, myometrial infiltration, lymph node status, curettage histology type, and DNA ploidy.

Conclusions: Nucleotyping in preoperative curettage specimens is an independent prognostic marker for disease-specific survival, with potential to supplement existing parameters for risk stratification to tailor treatment.

Impact: This is the first study to evaluate the prognostic impact of Nucleotyping in curettage specimens from endometrial carcinoma and shows that this may be a clinically useful prognostic marker in endometrial cancer. External validation is warranted. Cancer Epidemiol Biomarkers Prev; 26(1); 61–67. ©2016 AACR.

This article is featured in Highlights of This Issue, p. 1

Endometrial cancer is the most frequently diagnosed gynecologic malignancy and the fourth most common cancer among women in developed countries (1). Although the majority of endometrial cancers are detected at early stage, there is still a significant mortality from the disease that ranks 11 for cancer-related deaths among women in developed countries (1). Age, histologic subtype, grade, Federation Internationale des Gynaecologistes et Obstetristes (FIGO) surgical stage, myometrial invasion, and lymphovascular invasion represent established important prognostic factors (2–4). However, there is still a need for improved prognostic markers, preferably preoperatively, both to identify patients at high risk for recurrence and offer aggressive treatment, as well as identifying low-risk patients to avoid overtreatment (5). Several promising studies on molecular prognostic markers in endometrial cancer have been published, but no molecular biomarkers are widely applied in the routine clinical setting (6). Most studies of DNA ploidy are based on hysterectomy specimens with substantial evidence for prognostic impact, as reviewed by Mauland and colleagues (7). Current routine preoperative risk assessment consists of radiologic imaging combined with determination of histology subtype and grade in curettage or biopsy specimen. Recently, a study of DNA ploidy by image cytometry in the Molecular Markers in Endometrial Cancer (MoMaTEC1) trial showed that DNA ploidy status in curettage specimen is an independent predictor of lymph node status (8). In the current study, we evaluate the prognostic value of chromatin patterns assessed by light microscopy and nuclear texture analysis in the same patient series.

Presence of nondiploid cell populations is an indicator of genomic instability (9). Modified chromatin structure is, if nothing else, a logical effect of genomic instability. Assessing the complexity or disorder of chromatin structure may thus be a reasonable approach to determine the presence or earlier presence of genomic instability. Entropy is an established metric for assessing disorder, originating from thermodynamics, but applied in various fields, including image cytometry (10).

Nucleotyping provides descriptions of cell nuclei and in particular the chromatin structure in cancer cell nuclei from images captured with light microscopy. Properties regarding chromatin structure are assessed with texture analysis based on gray level entropy. The prognostic significance of Nucleotyping has been shown in several cancers, and particularly in gynecologic malignancies (11, 12). In endometrial cancer, however, reports on nuclear texture analyses are limited. One group found a statistically significant association between Markovian texture parameters and prognosis in multivariate analysis (13). A few studies report that chromatin patterns assessed with nuclear texture analysis differ between normal, precursor lesions, and malignant tumors (14–20). This study is the first to investigate the prognostic impact of nuclear texture analysis in curettage specimen from endometrial cancer and by far the largest study to investigate the prognostic impact of nuclear texture analysis in endometrial cancer regardless of specimen type.

In total, 1,046 endometrial cancer patients from the MoMaTEC trial were included in the study with approval from the Regional Committees for Medical and Health Research Ethics in Norway (ClinicalTrials.gov identifier NCT00598845). Of these, 402 patients were treated at the Department of Gynecology and Obstetrics, Haukeland University Hospital (Bergen, Norway), and 644 patients were included from 9 other hospitals. The patients were treated in the period of May 2001 to December 2012. Clinical follow-up data were available for 951 patients, and 160 were excluded from analyses by image cytometry due to poor tissue material quality, leaving 791 patients for survival analysis. Median age at primary treatment was 66 years (range, 28–98) and median follow-up time among censored patients was 37 months (0–96).

Information about recurrence and survival was collected from patient records and correspondence, with physicians responsible for outpatient follow-up. Relevant clinicopathologic parameters were collected from routine histologic reports.

Curettage specimens were formalin fixed and paraffin embedded (FFPE). Preoperative curettage histology reports were categorized as either high risk (histologic type reported as nonendometrioid or histologic grade 3 endometrioid carcinoma) or low risk (other histologic diagnoses, including grade 1 or 2 endometrioid carcinoma, hyperplasia, and benign endometrium). The risk assessment based on the histology report is referred to as curettage histology. The presence of malignant tissue was verified in all analyzed samples, irrespective of the preoperative routine histologic diagnosis.

We included only patients with FIGO stage I–II in the main multivariate analysis, evaluating the proposed prognostic marker together with the preoperatively available parameters age and curettage histology. Patients in this group received treatment with curative intent and thus constitute a homogeneous group with respect to curability. Furthermore, valid risk stratification is still a key clinical challenge in this patient group. Additional analyses with inclusion of patients with FIGO stages I–IV were conducted to evaluate the effect on all patients and demonstrate the performance in a dataset with FIGO stage and clinical outcome distribution similar to the training set in which the method was trained.

Sample preparation and DNA ploidy measurement

Samples were prepared from the FFPE curettage specimens. Tissue areas with the highest tumor grade were identified in hematoxylin- and eosin-stained slides. Depending on the curettage tissue depth, one or two 1-mm cylinders were punched out using a custom-made precision instrument (Beecher Instruments). Monolayers of cell nuclei were prepared from one or more 50-μm sections cut from the cylinder(s) and stained with Feulgen–Schiff according to the established protocols (21). Automated image-based DNA cytometry was carried out using Ploidy Work Station (PWS, Room 4 Ltd.) together with a Zeiss AxioPlan light microscope with a 546-nm green filter and a black and white digital camera (AxioCam MRm, Zeiss). This microscope setup provided a resolution of 162 nm/pixel. Automated imaging was followed by automatic segmentation of cell nuclei in the captured microscope images. Segmented cell nuclei were first classified using an automated cell classification system in PWS and thereafter validated by trained personnel such that preferably only epithelial nuclei were included in the further analyses. DNA ploidy results were classified as diploid or nondiploid by both a computerized algorithm and trained personnel. DNA ploidy measurements in the same material have been published recently (8). A minimum of 100 cell nuclei was required for inclusion in Nucleotyping analyses, and samples with poor technical quality were excluded. In the preparation and analysis pipeline, all but the sample preparation steps were automated. Automated cell classification and DNA ploidy histogram classification were still verified and corrected by trained personnel.

Nucleotyping by gray level entropy

Adaptive nuclear entropy features using the Grey Level Entropy Matrix 4D (GLEM4D) framework as described in ref. 11 and illustrated in Fig. 1 were computed. These features use entropy to quantify chromatin structure in cell nuclei. The method provides a framework for calculating a similarity score between entropy patterns in cell nuclei from patients with different outcome. We have used this framework to calculate similarity scores between each patient and the two defined outcome groups, that is, death from endometrial cancer and no death from endometrial cancer. Details about the GLEM4D framework and its application in this study are provided in the Supplementary Information. Four features were extracted from the GLEM4D, and these four features were also validated in the independent validation set. Nucleotyping is calculated on the same images as used for DNA ploidy analysis, and the calculation procedure is fully automated.

Training and validation sets

To provide realistic estimates of the method's prognostic performance, we divided the data set into a training set and an independent validation set, including every second patient in the training set sorted by surgery date (n = 398 and n = 393, respectively; ref. 22). The training set was used for training the method, while the validation set was used to provide an unbiased estimate on its prognostic value. All patients (i.e., FIGO stage I–IV) were included in the training and validation of the Nucleotyping classifier. Eventually, 2-fold cross-validation was used to provide information about the robustness of the method, that is, in addition to the described training-validation model, the validation set was used as a training set and the resulting classifier tested on the initial training set (23).

Classification

Patients were classified on the basis of their GLEM4D feature values using a minimum Euclidean distance classifier. The estimated outcome classes were called Nucleotyping high and nucleotyping low, referring to estimated good prognosis and poor prognosis, respectively.

Statistical analyses

Death from endometrial cancer was used as an endpoint in survival analyses and as a grouping variable in training the Nucleotyping classifier. Two prognosis groups were thus defined; patients in the good prognosis group did not die from endometrial cancer during the study period, while patients in the poor prognosis group did. Disease-specific survival (DSS) was calculated as months from primary treatment to last follow-up or death from any cause. The definitions regarding endpoints and time to event are the same as used in ref. 8. Survival distributions were compared using the Mantel–Cox log-rank test. Associations between categorical variables were assessed using Pearson χ2 test. Statistical significance was defined as two-sided P < 0.05. In multivariate analysis, the preoperatively available parameters, age at diagnosis and risk group, based on curettage histology were included in addition to the Nucleotyping classifier. MATLAB R2011b (The MathWorks) and IBM SPSS 22.0 (IBM) were used for image analyses and statistical analyses, respectively.

The prognostic value of the four Nucleotyping features extracted from GLEM4D was statistically significant in univariate analyses in the training set (all P < 0.001) and generalized well in the independent validation set (all P < 0.001; Supplementary Table S1). Two-fold cross-validation results based on training on the validation set and testing on the training set were similar to the original result for specificity, while the sensitivity declined (Supplementary Fig. S1). The classification results based on the four extracted features (pos, neg, diff, and posneg) from GLEM4D were highly correlated, and we selected only one of these features that we used in the further analyses (Supplementary Fig. S1). In the following, classification results based on the diff feature is reported and used in various analyses in the entire dataset, unless otherwise specified. For convenience, we will refer to this feature as Nucleotyping.

In univariate analysis, Nucleotyping was a statistically significant prognostic marker among patients with FIGO stage I–II [hazard ratio (HR) = 6.2; 95% confidence interval (CI), 3.0–12.9; P < 0.001; Fig. 2] and patients with all FIGO stages (HR = 3.7; 95% CI, 2.4–5.8; P < 0.001). Among patients with FIGO stage I–II, 5-year DSS was 97% and 83% in the Nucleotyping high and Nucleotyping low groups, respectively. Stratified by FIGO stage, Nucleotyping was a statistically significant prognostic marker among FIGO stage I (HR = 6.9; 95% CI, 2.9–16.2; P < 0.001) and II (HR = 4.6; 95% CI, 1.0–20.7; P = 0.03), but not among FIGO stage III (HR = 1.4; 95% CI, 0.7–2.8; P = 0.39) and IV (HR = 1.9; 95% CI, 0.8–4.7; P = 0.15; Fig. 3).

For multivariate survival analysis of FIGO stage I–II patients, Nucleotyping corrected for age and curettage histology was not significant in the training set (HR = 2.1; 95% CI, 0.7–6.2; P = 0.18), but significant in the validation set (HR = 4.0; 95% CI, 1.1–14.4; P = 0.032). When including patients of all FIGO stages, Nucleotyping demonstrated borderline significant impact on prognosis for both the training and the validation sets (HR = 1.8; 95% CI, 0.9–3.5; P = 0.073 and HR = 2.0; 95% CI, 1.0–3.9; P = 0.053). Among FIGO stages I–II patients in the entire dataset, Nucleotyping was significant in multivariate analysis (HR = 2.9; 95% CI, 1.2–6.5; P = 0.013; Table 1). Also among FIGO stage I–IV patients, the corresponding multivariate analysis result for Nucleotyping was statistically significant (HR = 1.9; 95% CI, 1.2–3.0; P = 0.009). Including FIGO stage as a variable in the stage I–II multivariate model did not change the prognostic impact of Nucleotyping (HR = 3.0; 95% CI, 1.3–6.7; P = 0.010).

Nucleotyping was significantly associated with the preoperatively available parameters age, curettage histology, and curettage DNA ploidy. It was also significantly associated with the postoperatively assessed parameters FIGO stage, histologic type, histologic grade, myometrial infiltration, and lymph node status (Table 2). Among patients with FIGO stage I–II, nucleotyping allowed the identification of 55 diploid patients at significantly increased risk of cancer death compared with the remaining diploid patients (HR = 4.6; 95% CI, 1.5–13.7; P = 0.003; Fig. 4). Also among nondiploid patients, the additional information provided by Nucleotyping over DNA ploidy was significant, with 48 nondiploid patients at reduced risk of cancer death compared with the remaining nondiploid patients (HR = 0.14; 95% CI, 0.02–1.08; P = 0.027).

We have shown that Nucleotyping is an independent prognostic marker for DSS in preoperative specimens from endometrial cancer. Nucleotyping identified 168 (25%) FIGO stage I–II patients at increased risk for poor outcome. Risk classification based on curettage histology is hampered by intra- and interobserver variation (24). Our method is an objective parameter that relatively easily can be implemented as a confirmatory and supplemental method. Nucleotyping was a significant prognostic factor for FIGO stages I and II for which valid risk stratification is still a key clinical challenge. The statistically robust study design with independent training and validation sets and the relatively high number of patients puts confidence to our results.

Studies have postulated DNA ploidy results from curettage specimens as inferior to hysterectomy specimens regarding prognostic impact (25, 26). Nucleotyping provides strong prognostic results in curettage specimens, indicating an even greater potential applied in hysterectomy specimens. Furthermore, performing risk stratification based on preoperatively available information, such as by curettage specimen, is essential in tailoring endometrial cancer treatment.

DNA ploidy and Nucleotyping are related methods, assessing DNA content and DNA structure, respectively. In curettage specimen from endometrial carcinoma, the two methods provided different types of information. DNA ploidy was recently reported to predict lymph node status, while Nucleotyping is a stronger prognostic marker of the two (8).

Results in the training and validation sets and the corresponding 2-fold cross-validation analysis indicated that the overfitting problem was minor in this study (Supplementary Fig. S1; Supplementary Table S1). Nucleotyping did not significantly impact outcome in multivariate analysis for the training set among FIGO stage I–II patients. However, the corresponding result in the validation set was significant, and among FIGO stage I–IV patients, the training and validation set results both demonstrated to predict outcome with borderline significance. The observed variation may be related to the relatively low fraction of patients who experience the event (about 10%) and the corresponding high expected variability in the classification result in this group (27). This problem was also demonstrated in the 2-fold cross-validation analysis and corresponding variable sensitivity (Supplementary Fig. S1) and represents a common challenge with datasets where the event of interest is unevenly distributed. Among patients with FIGO stage I–II, binary classification of DSS by Nucleotyping had a specificity of 77%, while the sensitivity was 63%. The relatively weak sensitivity is another result from the same dataset characteristic.

This, to our knowledge, is the first published study on the prognostic significance of Nucleotyping in endometrial cancer curettage specimens. Digital pathology platforms are increasingly established worldwide with histologic digitalization as a consequence. At the same time, image cytometry has come of age, and this combination provides great opportunities for image-based prognostics and diagnostics with attractive properties, such as high throughput and low cost.

A limitation of this method is the relatively high proportion of excluded samples due to poor monolayer quality, mainly caused by sample contamination by excessive blood. Our standard preparation procedure involves microdissection of the entire region of interest rather than merely a 1-mm cylinder within this region, and the problem with excessive blood is thus less pronounced. Because of the nature of the sample type in this dataset, punching samples with a cylinder seemed suitable. For routine use in the clinic, the high exclusion rate is clearly not ideal, and we will modify our preparation protocol for future studies.

Nucleotyping is influenced by properties from the majority of cell nuclei. It is generally accepted that subpopulations of cells constitute the driving force and have the ability to metastasize during carcinogenesis. To take this fact properly into account, improved cell-by-cell analyses should be carried out.

In conclusion, we have shown that Nucleotyping is an independent prognostic marker in preoperative curettage specimens from patients with FIGO stage I–II endometrial cancer, with a potential for tailoring patient treatment according to risk for recurrent disease. The presented results provide reason to consider image cytometry as a tool for improved risk stratification to tailor clinical care. External validation of the prognostic impact is warranted.

No potential conflicts of interest were disclosed.

The funding sponsors had no role in the study design, collection, analysis or interpretation of data, or in writing the report.

Conception and design: T.S. Hveem, T.S. Njølstad, J. Marcickiewicz, H.B. Salvesen, H.E. Danielsen

Development of methodology: B. Nielsen, J. Marcickiewicz, H.E. Danielsen

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): T.S. Njølstad, J. Marcickiewicz, S. Tingulstad, A.C. Staff, H.K. Haugland, R. Eraker, K. Oddenes, J.A. Rokne, J. Tjugum, M.S. Lode, F. Amant, H.M.J. Werner, L. Bjørge, H.B. Salvesen, J. Trovik, H.E. Danielsen

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): T.S. Hveem, T.S. Njølstad, R.A. Syvertsen, W. Kildal, K. Liestøl, H.B. Salvesen, J. Trovik, H.E. Danielsen

Writing, review, and/or revision of the manuscript: T.S. Hveem, T.S. Njølstad, J.A. Nesheim, W. Kildal, M. Pradhan, J. Marcickiewicz, A.C. Staff, H.K. Haugland, F. Amant, H.M.J. Werner, F. Albregtsen, K. Liestøl, H.B. Salvesen, J. Trovik, H.E. Danielsen

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): R.A. Syvertsen, J.A. Nesheim, M.L. Kjæreng, W. Kildal, H.B. Salvesen, J. Trovik

Study supervision: H.E. Danielsen

Other (histologic evaluation of included tumors): M. Pradhan

Other (sampling of material and clinical data, a small number of patients): M.S. Lode

The authors would like to thank Marian Seiergren for creating Figure 1. Furthermore, we thank Britt Edvardsen for technical assistance. We also thank participants responsible for recruiting patients at the other MoMaTEC (Molecular Markers in Treatment of Endometrial Cancer) centers.

This work was supported by the Helse Vest Research Fund, the Norwegian Research Council, the Norwegian Cancer Society (The Harald Andersen's legacy), and by the Research Council of Norway through its Centres of Excellence funding scheme, project number 179571.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Torre
LA
,
Bray
F
,
Siegel
RL
,
Ferlay
J
,
Lortet-Tieulent
J
,
Jemal
A
. 
Global cancer statistics, 2012
.
CA Cancer J Clin
2015
;
65
:
87
108
.
2.
Amant
F
,
Moerman
P
,
Neven
P
,
Timmerman
D
,
Van
LE
,
Vergote
I
. 
Endometrial cancer
.
Lancet
2005
;
366
:
491
505
.
3.
Prat
J
. 
Prognostic parameters of endometrial carcinoma
.
Hum Pathol
2004
;
35
:
649
62
.
4.
Creutzberg
CL
,
van Putten
WL
,
Koper
PC
,
Lybeert
ML
,
Jobsen
JJ
,
Warlam-Rodenhuis
CC
, et al
Surgery and postoperative radiotherapy versus surgery alone for patients with stage-1 endometrial carcinoma: multicentre randomised trial. PORTEC Study Group. Post operative radiation therapy in endometrial carcinoma
.
Lancet
2000
;
355
:
1404
11
.
5.
Salvesen
HB
,
Haldorsen
IS
,
Trovik
J
. 
Markers for individualised therapy in endometrial carcinoma
.
Lancet Oncol
2012
;
13
:
e353
61
.
6.
Werner
HM
,
Salvesen
HB
. 
Current status of molecular biomarkers in endometrial cancer
.
Curr Oncol Rep
2014
;
16
:
403
.
7.
Mauland
KK
,
Wik
E
,
Salvesen
HB
. 
Clinical value of DNA content assessment in endometrial cancer
.
Cytometry B Clin Cytom
2014
;
86
:
154
63
.
8.
Njølstad
TS
,
Trovik
J
,
Hveem
TS
,
Kjæreng
ML
,
Kildal
W
,
Pradhan
M
, et al
DNA ploidy in curettage specimens identifies high-risk patients and lymph node metastasis in endometrial cancer
.
Br J Cancer
2015
;
112
:
1656
64
.
9.
Lengauer
C
,
Kinzler
KW
,
Vogelstein
B
. 
Genetic instabilities in human cancers
.
Nature
1998
;
396
:
643
9
.
10.
Nielsen
B
,
Albregtsen
F
,
Danielsen
HE
. 
Statistical nuclear texture analysis in cancer research: a review of methods and applications
.
Crit Rev Oncog
2008
;
14
:
89
164
.
11.
Nielsen
B
,
Albregtsen
F
,
Kildal
W
,
Abeler
VM
,
Kristensen
GB
,
Danielsen
HE
. 
The prognostic value of adaptive nuclear texture features from patient gray level entropy matrices in early stage ovarian cancer
.
Anal Cell Pathol
2012
;
35
:
305
14
.
12.
Nielsen
B
,
Hveem
TS
,
Kildal
W
,
Abeler
VM
,
Kristensen
GB
,
Albregtsen
F
, et al
Entropy-based adaptive nuclear texture features are independent prognostic markers in a total population of uterine sarcomas
.
Cytometry A
2015
;
87
:
315
25
.
13.
Geisler
JP
,
Wiemann
MC
,
Zhou
Z
,
Miller
GA
,
Geisler
HE
. 
Markov texture parameters as prognostic indicators in endometrial cancer
.
Gynecol Oncol
1996
;
62
:
174
80
.
14.
Bartels
PH
,
Garcia
FA
,
Davis
J
,
da Silva
VD
,
Bartels
HG
,
Thompson
D
, et al
Progression curves for endometrial lesions
.
Anal Quant Cytol Histol
2001
;
23
:
1
8
.
15.
Garcia
FA
,
Davis
JR
,
Alberts
DS
,
Hatch
K
,
Weyn
B
,
Thompson
D
, et al
Nuclear chromatin patterns in normal, hyperplastic and atypical endometrium
.
Anal Quant Cytol Histol
2001
;
23
:
144
50
.
16.
Garcia
FA
,
Davis
JR
,
Hatch
K
,
Alberts
DS
,
Thompson
D
,
Bartels
PH
. 
Karyometry in endometrial adenocarcinoma of different grades
.
Anal Quant Cytol Histol
2002
;
24
:
93
102
.
17.
Bartels
PH
,
Garcia
FA
,
Trimble
CL
,
Kauderer
J
,
Curtin
J
,
Lim
PC
, et al
Karyometry in atypical endometrial hyperplasia: a Gynecologic Oncology Group study
.
Gynecol Oncol
2012
;
125
:
129
35
.
18.
Diegenbach
PC
,
Baak
JP
. 
Quantitative nuclear image analysis: differentiation between normal, hyperplastic, and malignant appearing uterine glands in a paraffin section. III. The use of texture features for differentiation
.
Eur J Obstet Gynecol Reprod Biol
1978
;
8
:
109
16
.
19.
Diegenbach
PC
,
Baak
JP
. 
Quantitative nuclear image analysis: differentiation between normal, hyperplastic, and malignant appearing uterine glands in a paraffin section. IV. The use of Markov chain texture features in discriminant analysis
.
Eur J Obstet Gynecol Reprod Biol
1978
;
8
:
157
62
.
20.
Baak
JP
,
Kurver
PH
,
Diegenbach
PC
,
Delemarre
JF
,
Brekelmans
EC
,
Nieuwlaat
JE
. 
Discrimination of hyperplasia and carcinoma of the endometrium by quantitative microscopy–a feasibility study
.
Histopathology
1981
;
5
:
61
8
.
21.
Pradhan
M
,
Abeler
VM
,
Danielsen
HE
,
Trope
CG
,
Risberg
BA
. 
Image cytometry DNA ploidy correlates with histological subtypes in endometrial carcinomas
.
Mod Pathol
2006
;
19
:
1227
35
.
22.
Schulerud
H
,
Kristensen
GB
,
Liestol
K
,
Vlatkovic
L
,
Reith
A
,
Albregtsen
F
, et al
A review of caveats in statistical nuclear image analysis
.
Anal Cell Pathol
1998
;
16
:
63
82
.
23.
Arlot
S
,
Celisse
A
.
A survey of cross-validation procedures for model selection
. 
2010
:
40
79
.
Available from
: “http://www.di.ens.fr/willow/pdfs/2010_Arlot_Celisse_SS.pdf.”
24.
Nofech-Mozes
S
,
Ismiil
N
,
Dube
V
,
Saad
RS
,
Ghorab
Z
,
Grin
A
, et al
Interobserver agreement for endometrial cancer characteristics evaluated on biopsy material
.
Obstet Gynecol Int
2012
;
2012
:
414086
.
25.
Baak
JP
,
Snijders
W
,
van Diermen
B
,
van Diest
PJ
,
Diepenhorst
FW
,
Benraadt
J
. 
Prospective multicenter validation confirms the prognostic superiority of the endometrial carcinoma prognostic index in international Federation of gynecology and obstetrics stage 1 and 2 endometrial carcinoma
.
J Clin Oncol
2003
;
21
:
4214
21
.
26.
Pradhan
M
,
Abeler
VM
,
Danielsen
HE
,
Sandstad
B
,
Trope
CG
,
Kristensen
GB
, et al
Prognostic importance of DNA ploidy and DNA index in stage I and II endometrioid adenocarcinoma of the endometrium
.
Ann Oncol
2012
;
23
:
1178
84
.
27.
Isaksson
A
,
Wallman
M
,
Göransson
H
,
Gustafsson
MG
. 
Cross-validation and bootstrapping are unreliable in small sample classification
.
Pattern Recogn Lett
2008
;
29
:
1960
5
.