Purpose:

To evaluate the usefulness of the radiomic model in predicting early (≤2 years) and late (>2 years) recurrence after curative resection in cases involving a single hepatocellular carcinoma (HCC) 2–5 cm in diameter using preoperative gadoxetic acid–enhanced magnetic resonance imaging (MRI), in comparison with the clinicopathologic model.

Experimental Design:

This retrospective study included 167 patients with surgically resected and pathologically confirmed single HCC 2–5 cm in diameter (n = 167, training set:validation set = 128:39) who underwent preoperative gadoxetic acid–enhanced MRI between January 2010 and December 2015. A radiomic model, a clinicopathologic model, and a combined clinicopathologic-radiomic (CCR) model were built using a random survival forest to predict disease-free survival (DFS) in the following conditions: early DFS versus late DFS, dynamic phases, and the peritumoral area included in the segmentation.

Results:

The radiomic model showed a prognostic performance comparable with the clinicopathologic model only with 3-mm peritumoral border extension [c-index difference (radiomic-clinicopathologic), −0.021, P = 0.758]. The CCR model with the 3-mm border extension showed the highest c-index value but no statistically significant improvement over the clinicopathologic model [CCR, 0.716 (0.627–0.799); clinicopathologic model, 0.696 (0.557–0.799)].

Conclusions:

The prognostic value of the preoperative radiomic model with 3-mm border extension showed comparable performance with that of the postoperative clinicopathologic model for predicting DFS of early recurrence of HCC using gadoxetic acid–enhanced MRI. This suggests the importance of including peritumoral changes in the radiomic analysis of HCC.

Translational Relevance

Only few reports have compared the prognostic performance of preoperative magnetic resonance imaging features that predict early recurrence of hepatocellular carcinoma with that of postoperative clinicopathologic findings. Radiomics mathematically extracts high-dimensional features from medical images, which can distinguish between images that are difficult to perceive with the human eye. This study showed that postoperative disease-free survival in patients with resectable hepatocellular carcinoma can be predicted to a comparable level with the postoperative clinicopathologic prediction model using radiomic approach of preoperative gadoxetic acid–enhanced magnetic resonance imaging with three-millimeter peritumoral border extension. Therefore, the radiomic model of the preoperative image with three-millimeter border extension may be useful for predicting prognosis before the invasive procedure and affect the direction of treatment for each patient. However, the combined clinicopathologic and radiomic model showed no improvement in prognostic performance compared with that of the clinicopathologic model.

Hepatocellular carcinoma (HCC) is one of the most common malignant tumors in the liver worldwide and is particularly prevalent in East Asia. Among the various therapeutic options, hepatic resection is considered one of the most efficient treatments in patients with HCC when liver function is preserved. Perioperative safety or long-term survival has recently increased after surgery, but postoperative recurrence of HCC remains high (1, 2).

The pathophysiology of intrahepatic recurrence may be intrahepatic metastasis of the primary carcinoma or de novo multicentric tumor (3). The distinction between these two types of recurrence is important for surveillance, prevention, and management strategies of recurrence (4, 5). A definitive method to distinguish between the two is a genetic or molecular study, but it is technically complex, making its application in clinical practice difficult (6, 7). Several recent studies have reported that aggressive pathological factors, such as high tumor grade, microvascular invasion (MVI), and microsatellite lesions, are associated with tumor recurrence within 2 years after surgery. However, late recurrence is associated with underlying liver conditions, such as liver cirrhosis (8–10). Because imaging is closely correlated with histopathologic examination, some imaging features may reasonably predict tumor recurrence.

Radiomics is an emerging field that converts medical imaging into high-dimensional mineable features using a large number of image-characterization algorithms (11). Prognostic prediction models may be built from noninvasively extracted radiomic features in tumor images (12). Recently, a few radiomic analyses of the early recurrence of HCC based on computed tomography (CT) images have been conducted, and the prediction accuracy was high (13). However, to the best of our knowledge, few attempts have been made to evaluate both early and late recurrence using radiomic approach. In addition, magnetic resonance imaging (MRI) is known to have a higher soft-tissue contrast than CT image, and gadoxetic acid contrast-enhanced MRI can provide functional information such as hepatic uptake. Therefore, we hypothesized that radiomics analysis using gadoxetic acid–enhanced MRI would have comparable or incremental value in predicting early and late recurrence compared with clinicopathologic models.

In this study, we aimed to evaluate the usefulness of the radiomic model in predicting early and late recurrence after curative resection in a single HCC larger than 2 cm and smaller than or equal to 5 cm in diameter using preoperative gadoxetic acid–enhanced MRI, compared with the clinicopathologic model.

Study population

From January 2010 to December 2015, consecutive patients who underwent curative resection for HCC confirmed pathologically after surgery were included. The inclusion criteria for our study were as follows: (i) a single HCC larger than 2 cm and smaller than or equal to 5 cm; (ii) no extrahepatic metastasis or major vascular invasion; (iii) no infiltrative type HCC, satellite nodule, serosal invasion, or adjacent organ invasion; (iv) patients who underwent preoperative gadoxetic acid–enhanced magnetic resonance (MR) imaging within 2 months before surgery; and (v) HCC with hypointensity on hepatobiliary phase (HBP). HCC is known to have different prognosis and prognostic factors based on whether the diameter of the tumor is less than 2 cm or more than 5 cm. More specifically, HCCs larger than 5 cm in maximum diameter are known to have a greater chance of spreading through MVI and poor prognosis (14–16), and small HCCs less than 2 cm are known to be associated with an excellent prognosis not being affected by MVI, histologic grade, or tumor marker level (16). Therefore, we included only single HCC with a diameter larger than 2 cm and smaller than or equal to 5 cm for analysis. The exclusion criteria for our study were as follows: (i) pretreatment history before hepatectomy (n = 44); (ii) patients with comalignancy (n = 27); (iii) postop follow-up loss or expired less than 3 months (n = 5), and inadequate for analysis due to motion artifact (n = 5; Supplementary Data S1).

The final study population consisted of 167 patients (mean age, 56 years; range, 30–77 years), including 131 men (mean age, 56 years; range, 30–77 years) and 36 women (mean age, 58 years; range, 39–72; Table 1). The median time between MR imaging and surgery was 23 days (range, 8–59 days). For temporally independent validation, patients who underwent surgery prior to October 2013 were assigned to a training cohort (n = 128), and the subsequent patients were assigned to a validation cohort (n = 39).

Table 1.

Patient characteristics in the training and validation sets

2–5 cm HCC
ParameterTraining (n = 128)Validation (n = 39)P
Mean age (y) 56.4 ± 9.0 56.8 ± 9.8 0.822 
M/F ratio 97: 31 34: 5 0.196 
Child–Pugh class   1.000 
 A 120 (93.8) 37 (94.9)  
 B 8 (6.3) 2 (5.1)  
MELD score 3.65 ± 3.26 2.96 ± 2.95 0.216 
Serum bilirubin level 0.81 ± 0.45 0.91 ± 0.54 0.287 
Cause of liver disease   0.371 
 HBV 113 (88.3) 32 (82.1)  
 HCV 6 (4.7) 1 (2.6)  
 Alcoholism 3 (2.3) 2 (5.1)  
 Others 6 (4.7) 4 (10.2)  
Cirrhosis 72 (56.3) 18 (46.2) 0.356 
AFP level (ng/mL) 305.2 ± 1643.2 327.9 ± 821.4 0.908 
PIVKA-II level (mAU/mL) 203.6 ± 546.0 71.2 ± 161.4 0.017a 
Major resection 49 (38.3) 11 (28.2) 0.338 
Capsule formation 105 (82.0) 35 (89.7) 0.370 
Differentiation (poor vs. others) 5 (3.9) 2 (5.1) 0.666 
MVI 49 (38.3) 13 (33.3) 0.711 
2–5 cm HCC
ParameterTraining (n = 128)Validation (n = 39)P
Mean age (y) 56.4 ± 9.0 56.8 ± 9.8 0.822 
M/F ratio 97: 31 34: 5 0.196 
Child–Pugh class   1.000 
 A 120 (93.8) 37 (94.9)  
 B 8 (6.3) 2 (5.1)  
MELD score 3.65 ± 3.26 2.96 ± 2.95 0.216 
Serum bilirubin level 0.81 ± 0.45 0.91 ± 0.54 0.287 
Cause of liver disease   0.371 
 HBV 113 (88.3) 32 (82.1)  
 HCV 6 (4.7) 1 (2.6)  
 Alcoholism 3 (2.3) 2 (5.1)  
 Others 6 (4.7) 4 (10.2)  
Cirrhosis 72 (56.3) 18 (46.2) 0.356 
AFP level (ng/mL) 305.2 ± 1643.2 327.9 ± 821.4 0.908 
PIVKA-II level (mAU/mL) 203.6 ± 546.0 71.2 ± 161.4 0.017a 
Major resection 49 (38.3) 11 (28.2) 0.338 
Capsule formation 105 (82.0) 35 (89.7) 0.370 
Differentiation (poor vs. others) 5 (3.9) 2 (5.1) 0.666 
MVI 49 (38.3) 13 (33.3) 0.711 

Abbreviations: HBV, hepatitis B virus; HCV, hepatitis C virus; MELD, model for end-stage liver disease; MV, microvascular invasion.

aStatistically significant.

The institutional review board approved this retrospective study, and the requirement for informed consent was waived because of the retrospective nature of the study. This study was conducted in accordance with the Declaration of Helsinki.

MR imaging protocol

MRI was performed using one of the three 3.0-T systems (Magnetom Trio a Tim, Siemens Medical Solutions; Achieva, Philips Medical Systems; Discovery, GE Medical Systems) or a 1.5-T system (Achieva 1.5-T, Philips Medical Systems).

All images were obtained in the transverse plane with a field of view of 44 × 33 cm or 40 × 30 cm. Two-dimensional dual-echo T1-weighted gradient-recalled-echo images were initially obtained (in-phase and opposed-phase). Dynamic images were obtained before and after contrast material administration in arterial, portal venous, hepatic venous, and final dynamic phases by using a T1-weighted three-dimensional gradient-echo sequence (Supplementary Data S2). To determine the imaging delay for arterial phase imaging, we used a bolus technique with 1 mL of gadoxetic acid disodium (Primovist; Bayer Schering Pharma) and a 20 mL of 0.9% saline chaser at an injection rate of 1 or 2 mL/s to determine the peak enhancement of the abdominal aorta. For dynamic imaging, 0.1 mL/kg (0.025 mmol/kg) of gadoxetic acid disodium was injected, followed by a 20 mL of saline chaser at the same rate as that used for the bolus injection. T2-weighted images were obtained with multishot and single-shot turbo spin-echo sequences by using a navigator-triggered technique. Diffusion-weighted images were also obtained using a navigator-triggered technique at b values of 50, 400, and 800 s/mm2, and the apparent diffusion coefficient was calculated by the MR units.

MR radiomic feature analysis

Image segmentation and preprocessing.

In the preoperative gadoxetic acid contrast-enhanced dynamic liver MRI, arterial phase (AP), portal phase (PP), and HBP images were downloaded in a Digital Imaging and Communications in Medicine format. Registration was then performed on the three phases with resampling the image to a voxel size of 1 × 1 × 1 mm. One abdominal radiologist semiautomatically segmented the tumor lesion in HBP images three-dimensionally using 3D slicer (www.slicer.org), and the generated mask was commonly used for AP, PP, and HBP images (17). The drawn mask expanded the 3 mm and 5 mm outward, creating three kinds of 3D masks with border extensions (0 mm, 3 mm, and 5 mm) to capture the radiomic features of the peritumoral area as well as inside the tumor. Another radiology resident independently performed tumor segmentation on randomly chosen 30 lesions to evaluate interobserver reproducibility.

Radiomic feature extraction.

Feature extraction and additional image preprocessing were performed using PyRadiomics (version 1.3.0; Computational Imaging and Bioinformatics Lab, Harvard Medical School; ref. 18). Before the feature extraction, normalization of the MRI signal intensities (SIs) was performed using PyRadiomics because MRI SI is usually relative with large differences between scanners and vendors. For 1,301 radiomic features were extracted for each dynamic phase, a maximum of 3,903 features were obtained when all three phases were used. Radiomic features were extracted for each phase combination (AP, PP, HBP, AP-PP, AP-HBP, PP-HBP, and AP-PP-HBP) to evaluate the most predictive combination of the phases, and for each border extension (0 mm, 3 mm, and 5 mm) to assess the effect of peritumoral change on prognosis. Subsequently, feature normalization was performed using z-score, and hierarchical feature clustering was performed using Spearman correlation coefficient (Fig. 1). Z-score normalization was performed first in the training cohort and then we recorded the mean and standard deviation values for each radiomic feature. This process did not include a validation cohort. In the validation phase, the radiomic feature values of the validation cases were z-score normalized using the mean and standard deviation values of each radiomic feature memorized in the training cohort. Therefore, validation cases can be predicted one at a time.

Figure 1.

Framework for radiomic analysis.

Figure 1.

Framework for radiomic analysis.

Close modal

Feature selection and prediction model building.

We performed radiomic feature selection using a random forest minimal depth algorithm and built the prognostic model using random survival forest (19, 20), which is one of the methods for survival data set. More specifically, we used two splitting methods in the random survival forest, namely, log-rank method (21) and the maximally selected rank statistics (maxstat) method (22). These methods have recently reported good results as model training methods for survival analysis (23). We presented both results of the two different splitting rules to assess the consistency of the predictions. Hyperparameter optimization was performed to increase model generalizability before building the final prognostic model using the nested cross-validation with five outer iterations and two inner iterations (23).

In addition to the radiomic model, a clinicopathologic model and a combined clinicopathologic-radiomic (CCR) model were built to evaluate the prognostic performance of all three models and incremental value of the radiomic model (Supplementary Data S3). Prognostic model training was performed n = 1,000 times using bootstrap samples of the training cohort. The performances were evaluated in the temporally independent validation set using the Harrell's concordance index (c-index; ref. 24). Diagnostic performances of the model were compared via the 95% confidence interval (CI) for the difference of the c-index. The difference was considered to be significant if the 95% CI did not include 0. The 95% CI was estimated using the percentile bootstrap method with 1,000 resamples (25). Details related to radiomic feature extraction and analysis are described in Supplementary Data S4.

Pathologic evaluation

The surgical specimens of all patients were subjected to histopathologic analysis. Pathologic data analyzed in this study were histologic tumor grade, capsule formation, fibrous capsule invasion, presence of gross and/or microscopic vascular invasion, presence of satellite nodule, and surgical margin abutting. Histologic tumor grade was determined by the following predominant features: well differentiated, moderately differentiated, or poorly differentiated. Capsule formation was recorded as present or absent. Vascular invasion was classified as macroscopic and microscopic depending on the involvement level by the tumor. MVI was defined as a tumor within a vascular space lined by endothelium that was visible only on microscopy.

Evaluation of outcome

All patients underwent contrast-enhanced CT or MRI at 3–6 months after surgery and were followed up for at least 2 years. During the follow-up period, we retrospectively reviewed the medical records for tumor recurrence. Tumor recurrence was determined by radiologic evidence of new tumor in imaging studies. Early recurrence was defined as recurrence within 2 years after curative resection of HCC and late recurrence was defined as recurrence after 2 years after curative resection of HCC. For early recurrence, disease-free survival (DFS) was defined as the interval between the date of surgery and the first date of tumor recurrence on imaging (event) within 2 years or the last follow-up date without recurrence (censored) within 2 years. All cases that did not relapse by 2 years were censored. For late recurrence, DFS was defined as the interval between the date of surgery and the first date of tumor recurrence on imaging (event) after 2 years or the last follow-up date without recurrence (censored). The early recurrence cases that recurred within 2 years were censored.

Statistical analysis

DFS was assessed by the Kaplan–Meier method, and differences in survival distributions between groups were compared using log-rank tests. Cox regression analysis was performed to identify the independent clinicopathologic prognostic factors for DFS. Multivariate Cox regression was performed for variables with a P value of ≤0.2 in the univariate Cox regression analysis. The Harrell's c-index was used to assess prognostic accuracy (24) and hazard ratios and 95% CIs were measured. The interobserver reproducibility of the radiomic features was evaluated in terms of the intraclass correlation coefficient (ICC). An ICC value greater than 0.75 was considered indicative of good reproducibility (26). More specific information related to radiomic analysis is described in Supplementary Data S4. All statistical and radiomic analyses were performed using the R software (version 3.3.1; R Foundation for Statistical Computing). A P value of <0.05 was considered statistically significant.

Patient characteristics

The characteristics of patients in the training and validation sets are summarized in Table 1. No significant differences were found in most clinicopathologic factors between the training and validation sets. Protein induced by vitamin K absence or antagonist-II showed a significant difference between the training and validation sets, but α-fetoprotein showed no significant difference.

In the final study population (n = 167), histologic examination of noncancerous liver parenchyma showed 90 patients with cirrhosis (53.9%). Underlying causes of HCC were hepatitis B infection (n = 145, 86.8%), hepatitis C infection (n = 7, 4.2%), alcoholic liver disease (n = 5, 3.0%), and other origin (n = 10, 6.0%). The surgical resection comprised 60 (35.9%) major resections (≥3 segments according to Couinaud classification) and 107 (64.1%) minor resections or wedge resections. Recurrences occurred in 61 patients (early recurrence, 32; late recurrence, 29) of the 167 patients, with estimated 1-, 2-, and 5-year cumulative global DFS rates being 90.9% (95% CI, 86.6–95.4), 80.4% (95% CI, 74.5–86.7), and 61.5% (95% CI, 53.8–70.2), respectively.

Clinicopathologic prognostic factors

Survival analysis for DFS was performed for early and late recurrences. The identified independent factors for early recurrence were age, hepatitis C infection, alcohol hepatitis, cirrhosis, tumor capsule, and MVI. For late recurrence, cirrhosis was the only independent prognostic factor (Table 2).

Table 2.

Prognostic factors of early versus late recurrence in 2–5 cm HCCs

2–5 cm HCC
Early recurrence (DFS)Late recurrence (DFS)
UnivariateMultivariateUnivariateMultivariate
HRPHR (95% CI)PHRPHR (95% CI)P
Age 1.055 0.009a 1.048 (1.006–1.092) 0.026a 1.027 0.199 1.021 (0.975–1.069) 0.372 
CPS 0.485 0.476   0.983 0.982   
MELD score 0.937 0.246 1.001 (0.857–1.169) 0.987 1.094 0.066 1.097 (0.922–1.305) 0.295 
Hepatitis C 2.564 0.121 6.805 (1.686–27.469) 0.007a 3.821 0.028 2.666 (0.582–12.213) 0.207 
Alcohol hepatitis 5.666 0.004 12.562 (3.330–47.383) <0.001a 2.297 0.415   
Cirrhosis 1.487 0.278 2.875 (1.272–6.495) 0.011a 2.273 0.041 2.366 (1.031–5.428) 0.042a 
Major resection 0.817 0.596   1.041 0.916   
AFP 0.999 0.354   0.999 0.557   
AFP100 0.828 0.678   0.469 0.161 0.478 (0.160–1.435) 0.188 
PIVKA 1.000 0.254 1.000 (0.999–1.001) 0.492 1.000 0.765   
Capsule 0.517 0.107 0.242 (0.0947–0.620) 0.003a 1.701 0.384   
Differentiation 1.494 0.583   1.000 0.997   
MVI 2.075 0.039a 2.711 (1.220–6.027) 0.014a 1.031 0.936   
Serum bilirubin 0.495 0.174 0.337 (0.0715–1.585) 0.168 1.524 0.138 0.698 (0.236–2.068) 0.517 
2–5 cm HCC
Early recurrence (DFS)Late recurrence (DFS)
UnivariateMultivariateUnivariateMultivariate
HRPHR (95% CI)PHRPHR (95% CI)P
Age 1.055 0.009a 1.048 (1.006–1.092) 0.026a 1.027 0.199 1.021 (0.975–1.069) 0.372 
CPS 0.485 0.476   0.983 0.982   
MELD score 0.937 0.246 1.001 (0.857–1.169) 0.987 1.094 0.066 1.097 (0.922–1.305) 0.295 
Hepatitis C 2.564 0.121 6.805 (1.686–27.469) 0.007a 3.821 0.028 2.666 (0.582–12.213) 0.207 
Alcohol hepatitis 5.666 0.004 12.562 (3.330–47.383) <0.001a 2.297 0.415   
Cirrhosis 1.487 0.278 2.875 (1.272–6.495) 0.011a 2.273 0.041 2.366 (1.031–5.428) 0.042a 
Major resection 0.817 0.596   1.041 0.916   
AFP 0.999 0.354   0.999 0.557   
AFP100 0.828 0.678   0.469 0.161 0.478 (0.160–1.435) 0.188 
PIVKA 1.000 0.254 1.000 (0.999–1.001) 0.492 1.000 0.765   
Capsule 0.517 0.107 0.242 (0.0947–0.620) 0.003a 1.701 0.384   
Differentiation 1.494 0.583   1.000 0.997   
MVI 2.075 0.039a 2.711 (1.220–6.027) 0.014a 1.031 0.936   
Serum bilirubin 0.495 0.174 0.337 (0.0715–1.585) 0.168 1.524 0.138 0.698 (0.236–2.068) 0.517 

Abbreviations: AFP100, AFP level over 100; CI, confidence interval; CPS, Child–Pugh score A versus B; DFS, disease-free survival; MELD, model for end-stage liver disease; MVI, microvascular invasion.

aStatistically significant.

MR radiomic analysis and model comparison

We evaluated the performance of the clinicopathologic, radiomic, and CCR models for three tumor border extensions (0, 3, and 5 mm).

For the prediction of DFS in early recurrence, the radiomic model showed a prognostic performance comparable with the clinicopathologic model without statistically significant difference, only with 3-mm border extension [c-index difference (radiomic-clinicopathologic), log-rank −0.021, P = 0.758; maxstat, 0.008, P = 0.986; Table 3]. When the 3-mm border extension was used, the highest radiomic prognostic performance was observed (c-index, log-rank 0.675; maxstat 0.679) and the highest prognostic performance of the CCR model was observed (log-rank 0.716, maxstat 0.707; Table 3; Supplementary Data S5).

Table 3.

Comparison of diagnostic performance of radiomic model versus clinicopathologic model according to tumor size and peritumoral border extension (all three dynamic phases were used)

SizeBaSbCPRADCCRDifference (CCR-CP)cPDifference (RAD-CP)dP
Early recurrence (c-index)
2–5 cm 0.701 0.470 0.516 −0.185 (−0.320, −0.018) 0.034e −0.231 (−0.381, −0.065) 0.018e 
HCCs  0.673 0.501 0.527 −0.146 (−0.320, 0.014) 0.064 −0.172 (−0.365, 0.031) 0.108f 
 0.696 0.675 0.716g 0.019 (−0.101, 0.162) 0.788 −0.021 (−0.160, 0.148) 0.758f 
  0.671 0.679 0.707g 0.036 (−0.092, 0.209) 0.620 0.008 (−0.156, 0.217) 0.986f 
 0.697 0.397 0.448 −0.250 (−0.402, −0.076) 0.008e −0.301 (−0.471, −0.109) 0.008e 
  0.671 0.364 0.404 −0.267 (−0.414, −0.094) 0.002e −0.308 (−0.492, −0.102) 0.004e 
Late recurrence (c-index) 
2–5 cm 0.746 0.491 0.594 −0.152 (−0.359, 0.058) 0.128 −0.255 (−0.492, −0.041) 0.046e 
HCCs  0.745 0.516 0.680 −0.065 (−0.342, 0.095) 0.404 −0.228 (−0.542, 0.075) 0.062 
 0.745 0.545 0.595 −0.150 (−0.508, 0.125) 0.402 −0.200 (−0.542, 0.108) 0.224 
  0.737 0.622 0.716 −0.021 (−0.358, 0.225) 0.894 −0.115 (−0.475, 0.258) 0.468 
 0.744 0.705 0.711 −0.033 (−0.208, 0.359) 0.628 −0.039 (−0.225, 0.408) 0.524 
  0.744 0.696 0.696 −0.048 (−0.292, 0.178) 0.636 −0.048 (−0.358, 0.358) 0.594 
SizeBaSbCPRADCCRDifference (CCR-CP)cPDifference (RAD-CP)dP
Early recurrence (c-index)
2–5 cm 0.701 0.470 0.516 −0.185 (−0.320, −0.018) 0.034e −0.231 (−0.381, −0.065) 0.018e 
HCCs  0.673 0.501 0.527 −0.146 (−0.320, 0.014) 0.064 −0.172 (−0.365, 0.031) 0.108f 
 0.696 0.675 0.716g 0.019 (−0.101, 0.162) 0.788 −0.021 (−0.160, 0.148) 0.758f 
  0.671 0.679 0.707g 0.036 (−0.092, 0.209) 0.620 0.008 (−0.156, 0.217) 0.986f 
 0.697 0.397 0.448 −0.250 (−0.402, −0.076) 0.008e −0.301 (−0.471, −0.109) 0.008e 
  0.671 0.364 0.404 −0.267 (−0.414, −0.094) 0.002e −0.308 (−0.492, −0.102) 0.004e 
Late recurrence (c-index) 
2–5 cm 0.746 0.491 0.594 −0.152 (−0.359, 0.058) 0.128 −0.255 (−0.492, −0.041) 0.046e 
HCCs  0.745 0.516 0.680 −0.065 (−0.342, 0.095) 0.404 −0.228 (−0.542, 0.075) 0.062 
 0.745 0.545 0.595 −0.150 (−0.508, 0.125) 0.402 −0.200 (−0.542, 0.108) 0.224 
  0.737 0.622 0.716 −0.021 (−0.358, 0.225) 0.894 −0.115 (−0.475, 0.258) 0.468 
 0.744 0.705 0.711 −0.033 (−0.208, 0.359) 0.628 −0.039 (−0.225, 0.408) 0.524 
  0.744 0.696 0.696 −0.048 (−0.292, 0.178) 0.636 −0.048 (−0.358, 0.358) 0.594 

NOTE: Each prognostic model training was performed n = 1,000 times using bootstrap samples of the training cohort. The performances were evaluated in the temporally independent validation set using the c-index. The number of the training set and the validation set was 128 and 39.

Abbreviations: 0, 3, 5, the peritumoral border extension (mm); ALL, all three phases of AP, PP, and HBP; AP, arterial phase; CCR, combined clinicopathologic-radiomic model; CP, clinicopathologic model; Difference, difference between c-indices of the prognostic models; HBP, hepatobiliary phase; L, log-rank split rule; M, maxstat split rule; PP, portal phase; RAD, radiomic model.

aB means peritumoral border extension (mm).

bS means splitting methods for the random survival forest.

cDifference (CCR-CP) means difference between c-indices of the CCR and CP; numbers in parentheses are 95% CIs.

dDifference (RAD-CP) means difference between c-indices of the RAD and CP; numbers in parentheses are 95% CIs.

eStatistically significant.

fThe radiomic model showed a prognostic performance comparable to the clinicopathologic model without statistically significant difference.

gThe highest value in the CCR models. However, the CCR model showed no statistically significant improvement over the clinicopathologic model.

The CCR model showed no statistically significant improvement over the clinicopathologic model in all cases (Table 3). Except for 3-mm border extension, most of the CCR models were less predictive than the clinicopathologic models with or without statistical significance.

For late recurrence DFS prediction, the radiomic model did not show statistically significant different c-index values from those of the clinicopathologic model, but numerically the radiomic model showed lower c-index values than the clinicopathologic model in all cases. The highest c-index value was observed in the clinicopathologic model (c-index, 0.746; Table 3).

For the analysis of combination of different phases of gadoxetic acid–enhanced MRI, the highest c-index was achieved when all three phases were used to predict early recurrence (c-index, log-rank 0.716; maxstat 0.707; Table 4). Among the single phases, HBP showed the highest c-index value (c-index, log-rank 0.669; maxstat 0.682; Table 4).

Table 4.

Comparison of the diagnostic performance of the radiomic model versus the clinicopathologic model according to dynamic phases included in the radiomic analysis (in the condition of 3-mm peritumoral border extension)

PhaseSaCPRADCCRDifference (CCR-CP)bPDifference (RAD-CP)cP
Early recurrence (c-index)
AP 0.701 0.609 0.633 −0.068 (−0.270, 0.127) 0.548 −0.093 (−0.303, 0.119) 0.410 
 0.673 0.580 0.599 −0.075 (−0.240, 0.115) 0.368 −0.093 (−0.268, 0.115) 0.348 
PP 0.698 0.542 0.595 −0.103 (−0.256, 0.094) 0.264 −0.156 (−0.328, 0.061) 0.116 
 0.673 0.584 0.667 −0.006 (−0.187, 0.182) 0.954 −0.089 (−0.287, 0.135) 0.408 
HBP 0.699 0.581 0.669d −0.030 (−0.189, 0.121) 0.75 −0.118 (−0.283, 0.051) 0.156 
 0.673 0.583 0.682d 0.009 (−0.137, 0.156) 0.892 −0.090 (−0.287, 0.111) 0.382 
AP + PP 0.701 0.561 0.604 −0.097 (−0.209, 0.037) 0.152 −0.140 (−0.258, 0.010) 0.072 
 0.670 0.528 0.577 −0.094 (−0.221, 0.051) 0.21 −0.142 (−0.293, 0.027) 0.098 
AP + HBP 0.700 0.611 0.700 <0.001 (−0.119, 0.125) 0.996 −0.089 (−0.250, 0.088) 0.300 
 0.672 0.591 0.660 −0.013 (−0.137, 0.109) 0.862 −0.081 (−0.273, 0.111) 0.398 
PP + HBP 0.699 0.626 0.704 0.004 (−0.113, 0.123) 0.912 −0.073 (−0.217, 0.092) 0.356 
 0.677 0.594 0.698 0.021 (−0.127, 0.164) 0.738 −0.083 (−0.293, 0.111) 0.394 
ALL 0.696 0.675 0.716e 0.019 (−0.101, 0.162) 0.788 −0.021 (−0.160, 0.148) 0.758 
 0.671 0.679 0.707e 0.036 (−0.092, 0.209) 0.620 0.008 (−0.156, 0.217) 0.986 
Late recurrence (c-index) 
AP 0.740 0.483 0.485 −0.255 (−0.475, 0.175) 0.070 −0.257 (−0.492, 0.192) 0.068 
 0.738 0.381 0.464 −0.273 (−0.575, 0.108) 0.138 −0.357 (−0.608, 0.092) 0.076 
PP 0.744 0.489 0.498 −0.245 (−0.408, 0.142) 0.066 −0.255 (−0.425, 0.142) 0.068 
 0.741 0.609 0.663 −0.078 (−0.275, 0.308) 0.376 −0.132 (−0.308, 0.309) 0.136 
HBP 0.744 0.516 0.519 −0.225 (−0.425, 0.175) 0.056 −0.228 (−0.442, 0.175) 0.056 
 0.744 0.427 0.504 −0.240 (−0.592, 0.042) 0.088 −0.317 (−0.642, 0.008) 0.054 
AP+PP 0.745 0.607 0.621 −0.124 (−0.342, 0.094) 0.212 −0.138 (−0.358, 0.094) 0.168 
 0.742 0.514 0.610 −0.132 (−0.408, 0.142) 0.268 −0.228 (−0.492, 0.158) 0.132 
AP+HBP 0.743 0.233 0.269 −0.474 (−0.675, −0.125) 0.026f −0.511 (−0.692, −0.125) 0.024f 
 0.740 0.398 0.569 −0.172 (−0.525, 0.125) 0.306 −0.342 (−0.625, 0.075) 0.064 
PP+HBP 0.744 0.509 0.545 −0.199 (−0.492, 0.108) 0.198 −0.236 (−0.508, 0.075) 0.112 
 0.744 0.529 0.570 −0.174 (−0.492, 0.092) 0.240 −0.216 (−0.475, 0.108) 0.138 
ALL 0.745 0.545 0.595 −0.150 (−0.508, 0.125) 0.402 −0.200 (−0.542, 0.108) 0.224 
 0.737 0.622 0.716 −0.021 (−0.358, 0.225) 0.894 −0.115 (−0.475, 0.258) 0.468 
PhaseSaCPRADCCRDifference (CCR-CP)bPDifference (RAD-CP)cP
Early recurrence (c-index)
AP 0.701 0.609 0.633 −0.068 (−0.270, 0.127) 0.548 −0.093 (−0.303, 0.119) 0.410 
 0.673 0.580 0.599 −0.075 (−0.240, 0.115) 0.368 −0.093 (−0.268, 0.115) 0.348 
PP 0.698 0.542 0.595 −0.103 (−0.256, 0.094) 0.264 −0.156 (−0.328, 0.061) 0.116 
 0.673 0.584 0.667 −0.006 (−0.187, 0.182) 0.954 −0.089 (−0.287, 0.135) 0.408 
HBP 0.699 0.581 0.669d −0.030 (−0.189, 0.121) 0.75 −0.118 (−0.283, 0.051) 0.156 
 0.673 0.583 0.682d 0.009 (−0.137, 0.156) 0.892 −0.090 (−0.287, 0.111) 0.382 
AP + PP 0.701 0.561 0.604 −0.097 (−0.209, 0.037) 0.152 −0.140 (−0.258, 0.010) 0.072 
 0.670 0.528 0.577 −0.094 (−0.221, 0.051) 0.21 −0.142 (−0.293, 0.027) 0.098 
AP + HBP 0.700 0.611 0.700 <0.001 (−0.119, 0.125) 0.996 −0.089 (−0.250, 0.088) 0.300 
 0.672 0.591 0.660 −0.013 (−0.137, 0.109) 0.862 −0.081 (−0.273, 0.111) 0.398 
PP + HBP 0.699 0.626 0.704 0.004 (−0.113, 0.123) 0.912 −0.073 (−0.217, 0.092) 0.356 
 0.677 0.594 0.698 0.021 (−0.127, 0.164) 0.738 −0.083 (−0.293, 0.111) 0.394 
ALL 0.696 0.675 0.716e 0.019 (−0.101, 0.162) 0.788 −0.021 (−0.160, 0.148) 0.758 
 0.671 0.679 0.707e 0.036 (−0.092, 0.209) 0.620 0.008 (−0.156, 0.217) 0.986 
Late recurrence (c-index) 
AP 0.740 0.483 0.485 −0.255 (−0.475, 0.175) 0.070 −0.257 (−0.492, 0.192) 0.068 
 0.738 0.381 0.464 −0.273 (−0.575, 0.108) 0.138 −0.357 (−0.608, 0.092) 0.076 
PP 0.744 0.489 0.498 −0.245 (−0.408, 0.142) 0.066 −0.255 (−0.425, 0.142) 0.068 
 0.741 0.609 0.663 −0.078 (−0.275, 0.308) 0.376 −0.132 (−0.308, 0.309) 0.136 
HBP 0.744 0.516 0.519 −0.225 (−0.425, 0.175) 0.056 −0.228 (−0.442, 0.175) 0.056 
 0.744 0.427 0.504 −0.240 (−0.592, 0.042) 0.088 −0.317 (−0.642, 0.008) 0.054 
AP+PP 0.745 0.607 0.621 −0.124 (−0.342, 0.094) 0.212 −0.138 (−0.358, 0.094) 0.168 
 0.742 0.514 0.610 −0.132 (−0.408, 0.142) 0.268 −0.228 (−0.492, 0.158) 0.132 
AP+HBP 0.743 0.233 0.269 −0.474 (−0.675, −0.125) 0.026f −0.511 (−0.692, −0.125) 0.024f 
 0.740 0.398 0.569 −0.172 (−0.525, 0.125) 0.306 −0.342 (−0.625, 0.075) 0.064 
PP+HBP 0.744 0.509 0.545 −0.199 (−0.492, 0.108) 0.198 −0.236 (−0.508, 0.075) 0.112 
 0.744 0.529 0.570 −0.174 (−0.492, 0.092) 0.240 −0.216 (−0.475, 0.108) 0.138 
ALL 0.745 0.545 0.595 −0.150 (−0.508, 0.125) 0.402 −0.200 (−0.542, 0.108) 0.224 
 0.737 0.622 0.716 −0.021 (−0.358, 0.225) 0.894 −0.115 (−0.475, 0.258) 0.468 

NOTE: Each prognostic model training was performed n = 1,000 times using bootstrap samples of the training cohort. The performances were evaluated in the temporally independent validation set using the c-index. The number of the training set and the validation set was 128 and 39.

Abbreviations: 0, 3, 5, the peritumoral border extension (mm); ALL, all three phases of AP, PP, and HBP; AP, arterial phase; CCR, combined clinicopathologic-radiomic model; CP, clinicopathologic model; Difference, difference between c-indices of the prognostic models; HBP, hepatobiliary phase; L, log-rank split rule; M, maxstat split rule; PP, portal phase; RAD, radiomic model.

aS means splitting methods for the random survival forest.

bDifference (CCR-CP) means difference between c-indices of the CCR and CP; numbers in parentheses are 95% CIs.

cDifference (RAD-CP) means difference between c-indices of the RAD and CP; numbers in parentheses are 95% CIs.

dAmong the single phases, HBP showed the highest c-index value.

eThe highest c-index was achieved when all three phases were used to predict early recurrence.

fStatistically significant.

In the CCR model, we obtained the optimal cutoff for differentiating the high-risk group and the low-risk group based on the hazard function (21) for the training data set (log-rank test, P < 0.001; Fig. 2). When the validation set was stratified into high- and low-risk groups, the survival curves showed statistically significant difference between the two groups (P = 0.031) in the validation set (Fig. 2). Meanwhile, the survival curves were significantly different in the training data for the clinicopathologic model (P < 0.001) or radiomic model (P < 0.001). However, the survival curves of the high- and low-risk groups were not significantly different in the validation set in these two models.

Figure 2.

CCR model for predicting DFS in early postoperative period (<2 years) in patients with 2–5 cm HCCs. Kaplan–Meier estimates of DFS and prediction curves of the CCR for DFS in the training set (A, B) and the validation set (C, D). The validation cohort stratified into a low- and a high-risk group based on a cutoff value determined on the training cohort. A significant patient stratification was shown (P = 0.031) with high predictive performance (c-index: 0.716). In the training cohort, recurrences occurred in 22 of 29 patients in the high-risk group and 3 of 99 patients in the low-risk group. In the validation cohort, recurrences occurred in 4 of 10 patients in the high-risk group and 3 of 29 patients in the low-risk group.

Figure 2.

CCR model for predicting DFS in early postoperative period (<2 years) in patients with 2–5 cm HCCs. Kaplan–Meier estimates of DFS and prediction curves of the CCR for DFS in the training set (A, B) and the validation set (C, D). The validation cohort stratified into a low- and a high-risk group based on a cutoff value determined on the training cohort. A significant patient stratification was shown (P = 0.031) with high predictive performance (c-index: 0.716). In the training cohort, recurrences occurred in 22 of 29 patients in the high-risk group and 3 of 99 patients in the low-risk group. In the validation cohort, recurrences occurred in 4 of 10 patients in the high-risk group and 3 of 29 patients in the low-risk group.

Close modal

The interobserver reproducibility of the radiomic feature extraction showed high ICC values (median, 0.991; range, 0.813–0.999 for the model with 3-mm border extension; Supplementary Data S6).

In the present study, we developed and validated a radiomic model and CCR model using preoperative gadoxetic acid–enhanced MRI for individualized prediction of DFS in patients with HCC. Our results demonstrated that the radiomic model with appropriate border extension using preoperative MRI showed comparable performance with the postoperative clinicopathologic model for predicting DFS of early recurrence. This result supports the clinical importance of using radiomics for preoperative clinical decision-making before the surgical pathology report in patients with HCC. More refined and personalized radiomic model using machine learning may improve the performance of prognostication of each patient with HCC. In addition, we developed a radiomic model using gadoxetic acid–enhanced multiphase MRI data to reflect the nature of tumor perfusion and the hepatic uptake function. The suggested optimal forms of the input data for the radiomic model in the present study could be the basis for future research in HCC.

The present study demonstrated the preoperative usefulness of radiomic features for predicting early recurrence of HCC. Multiple tumors, microscopic vascular invasion, serosal invasion, tumor markers, Milan criteria, positive surgical margin, and nonanatomic resection are known clinicopathologic factors that affect early recurrence (27–29). Imaging features, such as peritumoral enhancement, peritumoral hypointensity, irregular tumor margin, and arterial rim enhancement, are also associated with the early recurrence of HCC (30, 31). In previous studies, most preoperative imaging prognostic factors, which are generally subjective criteria by the observers, were reported to be correlated with postoperative histopathologic factors. However, in the present study, we observed that preoperative radiomic features showed comparable prognostic performance with that of postoperative clinicopathologic data. It is meaningful in clinical practice in several aspects. It is helpful in predicting prognosis and determining management plan in patients without histology. It is objective compared with image interpretation, which depends on the observer's subjectivity. In addition, the radiomic model may be useful for prognostic prediction after surgical histology is obtained to compensate for the pathologic report. This may be explained by the fact that radiomic features were derived by mathematical equations, which could capture histopathologic characteristics and gross morphologic features. Moreover, in vivo radiomics can analyze the entire tumor thoroughly in three dimensions, indicating that comparable or more information could be obtained compared with pathologic examination in which only small tissues are sampled and assessed. In this study, we did not observe an additional value of the radiomic model compared with that of the clinicopathologic model, but further study is required because the importance of radiomic features incorporated with clinicopathologic data has been reported in several oncologic studies (13, 32).

For the prognostic performance of late recurrence, the radiomic model did not show statistically significant difference from the clinical model, but the clinical usefulness of the radiomic model was low. Recurrence of HCC was speculated to be caused by either metastasis of primary tumor or metachronous multicentric occurrence in the underlying liver disease. Several studies have reported that early recurrence is due to primary tumor metastasis and late recurrence is due to multicentric occurrence (9, 33). In late recurrence, only liver cirrhosis has been reported as an independent factor (8, 9). Consistent with the previous studies, the only independent clinicopathologic factor of the late recurrence in our study was liver cirrhosis. Although the radiomic model did not show a significant difference with the clinicopathologic model, the only independent factor, liver cirrhosis, can be assessed before surgery more easily than with the radiomic model. Therefore, the clinical usefulness of the radiomic model is limited for predicting late recurrence.

We found that the peritumoral change had a significant effect on DFS through radiomics. The predictability of DFS increased when a peritumoral area of 3 mm diameter was included in the radiomic analysis. Our results are in good agreement with those reported in previous studies, which suggested that arterial peritumoral enhancement, non-smooth tumor margin, and peritumoral hypointensity on HBP are independent predictors of MVI, which is a potent prognostic factor of HCC (30, 31). All these suggested prognostic findings are presented in the peritumoral area. However, most previous oncologic radiomic studies did not consider the peritumoral area as a potential prognostic factor, because only the tumors were segmented (13, 32, 34). In the present study, the radiomic model showed a prognostic performance comparable to that of the clinicopathologic model only for 3-mm border extension, in which the highest prognostic performance was observed. This result indicates that appropriate border extension is required to achieve performance comparable to the clinicopathologic model. The radiomic model with 5-mm border extension showed lower prognostic performance than the clinicopathologic model. This may be due to the known nature of the distribution of microsatellites in HCC. A microsatellite is defined as a pathologically microscopic invasion into the portal vein and/or intrahepatic metastasis. The number of microsatellites is known to decrease with distance from the primary HCC and increase with the primary tumor size (35–37). Therefore, it is possible that the 5-mm border extension may have reduced prognostic performance due to the decrease in the number of microsatellites in the farther peritumoral area and inclusion of the peritumoral normal hepatic parenchyma. This suggested that proper border extensions may contribute to predictability.

Technically, a distinct tumoral boundary in HBP of gadoxetic acid–enhanced MRI enabled the border extension with high reproducibility. Segmentation is critical for radiomics because the subsequent feature data are generated from the segmented volumes. However, it is also challenging because tumors may have indistinct borders (12). In a previous radiomic study, HCC border segmentation was performed based on morphologic or perfusional difference in AP or PP of CT scan manually (13). However, in our experience, distinguishing between tumor itself and peritumoral change is difficult because of the indistinct border. We believe that our method could be an easy solution to better segmentation in MRI. By defining clear boundaries, we could analyze the various diameters of the peritumoral area, resulting in prognostic potential of morphologic changes within 3 mm of the peritumoral area. Moreover, the prediction of DFS showed the highest performance when all three dynamic phase images were used among all the possible combinations of three phases. Various prognostic imaging features, such as peritumoral enhancement or rim enhancement in AP, irregular or non-smooth tumor margin, and peritumoral hypointensity in HBP, have been demonstrated in different dynamic phases (30, 31, 38–41). Therefore, all three phases seem to be necessary to reflect all of these image features. Our result suggests that a multiparametric approach is required for postoperative prognostic analysis of HCC using MRI and should include at least three dynamic phases.

This study has several limitations. First, it has a limited sample size of 167 patients. Radiomic features were basically high-dimensional data, and machine learning–based random forest was used; thus, there was a risk of overfitting. Therefore, parameter optimization was performed using the nested 5-fold cross-validation of the two different methodologies (log-rank, maxstat) to achieve the generalized model. The prediction model was validated with a temporally independent validation set using bootstrap resampling (n = 1,000) of the training set for an objective evaluation of the model. Second, semiautomatic region of interest placement can be a limiting factor because interobserver variability is known to affect the result (42). However, because we used HBP images in gadoxetic acid–enhanced MRI for segmentation, high interobserver reproducibility could be achieved. Third, only HBP-hypointense HCCs were included (excluding iso-/hyperintense HCCs) for clear delineation of the HCCs in HBP. However, only a small proportion of HCCs may show iso to hyperintensity on HBP, and they are known to have good prognosis (43, 44). Fourth, we performed SI normalization and voxel size resampling prior to radiomics feature extraction for quantitative comparison of images between different MRI machines and patients, but did not have access to how this process affects the results. However, recent CT radiomics studies have reported that using gray-level normalization and voxel-size resampling greatly reduced the dependency of differences in scanners and CT parameters (45–47). It is presumed that these preprocesses would be effective for MRI, but further study is needed. Fifth, the training and validation cohorts were temporally disjoint and may cause problems if there were temporal changes in image acquisition or patient characteristics. However, this study was performed without any changes in the MRI unit or the image acquisition protocol between the training cohort and the validation cohort. No statistically significant difference was found in the clinical conditions between the training cohort and the validation cohort except for PIVKA-II (Table 1). PIVKA-II is a tumor marker associated with the diagnosis of HCC, but there is no evidence that it affects the prognosis (30, 31). In addition, it is known that temporal validation is better than random split according to the TRIPOD statement (48).

In conclusion, the prognostic value of the preoperative radiomic model with 3-mm border extension showed comparable performance with that of the postoperative clinicopathologic model for predicting DFS of early recurrence of HCC using gadoxetic acid–enhanced MRI. This suggests the importance of including peritumoral changes in the radiomic analysis of HCC.

No potential conflicts of interest were disclosed.

Conception and design: S. Kim, G.H. Choi, M.-J. Kim, J.-Y. Choi

Development of methodology: S. Kim

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): S. Kim, J. Shin, D.-Y. Kim, M.-J. Kim

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): S. Kim, J. Shin, D.-Y. Kim, G.H. Choi, J.-Y. Choi

Writing, review, and/or revision of the manuscript: S. Kim, J. Shin, J.-Y. Choi

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): S. Kim, J. Shin, M.-J. Kim, J.-Y. Choi

Study supervision: S. Kim, D.-Y. Kim, G.H. Choi, J.-Y. Choi

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Hwang
S
,
Lee
YJ
,
Kim
KH
,
Ahn
CS
,
Moon
DB
,
Ha
TY
, et al
The impact of tumor size on long-term survival outcomes after resection of solitary hepatocellular carcinoma: single-institution experience with 2558 patients
.
J Gastrointest Surg
2015
;
19
:
1281
90
.
2.
Wong
TC
,
Lo
CM
. 
Resection strategies for hepatocellular carcinoma
.
Semin Liver Dis
2013
;
33
:
273
81
.
3.
Cha
C
,
Fong
Y
,
Jarnagin
WR
,
Blumgart
LH
,
DeMatteo
RP
. 
Predictors and patterns of recurrence after resection of hepatocellular carcinoma
.
J Am Coll Surg
2003
;
197
:
753
8
.
4.
Raza
A
,
Sood
GK
. 
Hepatocellular carcinoma review: Current treatment, and evidence-based medicine
.
World J Gastroenterol
2014
;
20
:
4115
27
.
5.
Balogh
J
,
Victor
D
,
Asham
EH
,
Burroughs
SG
,
Boktour
M
,
Saharia
A
, et al
Hepatocellular carcinoma: a review
.
J Hepatocell Carcinoma
2016
;
3
:
41
53
.
6.
Chen
PJ
,
Chen
DS
,
Lai
MY
,
Chang
MH
,
Huang
GT
,
Yang
PM
, et al
Clonal origin of recurrent hepatocellular carcinomas
.
Gastroenterology
1989
;
96
(
2 Pt 1
):
527
9
.
7.
Ng
IO
,
Guan
XY
,
Poon
RT
,
Fan
ST
,
Lee
JM
. 
Determination of the molecular relationship between multiple tumour nodules in hepatocellular carcinoma differentiates multicentric origin from intrahepatic metastasis
.
J Pathol
2003
;
199
:
345
53
.
8.
Portolani
N
,
Coniglio
A
,
Ghidoni
S
,
Giovanelli
M
,
Benetti
A
,
Tiberio
GAM
, et al
Early and late recurrence after liver resection for hepatocellular carcinoma: prognostic and therapeutic implications
.
Ann Surg
2006
;
243
:
229
.
9.
Yamamoto
Y
,
Ikoma
H
,
Morimura
R
,
Konishi
H
,
Murayama
Y
,
Komatsu
S
, et al
Optimal duration of the early and late recurrence of hepatocellular carcinoma after hepatectomy
.
World J Gastroenterol
2015
;
21
:
1207
.
10.
Cheng
Z
,
Yang
P
,
Qu
S
,
Zhou
J
,
Yang
J
,
Yang
X
, et al
Risk factors and management for early and late intrahepatic recurrence of solitary hepatocellular carcinoma after curative resection
.
HPB (Oxford)
2015
;
17
:
422
7
.
11.
Aerts
HJ
,
Velazquez
ER
,
Leijenaar
RT
,
Parmar
C
,
Grossmann
P
,
Carvalho
S
, et al
Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
.
Nat Commun
2014
;
5
:
4006
.
12.
Gillies
RJ
,
Kinahan
PE
,
Hricak
H
. 
Radiomics: images are more than pictures, they are data
.
Radiology
2015
;
278
:
563
77
.
13.
Zhou
Y
,
He
L
,
Huang
Y
,
Chen
S
,
Wu
P
,
Ye
W
, et al
CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma
.
Abdom Radiol (NY)
2017
;
42
:
1695
704
.
14.
Kim
BK
,
Han
KH
,
Park
YN
,
Park
MS
,
Kim
KS
,
Choi
JS
, et al
Prediction of microvascular invasion before curative resection of hepatocellular carcinoma
.
J Surg Oncol
2008
;
97
:
246
52
.
15.
Kaibori
M
,
Ishizaki
M
,
Matsui
K
,
Kwon
AH
. 
Predictors of microvascular invasion before hepatectomy for hepatocellular carcinoma
.
J Surg Oncol
2010
;
102
:
462
8
.
16.
Shindoh
J
,
Andreou
A
,
Aloia
TA
,
Zimmitti
G
,
Lauwers
GY
,
Laurent
A
, et al
Microvascular invasion does not predict long-term survival in hepatocellular carcinoma up to 2 cm: reappraisal of the staging system for solitary tumors
.
Ann Surg Oncol
2013
;
20
:
1223
9
.
17.
Fedorov
A
,
Beichel
R
,
Kalpathy-Cramer
J
,
Finet
J
,
Fillion-Robin
JC
,
Pujol
S
, et al
3D Slicer as an image computing platform for the Quantitative Imaging Network
.
Magn Reson Imaging
2012
;
30
:
1323
41
.
18.
van Griethuysen
JJM
,
Fedorov
A
,
Parmar
C
,
Hosny
A
,
Aucoin
N
,
Narayan
V
, et al
Computational radiomics system to decode the radiographic phenotype
.
Cancer Res
2017
;
77
:
e104
e7
.
19.
Ishwaran
H
,
Kogalur
UB
,
Gorodeski
EZ
,
Minn
AJ
,
Lauer
MS
. 
High-dimensional variable selection for survival data
.
J Am Statist Assoc
2010
;
105
:
205
17
.
20.
Ishwaran
H
,
Kogalur
UB
,
Chen
X
,
Minn
AJ
. 
Random survival forests for high‐dimensional data
.
Stat Anal Data Mining
2011
;
4
:
115
32
.
21.
Ishwaran
H
,
Kogalur
UB
,
Blackstone
EH
,
Lauer
MS
. 
Random survival forests
.
Ann Appl Stat
2008
;
2
:
841
60
.
22.
Wright
MN
,
Dankowski
T
,
Ziegler
A
. 
Unbiased split variable selection for random survival forests using maximally selected rank statistics
.
Stat Med
2017
;
36
:
1272
84
.
23.
Leger
S
,
Zwanenburg
A
,
Pilz
K
,
Lohaus
F
,
Linge
A
,
Zophel
K
, et al
A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling
.
Sci Rep
2017
;
7
:
13206
.
24.
Harrell
FE
 Jr
,
Lee
KL
,
Mark
DB
. 
Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors
.
Stat Med
1996
;
15
:
361
87
.
25.
Efron
B
,
Tibshirani
R
.
An introduction to the bootstrap
.
New York
:
Chapman & Hall
; 
1993
.
xvi
,
436
pp.
26.
Landis
JR
,
Koch
GG
. 
The measurement of observer agreement for categorical data
.
Biometrics
1977
;
33
:
159
74
.
27.
Hayashi
M
,
Shimizu
T
,
Hirokawa
F
,
Inoue
Y
,
Komeda
K
,
Asakuma
M
, et al
Clinicopathological risk factors for recurrence within one year after initial hepatectomy for hepatocellular carcinoma
.
Am Surg
2011
;
77
:
572
8
.
28.
Shah
SA
,
Greig
PD
,
Gallinger
S
,
Cattral
MS
,
Dixon
E
,
Kim
RD
, et al
Factors associated with early recurrence after resection for hepatocellular carcinoma and outcomes
.
J Am Coll Surg
2006
;
202
:
275
83
.
29.
Imamura
H
,
Matsuyama
Y
,
Tanaka
E
,
Ohkubo
T
,
Hasegawa
K
,
Miyagawa
S
, et al
Risk factors contributing to early and late phase intrahepatic recurrence of hepatocellular carcinoma after hepatectomy
.
J Hepatol
2003
;
38
:
200
7
.
30.
An
C
,
Kim
DW
,
Park
YN
,
Chung
YE
,
Rhee
H
,
Kim
MJ
. 
Single hepatocellular carcinoma: preoperative MR imaging to predict early recurrence after curative resection
.
Radiology
2015
;
276
:
433
43
.
31.
Lee
S
,
Kim
SH
,
Lee
JE
,
Sinn
DH
,
Park
CK
. 
Preoperative gadoxetic acid-enhanced MRI for predicting microvascular invasion in patients with single hepatocellular carcinoma
.
J Hepatol
2017
;
67
:
526
34
.
32.
Huang
Y
,
Liu
Z
,
He
L
,
Chen
X
,
Pan
D
,
Ma
Z
, et al
Radiomics signature: a potential biomarker for the prediction of disease-free survival in early-stage (I or II) non-small cell lung cancer
.
Radiology
2016
;
281
:
947
57
.
33.
Poon
RT
. 
Differentiating early and late recurrences after resection of HCC in cirrhotic patients: implications on surveillance, prevention, and treatment strategies
.
Ann Surg Oncol
2009
;
16
:
792
.
34.
Wu
J
,
Aguilera
T
,
Shultz
D
,
Gudur
M
,
Rubin
DL
,
Loo
BW
 Jr
, et al
Early-stage non-small cell lung cancer: quantitative imaging characteristics of (18)F fluorodeoxyglucose PET/CT allow prediction of distant metastasis
.
Radiology
2016
;
281
:
270
8
.
35.
Sasaki
A
,
Kai
S
,
Iwashita
Y
,
Hirano
S
,
Ohta
M
,
Kitano
S
. 
Microsatellite distribution and indication for locoregional therapy in small hepatocellular carcinoma
.
Cancer
2005
;
103
:
299
306
.
36.
Pawlik
TM
,
Delman
KA
,
Vauthey
JN
,
Nagorney
DM
,
Ng
IO
,
Ikai
I
, et al
Tumor size predicts vascular invasion and histologic grade: Implications for selection of surgical treatment for hepatocellular carcinoma
.
Liver Transpl
2005
;
11
:
1086
92
.
37.
Shi
M
,
Zhang
CQ
,
Zhang
YQ
,
Liang
XM
,
Li
JQ
. 
Micrometastases of solitary hepatocellular carcinoma and appropriate resection margin
.
World J Surg
2004
;
28
:
376
81
.
38.
Kim
H
,
Park
MS
,
Choi
JY
,
Park
YN
,
Kim
MJ
,
Kim
KS
, et al
Can microvessel invasion of hepatocellular carcinoma be predicted by pre-operative MRI?
Eur Radiol
2009
;
19
:
1744
51
.
39.
Kim
KA
,
Kim
MJ
,
Jeon
HM
,
Kim
KS
,
Choi
JS
,
Ahn
SH
, et al
Prediction of microvascular invasion of hepatocellular carcinoma: usefulness of peritumoral hypointensity seen on gadoxetate disodium-enhanced hepatobiliary phase images
.
J Magn Reson Imaging
2012
;
35
:
629
34
.
40.
Ariizumi
S
,
Kitagawa
K
,
Kotera
Y
,
Takahashi
Y
,
Katagiri
S
,
Kuwatsuru
R
, et al
A non-smooth tumor margin in the hepatobiliary phase of gadoxetic acid disodium (Gd-EOB-DTPA)-enhanced magnetic resonance imaging predicts microscopic portal vein invasion, intrahepatic metastasis, and early recurrence after hepatectomy in patients with hepatocellular carcinoma
.
J Hepatobiliary Pancreat Sci
2011
;
18
:
575
85
.
41.
Kawamura
Y
,
Ikeda
K
,
Seko
Y
,
Hosaka
T
,
Kobayashi
M
,
Saitoh
S
, et al
Heterogeneous type 4 enhancement of hepatocellular carcinoma on dynamic CT is associated with tumor recurrence after radiofrequency ablation
.
AJR Am J Roentgenol
2011
;
197
:
W665
73
.
42.
Lambregts
DMJ
,
Beets
GL
,
Maas
M
,
Curvo-Semedo
L
,
Kessels
AGH
,
Thywissen
T
, et al
Tumour ADC measurements in rectal cancer: effect of ROI methods on ADC values and interobserver variability
.
Eur Radiol
2011
;
21
:
2567
74
.
43.
Kitao
A
,
Matsui
O
,
Yoneda
N
,
Kozaka
K
,
Kobayashi
S
,
Koda
W
, et al
Hypervascular hepatocellular carcinoma: correlation between biologic features and signal intensity on gadoxetic acid-enhanced MR images
.
Radiology
2012
;
265
:
780
9
.
44.
Choi
JW
,
Lee
JM
,
Kim
SJ
,
Yoon
JH
,
Baek
JH
,
Han
JK
, et al
Hepatocellular carcinoma: imaging patterns on gadoxetic acid-enhanced MR images and their value as an imaging biomarker
.
Radiology
2013
;
267
:
776
86
.
45.
Shafiq-Ul-Hassan
M
,
Zhang
GG
,
Latifi
K
,
Ullah
G
,
Hunt
DC
,
Balagurunathan
Y
, et al
Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels
.
Med Phys
2017
;
44
:
1050
62
.
46.
Shafiq-Ul-Hassan
M
,
Latifi
K
,
Zhang
G
,
Ullah
G
,
Gillies
R
,
Moros
E
. 
Voxel size and gray level normalization of CT radiomic features in lung cancer
.
Sci Rep
2018
;
8
:
10545
.
47.
Larue
R
,
van Timmeren
JE
,
de Jong
EEC
,
Feliciani
G
,
Leijenaar
RTH
,
Schreurs
WMJ
, et al
Influence of gray level discretization on radiomic feature stability for different CT scanners, tube currents and slice thicknesses: a comprehensive phantom study
.
Acta Oncol
2017
;
56
:
1544
53
.
48.
Moons
KGM
,
Altman
DG
,
Reitsma
JB
,
Ioannidis
JPA
,
Macaskill
P
,
Steyerberg
EW
, et al
Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration.
Ann Intern Med
2015
;
162
:
W1
W73
.

Supplementary data