Although the sphingolipid ceramide exhibits potent tumor suppressor effects, efforts to harness this have been hampered by poor solubility, uptake, bioavailability, and metabolic conversion. Therefore, identification of avenues to improve efficacy is necessary for development of ceramide-based therapies. In this study, we used mutant p53, triple-negative breast cancer (TNBC) cells, a type of breast cancer highly refractory to treatment, and cell-permeable nanoliposomal C6-ceramide in conjunction with the antiestrogen tamoxifen, which has been shown to be an effective modulator of ceramide metabolism. We show for the first time that nanoliposomal tamoxifen enhances nanoliposomal C6-ceramide cytotoxicity in cultured TNBC cells, a response that was accompanied by induction of cell-cycle arrest at G1 and G2, caspase-dependent induction of DNA fragmentation, and enhanced mitochondrial and lysosomal membrane permeability at 18 and 2 hours, respectively. Tamoxifen metabolites were also effective. Only tamoxifen promoted lysosomal membrane permeability. In addition, we show for the first time that tamoxifen inhibits acid ceramidase, as measured in intact cell assays; this effect was irreversible. Together, our findings show that tamoxifen magnifies the antiproliferative effects of C6-ceramide via combined targeting of cell-cycle traverse and lysosomal and mitochondrial integrity. We adduce that C6-ceramide–induced apoptosis is amplified by tamoxifen's impact on lysosomes and perhaps accompanying inhibition of acid ceramidase, which could result in decreased levels of sphingosine 1-phosphate. This drug regimen could serve as a promising therapy for chemoresistant and triple-negative types of breast cancer, and thus represents an indication for tamoxifen, irrespective of estrogen receptor status. Mol Cancer Ther; 11(11); 2352–61. ©2012 AACR.

Triple-negative breast cancer (TNBC), a difficult to treat problematic subtype of breast cancer, accounts for approximately 15% of all breast cancers and disproportionally affects African American and Hispanic women. Despite improvements in the clinical outcome of patients with breast cancer through use of endocrine-targeted agents and cytotoxic chemotherapies, overcoming drug resistance and the development of rational therapies for TNBC remain menacing therapeutic hurdles. The goal of the present work was to identify and evaluate new treatment combinations for TNBC, irrespective of estrogen and progesterone receptor status, HER2/neu expression, and the multidrug resistant phenotype. We have addressed this issue by formulating and evaluating novel combinatorial therapies consisting of short-chain ceramide-containing (C6-ceramide) nanoliposomes and adjuvants that are capable of regulating ceramide metabolism while affecting other independent targets as well, and thus propose this approach, which could restore breast tumor cell death programs that have been overridden by the cancer process.

Ceramide, a potent tumor suppressor lipid, can be generated in situ or administered exogenously. The sphingolipid-metabolizing machinery of cancer cells can work however to dampen ceramide's tumor-censoring effects. For example, conversion of ceramide to glucosylceramide by glucosylceramide synthase (GCS) is a main anabolic route used by cancer cells to neutralize ceramide downstream death signals (1–3). Ceramide cytotoxicity can also be limited by sphingomyelin synthase and ceramide kinase (CerK; refs. 4, 5). Ceramide hydrolysis via ceramidase is also an effective means to eliminate ceramide; however, the sphingosine generated can be phosphorylated by sphingosine kinase to yield sphingosine 1-phosphate (S1-P), a mitogenic sphingolipid with its own important place in cancer biology (6, 7). Maintaining balance between ceramide and S1-P has been viewed as paramount in maintaining ceramide's tumor suppressor properties, and to this end, a number of pharmalogic and molecular approaches have been investigated to enhance ceramide's inhibition of tumor cell proliferation (8–13). Apropos here is a recent study by van Vlerken and colleagues (14) that shows, through polymer-blend nanoparticles, that combination therapy with exogenous C6-ceramide or tamoxifen, used as a GCS inhibitor with paclitaxel, was more effective than single agent paclitaxel.

In the present study, we assessed a nanoliposomal formulation of C6-ceramide, which has shown enhanced activity in in vitro and in vivo models (15–18), and paired it with tamoxifen, to determine whether the antiproliferative effects of ceramide could be amplified. Although tamoxifen is an antiestrogen used for treatment of estrogen receptor–positive types of breast cancer, this drug has a number of estrogen receptor–independent actions, including circumvention of multidrug resistance (19), inhibition of ceramide glycosylation (20), and downregulation of survivin (21). In addition, we now show that tamoxifen inhibits acid ceramidase, an enzyme indispensible for cellular ceramide degradation. For the present study, we developed a stable formulation of tamoxifen nanoliposomes containing 30 mol% tamoxifen and evaluated the impact of C6-ceramide–tamoxifen combinatorial regimens in TNBC cells. This combination was synergistic for reduction of cell viability, and at 24 hours the combination induced cell-cycle arrest at G1 and G2, independent of retinoblastoma protein (Rb) expression. One function of Rb protein is to prevent excessive cell growth by inhibiting cell cycle, thus this protein can function as a tumor suppressor. Testing upstream revealed that enhanced mitochondrial and lysosomal membrane permeability were important elements of the apoptotic cascade.

Cell lines and reagents

Human breast cancer cell lines, MDA-MB-468, MDA-MB-231, and Hs578T, were obtained from the American Type Culture Collection (ATCC). The cell lines were expanded and cryopreserved in liquid nitrogen in the investigators laboratory. The cell lines were not tested or authenticated over and above documentation provided by the ATCC, which includes antigen expression, DNA profile, and cytogenic analysis. Cells were maintained (∼25 passages) in RPMI-1640 GlutaMax medium (Invitrogen) supplemented with 10% FBS (Atlanta Biologicals) and 100 units/mL each of penicillin and streptomycin plus 0.3 mg/mL l-glutamine. Cells were grown in humidified atmosphere, 95% air, 5% CO2 at 37°C and subcultured at confluence using 0.05% trypsin/0.53 mmol/L EDTA (Invitrogen). DM102 was provided by the Department of Biomedicinal Chemistry, Institute of Advanced Chemistry of Catalonia (Barcelona, Spain).

C6-ceramide (N-hexanoyl-d-erythro-sphingosine) was from Avanti Polar Lipids and it was dissolved in 100% ethanol and stored as a 10 mmol/L stock at −20°C. Tamoxifen–HCl, N-desmethyltamoxifen HCl, and (Z)-4-hydroxytamoxifen were from Sigma Chemical Co. and dissolved and stored as was C6-ceramide. Pan-caspase inhibitor, Z-VAD-FMK, from BD Pharmigen, was solubilized in ethanol and stored as 40 mmol/L stock at −20°C.

Preparation of nanoliposomes

Pegylated nanoliposomes were prepared on the basis of earlier studies (22–24). Liposome formulations were made from specific lipids, at particular molar ratios (shown later), before nano-sizing. We used 1,2-distearoyl-sn-glycero-3-phosphocholine, 1,2-dioleoyl-sn-glycero-3-phosphoethanolamine, 1,2-distearoyl-sn-glycero-3-phosphoethanolamine-N-[methoxy(polyethylene glycol)-2000] (PEG(2000)-DSPE), C6-ceramide, C8-ceramide-1-succinyl[methoxy(polyethylene glycol)-750] (PEG(750)-C8), and/or tamoxifen. Briefly, lipids dissolved in chloroform, dried to a film under a stream of nitrogen, and then hydrated by addition of 0.9% NaCl. Solutions were sealed, heated at 55°C (60 minutes), and subjected to vortex mixing and sonicated for 5 minutes until light no longer diffracted through the suspension. The lipid vesicle-containing solution was quickly extruded at 55°C by passing the solution 10 times through 100 nm polycarbonate filters in an Avanti Mini-Extruder (Avanti Polar Lipids). Nanoliposomal size for triplicate replicates of each formulation, and a neutral ζ-potential were validated using a Malvern Zetasizer Nano ZS at 25°C. Nanoliposome solutions were stored at room temperature until use. All formulations were designed to deliver equal masses of total lipids. Ghost nanoliposomes were prepared from the same components excluding C6-ceramide.

Cell viability

Viability was assessed by the Cell Titer 96 AQueous One solution cell proliferation assay (Promega). Cells were seeded (3,000 cells/well) in 96-well plates in medium containing 5% FBS and the following day treated with the indicated agents using 1% FBS medium (final FBS concentration, ∼2.5%) for 72 to 96 hours. Viability was measured as mean (n = 6) absorbance (minus ghost and/or ethanol vehicle) at 490 nm using a microplate reader FL600, Bio-Tek.

Cell-cycle and apoptosis analysis

Progression of cells through cell cycle and DNA fragmentation were examined using flow cytometry. DNA fragmentation was measured according to published protocol (25), with modification. MDA-MB-468 and MDA-MB-231 cells were seeded in 6-cm dishes at 200,000 cells per dish in 10% FBS medium. The following day, the medium was removed and cells were treated with indicated agents in medium containing 2.5% FBS for 24 hours. Cells were collected by trypsinization then washed with cold PBS, and fixed in 70% ethanol. DNA was stained for 4 hours in the dark with 0.5 mL hypotonic propidium iodide (PI) buffer (0.1% sodium citrate, 50 μg/mL DNase-free RNase A, and 0.1% Triton X-100). DNA content was analyzed using a FACScan, and cell-cycle analysis, and subG0 (apoptosis marker) was conducted using FCS Express 4 (De Novo Software).

Acid ceramidase

Acid ceramidase was measured in intact cells and in cell lysates by fluorgenic assay as described (26). For intact cell assay, MDA-MB 468 cells (10,000 cells/well) were seeded into 96-well plates in 10% FBS medium. After 24 hours, medium was removed and replaced with 5% FBS medium. Cells were preincubated at 37°C for 1 hour with either the acid ceramidase inhibitor DM102, [2R, 3Z]-N–(1-hydroxyoctadec-3-en-2-yl)pivaloylamine (26) or tamoxifen (ethanol vehicles), after which, fluoregenic substrate (ethanol vehicle) was added to a final concentration of 16 μmol/L, and the cells were incubated (125 μL final well volume) in a tissue culture incubator (37°C) for either 3 or 23 additional hours. At this point, the assay can either be continued to completion or the 96-well plates can be placed at −20°C and assayed the following day. To complete the assay, 50 μL methanol and 100 μL NaIO4 (2.5 mg/mL) in 0.1 mol/L glycine buffer, pH 10.6, was added and the plates were placed in the dark at room temperature for 2 hours. Fluorescence was measured in the UV range (365 nm excitation/410–460 nm emission) using a GloMax multi-detection system (Promega). To measure acid ceramidase in cell lysates, cultured cells were pretreated in the absence and presence of tamoxifen (5 and 10 μmol/L) for 4 hours. Cells were then washed, harvested, pelleted, briefly sonicated in 0.25 mol/L sucrose, and centrifuged to remove debris. Cell-free assays (37 μg protein) contained sodium acetate–acetic acid buffer, pH 4.5 (25 mmol/L), 40 μmol/L substrate and were incubated at 37°C for 3 hours (26). Fluorescence was then measured as earlier. Similar assays were conducted using cell lysates derived from tamoxifen-naïve cultures to which either DM102 or tamoxifen was then added.

Lysosomal stability and mitochondrial membrane potential

Cells were assessed for lysosomal stability using the acridine orange relocation methods (27, 28). Acridine orange is a metachromatic fluorochrome and a weak base that exhibits red fluorescence when highly concentrated in acidic lysosomes and green fluorescence when outside the lysosomes. Lysosomal integrity was evaluated by assessing red fluorescence (FL3) by flow cytometry. Briefly, cells were stained with acridine orange (5 μg/mL) for 15 minutes in RPMI-1640 complete media followed by PBS washing and treatment with tamoxifen for 2 hours. After treatment, cells were trypsinized, washed with PBS, and placed on ice for flow cytometry. The shift of red fluorescence indicates lysosomal permeabilization.

Mitochondrial membrane potential (MMP; Δψm) was measured using JC-1 (Cell Technology). Cells were plated as in the apoptosis experiments and treated the following day with agents for 18 hours. JC-1 is a ratiometric dye that exists as a monomer in the cytosol (green) and also accumulates as aggregates in the mitochondria, which stain red. Quantitative analysis of Δψm was detected by JC-1 at FL-1 (green) and FL-2 (red) using flow cytometry.

Statistical analysis

The results are expressed as means ± SE and were analyzed by ANOVA. Differences among the treatment groups were assessed by Tukey post hoc test. Differences were considered significant at P < 0.05.

Unless otherwise stated, all experiments were carried out using nanoliposomal formulations. The first experiments were carried out to assess the effects of combinatorial C6-ceramide and tamoxifen on cell viability. As shown in Fig. 1, these combinations were more effective than single agents in reducing viability in all cell lines. In some instances, the effects were supradditive. For example, in Hs578T cells (Fig. 1A), while tamoxifen did not affect viability, and C6-ceramide reduced viability to approximately 60% of control, concomitant exposure elicited amplified antiproliferative responses (viability < 20%). Similarly, in MDA-MB-231 cells (Fig. 1B), C6-ceramide, tamoxifen, and the combination reduced viability to 70%, 90%, and 40% of control, respectively, with like results in MDA-MB-468 cells (Fig. 1C). Interestingly, N-desmethyltamoxifen, the primary metabolite of tamoxifen, and 4-hydroxytamoxifen, a metabolite with more potent antiestrogenic activity than that of the parent drug, were also effective when coadministered with C6-ceramide (Fig. 1D), albeit N-desmethyltamoxifen was the most effective. We also compared the activity of free ceramide (ethanol vehicle) with nanoliposomal C6-ceramide and found that at a dose of 2.0 μmol/L, MDA-MD-468 cell viability was 72% ± 6.27% and 2% ± 10.28% of control after 96 hours exposure to the respective ceramide forms. These findings are similar with observations using free and nanoformulated C6-ceramide in a murine mammary adenocarcinoma cell line model (29).

Figure 1.

Effect of tamoxifen on C6-ceramide cytotoxicity in TNBC. A–D, cells were seeded in 96-well plates (3,000 cells/well) in RPMI-1640 medium containing 10% FBS and cultured at 37°C. The next day, growth medium was replaced with 2.5% FBS medium, agents were added, and cells were incubated for 96 hours. D. MDA-MB-468 cells treated with N-desmethyltamoxifen (dme-tam) and 4-hydroxytamoxifen (4-OH tam). Viability was evaluated by MTS assay. Concentrations, μmol/L. Data are the mean (n = 6) ± SE, relative to vehicle controls and/or the ghost. Experiments were repeated and yielded similar results.

Figure 1.

Effect of tamoxifen on C6-ceramide cytotoxicity in TNBC. A–D, cells were seeded in 96-well plates (3,000 cells/well) in RPMI-1640 medium containing 10% FBS and cultured at 37°C. The next day, growth medium was replaced with 2.5% FBS medium, agents were added, and cells were incubated for 96 hours. D. MDA-MB-468 cells treated with N-desmethyltamoxifen (dme-tam) and 4-hydroxytamoxifen (4-OH tam). Viability was evaluated by MTS assay. Concentrations, μmol/L. Data are the mean (n = 6) ± SE, relative to vehicle controls and/or the ghost. Experiments were repeated and yielded similar results.

Close modal

Because of the strong antiproliferative responses shown with C6-ceramide–tamoxifen combinations, we next sought to determine whether treatment impacted cell cycle. In MDA-MB-468 cells, which do not express Rb protein, the drug duo promoted cell-cycle arrest at G1 and G2 (Fig. 2A). Similar results were obtained in MDA-MB-231 cells (Fig. 2B), which express Rb-protein. With single agents, the data show that C6-ceramide principally influenced G1 as did tamoxifen, only although slightly; the combination also increased cells in G2 (Fig. 2B), indicating that the combination induces cell-cycle arrest independent of Rb expression.

Figure 2.

Effect of tamoxifen and C6-ceramide on cell-cycle progression in TNBC. Cells were treated with C6-ceramide, tamoxifen, and the mix for 24 hours, collected and fixed in ethanol, stained with PI, and analyzed by flow cytometry as detailed in Materials and Methods. The distribution of cells in the different phases of the cell cycle was analyzed using FCS express software. Data are mean ± SE from 3 independent experiments. Concentrations, μmol/L.

Figure 2.

Effect of tamoxifen and C6-ceramide on cell-cycle progression in TNBC. Cells were treated with C6-ceramide, tamoxifen, and the mix for 24 hours, collected and fixed in ethanol, stained with PI, and analyzed by flow cytometry as detailed in Materials and Methods. The distribution of cells in the different phases of the cell cycle was analyzed using FCS express software. Data are mean ± SE from 3 independent experiments. Concentrations, μmol/L.

Close modal

We next investigated the influence of single agents and combinatorial regimens on the type of cell death. It is well known that ceramide produces many of its tumor suppressor effects through the induction of apoptosis. Using DNA fragmentation as a gauge, the data in Fig. 3A show that C6-ceramide, as opposed to tamoxifen, was the major player in eliciting apoptosis in MDA-MB-468 cells; however, concurrent administration produced a response that was supra-additive. For example, while 5.0 μmol/L tamoxifen had no significant effect, and 2.5 μmol/L C6-ceramide increased DNA fragmentation to 13% (7% over control), the combination yielded 29% fragmentation (23% over control). A similar trend was evidenced in MDA-MB-231 cells (Fig. 3B). Apoptosis was caspase-dependent as shown by inhibition of DNA fragmentation in MDA-MB-468 cells exposed to C6-ceramide–tamoxifen in the presence of a pan-caspase inhibitor (Fig. 3A). N-desmethyltamoxifen also effectively enhanced DNA fragmentation when coadministered with C6-ceramide (Fig. 3C).

Figure 3.

Influence of C6-ceramide, tamoxifen, and caspase inhibition on apoptosis in TNBC. A, MDA-MB-468 cells. B, MDA-MB-231 cells. C, MDA-MB-468 cells with N-desmethyltamoxifen. Cells were treated with C6-ceramide, tamoxifen, N-desmethyltamoxifen, or the combination for 24 hours. Cells were then trypsinized, fixed in ethanol, stained with PI, and analyzed for DNA fragmentation by flow cytometry. SubG0 (apoptosis marker) was analyzed using FCS express software. Data are means ± SE from 3 independent experiments. Concentrations (μmol/L) noted in brackets. CaspI (50 μmol/L), caspase inhibitor. *, P < 0.05.

Figure 3.

Influence of C6-ceramide, tamoxifen, and caspase inhibition on apoptosis in TNBC. A, MDA-MB-468 cells. B, MDA-MB-231 cells. C, MDA-MB-468 cells with N-desmethyltamoxifen. Cells were treated with C6-ceramide, tamoxifen, N-desmethyltamoxifen, or the combination for 24 hours. Cells were then trypsinized, fixed in ethanol, stained with PI, and analyzed for DNA fragmentation by flow cytometry. SubG0 (apoptosis marker) was analyzed using FCS express software. Data are means ± SE from 3 independent experiments. Concentrations (μmol/L) noted in brackets. CaspI (50 μmol/L), caspase inhibitor. *, P < 0.05.

Close modal

Mitochondria are the orchestrators of the intrinsic pathway of apoptosis, and ceramide can elicit this process via mitochondrial-involved avenues (30, 31). One way that ceramide impacts mitochondria is by increasing outer membrane permeabilization through induction of mitochondrial transmembrane depolarization (Δψm); this promotes leakage of apoptotic proteins. To determine whether the C6-ceramide–tamoxifen regimen targeted the mitochondrial pathway, we measured Δψm in response to treatment. The data in Fig. 4A show a robust increase in Δψm in MDA-MB-468 cells in response to combination C6-ceramide–tamoxifen and C6-ceramide–N-desmethyltamoxifen (Fig. 4B). It is noteworthy that Δψm was evident as early as 18 hours and was promoted using concentrations of C6-ceramide and tamoxifen similar to those that affected decreased viability at 96 hours (see Fig. 1C). Evaluating the effects of single agents on Δψm revealed that C6-ceramide was the predominant player in eliciting altered mitochondrial membrane integrity (Fig. 4A). Although Δψm constitutes a major checkpoint in apoptotic cell death, lysosomal membrane permeabilization (LMP) has also been shown, in certain circumstances, to trigger cell death. LMP causes release of proteases into the cytosol with deleterious consequences. The experiment in Fig. 4C shows that tamoxifen affected enhanced LMP in MDA-MB-468 cells after only a 2 hours exposure (quantification data on right); the same was not produced upon exposure to C6-ceramide. Thus, LMP seems to be an early event contributing to cell death that is chiefly regulated by tamoxifen.

Figure 4.

Effect of C6-ceramide and tamoxifen on mitochondrial and lysosomal integrity. A, effect of C6-ceramide on mitochondrial depolarization. B, effect of C6-ceramide and N-desmethyltamoxifen on mitochondrial depolarization. MDA-MB-468 cells were treated with C6-ceramide, tamoxifen, N-desmethyltamoxifen, or the combination for 18 hours. Cells were then stained with JC-1 for 15 minutes in phenol red-free RPMI, trypsinized, washed with PBS, and placed on ice until quantitation by fluorescence-activated cell sorting. Quantitative analysis of Δψm was detected by JC-1 at FL-1 and FL-2. Data are mean from 3 to 4 independent experiments. C, effect of C6-ceramide and tamoxifen on lysosomal permeabilization. MDA-MB-468 cultures were stained with acridine orange for 15 minutes followed by PBS washing and treated with tamoxifen for 2 hours. After treatment, cells were trypsinized, washed with PBS, and placed on ice until fluorescence-activated cell sorting analysis. A shift in red fluorescence (FL-3) indicated lysosomal peremeabilization. Data were analyzed using FCS express software. Data are means ± SE from at least 3 independent experiments. *, P < 0.05.

Figure 4.

Effect of C6-ceramide and tamoxifen on mitochondrial and lysosomal integrity. A, effect of C6-ceramide on mitochondrial depolarization. B, effect of C6-ceramide and N-desmethyltamoxifen on mitochondrial depolarization. MDA-MB-468 cells were treated with C6-ceramide, tamoxifen, N-desmethyltamoxifen, or the combination for 18 hours. Cells were then stained with JC-1 for 15 minutes in phenol red-free RPMI, trypsinized, washed with PBS, and placed on ice until quantitation by fluorescence-activated cell sorting. Quantitative analysis of Δψm was detected by JC-1 at FL-1 and FL-2. Data are mean from 3 to 4 independent experiments. C, effect of C6-ceramide and tamoxifen on lysosomal permeabilization. MDA-MB-468 cultures were stained with acridine orange for 15 minutes followed by PBS washing and treated with tamoxifen for 2 hours. After treatment, cells were trypsinized, washed with PBS, and placed on ice until fluorescence-activated cell sorting analysis. A shift in red fluorescence (FL-3) indicated lysosomal peremeabilization. Data were analyzed using FCS express software. Data are means ± SE from at least 3 independent experiments. *, P < 0.05.

Close modal

The structure of tamoxifen is similar to other amphiphilic agents that have been shown to inhibit acid ceramidase (32), the lysosomal enzyme indispensable for cellular ceramide degradation. We were therefore interested in determining whether tamoxifen influenced acid ceramidase activity. Here, we show for the first time that tamoxifen inhibits acid ceramidase activity as assessed in intact cells. Inhibition was dose-dependent (Table 1, 4 hours) and N-desmethyltamoxifen and 4-hydroxytamoxifen were as effective as tamoxifen (Table 1, 24 hours). Free tamoxifen (ethanol vehicle) and tamoxifen nanoliposomes were equally effective. As negative control, we assessed triphenylbutene, which is the tamoxifen nucleus devoid of the dimethylaminoethanol function; it did not inhibit acid ceramidase activity in MDA-BM-468 cells when tested at concentrations of 5 to 20 μmol/L (10 μmol/L shown). Two other tricyclic cationic amphiphilic agents with a nitrogen-containing side group similar to tamoxifen, chloropromazine, and desipramine, inhibited acid ceramidase activity in intact MDA-MB-468 cells by 66% and 42%, respectively (at 10 μmol/L; data not shown). The action of the cysteine protease, cathepsin B, has been shown to exert a role in inhibition of acid ceramidase (32). To determine whether acid ceramidase inhibition by tamoxifen could be linked to degradation of acid ceramidase protein by lysosomal cathepsin, we used CA074ME, a cathepsin B/L inhibitor; however, when introduced over a concentration range of 5 to 20 μmol/L (1 hour preincubation), CA074ME failed to reverse the inhibitory effects of tamoxifen on acid ceramidase. These data indicate that inhibition of acid ceramidase by tamoxifen is not related to cathepsin degradation of the enzyme. Finally, we determined the effect of tamoxifen on acid ceramidase activity using MDA-MB-468 cell lysates. For this, 2 types of experiments were conducted: (i) preexposure of intact cells to tamoxifen followed by cell disruption and assay, and (ii) cell disruption and addition of tamoxifen or DM102 to the cell-free assay. As shown in Fig. 5A, preincubation of intact cells with tamoxifen resulted in inhibition of acid ceramidase activity as assayed in the resultant lysates. On the other hand, tamoxifen did not inhibit acid ceramidase activity when added directly to the cell lysates (Fig. 5B), showing that the mechanism underlying enzyme inhibition requires intact cells. DM102, an acid ceramidase substrate mimic, inhibited cell-free acid ceramidase activity.

Figure 5.

Effect of tamoxifen and DM102 on acid ceramidase (AC) activity in MDA-MB-468 cell lysates. A, cell-free assay. Cells were preincubated with tamoxifen for 4 hours, washed, lysed, and assayed for acid ceramidase activity as described in Materials and Methods. B, cell-free assays with tamoxifen (20 μmol/L) and DM102 (20 μmol/L) added after lysis. Data are means ± SE, n = 3 from 2 independent experiments.

Figure 5.

Effect of tamoxifen and DM102 on acid ceramidase (AC) activity in MDA-MB-468 cell lysates. A, cell-free assay. Cells were preincubated with tamoxifen for 4 hours, washed, lysed, and assayed for acid ceramidase activity as described in Materials and Methods. B, cell-free assays with tamoxifen (20 μmol/L) and DM102 (20 μmol/L) added after lysis. Data are means ± SE, n = 3 from 2 independent experiments.

Close modal
Table 1.

The effect of tamoxifen, tamoxifen metabolites, and other agents on acid ceramidase activity in intact MDA-MB-468 cells

4-hour incubation24-hour incubationa
ConditionsMean FUActivity (%)ConditionsMean FUActivity (%)
Control 7367 100 Control 16958 100 
tam 5 μmol/L 5367 73 tam (EtOH) 3891 23 
10 3167 43 tam (nano) 4458 26 
20 2067 28 desMe-tam 3925 23 
   4-OH-tam 3891 23 
   Triphenylbutene 17733 105 
   NH4Cl [20 μmol/L] 15262 90 
   pre-tam [20 μmol/L]b 6892 41 
4-hour incubation24-hour incubationa
ConditionsMean FUActivity (%)ConditionsMean FUActivity (%)
Control 7367 100 Control 16958 100 
tam 5 μmol/L 5367 73 tam (EtOH) 3891 23 
10 3167 43 tam (nano) 4458 26 
20 2067 28 desMe-tam 3925 23 
   4-OH-tam 3891 23 
   Triphenylbutene 17733 105 
   NH4Cl [20 μmol/L] 15262 90 
   pre-tam [20 μmol/L]b 6892 41 

Abbreviations: EtOH, ethanol vehicle; FU, fluorescence units; 4-OH-tam, 4-hydroxytamoxifen; tam (nano), nanoliposomal tamoxifen.

aTamoxifen and metabolites were tested at 10 μmol/L, except where noted.

bCells were exposed (pre-tam) to tamoxifen for 3 hours, rinsed, and reincubated in tamoxifen-free medium (10% FBS) for 24 hours before acid ceramidase activity was determined. Each data point is the mean of triplicate cultures. Repeated experiments yielded similar results. Error was within ±6% of the mean.

This promise of ceramide-based therapeutics has largely been boosted by the use of short-chain ceramides, which are more hydrophilic, and by the use of nanoliposomal formulations, which show enhanced bioavailability (15, 17, 33), few side effects (17), and efficacy via antiproliferative and proapoptotic properties (15, 18, 29). In some instances, nanoliposomal ceramides show enhanced activity when coadministered with other anticancer agents, such as sorafenib or gemcitabine, or the GCS inhibitor d-threo­-1-phenyl-2-decanoylamino-3-morpholino-1-propanol (PDMP; refs. 16, 22). The discovery that tamoxifen inhibits ceramide metabolism at the step of glycosylation (20) suggests that this antiestrogen could have estrogen receptor–independent use in the extant of ceramide therapies. A recent study from our group, using ethanol vehicle, showed the value of combination C6-ceramide–tamoxifen for limiting proliferation in several human cancer cell lines; however, neither the mechanism nor the targets were defined (34). In the present study, we have focused on breast cancer, and specifically, because we have used tamoxifen, on TNBC, which is devoid of estrogen receptor and thus not amiable to antiestrogen therapy.

The present work shows that a C6-ceramide–tamoxifen regimen produced cell-cycle arrest in G1 and G2, phases in which growth and preparation for mitosis occur. This regimen was effective in Rb-expressing and in Rb-nonexpressing breast cancer cells. This is important because Rb, which maintains cell proliferative balance, is functionally inactivated, mutated, or lost in most human neoplasms. From data accrued, it seems that both agents contribute (see Fig. 2). Ceramide and tamoxifen have been shown to induce cell-cycle arrest (35–37). We have also established that cell death is by caspase-dependent apoptosis, whereas C6-ceramide was the major single agent player, the addition of tamoxifen provided enhanced effects (see Fig. 3A). Interestingly, in mammary tumor studies, Mandlekar and colleagues (38) showed that N-desmethyltamoxifen as well as tamoxifen and 4-hydroxytamoxifen, induced caspase-dependent apoptosis, and in a comparative study in multidrug resistant human leukemia cells, both tamoxifen and N-desmethyltamoxifen functioned equally in restoring daunorubicin sensitivity. Tamoxifen is also a P-glycoprotein antagonist (39), and in this capacity functions to retard ceramide glycosylation by virtue of its competence to block sphingolipid trafficking at the Golgi (40, 41).

With regard to subcellular targets, we documented a clear impact on mitochondria, as gauged by an increase in Δψm in response to C6-ceramide–tamoxifen. An interesting aspect here, as seen in the viability studies, was that N­-desmethyltamoxifen was as effective as tamoxifen when coadministered with C6-ceramide. This has clinical relevance as N­-desmethyltamoxifen is the major in vivo tamoxifen metabolite in humans. Short-chain ceramides induce cytochrome C release from mitochondria (42), augment curcumin-induced cell death in melanoma via mitochondrial apoptosis (43), and induce cell death through induction of hari-kari gene (HRK)–mediated mitochondrial dysfunction (44). Our studies with nanoliposomal ceramide also show that mitochondrial targeting perpetuates cell death.

Lysosomal destabilization was the earliest upstream event that we detected. Using LMP as a measure, lysosomal destabilization was seen at 2 hours and was initiated by tamoxifen, dose dependently. C6-ceramide at 2 hours had no effect. LMP occurs in response to a myriad of stimuli, and this process is a factor in the regulation of apoptosis (45) through release of lysosomal constituents, such as destructive cathepsins. For this reason, lysosomes are promising therapeutic targets in cancer. Because tamoxifen induced LMP, we were interested in learning whether tamoxifen affected acid ceramidase activity, as acid ceramidase is localized in the lysosomal lumen, and acid ceramidase is another important target in cancer (46–49). Herein, we show that tamoxifen as well as tamoxifen metabolites inhibited acid ceramidase activity in intact MDA-MB-468 cells at concentrations that induced LMP. Tamoxifen did not inhibit acid ceramidase when added directly to the cell lysate, whereas cell preincubation with tamoxifen was effective in the inhibition of acid ceramidase when tested in the resultant lysate. Tamoxifen is a cationic amphilic drug, as are chlorpromazine and desipramine, which we showed, also inhibited acid ceramidase activity. Along these lines, Elojeimy and colleagues (32) showed that desipramine inhibited acid ceramidase in prostate cancer cells, based on downregulation of acid ceramidase protein, and that CA074ME, a cathepsin B/C inhibitor, blocked the effect on acid ceramidase. In our system, CA074ME failed to block tamoxifen-regulated inhibition of acid ceramidase, suggesting that cathepsin hydrolysis of acid ceramidase protein is not a factor in the mechanism of enzyme inhibition. Furthermore, exposure of cells to NH4Cl, which neutralizes lysosomal pH, did not affect cellular acid ceramidase activity. Although we have not defined the mechanism by which tamoxifen inhibits acid ceramidase, in light of the earlier discussion, we propose that lysosomal destabilization is a factor involved in inhibition of acid ceramidase by tamoxifen, and that this response requires intact cell machinery.

To our knowledge, this is the first report showing that tamoxifen inhibits acid ceramidase activity, an action we suggest is related to LMP. In MCF-7 cells, Hwang and colleagues (50) showed that tamoxifen-induced cell death was accompanied by increased oxidative stress and LMP. We have previously shown that tamoxifen blocks C6-ceramide glycosylation (34), which would, in our case, extend the intracellular residence time of C6-ceramide, and thus amplify cellular responses. That tamoxifen also inhibits acid ceramidase, leads to a number of interesting avenues for pursuit. For the present, we offer the following scenario (Fig. 6). Increasing the levels or residence time of intracellular C6-ceramide by virtue of tamoxifen's inhibitory effect on ceramide metabolism may not be a chief factor in cellular response. We propose that tamoxifen and C6-ceramide each have common as well as distinct targets, and this perpetuates the perfect storm. We suggest that initial targeting of lysosomes by tamoxifen initiates cellular demise. This action can directly engage apoptosis and/or elicit Δψm, which could be driven by lysosomal cathepsins. The overall C6-ceramide mitochondrial response could be magnified by tamoxifen's indirect impact on Δψm through LMP. Thus, apoptosis can be driven by a trio of events, which as shown, enlist lysosomal, mitochondrial, and antimitogenic responses.

Figure 6.

Scheme for C6-ceramide–tamoxifen elicitation of apoptosis. Tamoxifen can block C6-ceramide glycosylation and hydrolysis, the latter mediated via enhanced LMP. LMP can also impact mitochondria as can C6-ceramide, directly. Blocking acid ceramidase (AC) dampens S1-P mitogenic responses and potentiates lysosomal and mitochondrial-driven apoptosis.

Figure 6.

Scheme for C6-ceramide–tamoxifen elicitation of apoptosis. Tamoxifen can block C6-ceramide glycosylation and hydrolysis, the latter mediated via enhanced LMP. LMP can also impact mitochondria as can C6-ceramide, directly. Blocking acid ceramidase (AC) dampens S1-P mitogenic responses and potentiates lysosomal and mitochondrial-driven apoptosis.

Close modal

The Penn State Research Foundation has licensed ceramide nanoliposomes and other nanoliposomal nanotechnology to Keystone Nano Inc. (State College, PA). M. Kester is the Chief Medical Officer of Keystone Nano Inc. and has ownership interest (including patents) in Keystone Nano, Inc. No potential conflicts of interest were disclosed by the other authors.

Conception and design: S.A.F. Morad, M. Kester, M.C. Cabot

Development of methodology: S.A.F. Morad, J. Levin, S.S. Shanmugavelandy, M. Kester, G. Fabrias, M.C. Cabot

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): S.A.F. Morad, J. Levin, M. Kester, G. Fabrias, C. Bedia, M.C. Cabot

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): S.A.F. Morad, J. Levin, C. Bedia, M.C. Cabot

Writing, review, and/or revision of the manuscript: S.A.F. Morad, M. Kester, M.C. Cabot

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): S.A.F. Morad, J. Levin, M.C. Cabot

Study supervision: S.A.F. Morad, M.C. Cabot

This research was supported by the Association for Breast and Prostate Cancer Studies (Los Angeles), the Fashion Footwear Association of New York Charitable Foundation (New York), and National Institute of General Medical Sciences (grant no. GM77391).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Liu
YY
,
Han
TY
,
Giuliano
AE
,
Ichikawa
S
,
Hirabayashi
Y
,
Cabot
MC
. 
Glycosylation of ceramide potentiates cellular resistance to tumor necrosis factor-alpha-induced apoptosis
.
Exp Cell Res
1999
;
252
:
464
70
.
2.
Bleicher
RJ
,
Cabot
MC
. 
Glucosylceramide synthase and apoptosis
.
Biochim Biophys Acta
2002
;
1585
:
172
8
.
3.
Liu
YY
,
Han
TY
,
Giuliano
AE
,
Cabot
MC
. 
Expression of glucosylceramide synthase, converting ceramide to glucosylceramide, confers adriamycin resistance in human breast cancer cells
.
J Biol Chem
1999
;
274
:
1140
6
.
4.
Meng
A
,
Luberto
C
,
Meier
P
,
Bai
A
,
Yang
X
,
Hannun
YA
, et al
Sphingomyelin synthase as a potential target for D609-induced apoptosis in U937 human monocytic leukemia cells
.
Exp Cell Res
2004
;
292
:
385
92
.
5.
Gomez-Munoz
A
,
Gangoiti
P
,
Granado
MH
,
Arana
L
,
Ouro
A
. 
Ceramide-1-phosphate in cell survival and inflammatory signaling
.
Adv Exp Med Biol
2010
;
688
:
118
30
.
6.
Pitson
SM
. 
Regulation of sphingosine kinase and sphingolipid signaling
.
Trends Biochem Sci
2011
;
36
:
97
107
.
7.
Pyne
NJ
,
Pyne
S
. 
Sphingosine 1-phosphate and cancer
.
Nat Rev Cancer
2010
;
10
:
489
503
.
8.
Oskouian
B
,
Saba
JD
. 
Cancer treatment strategies targeting sphingolipid metabolism
.
Adv Exp Med Biol
2010
;
688
:
185
205
.
9.
Graf
C
,
Klumpp
M
,
Habig
M
,
Rovina
P
,
Billich
A
,
Baumruker
T
, et al
Targeting ceramide metabolism with a potent and specific ceramide kinase inhibitor
.
Mol Pharmacol
2008
;
74
:
925
32
.
10.
Senchenkov
A
,
Litvak
DA
,
Cabot
MC
. 
Targeting ceramide metabolism—a strategy for overcoming drug resistance
.
J Natl Cancer Inst
2001
;
93
:
347
57
.
11.
Shida
D
,
Takabe
K
,
Kapitonov
D
,
Milstien
S
,
Spiegel
S
. 
Targeting SphK1 as a new strategy against cancer
.
Curr Drug Targets
2008
;
9
:
662
73
.
12.
Beckham
TH
,
Elojeimy
S
,
Cheng
JC
,
Turner
LS
,
Hoffman
SR
,
Norris
JS
, et al
Targeting sphingolipid metabolism in head and neck cancer: rational therapeutic potentials
.
Expert Opin Ther Targets
2010
;
14
:
529
39
.
13.
Canals
D
,
Perry
DM
,
Jenkins
RW
,
Hannun
YA
. 
Drug targeting of sphingolipid metabolism: sphingomyelinases and ceramidases
.
Br J Pharmacol
2011
;
163
:
694
712
.
14.
van Vlerken
LE
,
Duan
Z
,
Little
SR
,
Seiden
MV
,
Amiji
MM
. 
Augmentation of therapeutic efficacy in drug-resistant tumor models using ceramide coadministration in temporal-controlled polymer-blend nanoparticle delivery systems
.
AAPS J
2010
;
12
:
171
80
.
15.
Stover
T
,
Kester
M
. 
Liposomal delivery enhances short-chain ceramide-induced apoptosis of breast cancer cells
.
J Pharmacol Exp Ther
2003
;
307
:
468
75
.
16.
Tran
MA
,
Smith
CD
,
Kester
M
,
Robertson
GP
. 
Combining nanoliposomal ceramide with sorafenib synergistically inhibits melanoma and breast cancer cell survival to decrease tumor development
.
Clin Cancer Res
2008
;
14
:
3571
81
.
17.
Zolnik
BS
,
Stern
ST
,
Kaiser
JM
,
Heakal
Y
,
Clogston
JD
,
Kester
M
, et al
Rapid distribution of liposomal short-chain ceramide in vitro and in vivo
.
Drug Metab Dispos
2008
;
36
:
1709
15
.
18.
Shabbits
JA
,
Mayer
LD
. 
Intracellular delivery of ceramide lipids via liposomes enhances apoptosis in vitro
.
Biochim Biophys Acta
2003
;
1612
:
98
106
.
19.
Hotta
T
,
Tanimura
H
,
Yamaue
H
,
Iwahashi
M
,
Tani
M
,
Tsunoda
T
, et al
Tamoxifen circumvents the multidrug resistance in fresh human gastrointestinal cancer cells
.
J Surg Res
1996
;
66
:
31
5
.
20.
Lavie
Y
,
Cao
H
,
Volner
A
,
Lucci
A
,
Han
TY
,
Geffen
V
, et al
Agents that reverse multidrug resistance, tamoxifen, verapamil, and cyclosporin A, block glycosphingolipid metabolism by inhibiting ceramide glycosylation in human cancer cells
.
J Biol Chem
1997
;
272
:
1682
7
.
21.
Guo
R
,
Huang
Z
,
Shu
Y
,
Jin
S
,
Ge
H
. 
Tamoxifen inhibits proliferation and induces apoptosis in human hepatocellular carcinoma cell line HepG2 via down-regulation of survivin expression
.
Biomed Pharmacother
2009
;
63
:
375
9
.
22.
Jiang
Y
,
Divittore
NA
,
Kaiser
JM
,
Shanmugavelandy
SS
,
Fritz
JL
,
Heakal
Y
, et al
Combinatorial therapies improve the therapeutic efficacy of nanoliposomal ceramide for pancreatic cancer
.
Cancer Biol Ther
2011
;
12
:
574
85
.
23.
Tagaram
HR
,
Divittore
NA
,
Barth
BM
,
Kaiser
JM
,
Avella
D
,
Kimchi
ET
, et al
Nanoliposomal ceramide prevents in vivo growth of hepatocellular carcinoma
.
Gut
2011
;
60
:
695
701
.
24.
Liu
X
,
Ryland
L
,
Yang
J
,
Liao
A
,
Aliaga
C
,
Watts
R
, et al
Targeting of survivin by nanoliposomal ceramide induces complete remission in a rat model of NK-LGL leukemia
.
Blood
2010
;
116
:
4192
201
.
25.
Riccardi
C
,
Nicoletti
I
. 
Analysis of apoptosis by propidium iodide staining and flow cytometry
.
Nat Protoc
2006
;
1
:
1458
61
.
26.
Bedia
C
,
Casas
J
,
Garcia
V
,
Levade
T
,
Fabrias
G
. 
Synthesis of a novel ceramide analogue and its use in a high-throughput fluorogenic assay for ceramidases
.
Chembiochem
2007
;
8
:
642
8
.
27.
Kirkegaard
T
,
Roth
AG
,
Petersen
NH
,
Mahalka
AK
,
Olsen
OD
,
Moilanen
I
, et al
Hsp70 stabilizes lysosomes and reverts Niemann-Pick disease-associated lysosomal pathology
.
Nature
2010
;
463
:
549
53
.
28.
Boya
P
,
Kroemer
G
. 
Lysosomal membrane permeabilization in cell death
.
Oncogene
2008
;
27
:
6434
51
.
29.
Stover
TC
,
Sharma
A
,
Robertson
GP
,
Kester
M
. 
Systemic delivery of liposomal short-chain ceramide limits solid tumor growth in murine models of breast adenocarcinoma
.
Clin Cancer Res
2005
;
11
:
3465
74
.
30.
Colombini
M
. 
Ceramide channels and their role in mitochondria-mediated apoptosis
.
Biochim Biophys Acta
2010
;
1797
:
1239
44
.
31.
Morales
A
,
Lee
H
,
Goni
FM
,
Kolesnick
R
,
Fernandez-Checa
JC
. 
Sphingolipids and cell death
.
Apoptosis
2007
;
12
:
923
39
.
32.
Elojeimy
S
,
Holman
DH
,
Liu
X
,
El-Zawahry
A
,
Villani
M
,
Cheng
JC
, et al
New insights on the use of desipramine as an inhibitor for acid ceramidase
.
FEBS Lett
2006
;
580
:
4751
6
.
33.
Struckhoff
AP
,
Bittman
R
,
Burow
ME
,
Clejan
S
,
Elliott
S
,
Hammond
T
, et al
Novel ceramide analogs as potential chemotherapeutic agents in breast cancer
.
J Pharmacol Exp Ther
2004
;
309
:
523
32
.
34.
Chapman
JV
,
Gouaze-Andersson
V
,
Messner
MC
,
Flowers
M
,
Karimi
R
,
Kester
M
, et al
Metabolism of short-chain ceramide by human cancer cells—implications for therapeutic approaches
.
Biochem Pharmacol
2010
;
80
:
308
15
.
35.
Lee
JY
,
Bielawska
AE
,
Obeid
LM
. 
Regulation of cyclin-dependent kinase 2 activity by ceramide
.
Exp Cell Res
2000
;
261
:
303
11
.
36.
Phillips
DC
,
Hunt
JT
,
Moneypenny
CG
,
Maclean
KH
,
McKenzie
PP
,
Harris
LC
, et al
Ceramide-induced G2 arrest in rhabdomyosarcoma (RMS) cells requires p21Cip1/Waf1 induction and is prevented by MDM2 overexpression
.
Cell Death Differ
2007
;
14
:
1780
91
.
37.
Ichikawa
A
,
Ando
J
,
Suda
K
. 
G1 arrest and expression of cyclin-dependent kinase inhibitors in tamoxifen-treated MCF-7 human breast cancer cells
.
Hum Cell
2008
;
21
:
28
37
.
38.
Mandlekar
S
,
Hebbar
V
,
Christov
K
,
Kong
AN
. 
Pharmacodynamics of tamoxifen and its 4-hydroxy and N-desmethyl metabolites: activation of caspases and induction of apoptosis in rat mammary tumors and in human breast cancer cell lines
.
Cancer Res
2000
;
60
:
6601
6
.
39.
Callaghan
R
,
Higgins
CF
. 
Interaction of tamoxifen with the multidrug resistance P-glycoprotein
.
Br J Cancer
1995
;
71
:
294
9
.
40.
Borst
P
,
Zelcer
N
,
van Helvoort
A
. 
ABC transporters in lipid transport
.
Biochim Biophys Acta
2000
;
1486
:
128
44
.
41.
Chapman
JV
,
Gouaze-Andersson
V
,
Cabot
MC
. 
Expression of P-glycoprotein in HeLa cells confers resistance to ceramide cytotoxicity
.
Int J Oncol
2010
;
37
:
1591
7
.
42.
Richter
C
,
Ghafourifar
P
. 
Ceramide induces cytochrome c release from isolated mitochondria
.
Biochem Soc Symp
1999
;
66
:
27
31
.
43.
Yu
T
,
Li
J
,
Sun
H
. 
C6 ceramide potentiates curcumin-induced cell death and apoptosis in melanoma cell lines in vitro
.
Cancer Chemother Pharmacol
2010
;
66
:
999
1003
.
44.
Rizvi
F
,
Heimann
T
,
Herrnreiter
A
,
O'Brien
WJ
. 
Mitochondrial dysfunction links ceramide activated HRK expression and cell death
.
PLoS ONE
2011
;
6
:
e18137
.
45.
Johansson
AC
,
Appelqvist
H
,
Nilsson
C
,
Kagedal
K
,
Roberg
K
,
Ollinger
K
. 
Regulation of apoptosis-associated lysosomal membrane permeabilization
.
Apoptosis
2010
;
15
:
527
40
.
46.
Liu
X
,
Cheng
JC
,
Turner
LS
,
Elojeimy
S
,
Beckham
TH
,
Bielawska
A
, et al
Acid ceramidase upregulation in prostate cancer: role in tumor development and implications for therapy
.
Expert Opin Ther Targets
2009
;
13
:
1449
58
.
47.
Liu
X
,
Elojeimy
S
,
Turner
LS
,
Mahdy
AE
,
Zeidan
YH
,
Bielawska
A
, et al
Acid ceramidase inhibition: a novel target for cancer therapy
.
Front Biosci
2008
;
13
:
2293
8
.
48.
Flowers
M
,
Fabrias
G
,
Delgado
A
,
Casas
J
,
Abad
JL
,
Cabot
MC
. 
C6-Ceramide and targeted inhibition of acid ceramidase induce synergistic decreases in breast cancer cell growth
.
Breast Cancer Res Treat
2012
;
133
:
447
58
.
49.
Gouaze-Andersson
V
,
Flowers
M
,
Karimi
R
,
Fabrias
G
,
Delgado
A
,
Casas
J
, et al
Inhibition of acid ceramidase by a 2-substituted aminoethanol amide synergistically sensitizes prostate cancer cells to N-(4-hydroxyphenyl) retinamide
.
Prostate
2011
;
71
:
1064
73
.
50.
Hwang
JJ
,
Kim
HN
,
Kim
J
,
Cho
DH
,
Kim
MJ
,
Kim
YS
, et al
Zinc(II) ion mediates tamoxifen-induced autophagy and cell death in MCF-7 breast cancer cell line
.
Biometals
2010
;
23
:
997
1013
.