## Abstract

Quantitative models from population biology and evolutionary game theory frame the tumor-host interface as a dynamical microenvironment of competing tumor and normal populations. Through this approach, critical parameters that control the outcome of this competition are identified and the conditions necessary for formation of an invasive cancer are defined. Perturbations in these key parameters that destabilize the cancer solution of the state equations and produce tumor regression can be predicted. The mathematical models demonstrate significant theoretical limitations in therapies based solely on cytotoxic drugs. Because these approaches do not alter critical parameters controlling system dynamics, the tumor population growth term will remain positive as long as any individual cells are present so that the tumor will invariably recur unless all proliferative cells are killed. The models demonstrate that such total effectiveness is rendered unlikely by the genotypic heterogeneity of tumor populations (and, therefore, the variability of their response to such drugs) and the ability of tumor cells to adapt to these proliferation constraints by evolving resistant phenotypes. The mathematical models support therapeutic strategies that simultaneously alter several of the key parameters in the state equations. Furthermore, the models demonstrate that administration of cytotoxic therapies will, by reducing the tumor population density, create system dynamics more conducive to perturbations by biological modifiers.

## Introduction

The tumor-host interface is a complex structure dominated by stochastic, non-linear processes for which there is no clear theoretical framework of understanding (1). Experience in the physical sciences has demonstrated that such systems cannot be fully understood using intuitive, linear reasoning alone (2). Rather, appropriate non-linear mathematical models are necessary to serve as the theoretical framework for synthesis of extant data and integration of rapidly accumulating new information. Mathematical models based on biological first principle serve to define critical underlying dynamics and interactions in these complex systems and predict the results of system perturbations through therapy.

Clinical medicine has not generally integrated quantitative methods into theoretical analysis of tumor biology. We submit that this has impeded progress in clinical oncology because the vast data generated by molecular biology and other new technologies have not been synthesized into integrative, conceptual models. Furthermore, in the absence of sound theoretical framework, design and evaluation of therapeutic strategies remains largely empiric. This is well summarized by the authors of Molecular Biology of the Cell: “In general progress with the vexing problem of anticancer therapy has been slow—a matter of trial and error and guesswork as much as rational calculation. In the search for better ways of curbing the survival, proliferation, and spread of cancer cells, it is important to examine more closely the strategies by which they thrive and multiply” (3).

The purpose of this paper is to provide the simplest possible mathematical framework that encompasses the critical behavior of the tumor-host interface, that is, the advance of tumor tissue into the surrounding host tissue, and to elucidate key biological parameters controlling this behavior. Within the context of this framework, developed from methods used in population biology and evolutionary game theory, we are able to gain insight into the effectiveness of current treatment approaches as well as suggest new therapeutic strategies.

We initially present non-evolutionary population models to illustrate critical, general parameters in the dynamics of the tumor-host interface. These lumped, phenomenologic terms provide insight into potential limitations of treatment strategies focused entirely on inducing tumor cell death and suggest methods that might optimize new approaches such as anti-angiogenesis and anti-epidermal growth factor receptor (anti-EGFR) agents in combination with cytotoxic agents. We then add the potential for tumor evolution due to accumulating random mutations to examine the dynamics of adaptation to iatrogenic proliferation constraints generated by treatments. The models demonstrate the need for multimodal therapy directed simultaneously at the critical parameters along with reduction of the tumor population through cytotoxic agents in a sufficiently short time period to prevent cellular evolution and adaptation.

We believe that this analysis provides insight into tumor biology and treatment not available by other means and illustrates the potentially critical role of mathematical analysis in successfully understanding and treating invasive cancer.

## Mathematical Models

### Non-Evolutionary Population Interactions

We have previously demonstrated that the tumor-host interface can be explored using the principles of population ecology (4, 5). In this paradigm, tumor populations are viewed as invading species disrupting the well-ordered, cooperative cellular societies that ordinarily form functioning tissue in multicellular organisms. Each tumor “species” arising within or entering this somatic microecology may experience one of three outcomes: (*a*) it may undergo unconstrained proliferation driving normal cellular populations to extinction thus forming an invasive cancer; (*b*) it may develop a stable coexistence with normal cells forming a “benign” tumor; and (*c*) it may be eliminated by the host tissue and suffer extinction. We first examine the dynamics of the tumor-host interface to determine the critical parameters that control these outcomes.

The simplest and most widely used model describing the interaction of multiple populations is of the Lotka-Volterra type (6). There are a variety of other mathematical models of tumor growth kinetics (see, *e.g.*, Chapter 3 of 6) for a survey of models). However, because all such models, including Lotka-Volterra, predict the same qualitative behavior, we have employed the one that is most simple to present and analyze mathematically (7–9). Here, we write Lotka-Volterra equations with one dominant, phenotypically stable (see below for populations that are mutagenic and, hence, evolutionary) tumor population (*N*_{T}) interacting with normal cells (*N*_{N}).

where *R*_{N} and *R*_{T} are maximum growth rates of normal and tumor cells, respectively (*i.e.*, the net result of tumor cell doubling minus tumor cell loss from apoptosis or necrosis); *K*_{N} and *K*_{T} denote the maximal normal and tumor cell densities; and *α*_{NT} and *α*_{TN} are the competition coefficients. These parameters represent a lumped phenomenologic terms with the following biological significance at the tumor-host interface. *α*_{TN} encompasses a variety of host defenses including the immune response that serve to decrease growth of the tumor populations, *α*_{NT} is the negative effect of tumor on normal tissue such as tumor-induced extracellular matrix breakdown and microenvironmental changes. The carrying capacity term, *K*, represents the summation of growth promotion and constraint within the tissue. This includes growth promoters such as the concentration of epidermal growth factor in the environment and the number of receptors (EGFR) expressed on the cell surface, negative growth factors such as β-catenin (including its regulators such as the APC gene product), and the availability of adequate substrate to allow synthesis of macromolecules for new cells. This is further discussed below.

In the absence of tumor, this system achieves a stable, nonzero steady state of normal cells but also exhibits solutions in which, under some circumstances, one population (*i.e.*, invasive cancer) may invade and destroy the other. If initial conditions specify normal cells at their carrying capacity (*K*_{N}) when a small number of tumor cells emerge in or migrate to the region, the following steady states [*i.e.*, (d*N*_{N})/(d*t*) = (d*N*_{T})/(d*t*) = 0] may result.

*N*_{N} = 0, *N*_{T} = 0. This is the trivial solution and is not biologically relevant.

*N*_{N} = *K*_{N}, *N*_{T} = 0. This corresponds to normal tissue with no tumor cell present. That is, the tumor regresses completely. Regardless of the starting point, the system will always arrive at this state if both *α*_{TN}*K*_{N}/*K*_{T} > 1 and *α*_{NT}*K*_{T}/*K*_{N} < 1. If the starting point is sufficiently close to *N*_{N} = *K*_{N}, *N*_{T} = 0 (as we suppose the initial conditions will be in early tumor development), only the former condition need be satisfied.

*N*_{N} = (*K*_{N} − *α*_{NT}*K*_{T})/(1 − *α*_{NT}*α*_{TN}), *N*_{T} = (*K*_{T} − *α*_{TN}*K*_{N})/(1 − *α*_{NT}*α*_{TN}). This corresponds to a stable coexistence of tumor and normal cells that we interpret as benign, non-invasive tumor. The system will always arrive at this state if both *α*_{NT}*K*_{T}/*K*_{N} < 1 and *α*_{TN}*K*_{N}/*K*_{T} < 1.

*N*_{N} = 0, *N*_{T} = *K*_{T}. This corresponds to an invasive cancer with complete overgrowth of the normal tissue by the tumor cells. Regardless of the starting point, the system will always arrive at this state if both *α*_{NT}*K*_{T}/*K*_{N} > 1 and *α*_{TN}*K*_{N}/*K*_{T} < 1. If the starting point is sufficiently close to *N*_{N} = 0, *N*_{T} = *K*_{T} (as would occur when tumor treatment is initiated), only the former condition need be satisfied.

For any given set of parameters, there are either five or six possible equilibrium points which lie in the non-negative orthant. These are determined by the intersection of isoclines defined by lines along which derivatives are zero. There are four isoclines defined by

One equilibrium point is the origin defined by the intersection of the first two isoclines. Four equilibrium points are defined by the intersection of the last two isoclines with the first two and the sixth possible equilibrium point defined by the intersection of the last two isoclines, provided that such an intersection exists. Fig. 1 illustrates possible situations. The three panels illustrate steady states 2, 3, and 4 as discussed above using the following parameter values:

In the first panel, the only stable equilibrium point is *N*_{N} = 10, *N*_{T} = 0. There are six equilibrium points in the middle panel with only the positive one stable at *N*_{N} = *N*_{T} = 6(2/3). In the last panel, the equilibrium point is given by *N*_{N} = *N*_{T} = 10. Note, in this example, the transition from an equilibrium point that does not allow tumor growth to one that allows tumor-normal cell coexistence (*i.e.*, benign tumor growth) to one in which the tumor cells drive normal populations to extinction (*i.e.*, invasive cancer) is dependent only on changes in the cellular interaction parameter *α*. Using the parameters cited above, we see in Fig. 2 how the system moves from the same initial starting condition to an equilibrium solution.

Given the initial conditions expected in carcinogenesis (*i.e.*, a small number of cancer cells numerically dominated by normal cell populations), an invasive cancer must possess the parameter values specified under steady state solution 2 (*α*_{NT}*K*_{T}/*K*_{N} > 1 and *α*_{TN}*K*_{N}/*K*_{T} < 1). Clearly this invasive cancer solution is favored if *K*_{T} > *K*_{N}. Biologically, the carrying capacity represents the restrictions on cell numbers due to: (*a*) normal tissue controls mediated by positive and negative growth factors (*e.g.*, oncogenes and tumor suppressor genes) encompassing interactions with other cells, the extracellular matrix, and soluble growth factors; and (*b*) substrate availability (the cell must be able to accumulate substrate in excess of basal metabolic demands to synthesize necessary components for mitosis). Thus, oncogene and tumor suppressor gene mutations that characteristically accumulate during carcinogenesis (10, 11) will tend to increase *K*_{T}. However, substrate availability due to lack of vascularity will tend to decrease *K*_{T} as demonstrated by the transition from non-invasive growth to invasive tumor growth coincident with the onset of angiogenesis (12).

The invasive cancer solution is also favored if *α*_{NT} is large. Recall that the *α* term encompasses the negative effects of one population on the other. Thus, the system dynamics dictating that development of an invasive cancer requires tumor cells exert a substantial negative effect on the normal cells. One clear mechanism for this is expression of the glycolytic phenotype. We have previously demonstrated that the consequent acidification of the extracellular tumor microenvironment will produce an acid gradient extending into adjacent normal tissue (4) resulting in normal cell death mediated (13) by p53-dependent apoptosis pathways [induced by increased caspase activity (14)] as well as degradation of extracellular matrix [through acid-induced release of Cathepsin B and other proteolytic enzymes (15)], inhibition of tumor immune response (16) and induction of angiogenesis [mediated by acid-induced release of VEGF and IL8 (17, 18)].

### Therapeutic Strategies in Non-Evolving Tumor Cells

To evaluate potential cancer therapies, we can examine the system after it has achieved the invasive cancer stable state of *N*_{T} = *K*_{T} and *N*_{N} = 0. The goal of therapy is to destabilize this solution so that the system will evolve to one of the other solutions in which the tumor cells are eliminated entirely or at least remain in stable equilibrium with normal tissue (*i.e.*, solutions 2 and 3 above).

Classical cytotoxic therapies seek to eliminate the tumor by directly killing as many individual tumor cells as possible. The fundamental flaw in this strategy is apparent from simple inspection of the system equations and Figs. 1 and 2. Reducing the population of tumor cells does not alter the basic system dynamics. That is, given the parameter values necessary for invasive cancer, the system will always tend to the state *N*_{N} = 0, *N*_{T} = *K*_{T} provided *N*_{T} > 0. In other words, cytotoxic therapies may eliminate a large number of malignant cells reducing the tumor size but, unless all of the malignant cells are eliminated, invasive cancer remains the stable steady state solution to the state equations so even a small surviving tumor population will inevitably repopulate the tissue so that the invasive cancer recurs. This is illustrated in Fig. 2. If we assume that cytotoxic therapy has reduced the tumor population from *K*_{T} to that shown at the starting point, system dynamics dictate (*i.e.*, case 4) return to the original state in which *N*_{T} = *K*_{T} (where *K*_{T} = 10 in our example) and *N*_{N} = 0.

Therapeutic strategies more consistent with the quantitative models would typically focus on altering the key parameters that confer stability on the invasive tumor solution. That is, alter the underlying system dynamics as shown in Fig. 2 to produce case 3 or, better, case 2. Assuming the initial conditions are *N*_{T} = *K*_{T} and *N*_{N} = 0, the tumor state will be destabilized only if both of the above inequalities are reversed so that *α*_{NT}*K*_{T}/*K*_{N} < 1 and *α*_{TN}*K*_{N}/*K*_{T} > 1. Assuming the carrying capacity of normal cells remain constant, three general strategies are apparent: decrease the value of *K*_{T}, increase *α*_{TN}, and decrease *α*_{NT}.

Translating these parameter changes to conventional tumor strategies, we note that anti-angiogenesis drugs will diminish the carrying capacity of the environment for tumor cells thus decreasing the value of *K*_{T}. Strategies that increase immune response to tumor antigens will increase the value of *α*_{TN} (the negative effects on the cancer cells due to the presence of normal host cells). Strategies that reduce activity of metaloproteinases will decrease the value of *α*_{NT} (the negative effects on normal cells generated by tumor cells). Note, however, that the latter two strategies will each affect only one of the two inequalities. Therefore, to fully destabilize the tumor solution, these approaches should ideally be combined or added to the anti-angiogenesis therapy. Interestingly, if the number of tumor cells is near *K*_{T}, then only the first inequality needs to be satisfied to insure stability of the tumor solution. Thus, immunotherapy, by increasing the value of *α*_{TN}, will typically exhibit a biological effect only in tumors that have already undergone a significant population decline due to chemotherapy, radiation therapy, or surgery. This appears consistent with clinical trials using the 17-1A antibody in colorectal cancer which demonstrated that treatment was more effective in eliminating microscopic metastatic disease than bulkier local recurrence (19–21). Similarly, anti-EGFR antibodies appear more effective when added to cisplatin (22) or doxorubicin (23).

Two caveats are also apparent from this simple analysis: (*a*) Threshold effects should be expected. Thus, for example, anti-angiogenesis therapy may result in some changes in tumor growth but it will not fundamentally change the dynamics and result in complete tumor regression unless the change in *K*_{T} is sufficient to satisfy the above inequalities. (*b*) Unexpected effects may confound therapeutic expectations. Again using anti-angiogenesis strategies, for example, note that if this approach also restricts the vascularity of normal tissue at the tumor-host interface, the resulting reduction in *K*_{N} will tend to *stabilize* the tumor solution.

In addition, we have thus far assumed the tumor phenotype to be stable. In fact, the mutagenic phenotype found in cancer cells confers the ability to evolve and adapt to changing environmental conditions. The potential effects of this property on therapeutic strategies are presented next.

### Cellular Heterogeneity and Evolution

In the previous section, we assumed that the tumor cells possess a homogeneous and static phenotype. This is, of course, biologically unrealistic because cellular populations in both malignant and preneoplastic lesions exhibit substantial spatial and temporal heterogeneity. In part this cellular heterogeneity is due to genetic instability induced by the mutator phenotype or a mutagenic environment such as chronic inflammation or acidosis. Because of this increased mutation rate, tumor cells possess the ability to evolve at a greater rate than non-transformed cells. That is, each mutation produces an “experimental” phenotype that interacts with environmental selection factors. Those mutations that confer a survival advantage are rewarded by proliferation and clonal expansion.

This evolutionary capacity is of enormous importance in developing therapeutic strategies because, as pointed out by Coldman and Goldie (24) “much experimental evidence has accrued that [cancer] cells which display inheritable resistance are the cause of treatment failure.” Several mathematical models describing the development of drug resistance in cancer therapy have been developed using the Lotka-Volterra competition model as presented above (25–31). The reader may want to consult the literature for other modelling approaches (26). Here we directly incorporate cellular evolution into the cellular and microenvironmental dynamics during cancer treatment.

This model, based on evolutionary game theory, requires some additional concepts and terms. The phenotypic properties of each cellular population of size *N*_{i} is identified by a “strategy” *u*_{i} that defines its interaction with environmental factors controlling cellular proliferation such as growth promoters or inhibitors (including chemotherapeutic agents as outlined below) and substrate delivery. Typically a range of possible strategies is available depending on the number of viable phenotypes that can be generated through mutations of the genome. Each mutation in the genome produces a new cellular strategy which, in turn, confers some fitness function *H*_{i} on the cellular population as defined by its ability to proliferate in the context of extant strategies employed by the current populations. The Lotka-Volterra equations, in terms of *n* different cellular populations currently in the tissue, are formulated in terms of fitness function as follows:

where

and *R*_{i} is the maximum proliferation rate, *k*(*u*_{i}) is the carrying capacity for population *i*, and *α*(*u*_{i}, *u*_{j}) is the competitive effect of population *j* using strategy *u*_{j} on the fitness of individuals of species *i* using strategy *u*_{i}. If the fitness function of a mutant cell results in increased proliferation (*i.e.*, the cell is more fit than its ancestors), clonal expansion results. Cells that are less fit do not proliferate and that strategy is eliminated. Neutral mutations may also be maintained in the population. Through this combination of random mutations interacting with microenvironmental selection parameters, progressively fitter populations emerge over time. This constitutes the somatic evolution of cancer so that the observed cellular strategy varies over time as new, fitter populations emerge sequentially during the transition from normal tissue to premalignant lesions to invasive cancer. This ability to evolve also confer an ability to adapt to environmental changes and is, thus, fundamentally important to the emergence of tumor resistance to any host response or therapeutic strategy.

This cellular fitness may be expressed using a fitness generating function, generally called a *G*-function(31, 34). A full exposition of this mathematical technique is beyond the scope of this article. A brief discussion for the mathematically inclined reader is included in Appendix A. Here, the *G*-function for a cellular community that includes tumor populations and has a range of possible strategies is defined by:

where *v* is a virtual variable. Note that by setting the virtual variable equal to the strategy used by any cellular population results in the fitness function for that population. That is

In this context, the *G*-function describes the evolutionary potential for all evolutionarily identical individuals.

In terms of the *G*-function, the population dynamics equations are written as

Given the fact that variability exists in the strategies *u*_{i}, it can be shown (25) that strategies evolve according to

where *σ*_{i} is related to the variance in strategies. In the simulations below, we assume that normal cells cannot evolve by setting *σ*_{1} = 0.

Consider again the simple two-population normal-cancer cell situation. By integrating Eqs. 6 and 7 we find that, over time, the tumor cells evolve, reach a maximum steady state of cellular fitness. This represent the state of an invasive cancer within the context of unmodified (*i.e.*, untreated) tissue dynamics. The top frame of Fig. 3 illustrates this situation. This figure shows the changes in population number of both the normal and cancer cells with time keeping the normal cell strategy constant, but allowing the cancer cells to evolve as illustrated in the bottom frame. The cancer cells are introduced in small numbers at a strategy different than the normal cell strategy with the normal cells at their carrying capacity (in the absence of cancer cells).

Fig. 4 illustrates the fact that, at equilibrium, the cancer cells are at a local maximum on an adaptive fitness landscape as defined by the *G*-function. This maximum represents a local evolutionary stability for the cancer cells, which makes it impossible for other cells with similar strategy values to invade. The slight rise in the number of normal cells initially is due to the fact that *σ*_{k} has changed that in turn changes the equilibrium solution.

We can now model institution of treatment of a cancer population that has reached steady state values shown in Fig. 3. Using cell-specific drugs to eliminate the cancer by adding an appropriate “harvesting” term so that the *G*-function becomes

where *k*_{h} is a term expressing the level of drug dosage, *ū* is the cancer cell strategy at which the drug is most effective, and *σ*_{h} is the variance in effectiveness. Starting with the equilibrium conditions above and integrating Eqs. E and F with the *G*-function defined by Eq. G (see 34 for details of parameter values), it is found that cytotoxic chemotherapy is effective initially, but the cancer cells ultimately recover because they can evolve and a new tumor equilibrium state is obtained as illustrated in the first frame of Fig. 5. The net effect is that rather than curing the cancer, the cell-specific drug caused the cancer to evolve to a new form (second frame of Fig. 5) that is now highly resistant to the current and any similar therapeutic strategies. This is illustrated in Fig. 6 by the fact that the cancer cells are again sitting at a local maximum. Note that the normal cells are at a fitness less than zero resulting in a zero equilibrium population. These results are essentially identical to evolution of multi-drug resistance observed in treated human tumors (35, 36).

## Discussion

Invasive cancer is an emergent phenomenon resulting from non-linear, stochastic interactions between evolving cellular phenotypes and multiple environmental selection factors. A clinically detectable cancer is the end result of this complex system dynamics. Through quantitative models, the critical parameters that control the interactions at the tumor-host interface can be identified as the environmental carrying capacity for the tumor cells and the interaction of the tumor cells with normal cellular populations. Both of these are lumped phenomenologic terms. The former encompasses response to normal tissue growth inhibitions such as cellular interactions with other cells, the extracellular matrix, and various soluble growth factors and substrate-mediated growth controls such as angiogenesis. The latter includes the negative effects of host immune response on cancer cells as well as the negative effects of cancer cells on normal tissue due, for example, to excess production and excretion of *H*^{+} ions as detailed above.

Although conclusions from these simple models must be drawn with some caution, the population models clearly raise doubts that cancer therapy based solely on systemic cytotoxic drugs will ever successfully eradicate a broad range of tumors. While a therapeutic strategy that relies solely on killing tumor cells (without altering critical system parameters) may be sufficiently effective to reduce the tumor size, the models clearly demonstrate that d*N*_{T}/d*t* will remain >0 as long as *N*_{T} is >0. That is, the tumor population will inevitably rebound unless all proliferative cells within the population are eliminated. Two fundamental barriers to this requirement of total eradication of the tumor cells emerge readily from evolutionary game theory. First, the heterogeneity of tumor phenotypes (defined by the strategy parameter *u*) related to the stochastic nature of the random underlying mutations and the non-linear interactions with microenvironmental selection parameters (which are also variable due to spatial and temporal variations in, *e.g.*, blood flow) will likely produce a broad range of cellular sensitivity to cytotoxic drugs. Second, a cytotoxic therapy that “harvests” a large number (but not all) of the tumor cells simply becomes an additional environmental selection parameter. Tumor cells, because of their ability to evolve, adapt to this new factor and resistant populations readily emerge. Ultimately, the tumor regrows as the resistant populations proliferate rendering the therapy ineffective.

In summary, any therapy relying solely on tumor cytotoxic effects will be curative only if it is sufficiently effective to overcome the tumor phenotypic diversity such that it kills all proliferative cells in a time period sufficiently short to prevent evolution of resistance.

On the other hand, the models support the current trend toward therapeutic strategies focused on disrupting the interaction of tumor cells with their environment such as blocking EGFRs or anti-angiogenesis drugs because they alter the fundamental system parameters in addition to directly killing tumor cells. By altering the critical parameters in the state equations, these strategies have the potential advantage of rendering the tumor solution unstable, driving the system to a new steady state in which tumor cells are absent or at least remain at equilibrium with normal cells. This effect will likely be more durable because under these conditions, d*N*_{T}/d*t* will be < 0 for *N*_{T} > 0 or *N*_{T} > *N*_{Tmax} depending on whether the solution admits the presence of any tumor cells.

Finally, integrative models may allow more rational therapeutic design. It is apparent from the models that several critical parameters govern system dynamics in invasive cancer. Therapy directed toward only one of these parameters may be ineffective because each term in the inequalities required for stability of the tumor solution (*α*_{NT}*K*_{T}/*K*_{N} > 1 and *α*_{TN}*K*_{N}/*K*_{T} < 1) contains three lumped parameters each of which, in turn, may consist of several biological processes. Furthermore, any therapy is subject to long-term failure due to evolving tumor phenotypes that adapt to and overcome proliferation constraints. It is likely that the most effective therapies will be rationally designed combinations targeting more than one parameter or more than one component of each parameter. For example, *K*_{T} could be reduced through the simultaneous administration of growth inhibitors and anti-angiogenesis drugs (37, 38). This may explain the surpraadditive (37) growth inhibition of anti-EGFR combined with C225 (a chimerized version of the anti-EGFR antibody MAb225 that inhibits expression of VEGF and vascular permeability factor, thus decreasing angiogenesis). Other valuable combinations predicted by the models include the addition of cytotoxic drugs to certain biological modifiers. Reduction of tumor population density below the carrying capacity (*K*_{T}) allows the tumor solution to be destabilized with reversal either *α*_{NT}*K*_{T}/*K*_{N} > 1 or *α*_{TN}*K*_{N}/*K*_{T} < 1 while only the former inequality needs to be satisfied when *K*_{T} tumor cells are present. Thus, immune therapy which increases *α*_{TN} will likely not be effective as initial tumor therapy but may strongly affect tumor dynamics after debulking with chemotherapy. In fact, studies with trastuzumab, an antibody against the HER-2/NEU receptor, demonstrated 16% response rate to the antibody alone (39) but 52% response to the antibody combined with an anthracycline (40) and 42% combined with a taxane regimen. Similarly, the addition of the antibody increased survival when compared to cytotoxic therapy alone by 16% at 1 year (41) and 25% at 29 months (42) as predicted by the mathematical models.

## Appendix A

The *G*-function (25–27, 31, 32, 34) is used to examine the evolutionary stability characteristics of this model. In the above model, it is assumed that the tumor populations can exhibit a range of strategies as given by specifying the virtual variable *v*.

where the *σ* variables denote variances due to genetic diversity. We have previously shown (25–27) that, by varying the environmental parameter *σ*_{k}, the dynamical system can have equilibrium solutions composed of one or more cell types. That is, given a constant strategy vector **u**, there exists at least one non-zero equilibrium solution *N** (*i.e.*, not every component of *N** is zero). There are two ways on which we can find an equilibrium solution for *N**. One way is to solve for *N** from the system of equations (see Eq. E)

Note that, in general, a solution to this system of equations will require a numerical procedure. If more than one equilibrium solution exists, then the particular solution obtained will depend on the initial guess made for the solution. A second method for determining an equilibrium solution (when such a solution is asymptotically stable), is to choose **u** and an initial condition **N** (0) and simply let the solution to equations determine the equilibrium point by integrating the differential equations.

At equilibrium, one or more components of **N*** must be positive and non-zero for there to be a viable solution. The corresponding strategies are those that can coexist in the population of cells. However, the equilibrium solution to the above system of equations only considers the outcome for those strategies already resident in the population and does not consider, nor can it consider, the potentially infinite number of feasible strategies that may occur in the future via selection and/or mutation.

For our model, consider the following parameter values

In this situation, for a given **u**, there is only one non-zero equilibrium solution to the system (Eq. E). Choosing different strategies will result in different equilibrium values. For example *u*_{i} = 0 results in *N*_{1}* = 100.0. However, this solution is not evolutionarily stable because it can be displaced by introducing another cell population (even at small numbers) using the strategy *u*_{2} = 1. We seek an evolutionarily stable strategy (ESS) that has the property that it cannot be displaced by introducing a mutant strategy. We can obtain such a strategy for this set of parameters by simply picking any strategy (*e.g.*, *u*_{1} = 0) and letting it evolve according to the system of Eqs. E and F. The equilibrium solution to this set of equations is *u*_{1}* = 1.213 and *N*_{1}* = 83.2 and it is an ESS solution. We view this as normal tissue. This stability is maintained in spite of a low baseline mutation rate. That is, a normal cellular population will remain evolutionarily stable so long as conditions do not change.

With this model, if we increase *σ*_{k} from the value given above to

*e.g.*, due to damage or changes in surrounding tissue), we have previously shown (25) that there exist two equilibrium solutions to Eq. E. As a consequence, if we introduce a cancer cell at some strategy other than 1.213, it can coexist. In fact if we allow both the normal cells and cancer cells to evolve according to Eqs. E and F, we would arrive at an ESS composed of two strategies (the normal cells would no longer be at 1.213). However, normal cells are limited in their ability to evolve significantly within the lifetime of the host, whereas, tumor cells have no such limitation and can evolve to an equilibrium condition. This was incorporated into the integration of Eqs. E and F by setting

*σ*

_{1}= 0 (no evolution of normal cells) and

*σ*

_{2}> 0 (evolution allowed for the cancer cells). The results are depicted in Fig. 3.

Whether a population of cancer cells will evolve depends on the nature of the environmental growth constraints. For example, if the host-immune system is capable of eliminating tumor cells that express certain cell-surface antigens, the system dynamics will favor strategies that do not express these antigens and, over time, these cells will proliferate and the tumor will successfully evade the host response. As illustrated in the text, application of any tumor therapy similarly introduces new environmental selection forces that will, over time, favor evolution or resistant phenotypes.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

## References

*et al.*Randomized trial of monoclonal antibody for adjuvant therapy of resected Dukes' C colorectal carcinoma. German Cancer Aid 17-1A Study Group.

*cis*-diamminedichloroplatinum on well established A431 cell xenografts.

*In:*Dynamics of Complex Interconnected Biological Systems, Mathematical Modeling, pp. 236–249. Boston, MA: Birkhäuser, 1990.

*In:*Modeling and Management of Resources under Uncertainty, Lecture Notes in Biomathematics, pp. 83–99. Heidelberg: Springer-Verlag, 1987.

*In:*T. Basar and A. Haurie (eds.), Advances in Dynamic Games and Applications, pp. 356–374. Boston, MA: Birkhauser, 1994.

*in vitro*and

*in vivo*: angiogenic implication for signal transduction therapy of solid tumors.