Planning Interventions for Gene Regulatory Networks as Partially Observable Markov Decision Processes

Planning Interventions for Gene Regulatory Networks as Partially Observable Markov Decision Processes

Daniel Bryce, Seungchan Kim
DOI: 10.4018/978-1-60566-685-3.ch023
(Individual Chapters)
No Current Special Offers


In this chapter, a computational formalism for modeling and reasoning about the control of biological processes is explored. It comprises five main sections: a survey of related work, a background on methods (including discussion of the Wnt5a gene regulatory network, the coefficient of determination method for deriving gene regulatory network models, and the partially observable Markov decision process model and its role in modeling intervention planning problems), a main section on the approach taken (including algorithms for solving the intervention planning problems and techniques for representing components of the problems), an empirical evaluation of the intervention planning algorithms on synthetic and the Wnt5a gene regulatory networks, and a conclusion and future directions section. The techniques described present a promising avenue of future research in reasoning algorithms for improved scalability in planning interventions in gene regulatory networks.
Chapter Preview


Gene regulatory network (GRN) models, in their many forms (de Jong, 2002), provide a mathematical basis for representing and reasoning about biological processes. Of prime importance, is the use of GRNs to generate predictions of how a biological process changes over time through various forms of inference. One of the primary forms of inference is projection through simulation, especially in probabilistic GRN models. This chapter explores how to move beyond simulating biological processes, modeled as GRNs, and focuses on strategically inserting (planning) intervening actions to control the process. With connections to both shortest path problems and automated theorem proving, planning is a well-studied problem in Artificial Intelligence that focuses on generating sequences of actions (plans) that will transform an initial state into a goal state. The solution to the planning task (in this case an intervention plan) identifies how one might intervene with external control actions in abnormal development and prohibit cells from reaching undesirable states. This chapter explores planning interventions to avoid metastasis prone states in the WNT5A GRN.

The specific GRN model, described herein, represents a biological process by a set of genes and their regulatory influences over each other. The GRN focuses solely on genes (omitting proteins or other molecules) to model the high level behavior of many genes versus the low level behavior of a much smaller system. Because practical GRN models are typically learned from microarray data (Kim et al., 2000), they are situated at the right level of granularity for automated parameterization. As described below, microarray experiments measure the activity level of thousands of genes from living tissue in terms of mRNA concentrations (the products of gene transcription used to code proteins). Correlations between observed gene activity levels help describe regulatory influences. Predictor functions characterize the regulatory influences and provide a dynamic model (e.g., when genes g1 and g2 are highly active, gene g3 becomes inactive). The state of the GRN models the activity levels of genes and the predictor functions describe possible next states.

Layered on top of the GRN model, a planning model includes outside interventions (e.g., using RNA interference to suppress a gene’s activity level) to alter the GRN predictor functions and effectively control the state evolution of the GRN. In an intervention plan, each action models either a possible intervention or non-intervention that will change the gene activity levels (i.e., the state of the GRN).

As one practical application of planning interventions, consider the treatment of cancer, where the chaining of multiple treatments is being aggressively explored. It is already clear that attention must be paid to the sequencing of these treatments. For example, cytotoxic drugs that induce replicative arrest and subsequent apoptosis (i.e. 5-fluorouracil or platinum containing drugs) rely on active replication to be effective. Cytostatic drugs (i.e. anti-estrogens, anti-angiogenics) on the other hand reduce ability to proliferate in order to reduce tumor load in patients and allow reductions in tumor size through active immunologic defense. Using a treatment sequence where a cytostatic drug is applied and still effective when the cytotoxic drug is given usually reduces the effectiveness of the cytotoxic drug, since far fewer tumor cells are actively replicating. The opposite sequence of treatment can be far more effective. It is likely that many treatment sequences will be either synergistic or antagonist based on timing and strengths of doses, and that a model that allows the physician to exploit the vulnerabilities of a disease to greatest therapeutic effect would be of great utility, not only for cancer treatment but also for other complex disorders.

Key Terms in this Chapter

Partially Observable Markov Decision Process: A formal model of extended decision making with stochastic actions and noisy observations of a partially observable environment.

Planning: Synthesizing a sequence of actions to achieve a goal.

Search: The process of evaluating alternative solutions by comparing partial or complete solutions.

Gene Regulatory Network: A systems biology model of intracellular components and their interactions.

Melanoma: A malignant skin tumor.

Dynamic Programming: A problem solving technique that extends optimal sub-solutions to optimal complete solutions.

Intervention: An externally controllable action that has a direct or indirect impact on the behavior of a biological system.

Complete Chapter List

Search this Book: