Planning Interventions for Gene Regulatory Networks as Partially Observable Markov Decision Processes

Introduction

Gene regulatory network (GRN) models, in their many forms (de Jong, 2002), provide a mathematical basis for representing and reasoning about biological processes. Of prime importance, is the use of GRNs to generate predictions of how a biological process changes over time through various forms of inference. One of the primary forms of inference is projection through simulation, especially in probabilistic GRN models. This chapter explores how to move beyond simulating biological processes, modeled as GRNs, and focuses on strategically inserting (planning) intervening actions to control the process. With connections to both shortest path problems and automated theorem proving, planning is a well-studied problem in Artificial Intelligence that focuses on generating sequences of actions (plans) that will transform an initial state into a goal state. The solution to the planning task (in this case an intervention plan) identifies how one might intervene with external control actions in abnormal development and prohibit cells from reaching undesirable states. This chapter explores planning interventions to avoid metastasis prone states in the WNT5A GRN.

The specific GRN model, described herein, represents a biological process by a set of genes and their regulatory influences over each other. The GRN focuses solely on genes (omitting proteins or other molecules) to model the high level behavior of many genes versus the low level behavior of a much smaller system. Because practical GRN models are typically learned from microarray data (Kim et al., 2000), they are situated at the right level of granularity for automated parameterization. As described below, microarray experiments measure the activity level of thousands of genes from living tissue in terms of mRNA concentrations (the products of gene transcription used to code proteins). Correlations between observed gene activity levels help describe regulatory influences. Predictor functions characterize the regulatory influences and provide a dynamic model (e.g., when genes g1 and g2 are highly active, gene g3 becomes inactive). The state of the GRN models the activity levels of genes and the predictor functions describe possible next states.

Layered on top of the GRN model, a planning model includes outside interventions (e.g., using RNA interference to suppress a gene’s activity level) to alter the GRN predictor functions and effectively control the state evolution of the GRN. In an intervention plan, each action models either a possible intervention or non-intervention that will change the gene activity levels (i.e., the state of the GRN).

As one practical application of planning interventions, consider the treatment of cancer, where the chaining of multiple treatments is being aggressively explored. It is already clear that attention must be paid to the sequencing of these treatments. For example, cytotoxic drugs that induce replicative arrest and subsequent apoptosis (i.e. 5-fluorouracil or platinum containing drugs) rely on active replication to be effective. Cytostatic drugs (i.e. anti-estrogens, anti-angiogenics) on the other hand reduce ability to proliferate in order to reduce tumor load in patients and allow reductions in tumor size through active immunologic defense. Using a treatment sequence where a cytostatic drug is applied and still effective when the cytotoxic drug is given usually reduces the effectiveness of the cytotoxic drug, since far fewer tumor cells are actively replicating. The opposite sequence of treatment can be far more effective. It is likely that many treatment sequences will be either synergistic or antagonist based on timing and strengths of doses, and that a model that allows the physician to exploit the vulnerabilities of a disease to greatest therapeutic effect would be of great utility, not only for cancer treatment but also for other complex disorders.