Temporal, multimodal microscopy imaging of live cells is becoming widely used in studies of cellular processes. In general, temporal sequences of images with functional and morphological data from live cells are acquired using multiple image sensors. The images from the different sources usually differ in resolution and have non-coincident fields of view, making the merging process complex. We present a new tool – iCellFusion – that performs data fusion of images from Phase-Contrast Microscopy and Fluorescence Microscopy in order to correlate the information on cell morphology, lineage and functionality. Prior to image fusion, iCellFusion performs automatic or computer-aided cell segmentation and establishes cell lineages. We exemplify its usage on time-lapse, multimodal microscopy images of bacteria producing fluorescent spots. We expect iCellFusion to assist research in Cell and Molecular Biology and the healthcare sector, where live-cell imaging is an increasingly important technique to detect and study diseases at the cellular level.
TopIntroduction
The benefits of using multisensory data in computer vision has encouraged research in several areas, including biomedical imaging (Sotiras, Davatzikos, & Paragios, 2013) and remote sensing (Pohl & Van Genderen, 1998). The combination of different sensorial sources of information into one model, named multisensory fusion, allows information from multiple sources to be used cooperatively (Luo, Yih, & Su, 2002). In recent years, several algorithms have been proposed for fusing images from both unimodal and multimodal image sensors (Dong, Zhuang, Huang, & Fu, 2009).
Multimodal image fusion is becoming particularly useful in biomedical research (Bonnet, 2000, 2004; Glasbey, Buildings, Eh, & Martin, 1996; James & Dasarathy, 2014) and in the healthcare sector. In general, the goal is to fuse functional and structural images (James & Dasarathy, 2014), in order to search for correlations between morphological and functional changes. Recent cardiac diagnosis techniques, for example, use fusion of real-time 3-dimensional ultrasound and magnetic resonance (MR) (Huang et al., 2009), while diagnosis techniques of several brain diseases often use a fusion of MR and computed tomography images (Swati & Kadbe, 2013).
Similarly, in Cell and Molecular Biology, the advent of novel techniques for tagging proteins and RNA molecules with fluorescent probes in live cells has created a need for techniques of image fusion from different microscopy sources, such as phase contrast (which informs about cell morphology) and fluorescence microscopy (which informs on functionalities such as the kinetics of fluorescent proteins). For example, studies of co-localizations of different cellular structures and specific proteins (Bolte & Cordelières, 2006) are based on the image fusion of different fluorescence channels.
One of the first applications of multimodal imaging was the study of double labelled DNA via the fusion of dual colour images (such as the red and green filter components) (E. M. M. Manders, Verbeek, & A, 1993; E. M. Manders, Stap, Brakenhoff, van Driel, & Aten, 1992). To assist this effort, studies began on techniques of image fusion (E. M. M. Manders et al., 1993), including images from different microscopy techniques. Illumination and contrast methods such as bright field, dark field, phase contrast, differential interference contrast, and fluorescence methods such as epifluorescence and confocal microscopy (Bonnet, 2000, 2004; Glasbey et al., 1996), have since been fused to correlate functional and morphological cell data. The use of these techniques, individually or combined, is currently a common practice in studies using data at the single-cell level (Stephens & Allan, 2003). For example, the fusion of fluorescence and phase contrast images has been used in studies of the partitioning of F-plasmids (Niki & Hiraga, 1997) and the localization of DNA segments on the chromosome of Escherichia coli cells (Niki, Yamaichi, & Hiraga, 2000). Similarly, to study the kinetics of the genetic circuit responsible for the utilization of lactose in E. coli, green fluorescence images were fused with inverted phase contrast images (Ozbudak, Thattai, Lim, Shraiman, & Van Oudenaarden, 2004). Also, protein numbers in individual cells were recently quantified at the single-molecule level using fused fluorescence and phase contrast images (Taniguchi et al., 2010).