Experiences in Applying Mixed-Methods Approach in Information Systems Research

Introduction

Research designs and methods adopted by Social Sciences researchers in general, and in the Information Systems (IS) field in particular, can be broadly classified into two main categories, namely quantitative and qualitative (Jick, 1979; Orlikowski & Baroudi, 1991; Mingers, 2001; Creswell, 2003; Saunders, et al., 2003). However, it is widely understood and recognized that both quantitative and qualitative designs have their own advantages and limitations. For example, questionnaire survey, as a typical quantitative method, is a very efficient and economical way for collecting data from a large sample in a wide geographical area at the same time (Bryman, 2004, pp. 133-134). Nonetheless, a questionnaire is arguably a less efficient method to be used in exploratory studies that aim to investigate and explore sophisticated social contexts (Robson, 2002, p. 234). On the other hand, interview as a typical qualitative tool is very useful and efficient in gathering and exploring in-depth human insights and perceptions on complex social phenomena (Saunders, et al., 2003, p. 246; Bryman, 2004, p. 321). Nevertheless, as interviews can very often last more than one hour, it is very time-consuming to carry out interviews with a large group of respondents.

The realization of the inherent limitations of quantitative and qualitative approaches results in the emergence and use of an alternative research design, namely mixed-methods research. Mixed-methods research integrates and combines both quantitative and qualitative methods to investigate the same underlying phenomenon in one single study (Leech & Onwuegbuzie, 2009). This approach is deemed to be efficient in supplementing the weaknesses of single method designs and thus leading to richer findings and higher quality research (Jick, 1979; Mingers, 2001; Creswell, 2003; Fidel, 2008; Leech & Onwuegbuzie, 2009). It has been used as a distinct approach in social sciences research for more than five decades (Campbell & Fiske, 1959; Jick, 1979; Rocco, et al., 2003). In the IS field, a considerable number of researchers have advocated the use of mixed-methods approach since the early 1990s (Galliers, 1991; Lee, 1991; Robey, 1996; Mingers, 2001; Petter & Gallivan, 2004; Fidel, 2008). Curiously, despite strong and continuous support from IS researchers, the actual use of the mixed-methods approach in IS research has not been prevalent. In particular, Mingers (2003) reviewed the IS literature between 1993 and 2000, and found that only 20% of articles published in this period of time adopted multiple research methods. In a more recent study, Fidel (2008) reviewed 465 articles published in four major journals in Library and Information Science (LIS) during 2005-2006. The study found that only 17% of these LIS articles adopted multiple methods, and only 5% could be considered as ‘truly’ mixed-methods research (Fidel, 2008). This low application rate of mixed-methods research in the IS field might be caused by a number of issues such as:

• •

  There may exist some misunderstandings among IS researchers about the actual meaning of mixed-methods research (i.e. is ‘mixed-methods’ the same as ‘multiple-methods’ design?).

• •

  IS researchers may find it difficult to decide which mixed-methods design would be suitable for a particular study (e.g. what priority or weight should be given to the quantitative and qualitative methods? In what sequence these methods should be conducted?).

• •

  IS researchers may have difficulties in integrating and making sense of quantitative and qualitative components effectively across the entire study.