Adapting Agile Practices for Data Warehousing, Business Intelligence, and Analytics

Adapting Agile Practices for Data Warehousing, Business Intelligence, and Analytics

Dinesh Batra (College of Business Administration, Florida International University, Miami, FL, USA)
Copyright: © 2017 |Pages: 23
DOI: 10.4018/JDM.2017100101


Business surveys indicate that fewer than 30% of data warehousing and business intelligence (DW/BI) projects meet the stated goals of the budget, schedule, and quality. Agile methods have been suggested as a possible solution, but because of the large size of the typical DW/BI project, it may be difficult to apply the agile values and principles. In this article, the following research questions are raised: Can agile practices be adapted for DW/BI development? What factors influence agile DW/BI development? Six semi-structured interviews were conducted using a questionnaire. The interview transcripts were coded using the grounded theory approach. Eight categories emerged from the analysis: business value, project management, agile development, shared understanding, technological capability, top management commitment, complexity, and organizational culture. Based on the categories, a research framework is proposed. The findings reveal that agile methods are suited for only certain aspects of DW/BI projects and need to be augmented with project management practices.
Article Preview


Business Intelligence and Analytics have emerged as an important area of study for both practitioners and researchers reflecting the magnitude and impact of data-related problems to be solved in contemporary business organizations (H. Chen, Chiang, & Storey, 2012). Data warehousing (DW) provides the foundation of this decision support infrastructure (Ariyachandra & Watson, 2010). Business analytics (BA) helps to understand the information contained in the data and to derive insights that are most important to future business decisions (Sharda, Delen, Turban, & King, 2015). BA subsumes Data Mining (DM), which employs analysis and artificial intelligence for nontrivial extraction of implicit, previously unknown and useful information using pattern recognition, statistical and mathematical techniques (Lee & Siau, 2001). Business Intelligence (BI) combines architecture, databases, data warehouses, analytical tools, and applications (Sharda et al., 2015). There is considerable overlap in the definition of BI and BA, or BI and analytics. This paper examines the use of agile development practices for DW/BI, which refers to business intelligence and analytics applications that employ data warehouses.

It has been found that 59% of BI projects fail and fewer than 30% of BI projects meet the objectives of the business ( last accessed on August 15, 2017). Sen, Ramamurthy, and Sinha (2012) ascribe the failures to lack of the maturity of the data warehousing processes. Takecian et al. (2013) assert that the traditional process for DW construction does not allow rapid and partial deliveries of functional features, and one of the most important causes for high failure of DW projects is the long development time, which leads to delays in delivery of functional features to the end-users. Often, when DW systems are finally available, some of the features implemented are already obsolete, while newer needs end up being postponed until future phases of development. Barrett and Barton (2006) state that a “big bang” approach to DW almost always ends in a disaster primarily because data warehouse projects do not scale well. The BI area also faces challenges because of the need to respond quickly to the large amount of external data that may need to be analyzed on a daily basis (Davenport, Barth, & Bean, 2012).

Data warehousing projects have typically been large and have always been difficult to develop and implement (Sen et al., 2012). In the earlier years of DW/BI, the platform investments were largely in IT-led consolidation and standardization projects for large-scale systems reporting. These projects tended to be highly governed and centralized, where IT-authored production reports were distributed to a broad array of information consumers and analysts (H. Chen et al., 2012) and a structured approach could still work. In recent years, the development of the DW/BI projects is facing additional challenges (Collier, 2011; Davenport et al., 2012). These include the size of the projects becoming even larger, more variety in the type of data stored, some of which are handled using NoSQL systems (Sadalage & Fowler, 2012), a wider range of business users demanding access to better predictions, and more interactive styles of analysis, insights and a dynamic environment leading to volatile demands.

Complete Article List

Search this Journal:
Open Access Articles
Volume 31: 4 Issues (2020): 1 Released, 3 Forthcoming
Volume 30: 4 Issues (2019)
Volume 29: 4 Issues (2018)
Volume 28: 4 Issues (2017)
Volume 27: 4 Issues (2016)
Volume 26: 4 Issues (2015)
Volume 25: 4 Issues (2014)
Volume 24: 4 Issues (2013)
Volume 23: 4 Issues (2012)
Volume 22: 4 Issues (2011)
Volume 21: 4 Issues (2010)
Volume 20: 4 Issues (2009)
Volume 19: 4 Issues (2008)
Volume 18: 4 Issues (2007)
Volume 17: 4 Issues (2006)
Volume 16: 4 Issues (2005)
Volume 15: 4 Issues (2004)
Volume 14: 4 Issues (2003)
Volume 13: 4 Issues (2002)
Volume 12: 4 Issues (2001)
Volume 11: 4 Issues (2000)
Volume 10: 4 Issues (1999)
Volume 9: 4 Issues (1998)
Volume 8: 4 Issues (1997)
Volume 7: 4 Issues (1996)
Volume 6: 4 Issues (1995)
Volume 5: 4 Issues (1994)
Volume 4: 4 Issues (1993)
Volume 3: 4 Issues (1992)
Volume 2: 4 Issues (1991)
Volume 1: 2 Issues (1990)
View Complete Journal Contents Listing