Article Preview
TopIntroduction
Business Intelligence and Analytics have emerged as an important area of study for both practitioners and researchers reflecting the magnitude and impact of data-related problems to be solved in contemporary business organizations (H. Chen, Chiang, & Storey, 2012). Data warehousing (DW) provides the foundation of this decision support infrastructure (Ariyachandra & Watson, 2010). Business analytics (BA) helps to understand the information contained in the data and to derive insights that are most important to future business decisions (Sharda, Delen, Turban, & King, 2015). BA subsumes Data Mining (DM), which employs analysis and artificial intelligence for nontrivial extraction of implicit, previously unknown and useful information using pattern recognition, statistical and mathematical techniques (Lee & Siau, 2001). Business Intelligence (BI) combines architecture, databases, data warehouses, analytical tools, and applications (Sharda et al., 2015). There is considerable overlap in the definition of BI and BA, or BI and analytics. This paper examines the use of agile development practices for DW/BI, which refers to business intelligence and analytics applications that employ data warehouses.
It has been found that 59% of BI projects fail and fewer than 30% of BI projects meet the objectives of the business (http://www.silvon.com/blog/bi-initiatives-fail/ last accessed on August 15, 2017). Sen, Ramamurthy, and Sinha (2012) ascribe the failures to lack of the maturity of the data warehousing processes. Takecian et al. (2013) assert that the traditional process for DW construction does not allow rapid and partial deliveries of functional features, and one of the most important causes for high failure of DW projects is the long development time, which leads to delays in delivery of functional features to the end-users. Often, when DW systems are finally available, some of the features implemented are already obsolete, while newer needs end up being postponed until future phases of development. Barrett and Barton (2006) state that a “big bang” approach to DW almost always ends in a disaster primarily because data warehouse projects do not scale well. The BI area also faces challenges because of the need to respond quickly to the large amount of external data that may need to be analyzed on a daily basis (Davenport, Barth, & Bean, 2012).
Data warehousing projects have typically been large and have always been difficult to develop and implement (Sen et al., 2012). In the earlier years of DW/BI, the platform investments were largely in IT-led consolidation and standardization projects for large-scale systems reporting. These projects tended to be highly governed and centralized, where IT-authored production reports were distributed to a broad array of information consumers and analysts (H. Chen et al., 2012) and a structured approach could still work. In recent years, the development of the DW/BI projects is facing additional challenges (Collier, 2011; Davenport et al., 2012). These include the size of the projects becoming even larger, more variety in the type of data stored, some of which are handled using NoSQL systems (Sadalage & Fowler, 2012), a wider range of business users demanding access to better predictions, and more interactive styles of analysis, insights and a dynamic environment leading to volatile demands.