Machine Learning and Data Science Project Management From an Agile Perspective: Methods and Challenges

Machine Learning and Data Science Project Management From an Agile Perspective: Methods and Challenges

Copyright: © 2022 |Pages: 16
DOI: 10.4018/978-1-7998-7872-8.ch005
(Individual Chapters)
No Current Special Offers


Successful implementations of machine learning (ML) and data science (DS) applications have enabled innovative business models and brought new opportunities for organizations. On the other hand, research studies report that organizations employing ML and DS solutions are at a high risk of failure and they can easily fall short of their objectives. One major factor is to adopt or tailor a project management method for the specific requirements of ML and DS applications. Therefore, agile project management (APM) may be proposed as a solution. However, there is significantly less study that explores ML and DS project management from an agile perspective. In this chapter, the authors discuss methods and challenges according to the background information and practice areas of ML, DS, and APM. This study can be viewed as an initial attempt to enhance these knowledge and practice domains in view of APM. Therefore, future research efforts will focus on the challenges as well as the experimental implementation of APM methods in real industrial case studies of ML and DS.
Chapter Preview


Contemporary developments in cutting - edge technologies, along with the advances in Artificial Intelligence (AI), have paved the way for various integrated and sophisticated systems. The proliferation of AI systems has enabled new business models and brought opportunities for organizations. The successful implementations of AI can have a great impact on the activities and competitiveness of an organization. On the other hand, there is also an overestimation of their benefits, opportunities, and return on investment of AI projects. How to realize AI as a long - term and stable solution is difficult, and defining the required procedures is not clear. To that aim, Bughin et al. (2017) indicate the factors for transforming an organization towards AI as: a good and reasonable business case, “the adaptation of business processes to the capabilities of AI”, effective and efficient AI environment, techniques and tools, solid data ecosystem, and organizational culture. Ransbotham et al. (2017) report the challenges in the adoption of AI systems as follows:

  • Difficulties in the acquisition of AI - related skills and knowledge,

  • “Competing of AI projects with other projects in the company”,

  • Safety and security aspects of systems using AI,

  • Cultural and organizational barriers in the AI adoption,

  • Limited technological capabilities,

  • Lack of leadership and management support for AI initiatives,

  • Unclear business cases for AI projects.

In this context, Machine Learning (ML), which has a close relationship with the Data Science (DS) discipline, can be viewed as one of the most popular subfields of AI. However, recent studies reveal the facts about the failure of ML and DS projects, inadequate return on investment, and unsatisfactory results. It is reported that organizations employing AI solutions with big data are at a high risk of failure and may fall short of project objectives (Kelly & Kaskade, 2013). There may be, of course, various domain / process / technology - specific causes when we view this issue from the perspectives of different disciplines, such as DS, Computer Science (CS), and Software Engineering (SE). Ponsard et al. (2017) and Saltz et al. (2019) indicate that the majority of ML or DS research studies focuses on technical aspects rather than project management issues. More than 80% of data scientist state that an explicit process model is not followed, and they note that more systematic methods would improve their performances. Therefore, one major factor has been adopting or tailoring a project management method suitable for the idiosyncratic requirements of ML and DS applications (Saltz, 2015; Cao, 2017; Nalchigar et al., 2021). Studies refer to various issues, however, there is significantly less research, which explores the DS and ML project management from an agile perspective. The objective of this chapter is, therefore, to present and discuss the findings from the literature according to the solutions, background of the ML, DS, and Agile Project Management (APM) knowledge domains.

Key Terms in this Chapter

Idiosyncratic: To have characteristic attributes or behaviours different from others.

Model: Representation of a ML problem by using a learner algorithm, test, and training data.

Algorithm: A stepwise solution to a problem represented by visual, textual or computer programming codes.

Kanban: Signboard in Japanese.

Knowledge Discovery in Databases (KDD): A method for discovering valuable knowledge from various sources of data.

Training: A process whereby a learner algorithm maps a function to the training data.

Complete Chapter List

Search this Book: