# Data Management Challenges in Agile Software Projects: A Systematic
Literature Review

Ahmed Fawzy  Amjed Tahir  Matthias Galster  Peng Liang

###### Abstract

Context: Agile software development follows an adaptive and iterative
approach. However, the management of data (e.g., development data or product
data) can pose significant challenges for projects and agile teams.

Objective: We aim to identify and characterize key challenges faced in data
management within agile projects and to examine potential solutions proposed
in the literature.

Method: We used a Systematic Literature Review (SLR) to collect and analyse
relevant studies. We identified 45 studies related to data management in agile
software development. We then manually analysed and mapped data from these
studies to categorise different data management aspects and identify
challenges and solutions as identified in those studies.

Results: Our findings reveal major challenges such as data integration and
quality assurance. We found implications of challenges on team members and the
product delivery process. We found that teams frequently struggle to integrate
heterogeneous data sources, ensuring data reliability and real-time analytics.
Additionally, fragmented data collection and a lack of standardized practices
can impede team collaboration and project transparency. The studies have also
proposed various solutions to address those challenges, including the use of
ontologies, diverse data management strategies, automated tools, and the
adoption of quality-focused development methods. Solutions also include
training to enhance data quality and analysis.

Conclusion: This SLR provides in-depth insights and recommendations for
practitioners, emphasizing the importance of robust data management
strategies. It suggests integrating advanced data management techniques into
agile frameworks to enhance decision-making and improve software project
outcomes. The study highlights the need for a more focused approach to data
management in agile environments, advocating tailored solutions to meet the
unique demands of agile software development.

###### keywords:

Data Management , Quality , Agile Software Development

††journal: Elsevier

[inst1]organization=School of Mathematical and Computational
Sciences,addressline=Massey University, country=New Zealand

[inst2]organization=Department of Computer Science and Software
Engineering,addressline=University of Canterbury, country=New Zealand

[inst3]organization=School of Computer Science,addressline=Wuhan University,
country=China

##  1 Introduction

Agile software development is a flexible and collaborative approach to
software development, focusing on iterative progress, customer feedback, and
adapting to change. It emerged as an alternative to rigid traditional methods,
emphasizing working software and user involvement over extensive
documentation. Agile values skilled, empowered teams and promotes responsive,
customer-centric development. Its principles, outlined in the 2001 Agile
Manifesto [11], prioritize individuals, working software, customer
collaboration, and adaptability over strict processes and plans. This approach
has made software development more efficient and responsive to market needs
[42].

In the realm of business success, data emerges as a paramount asset, playing a
pivotal role in informed decision-making and providing invaluable insights
into customer behavior and operational efficiency [49]. Business data, product
data, process data, project data, and operational data each contribute to the
success equation. However, managing data within agile projects presents
significant challenges that must be addressed for optimal organizational
success.

In addressing the intricate challenges of data management in agile software
development, we conducted a Systematic Literature Review (SLR) to investigate
in-depth common practices, issues, challenges and solutions. This is not
possible through the use of mapping reviews. The SLR facilitates an in-depth
exploration of these challenges, offering insights into the complexities of
managing data in an agile context, where rapid iterations and evolving
requirements are the norms. For instance, data integration challenges, such as
incorporating new data into existing processes and ensuring interoperability,
demand specific solutions like the use of intuitive tools and coordinated
efforts with relevant initiatives [62]. Similarly, data quality issues require
systematic collection and the employment of automated tools to ensure
completeness and accuracy [30] . Moreover, the analysis of large volumes of
structured and unstructured data for real-time recommendations necessitates
advanced technologies and agile approaches to analytics [17]. This review aims
to understand the challenges and solutions and provide insights and
recommendations for improved data management practices by answering the three
research questions (RQs):

  * RQ1. What are the data management aspects discussed in the context of agile software development?

  * RQ2. What challenges are faced regarding these aspects?

  * RQ3. What solutions are proposed to address these challenges?

Our results show that data management plays a pivotal role in agile software
development, emphasizing key aspects such as data integration, data
collection, data quality and data analysis. The results identify several data
management challenges, such as interoperability, semantic heterogeneity, and
the integration of data from diverse sources. We also highlighted several
solutions that have been suggested in literature such as employing ontologies,
implementing advanced Extract, Transform, Load (ETL) processes, decentralized
data management, and utilizing automation tools for enhanced data analysis.
Furthermore, the review emphasizes the impact of data management challenges on
decision-making, communication, and project efficiency in agile environments.

The remainder of this paper is structured as follows: Section 2 provides a
background of this work. Section 3 presents the methodology that has been
followed in the SLR. The key findings of this SLR are presented in Section 4,
which includes a categorization of data management aspects, interconnected
challenges and solutions in data management. Our findings are discussed in
Section 5, along with the implications of these challenges for agile teams and
the product delivery process. Section 7 then presents the conclusions of the
SLR.

##  2 Background

Data is an essential resource for organizations, as it significantly impacts
their success by facilitating informed decision-making and offering valuable
insights into customer behavior and operational efficiency [49]. In agile
development, the following examples illustrate the use of data:

Business Data: This term refers to information collected and used within a
business context to support decision-making processes [49]. For example, in
business data, the focus is on the use of Big Data to support strategic
business decisions, such as sales forecasts, predicting raw material needs,
and understanding customer behavior [23]. Matthies and Hesse [49] reported
that companies implementing data-driven decision-making (DDDM) experienced a
5-6 % increase in output and productivity, exceeding the expected outcomes
based on their use of information technology and investments. This highlights
the tangible benefits of utilizing business data to enhance company
performance. The study highlighted that in the field of software development,
examining data collected to obtain direct feedback from users or consumers
about their experiences with software releases is crucial. It enables prompt
feedback loops between customers and software developers. This process
facilitates the development of products that are more accurately tailored to
customer needs. Software developers can directly observe real user
interactions with the software and make well-informed decisions based on these
valuable insights.

Product Data: Focusing on the software product itself, encompassing elements
such as source code and design documents. It quantifies key attributes of the
software product, including its size, complexity, and design structure, all of
which significantly influence the quality of the software [54, 44, 20].

Process Data: Focusing on the software development process, this data
encompasses various aspects that could influence the effort required to design
a software system. Utilizing process data is crucial for understanding and
improving the software development lifecycle [54, 44].

Project Data: Focusing on a software project’s overall health and status. It
encompasses aspects such as project timelines, budgets, resources, and risks.
Project managers commonly use these data to monitor project progress and guide
decisions in project planning and control [20].

Operational Data: This data is related to the day-to-day operations of a
company. For instance, it involves managing personal information, privacy
engineering, and the deployment and operation of software in cloud
environments [32]. Operational data is integral to the daily operation of the
company and includes sensitive corporate information, classified documents,
and data stored in databases essential for enterprise operations [50].

These examples demonstrate how various forms of data may be utilized to
strengthen decision-making processes, promote customer involvement, and
optimize software development practices, ultimately leading to the success of
an organization.

Consequently, managing these pivotal data is essential. Data management refers
to the process of securely, efficiently, and cost-effectively gathering,
storing, and utilizing data. It guarantees that data is easily accessible,
reliable, and promptly available to its consumers. Data management is crucial
due to its capacity to facilitate decision-making, enhance business
performance, and enhance customer comprehension and service [3]. However, data
management within agile projects presents significant challenges.

The study of Graetsch et al. [31] illuminates various data management
challenges encountered in software development. They address the complexities
inherent in data integration, particularly the alignment of data from multiple
sources, which is further complicated by the challenges in understanding the
data. While agile methodologies, characterized by their iterative and
incremental nature [11], do not directly cause these integration challenges,
they can exacerbate the complexity if not managed effectively. This issue
arises from frequent changes and the need for continuous integration, which
are intrinsic to agile methodologies. Furthermore, Graetsch et al. [31]
identify challenges related to data analysis. Specifically, poor data quality
can hinder the analysis process and may lead to the development of solutions
that do not achieve the requisite predictive accuracy. This raises a
significant concern, as the focus of agile methodologies on delivering
functional software [42] might occasionally diminish the emphasis on data
quality, a factor essential for effective data analysis. The challenges also
extend to data collection and sharing. Graetsch et al. [31] noted that
obtaining access to data frequently requires authorization from data owners,
thereby increasing both time and complexity in the development process.
Although agile methodologies are known for promoting collaboration, they might
not fully address the complexities associated with data ownership and sharing,
particularly in a multidisciplinary environment.

Additionally, Graetsch et al. [31] highlighted prevalent issues in data
quality, such as missing data and the difficulties associated with integrating
data from various distributed sources. It is evident that agile methodologies
may not inherently resolve these data quality issues, suggesting the need for
a heightened focus on data management practices. Furthermore, Graetsch et al.
[31] discussed the necessity for substantial domain knowledge and specific
configurations to effectively utilize monitoring and visualization tools.
These tools do not automatically address data quality challenges, and agile
methodologies might not provide explicit guidance in these areas. Although
agile methodologies are proficient in managing software development processes
through their iterative and incremental approach, they may not adequately
address complex data management challenges in data-intensive software projects
and general software development. To address these challenges, agile teams are
recommended to adopt additional data-focused practices and tools that
complement the agile framework, ensuring robust data management throughout the
software development life cycle.

##  3 Methodology

###  3.1 Overview

We designed this review by following the Systematic Literature Review (SLR)
guidelines outlined by Kitchenham and Charters [41] and drew upon works
related to agile literature reviews, such as Dikert et al. [24] and Campanelli
and Parreiras [14], to inform our process. The review process is summarized in
Figure 1.

###  3.2 Search Process

Search Engine: We have chosen Scopus to locate relevant literature. This
decision is based on Scopus’ comprehensive indexing of major publications in
software engineering, as highlighted in studies by Mourão et al. and Carrera-
Rivera et al. [51, 16]. The extensive coverage of Scopus provides access to
high-quality, peer-reviewed literature, which is crucial for an in-depth
exploration of agile software development. Moreover, data management in agile
is a topic at the intersection of multiple disciplines (including software
engineering, project management, etc.). Scopus provides coverage of
publications on topics outside of computer science and software engineering
(compared to IEEE Xplore and ACM Digital Library). In our search, we
considered publications published before October 2023.

Search String: We constructed a search string covering the main keywords of
the studies we aim to review (i.e., data management, agile, and challenges).
These initial keywords were identified, and the search string was refined in
Subsection 3.4. We experimented with different search strings until we reached
a string that returned relevant results. The experimented process is discussed
in Subsection 3.4. Our search string is shown below:

(agile AND software AND data) AND (challenge OR practice) AND ("data quality"
OR "data management" OR "data security" OR "data governance" OR "data
integration" OR "data storage" OR "data privacy" OR "data access" OR "data
analytics" OR "data validation" OR "data capture" OR "data modeling" OR "data
virtualization" OR "data cataloging" OR "data Versioning" OR "data monitoring"
OR "data transformation" OR "data archiving" OR "backup" OR "disaster
recovery" OR "data-driven"

Figure 1: The review process

###  3.3 Selection and Exclusion Criteria

We selected studies that met all of the following inclusion criteria:

  1. 1.

Studies that discuss at least one aspect of data management in the context of
agile software development (including challenges and solutions).

  2. 2.

Studies where the full text is available for comprehensive analysis.

  3. 3.

Studies published in English.

We apply the following exclusion criteria:

  1. 1.

Studies that do not specifically mention agile software development or agile
methodologies.

  2. 2.

Duplicate studies to ensure the uniqueness of each data point.

  3. 3.

Studies that focus exclusively on technical or other aspects of software
development without demonstrating a clear link to data management challenges
in agile software development.

  4. 4.

Non-research materials such as tutorials, books, workshop summaries, or event
proceedings.

  5. 5.

Opinion pieces, editorials, or non-peer-reviewed articles to ensure the
academic rigor of the review.

###  3.4 Pilot Search Process and Search String Refinement

Before finalizing our search keywords and search string and establishing our
inclusion and exclusion criteria, we initiated a pilot search with a basic
string (“agile AND data”) in the Scopus database. This initial search
generated 9,900 studies, yet a review of a sample of titles and abstracts
revealed their lack of relevance to our focus on data management challenges in
agile software development, being too broad for our purposes.

To refine our search and secure more pertinent results, we expanded the string
by incorporating “challenge” and adjusting synonyms like “obstacle”, “issue”,
and “problem” with the OR operator. This refinement reduced the number of
results to 3,068 studies. However, a further examination of a new sample of
titles and abstracts showed that they were not explicitly related to software
development, as our research specifically targeted agile software development,
not agile methodologies in a broader sense.

We continued to refine our approach, modifying the search string to “(agile
AND data AND software) AND (challenge OR practice)”, which led to 1,396
studies. Yet, these results still did not focus explicitly on data management.
To address this, we further refined the search to “(agile AND data AND
software) AND (challenge OR practice) AND (“data management”)”, achieving more
targeted results. However, this specific approach risked missing relevant
studies where “data management” appeared in the context of its various aspects
rather than as an explicit term.

To overcome this, we expanded the search string to include a range of terms
associated with data management, resulting in a more balanced set of 181
studies as of the final execution in October 2023. To avoid missing related
studies by focusing narrowly on specific terms, we used Boolean operators (OR)
among them or included the general term “data management”. A random review of
sample titles and abstracts from these 181 studies demonstrated greater
relevance to our research question.

Before advancing 181 studies to the next stage, we verified that the search
strings effectively returned relevant studies and represented a robust search
strategy. We also incorporated studies known to be relevant into the dataset
sheet, complemented by an Excel formula to determine their presence in the
search results. These 181 studies proceeded to the next stage of review, as
illustrated in Figure 1. Additionally, we conducted a full pilot review
process on a sample of these studies. We applied inclusion and exclusion
criteria, performed quality assessments, and extracted results to ensure the
final selection was most aligned with our research objectives.

###  3.5 Data Filtration

The 181 studies were then screened, where their titles, abstracts, and
keywords were examined. Applying inclusion and exclusion criteria, we filtered
out 114 irrelevant studies, leaving 67 relevant studies. To ensure that these
studies precisely matched our research questions and scope, we conducted a
full-text review, applying the same criteria. This review led to the exclusion
of 22 studies from the initial 67. Reasons for exclusion included factors such
as language barriers (e.g., one study was in a non-English language, with only
the abstract in English), content type (e.g., one was a tutorial, another an
event), or lack of depth or relevance to our scope. Consequently, 45 studies
were included after a quality assessment Section 3.7.

###  3.6 Data Extraction

We then addressed RQ1 by exploring the data management aspects discussed in
the 45 studies. This exploration resulted in categories such as data
integration, data analysis, data quality, data collection, and other related
data management aspects, as shown in Table 2. In addition, the data types
discussed in the studies are provided in Table 1. Building on these aspects,
we explored the second and third RQs, focusing on identifying challenges and
their solutions in these areas. The findings from this analysis were then
synthesized and discussed in the discussion section, culminating in the
conclusion.

We provide a full dataset of all articles we have extracted together with our
detailed analysis in our replication package [29].

Table 1: Types of Data Discussed in the Studies Included in the SLR

Process Data and Project Data | Operational Data and Business Data | Process Data and Product Data
---|---|---
[15] | [53] | [17]
[6] | [59] | Project Data
[38] | [8] | [46]
[10] | [62] | Product Data and Project Data
[5] | [12] | [57]
[40] | Operational data | Product Data and Operational Data
[48] | [32] | [34]
[45] | [22] | Product Data
[9] | [26] | [19]
[43] | [50] | Business data
[28] | [4] | [56]
[27] | [52] | [23]
[36] | [55] | [21]
[35] | [58] | [61]
[25] | [39] | [37]
[30] | Process Data , Project Data and Product Data |
[47] | [60] |
[33] | Process Data , Product Data , and Operational Data |
[49] | [18] |

###  3.7 Quality Assessment

Throughout the stages of our review, we have conducted comprehensive quality
checks to ensure rigor, integrity, and relevance. This process began with
examining selected samples of search string results, aligning them carefully
with our established inclusion and exclusion criteria to ensure the focus and
relevance of our study. Furthermore, we conducted an in-depth quality
assessment of the 45 studies, employing a streamlined, three-point scoring
system across six key criteria [2]: Clarity of Objectives, Appropriateness of
Methodology, Adequacy of Data Analysis, Relevance to Research Questions (RQs),
Rigor of Data Collection, and Quality of Reporting. Each criterion is rated 1
(adequate), 2 (good), or 3 (excellent), depending on the study’s clarity,
methodological soundness, analytical rigor, relevance, data collection
thoroughness, and quality of reporting. We calculate the total score for each
study by summing the scores from all criteria, with the maximum possible score
being 18. The studies are then categorized according to their total scores,
with 7-11 indicating adequate quality, 12-15 indicating good quality, and
16-18 representing excellent quality. This meticulous approach ensures a
comprehensive and nuanced assessment of each study’s quality, aligning with
our goal to exclude low-quality research and thereby maintaining the integrity
and reliability of our findings. The quality assessment confirms that the 45
studies are of good quality based on their clarity of objectives,
appropriateness of methodology, adequacy of data analysis, relevance to
research questions (RQs), the rigor of data collection, and quality of
reporting. The details of the quality assessment can be found in the shared
dataset [29].

##  4 Results

The results section provides a detailed analysis of different aspects of data
management in agile software development. First, we define those different
aspects and then discuss the challenges and solutions associated with the top
aspects that we identified. We answer RQ1 in Section 4.1. RQ2 and RQ3 are
discussed across Sections 4.3.1, 4.3.2 , 4.3.3 and 4.3.4.

###  4.1 Data Management Aspects (RQ1)

Upon reviewing the studies that discuss various aspects of data management, as
referenced in Table 2, we have derived the following definitions for each
aspect.

Data Integration refers to the process of combining data from various sources
and systems into a unified and consistent view. This consolidated data is then
utilized for analysis, reporting, and decision-making purposes, as referred to
in the studies that discussed the data integration aspect in Table 2.

Data Collection is an activity that involves gathering and distributing
information. This information is subsequently utilized for analysis and
decision-making, as mentioned in the studies discussed data collection aspect
in Table 2.

Data Quality is defined by the accuracy, completeness, and consistency of a
dataset. The importance of high-quality data is underscored by its critical
role in effective data analysis and decision-making processes. This is
according to the studies that discussed the data quality aspect in Table 2.

Data Analysis is a comprehensive activity that includes inspecting, cleaning,
transforming, and modeling data to unearth valuable information, obtain
conclusions, and strengthen decision-making. This activity necessitates the
management of diverse data types, including structured and unstructured data,
and requires diverse techniques and tools. This is according to the studies
that discussed the data analysis aspect in Table 2.

Distribution of Studies across Data Management Aspects: We classified each of
the 45 studies based on the data management aspects they address. Table 2
lists all 15 data management aspects across studies. Figure 2 shows the
distribution of studies across the fifteen data management aspects we
identified.

Due to space limitations, we have provided the definitions of the other 11
data management aspects, along with their challenges and solutions, in the
replication package. Interested readers can refer to the replication package
[29]

Figure 2: Total number of studies discussing challenges and solutions
associated with fifteen aspects of data management Table 2: Classifications of
Studies in Data Management Aspects

| Data Management Aspect |  |  | Ref. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
1 | Data Integration | [17] | [34] | [25] | [18] | [60] | [4] | [9] | [55] | [61] | [58] | [39] | [36] | [56] | [46] | [57] | [23] | [26] | [62] |
2 | Data Storage | [17] | [34] | [8] | [9] | [57] | [23] | [37] | [26] |  |  |  |  |  |  |  |  |  |  |
3 | Data Validation and Governance | [17] | [40] | [47] | [36] | [35] |  |  |  |  |  |  |  |  |  |  |  |  |  |
4 | Data Quality | [30] | [53] | [59] | [34] | [10] | [6] | [47] | [43] | [58] | [39] | [36] | [56] | [46] |  |  |  |  |  |
5 | Data Ingestion | [17] | [25] | [22] | [21] | [43] | [37] |  |  |  |  |  |  |  |  |  |  |  |  |
6 | Data Collection | [53] | [49] | [59] | [18] | [25] | [27] | [60] | [21] | [10] | [8] | [40] | [48] | [47] | [52] | [43] | [23] | [45] | [37] | [28]
7 | Data Security and Compliance | [60] | [50] |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
8 | Data Privacy | [15] | [32] | [61] | [8] | [52] | [57] |  |  |  |  |  |  |  |  |  |  |  |  |
9 | Data Analysis | [17] | [34] | [25] | [49] | [59] | [18] | [21] | [10] | [48] | [52] | [9] | [43] | [55] | [23] | [45] | [26] | [33] | [28] | [27]
10 | Data Visualization | [34] | [59] | [47] | [55] | [39] | [45] | [33] | [28] | [43] |  |  |  |  |  |  |  |  |  |
11 | Data-Driven Decision Making | [53] | [25] | [49] | [59] | [9] | [12] | [45] |  |  |  |  |  |  |  |  |  |  |  |
12 | Data-Driven Development | [8] | [5] | [52] | [26] |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
13 | Data Testing: | [60] | [10] | [6] | [46] |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
14 | Data maintenance strategy | [6] | [61] |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
15 | Data Management Development | [35] | [38] | [19] | [26] | [17] | [18] |  |  |  |  |  |  |  |  |  |  |  |  |

###  4.2 Studies Discussing the Same Data Management Aspects (Overlapping)

####  4.2.1 Studies Discussing Data Collection and Analysis:

We observed the most significant overlap between data collection and analysis
in seven studies focused on their challenges and solutions, as illustrated in
Figure 3. This overlap suggests a potential link between these aspects. For
example, the Analytics-Driven Testing (ADT) process proposed by Batarseh and
Gonzalez [10] addresses the challenges in both data collection (unstructured
data) and analysis (extracting insights from unstructured data), showcasing
the link between these aspects in agile software development and decision-
making. Another example is the solution provided by Matthies and Hesse [49],
which addresses both the data collection challenges (variety and volume of
sources) and the data analysis challenges (integration and interpretation of
diverse data). It demonstrates how effective tool usage can bridge the gap
between collecting and analyzing a wide range of data to derive actionable
software development insights.

####  4.2.2 Studies Discussing Data Integration and Data Quality:

Spengler et al. [58] show that addressing data integration challenges, as
outlined in Section 4.3.1, inherently resolves data quality issues, which are
discussed in Section 4.3.3. The solution presented a cloud-based platform that
includes a generic agile data-loading pipeline to integrate data from
distributed and heterogeneous sources. Additionally, this pipeline is designed
to automatically detect the syntax and format of input data, handle different
encodings, and manage missing and duplicate data. Another example, the
solution proposed by Rosenkranz et al. [56] addresses the challenges in both
data integration (harmonizing data from different sources) and data quality
(ensuring accuracy and consistency), highlighting the interconnected nature of
these two aspects in effective data management.

####  4.2.3 Studies Discussing Data Integration and Analysis:

The studies depicted in Figure 3 discussed challenges and solutions related to
data integration and analysis, underscoring their role in agile project value
delivery, characterized by swift, flexible responses to change. For example,
the solution proposed by Chen et al. [17] (adopting an architecture-centric
approach) addresses the challenges in both data integration (combining diverse
data types from Enterprise Data Warehouses (EDWs) and Not Only SQL (NoSQL)
systems) and data analysis (applying real-time analytics), while embedding
these processes within an agile framework. This approach supports agile’s
flexible data structure needs for continuous delivery. The interplay between
data integration and analysis in agile development is essential for enabling
prompt, informed decision-making, adapting swiftly to changes, and maintaining
flexible development.

RQ1 Findings
We identified 15 aspects related to data management in agile software
development. Among those, data integration, data collection, data quality and
data analysis are the most discussed aspects in the literature.

Figure 3: Overlapping Studies Between the Key Focus Aspects in Data Management

###  4.3 Data Management Challenges and Solutions (RQ2 and RQ3)

Below, we discuss all the challenges and solutions identified in the studies.
Note that we discuss the challenges and solutions together, as this will
provide the reader with a better context of each challenge and the related
solution as discussed in the associated study. Those challenges and solutions
are discussed across the different aspects identified in Section 4.1. Given
the space limitation, we only report those for the top aspects we identified
in the previous section but provide our full comprehensive analysis online
[29].

####  4.3.1 Data Integration Challenges and Solutions

We have identified several challenges associated with data integration in
agile. These challenges and their proposed solutions are discussed in the
following subsections. A summary of these challenges and their corresponding
solutions are presented in Table 3.

Table 3: Summary of Data Integration Challenges and Solutions

Data Integration Challenge | Data Integration Solution | Solution Status | Ref.
---|---|---|---
Data Harmonization and Interoperability | Development of ontologies for MRO systems (ODBI) | Proposed | [4]
| Cloud-based platform with agile data-loading pipeline | Implemented | [58]
| Agile workflow for integrating climate data (AMCDS toolbox) | Proposed | [62]
| Translating the metadata of the datasets into HL7 FHIR profiles standard | Implemented | [57]
Semantic Heterogeneity | Communication-centric approach with agile methods | Implemented | [56]
| Development of ontology-based approach ’Immigrant’ | Proposed | [25]
| Development of domain ontologies for CSE tool integration | Implemented | [9]
| Implementing a modular and agile information processing framework | Implemented | [55]
Data Transformation and Extraction | Improvement of DBpedia Information Extraction Framework (DIEF) | Implemented | [36]
| Data visualization and federation layer | Proposed | [26]
| Replicable ETL process for EHR into EDW | Implemented | [39]
| Shift to advanced analytic platforms | Proposed | [23]
Managing Data Integration | Adoption of ‘data mesh’ approach | Proposed | [61]
| Adoption of the Microsoft Solutions Framework (MSF) | Implemented | [46]
| Architecture-centric approach with AABA methodology | Implemented | [17]
Diverse and Decentralized Data Sources | Common product line architecture for data cleansing and integration | Implemented | [34]
| Automated Continuous Quality (ACQ) Metrics dashboard | Further implementation needed | [18]
| Solution referenced in 4.3.2 | Further implementation needed | [60]

Data Harmonization and Interoperability: Abdallah and Fan [4] discussed the
challenge of harmonization and interoperability between heterogeneous
Maintenance Repair and Overhaul (MRO) information systems, which is a key
issue in integrating maintenance records over the lifespan of an aircraft.
Spengler et al. [58] discussed the challenge of integrating data from
distributed and heterogeneous sources at the technical, structural, and
semantic levels. Vøgt et al. [62] addressed incorporating new climate data
into local government processes. Schüttler et al. [57] focused on coordinating
the definition and integration of datasets to ensure interoperability and
avoid the development of parallel structures, necessitating alignment with
other research infrastructures like the European Biobanking and BioMolecular
Resources Research Infrastructure - European Research Infrastructure
Consortium (BBMRI-ERIC) and the German Medical Informatics Initiative (MII),
which is a national initiative aimed at improving medical research and
healthcare by facilitating the sharing and integration of data across various
medical institutions.

To address data harmonization and interoperability challenges Abdallah and Fan
[4] proposed a solution, which is the development of ontologies to
semantically integrate heterogeneous maintenance records of MRO systems,
enabling consistent understanding and data exchange across different systems.
This approach, known as Ontology-Based Data Integration (ODBI), is supported
by the Agile Development for Ontology-Based Applications (ADOBA) methodology,
which combines ontology development with application integration in an agile
framework. The solution was conceptualized and detailed within the framework
of the ADOBA methodology. However, Abdallah and Fan [4] indicated that further
validation is necessary to assess the accuracy and applicability of the
methodology in developing ontology-based applications. Spengler et al. [58]
presented a cloud-based platform as a solution that includes a generic agile
data-loading pipeline. This pipeline is designed to support deploying and
customizing clinical and translational data warehousing solutions,
specifically i2b2 and tranSMART. It addresses the need for technical and
medical expertise in setting up secure data warehousing systems and managing
multiple warehouse instances for various data-driven research projects. The
platform simplifies the complex installation process and enables rapid
instantiation of new instances of data warehousing systems, supporting both
i2b2 and tranSMART. Additionally, this pipeline is designed to automatically
detect the syntax and format of input data, handle different encodings, and
manage missing and duplicate data, thus significantly reducing the efforts
required for data cleansing and preprocessing. The solution was implemented
and evaluated.

Vøgt et al. [62] proposed the development of an agile workflow to integrate
climate data into the administrative process of urban land use planning,
illustrated through the City of Constance as a case study. This process
involves incorporating the Advanced Municipal Climate Data Store (AMCDS)
toolbox to facilitate data-informed decision-making across various activities,
including analyzing the current situation and its effects, devising measures,
testing and adapting them, and monitoring and assessing progress. The solution
was developed and proposed, and the next steps involved presenting the
workflow to administrative staff in participatory and co-creative workshops to
gather feedback, which would then be integrated into an adjusted workflow.
Sch”uttler et al. [57] proposed a solution to ensure the interoperability of
the collected data by closely coordinating the definition and integration of
datasets with MII. This process included translating the metadata of the
sample, donor, and biobank datasets into HL7 FHIR profiles (Health Level Seven
Fast Healthcare Interoperability Resources), which standardized the relevant
data within the established IT infrastructure and made them easier to query.
Furthermore, the collaboration with BBMRI-ERIC resulted in significant
synergies, enabling the quick technical integration of tools within the IT
infrastructure, leading to international visibility and a high degree of
interoperability. The solution was implemented and evaluated.

Semantic Heterogeneity: refers to the challenge of integrating data with
divergent interpretations and meanings. Rosenkranz et al. [56] highlighted the
problem of semantic heterogeneity during the design of data integration
requirements in the development of data warehouses (DW). Dos Santos Júnior et
al. [25] discussed the difficulty in accessing, integrating, analyzing, and
viewing data across heterogeneous applications with varying semantics.
Abdallah and Fan [4] discussed semantic heterogeneity in heterogeneous MRO
systems. Barcellos and Perini [9] discussed the difficulty of integrating data
from different agents and tools due to semantic heterogeneity. Rix et al. [55]
discussed the difficulty of integrating and making sense of diverse data
sources with varying formats and meanings within the high-pressure die casting
(HPDC) manufacturing process.

To resolve the semantic heterogeneity challenges, Rosenkranz et al. [56]
proposed a communication-centric approach that combines agile software
development practices with communication theory. This approach includes three
core artifacts: a detailed template for data field specifications, a procedure
model for iterative development, and the use of agile methods to facilitate
continuous stakeholder communication. These components work together to
enhance stakeholder comprehension and ensure consistent data usage across the
organization. The solution was implemented and evaluated. Dos Santos Júnior et
al. [25] proposed developing an ontology-based approach known as Immigrant as
a solution. By mapping the diverse semantics of various applications onto a
common ontology, Immigrant seeks to resolve semantic conflicts and provide a
cohesive and integrated view of the data. This enables the presentation of
meaningful information through dashboards, facilitating data-driven decision-
making in agile software organizations. The approach is currently in
development, with plans for a proof of concept to validate its effectiveness.
In addition to the data harmonization and interoperability challenge addressed
by Abdallah and Fan [4] suggested the development of ontologies discussed
above to resolve data harmonization, interoperability, and semantic
heterogeneity in heterogeneous MRO systems.

Barcellos and Perini [9] proposed using ontologies to establish a common
conceptualization across different tools and data sources. This approach helps
to reduce semantic conflicts and enables correct data integration by acting as
an interlingua to map the concepts used by various agents and tools, thus
facilitating understanding and communication. They developed and used domain
ontologies as reference models to integrate tools that support Continuous
Software Engineering (CSE) processes. They started by developing and
integrating a Scrum Reference Ontology with ontologies from the Software
Engineering Ontology Network (SEON), covering requirements and project
management aspects. This ontology was then applied as a basis to integrate
tools such as Clockify and Azure DevOps used by development teams in the
software unit of a Brazilian government agency. As a result, data from
different tools were integrated and displayed in dashboards, providing useful
information for managers to make decisions. Rix et al. [55] proposed a
solution that implemented a modular and agile information processing
framework. This framework utilizes annotation services, which add metadata to
data sources, and a Master Data Management (MDM) repository, a centralized
database that stores and manages master data. It enables real-time data
streaming for data mining analyses and visualization of results. The framework
ensures that data from various sources are annotated with metadata,
facilitating pervasive traceability and simplifying information aggregation.
Additionally, using dynamic Object-Relational Mapping (ORM) frameworks, such
as Hibernate, in conjunction with open interfaces like OPC-UA (Open Platform
Communications Unified Architecture), addresses semantic heterogeneity. This
is accomplished by enabling the labeling and exchange of information in a
unified manner. The solution was implemented and evaluated at the Audi testing
foundry.

Data Transformation and Extraction: Data transformation and extraction involve
converting and transferring data from various sources into a usable format for
analytics and decision-making. Hofer et al. [36] the challenge of extracting
and refining data from semi-structured, mixed-quality crowd-sourced data such
as Wikipedia and Wikidata. Dursun et al. [26] emphasized the need for a
unified view of data from various sources and to integrate data from
heterogeneous stores into a single, coherent data store for analytics and
visualization. Kannan et al. [39] discussed the importance of an efficient ETL
process for extracting data from Electronic Health Records (EHR) into an
Enterprise Data Warehouse (EDW). Dharmapal et al. [23] highlighted the
complexity of integrating diverse data types without complex, time-consuming
IT engineering efforts and suggested that modern analytic processing tools
should transition away from traditional, retrospective BI tools and platforms
towards more progressive analytic platforms.

To address the data transformation and extraction challenges, Hofer et al.
[36] proposed a solution for improving the DBpedia Information Extraction
Framework (DIEF) through the implementation of advanced ETL procedures. This
approach included a systematic and test-driven method for handling data and
code-related issues using Linked Data, which enhanced traceability and issue
management. Additionally, the solution introduced two key technical
enhancements: explicitly associating data artifacts with the corresponding
code and implementing a library for continuous testing and validation of the
data extraction process. The solution was implemented and evaluated. Dursun et
al. [26] proposed implementing a data visualization layer to simplify user
access to data, thereby abstracting the complexities of different storage
structures and technologies. In addition, it suggested the establishment of a
data federation layer to integrate data from various autonomous stores into a
single cohesive data store for users. Furthermore, the proposed solution
included an extensive modeling process. This process encompasses defining the
goal to comprehend the business value, preparing the data, selecting the type
of data-driven analytics, choosing suitable tools such as data mining, machine
learning, optimization, and fuzzy inference systems, selecting tasks and
algorithms from each tool, building the model, validating the model, and
ultimately deploying the model. Kannan et al. [39] proposed developing a
replicable ETL process that allowed the extraction of data from the EHR into
the EDW using standard EHR fields and a core set of EHR structures for
capturing custom data. This process supported the parallel development of EHR-
based registries and associated data collection tools. The solution was
implemented and evaluated. Dharmapal et al. [23] suggested a shift from
conventional, retrospective business intelligence (BI) tools towards more
advanced analytic platforms that support seamless integration with diverse
data sources, including external ones. It further recommended adopting an
agile methodology to adapt flexibly to changes in data sources, methodologies,
algorithms, or tools to achieve business goals.

Managing Data Integration: Vestues et al. [61] discussed the difficulty of
managing data across various silos within the organization and supporting
analytical solutions that require cross-organizational insights. Little et al.
[46] mentioned that a key business value of Landmark Graphics (a leading
supplier of software and services) application suite arises from the
integration of these products through a common data model with over 800
tables, 12 major entities, and 90 data types. This complexity presents a
challenge in managing and integrating data across different products. Chen et
al. [17] discussed the challenge of effectively managing internal and external
data integration, including data from existing Enterprise Data Warehouses
(EDW) and new NoSQL systems, in big data projects.

To confront the challenges related to managing data integration, Vestues et
al. [61] proposed the adoption of a “data mesh” approach. This approach
comprises four core principles: domain-oriented decentralized data ownership
and architecture, data as a product, self-serve data infrastructure, and
federated computational governance. The data mesh model advocates for a shift
from centralized data management to a distributed model where domain teams
have increased ownership of the data produced by their applications, aiming to
improve coordination and enable more agile and automated approaches to data
analytics. Vestues et al. [61] focused on reporting findings from a case study
of a public sector organization in Norway that has begun the transition from
centralized to distributed data management. Little et al. [46] presented that
Landmark Graphics adopted the Microsoft Solutions Framework (MSF), a
milestone-based iterative development framework, to standardize their
development process across different product teams. This standardization was
aimed at supporting the development of integrated products and ensuring
consistency in the integration efforts, which was crucial given the complexity
of their common data model. Chen et al. [17] adopted an architecture-centric
approach, a methodology that strongly emphasizes designing and developing a
system’s architecture as a crucial element of the development process. This
approach entails the creation of a well-defined architecture, which acts as a
blueprint for the system’s development. It guides the selection of
technologies, allocation of resources, and risk management throughout the
development life cycle. This approach complements agile’s modular design, a
software development approach that emphasizes breaking down a project into
smaller, more manageable modules or components to manage the integration of
diverse data types effectively. The approach includes using data lakes and
cloud storage to handle various data types, supporting agile’s requirement for
flexible data structures and continuous delivery. The resulting AABA
(Architecture-centric Agile Big Data Analytics) methodology was implemented
and validated through multiple case studies encompassing 10 big data analytics
projects.

Diverse and Decentralized Data sources: Upender and Barg [60] discussed the
complexity involved in integrating patient data and transitioning to a
centralized data management system within a diverse and decentralized user
environment. Harper and Dagnino [34] highlighted the need to merge and clean
data sets from different sources to enable advanced data analytics. The
challenge is to initiate and sustain processes while ensuring the exchange of
data and requests between components. The proposed solution is to distribute
work requests across computing resources. Chhillar and Sharma [18] discussed
the challenge of consolidating reports from various sources, such as
development, testing, and bug-tracking tools, into a unified data analytics
tool. This challenge pertains to seamlessly integrating diverse data sources
and formats to generate real-time metrics and reports for software quality
analysis.

To address the diverse and decentralized data sources challenges, Harper and
Dagnino [34] proposed developing a common product line architecture with
built-in capabilities for data cleansing and integration, designed to serve a
wide range of domains and reduce startup costs for new analytics applications.
The solution was implemented as part of an agile evolutionary approach to
build the advanced analytics product line architecture for various industrial
application scenarios at ABB (ASEA Brown Boveri), a multinational corporation.
Chhillar and Sharma [18] proposed an Automated Continuous Quality (ACQ)
Metrics dashboard. This dashboard integrates data from various sources, such
as functional, performance, security testing, Continuous Integration and
Deployment (CI/CD) build details, and development reports into a data
analytics tool. This integration facilitates the auto-generation of quality
metrics, enabling real-time tracking and graphical representation of metrics.
Chhillar and Sharma [18] also noted that further endeavors are necessary for
its implementation. Upender and Barg [60] proposed a solution, discussed in
Section 4.3.2.

####  4.3.2 Data Collection Challenges and Solutions

We have identified multiple challenges associated with Data collection in
Agile development. These challenges and their proposed solutions are discussed
in the studies reviewed below. A summary of these challenges and their
corresponding solutions is presented in Table 4.

Table 4: Summary of data collection challenges and solutions

Data Collection Challenge | Data Collection Solution | Solution Status | Ref.
---|---|---|---
Capturing Diverse Data | User-centered design strategies for data collection | Implemented | [53]
| Continuous Integration servers, static analysis tools, and other software development tools | Proposed | [49]
| Analytics-Driven Testing (ADT) process | Proposed | [10]
Data Collection Method Challenges | Automated Continuous Testing (ACT) TestBot and ACQ Metrics dashboard | Implementation requires more effort | [18]
| Automation of data collection and visualization tool chain | Proposed | [43]
| Human-centred Agile Software Engineering (HASE) platform | Proposed | [45]
| Automated methods for data collection and visualization | Proposed | [59]
| Centralized data management system with Scrum and XP practices | Proposed | [60]
Data Sharing and Collaboration | Diagnostic model for understanding data collection and sharing practices | Proposed | [27]
| Data-Driven Systems Engineering (DDSE) methodologies | Proposed | [40]
| Legal advisor for data sensitivity assessment | Proposed | [8]
Informative Data collection challenges | Development of ontology-based approach ’Immigrant’ | Proposed in Section 4.3.1 | [25]
| Integration of development project data into Retrospective agendas | Proposed | [48]
| Q-Rapids tool for acquiring valuable information | Implemented | [47]
| Agile methodology and Data Analyst collaboration for data collection | Proposed | [23]
Comprehensive Data Collection | Two-fold approach: Creation of a generalized dataset and standardized data processing | Proposed | [21]
| Big data stack with scalable storage technologies | Proposed | [37]
| Project Management Information System (PMIS) for automated data collection | Proposed | [28]
| Automated data collection mechanisms from software in use | Proposed | [52]

Capturing Diverse Data: Pater et al. [53] focused on capturing data for
personalized healthcare and human services while addressing barriers to
widespread adoption and standardization of information systems within
healthcare. Matthies and Hesse [49] discussed the need for effective data
collection and analysis from diverse sources, including software project data,
customer feedback, and operational metrics. Furthermore, Batarseh and Gonzalez
[10] highlighted the issue of data is often not collected in a structured and
organized manner, making it difficult to analyze and use for decision-making.

To address capturing diverse data challenges, Pater et al. [53] proposed a
solution that involves leveraging a variety of user-centered design
strategies, such as participatory design, empathic design, and design
thinking, to guide the data collection methods. This approach maintains the
user at the center of the process and continuously incorporates user-driven
input into each phase of the agile development cycle. The solution was
implemented to modernize the Army Community Service’s information technology
infrastructure. Matthies and Hesse [49] highlighted the presence of tools such
as Continuous Integration servers, static analysis tools, and other software
development tools that provide valuable data points on the current status and
health of the developed software project. They highlighted that the
combination of data analysis and interpretation by agile teams can enable
better-informed decisions, both in the domain of business as well as software
development processes. Batarseh and Gonzalez [10] proposed the Analytics-
Driven Testing (ADT) process. ADT involves a structured data collection and
analysis approach within the agile software development lifecycle. During each
sprint, data are systematically collected using the ADT tool. This includes
recording all software failures, measuring the Mean Time Between Failures
(MTBF), and collecting other relevant data such as software module, sprint
number, and run time.

Data Collection Method Challenges: Chhillar and Sharma [18] discussed the
difficulty in generating real-time software quality metrics and the need for
actual data collection processes. In contrast, Lehtonen et al. [43] discussed
the manual nature of data collection from the issue management system, which
can be time-consuming and susceptible to errors. Upender [60] discussed the
challenge involves creating a system that allows for the electronic capture of
clinical research data, replacing the existing rudimentary tools such as Excel
and Access that various independent research groups use. Lin et al. [45]
discussed the difficulty of collecting data on software engineering
activities, such as task allocation, collaboration, and mood stability,
without disrupting the normal Agile Software Development (ASD) process.
Practitioners surveyed by Svensson et al. [59] expressed concerns regarding
data availability, excessive quantity, and ambiguity in terms of usability,
relevance, and its connection to decision-making.

To overcome the challenges related to the data collection method, Chhillar and
Sharma [18] proposed an Automated Continuous Testing (ACT) TestBot and ACQ
Metrics dashboard. The ACT TestBot automates the execution of tests and the
monitoring of application logs, while the ACQ Metrics dashboard aggregates and
visualizes the data, enabling the automatic generation of quality metrics
reports. This integrated approach ensures the reliability of data and
facilitates the real-time tracking of software quality. Chhillar and Sharma
[18] noted the implementation requires more effort. Lehtonen et al. [43]
recommended automating the data collection by implementing a toolchain that
covers all steps from the initial data collection to the visualization of the
data. This automated approach would enhance the accuracy and efficiency of
data collection, thereby improving the quality of the visualizations used to
analyze the software development process. Additionally, the development of an
interactive visualization tool was suggested to recognize patterns and
anomalies in the process, facilitating a deeper understanding and validation
of the evolution towards a more agile process.

Lin et al. [45] presented a Human-centred Agile Software Engineering (HASE)
platform. The HASE platform is an online Agile Project Management (APM) tool
allowing unobtrusive data collection during the normal ASD process. It
collects data on software engineering activities from the participants, such
as task allocation, collaboration, and mood stability, without requiring
additional effort from the participants. This approach ensures that the data
collection process does not disrupt the regular workflow of the software
engineering teams while still providing accurate and objective data for
assessing software engineering skills. Svensson et al. [59] underscored that
future research should develop novel automated methods for collecting,
analyzing, and visualizing data to enhance existing agile decision-making
processes by associating pertinent data with specific decision scenarios.
Upender [60] proposed a three-year project to build a centralized data
management system that addresses the needs of electronically collecting and
integrating data with a gradual adoption and adaptation of Scrum and XP
(Extreme Programming) practices. They emphasized a ”just in time design”
approach, which meant they approached the design phase with a flexible
mindset. They used an evolutionary design process to handle ambiguous and
conflicting requirements. Initially, they built a basic version of the
clinical application to ensure it had essential functions, leaving the
implementation of more advanced features for future updates. At the time the
study was written, they were two years into the project.

Data Sharing and Collaboration: Fabijan et al. [27] noted the lack of sharing
of customer and product data among agile team members. This leads to
difficulties in accessing data collected by others in different development
phases and fragmented collection and storage of data. Kaur et al. [40]
pinpointed the absence of concurrent access as a significant challenge,
leading to unnecessary rework due to human error and inefficient communication
methods. Barbala et al. [8] discussed the challenge of the uncertainty
surrounding what data agile product teams are allowed to gather.

To tackle the data sharing and collection challenges, Fabijan et al. [27]
developed a model that serves as a diagnostic tool to understand data
collection and sharing practices within software development organizations.
This model identifies the types of data collected, the parties responsible for
collection, and the development phases in which the data is used. It also
highlights critical handovers where data loss can occur. The model can provide
valuable insights for companies deciding on actions to improve their data
sharing and collection practices. This can aid in addressing the fragmented
collection and storage of data, ultimately leading to better data analysis and
utilization. Kaur et al. [40] recommended using Data-Driven Systems
Engineering (DDSE) methodologies that facilitate concurrent access, thereby
enabling real-time collaboration and information exchange. DDSE tools manage
engineering data during the implementation phase, provide version control,
make it available collaboratively, and ensure full traceability for the entire
engineering team. Barbala et al. [8] presented that including a legal advisor
in the team helped assess the sensitivity of data when collecting, sharing,
and storing it and provided guidance on compliance with data protection
regulations.

Informative Data collection challenges: Dos Santos Júnior et al. [25] delved
into difficulty in accessing, integrating, analyzing, and viewing data handled
by heterogeneous applications, which often adopt different semantics (the same
information item is given divergent interpretations), posing a barrier to
integrated data usage. Matthies [48] discussed the challenge of relying on
subjective data gathered from team members’ perceptions and experiences during
Retrospective meetings. This subjective data, while easily available and
relevant to team satisfaction, may not provide a comprehensive or objective
view of the team’s performance and areas for improvement. Martínez-Fernández
et al. [47] addressed the data collection challenge of obtaining informative
user data. Dharmapal and Sikamani [23] underscored the significance of
collecting data from the right sources and filtering out unwanted data in the
development phase.

To resolve or mitigate the informative data collection challenges, Dos Santos
Júnior et al. [25] proposed a solution ” Development of ontology-based
approach ’Immigrant’ ” discussed in Section 4.3.1. This solution not only
resolved the semantic heterogeneity data integration challenge but also
assisted in collecting informative data that can be used for data-driven
decision-making in agile software development. Matthies [48] recommended
integrating development project data and insights from software repository
mining into Retrospective agendas. By employing project data from the last
iteration in Retrospective meetings, teams can gain additional insights based
on their team-specific data. This approach aims to provide a more thorough
overview of the team’s state, including facts and feelings, which can lead to
better results in Retrospectives. The use of project data measurements can
help track progress on common agile and software engineering challenges,
thereby enabling teams to identify and tackle improvement actions based on
empirical project evidence rather than solely on anecdotal experiences.

Martínez-Fernández et al. [47] proposed the utilization of the Q-Rapids tool
for acquiring valuable information for users. The Q-Rapids tool is designed to
gather diverse data related to software system development and usage. These
data are then structured within a Quality Model (QM) to analyze aggregated
quality-related strategic indicators. These indicators are made available to
decision-makers through a multi-dimensional and navigational dashboard
employed during ASD events like sprint planning or daily stand-up meetings.
The Q-Rapids solution was implemented and used in a real-world setting.
Martínez-Fernández et al. [47] described a case study across four companies
involving 26 practitioners to investigate the integration of a Quality Model
within the Q-Rapids software analytics tool. Dharmapal and Sikamani [23]
proposed utilizing a combination of agile methodology principles and the
expertise of a Data Analyst. During the planning stage, the Product Owner and
the Data Analyst collaborate to select suitable Big Data Analytics tools and
strategize the data source collection. This approach allows for incremental
data collection in response to specific needs. In the development phase, the
Data Analyst assumes a pivotal role in analyzing data gathered from various
sources. They integrate data from diverse origins into a unified dataset,
merge information based on shared attributes, and present it in a consolidated
format. An essential aspect of their role is the removal of irrelevant data,
employing an elimination technique that saves time and resources by excluding
unnecessary data from processing.

Comprehensive Data Collection: Das et al. [21] delved into the challenges
associated with collecting and storing substantial amounts of data for
specific research purposes, particularly when the data is non-traditional, or
the dataset is immense. Huang et al. [37] focused on efficiently ingesting and
governing large volumes of heterogeneous data from various sources in various
formats. Fagarasan et al. [28] stressed the importance of systematic and
comprehensive data collection, beginning at the lowest level of project
management. Olsson [52] discussed that traditional data collection methods,
such as surveys and interviews, are insufficient for continuous
experimentation.

To address comprehensive data collection challenges, Das et al. [21] advocated
for a two-fold approach: first, the creation of a generalized dataset aimed at
expanding data coverage (not volume) by incorporating additional attributes.
This approach ensures that a single data extraction step can serve multiple
projects, with adjustments made in subsequent workflows to cater to specific
research task requirements. Second, they emphasized standardized data
processing, which involves abstracting common data procedures, such as noise
reduction and language standardization, into reusable solutions and tools
applicable across various data-driven research endeavors. It makes Datto
systematically collect and analyze. Huang et al. [37] The solution they
proposed was to design a big data stack with scalable storage technologies and
a flexible architecture. This stack included a data lake for raw data storage
and integrated relational databases for real-time applications. Apache Spark
was used as a distributed data computation engine to enable parallel
processing for analytics tasks. The stack’s design allowed for swift data
analysis and agile development of data products, reducing the time required to
answer analytical questions and develop applications. Fagarasan et al. [28]
proposed using a market-available Project Management Information System (PMIS)
for the systematic collection and analysis of data. This system enables the
automated collection and reporting of key performance indicators, crucial for
monitoring project performance and ensuring cost-effective execution. The PMIS
serves as a foundation for organizing projects into a measurable portfolio,
with performance metrics deeply embedded in the software development workflow.
Olsson [52] proposed implementing mechanisms for automated data collection
directly from the software in use, particularly in the post-deployment stage.
This approach allows for the collection of real-time, actionable data that can
be used for analysis to improve current products and inform the development of
future ones, aligning with legal requirements for data utilization and user
consent.

RQ2 Findings
We identified several challenges associated with the different data management
aspects as noted in RQ1, in particular the top four aspects (data integration,
collection, quality, and analysis). These challenges encompass the
complexities of interoperating, collecting, managing, and ensuring the quality
of heterogeneous and diverse data. Agile teams also face difficulties with
real-time analytics. Additionally, fragmented data collection and a lack of
standardized practices can impede team collaboration and project transparency.

####  4.3.3 Data Quality Challenges and Solutions

We have identified multiple challenges associated with Data Quality in agile
development. These challenges and their proposed solutions are discussed in
the studies reviewed below. A summary of these challenges and their
corresponding solutions is presented in Table 5.

Table 5: Summary of Data Quality Challenges and Solutions

Data Quality Challenge | Data Quality Solution | Solution Status | Ref.
---|---|---|---
Ensuring Data Accuracy and Consistency Across Varied Sources | Solution referenced in 4.3.1 | Implemented | [56]
| Solution referenced in 4.3.1 | Implemented | [34]
| Solution referenced in 4.3.1 | Implemented | [46]
| Solution referenced in 4.3.2 | Implemented | [53]
| Solution referenced in 4.3.1 | Implemented | [39]
Missing Quality Data | Implementing quality-aware software development methodology. | Implemented | [30]
| Solution referenced in 4.3.2 | Proposed | [43]
| Solution referenced in 4.3.2 | Proposed | [47]
Inadequate Data Quality Management | Utilizing agile methods and test-driven approaches. | Proposed | [6]
| Solution referenced in 4.3.1 | Implemented | [36]
| Solution referenced in 4.3.1 | Implemented | [59]
Data Quality Standardization | Solution referenced in 4.3.2 | Implemented | [58]
| Solution referenced in 4.3.2 | Proposed | [10]

Ensuring Data Accuracy and Consistency Across Varied Sources: Rosenkranz et
al. [56] highlighted the importance of data quality ( as it is defined in
their study by completeness, ambiguity, meaningfulness, and correctness) in
application systems and project performance in IT or Data Warehouse (DW)
projects. The challenge lies in ensuring the accuracy and consistency of data
from disparate sources. Harper and Dagnino [34] also emphasized the need to
merge and clean datasets from different sources. The challenge here is
ensuring the accuracy and consistency of data for analysis. Little et al. [46]
mentioned that the products are released regularly, with release cycles
ranging from 3 to 18 months. This regularity poses a challenge in managing
data and ensuring consistency across different product versions. Pater et al.
[53] underscored the necessity to eradicate data redundancy and business
process fragmentation in the context of IT system modernization. The challenge
lies in aligning the diverse IT needs of users with the broader organizational
data requirements. Kannan et al. [39] highlighted the challenge of ensuring
the completeness, accuracy, and internal consistency of the critical few EHR
fields that are essential for registry inclusion and eCQM (electronic Clinical
Quality Measure) calculation.

Note that the solutions suggested to resolve the data quality challenges
discussed above are the same as those of the data integration and collection
(as they co-occur together). We discussed those solutions in detail for data
integration in Section 4.3.1 and for data collection in Section 4.3.2.

Missing Quality Data: Franch et al. [30] emphasized the challenge when
necessary quality data is unavailable (inaccessible, incomplete, inconsistent,
and incorrect data) due to the absence of suitable data collection tools or
processes or inaccessible data storage systems. For instance, SonarQube
(mentioned as an existing tool that focuses on product quality or continuous
integration) does not furnish raw data for specific base metrics. Lehtonen et
al. [43] highlighted potential data quality problems in software engineering
data, including noise, outliers, low precision, missing values, coverage
errors, and clones. An example given is the potential for inaccurate data due
to developers forgetting to update the issue management system task state when
development work starts or ends, leading to inaccurate timestamps. Mart’ınez-
Fern’andez [47] discussed the need for transparency and clarity on the raw
data used to compute factors and indicators in the software analytics tool
(Q-Rapids tool). End-users like product owners may require a clear
understanding of the raw data used to derive specific values and the
associated decision-making processes.

To address the missing data Franch et al. [30] discussed the Q-Rapids project,
mentioned as an initiative aimed at developing a quality-aware rapid software
development methodology. The Q-Rapids project addressed data quality
challenges by selecting and configuring tailored data collection tools aligned
with business objectives, specifying precision requirements, implementing
real-time checks to prevent errors, and integrating diverse data sources while
transforming them for compatibility with specific analysis methods and tools.
The solution was implemented, evaluated, and refined through three releases
deployed by the four industry partners in the project. Note that the solutions
suggested to resolve the remaining data quality challenges, as discussed by
Lehtonen et al. [43] and Mart’ınez-Fern’andez [47], are the same as those of
data collection (as they co-occur together). We discussed those solutions in
detail in Section 4.3.2.

Inadequate Data Quality Management: Ambler [6] highlighted that data quality
issues cost US organizations an estimated 600 billion dollars annually,
stemming from the use of traditional data management approaches to ensure data
quality within organizations, according to the Data Warehouse Institute as
indicated by a survey conducted by Ambler [6]. Svensson et al. [59] emphasized
that the quality of the data, as well as the processing techniques and tools
used to handle it, will impact the quality of decisions made using Data-Driven
Decision Making (DDDM). Hofer et al. [36] discussed the data quality challenge
of ensuring the ”fitness for use” of data, which is often neglected or delayed
in the software engineering process until the end-user evaluation phase, known
as the “point-of-truth”. This delay impacts the cost-effectiveness of data
quality management and contradicts agile principles that emphasize early and
continuous delivery of valuable software.

To resolve inadequate data quality management, Ambler [6] Suggested that
utilizing agile methods, which typically involve a test-driven approach to
development, results in greater quality with more and early testing.
Additionally, developers and data professionals should apply concrete,
quality-focused techniques for evolutionary development. Note that the
solutions suggested to resolve the remaining data quality challenges, as
discussed by Svensson et al. [59] and Hofer et al. [36], are the same as those
of data collection and data integration (as they co-occur together). We
discussed those solutions in detail in the data collection Section 4.3.2 and
for data integration Section 4.3.1.

Data Quality Standardization: Spengler et al. [58] mentioned the need for
significant data restructuring and cleansing due to heterogeneous data to
ensure suitability for proper integration into clinical data warehousing
solutions. Batarseh and Gonzalez [10] highlighted the need to enhance dataset
quality, standardize variables, and eliminate unnecessary data. The challenge
here involves dealing with poor-quality data and standardizing data for
analysis.

Note that the solutions suggested to resolve the data quality challenges are
the same as those of data integration and collection (as they co-occur
together). We discussed those solutions in detail for data integration in
Section 4.3.1 and for data collection in Section 4.3.2.

####  4.3.4 Data Analysis Challenges and Solutions

We also identified multiple challenges associated with Data analysis in Agile
development. These challenges and their proposed solutions are discussed in
the studies reviewed below. A summary of these challenges and their
corresponding solutions is presented in Table 6.

Table 6: Summary of Data Analysis Challenges and solutions

Data Analysis Challenge | Data Analysis Solutions | Solution Status | Ref.
---|---|---|---
Analyzing Large and Complex Data | Solution referenced in 4.3.1 | Proposed | [17]
| Solution referenced in 4.3.2 | Proposed | [23]
| Solution referenced in 4.3.2 | Proposed | [21]
| Solution referenced in 4.3.2 | Proposed | [45]
| Well-documented tools, developer training, and simplified visualizations. | Proposed | [33]
Analyzing Semantic Heterogeneity Data | Solution referenced in 4.3.1 | Proposed | [25]
| Solution referenced in 4.3.1 | Implemented | [9]
| Solution referenced in 4.3.1 | Implemented | [55]
Efficient Data Analysis and Visualization | Solution referenced in 4.3.1 | Proposed | [26]
| Solution referenced in 4.3.2 | Proposed | [59]
| Solution referenced in 4.3.2 | Proposed | [43]
| Solution referenced in 4.3.2 | Proposed | [27]
Real-Time Data Analytics and Decision Making | Solution referenced in 4.3.1 and, 4.3.2 | Proposed | [18]
| Solution referenced in 4.3.2 | Proposed | [52]
Selection of Appropriate Analytical Techniques | Solution referenced in 4.3.1 | Implemented | [34]
| Solution referenced in 4.3.2 | Proposed | [49]
| Solution referenced in 4.3.2 | Proposed | [48]
| Solution referenced in 4.3.2 | Proposed | [28]
| Solution referenced in 4.3.2 | Proposed | [10]

Analyzing Large and Complex Data: Chen et al. [17] delved into the challenge
of developing a big data system that effectively supports advanced analytics
by addressing the 5Vs of big data (Volume, Velocity, Variety, Veracity, and
Value) and facilitating effective collaboration between data scientists and
software engineers to maximize the value from big data. Dharmapal and Sikamani
[23] underscored the challenge of managing and processing the increased volume
of unstructured data, which is more complex, requires enhancements in data
management and analysis techniques. Das et al. [21] discussed the challenge of
analyzing extensive volumes of data, particularly when the data is non-
traditional or when the dataset is massive. Traditional analysis techniques
often fall short due to these challenges. Lin et al. [45] discussed the
challenge of the high dimensionality of the datasets, which included detailed
interactions and decisions made by the participants (students in the study).
This complexity made it difficult to identify which features (which could
include metrics such as the number of collaborators per task, the frequency of
task updates, mood stability, and other behavioral indicators) or a
combination of features could accurately predict certain behaviors of
interest. Hamer et al. [33] highlighted that interpreting the data provided by
data-driven tools such as Git, Jira, and SonarQube can pose a significant
barrier to beginning the effective use of these tools due to the large volume
and complexity of the data generated. Furthermore, analyzing this data
necessitates training to utilize the information correctly.

Note that the solutions suggested to resolve the challenge of analyzing large
and complex data are the same as those of the data integration and collection
(as they co-occur together). We discussed those solutions in detail for data
integration in Section 4.3.1 and for data collection in Section 4.3.2.
Additionally, Lin et al. [45] solution is to employ the application of
Exploratory Data Analysis (EDA) to analyze the collected data. EDA is used to
summarize the main characteristics of data sets, often with visual methods. It
aims to understand what can be learned from the data beyond the formal
modeling or hypothesis-testing tasks. Furthermore, Hamer et al. [33] solution
recommends that these tools should be well-documented and provide tutorials to
facilitate easier adoption and usage. Developers should be trained to
comprehend and leverage the provided information effectively. Future work
could involve creating simplified visualizations that enhance information
understanding. This proposed solution has not been implemented.

Analyzing Semantic Heterogeneity Data: Dos Santos J’unior et al. [25]
highlighted the difficulties organizations encounter when accessing,
integrating, analyzing, and viewing data managed by disparate applications.
Barcellos [9] discussed the need for continuous software measurement, which
involves collecting and analyzing data to provide useful information for daily
activities and decision-making in software development. Rix et al. [55]
addressed the challenge of analyzing real-time data streams using data mining
methods.

Note that the solutions suggested to resolve the challenge of semantic
heterogeneity data are the same as those of the data integration (as they co-
occur together). We discussed those solutions in detail for data integration
in Section 4.3.1.

Efficient Data Analysis and Visualization: Dursun et al. [26] discussed the
challenges of efficiently analyzing and visualizing vast and diverse oil and
gas data, as well as developing intelligent data-driven analytics software to
optimize decision-making and minimize. Svensson et al. [59] addressed data
analysis as a crucial step in data-driven decision-making (DDDM). The
challenge lies in the fact that the quality of decisions is directly
proportional to the quality of processing techniques and tools. In other
words, ineffective or incorrect data analysis can lead to suboptimal or
erroneous decisions. Lehtonen et al. [43] explored the utilization of
visualizations for data analysis. The challenge lies in the potential
complexity of these visualizations, which may hinder understanding. Fabijan et
al. [27] highlighted that companies are not capitalizing fully on the data
they amass. This is attributed to challenges associated with efficiently and
meaningfully integrating and analyzing customer data.

Note that the solutions suggested to resolve the challenge of efficient data
analysis and visualization, are the same as those of the data integration and
collection (as they co-occur together). We discussed those solutions in detail
for data integration in Section 4.3.1 and for data collection in Section
4.3.2.

Real-Time Data Analytics and Decision Making: Chhillar and Sharma [18]
discussed using data analytics tools for generating metric reports. These
reports are vital for indicating the overall health of the build post-
deployment. The challenge here is the requirement for real-time data analytics
to support decision-making processes. Olsson [52] discussed the difficulty in
handling and analyzing the post-deployment data collected from embedded
systems.

Note that the solutions suggested to resolve the challenge of real-time data
analytics and decision-making are the same as those of the data integration
and collection (as they co-occur together). We discussed those solutions in
detail for data integration in Section 4.3.1 and for data collection in
Section 4.3.2.

Selection of Appropriate Analytical Techniques: Harper and Dagnino [34]
discussed the need to specify analysis strategy, process substantial amounts
of data, and complete analyses promptly. The challenge lies in fulfilling
these expectations while managing the complexity inherent in advanced
analytics applications. Matthies and Hesse [49] discussed data generation
during software development processes such as work item descriptions,
documentation, and version control information. The challenge lies in
efficiently managing and analyzing this data. Matthies [48] discussed using
various metrics designed for agile practices or tools that developers are
already familiar with, such as the git command line, for project data
analysis. The challenge is that improvement actions are often based on
anecdotal evidence and experiences rather than empirical project evidence.
Fagarasan et al. [28] discussed the necessity for effective data analysis to
extract meaningful insights from collected data. The challenge lies in
interpreting, analyzing, and converting this information into actionable
insights. Batarseh and Gonzalez [10] underscored the need to define variables
for model development and investigate different models. The challenge here
lies in selecting appropriate variables and models for analysis.

Note that the solutions suggested to resolve the challenge of selection of
appropriate analytical techniques, are the same as those of the data
integration and collection (as they co-occur together). We discussed those
solutions in detail for data integration in Section 4.3.1 and for data
collection in Section 4.3.2.

RQ3 Findings
To address the challenges identified in RQ2, we found that the proposed or
implemented solutions vary across studies, depending on the nature of the
systems and the maturity of the processes followed. We also note that most
solutions are mainly proposed with no empirical evaluation. The solutions
involve ontologies, strategies, advanced ETL processes, decentralized
approaches, and automation tools. Additionally, they emphasize quality-aware
development methodologies, as well as the use of well-documented tools and
training for data quality and analysis.

##  5 Discussion and Implications

The challenges identified in this SLR suggest that without addressing key data
management issues, agile teams may face significant obstacles that can impede
the efficiency and effectiveness of their work. The proposed solutions, as
discussed in Section 4, indicate a need for innovative approaches and tools to
overcome these challenges. For example, decentralized data management [61]
distributes responsibility for data handling, reducing bottlenecks and
enhancing collaboration. This approach requires agile teams to be adaptable
and innovative in their project execution strategies, ensuring effective data
management across different team members. As the industry continues to evolve
towards more data-driven decision-making [59] and [49], the insights from this
SLR will likely influence how practitioners approach data management, leading
to the adoption of new strategies that enhance agility, improve product
quality, and facilitate better project outcomes. To illustrate this, we note
some specific results and implications for the top data management aspects
(data integration, collection, quality and analysis) below:

Data Integration: Agile teams are expected to deliver value swiftly and adapt
to change flexibly [11], but challenges with data integration can hinder this.
Interoperability challenges and semantic heterogeneity can lead to
difficulties in combining data from various sources, causing delays and
miscommunication within the team. This can impede the team’s ability to make
prompt and informed decisions, which is crucial for maintaining the momentum
of agile development. The complexity of managing large volumes of data and
ensuring data quality can also strain the resources of an agile team,
potentially impacting the efficiency and effectiveness of the product delivery
process. Therefore, these data integration challenges necessitate a holistic
approach in agile development, emphasizing the importance of continuous skill
development and adaptability in both team dynamics and project execution
strategies. For example, team members might need to learn new data integration
tools or techniques to handle semantic heterogeneity more effectively, as
discussed in Section 4.3.1. Moreover, adaptable team dynamics and project
execution, quickly adjusting to changes in project requirements and data
environments, are vital.

Data Collection: Agile teams rely on the ability to collect and utilize data
effectively to inform decision-making and adapt to changes rapidly, as
discussed in Section 4.3.2, [8], [48] and [45]. However, when data collection
is fragmented or lacks structure, it can lead to inefficiencies and hinder the
team’s ability to make informed decisions. One of the important implications
is the difficulty in accessing data collected by other agile team members
during different development phases [27], which can result in unnecessary
rework and inefficient communication methods. This fragmentation can also lead
to data loss during critical “handovers” (i.e., transfer of data from one team
or individual to another during different development phases) [27], negatively
impacting the product delivery process. The absence of concurrent access to
data [40] is another significant challenge for agile teams, as it prevents
real-time collaboration and information exchange, which are essential for
agile methodologies. Additionally, the uncertainty surrounding what data agile
product teams are allowed to gather [8] can create legal and compliance
issues, further complicating the data collection process. These challenges
necessitate a comprehensive approach to data collection, emphasizing
systematic, efficient, and compliant methods to ensure effective agile
development.

Data Quality: Agile practitioners are tasked with ensuring data completeness,
accuracy, and internal consistency, which is essential for various aspects of
product development [56, 34]. Inaccuracies and inconsistencies in data from
varied sources can lead to poor decision-making and hinder the performance of
IT or Data Warehouse projects, as discussed in Section 4.3.3 and [56].
Furthermore, a common release cycle typically ranging from 3 to 18 months [46]
poses challenges in managing data consistency across different product
versions. This can affect the agile team’s ability to deliver high-quality
products on time due to the need to constantly adapt to changes and ensure
data reliability throughout the development cycle. The necessity to eradicate
data redundancy and business process fragmentation is crucial in IT system
modernization [53], where agile practitioners must align diverse IT needs with
broader organizational data requirements. Challenges like these, coupled with
the complexity of data generated by tools such as Git, Jira, and SonarQube,
can make it difficult for agile practitioners to identify relevant features
and effectively utilize the data, thereby creating barriers to the effective
use of these tools and necessitating training for proper data analysis and
interpretation. Agile teams are required to invest significant effort in
ensuring data accuracy, completeness, and consistency. This involves adopting
robust data management practices and techniques, which can impact project
timelines, resource allocation, and, ultimately, the success of the product
delivery process.

Data Analysis: Agile teams often need to analyze large and diverse datasets
due to the difficulties posed by the 5 Vs of big data: Volume, Velocity,
Variety, Veracity, and Value [17]. This complexity necessitates advanced
analytical skills for effective data processing and interpretation, which is
essential for informed decision-making. Inefficient data analysis can result
in poor decision-making related to the development of a system [59], as the
quality of decisions depends heavily on the quality of data processing
techniques and tools. Inadequate data analysis can lead to misguided
strategies and actions, potentially compromising product success.
Additionally, the high dimensionality of datasets, particularly those
generated by tools such as Git, Jira, and SonarQube, makes it difficult to
identify relevant features for analysis. Agile practitioners may require
additional training to analyze and interpret complex data effectively. For the
product delivery process, these challenges can lead to delays and
inefficiencies, as agile teams may spend considerable time integrating and
analyzing data, potentially slowing down the iterative cycles characteristic
of agile methodologies.

##  6 Threats to Validity

In conducting this SLR, we have identified and addressed various threats that
could impact the validity of our study. We have followed the SLR guidelines
proposed by Kitchenham and Charters [41] to mitigate these threats. The
potential threats are analyzed based on the four types: internal validity,
external validity, construct validity, and conclusion validity [7, 63, 64].

###  6.1 Internal Validity

Internal validity examines how various elements influence the outcomes and
interpretation of the data gathered. A comprehensive search strategy was
employed to counteract Search String Bias (Section 3.2). This involved
conducting a pilot search on Scopus to refine search terms and ensure that
they were broad enough to capture all relevant literature while being specific
enough to exclude unrelated studies. The search strings were iteratively
tested and adjusted based on the pilot search results, which helped identify
the most effective combination of keywords and search parameters.

Mitigating Selection and Exclusion Bias was achieved by establishing explicit
inclusion and exclusion criteria (Section 3.3) that were consistently applied
throughout the study selection process. Periodic sampling of studies was
conducted to review the application of these criteria, ensuring that relevant
studies, even those indirectly related to agile methodologies, were not
inadvertently excluded. This approach helped maintain a balanced and
comprehensive set of studies for the review. We have also conducted
comprehensive quality checks to ensure rigor, integrity, and relevance of the
quality of each selected study (see Section 3.7).

To reduce personal bias during data extraction, a dedicated data extraction,
as outlined in Section 3.6, is employed to extract relevant data consistently.
The first author initially performs the data extraction, followed by meetings
where other authors participate to verify the data and resolve any
uncertainties. Additionally, these authors engage in discussion meetings for
verification and peer review.

To diminish the risk of subjective interpretation bias in data classification,
we adhered to thematic classification guidelines [13] and utilized well-
established descriptive statistical methods for both quantitative and thematic
qualitative data.

###  6.2 External Validity

The concept of external validity focuses on the extent to which the findings
of our study can be generalized. While we do not claim that our findings are
universally applicable, we sought to maximize their relevance. This objective
was pursued by providing an in-depth review of the challenges and solutions in
data management in conjunction with a stringent, protocol-based SLR method as
described in [41]. The detailed methodologies, adherence to the SLR protocol,
and the systematic approach to data extraction are all designed to facilitate
the discovery and amalgamation of pertinent studies, thereby reducing the
risks associated with external validity.

###  6.3 Construct validity

Construct validity focuses on the accuracy of operational measures used in the
study to investigate specific concepts. To ensure construct validity, we have
implemented these strategies: Operational Measures: We have established
specific and well-defined research questions and clear inclusion and exclusion
criteria, ensuring that the studies selected are relevant to the intended
concepts. Data Extraction: We adopted a detailed data extraction process for
consistent data collection, as outlined in Section 3.6. This is followed by
meetings for peer review and further verification.

###  6.4 Conclusion validity

Conclusion validity deals with how reasonable the conclusions drawn about
relationships in our data are. To strengthen conclusion validity, we have
followed a cross-validation process involving multiple authors in analyzing
the data to cross-validate findings and interpretations, thereby minimizing
the likelihood of inaccurate conclusions.

##  7 Conclusions

This review provides insights into the data management challenges faced within
agile software projects. We identified 45 relevant studies that formed the
basis of our analysis. The review process allowed us to define key data
management aspects and derive insightful definitions for each aspect, as
presented in Table 2. Furthermore, our exploration of the challenges and
solutions in data integration, data analysis, data quality, and data
collection has shed light on the complexities and implications of managing
data within agile environments. Our quality assessment of the selected
studies, as detailed in Section 3.7, ensured that the included studies met
high rigor, integrity, and relevance standards. This approach bolstered the
credibility and reliability of our findings, ultimately contributing to a
comprehensive understanding of the data management landscape in agile software
projects.

The review underscores data analysis, data integration, data collection, and
data quality as the key areas of concern in agile software projects. It delves
into challenges such as interoperability, semantic heterogeneity, and the need
for real-time analytics to support decision-making processes. Other challenges
include ensuring data accuracy and consistency across varied sources,
addressing the variety and volume of data sources, and managing the regularity
of product releases. In response to these challenges, the review identified a
range of solutions. These include using cloud-based platforms with agile data-
loading pipelines for data integration, employing automated tools for data
quality assurance, and adopting architecture-centric approaches to support
flexible data structure needs for continuous delivery in agile software
projects. Additionally, the implementation of real-time analytics tools and
market-available PMIS for systematic data collection and analysis are
suggested. Furthermore, we recommend that future work should focus on the
empirical validation of the solutions proposed by the studies in the review in
real-world settings.

By synthesizing the findings and discussing their implications, we provided
actionable recommendations for agile practitioners to improve their data
management practices and ultimately enhance the success of agile software
development initiatives. We hope that the insights and recommendations
presented in this review will serve as a valuable resource for agile
practitioners, researchers, and organizations striving to navigate the
complexities of data management in agile software projects.

## References

  * [1]
  * Qua [[n. d.]] [n. d.].  Study Quality Assessment Tools.  https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools.  Accessed: 2023-12-21.
  * dat [[n. d.]] [n. d.].  What is data management?  https://www.ibm.com/topics/data-management.  Accessed: 2023-12-07.
  * Abdallah and Fan [2022] Abdallah A Abdallah and Ip-Shing Fan. 2022.  Towards Building Ontology-Based Applications for Integrating Heterogeneous Aircraft Maintenance Records. In _Proceedings of the 20th IEEE International Conference on Industrial Informatics (INDIN)_. IEEE, 293–299.
  * Alsaadi and Saeedi [2022] Bashaer Alsaadi and Kawther Saeedi. 2022.  Data-driven effort estimation techniques of agile user stories: a systematic literature review.  _Artificial Intelligence Review_ 55, 7 (2022), 5485–5516.
  * Ambler [2008] Scott Ambler. 2008.  When it gets cultural: Data management and agile development.  _IT Professional_ 10, 6 (2008), 11–14.
  * Ampatzoglou et al. [2019] Apostolos Ampatzoglou, Stamatia Bibi, Paris Avgeriou, Marijn Verbeek, and Alexander Chatzigeorgiou. 2019.  Identifying, categorizing and mitigating threats to validity in software engineering secondary studies.  _Information and Software Technology_ 106 (2019), 201–230.
  * Barbala et al. [2023] Astri Barbala, Tor Sporsem, and Viktoria Stray. 2023.  Data-Driven Development in Public Sector: How Agile Product Teams Maneuver Data Privacy Regulations. In _Proceedings of the 24th International Conference on Agile Software Development (XP)_. Springer, 165–180.
  * Barcellos [2020] Monalessa Perini Barcellos. 2020.  Towards a framework for continuous software engineering. In _Proceedings of the XXXIV Brazilian Symposium on Software Engineering (SBES)_. ACM, 626–631.
  * Batarseh and Gonzalez [2018] Feras A Batarseh and Avelino J Gonzalez. 2018.  Predicting failures in agile software development through data analytics.  _Software Quality Journal_ 26 (2018), 49–66.
  * Beck et al. [2001] Kent Beck, Mike Beedle, Arie Van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, et al. 2001\.  Manifesto for agile software development.  (2001).
  * Bosch [2019] Jan Bosch. 2019.  Towards a digital business operating system. In _Proceedings of the 13th International Conference on Research Challenges in Information Science (RCIS)_. IEEE, 1–9.
  * Braun and Clarke [2006] Virginia Braun and Victoria Clarke. 2006.  Using thematic analysis in psychology.  _Qualitative Research in Psychology_ 3, 2 (2006), 77.
  * Campanelli and Parreiras [2015] Amadeu Silveira Campanelli and Fernando Silva Parreiras. 2015.  Agile methods tailoring–A systematic literature review.  _Journal of Systems and Software_ 110 (2015), 85–100.
  * Canedo et al. [2022] Edna Dias Canedo, Angelica Toffano Seidel Calazans, Ian Nery Bandeira, Pedro Henrique Teixeira Costa, and Eloisa Toffano Seidel Masson. 2022.  Guidelines adopted by agile teams in privacy requirements elicitation after the Brazilian general data protection law (LGPD) implementation.  _Requirements Engineering_ 27, 4 (2022), 545–567.
  * Carrera-Rivera et al. [2022] Angela Carrera-Rivera, William Ochoa, Felix Larrinaga, and Ganix Lasa. 2022.  How-to conduct a systematic literature review: A quick guide for computer science research.  _MethodsX_ 9 (2022), 101895.
  * Chen et al. [2016] Hong-Mei Chen, Rick Kazman, and Serge Haziyev. 2016.  Agile big data analytics development: An architecture-centric approach. In _Proceedings of the 49th Hawaii International Conference on System Sciences (HICSS)_. IEEE, 5378–5387.
  * Chhillar and Sharma [2019] Dheeraj Chhillar and Kalpana Sharma. 2019.  ACT Testbot and 4S Quality Metrics in XAAS Framework. In _Proceedings of the International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon)_. IEEE, 503–509.
  * Chung and Hartford [2006] Sam Chung and Eric Hartford. 2006.  Bridging the gap between data models and implementations: XMI2SQL. In _Proceedings of the International Conference on Telecommunications and International Conference on Internet and Web Applications and Services (AICT-ICIW)_. IEEE, 201–201.
  * Colakoglu et al. [2021] Fatima Nur Colakoglu, Ali Yazici, and Alok Mishra. 2021.  Software product quality metrics: A systematic mapping study.  _IEEE Access_ 9 (2021), 44647–44670.
  * Das et al. [2015] Manirupa Das, Renhao Cui, David R Campbell, Gagan Agrawal, and Rajiv Ramnath. 2015.  Towards methods for systematic research on big data. In _Proceedings of the IEEE International Conference on Big Data (BigData)_. IEEE, 2072–2081.
  * Dautov et al. [2022] Rustem Dautov, Erik Johannes Husom, and Fotis Gonidis. 2022.  Towards MLOps in Mobile Development with a Plug-in Architecture for Data Analytics. In _Proceedings of the 6th International Conference on Computer, Software and Modeling (ICCSM)_. IEEE, 22–27.
  * Dharmapal and Sikamani [2016] Surend Raj Dharmapal and K Thirunadana Sikamani. 2016.  Big data analytics using agile model. In _Procedings of the International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT)_. IEEE, 1088–1091.
  * Dikert et al. [2016] Kim Dikert, Maria Paasivaara, and Casper Lassenius. 2016.  Challenges and success factors for large-scale agile transformations: A systematic literature review.  _Journal of Systems and Software_ 119 (2016), 87–108.
  * dos Santos Júnior et al. [2021] Paulo Sérgio dos Santos Júnior, Monalessa P Barcellos, and João Paulo A Almeida. 2021.  An Ontology-based Approach to Enable Data-Driven Decision-Making in Agile Software Organizations. In _Proceedings of the XIV Seminar on Ontology Research in Brazil (ONTOBRAS)_. 279–284.
  * Dursun et al. [2014] Serkan Dursun, Kaan Duman, Tayfun Tuna, Mamta Abbas, and James Ding. 2014.  A workflow for intelligent data-driven analytics software development in oil and gas industry. In _Proceedings of the SPE Annual Technical Conference and Exhibition (ATCE)_. SPE, SPE–170859.
  * Fabijan et al. [2016] Aleksander Fabijan, Helena Holmstrøm Olsson, and Jan Bosch. 2016.  The lack of sharing of customer data in large software organizations: challenges and implications. In _Proceedings of the 17th International Conference on Agile Software Development (XP)_. Springer, 39–52.
  * Fagarasan et al. [2023] Cristian Fagarasan, Ciprian Cristea, Maria Cristea, Ovidiu Popa, and Adrian Pisla. 2023.  Integrating Sustainability Metrics into Project and Portfolio Performance Assessment in Agile Software Development: A Data-Driven Scoring Model.  _Sustainability_ 15, 17 (2023), 13139.
  * Fawzy et al. [2024] Ahmed Fawzy, Amjed Tahir, Matthias Galster, and Peng Liang. 2024.  Dataset of the Paper “Data Management Challenges in Agile Software Projects: A Systematic Literature Review”.  https://doi.org/10.5281/zenodo.10597817
  * Franch et al. [2019] Xavier Franch, Lidia Lopez, Silverio Martínez-Fernández, Marc Oriol, Pilar Rodríguez, and Adam Trendowicz. 2019.  Quality-aware rapid software development project: the Q-rapids project. In _Proceedings of the 51st International Conference on Software Technology: Methods and Tools (TOOLS)_. Springer, 378–392.
  * Graetsch et al. [2023] Ulrike M Graetsch, Hourieh Khalajzadeh, Mojtaba Shahin, Rashina Hoda, and John Grundy. 2023.  Dealing with data challenges when delivering data-intensive software solutions.  _IEEE Transactions on Software Engineering_ (2023).
  * Grünewald [2021] Elias Grünewald. 2021.  Cloud native privacy engineering through DevPrivOps.  In _Proceedings of the IFIP International Summer School on Privacy and Identity Management (IFIP)_. Springer, 122–141.
  * Hamer et al. [2023] Sivana Hamer, Christian Quesada-López, and Marcelo Jenkins. 2023.  Students’ perceptions of integrating a contribution measurement tool in software engineering projects. In _Proceedings of the 35th IEEE International Conference on Software Engineering Education and Training (CSEE &T)_. IEEE, 21–30.
  * Harper and Dagnino [2014] K Eric Harper and Aldo Dagnino. 2014.  Agile software architecture in advanced data analytics. In _Proceedings of the 11th Working IEEE/IFIP Conference on Software Architecture (WICSA)_. IEEE, 243–246.
  * Harriman et al. [2004] Alan Harriman, Paul Hodgetts, and Mike Leo. 2004.  Emergent database design: liberating database development with agile practices. In _Proceedings of the 5th International Conference on Agile Software Development (XP)_. IEEE, 100–105.
  * Hofer et al. [2020] Marvin Hofer, Sebastian Hellmann, Milan Dojchinovski, and Johannes Frey. 2020.  The new dbpedia release cycle: Increasing agility and efficiency in knowledge extraction workflows. In _Proceedings of the 16th International Conference on Semantic Systems (SEMANTiCS)_. Springer, 1–18.
  * Huang et al. [2021] He Huang, Yixin He, Longpeng Zhang, Zhicheng Zeng, Tu Ouyang, and Zhimin Zeng. 2021.  Leveraging Modern Big Data Stack for Swift Development of Insights into Social Developments. In _Proceedings of the International Conference on Wireless Communications, Networking and Applications (WCNA)_. Springer, 325–333.
  * Jenness et al. [2018] Tim Jenness, Frossie Economou, Krzysztof Findeisen, Fabio Hernandez, Josh Hoblitt, K Simon Krughoff, KT Lim, Robert H Lupton, Fritz Mueller, William O’Mullane, et al. 2018\.  LSST data management software development practices and tools. In _Proceedings of the Software and Cyberinfrastructure for Astronomy V (SPIE)_. SPIE, 50–69.
  * Kannan et al. [2017] Vaishnavi Kannan, Jason S Fish, Jacqueline M Mutz, Angela R Carrington, Ki Lai, Lisa S Davis, Josh E Youngblood, Mark R Rauschuber, Kathryn A Flores, Evan J Sara, et al. 2017\.  Rapid development of specialty population registries and quality measures from electronic health record data.  _Methods of Information in Medicine_ 56, S01 (2017), e74–e83.
  * Kaur et al. [2020] Supreet Kaur, Stefan Siarov, Marco Witzmann, and Louise Lindblad. 2020.  A Dialogue on the Digitization of Requirements, Verification, and Test Management with Data-Driven Systems Engineering (DDSE). In _Proceedings of the 71st International Astronautical Congress (IAC)_. IAF, 1–6.
  * Kitchenham et al. [2007] Barbara Kitchenham, Stuart Charters, et al. 2007\.  Guidelines for performing systematic literature reviews in software engineering version 2.3.  _Engineering_ 45, 4ve (2007), 1051.
  * Lee and Xia [2010] Gwanhoo Lee and Weidong Xia. 2010.  Toward agile: an integrated analysis of quantitative and qualitative field data on software development agility.  _MIS Quarterly_ 34, 1 (2010), 87–114.
  * Lehtonen et al. [2017] Timo Lehtonen, Timo Aho, Kati Kuusinen, and Tommi Mikkonen. 2017.  Visualizations for software development process management.  _Information Modelling and Knowledge Bases XXVIII_ 292 (2017), 1.
  * Li [1999] Wei Li. 1999.  Software product metrics.  _IEEE Potentials_ 18, 5 (1999), 24–27.
  * Lin et al. [2018] Jun Lin, Han Yu, Zhengxiang Pan, Zhiqi Shen, and Lizhen Cui. 2018.  Towards data-driven software engineering skills assessment.  _International Journal of Crowd Science_ 2, 2 (2018), 123–135.
  * Little et al. [2004] Todd Little, Forrest Greene, Tessy Phillips, Rex Pilger, and Robert Poldervaart. 2004.  Adaptive agility. In _Proceedings of the 5th International Conference on Agile Software Development (XP)_. IEEE, 63–70.
  * Martínez-Fernández et al. [2019] Silverio Martínez-Fernández, Anna Maria Vollmer, Andreas Jedlitschka, Xavier Franch, Lidia López, Prabhat Ram, Pilar Rodríguez, Sanja Aaramaa, Alessandra Bagnato, Michał Choraś, et al. 2019\.  Continuously assessing and improving software quality with software analytics tools: a case study.  _IEEE Access_ 7 (2019), 68219–68239.
  * Matthies [2020] Christoph Matthies. 2020.  Playing with your project data in scrum retrospectives. In _Proceedings of the 42nd ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)_. IEEE, 113–115.
  * Matthies and Hesse [2019] Christoph Matthies and Guenter Hesse. 2019.  Towards using data to inform decisions in agile software development: views of available data.  _arXiv preprint arXiv:1907.12959_ (2019).
  * Min et al. [2010] Zeng Min, Liu Qiong-mei, and Wang Cheng. 2010.  Practices of agile manufacturing enterprise data security and software protection. In _Proceedings of the 2nd International Conference on Industrial Mechatronics and Automation (ICIMA)_ , Vol. 1. IEEE, 318–321.
  * Mourão et al. [2020] Erica Mourão, João Felipe Pimentel, Leonardo Murta, Marcos Kalinowski, Emilia Mendes, and Claes Wohlin. 2020.  On the performance of hybrid search strategies for systematic literature reviews in software engineering.  _Information and Software Technology_ 123 (2020), 106294.
  * Olsson [2018] Helena Holmstrøm Olsson. 2018.  Challenges and strategies for undertaking continuous experimentation to embedded systems: Industry and research perspectives.  _Agile Processes in Software Engineering and Extreme Programming_ 277 (2018).
  * Pater et al. [2018] Jessica A Pater, Samantha Lie-Tjauw, Margarita Gonzalez, Moon Kim, Sheila Isbell, and David Severson. 2018.  Advancing the Agile Software Process: The case of modernizing the Army Community Service’s information technology infrastructure.
  * Rahman and Devanbu [2013] Foyzur Rahman and Premkumar Devanbu. 2013.  How, and why, process metrics are better. In _Proceedings of the 35th International Conference on Software Engineering (ICSE)_. IEEE, 432–441.
  * Rix et al. [2016] Michael Rix, Bernd Kujat, Tobias Meisen, and Sabina Jeschke. 2016.  An agile information processing framework for high pressure die casting applications in modern manufacturing systems.  _Procedia CIRP_ 41 (2016), 1084–1089.
  * Rosenkranz et al. [2017] Christoph Rosenkranz, Roland Holten, Marc Räkers, and Wolf Behrmann. 2017.  Supporting the design of data integration requirements during the development of data warehouses: a communication theory-based approach.  _European Journal of Information Systems_ 26 (2017), 84–115.
  * Schüttler et al. [2021] Christina Schüttler, Hans-Ulrich Prokosch, Michael Hummel, Martin Lablans, Björn Kroll, Cäcilia Engels, and German Biobank Alliance IT development team. 2021.  The journey to establishing an IT-infrastructure within the German Biobank Alliance.  _Plos One_ 16, 9 (2021), e0257632.
  * Spengler et al. [2020] Helmut Spengler, Claudia Lang, Tanmaya Mahapatra, Ingrid Gatz, Klaus A Kuhn, Fabian Prasser, et al. 2020\.  Enabling agile clinical and translational data warehousing: Platform development and evaluation.  _JMIR Medical Informatics_ 8, 7 (2020), e15918.
  * Svensson et al. [2019] Richard Berntsson Svensson, Robert Feldt, and Richard Torkar. 2019.  The unfulfilled potential of data-driven decision making in agile software development. In _Proceedings of the 20th International Conference on Agile Software Development (XP)_. Springer, 69–85.
  * Upender [2005] Barg Upender. 2005.  Staying agile in government software projects. In _Proceedings of the 6th International Conference on Agile Software Development (XP)_. IEEE, 153–159.
  * Vestues et al. [2022] Kathrine Vestues, Geir Kjetil Hanssen, Marius Mikalsen, Thor Aleksander Buan, and Kieran Conboy. 2022.  Agile data management in NAV: a case study. In _Proceedings of the 23rd International Conference on Agile Software Development (XP)_. Springer, 220–235.
  * Vøgt et al. [2023] Verena Vøgt, Jan-Albrecht Harrs, Vanessa Reinhart, Pia Hollenbach, Michael Max B h̆ler, and Tim Tewes. 2023.  Implementing Agile Data Workflows to Unlock Climate-Resilient Urban Planning.  _Climate_ 11, 9 (2023), 174.
  * Wohlin et al. [2012] Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012.  _Experimentation in software engineering_.  Springer Science & Business Media.
  * Zhou et al. [2016] Xin Zhou, Yuqin Jin, He Zhang, Shanshan Li, and Xin Huang. 2016.  A map of threats to validity of systematic literature reviews in software engineering. In _Proceedings of the 23rd Asia-Pacific Software Engineering Conference (APSEC)_. IEEE, 153–160.