# A Dataset and Analysis of Open-Source Machine Learning Products

Nadia Nahar<EMAIL_ADDRESS>Carnegie Mellon UniversityPittsburghPAUSA ,
Haoran Zhang  Carnegie Mellon UniversityPittsburghPAUSA ,  Grace Lewis
Carnegie Mellon Software Engineering InstitutePittsburghPAUSA ,  Shurui Zhou
University of TorontoTorontoOntarioCanada and  Christian Kästner  Carnegie
Mellon UniversityPittsburghPAUSA

###### Abstract.

Machine learning (ML) components are increasingly incorporated into software
products, yet developers face challenges in transitioning from ML prototypes
to products. Academic researchers struggle to propose solutions to these
challenges and evaluate interventions because they often do not have access to
close-sourced ML products from industry. In this study, we define and identify
open-source ML products, curating a dataset of 262 repositories from GitHub,
to facilitate further research and education. As a start, we explore six broad
research questions related to different development activities and report 21
findings from a sample of 30 ML products from the dataset. Our findings reveal
a variety of development practices and architectural decisions surrounding
different types and uses of ML models that offer ample opportunities for
future research innovations. We also find very little evidence of industry
best practices such as model testing and pipeline automation within the open-
source ML products, which leaves room for further investigation to understand
its potential impact on the development and eventual end-user experience for
the products.

††conference: arXiv; August 2024; USA

##  1\. Introduction

With the increasing popularity of machine learning (ML), more and more
software products are incorporating ML components to enable various functions,
such as face recognition in photo sharing apps and content recommendation in
social media. However, incorporating ML into products is not just about
developing the model and pipeline; there is a significant amount of effort to
integrate the model into the system, while considering aspects such as system
architecture, requirements, user experience design, safety and security
measures, and system testing and operations (rrC2w, ; RmKB, ; t8sQ, ). Thus,
developers find it challenging to convert ML prototypes into final products
and require support to overcome the challenges in building such software
products with ML components, in short, _ML products_ (kLg8, ; Nrd9, ; 7mTx, ;
a4ccL, ). Unfortunately, researchers rarely have access to source code of ML
products and hence face difficulty in (a) studying challenges in-depth and (b)
designing and evaluating interventions (e.g., tools, practices). Currently,
academic researchers rely primarily on an outside view gathered through
interviews or surveys with industry practitioners (7mTx, ; Nrd9, ; rrC2w, ;
YIKzr, ; a4ccL, ; J91H, ). Some researchers perform research within a company
and gain rich access (XWAQz, ; KABz, ), (Y9Vl9, ; 0m6Fy, ) (KZWF3, ; DIXbJ, ;
In4bN, ; lxnLR, ), but are limited to a single context and can rarely share
details. This inability to access and study ML products poses a significant
impediment to advancing research in this field, leading to a wealth of
academic literature identifying challenges through professionals’ testimonies,
but a dearth of research offering scientifically evaluated solutions or
interventions.

Table 1. Sample ML Products for Analysis, from the Curated Dataset of 262 ML Products: Mobile (P1-P10), Desktop (P11-P20), and Web Applications (P21-P30) ID |  Name |  Descrption | Star | Cont. | Users*
---|---|---|---|---|---
P1 |  Text Fairy |  OCR scanner app | 751 | 5 | 10M+
P2 |  Seek by iNaturalist |  App for identifying plants and animals | 92 | 8 | 1M+
P3 |  Pocket Code |  App for learning programming | 92 | 8 | 7M+
P4 |  ESP32 AI Camera |  ESP32-CAM processing AI tasks | 82 | 1 | 1K+
P5 |  NotionAI MyMind |  App to store and search for web/text/image | 182 | 2 | 1K+
P6 |  Organic Maps |  Offline map app | 4023 | 226 | 500K+
P7 |  VertiKin |  E-commerce app to search/browse products | 74 | 5 | N/A
P8 |  FlorisBoard |  Android keyboard | 3503 | 76 | N/A
P9 |  NewsBlur |  Personal news reader | 6123 | 83 | 50K+
P10 |  TfLite MNIST |  Handwritten digits classification | 214 | 1 | N/A
P11 |  AWIPS |  Advanced weather processing system | 129 | 6 | 97K/mo
P12 |  Audiveris |  Optical musical recognition app | 932 | 16 | 8.7K/mo
P13 |  Datashare |  Document analysis software for journalists | 438 | 15 | 1.2K/mo
P14 |  Algoloop |  Algorithmic trading application | 67 | 160 | N/A
P15 |  Subtitle Edit |  Editor for video subtitles | 4407 | 86 | 293K/mo
P16 |  DeepFaceLab |  Software for creating deepfakes | 35566 | 19 | N/A
P17 |  Faceswap |  Software for creating deepfakes | 42623 | 80 | 297K/mo
P18 |  HO |  Helper for Hattrick online football manager | 138 | 12 | 14M+
P19 |  BigBlueButton |  Web conferencing system | 7710 | 181 | 88K/mo
P20 |  PoseOSC |  Realtime human pose estimation | 63 | 2 | N/A
P21 |  OpenBB Terminal |  Investment research software | 17481 | 136 | 94K/mo
P22 |  Coffee Grind Size |  Coffee particle analyzer | 402 | 1 | N/A
P23 |  Celestial Detection |  Classifier of celestial bodies | 69 | 20 | N/A
P24 |  Electricity Maps |  Greenhouse gas intensity visualizer | 2566 | 268 | 3M+
P25 |  Galaxy |  Data intensive science for everyone | 1021 | 255 | 187K/mo
P26 |  GridCal |  Power systems planning software | 293 | 14 | N/A
P27 |  Honkling |  Keyword spotting system | 63 | 5 | 1.2K/mo
P28 |  Jitsi Meet |  App for video conferencing | 18813 | 374 | 10M+
P29 |  Code.org |  Professional learning program for CS | 712 | 132 | 82M+
P30 |  Basketball Analysis |  Analyze basketball shooting pose | 781 | 4 | N/A
*There is no reliable way to calculate the number of users; we report them using multiple ways if available, such as downloads in google play-store, self-reported on website, or website traffic tracker (similarweb.com) to count average monthly users (in ‘value/mo’ format)

Historically, the field of software engineering has greatly benefited from
open-source software in terms of research, practice, and education. Research
on mining open-source software repositories has allowed us to create vast
datasets and study aspects that would otherwise be challenging to investigate
(Ek3M, ; Zxtq, ; Tx4J, ), such as the effectiveness of continuous integration
(FW2m, ) and pull-request-based development (xt2Q, ). Open-source software
allow researchers to test interventions on software artifacts, forming the
basis for many research areas such as program repair and software testing
(azR0, ; pkk9, ; 2h5i, ). Many innovations developed and evaluated on open-
source software are later adopted in industry, such as Facebook’s adoption of
program repair (SsxR, ) and Google’s adoption of mutation testing (vzrE, ).
Beyond research and development, access to open-source software on a larger
scale has revolutionized education, offering students and professionals the
opportunity to study practices in open-source repositories (Uq5a, ; meZx, ),
and educators countless real-world examples (Hr7T, ; r2EI, ). In the same
manner, access to open-source _ML products_ could open opportunities for
research, education, and technology transfer. To this end, we aim in this
paper to identify a corpus of such ML products.

Many past studies have tried to study ML projects in open source, but usually
(a) only focus on one or two specific examples or (b) use a dataset composed
of notebooks, research projects, homework solutions, and demos, which are not
representative of real-world industrial ML products. While open source was
useful for studying ML libraries and notebooks (TBW6, ; IUr7, ; P8le, ),
researchers struggle to find good examples of open-source _ML_ _products_ for
studying: Several papers prominently highlight FaceSwap (1n0g, ) as an active
and fairly mature end-user open-source ML product and study it in depth, but
it is usually the only example ever identified or analyzed (Ku84, ; bGZK, ).
Several papers (OmEh, ; dx7s, ; TUkr, ) rely on a dataset of 4,500 projects
labeled as “ML applied,” curated by Gonzalez et al. (YDPU, ), but closer
inspection reveals that most of these projects are packages for developers,
research notebooks, tutorial-style projects, and toy projects – not a
promising dataset for those interested in studying ML products.

We have two goals for this paper, (a) to define and identify ML products in
open-source, beyond just FaceSwap, and (b) to analyze the curated dataset to
answer research questions of interest to the community. The first goal ended
up being surprisingly difficult due to the abundance of open-source ML
projects that are not ML products, making the keyword search approach used in
previous studies ineffective. In response, we designed a best-effort approach,
strategized specifically for finding _ML products_ rather than any ML
projects. We collected a total of 262 repositories (YDPU, ). While this is a
smaller corpus than past datasets on ML projects, it provides a considerable
number of open-source ML products that have a development history and are
fully transparent, providing opportunities not achievable with interviews,
surveys, and even industry collaborations.

Our second goal is to analyze the dataset to answer research questions for
which open-source software might provide useful insights. Instead of a shallow
quantitative analysis of all the products, we conducted an in-depth analysis
of a 30-product sample (Table 1) of the corpus, and reported 21 findings
around six research questions related to collaboration, system architecture,
development order, testing, operations, and responsible AI. Among others, our
findings reveal (a) more involvement of software engineers compared to data
scientists in developing ML products, with a lack of clear boundaries between
contributor responsibilities and ML and non-ML code, (b) diverse architectural
choices on incorporating models into products, heavily influenced by how the
model predictions are processed within the product, and (c) several industry
best practices for ML development, such as pipeline automation, model
evaluation, and monitoring in production, not being prevalent amongst these
open-source ML products. While the dataset is unlikely to fully represent
large-scale ML products in industry, its products provide ample opportunities
to study development practices and interventions around models and products.

To summarize, the primary contributions of this paper are: (a) a definition of
ML product, (b) a search strategy for retrieving a dataset of ML products from
GitHub, (c) an open-source dataset of 262 ML products, and (d) 21 findings
around six broad research questions, characterizing the nature of ML products
in the dataset.

##  2\. Defining ML Products

Throughout the paper we use the term _ML product_ to describe software
products for end-users that contain ML components, and to distinguish them
from other ML-related software projects and artifacts. Note that terminology
in this field is not standardized or consistent, as practitioners and
researchers may refer to the libraries that train models (e.g., Tensorflow),
the code to train models (e.g., in a notebook or with pipeline automation),
the trained or deployed models (e.g., GPT-3), or the products around those
models (e.g., FaceSwap) with names such as ML systems, ML projects, or ML
applications. Our notion of _ML product_ considers the entire software system
including non-ML components, in line with past research that used terms like
ML-enabled systems (OqUB, ; rrC2w, ), ML-infused systems (SDhx, ), or just ML
systems (fC0u, ; Nrd9, ).

During our research, we needed a clear definition for what we consider an _ML
product_ (especially because we had to classify thousands of repositories,
details in Section 4.1). We started with a working definition and refined it
repeatedly when we encountered ambiguous cases. We finally settled on the
following definition:

ML Product: A machine-learning product is a software project (a) for end-users
that (b) contains one or more machine-learning components.

To be considered _for end-users_ , the project must have a _clear_ _purpose_
and a _clear target audience_ , e.g., the purpose can be “fun” and the
audience can be “everybody.” Software developers and data scientists are
explicitly excluded from the target audiences. The software must be _complete,
usable_ , _polished, and documented_ (e.g., install and usage instructions) to
the level typically expected by the target audience (commensurate with the
expected skills of the target audience, e.g. “pip install” is okay when
targeting scientists but not for a general audience). For server-side
applications, more technical installation instructions are acceptable, if the
application is easy to use by clients once the server is installed.
Additionally, the product needs to use at least one machine-learned model that
is used for major or minor functionality of the software. The model can be
developed from scratch or called using an existing library or API, and may
have been trained with any ML algorithm.

For contrasting, we define ML library and ML project:

ML Library: Libraries, frameworks, or APIs that are used to perform ML tasks.
Examples of popular ML libraries include TensorFlow, Scikit-learn, and
PyTorch.

ML Project: ML Project represents any software project that integrates some
form of ML functionality or code. Examples include exploratory notebooks, code
to perform partial ML tasks, research artifacts associated with a paper, and
course homework. All ML products are ML projects, but most ML projects are not
ML products.

Why do we exclude software developers and data scientists as end-users? While
software developers and data scientists are technically end-users of ML
libraries (e.g., TensorFlow) and ML-based tools (e.g., GitHub CoPilot and
AutoML), we have deliberately excluded them from our definition of ML
products. This is because their expertise allows them to interact with
software differently compared to non-technical users, which may not reflect
the typical use and development of general-purpose software. Software for
these technical users may include complex installation instructions, making it
hard to distinguish ML products from other ML projects. Therefore, to avoid
confusion and focus on general-purpose software, we exclude these technical
users from consideration.

##  3\. Existing Research and Limitations

Building ML products is challenging and requires engineering beyond the model
and ML pipeline. Many researchers have studied the challenges that
practitioners face when turning an ML model or prototype into a product. Nahar
et al. (qMjN, ) collected the challenges from 50 papers that surveyed and
interviewed practitioners regarding the software engineering challenges faced
when building ML products. These papers illustrated numerous challenges, such
as vague ML specifications that make it difficult to align business goals with
performance metrics (7mTx, ; xgBR, ), architectural issues due to lack of
modularity in ML code that increase design complexity (swt2, ; Nrd9, ), and
team collaboration hindered by the absence of necessary skills (SuxN, ; rrC2w,
). While these papers provide us with a high-level understanding of the
problems, they often do not offer sufficient details or access to design
specific interventions.

Researcher tradeoff between internal and external validity for studying ML
products. For studying ML products (not just models and ML projects),
researchers have adopted different research designs. Some conducted interviews
with teams and organizations, e.g., (7mTx, ; Nrd9, ; swt2, ; rrC2w, ), while
others focused on surveying practitioners at scale, e.g., (O0jVj, ; a4ccL, ;
J91H, ; pvWrX, ). While these studies provide a broad sense of the challenges
(maximizing external validity), they rely on self-reported data without access
to artifacts. There are a few ethnographic studies (Y9Vl9, ; 0m6Fy, ),
industrial case studies (XWAQz, ; GtS4, ), experience reports in academic
venues (KZWF3, ; In4bN, ; lxnLR, ; rWfO, ; 9H3c, ), talks (aWlif, ; gOivy, ;
h2dfy, ), and blog posts (GkgWi, ; I1fwo, ; 9mveZ, ) – these can yield a
deeper understanding of specific cases (maximizing internal validity), but at
the cost of generalizability as they are usually based on a single case (a
common tradeoff (I0mg, )). Access to an open-source dataset of ML products can
provide new opportunities, enabling researchers to validate reported
challenges on tangible data, devise solutions, and evaluate interventions
across a larger number of cases.

Academic solutions exist for ML model and pipeline components where plenty of
notebooks and libraries exist in open source for studying, but not for
_product-level_ problems to which academics hardly have access. Academics can
design and evaluate solutions when adequate data is available, such as the
millions of notebooks on GitHub (IUr7, ; P8le, ) supporting studies and
evaluations of interventions on dependency management, documentation
generation, and collaboration practices in notebooks (CN2b, ; Krxa, ; 1JJ8, ;
IUr7, ). We also find solutions around components surrounding the model, such
as tools for data validation (Ig3x, ; pxhw, ; xpTX, ), training data creation
(wCk6, ; enBo, ), model building (KcIo, ; 6nq1, ), and fairness assessment
(zE76, ; sIXj, ; PTwM, ). However, as academics usually do not have sufficient
access to study ML products, it is challenging to design and rigorously
evaluate interventions at the product level (including architecture, testing,
collaboration, and documentation).

Open-source datasets exist for ML projects, but they are not representative of
_ML products,_ and not suitable for answering research questions related to ML
product development. We find papers introducing datasets of ML _projects_ on
GitHub (not necessarily ML _products_). For instance, Gonzalez et al. (YDPU, )
classified over 4,500 repositories in their dataset as “applied AI and ML.”
The quality of this dataset was criticized for inclusion of toy projects by
Rzig et al. (Ku84, ), who then curated a new dataset with 2,915 ML projects
for studying the adoption of CI. Similar datasets of ML projects were curated
by van Oort et al. (EMW5, ), and Tang et al. (wI1Q, ) for purposes such as
discovering the prevalence of code smells and refactoring practices in ML
projects. Widyasari et al. (4LDy, ) proposed a manually labeled dataset of 572
‘engineered ML projects’, referring to the need for a higher-quality dataset
that should include engineering practices. While all these datasets can
provide various insights about _ML_ _projects_ , they are not promising for
studying concerns related to developing ML _products_. We argue that insights
about architecture (OmEh, ), technical debt (TUkr, ), and differences in
addressing ML and non-ML issue reports (dx7s, ) from studying ML projects
might not well generalize to ML products.

Despite curating the many datasets of ML-related projects presented above,
researchers have not succeeded in finding ML products in open source, apart
from the one sole but frequently highlighted example, FaceSwap (1n0g, ).
Closest to our work, Wan et al. curated a dataset of ML products where a model
is used in the context of some non-ML code (in their case to test the code
_using_ the model). Unfortunately, all their projects are toy projects, such
as a 0-starred fire-alarm “application” with 3 commits that uses an object
detection

model to get object labels from an input photo and prints “alarm” if it
detects the keyword “fire” in the label (N5c4, ).

Existing datasets of ML projects and toy examples limit the generalizability
of research to real-world industry-style _ML_ _products_. Even worse, if a
reader is not aware of the types of projects in a dataset (which is not
obvious when the example provided is FaceSwap), it is easy to incorrectly
generalize findings to _ML products_. Thus, there is value in a dataset
specifically dedicated to ML products, which motivates us to curate and study
such a dataset.

##  4\. Contribution A: Acquiring the Dataset

As mentioned earlier, compiling a dataset of open-source ML products can
provide great research opportunities. In this section, we discuss how we
curated such a dataset.

###  4.1. Research Method

####  4.1.1. Search Space and Scope

As defined in Section 2, we search for _ML products_ in open-source that (a)
are developed for end-users, and (b) contain one or more ML components. We
select GitHub as our search space for ML products as it is the most popular
open-source platform for software projects, unlike platforms such as Hugging
Face which only host ML models. We only include popular project repositories
(over 50 stars) that have been maintained recently (updated after 2019-01-01),
and that are documented in English.

####  4.1.2. Search Strategy

Existing papers that mine ML projects (not necessarily ML products, cf.
Section 3) employ a keyword-based search approach to find repositories by
matching ML or AI-related keywords in project titles, descriptions, and readme
files. We identify two problems why this approach is not suitable for
effectively finding _ML products_ on GitHub. First, the search keywords are
associated with many projects that are not products, making it hard to
distinguish the actual products. Second, the visibility of ML functionality
within a larger product is often limited, making it unlikely to get all
relevant repositories solely through keyword searches.

_Finding “Products:”_ We are looking for ML _products_ , not just any ML
project. There is an abundance of other types of projects outside of our scope
in GitHub, such as ML libraries, tutorials, and toy projects. These projects
contain similar keywords to ML products, and therefore keyword searches return
a large number of unrelated projects, causing ML products to get lost in the
crowd.

To address this issue, we use _code search_ , instead of relying solely on
keyword search. We differentiate products from projects based on code
fragments that indicate end-user applications, such as the use of GUI
frameworks.

_Finding “ML:”_ While some end-user products prominently advertise their use
of ML, others may just use ML internally without mentioning it in the product
description, especially when ML is associated with minor functionality – this
is the case for 13 of our 30 analyzed ML products, e.g., product P29 in our
sample, uses facial expression detection as an add-on within a video-
conferencing application. In such cases, the existing strategies to rely on
keyword search may miss these products.

To identify ML, we again rely on code search rather than keyword search in
readme files. We detect different ways the ML-related code can be incorporated
into the repository – e.g., calling libraries, pre-trained ML models, or
third-party APIs.

_Validation of “ML Products:”_ We combine the two code-search strategies to
find ML products. However, we realize that manual validation is unavoidable as
the detection methods only serve as indicators for ML products, but are not
always conclusive. Our attempts at fully automating detection of relevant ML
products (with and without using ML itself) yielded only poor accuracy. Even
humans can find it difficult to classify these projects and in many cases,
often multiple authors had to discuss projects to agree on a classification.
So, instead of trying to automate detection entirely, we used our indicators
to inform and guide _manual inspections_.

Figure 1. Overall Process of ML Product Mining in GitHub

####  4.1.3. Implementing and Scaling the Search

The key to our implementation is to break down the vast search space into
manageable units, making the process more feasible and efficient. We recognize
three genres of end-user ML products (i.e., mobile, desktop, web) and
customize search steps for each genre. To optimize the search, we filter
projects in multiple stages (illustrated in Figure 1), as early as possible to
minimize effort for downloading and manual analysis.

_GitHub API Search:_ Due to the millions of public repositories on GitHub, it
is impractical to download and search through all of them in our study. While
the GitHub API allows to query repositories based on various parameters, such
as keywords, language, and date, it does not provide an efficient method to
search for specific code fragments across all GitHub projects. We perform code
search locally after cloning repositories, but only after first reducing the
number of candidate repositories with an initial filter.

In addition to the previously mentioned filters for popularity (over 50 stars)
and recency (updated after 2019-01-01), we consider only repositories with the
following programming languages as their top language: Javascript, Python,
Java, and C#, which are the most common languages for desktop and web
applications (8yZB, ; IeKU, ). We additionally include projects matching the
keywords “android” and “ios,” because these keywords are reliable for
detecting mobile applications. All these conditions can be expressed in the
GitHub search API. Where necessary, we further partition the search space by
date to overcome GitHub’s maximum of 1000 search results. At this stage, we
identified 430,902 candidate repositories.

_Initial Filter:_ Next, we collected metadata and readme files to further
exclude projects: We exclude repositories containing keywords such as
“deprecated” and “obsolete” in metadata or readme, archived projects, forked
projects, and projects with non-English descriptions (determined with an
existing ML classifier (tBF5, )). We also exclude obvious non-products by
matching keywords such as “framework”, “tutorial,” and “demo” in the
repository description (full list in the appendix (LwCB, )). To ensure that we
did not exclude relevant projects during the filtering steps, we validated a
random set of filtered projects finding no incorrectly filtered projects. A
total of 300,508 repositories remained after applying this filter.

_Clone Code and Product Filter:_ Next, we cloned each candidate repository and
performed code search locally. To develop code search strategies for likely
products, we examined each product genre (mobile, desktop, web) for
identifiable code structures. We discovered language and platform-specific
code structures and frameworks for each genre. For instance, we found that a
_gradle.build_ file must contain the string $com.android.application$ to make
an Android application runnable. In desktop and web applications, certain
frameworks (e.g., javafx, spring, pyqt5) and code fragments (e.g.,
$make\\_server$, $.serve\\_forever()$) are used for GUI and server code. We
compiled a list of such frameworks and code fragments (available in the
appendix (LwCB, )) and searched for them in the cloned project code. Projects
that did not contain these code fragments or frameworks were filtered out,
leaving us with 26,386 potential products for further analysis.

_ML Clues:_ Next, we search for the presence of ML components in the source
code of the remaining repositories. We use four criteria to search and count
occurrences of the following clues in the entire codebase – (1) popular ML
libraries and frameworks such as TensorFlow, and programming language-specific
ML libraries such as Deeplearning4j (e.g., detected through import
statements), (2) code references to popular external ML APIs and libraries
such as Google’s Vision API, (3) trained model files with known extensions
such as _.tflite_ , and _.mlmodel_ , and (4) general ML keywords such as
“machine learning” and “NLP” in all code and text files (which included readme
files and source code comments), accounting for the potential use of less
popular or custom APIs. These counts act as signals that the repository might
contain ML components, but they are not conclusive and often affected by
noise. Some signals, such as the use of the TensorFlow library, are strong,
while others, such as the occurrence of “NLP” in text are weak. Therefore,
instead of using these counts to filter out projects directly, we employ them
for guiding our manual inspection in the final stage. At this point, 11,257
projects have at least one ML clue.

_Manual Inspection:_ We manually inspect candidate projects to verify whether
they truly contain a “product” with “ML” components. However, considering the
large number of repositories that could potentially contain ML products,
inspecting all of them manually would be impractical. Instead, we guide manual
inspection based on the ML clues identified earlier.

We prioritize repositories with strong ML clues, such as specific ML libraries
(clue 1), and then move on to repositories with weaker ML clues, such as
general ML keywords (clue 4). To streamline the process, we perform multiple
iterations, ranking the repositories by different clues each time. During each
iteration, we manually inspect the top-k repositories, dynamically selecting
an appropriate k. We stop our manual inspection when we observe a significant
number of projects that do not fit our definition of ML product, typically
after encountering 30 consecutive cases. Using this method, we examined
approximately 4,000 repositories (spending 30 seconds to 20 minutes on each)
and identified 262 ML products.

####  4.1.4. Limitations and Threats to Validity

Our approach is not exhaustive and we do not claim completeness. We designed a
best-effort attempt to compile a dataset of ML products, but recognize ML
products in specific ways based on GUI and ML code fragments, that might not
capture all products. We focused on the most popular categories for end-user
applications but may have missed products from other possible categories, such
as embedded systems, smartwatch applications, and games. Also, despite our
best efforts to compile a list of ML-related libraries and keywords for code
search in the _ML Clues_ stage, we might have missed some, and may hence miss
some ML functionality. Additionally, our approach involved manual inspection,
which despite best efforts opens the possibility of human error and
subjectivity.

While we would have preferred to evaluate recall (i.e., whether we have missed
any ML products) by comparing our dataset against an existing ground-truth
dataset of ML products, such a dataset does not exist. Instead, we attempted
to collect ML products independently to evaluate whether our approach would
find them, by seeking input from industry practitioners through platforms like
Quora, Reddit, LinkedIn, Twitter, and a Slack channel with 32k members in the
field of data science; but we only got two projects in response (and a number
of answers expressing interest in our dataset), both of which we determined
not to be ML products according to our definition. Additionally, we compared
our dataset with other existing datasets (YDPU, ; 4LDy, ; TBW6, ; wI1Q, ) but
did not find any additional projects that we consider ML products in those
datasets. In fact, those datasets only contained a total of four ML products,
all of which we detected for our dataset.

###  4.2. The Open-Source ML Product Dataset

In total, we found 262 ML products (cf. Table 2, full dataset in the appendix
(LwCB, )). The average ML product has 1,495 stars, 28 contributors, and 325MB
of codebase size (cf. Figure 2). Over half of the ML products are written in
Python; most are web applications.

Table 2. Number of Retrieved Projects after Each Step After | Mobile App | Desktop App | Web App | Total
---|---|---|---|---
android | iOS | js | py | java | C# | js | py | java
GitHub Search | 12044 | 10969 | 72793 | 67626 | 55892 | 15267 | 72793 | 67626 | 55892 |
Initial Filter | 3358 | 4055 | 83777 | 36396 | 22802 | 7145 | 83777 | 36396 | 22802 |
Product Filter | 2296 | 2801 | 1100 | 1663 | 1909 | 2590 | 3025 | 8747 | 2255 |
Manual Check* | 33 | 14 | 19 | 43 | 17 | 12 | 42 | 104 | 5 |  = 262**
*Based on the ML clue counts, we inspected around 4k projects manually. **Removing duplicates.

The dataset comprises a diverse range of products, some of which, have a
significantly larger number of users and a more professional look than others.
For instance, _Seek_ (P2 in Table 1) is a mobile app for identifying plants
and animals using image recognition, downloaded over 1 million times and
reviewed by over 38k
users.111https://play.google.com/store/apps/details?id=org.inaturalist.seek,
https://apps.apple.com/app/apple-store/id1353224144 _Seek_ appears to have
robust support from the established _iNaturalist_ community, who maintains a
dedicated website and continuously improves and maintains the app. In
contrast, _NotionAI MyMind_ (P5), an Android app developed by a single
contributor, enables users to add images or web articles, which are
subsequently tagged automatically by its ML classifier for easy searching
capability. This app has a relatively simple user interface, is rarely
updated, and only attracted over 1,000 downloads from the Play
Store.222https://play.google.com/store/apps/details?id=com.elblogbruno.notion_ai_my_mind

Approximately half of the products in our dataset have a professional
presentation like _Seek_ ; those generally have more stars and a larger
codebase. The others, including _NotionAI MyMind,_ are personal-interest
projects released as a product.

Figure 2. Density plot of number of stars, contributors, and codebase, with
outliers removed

##  5\. Contribution B: Learning from the Dataset

We created this dataset due to the limited access to industry products, which
hinders research advancements in the field. Now that we have this dataset, we
can finally attempt to answer numerous research questions about ML products
that have accumulated in many past studies with open-source products, where
previously we had to rely on interviews or experience reports from industry
practitioners. Given the breadth of topics, we cannot cover everything in a
single study. In this paper, we explore a wide range of topics rather than
going into depth on a single one, allowing us to answer many research
questions and report findings that may guide future research directions. In
addition to contributing new knowledge, this approach helps us achieve two
secondary objectives: _(1) Characterize the dataset:_ Our analysis with broad
research questions will implicitly characterize the products in our dataset
and enable other researchers to effectively use it and interpret derived
findings (cf. limitations of existing work in Section 3) – this also helps to
explore how similar the ML products in our dataset are to ML products
described in interviews and experience reports. (2) _Identify when deeper
analysis of the dataset is feasible and promising:_ Our research questions
from different topics, such as collaboration, architecture, and development
process will identify what kind of questions are worth going into deeper,
before committing resources to in-depth analyses for individual research
questions.

###  5.1. Research Method

First, we curated a list of research questions relevant to the study and
designed qualitative and quantitative strategies to answer them. Given the
novelty of the dataset and questions, we heavily rely on qualitative analysis
involving substantial manual effort. Therefore, we decided it would be more
manageable to analyze a sample of 30 products from the dataset.

####  5.1.1. Deriving Research Questions

We selected research questions that are not only of interest to the research
community but can also be feasibly answered by analyzing open-source ML
products (e.g, we did not commonly find artifacts that would allow us to
answer _“How do data scientists elicit, document, and analyze requirements for
ML systems?”_). For this selection, we employed a two-step process.

First, we explored existing literature to identify topics that are of interest
to researchers in the field, such as the challenges faced by practitioners
building ML products, collected in a recent meta-survey of interviews and
surveys (qMjN, ). We identified numerous topics of interest, such as
collaboration, architecture, development process, quality assurance, MLOps,
and responsible AI.

Second, we examined our dataset of ML products to identify potential research
questions (RQs) that could plausibly be answered with open-source data. We
randomly explored 15 ML products qualitatively, taking multiple pages of notes
for each product. We immersed ourselves in the source code, documentation,
contributor profiles, issues, and any other available information provided on
associated websites; we identified the ML and non-ML components to become
familiarized with common structures.

In the end, we selected a broad set of questions spanning the entire life
cycle: $\bullet$ RQ #1 (Collaboration): How interdisciplinary are open-source
ML product teams and how do they divide work? $\bullet$ RQ #2 (Architecture):
How are open-source ML products architected to incorporate models? $\bullet$
RQ #3 (Process): What model-product development trajectory do open-source ML
products follow? $\bullet$ RQ #4 (Testing): What and how are the open-source
ML products and their parts tested? $\bullet$ RQ #5 (Operations): How are
open-source ML products designed for operation? $\bullet$ RQ #6 (Responsible
AI): What responsible AI practices are used in open-source ML products?

While each identified topic could warrant a dedicated study and deeper
analysis, here, we provide initial answers for each and explore opportunities
for future research.

####  5.1.2. Analysis and Synthesis

Without existing established measures, we found manual mostly-qualitative
analysis to be more responsive and effective than a narrow quantitative
analysis at scale. To find answers to the RQs, the products in the dataset
required an in-depth examination of the code, analysis of contributor
activities, and thorough inspection of related documents. While we automated
some measures, such as contributor percentages and modularity scores,
designing them also required initial manual investigation and manual
classification of ML components. This made the process quite labor-intensive,
requiring 10-15 hours per product. Consequently, we employed a sampling
methodology rather than analyzing every product in the dataset.

_Sampling:_ We analyzed 30 products (11.5% of the dataset, shown in Table 1),
which was manageable for our manual analysis. Our goal was not to make
statistical generalizations but rather to gain insights into the dataset by
sampling a wide range of products; thus we used different qualitative sampling
techniques to ensure diversity in our sample. Following case-study research
logic (oMJD, ), we use _information-oriented selection_ and stratification to
maximize the utility of information from small samples: We select popular and
large products (_extreme/deviant cases_), which may be closer to industry
products, to explore the potential of this dataset as an alternative to
industry products; we stratify selection across the three genres mobile,
desktop, and web; and also include a random selection of other products
(_average cases_). Specifically, we select two products with the most stars
per genre (P8, P9, P16, P17, P21, P28), two products with the most
contributors per genre (P3, P6, P14, P19, P24, P25), the product with the
largest code size per genre (P1, P11, P29), and five randomly selected
products per genre.

_Analyzing Products and Card Sorting:_ We conducted a comprehensive
qualitative analysis of the sampled products. We describe the specific
analysis steps separately for each research question below, but generally, we
follow the strategy: two researchers carefully examined the GitHub
repositories, addressing each research question individually for each product,
involving tasks such as reading documentation, identifying ML and non-ML
components in the source code, measuring modularity, examining contributor
profiles, analyzing commit history, and reviewing issues. The entire research
team regularly met to discuss, clarify understanding, resolve disagreements,
and organize findings. To organize and find patterns among the products, we
performed card sorting (ZAks, ). Each product was represented by a card for
each RQ, describing our findings for that particular RQ, and we iteratively
grouped these cards to identify patterns within and across RQs. Additionally,
we sought to understand relationships among characteristics across multiple
questions by comparing how products were classified across different
dimensions of analysis. We share analysis artifacts, including a Miro board
and spreadsheets, in the appendix (LwCB, ).

####  5.1.3. Threats to Credibility and Validity

This part of the research shares common threats encountered in qualitative
research. Given the small sample size and sampling strategy, statistical
generalization is not suitable and not advised. While we followed standard
practices for coding and memoing during the analysis of the products, as is
typical in qualitative research, we cannot completely eliminate biases
introduced by the researchers. In addition, we only access public information
and do not have access to offline activities, and did not talk to developers
in those products – hence, our findings should be interpreted accordingly
(construct validity), for example, regarding developer backgrounds and
development order.

### RQ #1: How interdisciplinary are open-source ML product teams and how do
they divide work?

Interdisciplinary collaboration is difficult, and this has been found also in
building ML products (y2A6, ; rrC2w, ). The transparency of open-source
development allows researchers to study many aspects of collaboration, as
demonstrated by numerous past studies on team collaboration (iYmX, ; gBO9, ),
pull requests (G2dk, ), and diversity (B8Ur, ). To understand
_interdisciplinary_ collaboration in open-source ML products, we explore team
composition in terms of the number of contributors and their backgrounds, who
works on the ML and the non-ML code of the product, and how tangled ML and
non-ML code are in terms of co-changes. The findings can help study existing
challenges such as siloed development and uncover further opportunities for
studying collaboration.

####  5.1.4. Method

To analyze contributor backgrounds and numbers (_Findings 1-2_), we collected
contribution data from GitHub and identified the _core_ contributors as those
collectively responsible for 80% of all commits (in line with past work (RJSl,
; Jz8a, )). We manually classified each contributor’s background as _SE-
focused,_ _ML-focused_ , or _other_ (e.g., physics, finance), based on public
self-description, professional title, and education history stated in their
GitHub profile, LinkedIn profile, and personal or company websites, if
available. We also examined the repository organization and related web pages
for finding responsibility assignments. If the classification was not obvious
(e.g., because of limited public information) we refrained from assigning a
background and classify them as _“unsure.”_

To study how developers from SE and ML backgrounds contribute to ML and non-ML
code (_Findings 3-4_), we separated the ML and non-ML code (documentation and
binaries excluded) manually: We categorized code associated with model
training, prediction, and pipeline as ML-related, while all other software
infrastructure and graphical user interface (GUI) code fell under non-ML,
typically at the granularity of files. Finally, we analyze the commit history
to attribute code changes to contributors.

To analyze coupling of the ML and non-ML code (_Finding 5_), we analyze co-
edits in the history of the product, known as _logical coupling_ (Di9F, ;
iqpc, ). Specifically, we compute a _relative coupling index_ that indicates
whether the ML and non-ML parts are more or less coupled than would be
expected if all changes were randomly distributed across files. A low
_relative coupling index_ indicates that changes are typically isolated to
only ML code or only non-ML code, whereas a high _relative coupling index_
indicates that ML and non-ML code are often changed together and is a sign of
low modularity. To compute _relative coupling index_ , we use _evolutionary
coupling index (ECI)(Di9F, )_ to compute coupling and then divide _ECI_ by the
probability of coupling of random edits to normalize the effect of size, as ML
and non-ML code size differs significantly (the average product has 971k LOC
non-ML code and 23k LOC ML code).

####  5.1.5. Finding 1: Many products have a single core contributor (13/30).
Finding 2: Most of the core contributors are software engineers (74/92)

In our sample of 30 products, we identified 140 core contributors, among whom
we could classify 92. Of these, 74 contributors are _SE-focused,_ whereas only
10 (spread across eight products) are _ML-focused_. For the products where we
could classify all core contributors, most (9/16) had exclusively _SE-focused_
contributors. We found a single contributor (in P24) who self-identified as an
expert in both SE and ML, which is a rare and highly sought-after combination
in industry, often referred to as “unicorn.”

In line with general trends in open source (OXc0, ; pYnK, ), we find
relatively small teams, in contrast to often larger teams reported for
industry ML products (swt2, ; 0qnc, ; MPUw, ). Studies of industry products
tend to report a much more significant involvement of data scientists, often
as the dominant role in the product, which we observed as less common in open
source. Overall, GitHub remains primarily a platform for software engineers,
with limited apparent participation from data scientists, even in the context
of ML products.

####  5.1.6. Finding 3: There is little evidence of clear silos, with core
contributors commonly committing to both ML and non-ML code, regardless of
background

We do not find a clear delineation where _ML-focused_ contributors work solely
on ML code and _SE-focused_ contributors work exclusively on non-ML code. In
products with substantial amounts of both ML and non-ML code, we often find
contributors of either background working on

both parts – in contrast to the widely reported problem of _siloing_ in
industry teams (rrC2w, ).

####  5.1.7. Finding 4: Team responsibilities are rarely assigned and
recognizable in the commit history

Only five products (P13, P21, P24, P25, P29) publicly documented team
structures with assigned roles and responsibilities for team members. However,
the assigned responsibilities are not always reflected in the commit history.
For instance, even though P29 has an explicit data team and a separate ML
repository, we do not find commits from the data team contributors to the ML
repository. Instead, it is the _SE-focused_ contributors who commit to both
the ML and non-ML repositories. We conjecture there may be some offline
collaboration between the _SE-focused_ and _ML-focused_ contributors, which is
not visible in the open-source repository.

Previous studies analyzing team collaboration in industry reported misaligned
responsibility assignments that do not reflect developers’ abilities or
preferences (y2A6, ; FzPp, ; rrC2w, ). Our findings signal towards similar
trends as team responsibilities are not clearly assigned, and developers
perform model and engineering work, regardless of their backgrounds.

####  5.1.8. Finding 5: ML and non-ML code are often changed together,
indicating low levels of modularity

While one could expect ML and non-ML code to change independently as the model
is often simply called as a function and could be developed as a distinct
service by an independent team, our analysis reveals that a substantial
portion of the products (12/30) in our sample exhibit frequent co-changes of
ML and non-ML code. For example, P27 has a very high _relative coupling index_
between ML and non-ML code – upon manual inspection, we found that this
product has a custom script for training a speech recognition model, where the
ML code directly updates the button color in the user interface (UI) based on
the prediction result; any modification to the UI properties requires an
update to the model script to accommodate the change, causing frequent co-
changes of the UI files and the model script. Conversely, although we indeed
found many products (18/30) that have coupling lower than random, it is not as
low as could be expected from fully modular components of a product. These
numbers corroborate the existing modularity concerns, where changes in models
propagate beyond module boundaries in unexpected and non-monotone ways (ftJI,
; Ijae, ; JyHd, ). These complicated interactions pose significant challenges
to integrating models into products.

In summary, for RQ 1, our findings suggest that open-source ML products are
mostly developed by small groups of mostly software engineers, with limited
involvement of data scientists, and without a clear modular separation of ML
work.

### RQ #2: How are open-source ML products architected to incorporate models?

Researchers have highlighted how ML can influence the architecture of software
products (swt2, ; Nrd9, ). To comprehensively understand the structure of the
products and the incorporated models, we explore multiple architecturally
relevant aspects such as model type, usage, importance, existence of multiple
models, pipeline automation, documentation, and big data infrastructure.

####  5.1.9. Method

To understand the overall structure of the model and product, we conducted a
comprehensive manual analysis of the ML and non-ML code in the repositories.
We use our usual qualitative methods and focus on the following artifacts:
code structure and data flow as it pertains to models – identifying how the
models are created, where and how they are called, and how the model
predictions are processed and used. We also reviewed their documentation,
relevant blogs and forums, associated web pages, and related repositories
under the same personal or organizational account. We then sorted our findings
and grouped those into categories, using card sorting techniques (ZAks, ), and
guidance from previous research and domain knowledge from our research team.

####  5.1.10. Finding 6: About half of the products rely only on third-party
ML models (13/30)

We identified 15 products that use _third-party models_ via libraries (such as
Tesseract OCR), external APIs (such as ClarifyAI), or by loading pre-trained
model files (such as a tflite model) from a remote URL or repository. In
contrast, 17 products _self-train models_. Two products use both _third-party_
or _self-trained models_ (P12, P30). For instance, the optical music
recognition application, P12, uses a self-trained model to classify music
symbols and an existing OCR library for classifying text. With a significant
portion of the sampled products only integrating third-party ML models
(13/30), this finding resonates with _Finding 2_ , that most contributors,
being software engineers, develop ML products by leveraging pre-existing
models.

####  5.1.11. Finding 7: The importance of the ML models to the product
varies, with about half using them as optional functionality only (13/30)

We found the importance of ML models to vary considerably across different
products. For 11 products, the model is the _core_ functionality, as there
would be no product without the model (e.g., the OCR model in the OCR scanner
app P1). There are 6 products that may still provide value without the model,
but the model provides _significant_ functionality (e.g., the OCR model in the
video subtitle editor P15 that could potentially operate on manual inputs). In
13 products, the model provides _optional_ functionality, serving as a nice-
to-have add-on (e.g., facial expression recognition in the video conferencing
app P28).

Whether a _third-party_ or _self-trained_ model is used (see _Finding 6_) is
not necessarily associated with the model’s importance. We found products
investing substantial effort in self-trained models for _optional_
functionality (e.g., P6, P26, P29), and products that rely on a third-party
model for _core_ functionality (e.g., P1, P12, P13). However, we do observe
that models core to a product are usually developed first, as we will discuss
in _RQ #3_.

####  5.1.12. Finding 8: Automation using model predictions is uncommon
(5/30), with most products keeping humans in the loop

A central question in human-AI design is how to use or present model
predictions. While automation can improve user experience, it can also make it
hard to fix model mistakes leading to safety concerns. In contrast, _prompts_
and _augmentation_ are often viewed as safer human-AI design choices when
models make mistakes as humans control the decision (DeO4, ; ftJI, ). We find
only five products that use model predictions to _automate_ actions (e.g.,
keyword spotting app, P27, executes gameplay instructions automatically based
on a speech model that processes voice commands). Two products _prompt_ users
to confirm an action (e.g., deepfake software, P17, displays image previews to
users for verification after each stage of image processing). Most products
(23) in our sample use predictions to _augment_ the user interface (ftJI, ) to
display the prediction to a human who may or may not take actions (e.g.,
trading app, P21, graphically presents model predictions related to
investments to the end-users, letting them make decisions on possible
investigations).

####  5.1.13. Finding 9: Most products use raw model predictions without any
post-processing (21/30)

There are many ways for processing model results in products, some of which
are viewed as architectural tactics, such as _two-phased predictions_ as an
_availability tactic,_ ensuring the resilient serving of models (zLpk, ; slEk,
). However, most products in our sample trust models and their predictions,
and do not process or check predictions any further. Only two products
incorporate additional architectural tactics around the model predictions:
Plant identifier app _,_ P2, uses a two-phase prediction system, starting with
a local model, and if its confidence score is below a certain threshold, an
online model is called for consultation (architectural pattern known as
_cascading/two-phase prediction_ (zLpk, )). Subtitle editor software, P15,
performs rigorous checks on texts predicted via optical character recognition
(OCR) and language translation before presenting them to the end-user,
complemented with an editor for manual correction of errors. In addition,
three products incorporate a confidence score threshold to filter low
confidence predictions (P9, P27, P29) and another three offer a retraining
option for the model if performance proves unsatisfactory (P5, P12, P28). P11
is an exception to all, with the model itself serving as a post-processing
mechanism to fix forecasting errors retrieved from an API. All others directly
present the predictions to users as-is.

Directly relying on model predictions can expose users directly to mistakes.
This can be particularly problematic for third-party models that evolve over
time (XOW6, ). However, none of these products are safety-critical, where
inaccurate predictions could be harmful.

####  5.1.14. Finding 10: Products that automate actions based on model
predictions incorporate architectural tactics

Our findings suggest an association between how model predictions are used in
a product _(Finding 8)_ and the architectural decisions made around the model
(_Finding 9_). For instance, three of the five products that _automate_
actions offer users an option to retrain the model, demonstrating a
contingency plan for potential prediction inaccuracies. The fourth product
includes a confidence score threshold on model predictions, thus ensuring a
check on prediction reliability. These instances indicate that when models are
given the power to automate decisions, developers implement architectural
tactics to hand more control over to users (either to retrain model or adjust
prediction threshold).

####  5.1.15. Finding 11: It is common to use multiple models in a product
(18/30), though mostly the models work independently of each other (11/18)

Interactions among multiple models are a frequently raised challenge in
industry, where a minor change in one model can trigger cascading changes
across the product (Ijae, ; 94yr, ; t8sQ, ; ftJI, ). While 18 products use
multiple models, they work independently in 11, among which, 7 products use
models for _separate functions_ (e.g., P15 uses one model for OCR and another
for speech-to-text), and 7 products use models as _alternatives_ for the same
function (e.g., P26 provides a choice between two clustering models), with 3
products using models for both. There are 5 products that _sequentially
compose_ models (ftJI, ), with the output of one model used as an input to
another model (e.g., P12 passes text recognized by an OCR model into an entity
recognition model). Two products use models for _collective decision-making_
(e.g., P9 combines multiple classifiers to generate personalized news feeds).

####  5.1.16. Finding 12: Pipeline automation is not common in open-source ML
products

Among the 17 products that use self-trained models (_Finding 7_), training is
often not automated. We did not find model training pipeline for four products
(P2, P18, P22, P30) (we cannot tell if training happens offline or in a
private repository). Four products (P6, P10, P12, P16) have non-automated
training pipelines requiring manual execution of sequential steps, saving
predictions in CSV files, and loading models from asset folders through
product code. Four products (P9, P15, P27, P29) have GUI-integrated training
pipelines that can be separately activated via GUI actions. Only four products
(P14, P21, P23, P26) feature fully automated training pipelines that
consistently fetch the latest data for model retraining and use the most
recent model or allow end-users to initiate retraining at run time. We also
found two products (P11, P24) that include a partial pipeline only for data
retrieval.

Industry discussions often mention how dependencies between pipelines and
other system parts can complicate ML product architecture (ftJI, ). However,
ML products in our sample have simpler structures, lacking the intricate
pipelines seen in industrial settings.

####  5.1.17. Finding 13: We do not find much effort on data or model
documentation

Both industry and academia view documentation as an important artifact in ML
product development for many aspects, such as setting clear expectations for
collaboration, improving transparency and accountability for responsible AI,
and facilitating information sharing across different components (tPzV, ;
Fhfs, ; L6sa, ; agC2, ). Documentation is vital to architecture, since
architectural practices such as divide and conquer and information hiding at
interfaces between components, require documentation support. However, in our
sample, the 17 products using self-trained models (_Finding 7_) provided
minimal and mostly scattered documentation about models and data, if any.
_Model documentation:_ We only found one product, P29, which provides high-
quality model documentation in the form of a model card (9y7Z, ) for a model
that differentiates between fish and trash in the ocean in an AI-educational
game for children. Other products generally limited their model documentation
to brief instructions for using the model API or insights into inner model
architecture from a data scientist’s perspective. _Data documentation:_ Data
documentation was mostly limited to presenting a data schema, occasionally
mentioning the volume of training data, or simply providing a link to their
data sources. We found no integration of data documentation frameworks such as
datasheets (EggD, ). Thus, much like commercial platforms, where ML
documentation is rare and perceived as challenging (tPzV, ; Krxa, ), open-
source platforms are no different (Krxa, ).

####  5.1.18. Finding 14: Most products do not use/rely on big data
infrastructure (23/30)

Scalability is an important architectural challenge for ML products as they
often need to handle extensive training and telemetry data, and operate large
distributed jobs for model training, leading to reliance on big data
infrastructure. However, we did not find use of local or self-hosted big data
infrastructures (such as Hadoop and Spark) in any of the products. Seven
products contain code related to cloud services such as Amazon S3 (storage),
EC2 Mongo (storage), Amazon EC2 (compute), Amazon CloudWatch (monitoring), and
Elastic Cloud (search). Given limited funding, unlike revenue-generating
commercial products, most open-source products may not be able to afford cloud
services or big on-site installations, or find it incompatible with the spirit
of open source.

In summary, for RQ2 our findings revealed that ML products are architected
with single and multiple models of varying types, both self-trained and third-
party, which are used for tasks of varying importance. Notably, we found
products to employ different architectural decisions based on how they use
model predictions. We also observed a lack of post-processing of model
predictions, training pipeline automation, big data infrastructure, and
insufficient emphasis on documentation in most of these products.

### RQ #3: What model-product development trajectory do open-source ML
products follow?

Industry teams have been observed to develop model and product in different
orders, known as trajectories (gHwA, ; rrC2w, ), and face distinct challenges.
We investigate trajectories in open-source ML products.

####  5.1.19. Method

We analyzed the commit history to identify the timeline of model and product
development, and classify a product as following _model-first trajectory_ if
the model is developed first followed by the creation of a product around it
and _product-first trajectory_ if models are added to existing products
(rrC2w, ). Products that have a single large initial commit containing both
model and product code are classified as “ _unsure_.”

####  5.1.20. Finding 15: The _product-first trajectory_ (16/30) is more
common than the _model-first trajectory_ (7/30)

While we are not aware of representative numbers, prior work suggests that
both trajectories are prevalent in industry (rrC2w, ). In our open-source
sample, we observe a greater prevalence of the product-first trajectory, which
may be attributed to most contributors being software engineers (_Finding 2_)
and many products adding machine learning for optional functionality (_Finding
7_). Overall though, both trajectories can be found in open source, which is
encouraging for studies exploring phenomena or solutions to their respective
challenges.

####  5.1.21. Finding 16: When the model is the _core_ functionality, it is
always developed first

We observed that model importance (_Finding 7_) significantly impacts a
product’s development trajectory. The products with models as _core_ are
always developed _model-first_ , while the models as _optional or significant_
tend to follow the product-first trajectory. This pattern is plausible, as
developers prototype the risky project parts first. For example, software for
creating deepfakes, P17, first developed the model in 2017, and added the GUI
components a year later to make the model accessible as a product. In
contrast, video conferencing app, P28, follows a product-first trajectory, for
which the software development started in 2013, with the face detection model
(_optional_ functionality) being added several years later in 2020.

### RQ #4-6: Testing, Operation, and Responsible AI

Due to the page limit, we only report brief findings (methods and details are
available in the appendix (LwCB, )) for the remaining three research questions
(cf. Section 5.1).

####  5.1.22. Finding 17: Testing regular software functionality is common
(23/30). Model testing is notably scarce (8/30). Data validation is rare
(2/30)

Contrary to industry practices, model evaluation appears to be remarkably less
prevalent in our sample of open-source ML products. Even among the eight
products that included model evaluation scripts, three of them (P3, P9, P29)
approached ML model testing like unit testing, cross-checking predictions
against expected values. Data validation testing was almost non-existent, with
just two products implementing minimal checks.

####  5.1.23. Finding 18: Only a few products (8/17) have mechanisms for
evolving models

Of the 17 products with self-trained models, five products (P9, P16, P23, P27,
P30) offer users the option to retrain products at run time and three products
(P14, P21, P26) continually retrain their models by fetching up-to-date data
from their data sources. This aligns with the finding from Nahar et al.
(rrC2w, ) that many product teams have a static view of models and model teams
struggle to foster a more dynamic mindset.

####  5.1.24. Finding 19: Telemetry monitoring is almost non-existent (1/30)

Despite heavy emphasis on observability in industry and academic literature,
we found 29 out of the 30 products to not have any telemetry tracking or
monitoring, which is critical for detecting failure regardless of whether
_third-party_ or _self-trained_ models are used. The exception was P21, which
incorporated telemetry due to its inherent need in dealing with financial
forecasting. We also found a product (P9) that, despite already having a
monitoring infrastructure (Amazon CloudWatch), does not use it for model
telemetry monitoring. Some developers might argue that collecting telemetry
from users may not align with open-source principles.

####  5.1.25. Finding 20: We find little evidence of MLOps tools

We did not find use of any popular MLOps tools for tasks such as automating
deployment, testing, monitoring, and data cataloging, in any of the products.
Two products (P17, P21) only implemented basic checks for GPU availability to
execute models. Given that many of these products do not incorporate
retraining mechanisms (_Finding 18_), they have no immediate need for many
automation tasks that MLOps tools typically support. This finding aligns with
the finding of Caleato et al. [16] that the adoption of MLOps workflows in
GitHub ML projects is limited, but contrasts with findings from industrial
surveys where MLOps is deemed crucial and associated with several challenges
[56, 88].

####  5.1.26. Finding 21: Despite significant attention in academia,
responsible AI practices (e.g., fairness, safety, security) are not apparent
in open-source ML products

We do not find adoption of any practices in our sample to mitigate responsible
AI concerns. Only one product, P17, discuss ethical usage in their readme
file,333https://github.com/deepfakes/faceswap#faceswap-has-ethical-uses which
is limited to a disclaimer for end users. Others included general privacy
policies and disclaimers in readme files or web pages that are not related to
ML. One product, P29, advocates model cards, a model documentation framework
for transparent and responsible usage of ML models. This limited focus on
responsible AI aligns with the literature (8V2w, ; Krxa, ; rrC2w, ), which
suggests that companies (especially outside of big tech) often do not have
concerns about responsible AI and rarely adopt responsible AI practices or
tools.

##  6\. Discussion and Conclusion

We compiled a dataset of 262 open-source ML products to facilitate research
and education that can benefit from access to the development history and
artifacts of ML products, and report 21 findings from the six research
questions that suggest promising insights, and avenues for future exploration.

####  6.0.1. Educational Resource for Academics and Practitioners

The compiled dataset can serve as a valuable educational resource for both
academics and practitioners in ML product development. The dataset offers
diverse study materials suitable for a range of learning needs as it contains
both large ML products representative of industry applications, as well as
small ones. Even though practices such as model documentation are rare
_(Finding 13)_ , the dataset still contains positive examples for study as
well as many opportunities to try interventions: Especially, the larger and
more professionally-built ML products (cf. Section 4.2) within the dataset can
provide a close-up view of various development practices and architectural
tactics, including safety mechanisms, human-AI interaction, and deployment
strategies. The smaller personal-interest-based ML products (cf. Section 4.2)
can serve as guides or tutorials, illustrating the fundamentals and challenges
of building ML products on a manageable scale, particularly beneficial for
instructors and beginners looking for an accessible entry point.

####  6.0.2. Research Opportunities

Our investigations into various activities associated with building ML
products have disclosed numerous promising avenues for future research. Among
others, we believe further exploration regarding team conflict is worthwhile,
particularly in light of our findings revealing unclear team boundaries and
responsibilities in open-source ML products _(Findings 3-4)_ , an issue also
prevalent and frustrating for developers in industry (rrC2w, ): With access to
concrete data in open source, it becomes feasible to delve deeper into this
issue, potentially employing firehouse studies (mbZE, ; lcTM, ) to interview
developers when certain events occur.

In addition to the analysis of existing practices, our analysis provides
researchers with opportunities to evaluate new tools and interventions in
realistic settings with the open-source data. For instance, building on
insights from _Finding 10_ that developers integrate architectural tactics for
safeguarding against model predictions when used for automation, we can
evaluate the effectiveness of different architectural tactics as safety and
usability interventions. Similarly, our discovery of multiple interacting
models (_Finding 11_) offers opportunities to design and evaluate whether
data-flow tracking and architecturally limiting interactions could curb
feedback loops and modularity challenges (Ijae, ; t8sQ, ), among many other
potential research avenues.

## References

  * (1) Amershi, S. et al. 2019. Software Engineering for Machine Learning: A Case Study. _In Proc. 41st Int’l Conf. on Software Engineering: Software Engineering in Practice_ (2019), 291–300.
  * (2) Amershi, S. et al. 2015. ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. _In Proc. of 33rd Conf. on Human Factors in Computing Systems_ (2015), 337–346.
  * (3) Apel, S., Kästner, C. and Kang, E. 2022. Feature Interactions on Steroids: On the Composition of ML Models. _IEEE Software_. 39, 3 (2022), 120–124.
  * (4) Arnold, M. et al. 2019. FactSheets: Increasing trust in AI services through supplier’s declarations of conformity. _IBM journal of research and development_. 63, 4/5 (2019), 6:1–6:13.
  * (5) Arpteg, A., Brinne, B., Crnkovic-Friis, L. and Bosch, J. 2018. Software Engineering Challenges of Deep Learning. _In Proc. 44th Euromicro Conf. on Software Engineering and Advanced Applications_ (2018), 50–59.
  * (6) Avelino, G., Passos, L., Hora, A. and Valente, M.T. 2016. A novel approach for estimating Truck Factors. _In Proc. 24th Int’l Conf. on Program Comprehension_ (2016), 1–10.
  * (7) Banna, V. et al. 2021. An Experience Report on Machine Learning Reproducibility: Guidance for Practitioners and TensorFlow Model Garden Contributors. _arXiv [cs.SE]_.
  * (8) Barnett, M., Bird, C., Brunet, J. and Lahiri, S.K. 2015. Helping Developers Help Themselves: Automatic Decomposition of Code Review Changesets. _In Proc. 37th Int’l Conf. on Software Engineering_ (2015), 134–144.
  * (9) Bass, L., Clements, P. and Kazman, R. 2003. _Software Architecture in Practice_. Addison-Wesley Professional.
  * (10) Bellamy, R.K.E. et al. 2018. AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias. _arXiv [cs.AI]_.
  * (11) Bernardi, L., Mavridis, T. and Estevez, P. 2019. 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com. _In Proc. 25th ACM SIGKDD Int’l Conf. on Knowledge Discovery & Data Mining_ (2019), 1743–1751.
  * (12) Bhat, A. et al. 2023. Aspirations and Practice of ML Model Documentation: Moving the Needle with Nudging and Traceability. _In Proc. of CHI Conf. on Human Factors in Computing Systems_ (2023), 1–17.
  * (13) Boyd, K.L. 2021. Datasheets for Datasets help ML Engineers Notice and Understand Ethical Issues in Training Data. _Proc. ACM Hum.-Comput. Interact._ 5, CSCW2 (2021), 1–27.
  * (14) Brown, A. and Wilson, G. 2011. _The Architecture of Open Source Applications: Elegance, Evolution, and a Few Fearless Hacks_. Lulu.com.
  * (15) Chang, J. and Custis, C. 2022. Understanding Implementation Challenges in Machine Learning Documentation. _In Proc. Equity and Access in Algorithms, Mechanisms, and Optimization_ (2022), 1–8.
  * (16) Cummaudo, A., Barnett, S., Vasa, R., Grundy, J. and Abdelrazek, M. 2020. Beware the evolving “intelligent” web service! an integration architecture tactic to guard AI-first components. _In Proc. 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ (2020), 269–280.
  * (17) Dilhara, M., Ketkar, A. and Dig, D. 2021. Understanding Software-2.0: A Study of Machine Learning Library Usage and Evolution. _ACM Transactions on Software Engineering and Methodology_. 30, 4 (2021), 1–42.
  * (18) Faceswap: Deepfakes Software For All: https://faceswap.dev/. Accessed: 2023-03-24.
  * (19) Ferreira, M., Mombach, T., Valente, M.T. and Ferreira, K. 2019. Algorithms for estimating truck factors: a comparative study. _Software quality journal_. 27, 4 (2019), 1583–1617.
  * (20) Flyvbjerg, B. 2006. Five Misunderstandings About Case-Study Research. _Qualitative inquiry: QI_. 12, 2 (2006), 219–245.
  * (21) Follow, R. Bridging the Gap Between Data Science & Engineer: Building High-Performance Teams. _Slideshare_.
  * (22) Fraser, G. and Arcuri, A. 2014. A Large-Scale Evaluation of Automated Unit Test Generation Using EvoSuite. _ACM Transactions on Software Engineering and Methodology_. 24, 2 (2014), 1–42.
  * (23) Gall, H., Hajek, K. and Jazayeri, M. 1998. Detection of logical coupling based on product release history. _In Proc. Int’l Conf. on Software Maintenance_ (1998), 190–198.
  * (24) Gamma, E. and Beck, K. 2004. _Contributing to Eclipse: Principles, Patterns, and Plug-ins_. Addison-Wesley Professional.
  * (25) Godfrey, M.W. et al. 2009. Future of Mining Software Archives: A Roundtable. _IEEE Software_. 26, 1 (2009), 67–70.
  * (26) Goeminne, M. and Mens, T. 2011. Evidence for the pareto principle in open source software activity. _In Joint Proc. of the 1st Int’l Workshop on Model Driven Software Maintenance and 5th Int’l Workshop on Software Quality and Maintainability_ (2011), 74–82.
  * (27) Gonzalez, D., Zimmermann, T. and Nagappan, N. 2020. The State of the ML-universe: 10 Years of Artificial Intelligence & Machine Learning Software Development on GitHub. _In Proc. 17th Int’l Conf. on Mining Software Repositories_ (2020), 431–442.
  * (28) Goues, C.L., Pradel, M. and Roychoudhury, A. 2019. Automated program repair. _Communications of the ACM_. 62, 12 (2019), 56–65.
  * (29) Gousios, G., Pinzger, M. and van Deursen, A. 2014. An exploratory study of the pull-based software development model. _In Proc. 36th Int’l Conf. on Software Engineering_ (2014), 345–355.
  * (30) Gousios, G., Zaidman, A., Storey, M.-A. and van Deursen, A. 2015. Work Practices and Challenges in Pull-Based Development: The Integrator’s Perspective. _In Proc. 37th Int’l Conf. on Software Engineering_ (2015), 358–368.
  * (31) Haakman, M., Cruz, L., Huijgens, H. and van Deursen, A. 2020. AI Lifecycle Models Need To Be Revised. An Exploratory Study in Fintech. _Empirical Software Engineering_. 26, (2020), 1–29.
  * (32) Harris, J. 2020. Beyond the jupyter notebook: how to build data science products. _Towards Data Science_.
  * (33) Hassan, A.E. 2008. The road ahead for Mining Software Repositories. _Frontiers of Software Maintenance_ (2008), 48–57.
  * (34) Hassan, A.E. and Xie, T. 2010. Software intelligence: the future of mining software engineering data. _In Proc. FSE/SDP workshop on Future of software engineering research_ (2010), 161–166.
  * (35) Hazelwood, K. et al. 2018. Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective. _In Proc. Int’l Symposium on High Performance Computer Architecture_ (2018), 620–629.
  * (36) Head, A., Hohman, F., Barik, T., Drucker, S.M. and DeLine, R. 2019. Managing messes in computational notebooks. _In Proc. of CHI Conf. on Human Factors in Computing Systems_ (2019).
  * (37) Hermann, J. and Del Balso, M. 2017. Meet Michelangelo: Uber’s machine learning platform. _Uber_.
  * (38) Hulten, G. 2018. _Building Intelligent Systems: A Guide to Machine Learning Engineering_. Apress.
  * (39) Hynes, N., Sculley, D. and Terry, M. 2017. The data linter: Lightweight, automated sanity checking for ml data sets. _NIPS MLSys Workshop_. 1, 5 (2017).
  * (40) Ishikawa, F. and Yoshioka, N. 2019. How do engineers perceive difficulties in engineering of machine-learning systems? - questionnaire survey. _In Proc. Joint 7th Int’ Workshop on Conducting Empirical Studies in Industry and 6th Int’l Workshop on Software Engineering Research and Industrial Practice_ (2019), 2–9.
  * (41) Kalliamvakou, E., Damian, D., Singer, L. and German, D.M. 2014. The Code-Centric Collaboration Perspective: Evidence from Github. _Technical Report DCS-352-IR, University of Victoria_. (2014), 17.
  * (42) Kanagal, B. and Tata, S. 2018. Recommendations for All: Solving Thousands of Recommendation Problems Daily. _In Proc. 34th Int’l Conf. on Data Engineering_ (2018), 1404–1413.
  * (43) Karahasanovic, A. et al. 2021. User Involvement in the Design of ML-Infused Systems. _In Proc. 1st Int’l Conf. of Greek SIGCHI Chapter_ (2021), 1–5.
  * (44) Kästner, C. 2022. _Machine Learning in Production: From Models to Products_.
  * (45) Kim, M., Zimmermann, T., DeLine, R. and Begel, A. 2018. Data Scientists in Software Teams: State of the Art and Challenges. _IEEE Transactions on Software Engineering_. 44, 11 (2018), 1024–1038.
  * (46) Königstorfer, F. and Thalmann, S. 2022. AI Documentation: A path to accountability. _Journal of Responsible Technology_. 11, (2022), 100043.
  * (47) Lai, T.D., Simmons, A., Barnett, S., Schneider, J.-G. and Vasa, R. 2022. Comparative analysis of real bugs in open-source Machine Learning projects – A Registered Report. _arXiv [cs.SE]_.
  * (48) Lakshmanan, V., Robinson, S. and Munn, M. 2020. _Machine Learning Design Patterns_. O’Reilly Media, Inc.
  * (49) Lewis, G.A., Bellomo, S. and Ozkaya, I. 2021. Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems. _In Proc. 1st Workshop on AI Engineering-Software Engineering for AI_ (2021), 133–140.
  * (50) Lewis, G.A., Ozkaya, I. and Xu, X. 2021. Software Architecture Challenges for ML Systems. _In Proc. Int’l Conf. on Software Maintenance and Evolution_ (2021), 634–638.
  * (51) Lin, J. and Kolcz, A. 2012. Large-scale machine learning at twitter. _In Proc. ACM SIGMOD Int’l Conf. on Management of Data_ (2012), 793–804.
  * (52) Liu, H., Eksmo, S., Risberg, J. and Hebig, R. 2020. Emerging and Changing Tasks in the Development Process for Machine Learning Systems. _In Proc. Int’l Conf. on Software and System Processes_ (2020), 125–134.
  * (53) Lwakatare, L.E. et al. 2019. A taxonomy of software engineering challenges for machine learning systems: An empirical investigation. _In Proc. Int’l Conf. on Agile Software Development_ (2019), 227–243.
  * (54) Lwakatare, L.E. et al. 2020. Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions. _Information and Software Technology_. 127, (2020), 106368.
  * (55) Mäkinen, S. et al. 2021. Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help? _In Proc. 1st Workshop on AI Engineering - Software Engineering for AI_ (2021), 109–112.
  * (56) Marginean, A. et al. 2019. SapFix: Automated End-to-End Repair at Scale. _In Proc. 41st Int’l Conf. on Software Engineering: Software Engineering in Practice_ (2019), 269–278.
  * (57) Marmorstein, R. 2011. Open source contribution as an effective software engineering class project. _In Proc. 16th Annual Joint Conf. on Innovation and Technology in Computer Science Education_ (2011), 268–272.
  * (58) Martinez-Plumed, F. et al. 2020. CRISP-DM twenty years later: From data mining processes to data science trajectories. _IEEE transactions on knowledge and data engineering_. (2020), 1–1.
  * (59) McGlohon, M. 2021. Demystifying Machine Learning in Production: Reasoning about a Large-Scale ML Platform. _Usenix_.
  * (60) Mitchell, M. et al. 2019. Model Cards for Model Reporting. _In Proc. Conf. on Fairness, Accountability, and Transparency_ (2019), 220–229.
  * (61) Murphy-Hill, E., Zimmermann, T., Bird, C. and Nagappan, N. 2013. The design of bug fixes. _In Proc. 35th Int’l Conf. on Software Engineering_ (2013), 332–341.
  * (62) Nahar, N., Zhang, H., Lewis, G., Zhou, S. and Kästner, C. 2023. A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners. _In Proc. of 2nd Int’l Conf. on AI Engineering: Software Engineering for AI_ (2023).
  * (63) Nahar, N., Zhou, S., Lewis, G. and Kästner, C. 2022. Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process. _In Proc. 44th Int’l Conf. on Software Engineering_ (2022), 413–425.
  * (64) Nascimento, D.M.C., Chavez, C.F.G. and Bittencourt, R.A. 2018. The Adoption of Open Source Projects in Engineering Education: A Real Software Development Experience. _In Proc. Conf. Frontiers in Education (FIE)_ (2018), 1–9.
  * (65) Nehra, M. 2022. Top 10 Programming Languages for Desktop Apps in 2022. _decipherzone.com_. Decipher Zone.
  * (66) Nushi, B., Kamar, E., Horvitz, E. and Kossmann, D. 2017. On human intellect and machine failures: troubleshooting integrative machine learning systems. _In Proc. 31st AAAI Conf. on Artificial Intelligence_ (2017), 1017–1025.
  * (67) OBrien, D. et al. 2022. 23 shades of self-admitted technical debt: an empirical study on machine learning software. _In Proc. 30th ACM Joint European Software Engineering Conf. and Symposium on the Foundations of Software Engineering_ (2022), 734–746.
  * (68) van Oort, B., Cruz, L., Aniche, M. and van Deursen, A. 2021. The Prevalence of Code Smells in Machine Learning projects. _In Proc. 1st Workshop on AI Engineering - Software Engineering for AI_ (2021), 1–8.
  * (69) Paleyes, A., Urma, R.-G. and Lawrence, N.D. 2022. Challenges in Deploying Machine Learning: A Survey of Case Studies. _ACM Comput. Surv._ 55, 6 (2022), 1–29.
  * (70) Passi, S. and Jackson, S.J. 2018. Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects. _In Proc. of the ACM on Human-Computer Interaction_. 2, CSCW (2018), 1–28.
  * (71) Passi, S. and Sengers, P. 2020. Making data science systems work. _Big data & society_. 7, 2 (2020), 205395172093960.
  * (72) Perspective API: Using Machine Learning to Reduce Toxicity Online: 2017. https://www.perspectiveapi.com/.
  * (73) Petrović, G. and Ivanković, M. 2018. State of mutation testing at google. _In Proc. 40th Int’l Conf. on Software Engineering: Software Engineering in Practice_ (2018), 163–171.
  * (74) Pimentel, J.F., Murta, L., Braganholo, V. and Freire, J. 2019. A large-scale study about quality and reproducibility of jupyter notebooks. _In Proc. 16th Int’l Conf. on Mining Software Repositories_ (2019).
  * (75) Piorkowski, D., González, D., Richards, J. and Houde, S. 2020. Towards evaluating and eliciting high-quality documentation for intelligent systems. _arXiv [cs.SE]_.
  * (76) Piorkowski, D. et al. 2021. How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study. _Proc. of the ACM on Human-Computer Interaction_. 5, CSCW1 (2021), 1–25.
  * (77) Polyzotis, N., Zinkevich, M., Roy, S., Breck, E. and Whang, S. 2019. Data validation for machine learning. _In Proc. Machine Learning and Systems_ (2019), 334–347.
  * (78) Psallidas, F. et al. 2022. Data Science Through the Looking Glass: Analysis of Millions of GitHub Notebooks and ML.NET Pipelines. _ACM SIGMOD Rec._ 51, 2 (2022), 30–37.
  * (79) Quaranta, L., Calefato, F. and Lanubile, F. 2022. Eliciting Best Practices for Collaboration with Computational Notebooks. _In Proc. ACM Hum.-Comput. Interact._ 6, CSCW1 (2022), 1–41.
  * (80) Rahman, M.S., Rivera, E., Khomh, F., Guéhéneuc, Y.-G. and Lehnert, B. 2019. Machine Learning Software Engineering in Practice: An Industrial Case Study. _arXiv [cs.SE]_.
  * (81) Rakova, B., Yang, J., Cramer, H. and Chowdhury, R. 2020. Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices. _Proc. of the ACM on Human-Computer Interaction_. 5, CSCW1 (2020), 1–24.
  * (82) Ratner, A., Bach, S.H., Ehrenberg, H., Fries, J., Wu, S. and Ré, C. 2017. Snorkel: Rapid Training Data Creation with Weak Supervision. _Proceedings of the VLDB Endowment International Conference on Very Large Data Bases_. 11, 3 (2017), 269–282.
  * (83) Ré, C., Niu, F., Gudipati, P. and Srisuwananukorn, C. 2019. Overton: A data system for monitoring and improving machine-learned products. _arXiv [cs.LG]_.
  * (84) Rzig, D.E., Hassan, F., Bansal, C. and Nagappan, N. 2022. Characterizing the usage of CI tools in ML projects. _In Proc. Int’l Symposium on Empirical Software Engineering and Measurement_ (2022).
  * (85) Saleiro, P. et al. 2018. Aequitas: A Bias and Fairness Audit Toolkit. _arXiv [cs.LG]_.
  * (86) Sánchez, A.B. et al. 2022. Mutation testing in the wild: findings from GitHub. _Empirical Software Engineering_. 27, 6 (2022), 132.
  * (87) Schelter, S. et al. 2018. Automating large-scale data quality verification. _Proceedings of the VLDB Endowment International Conference on Very Large Data Bases_. 11, 12 (2018), 1781–1794.
  * (88) Sculley, D. et al. 2015. Hidden Technical Debt in Machine Learning Systems. _Advances in Neural Information Processing Systems_. 28, (2015), 2503–2511.
  * (89) Nahar, N. 2022. Supplementary Documents: A Dataset and Analysis of Open-Source Machine Learning Products. OSF. https://osf.io/gqyex/
  * (90) Serban, A., van der Blom, K., Hoos, H. and Visser, J. 2020. Adoption and Effects of Software Engineering Best Practices in Machine Learning. _In Proc. 14th Int’l Symposium on Empirical Software Engineering and Measurement_ (2020), 1–12.
  * (91) Serban, A. and Visser, J. 2022. Adapting Software Architectures to Machine Learning Challenges. _In Proc. Int’l Conf. on Software Analysis, Evolution and Reengineering_ (2022), 152–163.
  * (92) Siebert, J. et al. 2020. Towards Guidelines for Assessing Qualities of Machine Learning Systems. _In Proc. 13th Int’l Conf. Quality of Information and Communications Technology_ (2020), 17–31.
  * (93) Siegmund, J., Siegmund, N. and Apel, S. 2015. Views on Internal and External Validity in Empirical Software Engineering. _In Proc. 37th Int’l Conf. on Software Engineering_ (2015), 9–19.
  * (94) Smith, D. 2017. Exploring development patterns in data science. _Theorylane_.
  * (95) d. S. Nascimento, E. et al. 2019. Understanding Development Process of Machine Learning Systems: Challenges and Solutions. _In Proc. Int’l Symposium on Empirical Software Engineering and Measurement_ (2019), 1–6.
  * (96) Spencer, D. 2009. _Card Sorting: Designing Usable Categories_. Rosenfeld Media.
  * (97) Subramonyam, H., Im, J., Seifert, C. and Adar, E. 2022. Human-AI Guidelines in Practice: Leaky Abstractions as an Enabler in Collaborative Software Teams. _arXiv [cs.SE]_.
  * (98) Tang, Y. et al. 2021. An Empirical Study of Refactorings and Technical Debt in Machine Learning Systems. _In Proc. 43rd Int’l Conf. on Software Engineering_ (2021), 238–250.
  * (99) Vasilescu, B. et al. 2015. Gender and Tenure Diversity in GitHub Teams. _In Proc. 33rd Conf. on Human Factors in Computing Systems_ (2015), 3789–3798.
  * (100) Vasilescu, B., Yu, Y., Wang, H., Devanbu, P. and Filkov, V. 2015. Quality and productivity outcomes relating to continuous integration in GitHub. _In Proc. 10th Joint Meeting on Foundations of Software Engineering_ (2015), 805–816.
  * (101) Villamizar, H., Kalinowski, M. and Lopes, H. 2022. Towards Perspective-Based Specification of Machine Learning-Enabled Systems. _In Proc. 48th Euromicro Conf. on Software Engineering and Advanced Applications_ (2022), 112–115.
  * (102) Vogelsang, A. and Borg, M. 2019. Requirements Engineering for Machine Learning: Perspectives from Data Scientists. _In Proc. 27th Int’l Requirements Engineering Conference Workshops_ (2019), 245–251.
  * (103) Wan, C. et al. 2022. Automated Testing of Software that Uses Machine Learning APIs. _In Proc. 44th Int’l Conf. on Software Engineering_ (2022), 212–224.
  * (104) Wang, J., Li, L. and Zeller, A. 2021. Restoring Execution Environments of Jupyter Notebooks. _In Proc. 43rd Int’l Conf. on Software Engineering_ (2021), 1622–1633.
  * (105) Wang, Z. and Hahn, J. The effects of programming style on open source collaboration. _scholar.archive.org_.
  * (106) Wan, Z., Xia, X., Lo, D. and Murphy, G.C. 2019. How does Machine Learning Change Software Development Practices? _IEEE Transactions on Software Engineering_. (2019), 1–1.
  * (107) Web Languages-What language are most commonly used for web development: 2020. https://www.dotnetlanguages.net/web-languages-what-language-are-most-commonly-used-for-web-development/. Accessed: 2023-04-05.
  * (108) Wexler, J. et al. 2020. The What-If Tool: Interactive Probing of Machine Learning Models. _IEEE transactions on visualization and computer graphics_. 26, 1 (2020), 56–65.
  * (109) Widder, D.G., Nafus, D., Dabbish, L. and Herbsleb, J. 2022. Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes. _In Proc. Conf. on Fairness, Accountability, and Transparency_ (2022), 2035–2046.
  * (110) Widyasari, R. et al. 2023. NICHE: A Curated Dataset of Engineered Machine Learning Projects in Python. _arXiv [cs.SE]_.
  * (111) Yamashita, K. et al. 2015. Revisiting the applicability of the pareto principle to core development teams in open source software projects. _In Proc. 14th Int’l Workshop on Principles of Software Evolution_ (2015), 46–55.
  * (112) Zhang, B. et al. 2022. Architecture Decisions in AI-based Systems Development: An Empirical Study. _In Proc. Int’l Conf. on Software Analysis, Evolution and Reengineering_ (2022), 616–626.
  * (113) Zhang, X., Yang, Y., Feng, Y. and Chen, Z. 2019. Software Engineering Practice in the Development of Deep Learning Applications. _arXiv [cs.SE]_.
  * (114) Zimmermann, T., Diehl, S. and Zeller, A. 2003. How history justifies system architecture (or not). _In Proc. 6th Int’l Workshop on Principles of Software Evolution_ (2003), 73–83.
  * (115) Zinkevich, M. 2017. Rules of machine learning: Best practices for ML engineering. _Minegrado_. minegrado.ovh.