# Towards a User Privacy-Aware Mobile Gaming App Installation Prediction Model

Ido Zehori
Bigabid Inc.
Tel-Aviv, Israel
<EMAIL_ADDRESS>
& Nevo Itzhak
Software and Information Systems Engineering
Ben-Gurion University of the Negev
Beer-Sheva, 8410501, Negev, Israel
<EMAIL_ADDRESS>
& Yuval Shahar
Software and Information Systems Engineering
Ben-Gurion University of the Negev
Beer-Sheva, 8410501, Negev, Israel
<EMAIL_ADDRESS>
& Mia Dor Schiller
Bigabid Inc.
Tel-Aviv, Israel
<EMAIL_ADDRESS>
Ido Zehori is the corresponding author

###### Abstract

Over the past decade, programmatic advertising has received a great deal of
attention in the online advertising industry. A real-time bidding (RTB) system
is rapidly becoming the most popular method to buy and sell online advertising
impressions. Within the RTB system, demand-side platforms (DSP) aim to spend
advertisers’ campaign budgets efficiently while maximizing profit, seeking
impressions that result in high user responses, such as clicks or installs.

In the current study, we investigate the process of predicting a mobile gaming
app installation from the point of view of a particular DSP, while paying
attention to user privacy, and exploring the trade-off between privacy
preservation and model performance. There are multiple levels of potential
threats to user privacy, depending on the privacy leaks associated with the
data-sharing process, such as data transformation or de-anonymization.

To address these concerns, privacy-preserving techniques were proposed, such
as cryptographic approaches, for training privacy-aware machine-learning
models. However, the ability to train a mobile gaming app installation
prediction model without using user-level data, can prevent these threats and
protect the users’ privacy, even though the model’s ability to predict may be
impaired. Additionally, current laws might force companies to declare that
they are collecting data, and might even give the user the option to opt out
of such data collection, which might threaten companies’ business models in
digital advertising, which are dependent on the collection and use of user-
level data.

In this study, we show that learning a user privacy-aware mobile gaming app
installation prediction model leads to an AUC-ROC of 81% and an AUPRC of 49%
while using a non-privacy-aware model leads to an AUC-ROC of 89% and AUPRC of
66%.

We conclude that privacy-aware models might still preserve significant
capabilities, enabling companies to make better decisions, dependent on the
privacy-efficacy trade-off utility function of each case.

_Keywords_ demand-side platform $\cdot$ mobile gaming app $\cdot$ privacy-
aware machine learning

##  1 Introduction

Programmatic advertising has received great attention in the online
advertising industry in the last decade. Real-time bidding (RTB) is quickly
becoming the leading method (Spencer et al., 2011) to enable buying and
selling online advertising impressions (i.e., single-user view of an
advertisement) through real-time auctions. The RTB display advertising (Fig.
1) is composed of three main components: (1) the demand-side platform (DSP)
that enables advertisers to automate the purchasing of the advertising
impressions; (2) the supply-side platform (SSP) that enables the publishers
(e.g., application owners) to sell the advertising impression, and (3)
advertising exchange system that connect advertisers and publishers.

As illustrated in Figure 1, given an advertising slot offered by the publisher
to sell, the SSP provides impression information to an advertising exchange
system, which creates an auction and forwards that information to the DSPs
(e.g., Google Ads, Facebook Ads Manager, AppLovin, IronSource, Liftoff, or
Moloco). Once the auction is completed, the winner is chosen, and an
advertisement is served on the publisher’s website or application. For
example, suppose there are three auction participants (i.e., DSPs), $d_{1}$,
$d_{2}$, and $d_{3}$, that compete on a specific advertising impression, and
their offered prices, according to their secret bidding strategies, were $1.3,
$1.5, and $0.7, respectively. Then, the winning price is the highest of their
offers, which was $1.5, offered by DSP $d_{2}$. In mobile performance
advertising, a DSP’s objectives extend beyond getting the application
installation, but also the prediction of user engagement probabilities,
whether through in-app purchases or watching advertisements. It is worth
noting that while some companies may own different parts of the chain in the
real-time bidding procedure, transparency, and fairness are among the
challenges in today’s online advertising industry. In light of the potential
conflicts of interest created by this ownership structure, some players may
have an unfair advantage.

Figure 1: A high-level overview of the real-time bidding procedure. A
publisher offers an ad slot, the SSP provides information about the user and
the publisher to the ad exchange system, and the DSPs participate in an
auction.

From the DSP’s perspective, the aim is to spend the advertisers’ campaign
budget effectively and to achieve high profit (i.e., increasing the profit
margin). That means a DSP seeks advertising impressions that result in high
user responses, such as clicks or installs, but within budget constraints. To
achieve that, a DSP typically considers three aspects given an auction:
utility, cost, and deciding on a bidding strategy. The utility represents the
value to the DSP of having the user click on the ad, while utility estimation
represents the probability of a user being interested in the ads by clicking,
installing the app, or depositing money into the app. The cost refers to the
estimated winning price of that auction. Estimating the auction winning price
is critical in designing bidding strategies; however, it is challenging. In
practice, a DSP can not observe the other participants’ bids during the
auction or when the auction is completed, but only the highest bid price when
the auction is completed. For RTB with significant volume, in which the number
of bidders participating in the auction is potentially high, the average
winning rate is naturally lower for each participant. The last aspect is a
bidding strategy that incorporates that estimated utility and winning price to
decide on the offered price in an auction (if at all).

In today’s digital world, data have become a valuable asset for companies, and
it is important for organizations to protect these data to maintain the trust
of their customers. Data-driven advertising platforms, such as DSPs, often
collect and store user-level information for research purposes, including
sensitive data such as financial information. As a result, it is crucial for
DSPs to adhere to strict privacy standards and implement robust security
measures to protect these data. Multiple levels of security threats are
associated with data sharing, including data transformations and
deanonymization. Machine learning models that are privacy-aware were developed
using privacy-preserving techniques, such as cryptographic approaches.

The methods still have some limitations, such as flexibility to cope with
ongoing advances in machine learning, or scalability as they require
additional processing and communication costs (Al-Rubaie and Chang, 2019). In
this regard, the ability to learn a machine learning model without involving
user-level data can both prevent privacy and security-related threats, even
though the model’s prediction capability may be impaired. Additionally, more
and more companies, including Big Tech firms, are changing their policies to
put user privacy at the forefront. For Apple’s iOS mobile operating systems,
cross-site identity matching was restricted in April 2021. Thus, iOS
applications must receive explicit consent before exchanging user-level data
with third parties. Government regulations play an important role in these
changes as well, with regulations such as the General Data Protection
Regulation (GDPR) Hoofnagle et al. (2019) and the California Consumer Privacy
Act (CCPA) Pardau (2018) imposing strict requirements on user data collection,
storage, and use. As users’ consent rates are low (Wagner, 2021), such
restriction might threaten companies that are strongly dependent on the
collection and use of user-level data (Morrison, 2021), such as in online
advertising markets.

In this study, we investigate the utility modeling and prediction of mobile
app installation from the point of view of a particular DSP. We compare the
prediction performance of the mobile gaming app installation prediction models
with and without different feature sets, such as the user or publisher-level
data, and their combinations. We consider also an important aspect that is
typically ignored in previous studies, namely, that the bidding and winning
processes are executed when a user loads a page, which requires quick
decision-making from the DSP regarding the bidding. Most previous studies
evaluated their methods on publicly available RTB datasets iPinYou (Liao et
al., 2014), and YOYI (Ren et al., 2017). However, in this paper, we evaluated
our methods on a new dataset called The Bigabid dataset (Sec. 4), which
contains data from the real-world mobile gaming advertising industry.

The main contribution of the paper is the investigation of the trade-off
between users’ privacy and the prediction power of using user-level data for
predicting mobile gaming application installation in digital advertising. Our
study demonstrates that by considering user privacy in the mobile gaming app
installation prediction model, the performance of the model is less accurate,
with an AUC-ROC of 81% and an AUPRC of 49%, however, when user privacy is not
taken into account, the model’s performance is better with an AUC-ROC of 89%
and an AUPRC of 66%. These surprising findings showed that, despite lacking
user-level data, our models still managed to exhibit good performance in
predicting app installations.

The rest of this paper is organized as follows: we start by reviewing the
literature (Sec. 2), considering the RTB aspects - utility, cost, and bidding
strategy, and then, the topic of privacy-aware machine learning. We then
proceed with the Problem Definition (Sec. 3), Dataset (Sec. 4), Methods (Sec.
5), Evaluation and Results (Sec. 6–7), and finally summarize the work, present
conclusions, and discuss future work (Sec. 8).

##  2 Related Work

###  2.1 Utility Estimation

Compared to traditional offline advertising, online advertising offers the
ability to target individual users by user response (e.g., clicks,
conversions) prediction. Advertisers usually measure the effectiveness of
their ads by measuring user responses and makes payment only when these
actions are triggered by their advertisement. Creating effective advertising
requires DSPs to precisely predict users’ responses, since accurate
predictions could greatly improve the subsequent decision-making process, such
as bid optimization. Predicting whether a particular user will click a
specific ad in a specific context is the most fundamental prediction task,
which is referred to as click-through rate (CTR) prediction.

On large-scale advertising setups, logistic regression models have proven
successful in predicting clicks (Richardson et al., 2007; McMahan et al.,
2013) and conversions (Bhamidipati et al., 2017). Although linear models are
easy to implement, they do not learn the complex patterns to capture
interactions between the assumed conditionally independent raw features,
typically leading to low performance.

The use of nonlinear models may improve estimation performance because they
can make use of different feature combinations. One popular approach is to use
deep learning models, which have achieved state-of-the-art performance in many
prediction tasks, including CTR prediction (Zhou et al., 2018). Deep learning
models can effectively capture complex patterns in high-dimensional data,
which are often encountered in online advertising. Previous studies suggested
using historical data to estimate the expected value of a particular user’s
response. For example, Qu et al. (2019) proposed a method that uses the user’s
historical click and conversion data to estimate the expected value of a
user’s response. However, this approach relies on the assumption that the
user’s behavior remains consistent over time, which may not always be the
case.

###  2.2 Cost Prediction

Predicting the winning price in action is an essential task since the auction
participants (i.e., the DSPs) can use the predicted highest bid price to
compute the winning rate of a bid and decide on their final bid price.

The cost predicting aspect is typically defined, rather than predicting the
winning bid price, as a process of forecasting the market price distribution
for auctions of a specific ad, named bid landscape forecasting. The estimated
market price distribution and its cumulative distribution function allow
computing the winning probability given each bid price.

Learning the market price distribution plays a critical role in designing
bidding strategies; however, it is challenging. A DSP can not observe the
other participants’ bids during the auction or when the auction is completed,
but only the highest bid price when the auction is completed. Additionally, it
becomes even more challenging to forecast the market price distribution
according to its dynamic nature; different DSPs probably have different
strategies and work with multiple advertisers, which can behave differently
depending on the given bid. These challenges become more noticeable for RTB
with significant volume, in which the number of bidders participating in the
auction is potentially high, which naturally lowers the winning rate for each
participant.

Most recent studies considered in their methodology the second-price auction
mechanism, in which the winning DSP is charged the price offered by the
second-highest DSP. The second-price auction mechanism brings significant bias
to the bid landscape forecasting and requires using models that consider
censored information. However, since this auction mechanism is irrelevant to
our dataset, in which the highest price wins the bid, the second-price auction
mechanism will be ignored.

A regression model returns the excepted mean value for the winning price given
a bid while ignoring the other auction participants and their strategies.
However, the winning price is dynamic and depends on the other DSPs’
strategies. Therefore, a dynamic model considers the negative examples, and
the other DSPs’ strategies should be learned.

Based on empirical analysis of real-world datasets, researchers proposed
different function forms to model the market price distribution, such as the
log-normal (Wu et al., 2018a; Cui et al., 2011), Gaussian (Wu et al., 2015),
and Gamma (Zhu et al., 2017) distribution. Then, the distribution parameters
are found by maximizing the distribution’s likelihood function. Wu et al. (Wu
et al., 2015) combined a linear regression model and a censored linear
regression model (Tobin, 1958) for winning price estimation. Using linear
regression, assume the dependant variable (i.e., the winning price) is
Gaussian distributed, and as a result, the values range from
$(-\infty,+\infty)$. As a result, the Gaussian distribution is unsuitable for
modeling the winning price (Wu et al., 2018a; Zhu et al., 2017), which should
only be positive values. Gamma is a more suitable distribution for the winning
price, as the values range from $(0,+\infty)$, as proposed by Zhu et al. (Zhu
et al., 2017). Yet, modeling the winning price by assuming a specific
distribution is a strong assumption. Thus, Wu et al. (Wu et al., 2018a)
proposed combining different winning price distributions to the deep learning
models with censoring information to improve the model’s flexibility.

Non-parametric distributions were also considered. Wang et al. (Marmer et al.,
2013) clustered the feature vectors using decision trees and then used
survival analysis with non-parametric distribution to model the winning price
of each cluster.

###  2.3 Bidding Strategy Optimization

In RTB, ads are auctioned off in real-time to the highest bidder, making it
essential for DSPs to have a strong bidding strategy in order to be
successful. One important aspect of bidding strategy optimization in RTB is
the use of algorithms to make real-time decisions about bid prices (Liu et
al., 2022). By using these algorithms, DSPs can quickly and accurately
determine the optimal bid price for each auction, taking into account a range
of factors such as the expected impressions against the bid price (or bid
landscape), total budget, and the campaign’s remaining lifetime.

Bidding strategies can be divided into two main categories: static and
dynamic. A static bidding strategy uses pre-determined formulas, such as
linear (Perlich et al., 2012) or non-linear (Zhang et al., 2014), which are
based on the probability of winning the auction and the prior distribution of
the impression features. The simplicity and ease of deployment of static
bidding strategies are the key reasons they are so widely adopted on DSPs.

However, these static bidding strategies are not effective in the highly
dynamic nature of RTB auctions. Reinforcement learning-based bidding
strategies have shown some promise in this area, such as RLB (Cai et al.,
2017), DRLB (Wu et al., 2018b), and FAB (Liu et al., 2020). A recent empirical
analysis (Liu et al., 2022) has shown that a DRLB-based framework for
optimizing DSPs or advertisers’ real-time bidding strategies is a promising
solution. However, reinforcement learning-based bidding strategies are not yet
ready for commercial deployment.

###  2.4 Privacy-Aware Machine Learning

Privacy-aware machine learning is a subfield of machine learning that focuses
on developing algorithms and techniques that can learn from data while
preserving the privacy of individuals (Chaudhuri et al., 2011). There has been
a growing concern in recent years about the potential risks to privacy posed
by machine learning algorithms (Shokri and Shmatikov, 2015), which are often
able to extract sensitive information from data sets.

One approach to privacy-aware machine learning is the use of privacy-
preserving data perturbation techniques, which add noise to the data to
prevent sensitive information from being revealed (Wood et al., 2018). Another
approach is the use of homomorphic encryption (Mohassel and Zhang, 2017; Aono
et al., 2017), which allows data to be encrypted in such a way that it can be
used for machine learning tasks without revealing its original form.

In addition to these technical approaches, there has also been a focus on
developing frameworks and ethical guidelines for privacy-aware machine
learning (FAT, 2018).

##  3 Problem Definition

Given a user $u$ visits the publisher $s$’s application, the real-time bidding
(RTB) display advertising system offers advertising slot $ad$ to show to $u$
on $s$, the supply-side platform (SSP) provides $u$ and $p$’s information to
$n$ different demand-side platforms (DSPs)
$SP=\\{{d}_{1},{d}_{2},...,{d}_{n}\\}$ throughout an auction $c$ that created
by the advertising exchange system (Fig. 1). From a specific DSP’s $d\in SP$
perspective, given $c$ and $m$ different advertisers
$R=\\{{r}_{1},{r}_{2},...,{r}_{m}\\}$, $d$ aims of selecting $r\in R$ that
will result in high profit by considering $u$’s probability of clicking and
installing (utility; Sec. 2.1), estimated $c$’s winning price (cost; Sec.
2.2), and $r$’s campaign budget to decide on the offered price in $c$ (bidding
strategy optimization; Sec. 2.3). Once the auction $c$ is completed, given
bidding prices of the $n$ DSPs $B=\\{{b}_{1},{b}_{2},...,{b}_{n}\\}$, the
$i$-th DSP $d_{i}\in SP$ that offered $b_{i}\in B$ wins $c$ if $b_{i}=max(B)$
and as a result, $r$’s advertisement is presented to $u$ on $p$.

We frame the problem of predicting whether $u$ will click and then install the
mobile gaming application after watching the advertisement of $r\in R$ located
at $ad$ on $s$ as a classification problem. The goal is to predict the
probability that the user will watch the mobile gaming application
advertisement, click, and install the mobile gaming application, given $d\in
SP$ wins $c$ and present $r\in R$’s advertisement. Given a database
$D^{w\times f}$ that includes records of winning auctions of a single DSP, and
a labels vector $y^{w}$, where $w$ represents the number of auctions in the
dataset and $f$ represents the number of attributes. The label vector $y$
indicates whether the user clicked the mobile gaming app’s advertisement and
then install the mobile application.

##  4 The Bigabid Dataset

The data for this study were collected from real-mobile gaming advertising
campaigns run through the Bigabid DSP platform. Bigabid is a leading DSP that
specializes in the gaming app market. Bigabid operates at a high speed,
bidding on over one million opportunities per second. Bigabid employs advanced
machine learning techniques to train its business models on a vast dataset of
over 4 billion unique devices.

###  4.1 Dataset Creation

The data were collected over a two-day period in January 2021. The data
include 44,824 mobile gaming app impressions from 227 different mobile apps
(i.e., publishers) that resulted in 10,000 clicks, and 7,872 app
installations. The sampling methodology employed for the dataset creation is
designed to obfuscate the actual ratios. The data include numeric variables
only.

Additionally, since a DSP is unable to observe the outcome of auctions it did
not win (e.g., clicks or installs), the data are limited to the auctions that
Bigabid was successful in winning. The data include all the clicks and
installs that occurred during that period. Even though the outcome of all
auctions is not known, the distribution of the auctions that Bigabid won is
similar to the real-world distribution. This is because Bigabid’s bidding
strategy considers the estimated utility and cost of each auction before
deciding whether to bid on it. As a result, Bigabid may win auctions with high
or low estimated utilities. This is a common challenge in RTB from the
perspective of a DSP company. To address this issue, Bigabid runs exploration
campaigns to gather data that covers the entire feature space. This allows
Bigabid to thoroughly investigate the underlying relationships and patterns
within the data and scale the winning auctions data accordingly. This helps
Bigabid better predict auction outcomes and improve the performance of its
auction platform on real-world distribution.

However, as mentioned in the study limitations, this is not the exact
distribution in the real world, as there can be auctions with outliers utility
estimation. To accurately learn and evaluate the real-world distribution, we
would need to have all labels and predict from the perspective of the auction
organizer.

###  4.2 Data Structure

The data can be divided into three main categories: user information,
publisher, and meta-features:

  1. 1.

User features are characteristics of the users that do not change frequently
and reflect their preferences and "static" behavior.

  2. 2.

Publisher features are first-party data about the publisher in which Bigabid
is presenting the ad, such as performance metrics and the type of publisher
that is categorized according to Bigabid’s deep categories system (Raz, 2021).

  3. 3.

Meta features include details about the specific ad bid, including the ad slot
and the type of ad being presented. The meta-features help us to target our
ads more effectively and improve the performance of our campaigns.

##  5 Methods

XGBoost (eXtreme Gradient Boosting) (Chen and Guestrin, 2016) and LightGBM
(Light Gradient Boosting Machine) (Ke et al., 2017) are both gradient-boosting
frameworks that were used in this study. Gradient boosting means they use
decision trees as weak learners and optimize the model using gradient descent.
Gradient boosting is an iterative process, where each iteration builds a new
model that corrects the errors of the previous model. This allows XGBoost and
LightGBM to achieve high levels of accuracy, even with complex and nonlinear
data.

In gradient boosting for classification, the final classifier is a weighted
sum of the individual weak learners, where the weights are determined by
solving the following optimization problem:

$F(x)=\arg\min_{F}\sum_{i=1}^{n}L(y_{i},F(x_{i}))+\sum_{j=1}^{T}\nu_{j}f_{j}(x)$

where $L$ is a loss function, $T$ is the number of weak learners, and each
$\nu_{j}$ the weight assigned to the weak learner. The loss function $L$
measures the difference between the predicted class label $F(x_{i})$ and the
true class label $y_{i}$. The first term in the formula,
$\sum_{i=1}^{n}L(y_{i},F(x_{i}))$, is the loss function, which measures the
difference between the predicted class labels $F(x_{i})$ and the true class
labels $y_{i}$. The second term, $\sum_{j=1}^{T}\nu_{j}f_{j}(x)$, is the
weighted sum of the individual weak learners, where $\nu_{j}$ are the weights
assigned to each weak learner.

No imputation technique was used in this study as both XGBoost and LightGBM
are able to handle missing values, which are common challenges in mobile app
installation prediction. XGBoost and LightGBM use advanced techniques such as
regularization and feature selection to handle missing values and reduce
overfitting.

There are some key differences between the two algorithms. XGBoost uses a more
regularized model formalization, with an L2 regularization term on the weights
of the trees to control overfitting, whereas LightGBM uses a novel gradient-
based one-side sampling algorithm to speed up training. Additionally, LightGBM
uses histogram-based algorithms for decision tree learning, which can reduce
the calculation time compared to XGBoost.

To find the optimal hyperparameters for our XGBoost and LightGBM models, we
performed a grid search over the learning rate $\eta\in\\{0.001,0.01,0.1\\}$,
number of estimators $n_{\text{estimators}}\in\\{10,25,50\\}$, and the maximum
number of leaves in the trees $l_{\text{max}}\in\\{7,10,15\\}$. The learning
rate determines the step size at each iteration of the gradient descent
algorithm, and the number of estimators and maximum number of leaves control
the complexity of the models (Hastie et al., 2009).

##  6 Evaluation

We performed an experimental study to evaluate our method and compare its
performance, with and without a different set of features, to baseline models.
The evaluation was performed on the Bigabid dataset, to answer three research
questions that were defined.

The main research questions for this study were:

  1. RQ1.

Which features set achieves the best prediction performance?

  2. RQ2.

Which machine learning model (XGBoost or LightGBM) achieves the best
performance, in terms of prediction accuracy?

###  6.1 Experiments

We evaluated each combination of the data imputation techniques, feature
selection techniques, and the different machine learning models individually.

####  6.1.1 Experimental Setup

To avoid information leakage from the testing set (e.g., during data
preparation) it is recommended to partition a subset of the data at the
beginning of the project and reserve it for the final evaluation of the best-
performing model (Lones, 2021). Thus, 30% of the data was reserved for the
final evaluation of the best-performing model. The remaining 70% of the data
were split into 10 equal parts using 10-fold cross-validation with
stratification on the classes to answer our research questions. By using
stratified cross-validation, we are able to maintain the balance of the data
in terms of the distribution of classes within the data, ensuring that each
fold is representative of the overall dataset. To optimize the
hyperparameters, each iteration of the cross-validation was split into an
additional 10-fold cross-validation.

###  6.2 Evaluation Metrics

To evaluate the performance of our mobile gaming application installation
prediction models, we used two common evaluation metrics: the receiver
operating characteristic (ROC) curve and the precision-recall (PR) curve. The
ROC curve is a plot of the true positive rate against the false positive rate
at different classification thresholds, and the area under the ROC curve (AUC-
ROC) is a measure of the model’s ability to discriminate between positive and
negative classes. However, in imbalanced classification tasks such as in this
study, where there are significantly more negative examples than positive
ones, the AUC-ROC may not be an accurate measure of performance. For this
reason, we also computed the area under the precision-recall curve (AUPRC),
which is a more suitable metric for imbalanced classification tasks.

The AUPRC of a random estimator can be calculated by dividing the number of
positive examples by the total number of examples. Interested readers may wish
to read more about these metrics and the relationship between them in Davis
and Goadrich (2006) and Saito and Rehmsmeier (2015).

##  7 Results

In this section, we present the quantitative results of the compared methods
on the Bigabid dataset.

Figure 2: The performance of a model can be satisfactory while still
protecting the privacy of the users.

We tested the performance of seven features combination sets (Meta Features
[M], Publisher-Related [P], User-Related Features [U], Meta and Publisher
Features [MP], Meta and User Features [MU], Publisher and User Features [PU],
and Meta, User, and Publisher Features [MPU]) with two classifiers (XGBoost
[XGB] and LightGBM [LGB]) to answer research questions RQ1 and RQ2.

Figure 2 presents the combination of the feature sets results versus the two
classifiers. Figure 2 shows that including the user-related features (U, MU,
PU, MPU) in the models leads to better performance in both AUC-ROC and AUPRC.
In contrast, using the meta-features alone (M) results in poor performance of
the models. However, the results of using publisher-related features alone (P)
suggest that it is possible to train a model that protects user privacy while
still achieving quite good performance. Overall, both classifiers (XGB and
LGB) achieved quite similar performance, but XGB achieved a bit better AUC-ROC
and AUPRC results.

Figure 3: ROC and PR curves of the XGB model with the publisher-related
features.

Next, we used XGB, the best-performing model, with the publisher-related
features and learned a privacy-aware model using 70% of the data, and
evaluated the model on the 30% of that data that was reserved for the final
evaluation of the best-performing model (Sec. 6.1.1). A privacy-aware model
either excludes personal information or uses it only in an aggregated form,
which aims at preventing tracking across multiple apps and websites owned by
different companies, as well as avoiding the unique identification of
individual users or devices. In contrast, a non-privacy-aware model relies on
identifiable data such as names, emails, or physical addresses to achieve
precise targeting.

Figure 3 presents the ROC and PR curves of XGB with the publisher-related
features. As presented in Figure 3, in the PR curve, the precision of the
model is at 1 at the beginning, since at these decision cut-offs, the model
does not make any false alarms. However, the number of false positives quickly
becomes relatively large. For example, to reach a recall of around 0.5, the
precision of the model is reduced to below 0.5.

##  8 Discussion and Conclusions

In this study, we investigated the process of predicting a mobile gaming app
installation from the perspective of a DSP, while paying attention to user
privacy. This is important because current laws may force companies to declare
that they are collecting data and might even give users the option to opt out
of such data collection, which could threaten companies’ business models in
digital advertising that are dependent on the collection and use of user-level
data. We aimed to predict whether a user will install an app after being
presented with an impression, given that the DSP has won the auction in an RTB
process.

We found that learning a user privacy-aware mobile gaming app installation
prediction model leads to an AUC-ROC of 81% and an AUPRC of 49%, while using a
non-privacy-aware model leads to an AUC-ROC of 89% and AUPRC of 66%. This
suggests that it is possible to train a model that protects user privacy while
still achieving good performance.

As a commercial company, Bigabid has limitations on the data that can be
publicly shared due to legal requirements and the need to protect the
company’s customers and confidential business information. This may affect the
reproducibility of our study by other researchers. Moreover, the Bigabid data
are limited to the auctions that Bigabid was successful in winning. Thus, this
is not the exact distribution in the real world, as there can be auctions with
outliers utility estimation. To accurately learn and evaluate the real-world
distribution, we would need to have all labels and predict from the
perspective of the auction organizer. Additionally, the number of slots that
were offered and the percentage of auctions that Bigbid won or lost cannot be
disclosed.

In our future work, we aim to increase the space of hyperparameters considered
during the optimization of our models. Ensemble learning, imputation, and
feature selection can also improve model performance. Additionally, we plan to
further investigate the trade-off between privacy preservation and model
performance and explore ways to improve the performance of privacy-aware
models.

#### Conflict of interest

The authors declare that they have no conflict of interest.

## References

  * Spencer et al. (2011) S Spencer, J O’Connell, and M Greene.  The arrival of real-time bidding.  _IAB, Google, Forrester_ , 2011.
  * Al-Rubaie and Chang (2019) Mohammad Al-Rubaie and J Morris Chang.  Privacy-preserving machine learning: Threats and solutions.  _IEEE Security & Privacy_, 17(2):49–58, 2019\.
  * Hoofnagle et al. (2019) Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius.  The european union general data protection regulation: what it is and what it means.  _Information & Communications Technology Law_, 28(1):65–98, 2019.
  * Pardau (2018) Stuart L Pardau.  The california consumer privacy act: Towards a european-style privacy regime in the united states.  _J. Tech. L. & Pol’y_, 23:68, 2018.
  * Wagner (2021) Kurt Wagner.  Facebook (fb) advertisers impacted by apple (aapl) privacy ios 14 changes, Jul 2021.  URL https://www.bloomberg.com/news/articles/2021-07-14/facebook-fb-advertisers-impacted-by-apple-aapl-privacy-ios-14-changes.
  * Morrison (2021) Sara Morrison.  Why facebook and apple are fighting over your privacy, Jan 2021.  URL https://www.vox.com/recode/22254815/facebook-apple-privacy-ios-14-lawsuit.
  * Liao et al. (2014) Hairen Liao, Lingxiao Peng, Zhenchuan Liu, and Xuehua Shen.  ipinyou global rtb bidding algorithm competition dataset.  In _Proceedings of the Eighth International Workshop on Data Mining for Online Advertising_ , pages 1–6, 2014.
  * Ren et al. (2017) Kan Ren, Weinan Zhang, Ke Chang, Yifei Rong, Yong Yu, and Jun Wang.  Bidding machine: Learning to bid for directly optimizing profits in display advertising.  _IEEE Transactions on Knowledge and Data Engineering_ , 30(4):645–659, 2017.
  * Richardson et al. (2007) Matthew Richardson, Ewa Dominowska, and Robert Ragno.  Predicting clicks: estimating the click-through rate for new ads.  In _Proceedings of the 16th international conference on World Wide Web_ , pages 521–530, 2007.
  * McMahan et al. (2013) H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al.  Ad click prediction: a view from the trenches.  In _Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 1222–1230, 2013.
  * Bhamidipati et al. (2017) Narayan Bhamidipati, Ravi Kant, and Shaunak Mishra.  A large scale prediction engine for app install clicks and conversions.  In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_ , pages 167–175, 2017.
  * Zhou et al. (2018) Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai.  Deep interest network for click-through rate prediction.  In _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 1059–1068, 2018.
  * Qu et al. (2019) Xianshan Qu, Li Li, Xi Liu, Rui Chen, Yong Ge, and Soo-Hyun Choi.  A dynamic neural network model for click-through rate prediction in real-time bidding.  In _2019 IEEE international conference on big data (Big Data)_ , pages 1887–1896. IEEE, 2019.
  * Wu et al. (2018a) Wush Wu, Mi-Yen Yeh, and Ming-Syan Chen.  Deep censored learning of the winning price in the real time bidding.  In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 2526–2535, 2018a.
  * Cui et al. (2011) Ying Cui, Ruofei Zhang, Wei Li, and Jianchang Mao.  Bid landscape forecasting in online ad exchange marketplace.  In _Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 265–273, 2011.
  * Wu et al. (2015) Wush Chi-Hsuan Wu, Mi-Yen Yeh, and Ming-Syan Chen.  Predicting winning price in real time bidding with censored data.  In _Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , pages 1305–1314, 2015.
  * Zhu et al. (2017) Wen-Yuan Zhu, Wen-Yueh Shih, Ying-Hsuan Lee, Wen-Chih Peng, and Jiun-Long Huang.  A gamma-based regression for winning price estimation in real-time bidding advertising.  In _2017 IEEE International Conference on Big Data (Big Data)_ , pages 1610–1619. IEEE, 2017.
  * Tobin (1958) James Tobin.  Estimation of relationships for limited dependent variables.  _Econometrica: journal of the Econometric Society_ , pages 24–36, 1958.
  * Marmer et al. (2013) Vadim Marmer, Artyom Shneyerov, and Pai Xu.  What model for entry in first-price auctions? a nonparametric approach.  _Journal of Econometrics_ , 176(1):46–58, 2013\.
  * Liu et al. (2022) Mengjuan Liu, Zhengning Hu, Zhi Lai, Daiwei Zheng, and Xuyun Nie.  Real-time bidding strategy in display advertising: An empirical analysis.  _arXiv preprint arXiv:2212.02222_ , 2022.
  * Perlich et al. (2012) Claudia Perlich, Brian Dalessandro, Rod Hook, Ori Stitelman, Troy Raeder, and Foster Provost.  Bid optimizing and inventory scoring in targeted online advertising.  In _Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 804–812, 2012.
  * Zhang et al. (2014) Weinan Zhang, Shuai Yuan, and Jun Wang.  Optimal real-time bidding for display advertising.  In _Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 1077–1086, 2014.
  * Cai et al. (2017) Han Cai, Kan Ren, Weinan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, and Defeng Guo.  Real-time bidding by reinforcement learning in display advertising.  In _Proceedings of the Tenth ACM International Conference on Web Search and Data Mining_ , pages 661–670, 2017.
  * Wu et al. (2018b) Di Wu, Xiujun Chen, Xun Yang, Hao Wang, Qing Tan, Xiaoxun Zhang, Jian Xu, and Kun Gai.  Budget constrained bidding by model-free reinforcement learning in display advertising.  In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ , pages 1443–1451, 2018b.
  * Liu et al. (2020) Mengjuan Liu, Li Jiaxing, Zhengning Hu, Jinyu Liu, and Xuyun Nie.  A dynamic bidding strategy based on model-free reinforcement learning in display advertising.  _IEEE Access_ , 8:213587–213601, 2020.
  * Chaudhuri et al. (2011) Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate.  Differentially private empirical risk minimization.  _Journal of Machine Learning Research_ , 12(3), 2011.
  * Shokri and Shmatikov (2015) Reza Shokri and Vitaly Shmatikov.  Privacy-preserving deep learning.  In _Proceedings of the 22nd ACM SIGSAC conference on computer and communications security_ , pages 1310–1321, 2015.
  * Wood et al. (2018) Alexandra Wood, Micah Altman, Aaron Bembenek, Mark Bun, Marco Gaboardi, James Honaker, Kobbi Nissim, David R O’Brien, Thomas Steinke, and Salil Vadhan.  Differential privacy: A primer for a non-technical audience.  _Vand. J. Ent. & Tech. L._, 21:209, 2018.
  * Mohassel and Zhang (2017) Payman Mohassel and Yupeng Zhang.  Secureml: A system for scalable privacy-preserving machine learning.  In _2017 IEEE symposium on security and privacy (SP)_ , pages 19–38. IEEE, 2017.
  * Aono et al. (2017) Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, et al.  Privacy-preserving deep learning via additively homomorphic encryption.  _IEEE Transactions on Information Forensics and Security_ , 13(5):1333–1345, 2017.
  * FAT (2018) ML FAT.  Fairness, accountability, and transparency in machine learning.  _Retrieved December_ , 24:2018, 2018.
  * Raz (2021) Ido Raz.  Deep categories: Programmatic app categorization, Sep 2021.  URL https://www.bigabid.com/deep-categories/.
  * Chen and Guestrin (2016) Tianqi Chen and Carlos Guestrin.  Xgboost: A scalable tree boosting system.  In _Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining_ , pages 785–794, 2016.
  * Ke et al. (2017) Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu.  Lightgbm: A highly efficient gradient boosting decision tree.  _Advances in neural information processing systems_ , 30, 2017.
  * Hastie et al. (2009) Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman.  _The elements of statistical learning: data mining, inference, and prediction_ , volume 2.  Springer, 2009.
  * Lones (2021) Michael A Lones.  How to avoid machine learning pitfalls: a guide for academic researchers.  _arXiv preprint arXiv:2108.02497_ , 2021.
  * Davis and Goadrich (2006) Jesse Davis and Mark Goadrich.  The relationship between precision-recall and roc curves.  In _Proceedings of the 23rd International Conference on Machine Learning_ , pages 233–240, 2006.
  * Saito and Rehmsmeier (2015) Takaya Saito and Marc Rehmsmeier.  The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets.  _PloS One_ , 10(3):e0118432, 2015.