# Bayesian Inference Under Differential Privacy: Prior Selection
Considerations with Application to Univariate Gaussian Data and Regression

Zeki Kazan
Department of Statistical Science
Duke University
Durham, NC 27708
<EMAIL_ADDRESS>&Jerome P. Reiter
Department of Statistical Science
Duke University
Durham, NC 27708
<EMAIL_ADDRESS>
###### Abstract

We describe Bayesian inference for the mean and variance of bounded data
protected by differential privacy and modeled as Gaussian. Using this setting,
we demonstrate that analysts can and should take the constraints imposed by
the bounds into account when specifying prior distributions. Additionally, we
provide theoretical and empirical results regarding what classes of default
priors produce valid inference for a differentially private release in
settings where substantial prior information is not available. We discuss how
these results can be applied to Bayesian inference for regression with
differentially private data.

##  1 Introduction

Differential privacy (DP) [12] is the current gold standard for protecting
individual privacy when performing data releases involving confidential data.
A common method of ensuring a release satisfies DP is to add calibrated random
noise to sufficient statistics of the data. Once the noisy statistics are
released, analysts in theory can use them to perform statistical tasks such as
the estimation of unknown parameters of interest, prediction of new data
values, and quantification of the uncertainty around these quantities.
Performing these tasks in a principled way, however, can be nontrivial and is
an area of active research.

Bayesian methods are natural for statistical inference after a DP release. By
sampling from posterior distributions, analysts can perform estimation and
prediction tasks automatically with “built-in” uncertainty quantification. As
examples, Bernstein and Sheldon (2018) [4] and Ju et al. (2022) [19] use Gibbs
sampling to estimate differentially private posterior distributions, the
former for exponential families and the latter for likelihoods satisfying a
record additivity condition. Gong (2022) [16] uses approximate Bayesian
computation for settings where a perturbation mechanism is applied. Other
works investigate Bayesian estimation of subset proportions [28], linear
regression coefficients [5], and Poisson factorization [36]. These methods all
employ variations of the following scheme: repeatedly impute plausible values
of the nonprivate statistics based on their private counterparts, and obtain
samples of the parameters of interest via a traditional nonprivate Bayesian
analysis.

DP is related to Bayesian methods more generally. For example, releasing a
sample from a posterior distribution can be used to achieve DP [9, 15, 18, 31,
40, 42, 43]. Another line of work examines Bayesian semantics for DP and their
relationship to disclosure risk assessment [12, 21, 22, 23, 25, 26, 29]. We
also note that researchers have developed non-Bayesian methods for inference
under DP, including interval estimates [2, 10, 11, 13, 20, 37] and
significance tests [1, 2, 3, 7, 24, 35].

Existing methods for Bayesian inference under DP generally presume that the
analyst sets an informative prior distribution. In practice, however, the
analyst may not have sufficient prior knowledge to set an accurate,
informative prior. Additionally, methods in the literature typically rely on
prior distributions with unbounded support. Often, however, the confidential
data are assumed to be constrained in some way; for example, values lie within
an interval $[a,b]$ [5, 11, 37, 39, 41]. Arguably, samples from the posterior
distribution should reflect this constraint as well. Yet, existing methods for
Bayesian inference under DP can produce estimates or predictions outside the
feasible range. This may lead analysts to resort to ad hoc fixes such as
clipping estimates or predictions to be at the bounds of the interval, which
can lead to undesirable and unpredictable behavior.

We address issues resulting from ignoring constraints in Bayesian inference
under DP and the use of default prior distributions under DP. Our primary
contributions include:

  1. 1.

We illustrate the incorporation of constraints into a differentially private,
Bayesian inference procedure and demonstrate their effect on estimation and
prediction tasks.

  2. 2.

We present a first-of-its-kind examination of default prior distributions
analysts might consider in lieu of highly informative proper priors, and
present theoretical results demonstrating the need for caution when certain
weakly informative priors are used in a DP analysis.

Throughout, we use the univariate Gaussian setting as a motivating case study.
This setting is among the most important and commonly encountered in
statistics but is relatively understudied in the DP literature, with only a
few works [10, 11, 13, 20] providing frequentist methods for inference with
uncertainty quantification. General Bayesian DP inference methods such as
those in [4, 16, 19] can be applied to this setting but require approximations
or substantial computation time. As part of our contributions, we propose a
Gibbs sampling procedure for the univariate Gaussian setting without these
drawbacks.

##  2 Background and Case Study

In this section, we briefly review the definition of DP and several of its
properties. We then describe the DP univariate Gaussian setting and present
our novel Gibbs sampling scheme.

###  2.1 Differential Privacy

As described by [12], a release mechanism ${\mathcal{M}}$ satisfies
$\varepsilon$-DP if for all data sets $x,x^{\prime}$ that differ in only one
row and all $S\subseteq Range({\mathcal{M}})$,

$P[{\mathcal{M}}(x)\in S]\leq e^{\varepsilon}\,P[{\mathcal{M}}(x^{\prime})\in
S].$ (1)

DP has many desirable properties [12]. Here, we make use of post-processing.
For any data set $x$, if ${\mathcal{M}}(x)$ satisfies $\varepsilon$-DP and $g$
is not a function of $x$, then $g({\mathcal{M}}(x))$ satisfies
$\varepsilon$-DP. We also make use of composition. For any data set $x$, if
$m_{1}={\mathcal{M}}_{1}(x)$ satisfies $\varepsilon_{1}$-DP and
$m_{2}={\mathcal{M}}_{2}(x,m_{1})$ satisfies $\varepsilon_{2}$-DP, then
${\mathcal{M}}(x)=(m_{1},m_{2})$ satisfies
$(\varepsilon_{1}+\varepsilon_{2})$-DP.

For any function $f$ applied to any data set $x$, the sensitivity is defined
as $\Delta f=\max_{x,x^{\prime}}|f(x)-f(x^{\prime})|$ for all data sets
$x,x^{\prime}$ that differ in only one row. The sensitivity of $f$ is used in
the Laplace Mechanism, which is a common method for achieving DP. If a
function $f(x)$ has sensitivity $\Delta f$, then the mechanism
${\mathcal{M}}(x)=\textsf{Lap}(f(x),\Delta f/\varepsilon)$ satisfies
$\varepsilon$-DP.

###  2.2 The Univariate Gaussian Setting

Let $Y_{1},\ldots,Y_{n}$ be the confidential, scalar data values for $n$
individuals. We assume that all $Y_{i}$ are contained in some publicly known
interval $[a,b]$. The analyst models the data as independent and identically
distributed with $Y_{i}\sim{\mathcal{N}}(\mu,\sigma^{2})$ for all $i$.
Gaussian models are reasonable and commonly employed when the observed data
are symmetric with a range not near the bounds. For example, quantities like
proportions and standardized test scores have known ranges. Other quantities
like commute times and stock market returns have known lower bounds and
effective upper bounds.

The sufficient statistics are $\bar{Y}=\sum_{i=1}^{n}Y_{i}/n$ and
$S^{2}=\sum_{i=1}^{n}(Y_{i}-\bar{Y})^{2}/(n-1)$. As shown in [11], $\bar{Y}$
and $S^{2}$ have sensitivities $(b-a)/n$ and $(b-a)^{2}/n$, respectively. To
achieve $\varepsilon$-DP, the data curator can release
$\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},(b-a)/(\varepsilon_{1}n))$ and
$S^{2*}\sim\textsf{Lap}(S^{2},(b-a)^{2}/(\varepsilon_{2}n))$, where
$\varepsilon_{1}+\varepsilon_{2}=\varepsilon$. We make the simplifying
assumption that $a=0$ and $b=1$. We can do so without loss of generality, as
described in Theorem 1, which is proved in Appendix A.

######  Theorem 1.

Let $Y_{1},\ldots,Y_{n}\in[a,b]$ and let
$\tilde{Y}_{i}=(Y_{i}-a)/(b-a)\in[0,1]$. Let $\bar{Y}$ and $S^{2}$ be the
sufficient statistics for $\\{Y_{i}\\}$ and let $\tilde{\bar{Y}}$ and
$\tilde{S}^{2}$ be the sufficient statistics for $\\{\tilde{Y}_{i}\\}$.
Suppose each statistic is released via the Laplace Mechanism under
$\varepsilon$-DP and denote the DP statistics $\bar{Y}^{*}$, $S^{2*}$,
$\tilde{\bar{Y}}^{*}$, and $\tilde{S}^{2*}$. Then,
$\bar{Y}^{*}\overset{d}{=}(b-a)\tilde{\bar{Y}}^{*}+a$ and
$S^{2*}\overset{d}{=}(b-a)^{2}\tilde{S}^{2*}$.

For now, we suppose the analyst wishes to incorporate prior beliefs into their
analysis via an informative prior distribution. A conjugate choice for the
prior in the public setting is

$\displaystyle\sigma^{2}\sim\textsf{IG}\left(\nu_{0}/2,\nu_{0}\sigma_{0}^{2}/2\right),\qquad\mu\mid\sigma^{2}\sim{\mathcal{N}}\left(\mu_{0},\sigma^{2}/\kappa_{0}\right)$
(2)

with analyst-specified hyperparameters
$(\nu_{0},\kappa_{0},\mu_{0},\sigma_{0}^{2}).$

We now present a Gibbs sampler to estimate this model, building on the ideas
in [4, 5]. Our innovation is with respect to the full conditional for $S^{2}$,
for which we derive an exact form. As long as $\varepsilon_{2}<2$, the full
conditional for $S^{2}$ is a distribution we call a truncated gamma mixture
(TGM).111For the distribution of $S^{2}$ to be a TGM, we require
$\sigma^{2}<(n-1)/(2n\varepsilon_{2})$. We show in Corollary 1 that in this
setting we can guarantee $\sigma^{2}\leq 1/4$, which implies
$\sigma^{2}<(n-1)/(2n\varepsilon_{2})$ automatically when $\varepsilon_{2}<2$.
Within the Gibbs sampler, we reject and resample any draws for which
$\sigma^{2}\geq(n-1)/(2n\varepsilon_{2})$.

######  Definition 1.

A random variable $X\sim\textsf{TGM}(\alpha,\beta,\lambda,\tau)$ for
$\alpha>0$, $\beta>\lambda\geq 0$, and $\tau\in{\mathbb{R}}$ if it has
probability density function

$\displaystyle
p(x)=\begin{cases}\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha)}x^{\alpha-1}e^{-(\beta+\lambda)x},&\mbox{if
}\tau\leq 0;\\\
\pi_{1}\frac{(\beta-\lambda)^{\alpha}}{\gamma(\alpha,(\beta-\lambda)\tau)}x^{\alpha-1}e^{-(\beta-\lambda)x},&\mbox{if
}\tau>0\mbox{ and }x\leq\tau;\\\
\pi_{2}\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha,(\beta+\lambda)\tau)}x^{\alpha-1}e^{-(\beta+\lambda)x}&\mbox{if
}\tau>0\mbox{ and }x>\tau,\end{cases}$ (3)

where

$\displaystyle\pi_{1}=\frac{e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}}{e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}},\qquad\pi_{2}=\frac{e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}}{e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}}.$
(4)

Here, $\gamma(\alpha,x)$ is the lower incomplete gamma function and
$\Gamma(\alpha,x)$ is the upper incomplete gamma function. We provide an
algorithm for sampling from the TGM in Appendix B.

Derivations for all full conditionals for the Gibbs sampler are provided in
Appendix C. The full conditionals for $\mu$ and $\sigma^{2}$ are identical to
those of the nonprivate setting. To obtain the full conditional for $\bar{Y}$,
we follow the strategy of [4] based on the Bayesian LASSO [34] and use the
fact that $\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},1/(\varepsilon_{1}n))$ is equal
in distribution to $\bar{Y}^{*}\sim{\mathcal{N}}(\bar{Y},\omega^{2})$ for
$\omega^{2}\sim\textsf{Exp}(\varepsilon_{1}^{2}n^{2}/2)$. The full conditional
for $\bar{Y}^{*}$ is Gaussian, the full conditional for $1/\omega^{2}$ is
inverse-Gaussian, and the full conditional for $S^{2}$ is a TGM. The
parameters of the full conditionals are provided in Appendix D.

A major advantage of this sampler is its runtime. If $T$ is the number of
Gibbs iterations, then the computational complexity of this sampler is
$\mathcal{O}(T)$; the runtime does not scale with the sample size, $n$. The
Gibbs sampler of [4] for exponential families applied to the univariate
Gaussian setting also has computational complexity $\mathcal{O}(T)$. This
sampler, however, requires a Gaussian approximation for $S^{2}$, which,
particularly when $\sigma^{2}$ is near zero or $n$ is small, can lead to
unreliable results. The sampler of [19] does not require any approximations;
however, it has computational complexity $\mathcal{O}(nT)$, making it
potentially infeasible for large $n$. Our proposed sampler achieves both the
exact sampling of [19] and the $\mathcal{O}(T)$ computational complexity of
[4].

##  3 Incorporating Constraints

By incorporating constraints on data values, analysts can reduce the
uncertainty in estimates of the parameters of interest “for free.” They need
not make additional assumptions; they merely determine how the conditions on
the data values constrain the model used for analysis.

###  3.1 Constraints in the Univariate Gaussian Setting

In the univariate Gaussian setting from Section 2.2, the constraint that
$Y_{i}\in[0,1]$ affects the model in two ways. Firstly, the bound constrains
the parameters $\mu$ and $\sigma^{2}$, since bounded random variables have
bounded expectations and variances. Secondly, the bound constrains the
sufficient statistics $\bar{Y}$ and $S^{2}$, since samples of bounded random
variables have bounded means and variances. We demonstrate how we establish
these bounds in a manner that can be easily incorporated into the Gibbs
sampler from Section 2.2. See Appendix E for proofs.

First, Theorem 2 establishes conditional bounds for $\mu\mid\sigma^{2}$ and
$\sigma^{2}\mid\mu$ that can be incorporated into the prior. That is, since
the data values are bounded, we may replace the unbounded normal-inverse gamma
prior from (2) with a prior of the same form truncated to be within the
feasible region. This yields a posterior of the same form as in Section 2.2
truncated to the feasible region.

######  Theorem 2.

Let $Y_{1},\ldots,Y_{n}\sim{\mathcal{N}}(\mu,\sigma^{2})$ and suppose each
$Y_{i}\in[0,1]$. If $\mu$ is known, then $\sigma^{2}\in[0,\mu(1-\mu)]$. If
$\sigma^{2}$ is known, then
$\mu\in[1/2-\sqrt{1/4-\sigma^{2}},1/2+\sqrt{1/4-\sigma^{2}}]$.

######  Corollary 1.

If $Y_{1},\ldots,Y_{n}\sim{\mathcal{N}}(\mu,\sigma^{2})$ and $Y_{i}\in[0,1]$,
then $\sigma^{2}\leq 1/4$.

Theorem 3 establishes bounds for $\bar{Y}\mid S^{2}$ and $S^{2}\mid\bar{Y}$
that can be incorporated into the likelihood. This yields a posterior of the
same form as in Section 2.2 truncated to the feasible region.

######  Theorem 3.

Let $Y_{1},\ldots,Y_{n}\sim{\mathcal{N}}(\mu,\sigma^{2})$ and suppose each
$Y_{i}\in[0,1]$. If $\bar{Y}$ is known, then
$S^{2}\in[0,n/(n-1)\bar{Y}(1-\bar{Y})]$. If $S^{2}$ is known, then
$\bar{Y}\in[1/2-\sqrt{1/4-(n-1)/n\cdot S^{2}},1/2+\sqrt{1/4-(n-1)/n\cdot
S^{2}}]$.

The full conditionals under these constraints are described in Appendix D.

###  3.2 Example: Constraints for the Blood Lead Levels Dataset

We now demonstrate the effect of incorporating these constraints using genuine
data.222Code to reproduce all experiments can be found at
https://github.com/zekicankazan/dp_priors All computations are performed on
the $[0,1]$ scale, but all quantities are converted back to the original scale
for clarity of presentation. Interval estimates are those with highest
posterior density (HPD).333HPD intervals are preferred over quantile-based
intervals when the posterior distribution is skewed [27]. HPD intervals can be
computed with standard tools, such as the HDInterval package in R [30].

######  Example 1.

Researchers sampled the blood lead levels of $n=43$ policemen assigned to
outdoor work in Egypt to examine the effect of leaded gasoline on exposed
individuals [32]. The sample mean was $\bar{Y}=32.08\,\mu g/dL$, and the
sample variance was $S^{2}=16.98^{2}\,\mu g^{2}/dL^{2}$. An expert advises
that blood lead levels in this region are reasonably bounded above by
$100\,\mu g/dL$. A realization of the Laplace Mechanism with
$\varepsilon_{1}=\varepsilon_{2}=0.25$ yields noisy statistics
$\bar{Y}^{*}=34.30\,\mu g/dL$ and $S^{2*}=47.16^{2}\,\mu g^{2}/dL^{2}$. An
analyst wishes to estimate $\mu$, the average blood lead level of all
policemen assigned to outdoor work in Egypt. She believes blood lead levels
are reasonably approximated by a Gaussian distribution. To reflect prior
knowledge, she uses (2) with
$(\mu_{0}=12.5,\sigma_{0}^{2}=3.8^{2},\kappa_{0}=1,\nu_{0}=1)$, roughly
representing the equivalent information of a single “prior data point.”

Figure 1: Joint posterior distribution for $(\mu,\sigma^{2})$ in Example 1.
Here, $(\bar{Y}^{*}=34.30,S^{2*}=47.17^{2})$ is represented by the red circle,
the unreleased $(\bar{Y}=32.08,S^{2}=16.96^{2})$ is represented by the green
triangle, the analyst’s posterior mode is represented by the blue diamond, and
$(\mu_{0}=12.5,S^{2}=3.8^{2})$ is represented by the purple square. Upper and
lower panels display the posterior when constraints are and are not accounted
for, respectively. The shaded area represents the feasible region for
$(\mu,\sigma^{2})$ from Theorem 2. The main and secondary axes present the
original and $[0,1]$ scale, respectively. Plots based on 5,000 Gibbs sampler
iterations.

Figure 1 presents the joint posterior distribution for $(\mu,\sigma^{2})$ with
and without constraints accounted for. For this particular release, $S^{2*}$
is larger than the nonprivate $S^{2}$ and is near the upper bound on
$\sigma^{2}$ determined by Theorem 2. Thus, more than 10% of samples from the
unconstrained analysis have infeasibly large $\sigma^{2}$. Additionally, the
unconstrained analysis produces samples with $\mu$ either negative or above
the upper bound. The constrained analysis respects the bounds and produces
only reasonable samples for $\mu$ and $\sigma^{2}$.

In both analyses, the analyst’s posterior mode is around $\hat{\mu}\approx
16\,\mu\textup{g}/\textup{dL}$ and $\hat{\sigma}^{2}\approx
5^{2}\,\mu\textup{g}^{2}/\textup{dL}^{2}$. The analyses, however, have
substantially different amounts of uncertainty. The 95% HPD interval for
$\sigma^{2}$ is $[1.2^{2},50.8^{2}]$ without constraints and
$[1.0^{2},39.3^{2}]$ with constraints. Accounting for constraints allows the
analyst to rule out the region with infeasibly high $\sigma^{2}$, yielding a
tighter interval estimate. The 95% HPD interval for $\mu$ is $[3.4,48.2]$
without constraints and $[1.9,42.0]$ with constraints. Similarly to the above,
the tighter interval in the constrained analysis is due primarily to ruling
out $\mu$ in the region where $\sigma^{2}$ is infeasibly large. In both
analyses, the posterior concentration is highest near the prior values,
$\mu_{0}$ and $\sigma_{0}^{2}$; there is much less density near the released
noisy values. This indicates that the posterior inference is likely quite
sensitive to the analyst’s prior distribution.

We emphasize that the constrained posterior is not merely a truncated version
of the unconstrained posterior; truncation in this way would yield inaccurate
point and interval estimates. Instead, by truncating within the Gibbs sampler,
we have mathematically coherent estimates.

##  4 Default Prior Choices

As evident in Example 1, the results of a Bayesian analysis under DP can be
sensitive to the choice of prior distribution. This is unsurprising, since the
addition of noise yields data with less information about model parameters
than in the public setting, making it difficult to overwhelm the information
in the prior distribution. One potential fix—used in examples in the Bayesian
DP inference literature—is to increase the prior variance, yielding a weakly
informative prior. One also could examine the limit in which the prior
variance is infinite. We show, however, that this strategy may lead to
improper limiting posterior distributions, even in settings where the
nonprivate posterior is proper. In such cases, weakly informative priors may
yield unreliable inference and should be avoided [14].

###  4.1 Default Priors in the Univariate Gaussian Setting

For the unconstrained analysis in the univariate Gaussian setting, a weakly
informative prior of the form in (2) is created by making $\kappa_{0}$ and
$\nu_{0}$ very small. In the limit as these hyperparameters go to zero, this
produces the default prior $p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$. In the
nonprivate setting, this is called the independent Jeffrey’s prior [38]. While
it does not correspond to a proper probability distribution, the posterior
distribution it produces is not only proper, but also frequentist
matching.444In general, Bayesian credible intervals do not have the property
that a 95% interval will cover the true value 95% of the time under repeated
sampling. Frequentist matching priors, however, produce credible intervals
identical to corresponding frequentist confidence intervals and so 95%
intervals have 95% coverage (see [8]). In the private setting, unfortunately,
the posterior distribution produced by
$p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$ is not a proper probability
distribution. The following result, proved in Appendix F, demonstrates this.

######  Theorem 4.

For confidential data
$Y_{1},\ldots,Y_{n}\overset{iid}{\sim}{\mathcal{N}}(\mu,\sigma^{2})$ where
sufficient statistics
$\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},1/(\varepsilon_{1}n))$ and
$S^{2*}\sim\textsf{Lap}(S^{2},1/(\varepsilon_{2}n))$ are released, if an
analyst has prior $p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$, then their
posterior $p(\mu,\sigma^{2}\mid\bar{Y}^{*},S^{2*})$ is not a proper
probability distribution.

An alternative strategy for finding a default prior for this setting is to
focus on the likelihood’s Laplace portion. We show in Appendix G that for a
Laplace likelihood in the nonprivate setting, a uniform prior produces a
proper, frequentist matching posterior distribution. A uniform prior on
$(\mu,\sigma^{2})$ thus may be a reasonable choice in the private setting.
Theorem 5, proved in Appendix F, demonstrates that $p(\mu,\sigma^{2})\propto
1$ does indeed produce a proper posterior distribution.

######  Theorem 5.

For confidential data
$Y_{1},\ldots,Y_{n}\overset{iid}{\sim}{\mathcal{N}}(\mu,\sigma^{2})$ where
$n>3$ and sufficient statistics
$\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},1/(\varepsilon_{1}n))$ and
$S^{2*}\sim\textsf{Lap}(S^{2},1/(\varepsilon_{2}n))$ are released, if an
analyst has prior $p(\mu,\sigma^{2})\propto 1$, then their posterior
$p(\mu,\sigma^{2}\mid\bar{Y}^{*},S^{2*})$ is a proper probability
distribution.

Similarly, we can determine posterior propriety for the constrained analysis.
Since a constrained uniform distribution has bounded support, it is a proper
prior distribution and thus produces a proper posterior distribution.
Likewise, it can be verified empirically that the posterior produced by
$p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$ also is improper in the
constrained analysis.

Figure 2: The empirical coverage rate (top) and average length (bottom) of 95%
HPD intervals for $\mu$ for different sample sizes. Results based on 10,000
simulated datasets $Y_{i}\in[0,1]$ released with
$\varepsilon_{1}=\varepsilon_{2}=0.1$ and analyzed with prior
$p(\mu,\sigma^{2})\propto 1$. Analyses with constraints accounted for are
solid lines and not accounted for are dashed lines. Data generating model is
either ${\mathcal{N}}(\mu=0.1,\sigma^{2}=0.04^{2})$ (red) or
${\mathcal{N}}(\mu=0.5,\sigma^{2}=0.2^{2})$ (blue). Each Gibbs sampler is run
for 20,000 iterations.

We use simulation experiments to assess the properties of the posterior
distributions based on priors that use or disregard the constraints on the
data values. For each of 10,000 simulated datasets of a given size $n$, we
create a 95% HPD interval based on the Gibbs sampler in Appendix D. Figure 2
displays the results; see Appendix J for run-time details. Without enforcing
constraints, the 95% credible interval has approximately the nominal 95%
coverage rate regardless of $n$, and the average interval length is linearly
related to $n$ on the log-log scale.555A regression of log average interval
length on log $n$ has a slope of $-1$ (with $R^{2}>0.999$), indicating that
average interval length decays proportionally to $1/n$. A similar result holds
for the RMSE; see Appendix H. When enforcing the constraints, the results are
similar to those for the unconstrained analysis when $n$ is relatively large.
In these cases, the credible intervals for $\mu$ are far from the boundary, so
that the constraints have little effect. When $n$ is relatively small, the 95%
credible interval has lower than 95% coverage rate when the true $\mu$ is
close to the boundary and near 100% coverage when $\mu$ is in the middle of
the range. In fact, it can be shown that the prior $p(\mu,\sigma^{2})\propto
1$ over the region where $\mu\in[0,1]$ and $\sigma^{2}\in(0,\mu(1-\mu)]$
induces the marginal distribution $\mu\sim\textsf{Beta}(2,2)$. This Beta
distribution places more probability density in the center of the distribution
than near $0$ or $1$, resulting in the observed over and under coverage,
depending on the true value of $\mu$. This suggests that
$p(\mu,\sigma^{2})\propto 1$ is most appropriate when an analyst believes
$\mu$ more likely is near the center of the distribution than the tails. That
is, while the prior is “non-informative” in the unconstrained analysis, it is
not so in the constrained analysis.

Enforcing the constraint, however, does have advantages. For all points in
Figure 2, the average interval length for the constrained analysis is less
than $0.75$. Meanwhile, the average interval length without constraints is
greater than $1$ for all $n\leq 50$, indicating that the credible intervals
must include values that are not within $[0,1]$. Such intervals often include
all of $[0,1]$, yielding intervals that are useless in practice. Indeed, the
calibrated coverage rate for the unconstrained analysis is of dubious merit,
as these credible intervals are inflated by permitting probability mass on
infeasible regions of support. Additionally, when we estimate $\mu$ with its
posterior mode, the RMSEs are uniformly lower in the constrained analysis than
in the unconstrained; see Figure 8 in Appendix H. When $\mu=0.5$ the
constrained analysis delivers a “win-win” result: it has a higher coverage
rate with a shorter average interval length and a lower RMSE than the
unconstrained analysis.

The results described above use a total $\varepsilon$ of $0.2$. When
$\varepsilon$ is larger, the constrained and unconstrained analyses are more
similar; see Figure 9 in Appendix H for additional simulations with total
$\varepsilon=2$. When $\varepsilon$ is larger, the unconstrained analysis no
longer offers approximately exact coverage: the 95% posterior credible
intervals cover more than 95% of the time.

###  4.2 Default Priors for the Blood Lead Levels Example

Returning to the setting of Example 1, we examine the effect of replacing the
informative prior used in Section 3.2 with the default prior
$p(\mu,\sigma^{2})\propto 1$. Figure 3 presents a plot analogous to Figure 1
with the new prior. In the unconstrained analysis, the posterior mode is
approximately equal to the released statistics, but more than 50% of posterior
draws are outside of the feasible region. In the constrained analysis, as
discussed above, the posterior mode for $\mu$ is shifted towards the center of
the distribution. A similar effect is observed for $\sigma^{2}$; the prior has
more mass closer to zero and so the posterior mode is smaller than $S^{2*}$.

Figure 3: Plot of the joint posterior distribution for $(\mu,\sigma^{2})$ in
Example 1 under a uniform prior. The point
$(\bar{Y}^{*}=34.30,S^{2}=47.17^{2})$ is represented by the red circle, the
unreleased point $(\bar{Y}=32.08,S^{2}=16.96^{2})$ is represented by the green
triangle, and the analyst’s posterior mode is represented by the blue diamond.
The upper and lower panels provide the posterior when constraints are and are
not accounted for, respectively. The shaded area represents the feasible
region for $(\mu,\sigma^{2})$ from Theorem 2. The main axes and secondary axes
present the original and $[0,1]$ scale, respectively. This plot is based on
5,000 Gibbs iterations.

Analysts also might be interested in using the Bayesian model for prediction.
Figure 4 plots the posterior predictive distribution for the uniform prior,
which is computed via
$Y_{\textup{new}}^{(t)}\sim{\mathcal{N}}(\mu^{(t)},\sigma^{2(t)})$ for each
Gibbs iteration $t$, under the unconstrained and constrained analyses.666For
the constrained analysis, the distribution is replaced by a truncated
Gaussian. The constrained posterior predictive distribution only includes
values in the feasible region of $0$ to $100\,\mu\textup{g}/\textup{dL}$, with
the distribution centered around the posterior mode. The unconstrained
posterior predictive distribution, meanwhile, generates 24% of predictive
draws as negative values and 10% as values greater than
$100\,\mu\textup{g}/\textup{dL}$. To obtain reasonable predictions with the
draws from the unconstrained analysis, an analyst must resort to ad hoc
methods. They might, for example, re-code all negative predictions to $0$ and
all predictions greater than $100$ to $100$. This would, among other effects,
lead to 24% of predictions being exactly zero, which is not reasonable
scientifically. Alternatively, the analyst could sample predictions from a
truncated Gaussian distribution. The variance of this distribution is still
substantially inflated by the samples of overly large values of
$\sigma^{2(t)}$. Thus, predictions have larger variability under this ad hoc
approach than under the theoretically principled constrained analysis.

Figure 4: Posterior predictive distribution of a new observation given the
posterior draws from Example 1 and prior $p(\mu,\sigma^{2})\propto 1$. The
value $\bar{Y}^{*}=34.30$ is represented by the solid red line, the
(unreleased) value $\bar{Y}=32.08$ is represented by the dotted green line,
and the analyst’s posterior mode is represented by the blue dot-dashed line.
Upper and lower panels provide the posterior predictive distributions without
and without accounting for constraints, respectively. The shaded area
represents the feasible region for a new observation. The main axis and
secondary axis present the original and $[0,1]$ scale, respectively. Plots
based on 100,000 Gibbs sampler iterations.

##  5 Non-Univariate Gaussian Application

The strategies we discuss can be applied to any Bayesian analysis under DP. To
demonstrate, we consider the DP Bayesian linear regression method of [5]. In
particular, we adapt an example from [5] using data from [6] to illustrate how
the strategies for enforcing constraints in Section 3 can be used off-the-
shelf to supplement an existing method.

The data in [5, 6] include the response variable cirrhosis rate,
$\\{y_{i}\\}$, explanatory variable drinking rate, $\\{x_{i}\\}$, and sample
size $n=46$. The authors rescale both variables to lie in $[0,1]$ and assume
the range of both quantities is public information. Letting
${\mathbf{Y}}\in{\mathbb{R}}^{n}$ be the vector of responses and
${\mathbf{X}}\in{\mathbb{R}}^{n\times 2}$ be a design matrix with a column of
ones, the authors’ model is a simple linear regression of the form below with
the following informative, conjugate prior.

$\displaystyle{\mathbf{Y}}\sim{\mathcal{N}}_{n}({\mathbf{X}}{\boldsymbol{\theta}},\sigma^{2}{\mathbf{I}}_{n}),\qquad{\boldsymbol{\theta}}\mid\sigma^{2}\sim{\mathcal{N}}_{2}\left(\begin{bmatrix}1\\\
0\end{bmatrix},\frac{\sigma^{2}}{4}\mathbf{I}_{2}\right),\quad\sigma^{2}\sim\textsf{IG}\left(\frac{40}{2},\frac{1}{2}\right),$
(5)

where $\mathbf{I}_{p}$ is the $p\times p$ identity matrix. Their proposed
method involves applying the Laplace Mechanism to each element of the
sufficient statistics ${\mathbf{X}}^{\top}{\mathbf{X}}$,
${\mathbf{X}}^{\top}{\mathbf{Y}}$ and ${\mathbf{Y}}^{\top}{\mathbf{Y}}$. To
ensure a valid regression solution exists, the authors enforce that the matrix
${\mathbf{Z}}^{\top}{\mathbf{Z}}$, where ${\mathbf{Z}}=[{\mathbf{X}}\hskip
5.78172pt{\mathbf{Y}}]$, is positive semi-definite (PSD) by projecting sampled
values to the nearest PSD matrix. They apply their Gibbs-SS-Noisy method,
which also adds Laplace noise to the matrix of fourth sample moments of
${\mathbf{X}}$.

Since it is known that $x_{i},y_{i}\in[0,1]$ for all $i$, we can exploit the
resulting structure in the linear regression to constrain parameter updates in
the Gibbs sampler. In particular, letting
${\mathbf{X}}=[{\mathbf{1}}_{n}\hskip 5.78172pt{\mathbf{x}}_{1}]$, we show in
Appendix I that
$0\leq{\mathbf{x}}_{1}^{\top}{\mathbf{x}}_{1}\leq{\mathbf{x}}_{1}^{\top}{\mathbf{1}}\leq
n$ and
$0\leq{\mathbf{Y}}^{\top}{\mathbf{Y}}\leq{\mathbf{Y}}^{\top}{\mathbf{1}}\leq
n$. We also show that
${\mathbf{x}}_{1}^{\top}{\mathbf{Y}}\leq\min\\{{\mathbf{x}}_{1}^{\top}{\mathbf{1}},{\mathbf{Y}}^{\top}{\mathbf{1}}\\}$
and that $0\leq\sigma^{2}\leq 1/4$. Letting the regression coefficients be
${\boldsymbol{\theta}}=[\theta_{0}\hskip 5.78172pt\theta_{1}]^{\top}$, we know
that $(\theta_{1},\theta_{0})$ must lie in the region depicted in the right
panels of Figure 5. We create a constrained Gibbs sampler by drawing each
iterate from the unconstrained full conditional distribution and rejecting and
resampling whenever all constraints are not satisfied. We also reject and
resample if ${\mathbf{Z}}^{\top}{\mathbf{Z}}$ is not PSD.

Figure 5: Plot of posterior draws from the linear regression method of [5].
The left panels represent draws of the imputed sufficient statistics
${\mathbf{Y}}^{\top}{\mathbf{1}}$ and ${\mathbf{Y}}^{\top}{\mathbf{Y}}$, while
the right panels represent draws of the parameters $\theta_{1}$ and
$\theta_{0}$. The upper and lower panels provide the posterior when
constraints are and are not accounted for, respectively. The shaded areas
represent the feasible regions; points outside the feasible regions are
colored in red. This plot is based on 10,000 Gibbs iterations.

Figure 5 demonstrates some of the effects of enforcing these constraints for
one particular set777The addition of Laplace noise to the fourth sample
moments of ${\mathbf{X}}$ adds additional uncertainty which is not accounted
for in the procedure in [5]. Because of this, our rejection sampling approach
struggles when the released noisy moments are far from their nonprivate
counterparts, leading to the Gibbs sampling procedure being unable to find a
draw for the next Gibbs iterate satisfying the constraints and getting
“stuck.” For demonstrative purposes, we choose a seed where this issue is not
encountered. of released noisy values with $\varepsilon=0.1$ for each of the
11 queries. The shaded triangles in the left panels represent the constraints
that
$0\leq{\mathbf{Y}}^{\top}{\mathbf{Y}}\leq{\mathbf{Y}}^{\top}{\mathbf{1}}\leq
n$. We find that around 20% of the posterior draws without accounting for
constraints do not satisfy this inequality, as indicated by the red points.
The shaded regions in the right panels represent the feasible region for
${\boldsymbol{\theta}}$. In this case, less than 5% of the posterior draws did
not satisfy the constraint in ${\boldsymbol{\theta}}$. But notably, when all
relevant constraints are enforced, the posterior distribution is more spread
out throughout the feasible region. In this case, not enforcing the
constraints leads artificially to an estimated model with too little
uncertainty about its parameter estimates and thus underestimated uncertainty
around downstream predictions.

This regression example, as well as univariate Gaussian case, suggest that
incorporating constraints in Bayesian DP inference, thereby respecting the
actual support of the parameters, can result in potentially more accurate
representations of posterior distributions. Additionally, the examples
indicate that informative prior distributions can have significant impact on
DP posterior inferences. The use of weakly informative or default prior
distributions may be advantageous, although analysts must take care to ensure
posterior distributions are not improper, as demonstrated in Section 4.
Finally, these results suggest there is scope for further development of
default prior distributions to obtain theoretically valid and accurate
posterior distributions with desirable frequentist properties.

## Acknowledgments and Disclosure of Funding

This research was supported by NSF grant SES-2217456.

## References

  * [1] Daniel Alabi and Salil Vadhan.  Hypothesis testing for differentially private linear regression.  Advances in Neural Information Processing Systems, 35:14196–14209, 2022.
  * [2] Jordan Awan and Aleksandra Slavković.  Differentially private uniformly most powerful tests for binomial data.  Advances in Neural Information Processing Systems, 31, 2018.
  * [3] Andrés F Barrientos, Jerome P Reiter, Ashwin Machanavajjhala, and Yan Chen.  Differentially private significance tests for regression coefficients.  Journal of Computational and Graphical Statistics, 28(2):440–453, 2019.
  * [4] Garrett Bernstein and Daniel R Sheldon.  Differentially private Bayesian inference for exponential families.  Advances in Neural Information Processing Systems, 31, 2018.
  * [5] Garrett Bernstein and Daniel R Sheldon.  Differentially private Bayesian linear regression.  Advances in Neural Information Processing Systems, 32, 2019.
  * [6] Kenneth Alexander Brownlee.  Statistical theory and methodology in science and engineering.  https://people.sc.fsu.edu/~jburkardt/datasets/regression/x20.txt, 1965.  Accessed on 4 May 2024.
  * [7] Simon Couch, Zeki Kazan, Kaiyan Shi, Andrew Bray, and Adam Groce.  Differentially private nonparametric hypothesis testing.  In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 737–751, 2019.
  * [8] Gauri Sankar Datta and Trevor J Sweeting.  Probability matching priors.  In D K Dey and C R Rao, editors, Bayesian Thinking, volume 25 of Handbook of Statistics, pages 91–114. Elsevier, 2005.
  * [9] Christos Dimitrakakis, Blaine Nelson, Zuhe Zhang, Aikaterini Mitrokotsa, and Benjamin IP Rubinstein.  Differential privacy for Bayesian inference through posterior sampling.  Journal of Machine Learning Research, 18(11):1–39, 2017.
  * [10] Vito D’Orazio, James Honaker, and Gary King.  Differential privacy for social science inference.  Sloan Foundation Economics Research Paper, (2676160), 2015.
  * [11] Wenxin Du, Canyon Foot, Monica Moniot, Andrew Bray, and Adam Groce.  Differentially private confidence intervals.  arXiv preprint arXiv:2001.02285, 2020.
  * [12] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith.  Calibrating noise to sensitivity in private data analysis.  In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, pages 265–284. Springer, 2006.
  * [13] Cecilia Ferrando, Shufan Wang, and Daniel Sheldon.  Parametric bootstrap for differentially private confidence intervals.  In International Conference on Artificial Intelligence and Statistics, pages 1598–1618. PMLR, 2022.
  * [14] Andrew Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper).  Bayesian Analysis, 1(3):515–534, 2006.
  * [15] Joseph Geumlek, Shuang Song, and Kamalika Chaudhuri.  Rényi differential privacy mechanisms for posterior sampling.  Advances in Neural Information Processing Systems, 30, 2017.
  * [16] Ruobin Gong.  Exact inference with approximate computation for differentially private data via perturbations.  Journal of Privacy and Confidentiality, 12(2), 2022.
  * [17] Peter D Hoff.  A First Course in Bayesian Statistical Methods, volume 580.  Springer, 2009.
  * [18] Jack E Jewson, Sahra Ghalebikesabi, and Chris C Holmes.  Differentially private statistical inference through beta-divergence one posterior sampling.  Advances in Neural Information Processing Systems, 36, 2024.
  * [19] Nianqiao Ju, Jordan Awan, Ruobin Gong, and Vinayak Rao.  Data augmentation mcmc for Bayesian inference from privatized data.  Advances in neural information processing systems, 35:12732–12743, 2022.
  * [20] Vishesh Karwa and Salil Vadhan.  Finite sample differentially private confidence intervals.  arXiv preprint arXiv:1711.03908, 2017.
  * [21] Shiva P Kasiviswanathan and Adam Smith.  On the’semantics’ of differential privacy: A Bayesian formulation.  Journal of Privacy and Confidentiality, 6(1), 2014.
  * [22] Zeki Kazan and Jerome P Reiter.  Prior-itizing privacy: A Bayesian approach to setting the privacy budget in differential privacy.  arXiv preprint arXiv:2306.13214, 2023.
  * [23] Zeki Kazan and Jerome P Reiter.  Assessing statistical disclosure risk for differentially private, hierarchical count data, with application to the 2020 us decennial census.  Statistica Sinica, 34:to appear, 2024.
  * [24] Zeki Kazan, Kaiyan Shi, Adam Groce, and Andrew P Bray.  The test of tests: A framework for differentially private hypothesis testing.  In International Conference on Machine Learning, pages 16131–16151. PMLR, 2023.
  * [25] Daniel Kifer, John M Abowd, Robert Ashmead, Ryan Cumings-Menon, Philip Leclerc, Ashwin Machanavajjhala, William Sexton, and Pavel Zhuravlev.  Bayesian and frequentist semantics for common variations of differential privacy: Applications to the 2020 census.  arXiv preprint arXiv:2209.03310, 2022.
  * [26] Daniel Kifer and Ashwin Machanavajjhala.  Pufferfish: A framework for mathematical privacy definitions.  ACM Transactions on Database Systems (TODS), 39(1):1–36, 2014.
  * [27] John K Kruschke.  Bayesian approaches to testing a point (“null”) hypothesis.  In John K Kruschke, editor, Doing Bayesian Data Analysis (Second Edition), pages 335–358. Academic Press, Boston, second edition edition, 2015.
  * [28] Linlin Li and Jerome P Reiter.  Bayesian inference for estimating subset proportions using differentially private counts.  Journal of Survey Statistics and Methodology, 10(3):785–803, 2022.
  * [29] David McClure and Jerome P Reiter.  Differential privacy and statistical disclosure risk measures: An investigation with binary synthetic data.  Trans. Data Priv., 5(3):535–552, 2012.
  * [30] Mike Meredith and John Kruschke.  HDInterval: Highest (Posterior) Density Intervals, 2020.  R package version 0.2.2.
  * [31] Kentaro Minami, Hitomi Arai, Issei Sato, and Hiroshi Nakagawa.  Differential privacy without sensitivity.  Advances in Neural Information Processing Systems, 29, 2016.
  * [32] Wael Mortada, Mohamed Sobh, M M El-Defrawy, and S E Farahat.  Study of lead exposure from automobile exhaust as a risk for nephrotoxicity among traffic policemen.  American Journal of Nephrology, 21(4):274–279, 2001.
  * [33] Frederick Novomestky and Saralees Nadarajah.  truncdist: Truncated Random Variables, 2016.  R package version 1.0-2.
  * [34] Trevor Park and George Casella.  The Bayesian lasso.  Journal of the American Statistical Association, 103(482):681–686, 2008.
  * [35] Víctor Peña and Andrés F Barrientos.  Differentially private hypothesis testing with the subsampled and aggregated randomized response mechanism.  Statistica Sinica, 34:to appear, 2024.
  * [36] Aaron Schein, Zhiwei Steven Wu, Alexandra Schofield, Mingyuan Zhou, and Hanna Wallach.  Locally private Bayesian inference for count models.  In International Conference on Machine Learning, pages 5638–5648. PMLR, 2019.
  * [37] Or Sheffet.  Differentially private ordinary least squares.  In International Conference on Machine Learning, pages 3105–3114. PMLR, 2017.
  * [38] Dongchu Sun and James O Berger.  Objective Bayesian analysis for the multivariate normal model.  Bayesian Statistics, 8:525–562, 2007.
  * [39] Yu-Xiang Wang.  Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain.  arXiv preprint arXiv:1803.02596, 2018.
  * [40] Yu-Xiang Wang, Stephen Fienberg, and Alex Smola.  Privacy for free: Posterior sampling and stochastic gradient Monte Carlo.  In International Conference on Machine Learning, pages 2493–2502. PMLR, 2015.
  * [41] Jun Zhang, Zhenjie Zhang, Xiaokui Xiao, Yin Yang, and Marianne Winslett.  Functional mechanism: regression analysis under differential privacy.  Proceedings of the VLDB Endowment, 5(11):1364–1375, 2012.
  * [42] Wanrong Zhang and Ruqi Zhang.  Dp-fast mh: Private, fast, and accurate Metropolis-Hastings for large-scale Bayesian inference.  In International Conference on Machine Learning, pages 41847–41860. PMLR, 2023.
  * [43] Zuhe Zhang, Benjamin Rubinstein, and Christos Dimitrakakis.  On the differential privacy of Bayesian inference.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.

##  Appendix A Re-scaling The Data

Throughout the document, we assume that the data $Y_{1},\ldots,Y_{n}$ are
contained in the interval $[a,b]$ for known, public $a$ and $b$. In this
section, we prove that, without loss of generality, we may consider data on
the interval $[0,1]$ by re-scaling the data as follows to obtain
$\tilde{Y}_{i}\in[0,1]$,

$\displaystyle\tilde{Y}_{i}=\frac{Y_{i}-a}{b-a}.$ (6)

We can convert back to the original scale via the relationship
$Y_{i}=(b-a)\tilde{Y}_{i}+a$.

We now show that the sufficient statistics released via the Laplace Mechanism
on the $[0,1]$ scale are re-scaled versions of the sufficient statistics
released via the Laplace Mechanism on the $[a,b]$ scale. Thus no information
is lost from scaling before the DP release and re-scaling afterwards.

######  Theorem 1.

Let $Y_{1},\ldots,Y_{n}\in[a,b]$ and let
$\tilde{Y}_{i}=(Y_{i}-a)/(b-a)\in[0,1]$. Let $\bar{Y}$ and $S^{2}$ be the
sufficient statistics for $\\{Y_{i}\\}$ and let $\tilde{\bar{Y}}$ and
$\tilde{S}^{2}$ be the sufficient statistics for $\\{\tilde{Y}_{i}\\}$.
Suppose each statistic is released via the Laplace Mechanism under
$\varepsilon$-DP and denote the DP statistics $\bar{Y}^{*}$, $S^{2*}$,
$\tilde{\bar{Y}}^{*}$, and $\tilde{S}^{2*}$. Then,
$\bar{Y}^{*}\overset{d}{=}(b-a)\tilde{\bar{Y}}^{*}+a$ and
$S^{2*}\overset{d}{=}(b-a)^{2}\tilde{S}^{2*}$.

###### Proof.

To begin, note that we may relate the sample means on the two scales as
follows

$\displaystyle\bar{Y}$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}Y_{i}=\frac{1}{n}\sum_{i=1}^{n}\left[(b-a)\tilde{Y}_{i}+a\right]=(b-a)\frac{1}{n}\sum_{i=1}^{n}\tilde{Y}_{i}+a=(b-a)\tilde{\bar{Y}}+a.$
(7)

Similarly, we may relate the sample variances on the two scales as follows.

$\displaystyle S^{2}$
$\displaystyle=\frac{1}{n-1}\sum_{i=1}^{n}(Y_{i}-\bar{Y})^{2}$ (8)
$\displaystyle=\frac{1}{n-1}\sum_{i=1}^{n}([(b-a)\tilde{Y}_{i}+a]-[(b-a)\tilde{\bar{Y}}+a])^{2}$
(9)
$\displaystyle=(b-a)^{2}\frac{1}{n-1}\sum_{i=1}^{n}(\tilde{Y}_{i}-\tilde{\bar{Y}})^{2}$
(10) $\displaystyle=(b-a)^{2}\tilde{S}^{2}.$ (11)

We now consider the sensitivities of each of the four statistics. By Lemmas 11
and 12 in [11], the sensitivity of $\bar{Y}$ is $(b-a)/n$ and the sensitivity
of $S^{2}$ is $(b-a)^{2}/n$. Similarly, the sensitivity of $\tilde{\bar{Y}}$
is $1/n$ and the sensitivity of $\tilde{S}^{2}$ is $1/n$. Thus, when released
under $\varepsilon$-DP via the Laplace Mechanism, the released statistics have
distribution

$\displaystyle\bar{Y}^{*}$
$\displaystyle\sim\textsf{Lap}\left(\bar{Y},\frac{b-a}{\varepsilon
n}\right),\qquad
S^{2*}\sim\textsf{Lap}\left(S^{2},\frac{(b-a)^{2}}{\varepsilon n}\right)$ (12)
$\displaystyle\tilde{\bar{Y}}^{*}$
$\displaystyle\sim\textsf{Lap}\left(\tilde{\bar{Y}},\frac{1}{\varepsilon
n}\right),\qquad\hskip
13.00806pt\tilde{S}^{2*}\sim\textsf{Lap}\left(\tilde{S^{2}},\frac{1}{\varepsilon
n}\right).$ (13)

Recall that if $X$ has a Laplace distribution with location parameter $m$ and
scale parameter $b$, i.e., $X\sim\textsf{Lap}(m,b)$, then if $k>0$ and
$c\in{\mathbb{R}}$, it follows that $kX+c\sim\textsf{Lap}(km+c,kb)$. By this
property, $(b-a)\tilde{\bar{Y}}^{*}+a$ has location parameter
$(b-a)\tilde{\bar{Y}}+a=\bar{Y}$ and scale parameter $(b-a)/(\varepsilon n)$.
Similarly, $(b-a)^{2}\tilde{S}^{2*}$ has location parameter
$(b-a)^{2}\tilde{S^{2}}=S^{2}$ and scale parameter $(b-a)^{2}/(\varepsilon
n)$. The result follows. ∎

By this result, it is equivalent to release $\bar{Y}^{*}$ and $S^{2*}$ on the
$[a,b]$ scale directly via the Laplace distribution or to re-scale the data to
the $[0,1]$ scale, release $\tilde{\bar{Y}}^{*}$ and $\tilde{S}^{2*}$, and
then use the relationships in Theorem 1 to convert back to the $[a,b]$ scale.

##  Appendix B The Truncated Gamma Mixture Distribution

This section provides additional details related to the truncated gamma
mixture distribution (abbreviated TGM). The TGM distribution has four
parameters: $\alpha$ is analogous to the gamma distribution’s scale parameter,
$\beta$ is analogous to the gamma distribution’s rate parameter, $\lambda$
determines how far the TGM distribution is from a gamma distribution, and
$\tau$ determines the point of truncation. When $\tau>0$, the distribution is
a mixture of $\textsf{Gam}_{(0,\tau]}(\alpha,\beta-\lambda)$ and
$\textsf{Gam}_{(\tau,\infty)}(\alpha,\beta+\lambda)$ with the mixture weights
in (4). When $\tau\leq 0$, the distribution is equivalent to
$\textsf{Gam}(\alpha,\beta+\lambda)$.

Figure 6 compares the probability density of the TGM distribution to the
probability density of the gamma distribution with the same $\alpha$ and
$\beta$. We see that the shapes are similar, but the rates are different on
each side of $\tau$. Notably, the distribution is continuous but is clearly
not differentiable at $\tau$.

Figure 6: Comparison of the probability density functions of
$\textsf{TGM}(\alpha=2,\beta=2,\lambda=1,\tau=1)$ (the solid line) and
$\textsf{Gam}(\alpha=2,\beta=2)$ (the dashed line).

Algorithm 1 provides a straightforward procedure for sampling from the TGM
distribution. It assumes one is able to sample from a truncated gamma
distribution, which can be done via standard tools such as the rtrunc function
from the truncdist package in R [33].

Input: $\alpha$, $\beta$, $\lambda$, $\tau$

1 if  _$\tau\leq 0$_ then

2 Sample $X\sim\textsf{Gam}(\alpha,\beta+\lambda)$

3 else

4 Compute $\pi_{1}$ via (4)

5 Sample $U\sim\textsf{Unif}(0,1)$

6 if  _$U\leq\pi_{1}$_ then

7 Sample $X\sim\textsf{Gam}(\alpha,\beta-\lambda)$ truncated to $(0,\tau]$

8

9 else

10 Sample $X\sim\textsf{Gam}(\alpha,\beta+\lambda)$ truncated to
$(\tau,\infty)$

11

Output: $X$

Algorithm 1 Sample $X\sim\textsf{TGM}(\alpha,\beta,\lambda,\tau)$

We now prove that Algorithm 1 correctly samples from the TGM distribution.

######  Theorem 6.

$X$ sampled via Algorithm 1 has distribution
$X\sim\textsf{TGM}(\alpha,\beta,\lambda,\tau)$.

###### Proof.

If $\tau\leq 0$, then $X\sim\textsf{Gam}(\alpha,\beta+\lambda)$ and so has PDF

$\displaystyle
p(x)=\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha)}x^{\alpha-1}e^{-(\beta+\lambda)x}.$
(14)

Thus, the distribution of $X$ is as desired for $\tau\leq 0$.

We now consider the case where $\tau>0$. Let $F_{1}(x)$ and $f_{1}(x)$ be the
CDF and PDF, respectively, of $\textsf{Gam}(\alpha,\beta-\lambda)$ truncated
to $(0,\tau]$. Let $F_{2}(x)$ and $f_{2}(x)$ be the CDF and PDF, respectively,
of $\textsf{Gam}(\alpha,\beta+\lambda)$ truncated to $(\tau,\infty)$. By the
law of total probability, the CDF of $X$ is

$\displaystyle P[X\leq x]$ $\displaystyle=P[X\leq x\mid
U\leq\pi_{1}]\,P[U\leq\pi_{1}]+P[X\leq x\mid U>\pi_{1}]\,P[U>\pi_{1}]$ (15)
$\displaystyle=\pi_{1}\,F_{1}(x)+\pi_{2}\,F_{2}(x).$ (16)

This function is differentiable everywhere except $x=\tau$. When $x\neq\tau$,
we may take the derivative with respect to $x$ to obtain the PDF

$\displaystyle p(x)$
$\displaystyle=\frac{\textrm{d}}{\textrm{d}x}\left[\pi_{1}\,F_{1}(x)+\pi_{2}\,F_{2}(x)\right]$
(17) $\displaystyle=\pi_{1}f_{1}(x)+\pi_{2}f_{2}(x)$ (18)
$\displaystyle=\pi_{1}\frac{(\beta-\lambda)^{\alpha}}{\gamma(\alpha,(\beta-\lambda)\tau)}x^{\alpha-1}e^{-(\beta-\lambda)x}{\mathbf{1}}[x\leq\tau]+\pi_{2}\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha,(\beta+\lambda)\tau)}x^{\alpha-1}e^{-(\beta+\lambda)x}{\mathbf{1}}[x>\tau].$
(19)

Thus, we have shown that $X\sim\textsf{TGM}(\alpha,\beta,\lambda,\tau)$ for
all $X\neq\tau$. Since the event $X=\tau$ occurs with probability zero, this
completes the proof. ∎

The TGM distribution primarily arises in the following setting.

######  Theorem 7.

If $\tau\mid\mu\sim\textsf{Lap}(\mu,1/\lambda)$ and
$\mu\sim\textsf{Gam}(\alpha,\beta)$ for $\beta>\lambda$, then
$\mu\mid\tau\sim\textsf{TGM}(\alpha,\beta,\lambda,\tau)$.

###### Proof.

By Bayes’ Theorem, the desired distribution is

$\displaystyle
p(\mu\mid\tau)=\frac{p(\tau\mid\mu)\,p(\mu)}{\int_{0}^{\infty}p(\tau\mid\mu)\,p(\mu)\,\textrm{d}\mu}.$
(20)

We begin by investigating the numerator of (20), which has the following form.

$\displaystyle p(\tau\mid\mu)\,p(\mu)$
$\displaystyle=\frac{\lambda}{2}e^{-\lambda|\tau-\mu|}\,\frac{\beta^{\alpha}}{\Gamma(\alpha)}\mu^{\alpha-1}e^{-\beta\mu}$
(21)
$\displaystyle=\frac{\lambda\beta^{\alpha}}{2\Gamma(\alpha)}\mu^{\alpha-1}\left[e^{-\lambda\tau}e^{-(\beta-\lambda)\mu}{\mathbf{1}}[\mu\leq\tau]+e^{\lambda\tau}e^{-(\beta+\lambda)\mu}{\mathbf{1}}[\mu>\tau]\right]$
(22)
$\displaystyle=\frac{\lambda\beta^{\alpha}}{2\Gamma(\alpha)}\left[e^{-\lambda\tau}\,\mu^{\alpha-1}e^{-(\beta-\lambda)\mu}{\mathbf{1}}[\mu\leq\tau]+e^{\lambda\tau}\,\mu^{\alpha-1}e^{-(\beta+\lambda)\mu}{\mathbf{1}}[\mu>\tau]\right]$
(23)
$\displaystyle=\frac{\lambda\beta^{\alpha}}{2\Gamma(\alpha)}\bigg{[}e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}\,\frac{(\beta-\lambda)^{\alpha}}{\gamma(\alpha,(\beta-\lambda)\tau)}\mu^{\alpha-1}e^{-(\beta-\lambda)\mu}{\mathbf{1}}[\mu\leq\tau]$
$\displaystyle\qquad\qquad\quad+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}\,\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha,(\beta+\lambda)\tau)}\mu^{\alpha-1}e^{-(\beta+\lambda)\mu}{\mathbf{1}}[\mu>\tau]\bigg{]}.$
(24)

The denominator of (20) is then as follows.

$\displaystyle\int_{0}^{\infty}p(\tau\mid\mu)\,p(\mu)\,\textrm{d}\mu$
$\displaystyle=\frac{\lambda\beta^{\alpha}}{2\Gamma(\alpha)}\bigg{[}e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}\int_{0}^{\tau}\frac{(\beta-\lambda)^{\alpha}}{\gamma(\alpha,(\beta-\lambda)\tau)}\mu^{\alpha-1}e^{-(\beta-\lambda)\mu}\,\textrm{d}\mu$
$\displaystyle\qquad\qquad\quad+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}\int_{\tau}^{\infty}\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha,(\beta+\lambda)\tau)}\mu^{\alpha-1}e^{-(\beta+\lambda)\mu}\,\textrm{d}\mu\bigg{]}$
(25)
$\displaystyle=\frac{\lambda\beta^{\alpha}}{2\Gamma(\alpha)}\bigg{[}e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}\bigg{]}.$
(26)

The second equality follows by recognizing the integrands as the PDFs of
truncated gamma distributions. Thus, the distribution of $\mu\mid\tau$ is

$\displaystyle p(\mu\mid\tau)$
$\displaystyle=\frac{e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}}{e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}}\frac{(\beta-\lambda)^{\alpha}}{\gamma(\alpha,(\beta-\lambda)\tau)}\mu^{\alpha-1}e^{-(\beta-\lambda)\mu}{\mathbf{1}}[\mu\leq\tau]$
$\displaystyle\qquad+\frac{e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}}{e^{-\lambda\tau}\frac{\gamma(\alpha,(\beta-\lambda)\tau)}{(\beta-\lambda)^{\alpha}}+e^{\lambda\tau}\frac{\Gamma(\alpha,(\beta+\lambda)\tau)}{(\beta+\lambda)^{\alpha}}}\frac{(\beta+\lambda)^{\alpha}}{\Gamma(\alpha,(\beta+\lambda)\tau)}\mu^{\alpha-1}e^{-(\beta+\lambda)\mu}{\mathbf{1}}[\mu>\tau],$
(27)

which we recognize as the PDF of a $\textsf{TGM}(\alpha,\beta,\lambda,\tau)$.
∎

##  Appendix C Derivation of Full Conditionals for Univariate Gaussian Gibbs
Sampler

In this section, we derive the full conditionals for the Gibbs sampler in
Section 2.2. Recall that if
$Y_{1},\ldots,Y_{n}\overset{iid}{\sim}{\mathcal{N}}(\mu,\sigma^{2})$, then the
distributions of the sufficient statistics are

$\displaystyle\bar{Y}\mid\mu,\sigma^{2}\sim{\mathcal{N}}\left(\mu,\frac{\sigma^{2}}{n}\right),\qquad
S^{2}\mid\sigma^{2}\sim\textsf{Gam}\left(\frac{n-1}{2},\frac{n-1}{2\sigma^{2}}\right),$
(28)

where $(\bar{Y}\perp\\!\\!\\!\perp S^{2})\mid\mu,\sigma^{2}$. Assuming that
the $Y_{i}\in[0,1]$, then the statistics released via the Laplace Mechanism
have distributions

$\displaystyle\bar{Y}^{*}\mid\bar{Y}\sim\textsf{Lap}\left(\bar{Y},\frac{1}{\varepsilon_{1}n}\right),\qquad
S^{2*}\mid
S^{2}\sim\textsf{Lap}\left(S^{2},\frac{1}{\varepsilon_{2}n}\right).$ (29)

The release mechanism is such that $(\bar{Y}^{*}\perp\\!\\!\\!\perp
S^{2*})\mid\bar{Y},S^{2}$. It is convenient to replace the above distribution
of $\bar{Y}^{*}$ with the following equivalent formulation, as used in [4, 5,
34].

$\displaystyle\bar{Y}^{*}\mid\bar{Y},\omega^{2}\sim{\mathcal{N}}\left(\bar{Y},\omega^{2}\right),\qquad\omega^{2}\sim\textsf{Exp}\left(\frac{\varepsilon_{1}^{2}n^{2}}{2}\right).$
(30)

Finally, we take the conjugate prior from the public setting

$\displaystyle\sigma^{2}\sim\textsf{IG}\left(\frac{\nu_{0}}{2},\frac{\nu_{0}\sigma_{0}^{2}}{2}\right),\qquad\mu\mid\sigma^{2}\sim{\mathcal{N}}\left(\mu_{0},\frac{\sigma^{2}}{\kappa_{0}}\right).$
(31)

${\mu}$${\sigma^{2}}$Parameters${\omega^{2}}$${\bar{Y}}$${S^{2}}$Augmented
Variables${\bar{Y}^{*}}$${S^{2*}}$Released DP Statistics Figure 7: A graphical
model representing the structure of the univariate Gaussian setting.

Figure 7 presents the graphical model corresponding to this likelihood, which
we can factor as follows.

$\displaystyle
p(\bar{Y}^{*},S^{2*},\bar{Y},S^{2},\omega^{2}\mid\mu,\sigma^{2})$
$\displaystyle=p(\bar{Y}^{*}\mid\bar{Y},\omega^{2})\,p(S^{2*}\mid
S^{2})\,p(\bar{Y}\mid\mu,\sigma^{2})\,p(S^{2}\mid\sigma^{2})\,p(\omega^{2})$
(32)
$\displaystyle={\mathcal{N}}(\bar{Y}^{*};\bar{Y},\omega^{2})\,\textsf{Lap}\left(S^{2*};S^{2},\frac{1}{\varepsilon_{2}n}\right)\,{\mathcal{N}}\left(\bar{Y};\mu,\frac{\sigma^{2}}{n}\right)$
$\displaystyle\qquad\qquad\textsf{Gam}\left(S^{2};\frac{n-1}{2},\frac{n-1}{2\sigma^{2}}\right)\,\textsf{Exp}\left(\omega^{2};\frac{\varepsilon_{1}^{2}n^{2}}{2}\right)$
(33)

We now examine each full conditional. The full conditional for $\mu$ is

$\displaystyle
p(\mu\mid\bar{Y}^{*},S^{2*},\bar{Y},S^{2},\omega^{2},\sigma^{2})$
$\displaystyle\propto
p(\bar{Y}\mid\mu,\sigma^{2})\,p(\mu\mid\sigma^{2})={\mathcal{N}}\left(\bar{Y};\mu,\frac{\sigma^{2}}{n}\right)\,{\mathcal{N}}\left(\mu;\mu_{0},\frac{\sigma^{2}}{\kappa_{0}}\right),$
(34)

which we recognize as the full conditional from the public setting, as derived
in Chapter 5 of [17]. Similarly, the full conditional for $\sigma^{2}$ is

$\displaystyle
p(\sigma^{2}\mid\bar{Y}^{*},S^{2*},\bar{Y},S^{2},\omega^{2},\mu)$
$\displaystyle\propto
p(\bar{Y}\mid\mu,\sigma^{2})\,p(S^{2}\mid\sigma^{2})\,p(\sigma^{2})$ (35)
$\displaystyle={\mathcal{N}}\left(\bar{Y};\mu,\frac{\sigma^{2}}{n}\right)\,\textsf{Gam}\left(S^{2};\frac{n-1}{2},\frac{n-1}{2\sigma^{2}}\right)\,\textsf{IG}\left(\frac{\nu_{0}}{2},\frac{\nu_{0}\sigma_{0}^{2}}{2}\right).$
(36)

This is the same full conditional as in the public setting, as derived in
Chapter 6 of [17]. Thus, the full conditionals have the form

$\displaystyle\mu\mid\sigma^{2},\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}$
$\displaystyle\sim{\mathcal{N}}\left(\frac{n\bar{Y}+\kappa_{0}\mu_{0}}{n+\kappa_{0}},\frac{\sigma^{2}}{n+\kappa_{0}}\right)$
(37)
$\displaystyle\sigma^{2}\mid\mu,\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}$
$\displaystyle\sim\textsf{IG}\left(\frac{n+\nu_{0}}{2},\frac{\nu_{0}\sigma_{0}^{2}+(n-1)S^{2}+n(\bar{Y}-\mu)^{2}}{2}\right).$
(38)

We next examine the full conditional for $\bar{Y}$, which is

$\displaystyle
p(\bar{Y}\mid\bar{Y}^{*},S^{2*},S^{2},\omega^{2},\mu,\sigma^{2})$
$\displaystyle\propto
p(\bar{Y}^{*}\mid\bar{Y},\omega^{2})\,p(\bar{Y}\mid\mu,\sigma^{2})={\mathcal{N}}(\bar{Y}^{*};\bar{Y},\omega^{2})\,{\mathcal{N}}\left(\bar{Y};\mu,\frac{\sigma^{2}}{n}\right).$
(39)

We recognize this is equivalent to a Gaussian model with known variance and a
Gaussian prior on the mean. By Chapter 5 of [17], the full conditional for
$\bar{Y}$ is

$\displaystyle\bar{Y}\mid\mu,\sigma^{2},\omega^{2},S^{2},\bar{Y}^{*},S^{2*}$
$\displaystyle\sim{\mathcal{N}}\left(\frac{\frac{\bar{Y}^{*}}{\omega^{2}}+\frac{n\mu}{\sigma^{2}}}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}},\frac{1}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}}\right).$
(40)

The full conditional for $\omega^{2}$ is as follows, expressed in terms of its
inverse, $1/\omega^{2}$.

$\displaystyle
p(1/\omega^{2}\mid\bar{Y}^{*},S^{2*},\bar{Y},S^{2},\mu,\sigma^{2})$
$\displaystyle\propto p(\bar{Y}^{*}\mid\bar{Y},1/\omega^{2})\,p(1/\omega^{2})$
(41)
$\displaystyle={\mathcal{N}}(\bar{Y}^{*};\bar{Y},(1/\omega^{2})^{-1})\,\textsf{IG}\left(1/\omega^{2};1,\frac{\varepsilon_{1}^{2}n^{2}}{2}\right).$
(42)

We recognize this form from the Bayesian LASSO [34], which yields the
distribution

$\displaystyle
1/\omega^{2}\mid\mu,\sigma^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}$
$\displaystyle\sim\textsf{InvGaus}\left(\frac{\varepsilon_{1}n}{|\bar{Y}^{*}-\bar{Y}|},\varepsilon_{1}^{2}n^{2}\right).$
(43)

Finally, the full conditional for $S^{2}$ is

$\displaystyle
p(S^{2}\mid\bar{Y}^{*},S^{2*},\bar{Y},\omega^{2},\mu,\sigma^{2})$
$\displaystyle\propto p(S^{2*}\mid S^{2})\,p(S^{2}\mid\sigma^{2})$ (44)
$\displaystyle=\textsf{Lap}\left(S^{2*};S^{2},\frac{1}{\varepsilon_{2}n}\right)\,\textsf{Gam}\left(S^{2};\frac{n-1}{2},\frac{n-1}{2\sigma^{2}}\right).$
(45)

By Theorem 7 when $(n-1)/(2\sigma^{2})>\varepsilon_{2}n$, the distribution is

$\displaystyle S^{2}\mid\mu,\sigma^{2},\omega^{2},\bar{Y},\bar{Y}^{*},S^{2*}$
$\displaystyle\sim\textsf{TGM}\left(\frac{n-1}{2},\frac{n-1}{2\sigma^{2}},n\varepsilon_{2},S^{2*}\right).$
(46)

Note that by Corollary 1 of the main text, which is repeated in Section E
below, since $Y_{i}\in[0,1]$ it follows that $\sigma^{2}\leq 1/4$. Thus, we
may use this full conditional when

$\displaystyle\varepsilon_{2}<\frac{n-1}{n}\cdot\frac{1}{2\sigma^{2}}\leq
2\,\frac{n-1}{n}.$ (47)

If $\varepsilon_{2}>2(n-1)/n$, then the full conditional for $S^{2}$ is not
necessarily a TGM without additional assumptions. If reasonable for the
application, the analyst could impose the additional constraint that
$\sigma^{2}<(n-1)/(n\varepsilon_{2})$, which is sufficient to ensure the full
conditional for $S^{2}$ is of the form in (46). Alternatively, if $n$ is
large, an analyst could use an approximation of the full conditional for
$S^{2}$, such as the proposals of [4, 5].

##  Appendix D List of Full Conditionals

In this section, we summarize the full conditional under different prior
distributions for the DP univariate Gaussian setting with likelihood given by
$Y_{i}\overset{iid}{\sim}{\mathcal{N}}_{[0,1]}(\mu,\sigma^{2})$,
$\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},1/(\varepsilon_{1}n))$, and
$S^{2*}\sim\textsf{Lap}(S^{2},1/(\varepsilon_{2}n))$. Subscripts on a
distribution denote truncation of the distribution to an interval.

If the prior is
$\sigma^{2}\sim\textsf{IG}\left(\nu_{0}/2,\nu_{0}\sigma_{0}^{2}/2\right)$ and
$\mu\mid\sigma^{2}\sim{\mathcal{N}}\left(\mu_{0},\sigma^{2}/\kappa_{0}\right)$
with no constraints enforced, then

$\displaystyle\mu\mid\sigma^{2},\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}\left(\frac{n\bar{Y}+\kappa_{0}\mu_{0}}{n+\kappa_{0}},\frac{\sigma^{2}}{n+\kappa_{0}}\right)$
(48)
$\displaystyle\sigma^{2}\mid\mu,\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{IG}_{\left(0,\frac{n-1}{2n\varepsilon_{2}}\right)}\left(\frac{n+\nu_{0}}{2},\frac{\nu_{0}\sigma_{0}^{2}+(n-1)S^{2}+n(\bar{Y}-\mu)^{2}}{2}\right)$
(49)
$\displaystyle\bar{Y}\mid\mu,\sigma^{2},\omega^{2},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}\left(\frac{\frac{\bar{Y}^{*}}{\omega^{2}}+\frac{n\mu}{\sigma^{2}}}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}},\frac{1}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}}\right)$
(50) $\displaystyle
1/\omega^{2}\mid\mu,\sigma^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{InvGaus}\left(\frac{\varepsilon_{1}n}{|\bar{Y}^{*}-\bar{Y}|},\varepsilon_{1}^{2}n^{2}\right)$
(51) $\displaystyle
S^{2}\mid\mu,\sigma^{2},\omega^{2},\bar{Y},\bar{Y}^{*},S^{2*}\sim\textsf{TGM}\left(\frac{n-1}{2},\frac{n-1}{2\sigma^{2}},n\varepsilon_{2},S^{2*}\right).$
(52)

If the prior is
$\sigma^{2}\sim\textsf{IG}\left(\nu_{0}/2,\nu_{0}\sigma_{0}^{2}/2\right)$ and
$\mu\mid\sigma^{2}\sim{\mathcal{N}}\left(\mu_{0},\sigma^{2}/\kappa_{0}\right)$
with constraints enforced, then

$\displaystyle\mu\mid\sigma^{2},\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}_{\left[1/2-\sqrt{1/4-\sigma^{2}},1/2+\sqrt{1/4-\sigma^{2}}\right]}\left(\frac{n\bar{Y}+\kappa_{0}\mu_{0}}{n+\kappa_{0}},\frac{\sigma^{2}}{n+\kappa_{0}}\right)$
(53)
$\displaystyle\sigma^{2}\mid\mu,\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{IG}_{[0,\min\\{\mu(1-\mu),\frac{n-1}{2n\varepsilon_{2}})}\left(\frac{n+\nu_{0}}{2},\frac{\nu_{0}\sigma_{0}^{2}+(n-1)S^{2}+n(\bar{Y}-\mu)^{2}}{2}\right)$
(54)
$\displaystyle\bar{Y}\mid\mu,\sigma^{2},\omega^{2},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}_{\left[1/2-\sqrt{1/4-\frac{n-1}{n}S^{2}},1/2+\sqrt{1/4-\frac{n-1}{n}S^{2}}\right]}\left(\frac{\frac{\bar{Y}^{*}}{\omega^{2}}+\frac{n\mu}{\sigma^{2}}}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}},\frac{1}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}}\right)$
(55) $\displaystyle
1/\omega^{2}\mid\mu,\sigma^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{InvGaus}\left(\frac{\varepsilon_{1}n}{|\bar{Y}^{*}-\bar{Y}|},\varepsilon_{1}^{2}n^{2}\right)$
(56) $\displaystyle
S^{2}\mid\mu,\sigma^{2},\omega^{2},\bar{Y},\bar{Y}^{*},S^{2*}\sim\textsf{TGM}_{(0,\frac{n}{n-1}\bar{Y}(1-\bar{Y})]}\left(\frac{n-1}{2},\frac{n-1}{2\sigma^{2}},n\varepsilon_{2},S^{2*}\right).$
(57)

If the prior is $p(\mu,\sigma^{2})\propto 1$ with no constraints enforced,
then

$\displaystyle\mu\mid\sigma^{2},\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}\left(\bar{Y},\frac{\sigma^{2}}{n}\right)$
(58)
$\displaystyle\sigma^{2}\mid\mu,\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{IG}_{\left(0,\frac{n-1}{2n\varepsilon_{2}}\right)}\left(\frac{n-2}{2},\frac{(n-1)S^{2}+n(\bar{Y}-\mu)^{2}}{2}\right)$
(59)
$\displaystyle\bar{Y}\mid\mu,\sigma^{2},\omega^{2},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}\left(\frac{\frac{\bar{Y}^{*}}{\omega^{2}}+\frac{n\mu}{\sigma^{2}}}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}},\frac{1}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}}\right)$
(60) $\displaystyle
1/\omega^{2}\mid\mu,\sigma^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{InvGaus}\left(\frac{\varepsilon_{1}n}{|\bar{Y}^{*}-\bar{Y}|},\varepsilon_{1}^{2}n^{2}\right)$
(61) $\displaystyle
S^{2}\mid\mu,\sigma^{2},\omega^{2},\bar{Y},\bar{Y}^{*},S^{2*}\sim\textsf{TGM}\left(\frac{n-1}{2},\frac{n-1}{2\sigma^{2}},n\varepsilon_{2},S^{2*}\right).$
(62)

If the prior is $p(\mu,\sigma^{2})\propto 1$ with constraints enforced, then

$\displaystyle\mu\mid\sigma^{2},\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}_{\left[1/2-\sqrt{1/4-\sigma^{2}},1/2+\sqrt{1/4-\sigma^{2}}\right]}\left(\bar{Y},\frac{\sigma^{2}}{n}\right)$
(63)
$\displaystyle\sigma^{2}\mid\mu,\omega^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{IG}_{[0,\min\\{\mu(1-\mu),\frac{n-1}{2n\varepsilon_{2}})}\left(\frac{n-2}{2},\frac{(n-1)S^{2}+n(\bar{Y}-\mu)^{2}}{2}\right)$
(64)
$\displaystyle\bar{Y}\mid\mu,\sigma^{2},\omega^{2},S^{2},\bar{Y}^{*},S^{2*}\sim{\mathcal{N}}_{\left[1/2-\sqrt{1/4-\frac{n-1}{n}S^{2}},1/2+\sqrt{1/4-\frac{n-1}{n}S^{2}}\right]}\left(\frac{\frac{\bar{Y}^{*}}{\omega^{2}}+\frac{n\mu}{\sigma^{2}}}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}},\frac{1}{\frac{1}{\omega^{2}}+\frac{n}{\sigma^{2}}}\right)$
(65) $\displaystyle
1/\omega^{2}\mid\mu,\sigma^{2},\bar{Y},S^{2},\bar{Y}^{*},S^{2*}\sim\textsf{InvGaus}\left(\frac{\varepsilon_{1}n}{|\bar{Y}^{*}-\bar{Y}|},\varepsilon_{1}^{2}n^{2}\right)$
(66) $\displaystyle
S^{2}\mid\mu,\sigma^{2},\omega^{2},\bar{Y},\bar{Y}^{*},S^{2*}\sim\textsf{TGM}_{(0,\frac{n}{n-1}\bar{Y}(1-\bar{Y})]}\left(\frac{n-1}{2},\frac{n-1}{2\sigma^{2}},n\varepsilon_{2},S^{2*}\right).$
(67)

##  Appendix E Constraints for the Univariate Gaussian Setting

In this section, we provide proofs for results in Section 3 of the main text
regarding constraints on parameters and statistics in the univariate Gaussian
setting. The result regarding constraints on the parameters $\mu$ and
$\sigma^{2}$ is restated below.

######  Theorem 2.

Let $Y_{1},\ldots,Y_{n}\sim{\mathcal{N}}(\mu,\sigma^{2})$ and suppose each
$Y_{i}\in[0,1]$. If $\mu$ is known, then $\sigma^{2}\in[0,\mu(1-\mu)]$. If
$\sigma^{2}$ is known, then
$\mu\in[1/2-\sqrt{1/4-\sigma^{2}},1/2+\sqrt{1/4-\sigma^{2}}]$.

###### Proof.

Since $Y_{i}\in[0,1]$, it follows that $Y_{i}^{2}\leq Y_{i}$. and so by
monotonicity of expectation,

$\displaystyle\sigma^{2}=Var[Y_{i}]=E[Y_{i}^{2}]-E[Y_{i}]^{2}\leq
E[Y_{i}]-E[Y_{i}]^{2}=\mu-\mu^{2}.$ (68)

Thus, since $Y_{i}$ must have non-negative variance, if $\mu$ is known then
$\sigma^{2}\in[0,\mu(1-\mu)]$. On the other hand, if $\sigma^{2}$ is known
then by (68), $\mu$ must be such that $\mu^{2}-\mu+\sigma^{2}\leq 0$. Applying
the quadratic formula, we find that $\mu^{2}-\mu+\sigma^{2}$ has roots
$\mu=1/2\pm\sqrt{1/4-\sigma^{2}}$. Recognizing that
$\mu^{2}-\mu+\sigma^{2}\leq 0$ when $\mu$ is between these roots yields the
desired result. ∎

The corollary of Theorem 2 is restated below.

######  Corollary 1.

If $Y_{1},\ldots,Y_{n}\sim{\mathcal{N}}(\mu,\sigma^{2})$ and $Y_{i}\in[0,1]$,
then $\sigma^{2}\leq 1/4$.

###### Proof.

By Theorem 2 $\sigma^{2}\leq\mu(1-\mu)$ and since $f(\mu)=\mu(1-\mu)$ attains
its maximum value at $\mu=1/2$, it follows that $f(\mu)\leq f(1/2)=1/4$ for
all $\mu$. The result follows. ∎

The result regarding constraints on the sufficient statistics $\bar{Y}$ and
$S^{2}$ is restated below.

######  Theorem 3.

Let $Y_{1},\ldots,Y_{n}\sim{\mathcal{N}}(\mu,\sigma^{2})$ and suppose each
$Y_{i}\in[0,1]$. If $\bar{Y}$ is known, then
$S^{2}\in[0,n/(n-1)\bar{Y}(1-\bar{Y})]$. If $S^{2}$ is known, then
$\bar{Y}\in[1/2-\sqrt{1/4-(n-1)/n\cdot S^{2}},1/2+\sqrt{1/4-(n-1)/n\cdot
S^{2}}]$.

###### Proof.

We begin by expressing $S^{2}$ in a more convenient form, as follows.

$\displaystyle(n-1)S^{2}=\sum_{i=1}^{n}(Y_{i}-\bar{Y})^{2}=\sum_{i=1}^{n}Y_{i}^{2}+n\bar{Y}^{2}-2\bar{Y}\sum_{i=1}^{n}Y_{i}=\sum_{i=1}^{n}Y_{i}^{2}-n\bar{Y}^{2}.$
(69)

Since $Y_{i}\in[0,1]$, we have that $Y_{i}\leq Y_{i}^{2}$ and so

$\displaystyle(n-1)S^{2}=\sum_{i=1}^{n}Y_{i}^{2}-n\bar{Y}^{2}\leq\sum_{i=1}^{n}Y_{i}-n\bar{Y}^{2}=n\bar{Y}(1-\bar{Y}).$
(70)

Since $(n-1)S^{2}$ is a sum of squares, it is non-negative and so if $\bar{Y}$
is known, $S^{2}\in[0,n/(n-1)\bar{Y}(1-\bar{Y})]$. On the other hand, if
$S^{2}$ is known then $\bar{Y}$ must be such that
$\bar{Y}^{2}-\bar{Y}+(n-1)/n\cdot S^{2}\leq 0$. By an analogous argument to
the proof of Theorem 2, it follows that $\bar{Y}\in[1/2-\sqrt{1/4-(n-1)/n\cdot
S^{2}},1/2+\sqrt{1/4-(n-1)/n\cdot S^{2}}]$. ∎

##  Appendix F Determining Whether the Posterior Distribution is Proper

In this section, we prove that the posterior distribution in the
differentially private univariate Gaussian setting is not a proper probability
distribution when $p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$, but is a proper
probability distribution when $p(\mu,\sigma^{2})\propto 1$. The results are
reproduced below.

######  Theorem 4.

For confidential data
$Y_{1},\ldots,Y_{n}\overset{iid}{\sim}{\mathcal{N}}(\mu,\sigma^{2})$ where
sufficient statistics
$\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},1/(\varepsilon_{1}n))$ and
$S^{2*}\sim\textsf{Lap}(S^{2},1/(\varepsilon_{2}n))$ are released, if an
analyst has prior $p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$, then their
posterior $p(\mu,\sigma^{2}\mid\bar{Y}^{*},S^{2*})$ is not a proper
probability distribution.

###### Proof.

To begin, note that under the prior
$p(\mu,\sigma^{2})\propto(\sigma^{2})^{-1}$, for observed
$\bar{Y}^{*}=\bar{y}^{*}$ and $S^{2*}=s^{2*}$ the posterior distribution, if
proper, should be

$\displaystyle p(\mu,\sigma^{2}\mid\bar{y}^{*},s^{2*})$
$\displaystyle=\frac{p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,p(\mu,\sigma^{2})}{\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,p(\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\sigma^{2}}$
(71)
$\displaystyle=\frac{p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,(\sigma^{2})^{-1}}{\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,(\sigma^{2})^{-1}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}}.$
(72)

The posterior is a proper probability distribution if the integral in the
denominator of (72) is finite. We now examine this integral. Introducing the
latent variables $\bar{Y}=\bar{y}$ and $S^{2}=s^{2}$ and exploiting the
conditional independence structure, this integral is equivalent to

$\displaystyle\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,(\sigma^{2})^{-1}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}$
(73)
$\displaystyle=\int_{0}^{\infty}\int_{-\infty}^{\infty}\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*},\bar{y},s^{2}\mid\mu,\sigma^{2})\,(\sigma^{2})^{-1}\,\textrm{d}\bar{y}\,\textrm{d}s^{2}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}$
(74)
$\displaystyle=\int_{0}^{\infty}(\sigma^{2})^{-1}\int_{-\infty}^{\infty}\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\,p(s^{2*}\mid
s^{2})\,p(\bar{y}\mid\mu,\sigma^{2})\,p(s^{2}\mid\sigma^{2})\,\,\textrm{d}\bar{y}\,\textrm{d}s^{2}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}$
(75)
$\displaystyle=\int_{0}^{\infty}(\sigma^{2})^{-1}\left[\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\,p(s^{2}\mid\sigma^{2})\,\textrm{d}s^{2}\right]\left[\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\,p(\bar{y}\mid\mu,\sigma^{2})\,\textrm{d}\bar{y}\,\textrm{d}\mu\right]\,\textrm{d}\sigma^{2}.$
(76)

We examine the double integral in the second bracket. If we exchange the order
of integration,

$\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\,p(\bar{y}\mid\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\bar{y}$
$\displaystyle=\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^{2}}}e^{-\frac{(\bar{y}-\mu)^{2}}{2\sigma^{2}}}\,\textrm{d}\mu\,\textrm{d}\bar{y},$
(77)

since we recognize the quantity under the inner integral as the density
function of ${\mathcal{N}}(\bar{y},\sigma^{2})$, the integral is equal to $1$.
Thus,

$\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\,p(\bar{y}\mid\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\bar{y}$
$\displaystyle=\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\,\textrm{d}\bar{y}=\int_{-\infty}^{\infty}\frac{\varepsilon_{1}n}{2}e^{-\varepsilon_{1}n|\bar{y}^{*}-\bar{y}|}\,\textrm{d}\bar{y}=1,$
(78)

since we recognize the quantity under the integral as the density function of
$\textsf{Lap}(\bar{y}^{*},1/(\varepsilon_{1}n))$. By Fubini’s Theorem, since
this integral is finite and the quantity being integrated is nonnegative, it
is equivalent to the quantity in brackets above. Thus, the quantity of
interest reduces to

$\displaystyle\int_{0}^{\infty}$
$\displaystyle\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,(\sigma^{2})^{-1}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}=\int_{0}^{\infty}(\sigma^{2})^{-1}\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\,p(s^{2}\mid\sigma^{2})\,\textrm{d}s^{2}\,\textrm{d}\sigma^{2}.$ (79)

If $s^{2*}>0$, then we bound the inner integral below as follows.

$\displaystyle\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\,p(s^{2}\mid\sigma^{2})\,\textrm{d}s^{2}$
$\displaystyle=\frac{\varepsilon_{2}n\left(\frac{n-1}{2\sigma^{2}}\right)^{\frac{n-1}{2}}}{2\,\Gamma(\frac{n-1}{2})}\int_{0}^{\infty}e^{-\varepsilon_{2}n|s^{2*}-s^{2}|}\,(s^{2})^{\frac{n-1}{2}-1}e^{-\frac{n-1}{2\sigma^{2}}s^{2}}\,\textrm{d}s^{2}$
(80)
$\displaystyle=\frac{\varepsilon_{2}n\left(\frac{n-1}{2\sigma^{2}}\right)^{\frac{n-1}{2}}}{2\,\Gamma(\frac{n-1}{2})}\Bigg{[}e^{-\varepsilon_{2}ns^{2*}}\int_{0}^{s^{2*}}(s^{2})^{\frac{n-1}{2}-1}e^{-(\frac{n-1}{2\sigma^{2}}-\varepsilon_{2}n)s^{2}}\,\textrm{d}s^{2}$
$\displaystyle\hskip
79.49744pt+e^{\varepsilon_{2}ns^{2*}}\int_{s^{2*}}^{\infty}(s^{2})^{\frac{n-1}{2}-1}e^{-(\frac{n-1}{2\sigma^{2}}+\varepsilon_{2}n)s^{2}}\,\textrm{d}s^{2}\Bigg{]}$
(81)
$\displaystyle\geq\frac{\varepsilon_{2}n\left(\frac{n-1}{2\sigma^{2}}\right)^{\frac{n-1}{2}}}{2\,\Gamma(\frac{n-1}{2})}\Bigg{[}e^{-\varepsilon_{2}ns^{2*}}\int_{0}^{s^{2*}}(s^{2})^{\frac{n-1}{2}-1}e^{-(\frac{n-1}{2\sigma^{2}}+\varepsilon_{2}n)s^{2}}\,\textrm{d}s^{2}$
$\displaystyle\hskip
79.49744pt+e^{-\varepsilon_{2}ns^{2*}}\int_{s^{2*}}^{\infty}(s^{2})^{\frac{n-1}{2}-1}e^{-(\frac{n-1}{2\sigma^{2}}+\varepsilon_{2}n)s^{2}}\,\textrm{d}s^{2}\Bigg{]}$
(82)
$\displaystyle=\frac{\varepsilon_{2}n\left(\frac{n-1}{2\sigma^{2}}\right)^{\frac{n-1}{2}}}{2\,\Gamma(\frac{n-1}{2})}e^{-\varepsilon_{2}ns^{2*}}\int_{0}^{\infty}(s^{2})^{\frac{n-1}{2}-1}e^{-(\frac{n-1}{2\sigma^{2}}+\varepsilon_{2}n)s^{2}}\,\textrm{d}s^{2}$
(83)
$\displaystyle=\frac{\varepsilon_{2}n}{2}\left(\frac{\frac{n-1}{2\sigma^{2}}}{\frac{n-1}{2\sigma^{2}}+\varepsilon_{2}n}\right)^{\frac{n-1}{2}}$
(84)
$\displaystyle=\frac{\varepsilon_{2}n}{2}\left(\frac{\frac{n-1}{2\varepsilon_{2}n}}{\frac{n-1}{2\varepsilon_{2}n}+\sigma^{2}}\right)^{\frac{n-1}{2}}.$
(85)

The inequality follows since $e^{x}\geq e^{-x}$ for $x>0$ and since
$e^{-(a-b)x}\geq e^{-(a+b)x}$ for $a>b>0$ and $x>0$. If $s^{2*}\leq 0$, then
the same bound holds by an analogous argument (with the integral from $0$ to
$s^{2*}$ omitted). Thus, the quantity of interest is bounded as follows.

$\displaystyle\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,(\sigma^{2})^{-1}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}$
$\displaystyle\geq\frac{\varepsilon_{2}n}{2}\int_{0}^{\infty}(\sigma^{2})^{-1}\left(\frac{\frac{n-1}{2\varepsilon_{2}n}}{\frac{n-1}{2\varepsilon_{2}n}+\sigma^{2}}\right)^{\frac{n-1}{2}}\,\textrm{d}\sigma^{2}$
(86)
$\displaystyle\geq\frac{\varepsilon_{2}n}{2}\int_{0}^{1}(\sigma^{2})^{-1}\left(\frac{\frac{n-1}{2\varepsilon_{2}n}}{\frac{n-1}{2\varepsilon_{2}n}+\sigma^{2}}\right)^{\frac{n-1}{2}}\,\textrm{d}\sigma^{2}$
(87)
$\displaystyle\geq\frac{\varepsilon_{2}n}{2}\int_{0}^{1}(\sigma^{2})^{-1}\left(\frac{\frac{n-1}{2\varepsilon_{2}n}}{\frac{n-1}{2\varepsilon_{2}n}+1}\right)^{\frac{n-1}{2}}\,\textrm{d}\sigma^{2}$
(88)
$\displaystyle=\frac{\varepsilon_{2}n}{2}\left(\frac{\frac{n-1}{2\varepsilon_{2}n}}{\frac{n-1}{2\varepsilon_{2}n}+1}\right)^{\frac{n-1}{2}}\int_{0}^{1}(\sigma^{2})^{-1}\,\textrm{d}\sigma^{2}.$
(89)

Since this integral diverges, the quantity of interest diverges. Thus, (72) is
not a proper probability distribution. ∎

######  Theorem 5.

For confidential data
$Y_{1},\ldots,Y_{n}\overset{iid}{\sim}{\mathcal{N}}(\mu,\sigma^{2})$ where
$n>3$ and sufficient statistics
$\bar{Y}^{*}\sim\textsf{Lap}(\bar{Y},1/(\varepsilon_{1}n))$ and
$S^{2*}\sim\textsf{Lap}(S^{2},1/(\varepsilon_{2}n))$ are released, if an
analyst has prior $p(\mu,\sigma^{2})\propto 1$, then their posterior
$p(\mu,\sigma^{2}\mid\bar{Y}^{*},S^{2*})$ is a proper probability
distribution.

###### Proof.

To begin, note that under the prior $p(\mu,\sigma^{2})\propto 1$, for observed
$\bar{Y}^{*}=\bar{y}^{*}$ and $S^{2*}=s^{2*}$ the posterior distribution, if
proper, should be

$\displaystyle p(\mu,\sigma^{2}\mid\bar{y}^{*},s^{2*})$
$\displaystyle=\frac{p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,p(\mu,\sigma^{2})}{\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,p(\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\sigma^{2}}$
(90)
$\displaystyle=\frac{p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})}{\int_{0}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\sigma^{2}}.$
(91)

The posterior is a proper probability distribution if the integral in the
denominator of (91) is finite. By analogy to the proof of Theorem 4, this
integral is equivalent to

$\displaystyle\int_{0}^{\infty}$
$\displaystyle\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\sigma^{2}$
(92) $\displaystyle=\int_{0}^{\infty}\left[\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\,p(s^{2}\mid\sigma^{2})\,\textrm{d}s^{2}\right]\left[\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}p(\bar{y}^{*}\mid\bar{y})\,p(\bar{y}\mid\mu,\sigma^{2})\,\textrm{d}\bar{y}\,\textrm{d}\mu\right]\,\textrm{d}\sigma^{2}.$
(93)

By (78, the double integral in the second bracket is equal to 1. Thus, the
quantity of interest reduces to

$\displaystyle\int_{0}^{\infty}$
$\displaystyle\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,\textrm{d}\mu\,\textrm{d}\sigma^{2}=\int_{0}^{\infty}\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\,p(s^{2}\mid\sigma^{2})\,\textrm{d}s^{2}\,\textrm{d}\sigma^{2}.$ (94)

Exchanging the order of integration yields

$\displaystyle\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\int_{0}^{\infty}\,p(s^{2}\mid\sigma^{2})\,\textrm{d}\sigma^{2}\,\textrm{d}s^{2}.$
(95)

Since $n>3$, the inner integral is the kernel of an inverse-gamma distribution
and so

$\displaystyle\int_{0}^{\infty}\,p(s^{2}\mid\sigma^{2})\,\textrm{d}\sigma^{2}$
$\displaystyle=\int_{0}^{\infty}\frac{(\frac{n-1}{2\sigma^{2}})^{\frac{n-1}{2}}}{\Gamma(\frac{n-1}{2})}(s^{2})^{\frac{n-1}{2}-1}e^{-\frac{(n-1)s^{2}}{2\sigma^{2}}}\,\textrm{d}\sigma^{2}$
(96)
$\displaystyle=\frac{(\frac{n-1}{2})^{\frac{n-1}{2}}}{\Gamma(\frac{n-1}{2})}(s^{2})^{\frac{n-1}{2}-1}\int_{0}^{\infty}(\sigma^{2})^{-\frac{n-3}{2}-1}e^{-\frac{(n-1)s^{2}}{2\sigma^{2}}}\,\textrm{d}\sigma^{2}$
(97)
$\displaystyle=\frac{(\frac{n-1}{2})^{\frac{n-1}{2}}}{\Gamma(\frac{n-1}{2})}(s^{2})^{\frac{n-1}{2}-1}\frac{\Gamma(\frac{n-3}{2})}{(\frac{(n-1)s^{2}}{2})^{\frac{n-3}{2}}}$
(98)
$\displaystyle=\frac{\frac{n-1}{2}\Gamma(\frac{n-3}{2})}{\Gamma(\frac{n-1}{2})}$
(99) $\displaystyle=\frac{n-1}{n-3}.$ (100)

Thus,

$\displaystyle\int_{0}^{\infty}p(s^{2*}\mid
s^{2})\int_{0}^{\infty}\,p(s^{2}\mid\sigma^{2})\,\textrm{d}\sigma^{2}\,\textrm{d}s^{2}$
$\displaystyle=\frac{n-1}{n-3}\int_{0}^{\infty}\frac{\varepsilon_{2}n}{2}e^{-\varepsilon_{2}n|s^{2*}-s^{2}|}\,\textrm{d}s^{2}$
(101)
$\displaystyle=\frac{n-1}{n-3}\left(\frac{1}{2}+\frac{1}{2}\textup{sgn}\left(s^{2*}\right)\left(1-e^{-\varepsilon_{2}n|s^{2*}|}\right)\right),$
(102)

since we recognize the quantity under the integral as the density function of
$\textsf{Lap}(s^{2*},1/(\varepsilon_{2}n))$, we plug in the form of its CDF at
zero. By Fubini’s Theorem, since this integral is finite and the quantity
being integrated is nonnegative, it is equivalent to the original quantity of
interest. Thus,

$\displaystyle\int_{0}^{\infty}$
$\displaystyle\int_{-\infty}^{\infty}p(\bar{y}^{*},s^{2*}\mid\mu,\sigma^{2})\,(\sigma^{2})^{0}\,\textrm{d}\mu\,\textrm{d}\sigma^{2}=\frac{n-1}{n-3}\left(\frac{1}{2}+\frac{1}{2}\textup{sgn}\left(s^{2*}\right)\left(1-e^{-\varepsilon_{2}n|s^{2*}|}\right)\right)<\infty$
(103)

and so the posterior is a proper probability distribution. ∎

##  Appendix G Nonprivate Laplace-Uniform Posterior Distribution

In this section, we demonstrate that if $X\sim\textsf{Lap}(\xi,1/\lambda)$ and
$\lambda$ is known, then a uniform prior on $\xi$ yields a proper, frequentist
matching posterior. The main result is as follows.

######  Theorem 8.

Let $X\sim\textsf{Lap}(\xi,1/\lambda)$, where $\lambda$ is known. Then the
prior $p(\xi)\propto 1$ yields a proper posterior distribution that is
frequentist matching.

###### Proof.

Applying properties of the Laplace distribution, if
$X\sim\textsf{Lap}(\xi,1/\lambda)$, then $\xi-X\sim\textsf{Lap}(0,1/\lambda)$.
Note that this is a pivotal quantity, since its distribution does not depend
on any unknown parameters. Let $\ell_{\alpha/2}$ and $\ell_{1-\alpha/2}$ be
the $\alpha/2$ and $(1-\alpha/2)$ quantiles of $\textsf{Lap}(\xi,1/\lambda)$.
Then,

$\displaystyle P[X+\ell_{\alpha/2}\leq\xi\leq
X+\ell_{1-\alpha/2}]=P[\ell_{\alpha/2}\leq\xi-X\leq\ell_{1-\alpha/2}]=1-\alpha$
(104)

and so $[X+\ell_{\alpha/2},X+\ell_{1-\alpha/2}]$ is a $(1-\alpha)$ confidence
interval for $\xi$.

In the Bayesian setting, if $p(\xi)\propto 1$ then

$\displaystyle p(\xi\mid X)\propto
p(X\mid\xi)\,p(\xi)\propto\exp\left\\{-\lambda|X-\xi|\right\\}=\exp\left\\{-\lambda|\xi-X|\right\\}.$

Thus, the posterior distribution is $(\xi\mid
X)\sim\textsf{Lap}(X,1/\lambda)$. Again applying properties of the Laplace
distribution, $(\xi-X\mid X)\sim\textsf{Lap}(0,1/\lambda)$. Thus,

$\displaystyle P[X+\ell_{\alpha/2}\leq\xi\leq X+\ell_{1-\alpha/2}\mid
X]=P[\ell_{\alpha/2}\leq\xi-X\leq\ell_{1-\alpha/2}\mid X]=1-\alpha$ (105)

and so $[X+\ell_{\alpha/2},X+\ell_{1-\alpha/2}]$ is a $(1-\alpha)$ credible
interval. Since this interval is the same as the confidence interval above,
this posterior distribution is frequentist matching. ∎

##  Appendix H Additional Simulation Studies

Figure 8: Root mean square error (RMSE) for estimating $\mu$ with the
posterior mode for different sample sizes for the simulation in Section 4 of
the main text. Results based on 10,000 simulated datasets $Y_{i}\in[0,1]$
released with $\varepsilon_{1}=\varepsilon_{2}=0.1$ and analyzed with prior
$p(\mu,\sigma^{2})\propto 1$. Analyses with constraints accounted for are
solid lines and not accounted for are dashed lines. Data generating model is
either ${\mathcal{N}}(\mu=0.1,\sigma^{2}=0.04^{2})$ (red) or
${\mathcal{N}}(\mu=0.5,\sigma^{2}=0.2^{2})$ (blue). Each Gibbs sampler is run
for 20,000 iterations.

Figure 8 presents the RMSE from estimating $\mu$ with the posterior mode in
the simulation study from Section 4 used to make Figure 2 of the main text.
The RMSE is uniformly lower in the constrained analysis than in the
unconstrained analysis. For the constrained analysis, a regression of log RMSE
on log $n$ has a slope of $-1$ (with $R^{2}>0.999$), indicating that RMSE
decays proportionally to $1/n$

Figure 9 presents the results of a simulation study identical to that of
Figure 2 and Figure 8 of the main text, but with $\varepsilon$ increased by a
factor of $10$. There is little practical difference between the constrained
and unconstrained analysis at this larger $\varepsilon$.

Figure 9: Empirical coverage rate of 95% HPD intervals for $\mu$ (top),
average 95% HPD interval length for $\mu$ (middle), and root mean square error
for estimating $\mu$ (bottom) for different sample sizes. Results based on
10,000 simulated datasets $Y_{i}\in[0,1]$ released with
$\varepsilon_{1}=\varepsilon_{2}=1$ and analyzed with prior
$p(\mu,\sigma^{2})\propto 1$. Analyses with constraints accounted for are
solid lines and not accounted for are dashed lines. Data generating model is
either ${\mathcal{N}}(\mu=0.1,\sigma^{2}=0.04^{2})$ (red) or
${\mathcal{N}}(\mu=0.5,\sigma^{2}=0.2^{2})$ (blue). Each Gibbs sampler is run
for 20,000 iterations.

##  Appendix I Constraints in Simple Linear Regression

In this section, we derive constraints on the model proposed by [5] in the
simple linear regression setting. Letting
${\mathbf{X}}=[{\mathbf{1}}_{n}\hskip 5.78172pt{\mathbf{x}}_{1}]$, the authors
release ${\mathbf{X}}^{\top}{\mathbf{X}}$, ${\mathbf{X}}^{\top}{\mathbf{Y}}$,
and ${\mathbf{Y}}^{\top}{\mathbf{Y}}$, which include the following elements.

$\displaystyle{\mathbf{X}}^{\top}{\mathbf{X}}=\begin{bmatrix}{\mathbf{1}}_{n}^{\top}{\mathbf{1}}_{n}&{\mathbf{1}}^{\top}{\mathbf{x}}_{1}\\\
{\mathbf{x}}_{1}^{\top}{\mathbf{1}}_{n}&{\mathbf{x}}_{1}^{\top}{\mathbf{x}}_{1}\end{bmatrix},\qquad{\mathbf{X}}^{\top}{\mathbf{Y}}=\begin{bmatrix}{\mathbf{1}}_{n}^{\top}{\mathbf{Y}}\\\
{\mathbf{x}}_{1}^{\top}{\mathbf{Y}}\end{bmatrix}.$ (106)

Note that ${\mathbf{1}}_{n}^{\top}{\mathbf{1}}_{n}=n$ and as sums of squares,
${\mathbf{x}}_{1}^{\top}{\mathbf{x}}_{1}=\sum_{i=1}^{n}x_{i}^{2}\geq 0$ and
${\mathbf{Y}}^{\top}{\mathbf{Y}}=\sum_{i=1}^{n}y_{i}^{2}\geq 0$. The
requirement that $x_{i},y_{i}\in[0,1]$ for all $i\in\\{1,\ldots,n\\}$ implies
the following additional constraints.

  1. 1.

Since $x_{i}\leq 1$ for all $i$, it follows that
${\mathbf{x}}_{1}^{\top}{\mathbf{1}}_{n}={\mathbf{1}}^{\top}{\mathbf{x}}_{1}=\sum_{i=1}^{n}x_{i}\leq
n$.

  2. 2.

Since $y_{i}\leq 1$ for all $i$, it follows that
${\mathbf{1}}_{n}^{\top}{\mathbf{Y}}=\sum_{i=1}^{n}y_{i}\leq n$.

  3. 3.

Since $x_{i},y_{i}\geq 0$ for all $i$, it follows that
${\mathbf{x}}_{1}^{\top}{\mathbf{Y}}=\sum_{i=1}^{n}x_{i}y_{i}\geq 0$.

  4. 4.

Since $x_{i}\leq 1$ for all $i$, it follows that
${\mathbf{x}}_{1}^{\top}{\mathbf{Y}}=\sum_{i=1}^{n}x_{i}y_{i}\leq\sum_{i=1}^{n}y_{i}={\mathbf{1}}_{n}^{\top}{\mathbf{Y}}$.

  5. 5.

Since $y_{i}\leq 1$ for all $i$, it follows that
${\mathbf{x}}_{1}^{\top}{\mathbf{Y}}=\sum_{i=1}^{n}x_{i}y_{i}\leq\sum_{i=1}^{n}x_{i}={\mathbf{1}}_{n}^{\top}{\mathbf{x}}_{1}$.

  6. 6.

Since $0\leq x_{i}\leq 1$ for all $i$, $x_{i}^{2}\leq x_{i}$ and so
${\mathbf{x}}_{1}^{\top}{\mathbf{x}}_{1}=\sum_{i=1}^{n}x_{i}^{2}\leq\sum_{i=1}^{n}x_{i}={\mathbf{1}}_{n}^{\top}{\mathbf{x}}_{1}$.

  7. 7.

Since $0\leq y_{i}\leq 1$ for all $i$, $y_{i}^{2}\leq y_{i}$ and so
${\mathbf{Y}}^{\top}{\mathbf{Y}}=\sum_{i=1}^{n}y_{i}^{2}\leq\sum_{i=1}^{n}y_{i}={\mathbf{1}}_{n}^{\top}{\mathbf{Y}}$.

The form of the model implies constraints on the parameters $\theta_{0}$,
$\theta_{1}$, and $\sigma^{2}$. We can rewrite the model in the following
form, where for each $i\in\\{1,\ldots,n\\}$,

$\displaystyle y_{i}\sim{\mathcal{N}}(\theta_{0}+\theta_{1}x_{i},\sigma^{2}).$
(107)

For the regression coefficients, $\theta_{0}$ and $\theta_{1}$, we observe
that by monotonicity of expectation, since $y_{i}\in[0,1]$, it follows that
$E[y_{i}]=\theta_{0}+\theta_{1}x_{i}\in[0,1]$. Using this fact, for a given
value of $\theta_{1}$, we obtain the following constraints on $\theta_{0}$.

  1. 1.

When $\theta_{1}\geq 0$, since $x_{i}\geq 0$ for all $i$, it follows that
$\theta_{0}\leq\theta_{0}+\theta_{1}x_{i}\leq 1$.

  2. 2.

When $\theta_{1}\geq 0$, since $x_{i}\leq 1$ for all $i$, it follows that
$\theta_{0}+\theta_{1}\geq\theta_{0}+\theta_{1}x_{i}\geq 0$ and so
$\theta_{0}\geq-\theta_{1}$.

  3. 3.

When $\theta_{1}<0$, since $x_{i}\geq 0$ for all $i$, it follows that
$\theta_{0}\geq\theta_{0}+\theta_{1}x_{i}\geq 0$.

  4. 4.

When $\theta_{1}<0$, since $x_{i}\leq 1$ for all $i$, it follows that
$\theta_{0}+\theta_{1}\leq\theta_{0}+\theta_{1}x_{i}\leq 1$ and so
$\theta_{0}\leq 1-\theta_{1}$.

We summarize these constraints as follows.

$\displaystyle\begin{cases}0\leq\theta_{0}\leq 1-\theta_{1},&\mbox{if
}\theta_{1}<0;\\\ -\theta_{1}\leq\theta_{0}\leq 1,&\mbox{if }\theta_{1}\geq
0.\end{cases}$ (108)

Finally, we use a similar argument to Corollary 1 to obtain a constraint on
$\sigma^{2}$. By monotonicity of expectation and since $y_{i}\in[0,1]$ implies
$y_{i}^{2}\leq y_{i}$, for all $i$,

$\displaystyle\sigma^{2}=Var[y_{i}]=E[y_{i}^{2}]-E[y_{i}]^{2}\leq
E[y_{i}]-E[y_{i}]^{2}\leq 1/4.$ (109)

The final inequality is obtained by observing that the quadratic
$f(x)=x-x^{2}$ is minimized at $x=1/2$ and has minimum $f(1/2)=1/4$. Since the
variance of $y_{i}$ must be nonnegative, it follows that
$\sigma^{2}\in[0,1/4]$.

##  Appendix J Compute Details

In this section, we describe information on the computer resources needed to
reproduce the experiments. We note that Figures 1, 3, 4, and 6 can be run on a
personal computer in a few seconds and do not require any additional computing
resources. Figure 5 can be run on a personal computer in under five minutes
and so also does not require any additional computing resources.

Figures 2, 8, and 9 are based on a large simulation study run on an internal
shared compute cluster. The cluster was primarily used for parallelization of
computations over a large number of cores; no substantial memory or storage
was required. Simulations were run for two values of $\varepsilon$, two data
generating distributions, and 21 sample sizes for each the unconstrained and
constrained analysis. This yielded 168 input combinations, each of which was
ran on a different core. The compute time required to run the Gibbs Sampler on
$10{,}000$ simulated datasets varied for each input combination, but was
generally between 10 and 20 hours.


$\left[\alpha_{\mathsf{z}}\right]_{1},\gamma_{\mathsf{z}},\psi,\mathsf{r},\left[\Gamma_{\vec{\mathbf{x}}\odot\vec{\mathbf{y}}}\right]_{1}$
using the respective keys with $\mathcal{A}$.
$\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{1}}$ samples
$\left[\alpha_{\mathsf{z}}\right]_{2}$ randomly on behalf of the respective
honest parties. – Steps corresponding to $\Pi_{\mathsf{jmp4}}$ are simulated
according to $\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{l}}$ where $P_{1}$ acts as
the server outside the computation for
$\left[\Gamma_{\vec{\mathbf{x}}\odot\vec{\mathbf{y}}}\right]_{2}$. –
$\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{1}}$ computes $\chi_{1}$. Steps
corresponding to $\Pi_{\mathsf{jmp4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{i}}$ where $P_{1}$ acts as one of the
sender for $\chi_{1}$. – Steps corresponding to $\Pi_{\mathsf{jmp4}}$ are
simulated according to $\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{l}}$ where
$P_{1}$ acts as the server outside the computation for $\chi_{2}$. Online: –
$\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{1}}$ computes
$\left[\mathsf{\beta}_{\mathsf{z}}^{\star}\right]_{1},\left[\mathsf{\beta}_{\mathsf{z}}^{\star}\right]_{2}$.
Steps corresponding to $\Pi_{\mathsf{jmp4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{i}}$ and
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{k}}$, where $P_{1}$ acts as one of the
sender for $\left[\mathsf{\beta}_{\mathsf{z}}^{\star}\right]_{1}$, and $P_{1}$
acts as the receiver for
$\left[\mathsf{\beta}_{\mathsf{z}}^{\star}\right]_{2}$. –
$\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{1}}$ computes
$\beta_{\mathsf{z}}+\gamma_{\mathsf{z}}$. Steps corresponding to
$\Pi_{\mathsf{jmp4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{i}}$ where $P_{1}$ acts as one of the
sender for $\beta_{\mathsf{z}}+\gamma_{\mathsf{z}}$.

Figure 57: Simulator $\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{1}}$ for corrupt
$P_{1}$

The case for corrupt $P_{3}$ is given in Fig. 58.  Preprocessing: –
$\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{3}}$ samples
$\left[\alpha_{\mathsf{z}}\right]_{1},\left[\alpha_{\mathsf{z}}\right]_{2},\gamma_{\mathsf{z}},\psi,\mathsf{r},\left[\Gamma_{\vec{\mathbf{x}}\odot\vec{\mathbf{y}}}\right]_{1}$
using the respective keys with $\mathcal{A}$.
$\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{3}}$ computes
$\left[\Gamma_{\vec{\mathbf{x}}\odot\vec{\mathbf{y}}}\right]$ honestly. –
Steps corresponding to $\Pi_{\mathsf{jmp4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{j}}$ where $P_{3}$ acts as one of the
sender for $\left[\Gamma_{\vec{\mathbf{x}}\odot\vec{\mathbf{y}}}\right]_{2}$.
– $\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{3}}$ computes $\chi_{1},\chi_{2}$.
Steps corresponding to $\Pi_{\mathsf{jmp4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{j}}$ where $P_{3}$ acts as one of the
sender for $\chi_{1},\chi_{2}$. Online: – Steps corresponding to
$\Pi_{\mathsf{jmp4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jmp4}}}^{P_{l}}$ where $P_{3}$ acts as the server
outside the computation for
$\left[\mathsf{\beta}_{\mathsf{z}}^{\star}\right]_{1},\left[\mathsf{\beta}_{\mathsf{z}}^{\star}\right]_{2}$,
$\beta_{\mathsf{z}}+\gamma_{\mathsf{z}}$.

Figure 58: Simulator $\mathcal{S}_{\Pi_{\mathsf{dotp4}}}^{P_{3}}$ for corrupt
$P_{3}$

####  C.2.7  Truncation Pair Generation

Here we give the simulation steps for $\Pi_{\mathsf{trgen4}}$. The case for
corrupt $P_{0}$ is given in Fig. 59. The case for corrupt $P_{3}$ is similar.
– $\mathcal{S}_{\Pi_{\mathsf{trgen4}}}^{P_{0}}$ samples $R_{1},R_{2}$ using
the respective keys with $\mathcal{A}$. – Steps corresponding to
$\Pi_{\mathsf{jsh4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jsh4}}}^{P_{0}}$ (Fig. 53).

Figure 59: Simulator $\mathcal{S}_{\Pi_{\mathsf{trgen4}}}^{P_{0}}$ for corrupt
$P_{0}$

The case for corrupt $P_{1}$ is given in Fig. 60. The case for corrupt $P_{2}$
is similar.  – $\mathcal{S}_{\Pi_{\mathsf{trgen4}}}^{P_{1}}$ samples $R_{1}$
using the respective keys with $\mathcal{A}$, and samples $R_{2}$ randomly. –
Steps corresponding to $\Pi_{\mathsf{jsh4}}$ are simulated according to
$\mathcal{S}_{\Pi_{\mathsf{jsh4}}}^{P_{1}}$ (Fig. 54).

Figure 60: Simulator $\mathcal{S}_{\Pi_{\mathsf{trgen4}}}^{P_{1}}$ for corrupt
$P_{1}$