# Using auxiliary marginal distributions in imputations for nonresponse while
accounting for survey weights, with application to estimating voter turnout

Jiurui Tang  D. Sunshine Hillygus  Jerome P. Reiter

###### Abstract

The Current Population Survey is the gold-standard data source for studying
who turns out to vote in elections. However, it suffers from potentially
nonignorable unit and item nonresponse. Fortunately, after elections, the
total number of voters is known from administrative sources and can be used to
adjust for potential nonresponse bias. We present a model-based approach to
utilize this known voter turnout rate, as well as other population marginal
distributions on demographic variables, in multiple imputation for unit and
item nonresponse. In doing so, we ensure that the imputations produce design-
based estimates that are plausible given the known margins. We introduce and
utilize a hybrid missingness model comprising a pattern mixture model for unit
nonresponse and selection models for item nonresponse. Using simulation
studies, we illustrate repeated sampling performance of the model under
different assumptions about the missingness mechansisms. We apply the model to
examine voter turnout by subgroups using the 2018 Current Population Survey
for North Carolina. As a sensitivity analysis, we examine how results change
when we allow for over-reporting, i.e., individuals self-reporting that they
voted when in fact they did not.

Key Words: missing; multiple imputation; pattern mixture; selection.

##  1 Introduction

When data suffer from unit nonresponse (no values are observed for some units)
or item nonresponse (some values are observed and some values are missing for
some units), analysts generally need to make strong assumptions about the
reasons for missingness. For example, they may need to assume data are missing
at random (Rubin, 1976; Mealli and Rubin, 2015) or that they follow a specific
nonignorable missingness mechanism (Linero and Daniels, 2018). Such
assumptions are inescapable with missing data.

One way to reduce reliance on strong assumptions is to utilize auxiliary
information from external data sources. Here, we consider sets of population
percentages or totals for categorical variables. For example, we may have
accurate estimates of population percentages of demographic variables from
censuses, large-sample surveys, or administrative databases. To motivate how
such auxiliary information can help with missing data, suppose we have data
from a survey that includes a question on whether or not one votes in an
election. The survey suffers from missingness on vote, which we want to
(multiply) impute to facilitate estimation of the turnout rate. Among survey
respondents, 70% indicate that they voted in the election. However, using
population size estimates and post-election administrative data, we know that
only 50% of voters turned out. Assuming respondents do not misreport, this
auxiliary marginal information suggests that the missingness for vote is
nonignorable; that is, people who did not vote are more likely not to tell
their voting status. Knowing the margin, we can impute values for the missing
votes so that the completed data estimates of turnout are closer to 50% than
70%. Without this margin, we would likely impute the missing votes using a
missing at random (MAR) assumption—which would perpetuate the nonresponse
bias—or have to make some heroic, unverifiable assumption about the vote
distribution of the nonrespondents.

In this article, we present methods for leveraging auxiliary marginal
distributions with application to estimating turnout in the 2018 U.S.
Congressional election in North Carolina using the 2018 Current Population
Survey (CPS). To do so, we build on the missing data with auxiliary margins
(MD-AM) framework developed by Akande et al. (2021), which we review in
Section 2. Specifically, because the CPS relies on a complex sampling
design—as do many large-scale government and social surveys—we extend the MD-
AM framework to account for survey weights from unequal probability sampling
designs. The basic idea is to generate multiple imputations so that the
completed data result in plausible design-based estimates of the known
margins. This strategy was introduced by Akande and Reiter (2022), who
illustrated it for stratified simple random samples subject to item
nonresponse only. We leverage and extend this strategy to surveys with weights
generated by other complex sampling designs and subject to both item and unit
nonresponse. In doing so, we introduce a hybrid missing data mechanism that
uses a pattern mixture model formulation for unit nonresponse and a selection
model formulation for item nonresponse. Using simulation studies, we show that
using a hybrid missingness MD-AM model can result in more reasonable survey-
weighted inferences than approaches that do not utilize the margins. We then
use a hybrid missingness MD-AM model to estimate voter turnout for subgroups
in the 2018 CPS. More broadly, the CPS analysis is illustrative of a general,
multiple imputation strategy for using auxiliary margins to handle
nonignorable missing data in surveys with complex sampling designs.

The remainder of this article is organized as follows. In Section 2, we review
the MD-AM framework of Akande et al. (2021) for simple random samples,
explaining how it can leverage auxiliary margins to handle unit and item
nonresponse. In Section 3, we present an MD-AM model that uses the hybrid
missingness mechanism and accounts for survey weights. We also describe
methods for estimating the parameters of this model and generating multiple
imputations of missing values, and we summarize results of simulation studies
evaluating the model fitting procedure. The simulation studies are described
in detail in the supplementary material. In Section 4, we use a hybrid
missingness MD-AM model to estimate turnout in demographic subgroups using the
2018 CPS in North Carolina. We also assess the sensitivity of results to
potential reporting error in the vote responses. In Section 5, we provide some
concluding remarks.

##  2 MD-AM Modeling with Simple Random Samples

When presenting the MD-AM framework for simple random samples, Akande et al.
(2021) describe a two step process. In step 1, the analyst specifies a joint
distribution for the survey variables and for the indicators for nonresponse
that are identifiable from the observed data alone. This model typically uses
default assumptions about the missingness, such as MAR or itemwise
conditionally independent nonresponse (ICIN, Sadinle and Reiter, 2017). In
step 2, the analyst, informed by the available margins, adds parameters to the
model while ensuring that the model as a whole can be identified as in Sadinle
and Reiter (2019). Adding these parameters weakens the assumptions about the
missingness mechanisms. In this section, we review the MD-AM framework for
simple random samples, beginning with general notation that we use throughout.

###  2.1 Notation

Let $\mathcal{D}$ comprise the data that were planned to be collected from a
survey of $i=1,\dots,n$ individuals, and let $\mathcal{A}$ comprise the
auxiliary margins. Let $X=(X_{1},\dots,X_{k})$ represent the $p$ variables in
both $\mathcal{A}$ and $\mathcal{D}$, where each
$X_{k}=(X_{1k},\dots,X_{nk})^{T}$ for $k=1,\dots,p$. Let
$Y=(Y_{1},\dots,Y_{q})$ represent the $q$ variables in $\mathcal{D}$ but not
in $\mathcal{A}$, where each $Y_{k}=(Y_{1k},\dots,Y_{nk})^{T}$ for
$k=1,\dots,q$. We disregard variables in $\mathcal{A}$ but not in
$\mathcal{D}$ as they are not of primary interest. We assume that
$\mathcal{A}$ contains only sets of marginal distributions for variables in
$X$, e.g., available in some external database.

For each $k=1,\dots,p$, let $R_{ik}^{x}=1$ if individual $i$ would not respond
to the question on $X_{k}$ in the survey and $R_{ik}^{x}=0$ otherwise.
Similarly, for each $k=1,\dots,q$, let $R_{ik}^{y}=1$ if individual $i$ would
not respond to the question on $Y_{k}$ in the survey and $R_{ik}^{y}=0$
otherwise. Let $R^{x}=(R_{1}^{x},\dots,R_{p}^{x})$ and
$R^{y}=(R_{1}^{y},\dots,R_{q}^{y})$ be the vectors of item nonresponse
indicators for variables in $X$ and $Y$ respectively, where each
$R_{k}^{x}=(R_{ik}^{x},\dots,R_{nk}^{x})^{T}$ and
$R_{k}^{y}=(R_{ik}^{y},\dots,R_{nk}^{y})^{T}$. Let $U_{i}=1$ when individual
$i$ in $\mathcal{D}$ does not respond to the survey, so that we do not have
their responses. Let $U_{i}=0$ when observe at least some values of the survey
variables for individual $i$. Let $U=(U_{1},\dots,U_{n})$.

###  2.2 Illustrative Model Specification

We illustrate the ideas underpinning the MD-AM framework using a $\mathcal{D}$
comprising two binary variables, $X_{1}$ and $X_{2}$. We have auxiliary
information for both from $\mathcal{A}$; thus, this illustrative example does
not include any $Y$ variables (whereas our 2018 CPS analysis does use $Y$
variables). We suppose $X_{1}$ suffers from item nonresponse and $X_{2}$ is
fully observed, so that $R_{1}^{x}$ is the vector of item nonresponse
indicators for $X_{1}$. For units with $U_{i}=1$, we do not observe
$(X_{1i},X_{2i},R_{1i}^{x})$. We do not concern ourselves with $R_{2}^{x}$
since $X_{2}$ is fully observed for cases with $U_{i}=0$. $U$ is fully
observed for all records in $\mathcal{D}$. Figure 1 displays the observed data
and the auxiliary information on $X_{1}$ and $X_{2}$. We use this same example
to introduce the hybrid missingness MD-AM model in Section 3.

Figure 1: Illustrative scenario for the MD-AM model in simple random samples. $\mathcal{D}$ comprises two binary variables, $X_{1}$ and $X_{2}$. $X_{2}$ is fully observed; $X_{1}$ suffers from item nonresponse; and, some units do not respond. Here, “$\checkmark$” represents observed values, “?” represents missing values. $X_{1}$ | $X_{2}$ | $R_{1}^{x}$ | $U$ |
---|---|---|---|---
$\checkmark$ | $\checkmark$ | 0 | 0 |  }31mm[$\mathcal{D}$]
? | $\checkmark$ | 1 |
? | ? | ? | 1 |
$\checkmark$ |  |  |  |  }21mm[$\mathcal{A}$]
| $\checkmark$ |  |  |

Akande et al. (2021) recommend using selection models for all missingness
indicators. With their modeling strategy, we write the joint distribution of
$(X_{1},X_{2},R_{1}^{x},U)$ as a product of sequential conditional
distributions, using selection models for the unit and item nonresponse
indicators.

$\displaystyle\mathbb{P}(X_{1},X_{2},R_{1}^{x},U)$
$\displaystyle=\mathbb{P}(R_{1}^{x}|X_{1},X_{2},U)\mathbb{P}(U|X_{1},X_{2})\mathbb{P}(X_{1}|X_{2})\mathbb{P}(X_{2}).$
(1)

The joint distribution in (1) can be fully parameterized with fifteen
parameters plus the constraint that probabilities sum to one. However,
$\mathcal{D}$ alone provides enough information to identify only six
parameters. This is evident in the six observable quantities in Figure 1,
namely the probabilities for $U$ and $(R_{1}^{x}\mid U=0)$, the two
probabilities for $(X_{2}\mid R_{1}^{x}=r,U=0)$ where $r\in\\{0,1\\}$, and the
two probabilities for $(X_{1}\mid X_{2}=x,R_{1}^{x}=0,U=0)$ where
$x\in\\{0,1\\}$. Thus, we need to make some identifying assumptions about the
missingness in order to estimate the joint distribution.

The auxiliary information about the marginal distributions of $X_{1}$ and
$X_{2}$ add constraints on the joint distribution in (1). The MD-AM framework
leverages this additional information. We begin with step 1: specify an
identifiable model based on $\mathcal{D}$ alone with at most six parameters.
Using selection models for the nonresponse indicators, one natural choice is

$\displaystyle X_{2}$ $\displaystyle\sim
Bernoulli(\pi_{x_{2}}),\,\,\,logit(\pi_{x_{2}})=\alpha_{0}$ (2) $\displaystyle
X_{1}|X_{2}$ $\displaystyle\sim
Bernoulli(\pi_{x_{1}}),\,\,\,logit(\pi_{x_{1}})=\beta_{0}+\beta_{1}X_{2}$ (3)
$\displaystyle U|X_{1},X_{2}$ $\displaystyle\sim Bernoulli(\nu_{0})$ (4)
$\displaystyle R_{1}^{x}|U,X_{1},X_{2}$ $\displaystyle\sim
Bernoulli(\pi_{R}),\,\,\,logit(\pi_{R})=\gamma_{0}+\gamma_{1}X_{2}.$ (5)

This default model has unit nonresponse following a missing completely at
random mechanism (MCAR) mechanism and item nonresponse in $X_{1}$ following an
ICIN mechanism.

In step 2, we add one term related to $X_{1}$ and one term related to $X_{2}$
to the specifications in (4) – (5). One approach is to add both terms to the
model for unit nonresponse in (4), leaving the model for item nonresponse as
in (5). With slight re-use and abuse of notation, we replace (4) with

$U\mid X_{1},X_{2}\sim
Bernoulli(\pi_{U}),\,logit(\pi_{U})=\nu_{0}+\nu_{1}X_{1}+\nu_{2}X_{2}.$ (6)

This is an example of an additive nonignorable model (Hirano et al., 2001;
Deng et al., 2013; Schifeling et al., 2015; Si et al., 2016) for unit
nonresponse. It includes the MCAR mechanism as a special case
($\nu_{1}=\nu_{2}=0$), but allows for nonignorable mechanisms as well (either
$\nu_{1}\neq 0$ or $\nu_{2}\neq 0$). Thus, the data can inform whether MCAR is
plausible, or if not allow for unit nonresponse to depend on $X_{1}$ or
$X_{2}$.

There is a key identifying assumption in (6): there is no interaction between
$X_{1}$ and $X_{2}$. With only marginal information on $X_{1}$ and $X_{2}$, we
have no way to identify, for example, a differential effect of $X_{1}$ on $U$
at $X_{2}=1$ versus $X_{2}=0$. If we had the joint margin of $(X_{1},X_{2})$,
we could add their interaction to (6). Nonetheless, using $\mathcal{A}$ has
allowed us to weaken substantially the strong MCAR assumption in (4).

As an alternative to (6), we could specify an additive nonignorable model for
$R^{x}_{1}$, which would allow the mechanism for item nonresponse in $X_{1}$
to depend on the value of $X_{1}$ itself. In doing so, however, we have to
remove the direct dependence of $U$ on $X_{1}$, as the margin for $X_{1}$ only
provides enough information to estimate one additional parameter. Thus, again
re-using notation for the regression parameters, we could replace (4) and (5)
with the alternative specification,

$\displaystyle U|X_{1},X_{2}$ $\displaystyle\sim
Bernoulli(\pi_{U}),\,\,\,logit(\pi_{U})=\nu_{0}+\nu_{1}X_{2}$ (7)
$\displaystyle R_{1}^{x}|X_{1},X_{2},U$ $\displaystyle\sim
Bernoulli(\pi_{R}),\,\,\,logit(\pi_{R})=\gamma_{0}+\gamma_{1}X_{1}+\gamma_{2}X_{2}.$
(8)

As this example suggests, analysts can use $\mathcal{A}$ to enrich the unit
nonresponse model or item nonresponse model. Akande et al. (2021) suggest that
analysts use the auxiliary margins to help model unit nonresponse when the
unit nonresponse rate is higher than the item nonresponse rate or when the
primary concern is about nonignorable unit nonresponse. Similarly, analysts
can enrich the item nonresponse model when item nonresponse is a bigger
concern than unit nonresponse. Researchers can fit multiple model
specifications and compare their results as a sensitivity analysis.

In the CPS analysis described in Section 4, unit nonresponse is more
substantial than item nonresponse. Hence, when describing our adaptation of
the MD-AM framework for handling unequal survey weights, we focus on using
$\mathcal{A}$ to enrich modeling of unit nonresponse.

##  3 MD-AM with Hybrid Missingness Model and Weights

The MD-AM model in Section 2 is intended for simple random samples. In complex
designs, it can result in biased estimates of finite population quantities, as
the modeling and imputation steps do not account for unequal probabilities of
selection. Akande and Reiter (2022) describe how to modify the MD-AM model to
account for the special case of stratified simple random sampling subject to
item nonresponse only. To do so, they use a rejection sampler: one proposes
imputations of the missing values, computes design-based estimates of totals
for variables with known margins, and accepts or rejects the proposals
according to how close in distribution the estimated totals are the known
marginal total. Their model employs selection models for the item nonresponse
indicators.

We attempted to apply the selection modeling approach in (1) and the strategy
in Akande and Reiter (2022) to account for more general unequal probability
sampling designs with both item and unit nonresponse. However, the acceptance
rate of the rejection sampler is generally too low to be practically useful,
and it can be difficult to generate imputations that result in plausible
design-based estimates for arbitrary survey weights. We instead propose a
hybrid missingness model that uses a pattern mixture model for unit
nonresponse and a selection model for item nonresponse. Using the example in
Section 2, we specify a model for $U$, a model for $(X_{1},X_{2}\mid U)$, and
a model for $(R_{1}^{x}\mid X_{1},X_{2},U)$, as we now describe.

We modify step 1 in the MD-AM framework to reflect the alternate
factorization. Again re-using some notation for regression parameters, we have
the default model,

$\displaystyle U$ $\displaystyle\sim Bernoulli(\pi_{U})$ (9) $\displaystyle
X_{2}|U$ $\displaystyle\sim
Bernoulli(\pi_{x_{2}}),\,\,\,logit(\pi_{x_{2}})=\alpha_{0}$ (10)
$\displaystyle X_{1}|X_{2},U$ $\displaystyle\sim
Bernoulli(\pi_{x_{1}}),\,\,\,logit(\pi_{x_{1}})=\beta_{0}+\beta_{1}X_{2}$ (11)
$\displaystyle R_{1}^{x}|X_{1},X_{2},U$
$\displaystyle=Bernoulli(\pi_{x_{1}}),\,\,\,logit(\pi_{R})=\gamma_{0}+\gamma_{1}X_{2}.$
(12)

This is essentially the same model proposed for step 1 in Section 2.2, re-
organized here to clearly show the pattern mixture modeling for $U$. We have
to assume $R_{1}^{x}$ is conditionally independent of $U$ given $\mathcal{D}$
to enable identification of $(\gamma_{0},\gamma_{1})$, as we have no
information to distinguish distributions of $R_{1}^{x}$ for unit respondents
and nonrespondents.

Following step 2 of the MD-AM framework, we next add terms related to $X_{1}$
and $X_{2}$ to leverage the information in the known margins. We dedicate both
terms to unit nonresponse modeling by adding an indicator variable for unit
nonresponse, which we label as $U$, to (10) and (11). We have

$\displaystyle X_{2}|U$ $\displaystyle\sim
Bernoulli(\pi_{x_{2}}),\,\,\,logit(\pi_{x_{2}})=\alpha_{0}+\alpha_{1}U$ (13)
$\displaystyle X_{1}|X_{2},U$ $\displaystyle\sim
Bernoulli(\pi_{x_{1}}),\,\,\,logit(\pi_{x_{1}})=\beta_{0}+\beta_{1}X_{2}+\beta_{2}U.$
(14)

Here, (13) implies that the distribution of $X_{2}$ differs for unit
nonrespondents and unit respondents, and $\alpha_{1}$ controls the strength of
nonresponse bias. Specifically, $logit(\pi_{x_{2}})=\alpha_{0}$ for unit
respondents, and $logit(\pi_{x_{2}})=\alpha_{0}+\alpha_{1}$ for unit
nonrespondents. As a result, with larger $|\alpha_{1}|$, the distribution of
$X_{2}$ is less homogeneous across unit nonrespondents and unit respondents.
Similar interpretations can be applied for (14). Essentially, we can view (14)
as a model with different intercepts for unit respondents and unit
nonrespondents, but with slopes preserved.

For the hybrid missingness MD-AM model, one key identifying assumption is no
interactions with $U$. This is unavoidable when using only univariate margins.
With bivariate margins, analysts can add interaction effects. For example,
when the joint margin of $(X_{1},X_{2})$ is known, analysts can add an
interaction between $U$ and $X_{2}$ in (14).

###  3.1 Incorporating Survey Weights

We now incorporate survey weights into the hybrid missingness MD-AM model. Our
overarching goal is to facilitate valid design-based estimation, which is the
preferred approach to finite population inference for many practitioners. Let
$N$ denote the population size from which the $n$ survey units in
$\mathcal{D}$ are sampled. For $i=1,\dots,N$, let $\pi_{i}$ be the probability
that unit $i$ is selected to be in the sample, and let $w_{i}^{d}=1/\pi_{i}$
be its design weight (also known as its sampling weight).

####  3.1.1 Using Margins for Probabilistic Constraints

In finite population inference, often we seek to estimate the population
totals or means of survey variables. For example, we may seek to estimate the
total of some variable $X_{k}$, which we write as
$T_{X_{k}}=\sum_{i=1}^{N}X_{ik}$. Analysts can estimate $T_{X_{k}}$ with the
Horvitz and Thompson (1952) estimator,

$\hat{T}_{X_{k}}=\sum_{i\in\mathcal{D}}w_{i}^{d}X_{ik}.$ (15)

Finite population central limit theorems ensure that when $n$ is large enough,
for fully observed data we have

$\hat{T}_{X_{k}}\sim N(T_{X_{k}},V_{X_{k}})$ (16)

with some variance $V_{X_{k}}$. Analysts can estimate $V_{X_{k}}$ using
design-based principles (Fuller, 2009).

However, we cannot compute $\hat{T}_{X_{k}}$ directly when units in
$\mathcal{D}$ are missing values of $X_{k}$, either due to unit or item
nonresponse. Nonetheless, (16) still holds for the unobserved value of
$\hat{T}_{X_{k}}$. Furthermore, we know $T_{X_{k}}$ (or the population
percentage, $T_{X_{k}}/N$) for any $X_{k}$ with a margin in $\mathcal{A}$.
Thus, as suggested in Akande and Reiter (2022), we should impute the missing
values for $X_{k}$ so that any completed data set produces a reasonable
$\hat{T}_{X_{k}}$ based on (16).

We implement this probabilistic constraint in the illustrative model with
binary $X_{1}$ and $X_{2}$ as follows. For all $i\in\mathcal{D}$ and for all
$k$, let $X_{ik}^{\star}=X_{ik}$ when $R_{ik}^{x}=0$, and let $X_{ik}^{\star}$
be an imputed value when $R_{ik}^{x}=1$ or $U_{i}=1$. In addition to following
(12) – (14), the imputations of missing $(X_{1},X_{2})$ should satisfy,

$\sum_{i\in\mathcal{D}}w_{i}^{d}X_{i1}^{\star}\sim
N(T_{X_{1}},V_{X_{1}}),\,\,\,\,\,\,\,\,\sum_{i\in\mathcal{D}}w_{i}^{d}X_{i2}^{\star}\sim
N(T_{X_{2}},V_{X_{2}}).\\\ $ (17)

When values of $X_{k}$ are missing, the design-based estimate of $V_{X_{k}}$
also is not directly computable. We therefore consider $V_{X_{k}}$ as a
parameter set by the analyst to reflect how closely $\hat{T}_{X_{k}}$ should
match $T_{X_{k}}$ in any completed data set. When we set $V_{X_{k}}$ to be
relatively small, we encourage imputations that result in $\hat{T}_{X_{k}}$
close to $T_{X_{k}}$. We note, however, that with small $V_{X_{k}}$ it can be
challenging computationally to find imputations that satisfy (17) and
adequately explore the space of plausible imputations of the missing values
(Tang, 2022). On the other hand, when we let $V_{X_{k}}\to\infty$, we
effectively impose no probabilistic constraint on $\hat{T}_{X_{k}}$, in which
case the MD-AM model is not identifiable.

We suggest setting $V_{X_{k}}$ to be plausibly close to what its design-based
estimate would have been absent missing data. To do so, analysts can use the
following procedure. First, generate a completed data set by imputing $X_{ik}$
for units with $R_{ik}^{x}=1$ using a common MAR mechanism, such as via
multiple imputation by chained equations (Azur et al., 2011) or, if available,
by using the imputations made by the agency responsible for collecting
$\mathcal{D}$. Second, compute the survey-weighted estimate of variance of
$\hat{T}_{X_{k}}$ with this completed data set. We denote this estimate as
$\widetilde{V}_{X_{k}}$ and use it for $V_{X_{k}}$. For the survey-weighted
estimation, we assume that the analyst uses respondent weights that are
adjusted for unit nonresponse, for example, by the agency responsible for
collecting $\mathcal{D}$.

####  3.1.2 Weights for Unit Nonrespondents

Data analysts do not always have access to design weights for survey
nonrespondents; indeed, federal agencies routinely do not provide any weights
for survey nonrespondents. Therefore, we develop methods for scenarios where
weights are unavailable for unit nonrespondents. For convenience in notation,
when using design weights in estimates, we set $w_{i}^{d}=0$ whenever
$U_{i}=1$.

The procedure for estimating the MD-AM model, which we describe in section
3.2, requires the availability of weights for all units. When the design
weights for survey respondents are available as part of $\mathcal{D}$, we need
to create weights for the survey nonrespondents. To do so, we treat all
nonrespondents equally and let each have the same weight. We generate weights
for nonrespondents so that the sum of the weights for all $n$ units in
$\mathcal{D}$ equals $N$. Therefore, when design weights are available, the
weights that we use for analysis are as follows:

$w_{i}=\left\\{\begin{array}[]{lr}w_{i}^{d}&\text{if }U_{i}=0\\\
\frac{N-\sum_{j\in\mathcal{D}}w_{j}^{d}}{\sum_{j\in\mathcal{D}}U_{j}}&\text{if
}U_{i}=1.\end{array}\right.$ (18)

####  3.1.3 Using Adjusted Weights

It is a common practice for design weights to be adjusted for unit nonresponse
so that the respondents weight up to the full population (Valliant et al.,
2013). Indeed, adjusted weights, which we denote as $w_{i}^{a}$, are often the
only weights available to analysts. Weight adjustment methods include but are
not limited to weighting class adjustment, propensity score adjustment,
poststratification and raking (Valliant et al., 2013; Lohr, 2010; Si and Zhou,
2021). If we use adjusted weights in 18, we may end up with very small or even
negative values of $w_{i}$ because the sum of the $w_{i}^{a}$ for the unit
respondents already might be close to or even exceed $N$.

We avoid this undesirable outcome by down-weighting each $w_{i}^{a}$ to
attempt to get close to $w_{i}^{d}$. In some instances, enough information
about the weighting adjustment procedures is provided to enable analysts to do
so exactly. Often, however, this is not the case. Absent information to allow
reverse-engineering of the design weights, analysts can use an ad hoc and
simple technique as follows. As before, set $w_{i}^{a}=0$ whenever $U_{i}=1$.
For all $i\in\mathcal{D}$, we create and use for estimation,

$w_{i}^{*}=\left\\{\begin{array}[]{lr}w_{i}^{a}\times(1-\frac{\sum_{j\in\mathcal{D}}U_{j}}{n}),&\text{if
}U_{i}=0\\\ \frac{\sum_{j\in\mathcal{D}}w_{j}^{a}}{n},&\text{if
}U_{i}=1.\end{array}\right.$ (19)

Here, (19) down-weights the adjusted weights for unit respondents by the unit
response rate, and assigns the remaining weight evenly to the unit
nonrespondents. After this re-assignment step,
$\sum_{i\in\mathcal{D}}w_{i}^{*}=\sum_{i\in\mathcal{D}}w_{i}^{a}$, which
should be a reasonable estimate of $N$.

Analysts could create weights for unit nonrespondents by instead requiring
$\sum w_{i}^{*}=N$. We find in simulation studies that setting $w_{i}^{*}$ so
that $\sum w_{i}^{*}=N$ or $\sum w_{i}^{*}=\sum w_{i}^{a}$ does not make much
difference to the ultimate inferences (Tang, 2022).

###  3.2 Estimation Strategy for Illustrative Model

In this section, we illustrate a general estimation strategy for the hybrid
missingness MD-AM model using the model defined by (12)—(14). The strategy can
be extended to handle additional variables by replicating the various sampling
steps. We present the sampler when design weights for unit respondents are
published. When those weights are not available, we substitute $w_{i}^{*}$ for
$w_{i}$. We assume $T_{X_{1}}$ and $T_{X_{2}}$ are available in $\mathcal{A}$,
and that the analyst has specified ${V}_{X_{1}}$ and ${V}_{X_{2}}$.

We use normal approximations for the distributions of the estimated
coefficients of the logistic regressions to simplify computations (Raghunathan
et al., 2001). Drawing from the exact posterior distribution can be achieved
via data augmentation using Pólya-Gamma latent variables (Polson et al.,
2013), although with large enough samples the normal approximations often are
reasonable.

Let $n_{U}$ be the number of unit nonrespondents in $\mathcal{D}$. We get
starting values for the sampler by imputing $X_{i1}$ for units with
$R_{i1}^{x}=1$ under the assumption of MAR. We then compute maximum likelihood
estimates for all parameters in the logistic regressions from this completed
data set, which we use as starting values. The sampler proceeds as follows
from any iteration $t$.

  1. IM1.

Draw a value $\hat{T}_{X_{2}}^{(t+1)}\sim N(T_{X_{2}},{V}_{X_{2}})$.

  2. IM2.

Calculate the number of times $X_{i2}^{(t+1)}$ should be imputed as 1 when
$U_{i}=1$ so that the weighted sum of $X_{2}^{(t+1)}$ is as close to
$\hat{T}_{X_{2}}^{(t+1)}$ as possible; denote this number as $n_{2U}$.
Specifically,

$n_{2U}=\lfloor\frac{\hat{T}_{X_{2}}^{(t+1)}-\sum_{i\in\mathcal{D}}w_{i}X_{i2}^{(t+1)}I(U_{i}=0)}{\sum_{i\in\mathcal{D}}w_{i}I(U_{i}=1)/n_{U}}\rfloor.$
(20)

  3. IM3.

Let $\hat{\alpha}_{0}$ and $V_{\hat{\alpha}_{0}}$ denote the maximum
likelihood estimate and the variance of $\alpha_{0}$ based on the $U_{i}=0$
observations. Sample $\alpha_{0}^{(t+1)}$ from its approximate posterior
distribution, $N(\hat{\alpha}_{0},V_{\hat{\alpha}_{0}})$.

  4. IM4.

Calculate the proportion of unit nonrespondents that should be imputed as 1
for $X_{2}$. Denote this number as $\hat{p_{2}}=\frac{n_{2U}}{n_{U}}$. Then,
set $\alpha_{1}^{(t+1)}$ so that
$\mathbb{E}[\alpha_{0}^{(t+1)}+\alpha_{1}^{(t+1)}]=logit(\hat{p}_{2})$.

  5. IM5.

Draw imputations of $X_{2}$ for unit nonrespondents from $X_{2}|U=1\sim
Bernoulli\Big{(}1/\big{(}1+\exp(-\alpha_{0}^{(t+1)}-\alpha_{1}^{(t+1)})\big{)}\Big{)}$.

  6. IM6.

Draw a value $\hat{T}_{X_{1}}^{(t+1)}\sim N(T_{X1},{V}_{X_{1}})$.

  7. IM7.

Calculate the number of times $X_{i1}^{(t+1)}$ should be imputed as 1 when
$U_{i}=1$ so that the weighted sum of $X_{1}^{(t+1)}$ is as close to
$\hat{T}_{X1}^{(t+1)}$ as possible; denote this number as $n_{1U}$.
Specifically,

$n_{1U}=\lfloor\frac{\hat{T}_{X_{1}}^{(t+1)}-\sum_{i}^{n}w_{i}X_{i1}^{(t)}I(U_{i}=0)}{\sum_{i}^{n}w_{i}I(U_{i}=1)/n_{U}}\rfloor.$
(21)

  8. IM8.

Let $\hat{\beta}$ and $V_{\hat{\beta}}$ denote the maximum likelihood
estimates and the covariance matrix of $\begin{bmatrix}\beta_{0}\\\
\beta_{1}\end{bmatrix}$ based on the $U_{i}=0$ observations. Sample
$\begin{bmatrix}\beta_{0}\\\ \beta_{1}\end{bmatrix}^{(t+1)}$ from its
approximate posterior distribution $N(\hat{\beta},V_{\hat{\beta}})$.

  9. IM9.

Calculate the proportion of unit nonrespondents that should be imputed as 1
for $X_{1}$. Denote it as $\hat{p_{1}}=\frac{n_{1U}}{n_{U}}$. Then, set
$\beta_{2}^{(t+1)}$ so that
$\mathbb{E}[\beta_{0}^{(t+1)}+\beta_{1}^{(t+1)}X_{2}+\beta_{2}^{(t+1)}]=logit(\hat{p}_{1})$.

  10. IM10.

Draw imputations of $X_{1}$ for unit nonrespondents from $X_{1}|U=1\sim
Bernoulli\Big{(}1/\big{(}1+\exp(-\beta_{0}^{(t+1)}-\beta_{1}^{(t+1)}X_{2}-\beta_{2}^{(t+1)})\big{)}\Big{)}$.

  11. IM11.

Let $\hat{\gamma}$ and $V_{\hat{\gamma}}$ denote the maximum likelihood
estimates and the covariance matrix of $\begin{bmatrix}\gamma_{0}\\\
\gamma_{1}\end{bmatrix}$ based on $U_{i}=0$ observations. Sample
$\begin{bmatrix}\gamma_{0}\\\ \gamma_{1}\end{bmatrix}^{(t+1)}$ from its
approximate posterior distribution, $N(\hat{\gamma},V_{\hat{\gamma}})$.

  12. IM12.

Draw imputations of $X_{1}^{(t+1)}$ when $(R_{i1}^{x}=1,U_{i}=0)$ from a
$Bernoulli\Big{(}1/\big{(}1+\exp(-\beta_{0}^{(t+1)}-\beta_{1}^{(t+1)}X_{2}\big{)}\Big{)}$.

We update $X_{2}$ first and use it as a predictor of $X_{1}$ because, in our
illustrative setting, $X_{2}$ only suffers from unit nonresponse whereas
$X_{1}$ also suffers from item nonresponse. When both $X_{2}$ and $X_{1}$ have
item nonresponse, analysts can generate starting values for both using MICE.
In general, we recommend updating variables with smaller item nonresponse
rates earlier in the sequence.

This algorithm can be viewed as a generalized version of the one proposed in
Pham et al. (2019), which incorporates an offset called the
“calibrated-$\delta$ adjustment” calculated from auxiliary information. The
algorithm in Pham et al. (2019) handles item nonresponse only, whereas our
algorithm, which we call the intercept matching algorithm, handles both item
and unit nonresponse.

The intercept matching algorithm can be readily generalized to more than two
variables with auxiliary margins, assuming we use the margins to add the
indicator $U$ to the models for the survey variables. This can be done via
repeating IM1–IM5 or IM6–IM10: generate parameter draws from their full
conditional based on the $U_{i}=0$ cases alone, use the auxiliary margin to
adjust the intercept, and draw imputations for this variable for unit
nonrespondents using the adjusted intercept. When the model for $X_{k}$ is a
multinomial logistic regression, as is the case for some variables in the 2018
CPS hybrid missingness MD-AM model, we estimate parameters and set $U$
coefficients in the logit expression for each level of $X_{k}$.

The algorithm also easily incorporates variables without known margins. We add
a conditional model for each $Y_{k}$ to the collection, without including
terms for $U$. For example, for a binary $Y_{k}$, we follow IM3 (or IM8) to
generate parameter draws based on the $U_{i}=0$ cases alone. We skip IM4 (or
IM9) as there is no auxiliary margin associated with $Y_{k}$ for the intercept
adjustment. As a result, we draw imputations of $Y_{k}$ for unit
nonrespondents from the model used for unit respondents, without changing the
intercept. When $Y_{k}$ also suffers from item nonresponse, we follow
IM11–IM12 to draw imputations for cases with $R_{ik}^{Y}=1$. In general, to
facilitate modeling, we recommend specifying models first for $U$ and $X\mid
U$, followed by models for $Y\mid X$, $R^{x}$, and $R^{y}$.

###  3.3 Summary of Simulation Results

In the supplementary material, we present results of a series of simulations
that investigate the performance of the intercept matching algorithm for
fitting the hybrid missingness MD-AM model with unequal probability samples
subject to unit and item nonresponse. The simulations vary the departure from
ignorability for the unit and item nonresponse, and the strength of
association between $X_{1}$ and $X_{2}$. To conserve space, here we summarize
the main findings from those simulations.

We begin with the simulations that assume the design weights for unit
respondents are known. Across all scenarios we investigated, the MD-AM model
fit with the intercept matching algorithm has better repeated sampling
properties than default ICIN models that do not utilize the auxiliary margins.
In particular, when missingness is nonigorable, the MD-AM models generally
result in multiple imputation point estimates with much lower bias than the
default ICIN models. Confidence interval coverage rates for the MD-AM models
generally are close to or exceed 95%, whereas those for the default ICIN model
often have zero coverage due to the bias. Interestingly, the multiple
imputation variance estimates with the MD-AM model are positively biased. This
mirrors a finding in Reiter (2008), who noted that the multiple imputation
variance estimator of Rubin (1987) has positive bias when the imputation
models use more information—in this case, the auxiliary margins—than the
analysis models.

When the design weights for unit respondents are not known, so that we use
(19) to make analysis weights, the performance of the MD-AM model suffers
compared to when we can use (3.1.1). This is because we are forced to estimate
the design weights, and errors in those estimates carry through to the survey
analysis. Nonetheless, the point and interval estimates from the MD-AM model
continue to outperform those from the default ICIN model. In fact, for
settings where the bias due to nonresponse is not extreme, the MD-AM model
produces results with good repeated sampling properties. We refer the reader
to the supplementary material for details.

The bottom line for the simulation studies is that the inferences from the
hybrid missingness MD-AM model with the intercept matching algorithm
outperform those from default models that do not use the margins, even in the
case where only adjusted weights are available. These promising results
encourage us to estimate a hybrid missingness MD-AM model with the intercept
matching algorithm to analyze voter turnout in the 2018 CPS data, to which we
now turn.

##  4 Estimating Voter Turnout with 2018 CPS Data

In every Congressional and Presidential election year since 1964, the Voting
and Registration Supplement has been included biennially in the November basic
monthly survey of the Current Population Survey. The resulting data are
regarded as one of the premier sources for studying turnout in the U. S. The
large sample size and detailed demographic information enable researchers to
examine voter turnout by state and within subgroups, for example, among racial
and ethnic minority groups (Ansolabehere et al., 2022).

In spite of its status as a gold standard for research on turnout, the CPS
suffers from unit and item nonresponse. To handle unit nonresponse, the Census
Bureau adjusts the design weights of CPS survey respondents so that the
resulting weighted estimates match certain demographic totals. To handle item
nonresponse, the Census Bureau uses various imputation methods. Particularly
relevant for turnout studies, the Census Bureau imputes all “Don’t Know,”
“Refused,” and “No Response” answers to the question on turnout as non-voters.
Taken together, the Census Bureau’s treatment of missingness represents very
strong modeling assumptions (Hur and Achen, 2013). To be explicit, according
to the Census Bureau missingness assumptions, unit nonresponse is
conditionally independent of voting status, and item nonresponse on vote
perfectly predicts voting status.

We instead handle the unit and item nonresponse with a hybrid missingness MD-
AM model. We note that Akande et al. (2021) previously analyzed CPS turnout
data with an MD-AM model. Their analysis uses data from the 2012 CPS, treats
the CPS as a simple random sample (i.e., disregards the survey weights), and
uses selection models for all nonresponse indicators rather than the hybrid
missingness mechanism.

###  4.1 Data

We use 2018 CPS data for North Carolina downloaded from the IPUMS website
(https://cps.ipums.org/cps/) and the variables described in Table 1. The data
are collected via a multi-stage design, including stratification within
states, probability proportional to size sampling, and oversampling of certain
demographic populations. As a result, the design weights vary across sampled
units. This variation can be compounded by the weighting adjustments for unit
nonresponse.

The CPS reports unit nonresponse rates at the household level, that is,
whether or not the household responds, rather than the individual person
level. We therefore need to approximate the person level unit nonresponse rate
among those eligible for the November supplement (U. S. citizens at least 18
years old). To do so, we use the strategy developed by Akande et al. (2021),
which we summarize here. First, using information from the CPS data file
describing why sampled households failed to respond, we exclude “Type C”
households; these are deemed ineligible for the survey. Second, we estimate
the average number of adult citizens per household in North Carolina. Our
numerator derives from the Census Bureau’s special tabulation of the Citizen
Voting Age Population (CVAP) using five-year American Community Survey (ACS)
data in North Carolina, and our denominator is the estimated total number of
households in North Carolina also from the five-year ACS data. Third, we
multiply the number of eligible nonresponding households in the CPS data by
the estimated average number of adult citizens per household, and round the
product to the nearest person. We then append records to the observed CPS data
comprising no information (other than that they reside in North Carolina and
are eligible to vote).

Table 1: Description of variables used in the analysis of turnout in the 2018 CPS data, along with notation used for modeling. Variable | Notation | Categories
---|---|---
Sex | S | 0 = Male, 1 = Female
Race | E | 1 = White alone, 2 = Black alone, 3 = Hispanic or Latino, 4 = Remaining people
Education | C | 1 = High school or less (HS-), 2 = Some college, 3 = Bachelor’s and more (BS+)
Age | A | 1 = 18 - 29, 2 = 30 - 39, 3 = 40 - 49, 4 = 50 - 59, 5 = 60 - 69, 6 = 70 - 79, 7 = 80+
Vote | V | 0 = Did not vote; 1 = Voted

Using this approach, we generate 913 unit nonrespondents to add to the 2013
unit respondents; thus, we have $n=2926$ in $\mathcal{D}$ with 31% unit
nonresponse. Two observations in the CPS data have all variables missing
except their survey weights. We treat them as unit nonrespondents, without
modifying their weights. Item nonresponse rates among respondents are modest
among sex (0%), race (2%), education (4%), and age (4%). Item nonresponse
among respondents for vote is more substantial at 18%.

We use auxiliary margins for several of the variables in Table 1. For vote, we
use the total ballots counted voter-eligible population (VEP) turnout rate for
North Carolina in the 2018 election, which is 49%
(http://www.electproject.org/2018g). For sex and race, we utilize auxiliary
margins for the VEP from the 2018 ACS for North Carolina. Specifically, we
have 52% female and 48% male; we have 69.9% with white race alone, 21.8% with
black race alone, 3.9% with Hispanic, and 4.4% comprising people not in these
categories. The standard errors of these statewide ACS estimates are small
enough that we can treat them as known population margins.

###  4.2 Hybrid missingness MD-AM model

For any individual $i\in\mathcal{D}$, let $A_{i}$ denote their age, $S_{i}$
denote their sex, $E_{i}$ denote their race/ethnicity, $C_{i}$ denote their
education, and $V_{i}$ denote their vote. We use this notation instead of
$X_{ik}$ and $Y_{ik}$ to make it easier to identify the variables. Note that
$\\{G,E,V\\}\in\mathcal{A}$ and $\\{A,C\\}\notin\mathcal{A}$. For any
individual $i\in\mathcal{D}$, let
$(U_{i},R^{E}_{i},R^{C}_{i},R^{A}_{i},R^{V}_{i})$ represent the indicators for
unit nonresponse and item nonresponse for race/ethnicity, education, age, and
vote, respectively.

Following the MD-AM framework, we first specify an identifiable model for the
collection of survey variables and nonresponse indicators without using
auxiliary margins. Table 2 summarizes the sequence of conditional models
before using the margins. Exploratory analysis of the missing data patterns
reveals that whenever an individual’s age is missing, that person’s vote also
is missing. Hence, when specifying the default model for $R^{A}$, we cannot
include a term for $V$; there is not sufficient information to identify a
coefficient of vote. We set $R^{V}_{i}=1$ whenever $R^{A}_{i}=1$. In the model
for $V$, we include interactions between $S$ and $E$, $S$ and $C$, and $S$ and
$A$ as we are especially interested in estimating voter turnout in these
subgroups. Overall, the collection of models specifies fewer parameters than
can be identified using the CPS data alone; however, the simplified models
explain the data reasonably well while avoiding the need to estimate extra
parameters, especially in the multinomial models.

Table 2: Hybrid missingness MD-AM model in the analysis of turnout in the 2018 CPS data. Abbreviations for models include “B” for Bernoulli distribution, “MR” for multinomial logistic regression, “LR” for logistic regression. For the predictors, “I” stands for an intercept. Categorical predictors with $d$ levels are modeled with a series of $d-1$ indicator variables. Interaction effects between any two variables $A$ and $B$ represented by $A:B.$ Variable | Model | Predictors
---|---|---
$U$ | B | $I$
$S$ | B | $I$
$E$ | MR | $I+S$
$C$ | MR | $I+S+E$
$A$ | MR | $I+S+E+C$
$V$ | LR | $I+S+E+C+A+S:E+S:C+S:A$
$R^{E}$ | LR | $I+S+C+A+V$
$R^{C}$ | LR | $I+S+E+A+V$
$R^{A}$ | LR | $I+S+E+C$
$R^{V}$ | LR | $(R^{A}=0):I+S+E+C+A$
|  | $(R^{A}=1):R^{V}=1$

Following step 2 of the MD-AM framework with hybrid missingness, we add main
effects for $U$ to the models for $S$, $E$, and $V$. We do not add $U$ to the
models for $A$ and $C$ since we do not utilize auxiliary margins on age and
education.

As the CPS file from IPUMS includes only adjusted weights, we apply the
algorithm in (19) to create the analysis weights. We use non-informative
priors for all model parameters and normal approximations for the coefficients
of all regressions. We employ the intercept matching algorithm for 10000
iterations and discard the first 5000 iterations as burn-in. Using a non-
optimized code in $R$ and a standard laptop, it takes about 14 hours to
generate the 10000 draws. Evaluations of trace plots of model parameters
suggest that the sampler converges. Among the 5000 posterior samples, we
retrieve every 100th posterior sample to create $L=50$ multiple imputation
data sets, and do survey-weighted multiple imputation inference using those
completed data sets.

###  4.3 Results

We begin by examining the completed-data marginal distributions of the
variables other than vote, i.e., $S$, $E$, $C$ and $A$. Table 3 displays the
multiple imputation point estimates, $\bar{q}_{L}$, and corresponding multiple
imputation standard errors using the 50 completed data sets generated under
the hybrid missingness MD-AM model. Survey-weighted estimates are computed
with the “survey” package (Lumley, 2021) in $R$, assuming each completed data
set is a probability proportional to size with replacement sample. This is a
common estimation strategy for analyzing complex surveys (Lohr, 2010). For
comparisons, we also generate 50 completed data sets using a default
implementation of MICE with only the unit respondents. Estimates for the MICE-
completed data sets are based on the CPS adjusted weights on the IPUMS file.
Finally, we also display weighted estimates based on complete cases.

Table 3: Estimated marginal distributions of sex, age, and race based on 50 imputations generated from the hybrid missingness MD-AM model and default MICE. The entries under the column labeled $Q$ are the available population percentages used as auxiliary margins. The entries in the columns labeled CC are survey-weighted estimates using the complete cases.  |  | MD-AM | MICE | CC
---|---|---|---|---
| $Q$ | $\bar{q}_{L}$ | SE | $\bar{q}_{L}$ | SE | Est | SE
Male | .48 | .478 | .017 | .470 | .011 | .466 | .013
Female | .52 | .522 | .017 | .530 | .011 | .534 | .013
White | .699 | .699 | .013 | .696 | .011 | .709 | .012
Black | .218 | .219 | .013 | .220 | .010 | .213 | .011
Hispanic | .039 | .038 | .005 | .035 | .004 | .033 | .005
Rest | .044 | .044 | .005 | .049 | .005 | .046 | .005
$(0,29]$ |  | .213 | .014 | .216 | .010 | .205 | .011
$(29,39]$ |  | .160 | .021 | .141 | .008 | .144 | .009
$(39,49]$ |  | .189 | .018 | .161 | .009 | .150 | .009
$(49,59]$ |  | .162 | .013 | .173 | .009 | .177 | .009
$(59,69]$ |  | .147 | .010 | .161 | .008 | .168 | .009
$(69,79]$ |  | .092 | .009 | .103 | .007 | .111 | .008
$>79$ |  | .036 | .005 | .043 | .004 | .045 | .005
HS- |  | .372 | .012 | .375 | .011 | .366 | .012
Some College |  | .297 | .011 | .300 | .011 | .306 | .012
BA+ |  | .331 | .012 | .325 | .011 | .328 | .012
Table 4: Estimated proportion who voted in subgroups based on 50 imputations generated from the hybrid missingness MD-AM model and default MICE. Survey-weighted estimates for complete case analyses (CC) and estimates released by the Census Bureau (Census) also included. The auxiliary margin for voted in North Carolina is .49. | MD-AM | MICE | CC | Census
---|---|---|---|---
| $\bar{q}_{L}$ | SD | $\bar{q}_{L}$ | SD | Est | SD | Est | SD
Full | .501 | .016 | .630 | .012 | .637 | .012 | .524 | .012
Male | .499 | .021 | .630 | .018 | .639 | .018 | .521 | .017
Female | .504 | .020 | .631 | .017 | .636 | .017 | .526 | .016
White | .510 | .018 | .645 | .014 | .648 | .014 | .544 | .013
Black | .519 | .032 | .647 | .031 | .658 | .030 | .516 | .027
Hispanic | .390 | .061 | .494 | .070 | .489 | .073 | .386 | .061
Rest | .378 | .057 | .449 | .058 | .467 | .060 | .371 | .049
$(0,29]$ | .325 | .027 | .419 | .030 | .421 | .030 | .331 | .026
$(29,39]$ | .398 | .038 | .558 | .033 | .571 | .034 | .466 | .031
$(39,49]$ | .492 | .041 | .679 | .029 | .681 | .030 | .517 | .028
$(49,59]$ | .595 | .038 | .698 | .027 | .700 | .027 | .596 | .026
$(59,69]$ | .700 | .035 | .796 | .024 | .794 | .024 | .664 | .025
$(69,79]$ | .672 | .039 | .758 | .030 | .761 | .030 | .691 | .031
$>79$ | .467 | .060 | .545 | .055 | .556 | .054 | .485 | .051
HS- | .347 | .020 | .456 | .021 | .464 | .021 | .369 | .018
Some College | .519 | .023 | .653 | .022 | .653 | .022 | .552 | .021
BA+ | .659 | .029 | .810 | .017 | .813 | .017 | .676 | .018
Table 5: Estimated proportion who voted in detailed subgroups based on 50 imputations generated from the hybrid missingness MD-AM model and default MICE. Survey-weighted estimates for complete case analyses (CC) and estimates released by the Census Bureau (Census) also included. The auxiliary margin for voted in North Carolina is .49. | MD-AM | MICE | CC | Census
---|---|---|---|---
| $\bar{q}_{L}$ | SD | $\bar{q}_{L}$ | SD | Est | SD | Est | SD
Male, White | .513 | .023 | .648 | .020 | .650 | .020 | .550 | .040
Male, Black | .475 | .047 | .612 | .046 | .618 | .045 | .463 | .018
Male, Hispanic | .459 | .100 | .562 | .103 | .597 | .106 | .443 | .086
Male, Rest | .427 | .081 | .509 | .086 | .552 | .090 | .416 | .074
Female, White | .508 | .023 | .642 | .019 | .647 | .019 | .539 | .018
Female, Black | .557 | .038 | .676 | .039 | .687 | .040 | .559 | .037
Female, Hispanic | .323 | .080 | .432 | .094 | .401 | .100 | .329 | .085
Female, Rest | .334 | .072 | .396 | .076 | .397 | .078 | .332 | .066
Male, $(0,29]$ | .313 | .038 | .405 | .042 | .408 | .043 | .317 | .036
Male, $(29,39]$ | .396 | .049 | .553 | .048 | .573 | .049 | .460 | .045
Male, $(39,49]$ | .451 | .053 | .640 | .045 | .639 | .046 | .482 | .042
Male, $(49,59]$ | .577 | .048 | .694 | .037 | .689 | .038 | .596 | .037
Male, $(59,69]$ | .714 | .049 | .813 | .034 | .810 | .035 | .656 | .037
Male, $(69,79]$ | .764 | .050 | .828 | .040 | .840 | .039 | .781 | .042
Male, $>79$ | .460 | .094 | .561 | .090 | .565 | .091 | .469 | .083
Female, $(0,29]$ | .337 | .038 | .433 | .041 | .432 | .043 | .344 | .037
Female, $(29,39]$ | .401 | .050 | .562 | .045 | .569 | .046 | .472 | .042
Female, $(39,49]$ | .530 | .048 | .710 | .037 | .714 | .038 | .545 | .037
Female, $(49,59]$ | .594 | .046 | .702 | .039 | .711 | .037 | .596 | .037
Female, $(59,69]$ | .687 | .043 | .780 | .034 | .781 | .034 | .671 | .035
Female, $(69,79]$ | .596 | .054 | .700 | .044 | .694 | .044 | .619 | .044
Female, $>79$ | .473 | .072 | .535 | .068 | .551 | .068 | .494 | .065
Male, HS- | .347 | .026 | .459 | .029 | .468 | .030 | .369 | .025
Male, Some College | .520 | .035 | .660 | .032 | .656 | .033 | .552 | .031
Male, BA+ | .677 | .039 | .835 | .025 | .837 | .024 | .699 | .027
Female, HS- | .348 | .026 | .452 | .030 | .460 | .029 | .370 | .026
Female, Some College | .519 | .031 | .648 | .030 | .650 | .030 | .552 | .029
Female, BA+ | .645 | .034 | .791 | .024 | .794 | .024 | .658 | .025
White, HS- | .341 | .023 | .445 | .022 | .452 | .025 | .374 | 022
White, Some College | .512 | .026 | .647 | .024 | .654 | .026 | .556 | .024
White, BA+ | .663 | .030 | .812 | .017 | .826 | .018 | .694 | .019
White, Male, HS- | .341 | .031 | .452 | .035 | .453 | .035 | .378 | .031

Overall, estimates are similar across all the survey-weighted analyses. This
is not surprising, as the CPS adjusts design weights so that the survey-
weighted estimates for some of these demographic variables approximately match
their known demographic margins. We point out that, for $S$ and $E$ where we
have auxiliary margins, the multiple imputation standard deviations for the
MD-AM model are biased high. The true standard deviation for each of these two
$\bar{q}_{L}$ is close to zero, as the model essentially forces each of these
two $\bar{q}_{L}$ to match their corresponding known margins.

The benefits of handling item and unit nonresponse with the hybrid missingness
MD-AM model become apparent when we examine voter turnout. Table 4 displays
estimates of turnout by each category of the demographic variables for the
three analysis strategies. Additionally, Table 4 includes the turnout
estimates using the Census Bureau procedures for handling missing values. That
is, we use the CPS data for unit respondents and their survey weights on the
file; we use the Census Bureau’s imputations for $S$, $E$, $A$, and $C$; and,
we set all missing responses for $V$ as not voting.

Statewide, the MD-AM model estimates around 50.1% of CPS participants are
voters, which is close to the population turnout of 49%. In contrast,
estimates from MICE and complete cases overestimate the turnout rate overall
and seemingly for most subgroups—a bias likely more than 10 percentage points
in many instances. For subgroups, estimates of turnout under the MD-AM model
tend to be more plausible than those under MICE and the complete case
analyses. For example, college graduates are estimated to have a turnout rate
of more than 80% using complete case or MICE analysis, where as the MD-AM
estimates a turnout rate around 66%.

Table 5 displays turnout for bivariate combinations of the demographic
variables. Here again, the MD-AM model estimates are smaller and more
plausible than those under MICE and complete case analysis. For some of the
most substantively interesting population subgroups, the MD-AM estimates very
different turnout rates than the MICE or complete case analyses. For example,
political pundits have focused considerable attention on voting behavior of
non-college educated white men and women. Whereas the complete case analysis
estimated they had a turnout rate of 45% in 2018, the MD-AM estimates their
turnout rate at just 34%. As another example, the MD-AM model estimates that
Black men had a much lower turnout rate (47.5%, which is in line with the
statewide turnout rate) compared to the complete case estimates (61.8%, over
10 points higher than the statewide rate).

The results from the MD-AM model and Census Bureau imputations are somewhat
similar. By imputing all item nonrespondents on vote to be non-voters, the
Census Bureau essentially is forcing the overall turnout margin to be closer
to 49% than the complete case results. Still, the overall turnout estimates
under the MD-AM model are closer to 49%. Additionally, it seems implausible
that all item nonrespondents are non-voters. In contrast, the MD-AM model
imputes on average 60% of the item nonrespondents as voters—close to the
complete cases turnout estimate of 63.7%, as expected from the ICIN assumption
for $R^{V}$—and, to make the completed-data design-based estimates of turnout
closely match 49%, on average 22% of the unit nonrespondents as voters.

While it is impossible to assess how accurately the MD-AM model describes the
nonresponse mechanism in the CPS, we can assess how well the MD-AM model
reproduces the observed data. To do so, we compare unweighted, observed
percentages from $\mathcal{D}$ with estimates from replicated data generated
from posterior predictive distributions (He and Zaslavsky, 2012). These
comprise the values of
$\\{(A_{i},S_{i},E_{i},C_{i},V_{i}):U_{i}=R^{A}_{i}=R_{i}^{E}=R_{i}^{C}=R_{i}^{V}=0\\}$.
Using each of the 5000 draws of the parameters, we generate new values of all
variables for all $n$ units in $\mathcal{D}$, and compute the replicated data
percentages for those generated to be respondents to all questions. We
construct 95% intervals for marginal probabilities for the replicated data for
all variables and voter turnout probabilities for subgroups. Figure 2 displays
the posterior predictive intervals. All intervals contain the observed data
estimates, giving us confidence in the reasonableness of the hybrid
missingness MD-AM model for the observable portion of the data.

Figure 2: Posterior predictive intervals for all marginal and conditional
probabilities involving vote for completed data only. Crosses are observed
data estimates. The first panel shows marginal probabilities of vote, sex,
race/ethnicity, age and education. The second panel shows voter turnout
probabilities in various sex, race/ethnicity, age and education subgroups. The
third panel displays voter turnout probabilities in sex crossed with
race/ethnicity, gender crossed with age and sex crossed with education
subgroups.

The simulation studies in the supplementary material suggest that, since
design weights for unit respondents are not available, the performance of the
MD-AM model in estimating voter turnout should depend on the strength of the
unit nonresponse bias. The similarity of estimates in Table 3 for the MD-AM
model, MICE and complete case analysis suggest practically irrelevant biases
for these three variables due to unit nonresponse (after the weighting
adjustments). On the contrary, the voter turnout estimated under the MD-AM
model is lower than the turnout under MICE and complete case analysis,
suggesting the unit nonresponse bias on vote is important. The simulation
results under such scenarios in the supplementary material (namely, scenarios
“b” or “d” in Table 8 in the supplement) suggest that results from the MD-AM
model are more accurate than those from models that do not utilize the
information in the margins.

###  4.4 Incorporating measurement error

Beyond the effects of item and unit nonresponse, it is well-recognized that
self-reported voter turnout suffers from measurement error. More specifically,
people often report that they voted when they did not in fact do so. In
contrast, people who did vote rarely report that they did not (Enamorado and
Imai, 2019; Jackman and Spahn, 2019; DeBell et al., 2020). Thus, some of the
apparent bias in the CPS complete cases results could be the result of over-
reporting of turnout.

Figure 3: Posterior distribution of voter turnout among item nonrespondents,
respondents, and unit nonrespondents for the MD-AM model. For each response
type, the box to the right represents the turnout rate in the MD-AM model
assuming no measurement error among respondents’ vote answers. The box to the
left represents the turnout rate in the MD-AM model assuming measurement
error.

To assess the sensitivity of results to reporting error, we layer a
measurement error model on top of the hybrid missingness MD-AM model. Previous
research suggests a dose-response relationship between over-reporting of vote
and education: increasing education is associated with increasing rates of
over-reporting (Silver et al., 1996; Karp and Brockington, 2005).
Unfortunately, we do not have a validated sample from the CPS to assess the
over-reporting rate by educational attainment. Thus, we represent the
measurement error in vote using informative prior distributions.

For each individual $i\in\mathcal{D}$ among those who have a response to the
question about vote, let $Z_{i}=1$ when the person is reported as a voter, and
let $Z_{i}=0$ when the person is reported as a nonvoter. We seek to tie each
$Z_{i}$ to the underlying true voting status, $V_{i}$. Note that we define
$Z_{i}$ only for units with $U_{i}=R^{V}_{i}=0$. Let the misreporting
probabilities for nonvoters given their education level be as follows.

$\displaystyle\mathbb{P}(Z_{i}=1|V_{i}=0,C_{i}=1,U_{i}=R^{V}_{i}=0)$
$\displaystyle=\theta_{1}$ (22)
$\displaystyle\mathbb{P}(Z_{i}=1|V_{i}=0,C_{i}=2,U_{i}=R^{V}_{i}=0)$
$\displaystyle=\theta_{2}$ (23)
$\displaystyle\mathbb{P}(Z_{i}=1|V_{i}=0,C_{i}=3,U_{i}=R^{V}_{i}=0)$
$\displaystyle=\theta_{3},$ (24)

where $C_{i}\in\\{1,2,3\\}$ per Table 1. Among nonvoters who have at most a
high school education, we assume around 6% would misreport that they actually
voted ($\theta_{1}$). Among nonvoters with some college education, we assume
around 13% would misreport that they actually voted ($\theta_{2}$). Among
nonvoters who have at least a bachelor’s degree, we assume around 19% would
misreport that they voted ($\theta_{3}$). We treat these best guesses as
measures of central tendency in Beta prior distributions. For each
$\theta_{c}$, we assume $\theta_{c}\sim Beta(a_{c},b_{c})$, where
$(a_{1},b_{1})=(60,940)$, $(a_{2},b_{2})=(130,870)$, and
$(a_{3},b_{3})=(190,810)$. These correspond to prior means of misreporting
rates of $(0.06,0.13,0.19)$, respectively, and prior standard deviations
around 1%. In accordance with past research from other studies, we assume that
anyone who truly voted is reported that they did so. We select these prior
distributions based on the research findings from previous voter validation
studies that have matched survey respondents to voter files (e.g., Jackman and
Spahn, 2019), accounting for differences in observed bias across election and
survey type.

Adding the measurement error models and prior distributions to the MD-AM model
in Section 4.2 requires two additional steps in the intercept matching
algorithm: update $\theta_{c}$, and sample true voting status $V_{i}$ for
individuals with $(U_{i}=0,R^{V}_{i}=0,Z_{i}=1)$. The full conditionals for
these updates are described in the supplementary material.

Figure 3 displays estimated voter turnout rates among respondents, item
nonrespondents on vote, and unit nonrespondents before and after taking the
measurement error into consideration. With the additional measurement error
assumptions, the MD-AM model changes some of the reported voters to nonvoters,
lowering the voting proportion among observed cases. Under the ICIN model for
$R^{V}$, we therefore impute fewer voters among respondents with
$R^{V}_{i}=1$. Consequently, we need to impute more voters among unit
nonrespondents to match the auxiliary margin for vote.

Tables containing results from the hybrid missingness MD-AM model with
measurement error like those in Table 3, Table 4, and Table 5 are displayed in
the supplementary material. Overall, despite the measurement error, the
estimates are quite similar with a few estimates increasing or decreasing by a
point or two, and all changes are well within one estimated standard error.

##  5 Concluding Remarks

The hybrid missingness MD-AM model allows analysts to take advantage of known
auxiliary margins when imputing values for unit and item nonresponse in
surveys with complex sampling designs. In the 2018 CPS analysis, we enriched
the models to account for unit nonresponse, as it was more prevalent than item
nonresponse. In other situations, analysts may prefer to enrich one or more of
the models for the item nonresponse indicators. We explored adding an
indicator for $V$ to the model for $R^{V}$ and removing $U$ from the model for
$V$ using a rejection sampler akin to that used in Akande and Reiter (2022).
However, with diverse values of weights, our sampler had a difficult time
exploring the parameter space adequately. Further research is needed on
computational algorithms for the version of the hybrid missingness MD-AM model
that enriches item nonresponse models.

The simulation studies demonstrate that the hybrid missingness MD-AM results
are more accurate when we use the unit respondents’ design weights as opposed
to having to down-weight adjusted weights. This finding suggests that survey
providers would help analysts by including design weights in survey data
files, even when they also include adjusted weights on those files.

## References

  * Akande and Reiter (2022) Akande, O. and Reiter, J. P. (2022).  Multiple imputation for nonignorable item nonresponse in complex surveys using auxiliary margins.  In A. L. Carriquiry, J. M. Tanur, and W. F. Eddy, editors, Statistics in the Public Interest: In Memory of Stephen E. Fienberg, pages 289–306. Springer, Berlin.
  * Akande et al. (2021) Akande, O., Madson, G., Hillygus, D. S., and Reiter, J. P. (2021).  Leveraging auxiliary information on marginal distributions in nonignorable models for item and unit nonresponse in surveys.  Journal of the Royal Statistical Society: Series A, 184, 643–662.
  * Ansolabehere et al. (2022) Ansolabehere, S., Fraga, B. L., and Schaffner, B. F. (2022).  The CPS voting and registration supplement overstates minority turnout.  Journal of Politics, page forthcoming.
  * Azur et al. (2011) Azur, M. J., Stuart, E. A., Frangakis, C., and Leaf, P. J. (2011).  Multiple imputation by chained equations: what is it and how does it work?  International Journal of Methods in Psychiatric Research, 20, 40–49.
  * DeBell et al. (2020) DeBell, M., Krosnick, J. A., Gera, K., Yeager, D. S., and McDonald, M. P. (2020).  The turnout gap in surveys: Explanations and solutions.  Sociological Methods & Research, 49(4), 1133–1162.
  * Deng et al. (2013) Deng, Y., Hillygus, D. S., Reiter, J. P., Si, Y., and Zheng, S. (2013).  Handling attrition in longitudinal studies: the case for refreshment samples.  Statistical Science, 28, 238–256.
  * Enamorado and Imai (2019) Enamorado, T. and Imai, K. (2019).  Validating self-reported turnout by linking public opinion surveys with administrative records.  Public Opinion Quarterly, 83, 723–748.
  * Fuller (2009) Fuller, W. A. (2009).  Sampling Statistics.  John Wiley & Sons, Inc, New York.
  * He and Zaslavsky (2012) He, Y. and Zaslavsky, A. M. (2012).  Diagnosing imputation models by applying target analyses to posterior replicates of completed data.  Statistics in Medicine, 31, 1–18.
  * Hirano et al. (2001) Hirano, K., Imbens, G. W., Ridder, G., and Rubin, D. B. (2001).  Combining panel data sets with attrition and refreshment samples.  Econometrica, 69, 1645–1659.
  * Horvitz and Thompson (1952) Horvitz, D. G. and Thompson, D. J. (1952).  A generalization of sampling without replacement from a finite universe.  Journal of the American Statistical Association, 47, 663–685.
  * Hur and Achen (2013) Hur, A. and Achen, C. H. (2013).  Coding voter turnout responses in the current population survey.  Public Opinion Quarterly, 77, 985–993.
  * Jackman and Spahn (2019) Jackman, S. and Spahn, B. (2019).  Why does the american national election study overestimate voter turnout?  Political Analysis, 27(2), 193–207.
  * Karp and Brockington (2005) Karp, J. A. and Brockington, D. (2005).  Social desirability and response validity: A comparative analysis of overreporting voter turnout in five countries.  Journal of Politics, 67, 825–840.
  * Linero and Daniels (2018) Linero, A. R. and Daniels, M. J. (2018).  Bayesian approaches for missing not at random outcome data: The role of identifying restrictions.  Statistical Science, 33, 198–213.
  * Lohr (2010) Lohr, S. L. (2010).  Sampling: Design and Analysis. 2nd Edition.  Brooks/Cole, Boston.
  * Lumley (2021) Lumley, T. (2021).  Analysis of complex survey samples.  R ($>=$ 3.5.0), grid, methods, Matrix, survival.
  * Mealli and Rubin (2015) Mealli, F. and Rubin, D. B. (2015).  Clarifying missing at random and related definitions, and implications when coupled with exchangeability.  Biometrika, 102, 995–1000.
  * Pham et al. (2019) Pham, T. M., Carpenter, J. R., Morris, T. P., Wood, A. M., and Petersen, I. (2019).  Population-calibrated multiple imputation for a binary categorical covariate in categorical regression models.  Statistics in Medicine, 38, 792–808.
  * Polson et al. (2013) Polson, N. G., Scott, J. G., and Windle, J. (2013).  Bayesian inference for logistic models using Pólya-–Gamma latent variables.  Journal of the American Statistical Association, 108, 1339–1349.
  * Raghunathan et al. (2001) Raghunathan, T. E., Lepkowski, J. M., Hoewyk, J. V., and Solenberger, P. (2001).  A multivariate technique for multiply imputing missing values.  Survey Methodology, 27, 85–95.
  * Reiter (2008) Reiter, J. P. (2008).  Multiple imputation when records used for imputation are not used or disseminated for analysis.  Biometrika, 95, 933–946.
  * Rubin (1976) Rubin, D. B. (1976).  Inference and missing data.  Biometrika, 63, 581–592.
  * Rubin (1987) Rubin, D. B. (1987).  Multiple Imputation For Nonresponse In Surveys.  John Wiley & Sons, New York.
  * Sadinle and Reiter (2017) Sadinle, M. and Reiter, J. P. (2017).  Itemwise conditionally independent nonresponse modelling for incomplete multivariate data.  Biometrika, 104, 207–220.
  * Sadinle and Reiter (2019) Sadinle, M. and Reiter, J. P. (2019).  Sequentially additive nonignorable missing data modelling using auxiliary marginal information.  Biometrika, 106, 889–911.
  * Schifeling et al. (2015) Schifeling, T., Cheng, C., Reiter, J. P., and Hillygus, D. S. (2015).  Accounting for nonignorable unit nonresponse and attrition in panel studies with refreshment samples.  Journal of Survey Statistics and Methodology, 3, 265–295.
  * Si and Zhou (2021) Si, Y. and Zhou, P. (2021).  Bayes-raking: Bayesian finite population inference with known margins.  Journal of Survey Statistics and Methodology, 8, 833–855.
  * Si et al. (2016) Si, Y., Reiter, J. P., and Hillygus, D. S. (2016).  Bayesian latent pattern mixture models for handling attrition in panel studies with refreshment samples.  Annals of Applied Statistics, 10, 118–143.
  * Silver et al. (1996) Silver, B. D., Anderson, B. A., and Abramson., P. R. (1996).  Who overreports voting?  The American Political Science Review, 80, 613–24.
  * Tang (2022) Tang, J. (2022).  Bayesian Models for Combining Information from Multiple Sources.  Ph.D. thesis, Duke University.
  * Valliant et al. (2013) Valliant, R., Dever, J. A., and Kreuter, F. (2013).  Practical Tools for Designing and Weighting Sample Surveys.  Springer-Verlag, New York.