#  lmw: Linear Model Weights for Causal Inference††thanks: We thank Bijan
Niknam for helpful commments. This work was supported by the Alfred P. Sloan
Foundation (G-2020-13946) and the Patient-Centered Outcomes Research Institute
(PCORI, ME-2022C1-25648).

Ambarish Chattopadhyay Stanford Data Science, Stanford University, 450 Jane
Stanford Way Wallenberg, Stanford, CA 94305; email<EMAIL_ADDRESS>Noah Greifer Institute for Quantitative Social Sciences (IQSS), Harvard
University, CGIS Knafel Building, Office K303, 1737 Cambridge Street,
Cambridge, MA 02138; emai<EMAIL_ADDRESS>José R. Zubizarreta
Departments of Health Care Policy, Biostatistics, and Statistics, Harvard
University, 180 A Longwood Avenue, Office 307-D, Boston, MA 02115; email:
<EMAIL_ADDRESS>
###### Abstract

The linear regression model is widely used in the biomedical and social
sciences as well as in policy and business research to adjust for covariates
and estimate the average effects of treatments. Behind every causal inference
endeavor there is at least a notion of a randomized experiment. However, in
routine regression analyses in observational studies, it is unclear how well
the adjustments made by regression approximate key features of randomization
experiments, such as covariate balance, study representativeness, sample
boundedness, and unweighted sampling. In this paper, we provide software to
empirically address this question. In the new lmw package for R, we compute
the implied linear model weights for average treatment effects and provide
diagnostics for them. The weights are obtained as part of the design stage of
the study; that is, without using outcome information. The implementation is
general and applicable, for instance, in settings with instrumental variables
and multi-valued treatments; in essence, in any situation where the linear
model is the vehicle for adjustment and estimation of average treatment
effects with discrete-valued interventions.

Keywords: Causal Inference; Linear Models; R Programming Language; Regression
Modeling; Observational Studies

###### Contents

  1. 1 Introduction
  2. 2 Setup
  3. 3 Methods
    1. 3.1 Uni-regression imputation
    2. 3.2 Multi-regression imputation
    3. 3.3 Extensions
  4. 4 Data
  5. 5 Using lmw
    1. 5.1 URI for the ATE
    2. 5.2 MRI for the ATT
    3. 5.3 MRI after matching for the ATT
    4. 5.4 Multi-valued treatments
    5. 5.5 Two-stage least squares

##  1 Introduction

In observational studies, the linear regression model is widely used to
estimate the causal effect of a treatment, intervention, or exposure on an
outcome variable. Examples are ubiquitous across the biomedical and social
sciences, as well as in policy and business research. In common practice, the
linear regression model is fitted by least squares, including as explanatory
or predictor variables of the outcome a treatment indicator plus a series of
control variables or covariates. The estimated coefficient is interpreted as
the average effect of the treatment on the response variable, holding the
other predictors constant (e.g., Angrist and Pischke 2008, Chapter 3; Imbens
and Rubin 2015, Section 12.2.4).

The linear regression model has been extensively studied. Among its properties
is that the least squares estimator is the best (minimum variance) linear
unbiased estimator of the treatment indicator coefficient, provided the model
is correctly specified. Furthermore, the model is easy to fit and implement in
practice, and is broadly accepted across disciplines as a valid method for
data analysis. However, when it comes to estimating the causal effect of a
treatment, typical use of regression can blur certain notions and principles
that are central to causal inference in observational studies.

One of them is the idea of approximating or emulating the randomized
experiment that would have been conducted under controlled circumstances (Dorn
1953; Cochran 1965; Rosenbaum 2010; Hernán and Robins 2016). Underlying, is
the notion of balance, that is, of constructing treatment and control groups
that are comparable in terms of confounding factors. Also central is a clear
definition of the target population to which the inferences apply. Finally,
another important consideration is the extent to which the estimator
extrapolates beyond the region restricted by the observed data and relies on
parametric assumptions that are untestable. In routine practice, it is unclear
how linear regression adheres to these principles.

Arguably, other methods for adjustment in observational studies, such as
matching and weighting, follow these principles more explicitly than
regression in common practice. For instance, weighting adjusts the treatment
and control groups in order to balance their observed covariate distributions,
characterizing the target population in an open manner, and producing an
estimator that is sample bounded. Moreover, weighting does not require outcome
information to complete the adjustment process. As argued by Rubin (2007),
this helps to preserve the objectivity and validity of an observational study.

Recently, Chattopadhyay and Zubizarreta (2022) offered a connection between
weighting and linear regression methods for causal inference. In particular,
they obtained new closed form expressions for the implicit weights in
different methods involving the linear regression model. See Abadie et al.
(2015), Imbens (2015), Gelman and Imbens (2018), and Ben-Michael et al. (2021)
for related analyses. Chattopadhyay and Zubizarreta (2022) examined properties
of the implied weights of linear regression in both finite- and large-sample
regimes. In finite samples, they showed that the implied weights of general
regression problems can be obtained equivalently by solving a convex
optimization problem akin to the stable balancing weights (Zubizarreta 2015),
which involve minimizing the variance of the weights subject to constraints on
covariate balance. In large samples, they studied multiply robust properties
of regression estimators from the perspective of their implied weights. Among
others, they proposed new regression diagnostics for causal inference that are
part of the design stage of an observational study.

In this paper, we implement and demonstrate this weighting approach to
regression. In particular, we illustrate the new _linear model weights_ lmw
package for R, which is freely available on CRAN. Using it, we explore a
number of settings. For this, in Section 2 we present the notation, estimands,
and basic assumptions. In Section 3, we discuss two regression approaches,
termed uni- and multi-regression imputation. In Section 4, we introduce our
running example. In Section 5, we illustrate the use of the lmw package in
settings with binary treatments, multivariate matching, multi-valued
treatments, and instrumental variables, explaining step by step the code.

##  2 Setup

Consider an observational study based on a random sample of $n$ independent
and identically distributed units drawn from a population. For each unit
$i\in\\{1,2,...,n\\}$, let $A_{i}$ be their treatment assignment indicator,
with $A_{i}=1$ if the unit is assigned to treatment, and $A_{i}=0$ otherwise;
$\bm{X}_{i}$ be a vector of $k$ observed covariates; and $Y_{i}$ be the
observed outcome. Define $n_{t}$ and $n_{c}$ as the sample sizes of the
treatment and control groups, respectively. Denote $\bar{\bm{X}}_{t}$ and
$\bm{V}_{t}$, and $\bar{\bm{X}}_{c}$ and $\bm{V}_{c}$, as the mean vector and
covariance matrix of the covariates in the treatment and control groups,
correspondingly. Write $\bar{\bm{X}}$ for the mean vector of the covariates in
the entire sample.

Following the potential outcome framework (Neyman 1923,1990; Rubin 1974), let
$Y_{i}(a)$ be the potential outcome of unit $i$ under treatment
$a\in\\{0,1\\}$ and $Y^{\text{obs}}_{i}=A_{i}Y_{i}(1)+(1-A_{i})Y_{i}(0)$ be
their observed outcome. In this paper, we are interested in two causal
estimands: the average treatment effect
$\text{ATE}=\mathbb{E}[Y_{i}(1)-Y_{i}(0)]$ and the average treatment effect on
the treated $\text{ATT}=\mathbb{E}[Y_{i}(1)-Y_{i}(0)|A_{i}=1]$. For
identification of these estimands, for the most of this paper we assume the
following conditions hold (Rosenbaum and Rubin 1983): (a) positivity:
$0<\Pr(A_{i}=1\mid\bm{X}_{i}=\bm{x})<1$ for all
$\bm{x}\in\text{supp}(\bm{X}_{i})$, and (b) unconfoundedness:
$A_{i}\perp\\!\\!\\!\perp\\{Y_{i}(0),Y_{i}(1)\\}\mid\bm{X}_{i}$.

##  3 Methods

Following Chattopadhyay and Zubizarreta (2022), we focus on two common
regression modeling approaches. In the first approach, the outcome is
regressed on the covariates and the treatment indicator without any treatment-
covariate interactions, and the average treatment effect is estimated by the
coefficient of the treatment indicator. This approach is called uni-regression
imputation (URI) because it is equivalent to an imputation estimator based on
a single (uni) linear outcome model. In the second approach, the outcome is
regressed on a similar model that adds treatment-covariate interactions. This
approach is called multi-regression imputation (MRI) because it is equivalent
to an imputation estimator based on two (multi) linear outcome models, one
fitted in each treatment group.

###  3.1 Uni-regression imputation

URI is arguably the most commonly used method for regression adjustments in
observational studies (Aronow and Samii 2016, Angrist and Pischke 2008 Chapter
3). The URI estimator of the ATE is
$\hat{\tau}^{\scriptscriptstyle\text{OLS}}$, the ordinary least squares (OLS)
estimator of $\tau$ in the model
$Y^{\text{obs}}_{i}=\beta_{0}+\bm{\beta}^{\top}_{1}\bm{X}_{i}+\tau
A_{i}+\epsilon_{i}$. Implementations of URI and associated inferences are
straightforward and can be done using, e.g., the lm function in R. However,
standard software implementations do not make clear how faithfully URI designs
an observational study in terms of approximating features of randomized
experiments.

To address this concern, we consider the implied weighting framework of
Chattopadhyay and Zubizarreta (2022). This framework represents the URI
estimator as a weighting estimator, i.e.,
$\hat{\tau}^{{\scriptscriptstyle\text{OLS}}}=\sum_{i:A_{i}=1}w^{{\scriptscriptstyle\text{URI}}}_{i}Y^{\text{obs}}_{i}-\sum_{i:A_{i}=0}w^{{\scriptscriptstyle\text{URI}}}_{i}Y^{\text{obs}}_{i}$,
where
$w^{{\scriptscriptstyle\text{URI}}}_{i}=\frac{1}{n_{t}}+(\bm{X}_{i}-\bar{\bm{X}}_{t})^{\top}(n_{t}\bm{V}_{t}+n_{c}\bm{V}_{c})^{-1}(\bar{\bm{X}}_{c}-\bar{\bm{X}}_{t})$
if unit $i$ is in the treatment group and
$w^{{\scriptscriptstyle\text{URI}}}_{i}=\frac{1}{n_{c}}+(\bm{X}_{i}-\bar{\bm{X}}_{c})^{\top}(n_{t}\bm{V}_{t}+n_{c}\bm{V}_{c})^{-1}(\bar{\bm{X}}_{t}-\bar{\bm{X}}_{c})$
if unit $i$ is in the control group. The weights
$w^{{\scriptscriptstyle\text{URI}}}_{i}$ sum to one within each group and,
moreover, are functions of the covariates and treatment but not the outcome.
Thus, these implied weights make precise how URI acts on treated and control
units in the sample without using outcome information. As a result, we can
characterize in finite samples the adjustments under URI in terms of covariate
balance, study representativeness, model extrapolation, and weight dispersion.

In particular, the URI weights exactly balance the means of the covariates,
but towards a population that is different from the “natural” populations
corresponding to treated population, the control population, or the overall
study population, i.e.,
$\sum_{i:A_{i}=1}w^{{\scriptscriptstyle\text{URI}}}_{i}\bm{X}_{i}=\sum_{i:A_{i}=0}w^{{\scriptscriptstyle\text{URI}}}_{i}\bm{X}_{i}=\bm{X}^{*{\scriptscriptstyle\text{URI}}}:=n_{c}\bm{V}_{c}(n_{t}\bm{V}_{t}+n_{c}\bm{V}_{c})^{-1}\bar{\bm{X}}_{t}+n_{t}\bm{V}_{t}(n_{t}\bm{V}_{t}+n_{c}\bm{V}_{c})^{-1}\bar{\bm{X}}_{c}$.
Also, the URI weights can be negative, which may lead to extrapolation beyond
the support of the observed data. Finally, the URI weights are optimal in that
they are the weights of minimum variance that sum to one and exactly balance
the means of the covariates towards the profile
$\bm{X}^{*{\scriptscriptstyle\text{URI}}}$.

###  3.2 Multi-regression imputation

MRI is another widely used method for regression adjustments in observational
studies, also known as g-computation or regression estimation in the causal
inference literature (Hernán and Robins 2020, Schafer and Kang 2008). In MRI,
we fit a linear regression model
$Y^{\text{obs}}_{i}=\beta_{0t}+\bm{\beta}^{\top}_{1t}\bm{X}_{i}+\epsilon_{it}$
in the treatment group and a separate linear regression model
$Y^{\text{obs}}_{i}=\beta_{0c}+\bm{\beta}^{\top}_{1c}\bm{X}_{i}+\epsilon_{ic}$
in the control group using OLS. The MRI estimators of the ATE and the ATT are
then
$\widehat{\text{ATE}}=\frac{1}{n}\sum_{i=1}^{n}(\hat{\beta}_{0t}+\hat{\bm{\beta}}^{\top}_{1t}\bm{X}_{i})-\frac{1}{n}\sum_{i=1}^{n}(\hat{\beta}_{0c}+\hat{\bm{\beta}}^{\top}_{1c}\bm{X}_{i})$
and
$\widehat{\text{ATT}}=\frac{1}{n}\sum_{i:A_{i}=1}^{n}Y^{\text{obs}}_{i}-\frac{1}{n_{t}}\sum_{i:A_{i}=1}(\hat{\beta}_{0c}+\hat{\bm{\beta}}^{\top}_{1c}\bm{X}_{i})$.

Using the implied weighting framework, we can represent the MRI estimators for
the estimands of interest as weighting estimators, i.e.,
$\widehat{\text{ATE}}=\sum_{i:A_{i}=1}w^{{\scriptscriptstyle\text{MRI}}}_{i}Y^{\text{obs}}_{i}-\sum_{i:A_{i}=0}w^{{\scriptscriptstyle\text{MRI}}}_{i}Y^{\text{obs}}_{i}$
and
$\widehat{\text{ATT}}=\sum_{i:A_{i}=1}w^{{\scriptscriptstyle\text{MRI}}}_{i}Y^{\text{obs}}_{i}-\sum_{i:A_{i}=0}w^{{\scriptscriptstyle\text{MRI}}}_{i}Y^{\text{obs}}_{i}$,
where the weights sum to one within each group. For the ATE,
$w^{{\scriptscriptstyle\text{MRI}}}_{i}=\frac{1}{n_{t}}\\{1+(\bm{X}_{i}-\bar{\bm{X}}_{t})^{\top}\bm{V}_{t}^{-1}(\bar{\bm{X}}-\bar{\bm{X}}_{t})\\}$
if unit $i$ is treated and
$w^{{\scriptscriptstyle\text{MRI}}}_{i}=\frac{1}{n_{c}}\\{1+(\bm{X}_{i}-\bar{\bm{X}}_{c})^{\top}\bm{V}_{c}^{-1}(\bar{\bm{X}}-\bar{\bm{X}}_{c})\\}$
otherwise. For the ATT,
$w^{{\scriptscriptstyle\text{MRI}}}_{i}=\frac{1}{n_{t}}$ if unit $i$ is
treated and
$w^{{\scriptscriptstyle\text{MRI}}}_{i}=\frac{1}{n_{c}}\\{1+(\bm{X}_{i}-\bar{\bm{X}}_{c})^{\top}\bm{V}_{c}^{-1}(\bar{\bm{X}}_{t}-\bar{\bm{X}}_{c})\\}$
otherwise. Similar to URI, the MRI weights do not depend on the outcome, so
they can also be obtained in the design stage of a causal study. Specifically,
the MRI weights exactly balance the means of the covariates, but unlike the
URI weights, they balance them toward the intended target populations. That
is,
$\sum_{i:A_{i}=1}w^{{\scriptscriptstyle\text{MRI}}}_{i}\bm{X}_{i}=\sum_{i:A_{i}=0}w^{{\scriptscriptstyle\text{MRI}}}_{i}\bm{X}_{i}=\bm{X}^{*{\scriptscriptstyle\text{MRI}}}$
where $\bm{X}^{*{\scriptscriptstyle\text{MRI}}}$ equals $\bar{\bm{X}}$ and
$\bar{\bm{X}}_{t}$ when the estimand is the ATE and the ATT, respectively.
Further, the MRI weights the weights of minimum variance that sum to one and
exactly balance the means of the covariates towards the profile
$\bm{X}^{*{\scriptscriptstyle\text{MRI}}}$. Finally, similar to URI, the MRI
weights can take negative values, leading to possible extrapolation beyond the
support of the observed data.

In large samples, both URI and MRI weights converge to inverse probability
weights, albeit under stringent assumptions on the treatment assignment
process. Moreover, leveraging this convergence, the URI and MRI estimators can
be shown to be multiply robust under conditions for the true treatment and
outcome models. See Chattopadhyay and Zubizarreta (2022) for details.

###  3.3 Extensions

The implied weighting framework applies to regression estimators in more
general settings, such as those with multi-valued treatments and instrumental
variables. First, consider a multi-valued treatment taking values in a finite
set $\mathcal{A}$ of treatment levels. Without loss of generality, let the
estimand be $\text{ATE}_{a,1}:=\mathbb{E}[Y_{i}(a)-Y_{i}(1)]$,
$a\in\\{2,...,|\mathcal{A}|\\}$, i.e., the average treatment effect of level
$a$ (the active level) over level $1$ (the reference level). The MRI approach
fits separate linear regression models in the groups $A_{i}=a$ and $A_{i}=1$
and hence, the MRI weights and their properties for binary treatment settings
directly apply to this setting. The URI approach fits the model
$Y^{\textrm{obs}}_{i}=\beta_{0}+\bm{\beta}^{\top}_{1}\bm{X}_{i}+\sum_{a=2}^{|\mathcal{A}|}\tau_{a,1}\mathbbm{1}(A_{i}=a)+\epsilon_{i}$
and estimates $\text{ATE}_{a,1}$ by
$\hat{\tau}^{\scriptscriptstyle\text{OLS}}_{a,1}$. This estimator admits the
weighting representation
$\hat{\tau}^{\scriptscriptstyle\text{OLS}}_{a,1}=\sum_{i:A_{i}=a}w^{{\scriptscriptstyle\textrm{URI}}}_{i,a,1}Y^{\textrm{obs}}_{i}-\sum_{A_{i}\neq
a}w^{{\scriptscriptstyle\textrm{URI}}}_{i,a,1}Y^{\textrm{obs}}_{i}$, where the
weights $w^{{\scriptscriptstyle\textrm{URI}}}_{i,a,1}$ sum to one within the
groups $A_{i}=a$ and $A_{i}\neq a$. In general, the URI weights may be non-
zero for units in groups other than groups $a$ and $1$. That is, other
treatment groups besides $a$ and $1$ have a say in the contrast between groups
$a$ and $1$. In fact, when estimating $\text{ATE}_{a,1}$, URI borrows strength
from all the treatment groups using linearity. Finally, while the weights
exactly balance the means of the covariates between groups $A_{i}=a$ and
$A_{i}\neq a$, the implied target population under URI varies with the active
treatment level $a$ (that is, it varies within the same fitted model), and in
general, none of these populations match the overall study population.

Second, consider an instrumental variable (IV) setting with a binary
instrument $Z_{i}$, which takes the value $Z_{i}=1$ if unit $i$ is encouraged
to receive treatment, and $Z_{i}=0$ otherwise. A common method to estimate the
effect of treatment in this setting is two-stage least squares (2SLS)
regression. In the first stage of 2SLS, one fits the regression model
$A_{i}=\delta_{0}+\bm{\delta}^{\top}_{1}\bm{X}_{i}+\gamma Z_{i}+\epsilon_{i}$
using OLS. In the second stage, one uses $\hat{A}_{i}$, the predicted values
of $A_{i}$ from the first stage, to fit the regression model
$Y^{\textrm{obs}}_{i}=\alpha_{0}+\bm{\alpha}^{\top}_{1}\bm{X}_{i}+\tau\hat{A}_{i}+\eta_{i}$
again using OLS. The 2SLS estimator is
$\hat{\tau}^{\scriptscriptstyle\text{2SLS}}$, the estimator of $\tau$ from the
second stage regression. Using the implied weighting framework,
$\hat{\tau}^{\scriptscriptstyle\text{2SLS}}$ can also be represented as a
weighting estimator:
$\hat{\tau}^{{\scriptscriptstyle\text{2SLS}}}=\sum_{i:A_{i}=1}w^{{\scriptscriptstyle\textrm{2SLS}}}_{i}Y^{\textrm{obs}}_{i}-\sum_{i:A_{i}=0}w^{{\scriptscriptstyle\textrm{2SLS}}}_{i}Y^{\textrm{obs}}_{i}$,
where the weights $w^{{\scriptscriptstyle\textrm{2SLS}}}_{i}$ sum to one
within each treatment group and depend only on the covariates, treatment, and
the instrument. Similar to the URI and MRI weights, the 2SLS weights exactly
balance the means of the covariates. Moreover, these weights allow us to
explicitly characterize the implied target populations of 2SLS regression in
finite samples. In general, the implied population of 2SLS differs from the
treated, control, or the overall study population, which is why the 2SLS
estimand, the local average treatment effect (LATE), often differs from the
true ATE. Finally, the weights under 2SLS can be negative and, in fact, such
negative weights occur more frequently in this setting than under URI
(Chattopadhyay and Zubizarreta 2021; Chattopadhyay 2022).

##  4 Data

To illustrate the lmw package, we use as a running example a part of the
famous Lalonde observational study (LaLonde 1986). The study evaluates the
effect of a labor training program (the National Supported Work Demonstration)
on future earnings. In our analysis, the treatment group corresponds to a
subset of $n_{t}=185$ units enrolled in the program from the Lalonde
experimental study (Dehejia and Wahba 1999), and the control group corresponds
to a subset of $n_{c}=2490$ non-experimental units from the Population Survey
of Income Dynamics (PSID). The dataset can be downloaded from
https://users.nber.org/~rdehejia/data/.nswdata2.html. There are $k=7$ baseline
covariates: age (age in years), education (years of education), race (Black,
Hispanic, and White), married (indicator for married status), nodegree
(indicator for high school dropout), re74 (earnings in 1974), and re75
(earnings in 1975). The outcome variable of interest is re78 (earnings in
1978). The standard estimand in this study is the ATT, i.e., the average
effect of the training program on the earnings of subjects who are part of the
program. However, for illustration, we also consider estimating the ATE.

In addition, we use the covariate data to generate an artificial multi-valued
treatment with three levels. We label the original treatment group as group 1
and split the original control group into two groups: group 2 (of size 901)
and group 3 (of size 1589) based on the logistic regression model
$\text{logit}\\{P(A_{i}=3|\bm{X}_{i},A_{i}\neq
1)\\}=1-\texttt{age}+1.2\times\texttt{education}-\texttt{black}+1.2\times\texttt{re75}$.
We also generate an artificial binary instrument $Z_{i}$ based on the logistic
regression model $\text{logit}\\{P(Z_{i}=1|\bm{X}_{i},A_{i}\neq
1)\\}=1-\texttt{age}+0.5\times\texttt{education}-0.5\times\texttt{hispanic}+0.8\times\text{black}+0.8\times\texttt{re74}+3\times\mathbbm{1}(A_{i}=1)$.
The generated instrument, by construction, has a significant partial
correlation with treatment after adjusting for the covariates. Also, the
instrument passes a pseudo-outcome-based heuristic test for the assumptions of
exclusion restriction and independence with unobserved covariates (see
Baiocchi et al. 2014).

##  5 Using lmw

Although the usual target estimand with the Lalonde data is the ATT, here we
target the ATE to demonstrate the features of lmw that can be used to diagnose
balance, representativeness, and extrapolation. We will follow with an example
targeting the ATT, an example using a multi-valued treatment, and an example
using 2SLS with an instrumental variable.

The primary function of the lmw package is lmw(), which computes the implied
regression weights. The syntax of lmw() is very similar to the linear modeling
function lm(), with a few adjustments.

    lmw_out <- lmw(~ A + X1 + X2 + X3, data = dataset,
                   estimand = "ATE", method = "URI",
                   treat = "A")

The first argument specifies the regression model formula, with the treatment
and covariates on the right-hand-side of ~. Unlike with lm(), an outcome does
not need to be specified because the linear regression weights do not depend
on the outcome; this allows for the preservation of the distinction between
the design and analyses stages, where the outcome is not involved until the
final step of estimating the treatment effect. The second argument, data,
specifies the dataset containing the variables in the model formula. The
estimand argument specifies the desired target estimand for the analysis. The
method argument specifies whether URI or MRI is to be used. Finally, the treat
argument specifies which variable in the model formula is to be regarded as
the treatment variable; if omitted, the first variable is assumed to be the
treatment. lmw() has several other arguments for more advanced uses that will
be discussed later.

###  5.1 URI for the ATE

Below is an example of using lmw() to compute the URI weights for the Lalonde
dataset. After running lmw(), the output is printed, displaying a summary of
the resulting lmw object.

    lmw_ate_out <- lmw(~ treat + age + education + married + race +
                         nodegree + re74 + re75, data = lalonde,
                       estimand = "ATE", method = "URI", treat = "treat")
    lmw_ate_out

    ## An lmw object
    ##  - treatment: treat (2 levels)
    ##  - method: URI (uni-regression imputation)
    ##  - number of obs.: 2675
    ##  - sampling weights: none
    ##  - base weights: none
    ##  - target estimand: ATE
    ##  - covariates: age, education, married, race, nodegree, re74, re75

The next step is to run summary() on the resulting object to examine the
balance and representativeness of the sample weighted by the regression
weights.

    lmw_ate_sum <- summary(lmw_ate_out)
    lmw_ate_sum
    ##
    ## Call:
    ## lmw(formula = ~ treat + age + education + married + race + nodegree +
    ##     re74 + re75, data = lalonde, estimand = "ATE", method = "URI",
    ##     treat = "treat")
    ##
    ## Summary of Balance for Unweighted Data:
    ##                 SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age          -1.009        0.070       -0.940 0.377       0.026       0.351
    ## education    -0.681        0.047       -0.633 0.403       0.028       0.375
    ## married      -1.845        0.128       -1.718 0.677       0.047       0.630
    ## raceblack     1.482       -0.102        1.379 0.593       0.041       0.552
    ## racehispanic  0.129       -0.009        0.120 0.027       0.002       0.025
    ## racewhite    -1.625        0.112       -1.512 0.620       0.043       0.577
    ## nodegree      0.880       -0.061        0.820 0.403       0.028       0.375
    ## re74         -1.718        0.119       -1.599 0.729       0.050       0.679
    ## re75         -1.774        0.123       -1.652 0.774       0.054       0.720
    ##
    ## Summary of Balance for Weighted Data:
    ##              SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age            0       -0.891       -0.891 0.127       0.350       0.332
    ## education      0       -0.615       -0.615 0.081       0.352       0.352
    ## married        0       -1.574       -1.574 0.000       0.578       0.578
    ## raceblack      0        1.338        1.338 0.000       0.535       0.535
    ## racehispanic   0        0.129        0.129 0.000       0.027       0.027
    ## racewhite      0       -1.474       -1.474 0.000       0.562       0.562
    ## nodegree       0        0.769        0.769 0.000       0.352       0.352
    ## re74           0       -1.578       -1.578 0.578       0.528       0.665
    ## re75           0       -1.636       -1.636 0.566       0.523       0.711
    ##
    ## Effective Sample Sizes:
    ##          Control Treated
    ## All       2490.    185.
    ## Weighted   367.3   180.6

summary() produces two tables, one containing balance statistics for the
original unweighted sample and one containing balance statistics for the
sample weighted by the regression weights. Each row corresponds to a covariate
and each column is a balance statistic. The statistics include SMD, the
standardized mean difference (SMD) of the covariate between the treatment and
control groups; TSMD Control and TSMD Treated, the SMD between each treatment
group and the target population implied by the target estimand (in this case
is the full sample because estimand was set to "ATE"; see Chattopadhyay et al.
2020 for details); and the corresponding values of the Kolmogorv-Smirnov (KS)
statistics. The addlvariables argument can be supplied with the names of
additional variables or terms (e.g., transformations of the covariates or
interactions between them) to assess balance on them in addition to the
variables used in the model. The effective sample size (ESS) represents the
approximate size of an unweighted sample that contains the same precision as
the weighted sample in question for estimating a counterfactual mean.

One can also use plot() on the summary() output to produce a visualization of
the resulting balance. The plot can be customized to display difference
balance statistics, add threshold lines, and change the order of the
covariates.

    plot(lmw_ate_sum)

The first table in the summary() output reveals several imbalances between the
treatment groups and between each treatment group and the target population
before adjustments, as indicated by the high SMDs and KS statistics,
especially for the treated group; these are also indicated by the crosses
($\times$) in the plot. After applying the regression weights, the SMDs are
equal to 0 for all covariates, indicating good covariate balance between the
treatment groups (though some of the KS statistics remain high). However, the
target imbalance is severe, even more so than prior to adjustment by the
regression weights, as indicated by the TSMD and TKS statistics and the black
circles in the plots. In particular, target imbalance for the control group
becomes much worse than it was prior to weighting, while target imbalance for
the treated group barely changed. The distribution of the covariates in the
resulting weighted sample can be summarized using summary() with the
additional argument stat = "distribution":

    summary(lmw_ate_out, stat = "distribution")

    ## Call:
    ## lmw(formula = ~ treat + age + education + married + race + nodegree +
    ##     re74 + re75, data = lalonde, estimand = "ATE", method = "URI",
    ##     treat = "treat")
    ##
    ## Distribution Summary for Unweighted Data:
    ##              Mean Overall  SD Overall Mean Control  SD Control Mean Treated SD Treated
    ## age                34.226    (10.500)       34.851    (10.441)       25.816    (7.155)
    ## education          11.994     (3.054)       12.117     (3.082)       10.346    (2.011)
    ## married             0.819     (0.385)        0.866     (0.340)        0.189    (0.392)
    ## raceblack           0.292     (0.454)        0.251     (0.433)        0.843    (0.364)
    ## racehispanic        0.034     (0.182)        0.033     (0.177)        0.059    (0.236)
    ## racewhite           0.674     (0.469)        0.717     (0.451)        0.097    (0.296)
    ## nodegree            0.333     (0.471)        0.305     (0.461)        0.708    (0.455)
    ## re74            18230.003 (13722.252)    19428.746 (13406.877)     2095.574 (4886.620)
    ## re75            17850.894 (13877.777)    19063.338 (13596.955)     1532.055 (3219.251)
    ##
    ## Distribution Summary for Weighted Data:
    ##              Mean Target   SD Target Mean Control  SD Control Mean Treated SD Treated
    ## age               34.226    (10.500)       26.247     (6.071)       26.247    (7.283)
    ## education         11.994     (3.054)       10.394     (2.821)       10.394    (2.053)
    ## married            0.819     (0.385)        0.242     (0.428)        0.242    (0.428)
    ## raceblack          0.292     (0.454)        0.827     (0.378)        0.827    (0.378)
    ## racehispanic       0.034     (0.182)        0.061     (0.240)        0.061    (0.240)
    ## racewhite          0.674     (0.469)        0.112     (0.315)        0.112    (0.315)
    ## nodegree           0.333     (0.471)        0.685     (0.465)        0.685    (0.465)
    ## re74           18230.003 (13722.252)     2310.339 (19516.079)     2310.339 (5243.971)
    ## re75           17850.894 (13877.777)     1690.432 (20103.272)     1690.432 (3525.011)
    ##
    ## Effective Sample Sizes:
    ##          Control Treated
    ## All       2490.    185.
    ## Weighted   367.3   180.6

The first table contains the mean and standard deviation of each covariate in
the full sample and each treatment group prior to weighting, and the second
table contains the mean and standard deviation of each covariate in the target
population and in the treatment groups after weighting. The severe target
imbalances are apparent in this table. For example, the mean of age in the
target population is approximately 34 years, while in the sample weighted by
the regression weights, the mean of age is approximately 26 years. The
proportion of married subjects in the target population is approximately 82%,
and in the weighted sample is 24%. These extreme imbalances demonstrate that
the population implied by the URI weights is not representative of the
population that the ATE intends to target.

Using plot() directly on the lmw output object provides additional insight
into the distribution of the weights. The type argument can be used to select
which among three types of plots to display; we consider the extrapolation
plot (type = "extrapolation") here and the other two later.

    plot(lmw_ate_out, type = "extrapolation",
         var = ~ age + married + re74)

The extrapolation plot displays how the weights distribute by the covariates
listed in var. The vertical line indicates the mean of the covariate in each
treatment group after weighting, and the cross ($\times$) indicates the mean
of the covariate in the target population. Each point represents an
observation, with black and red points indicating units with positive and
negative weights, respectively, and the size of each point reflecting the
magnitude of its weight. From this plot it is apparent that the covariate
means in the treatment groups do not align with the means in the target
population, and extrapolation is fairly severe given the preponderance of
negative weights (red points). Negative weights often yield poor balance on
components of the covariate distribution not directly targeted by the model;
this can be seen in the high KS statistics in the weighted sample even while
the SMDs are 0.

Given the severe target imbalance and extrapolation induced by using URI to
estimate the ATE, one has a few options. First, one can attempt to use other
methods that retain the target estimand, such as MRI or other weighting or
matching methods (possibly in combination with MRI or URI). Second, one can
change the target estimand of interest to one that is better supported by the
data, though this choice should be determined primarily by substantive
considerations (see Greifer and Stuart (2021)). We use both of these
approaches below, changing the target estimand to the study’s more natural
estimand, the ATT, and investigating the performance of MRI and an initial
matching step to reduce extrapolation and dependence on the outcome model.

###  5.2 MRI for the ATT

Below, we use the MRI to target the ATT. The call to lmw() is almost
identical, except this time we set method = "MRI" and estimand = "ATT":

    lmw_att_out <- lmw(~ treat + age + education + married + race +
                         nodegree + re74 + re75, data = lalonde,
                         estimand = "ATT", method = "MRI", treat = "treat")
    lmw_att_out
    ## An lmw object
    ##  - treatment: treat (2 levels)
    ##  - method:  MRI (multi-regression imputation)
    ##  - number of obs.: 2675
    ##  - sampling weights: none
    ##  - base weights: none
    ##  - target estimand: ATT
    ##  - covariates: age, education, married, race, nodegree, re74, re75

The model formula is interpreted slightly differently when using MRI than when
using URI. With URI, the model formula corresponds exactly to the linear model
that a user would fit, e.g., using lm(); with MRI, the model formula simply
indicates the terms that will be fit in the linear model separately for each
treatment group, and the treatment variable can be omitted (but still must be
specified in the treat argument).

After computing the weights, target balance can be assessed using summary():

    lmw_att_sum <- summary(lmw_att_out)
    lmw_att_sum
    ##
    ## Call:
    ## lmw(formula = ~treat + age + education + married + race + nodegree +
    ##     re74 + re75, data = lalonde, estimand = "ATT", method = "MRI",
    ##     treat = "treat")
    ##
    ## Summary of Balance for Unweighted Data:
    ##                 SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age          -1.263        1.263            0 0.377       0.377           0
    ## education    -0.881        0.881            0 0.403       0.403           0
    ## married      -1.729        1.729            0 0.677       0.677           0
    ## raceblack     1.630       -1.630            0 0.593       0.593           0
    ## racehispanic  0.114       -0.114            0 0.027       0.027           0
    ## racewhite    -2.091        2.091            0 0.620       0.620           0
    ## nodegree      0.886       -0.886            0 0.403       0.403           0
    ## re74         -3.547        3.547            0 0.729       0.729           0
    ## re75         -5.446        5.446            0 0.774       0.774           0
    ##
    ## Summary of Balance for Weighted Data:
    ##              SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age            0            0            0 0.144       0.144           0
    ## education      0            0            0 0.085       0.085           0
    ## married        0            0            0 0.000       0.000           0
    ## raceblack      0            0            0 0.000       0.000           0
    ## racehispanic   0            0            0 0.000       0.000           0
    ## racewhite      0            0            0 0.000       0.000           0
    ## nodegree       0            0            0 0.000       0.000           0
    ## re74           0            0            0 0.593       0.593           0
    ## re75           0            0            0 0.579       0.579           0
    ##
    ## Effective Sample Sizes:
    ##          Control Treated
    ## All      2490.       185
    ## Weighted  333.61     185

    plot(lmw_att_sum)

This time, the SMDs and target SMDs after weighting by the MRI weights are all
exactly equal to 0, indicating that the requested target population is being
targeted accurately (in this case, the treated group, as indicated by the
TSMDs for the treated group prior to weighting). Some imbalances remain on
features of the covariate distributions beyond the means, however, as
indicated by the high KS statistics. The linearity of the model is not able to
account for imbalances in the whole covariate distribution, despite perfectly
balancing the means. Such imbalances can be particularly severe in the
presence of negative weights.

We can use plot() with the argument type = "weights" to examine the
distribution of the weights and, in particular, to examine the presence of
negative weights.

    plot(lmw_att_out, type = "weights")

The upper plot displays the distribution of the weights for the control group
(the treated group weights are all equal to 1). The distribution is depicted
using a kernel density plot and a rug plot, with the vertical red line
indicating the average of the weights (which is equal to the inverse of the
control group sample size, i.e., $1/2490$). This plot indicates significant
extrapolation given that a large share of the weights distribution takes
values less than 0 and with somewhat high magnitude. The high variability of
the weights is reflected in the low ESS (334) compared to the original sample
size.

As before, one can request an extrapolation plot to further examine the extent
of extrapolation using plot() with type = "extrapolation":

    plot(lmw_att_out, type = "extrapolation",
         var = ~age + married + re74)

These plots indicate somewhat severe extrapolation based on the parts of the
distributions with negative weights (red circles). This time, though, the
vertical black lines (indicating the weighted covariate means) align exactly
with the covariate means in the target population.

Finally, one can estimate the potential outcome means and the treatment effect
using lmw_est(), which fits the linear model specified in the lmw object. To
do so, the outcome variable must be supplied to the outcome argument. The
output of lmw_est() contains the fitted model, but to compute the effect
estimate and its standard error, summary() must be called on the resulting
output.

lmw_att_est <\- lmw_est(lmw_att_out, outcome = re78)lmw_att_est## An lmw_est
object## - outcome: re78## - standard errors: robust (HC3)## - estimand: ATT##
- method: MRI#### Use summary() to examine estimates, standard errors,
p-values,## and confidence intervals.summary(lmw_att_est)#### Effect
estimates:## Estimate Std. Error 95% CI L 95% CI U t value Pr(>|t|)##
E[Y1-Y0|A=1] 790.5 793.7 -765.7 2346.8 0.996 0.319#### Residual standard
error: 10060 on 2657 degrees of freedom#### Potential outcome means:##
Estimate Std. Error 95% CI L 95% CI U t value Pr(>|t|)## E[Y0|A=1] 5558.6
520.6 4537.8 6579.4 10.68 <2e-16 ***## E[Y1|A=1] 6349.1 599.1 5174.5 7523.8
10.60 <2e-16 ***## ---## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’
0.1 ’ ’ 1

By default, lmw_est() applies HC3 robust standard errors to the outcome model
parameters; this behavior can be controlled using the robust argument. The
summary() output displays potential outcome means estimates of $6349 for the
treated group and $5558 for the control group, yielding an ATT of $791 (SE:
$794, 95% CI: (-767, 2347), p = .319). When URI is used, the potential outcome
means will not be displayed because they are not estimated by the outcome
model directly; only the treatment effect is.

A common concern in weighting is that estimates may be dominated by a handful
of observations, in particular for continuous outcomes such as earnings. The
influence of individual observations can be assessed using the sample
influence curve (SIC), which is a function of the implied regression weights,
residuals, and hat values for the fitted outcome model. The influence()
function can be used on lmw or lmw_est objects to produce the usual measures
of influence (hat values and residuals) and SIC values. These can be plotted
by using plot() with type = "influence", which displays SIC values by ID,
highlighting the most influential cases.

    plot(lmw_att_est, type = "influence")

###  5.3 MRI after matching for the ATT

Performing regression after matching (e.g., propensity score matching) or
weighting (e.g., inverse probability weighting) can reduce residual imbalances
and extrapolation and add robustness to the effect estimate (i.e., robustness
to misspecification of the outcome model). lmw() accepts a base.weights
argument to incorporate balancing weights into the regression model for the
outcome. When base.weights is supplied, the dr.method argument can be set to
"WLS" to perform weighted least squares regression in the weighted sample or
"AIPW" to perform augmented inverse probability weighting using the weights
and outcome model; "WLS" is the default. lmw also provides direct
compatibility with the MatchIt and WeightIt packages for matching and
weighting, respectively; the user can supply the output of a call to
MatchIt::matchit() or WeightIt::weightit() to the obj argument of lmw(), which
will extract the weights and estimand.

Below, we use an initial matching step to attempt to balance the covariates
prior to fitting the outcome model for the treatment effect. We use
MatchIt::matchit() to perform nearest neighbor propensity score matching.

    library(MatchIt)
    matchit_out <- matchit(treat ~ age + education + married + race +
                         nodegree + re74 + re75, data = lalonde,
                       estimand = "ATT")
    matchit_out
    ## A matchit object
    ##  - method: 1:1 nearest neighbor matching without replacement
    ##  - distance: Propensity score
    ##              - estimated with logistic regression
    ##  - number of obs.: 2675 (original), 370 (matched)
    ##  - target estimand: ATT
    ##  - covariates: age, education, married, race, nodegree, re74, re75

Next, we supply the matchit output object to the obj argument of lmw() to
incorporate the matching weights into the outcome model. This effectively fits
a weighted least squares regression with the matching weights (though in this
case the matching weights are all either 0 or 1).

    lmw_match_out <- lmw(~ treat + age + education + married + race +
                           nodegree + re74 + re75, obj = matchit_out,
                         method = "MRI")
    lmw_match_out
    ## An lmw object
    ##  - treatment: treat (2 levels)
    ##  - method:  MRI (multi-regression imputation)
    ##  - number of obs.: 2675
    ##  - sampling weights: none
    ##  - base weights: matching weights from MatchIt
    ##  - doubly-robust method: weighted least squares (WLS)
    ##  - target estimand: ATT
    ##  - covariates: age, education, married, race, nodegree, re74, re75

When using summary() to assess balance, balance is presented for the
unadjusted sample, for the sample after matching, and for the sample after
matching and weighting by the regression weights.

    summary(lmw_match_out)
    ##
    ## Call:
    ## lmw(formula = ~treat + age + education + married + race + nodegree +
    ##     re74 + re75, method = "MRI", obj = matchit_out)
    ##
    ## Summary of Balance for Unweighted Data:
    ##                 SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age          -1.263        1.263            0 0.377       0.377           0
    ## education    -0.881        0.881            0 0.403       0.403           0
    ## married      -1.729        1.729            0 0.677       0.677           0
    ## raceblack     1.630       -1.630            0 0.593       0.593           0
    ## racehispanic  0.114       -0.114            0 0.027       0.027           0
    ## racewhite    -2.091        2.091            0 0.620       0.620           0
    ## nodegree      0.886       -0.886            0 0.403       0.403           0
    ## re74         -3.547        3.547            0 0.729       0.729           0
    ## re75         -5.446        5.446            0 0.774       0.774           0
    ##
    ## Summary of Balance for Base Weighted Data:
    ##                 SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age          -0.652        0.652            0 0.178       0.178           0
    ## education    -0.016        0.016            0 0.092       0.092           0
    ## married      -0.690        0.690            0 0.270       0.270           0
    ## raceblack     0.238       -0.238            0 0.086       0.086           0
    ## racehispanic -0.023        0.023            0 0.005       0.005           0
    ## racewhite    -0.274        0.274            0 0.081       0.081           0
    ## nodegree      0.190       -0.190            0 0.086       0.086           0
    ## re74         -0.492        0.492            0 0.416       0.416           0
    ## re75         -0.519        0.519            0 0.297       0.297           0
    ##
    ## Summary of Balance for Weighted Data:
    ##              SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age            0            0            0 0.147       0.147           0
    ## education      0            0            0 0.058       0.058           0
    ## married        0            0            0 0.000       0.000           0
    ## raceblack      0            0            0 0.000       0.000           0
    ## racehispanic   0            0            0 0.000       0.000           0
    ## racewhite      0            0            0 0.000       0.000           0
    ## nodegree       0            0            0 0.000       0.000           0
    ## re74           0            0            0 0.382       0.382           0
    ## re75           0            0            0 0.222       0.222           0
    ##
    ## Effective Sample Sizes:
    ##               Control Treated
    ## All           2490.       185
    ## Base weighted  185.       185
    ## Weighted        77.93     185

Although balance after matching alone was somewhat poor, using MRI after
matching improved balance relative to matching alone and to regression without
matching. Matching can also decrease extrapolation, as the plots below
indicate:

    plot(lmw_match_out, type = "extrapolation",
         var = ~age + married + re74)

Finally, one can estimate the treatment effect, again using lmw_est() and
summary(). Because pair matching was used to find the matches, we use cluster-
robust standard errors by setting cluster = ~subclass in the call to
lmw_est():

lmw_match_est <\- lmw_est(lmw_match_out, outcome = re78, cluster =
~subclass)lmw_match_est## An lmw_est object## - outcome: re78## - standard
errors: cluster robust (HC1)## - estimand: ATT## - method: MRI#### Use
summary() to examine estimates, standard errors, p-values,## and confidence
intervals.summary(lmw_match_est)#### Effect estimates:## Estimate Std. Error
95% CI L 95% CI U t value Pr(>|t|)## E[Y1-Y0|A=1] 1904.5 872.3 189.0 3620.1
2.183 0.0297 *## ---## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1
’ ’ 1#### Residual standard error: 6922 on 352 degrees of freedom####
Potential outcome means:## Estimate Std. Error 95% CI L 95% CI U t value
Pr(>|t|)## E[Y0|A=1] 4444.6 634.0 3197.8 5691.4 7.011 1.22e-11 ***## E[Y1|A=1]
6349.1 577.2 5213.9 7484.4 10.999 < 2e-16 ***## ---## Signif. codes: 0 ’***’
0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1

The summary() output indicates the potential outcome means of $6349 for the
treated group and $4445 for the control group, yielding an ATT of $1905 (SE:
$872, 95% CI: (189, 3620), p = .030).

###  5.4 Multi-valued treatments

The procedure for the multi-valued treatment setting is similar to that with
binary treatments. Here we estimate the average treatment effects with respect
to the two constructed control groups. Although it is possible to use URI with
multi-valued treatments, it carries a number of shortcomings, both the same
ones URI poses for binary treatments (i.e., failure to target the right
population, extrapolation across treatment groups) and shortcomings unique to
the multi-valued treatment case (i.e., treatment effects from the same model
generalizing to different populations, changes in the procedure depending on
which category serves as the reference). Given these problems, we recommend
using MRI to estimate the effects of multi-valued treatments, which we
demonstrate below.

The syntax for multi-valued treatments is essentially the same as with binary
treatments except that the focal argument needs to be supplied when estimand =
"ATT" to identify which treatment level is to be considered the “treated” or
“focal” level. (When URI is used, an additional contrast argument is required
to identify a pair of groups to be contrasted, since each contrast will
receive its own set of weights, which will be computed one at a time by
lmw().)

    lmw_multi_out <- lmw(~ treat_multi + age + education + married + race +
                         nodegree + re74 + re75, data = lalonde,
                       estimand = "ATT", method = "MRI",
                       treat = "treat_multi", focal = "1")
    lmw_multi_out
    ## An lmw object
    ##  - treatment: treat_multi (3 levels)
    ##  - method:  MRI (multi-regression imputation)
    ##  - number of obs.: 2675
    ##  - sampling weights: none
    ##  - base weights: none
    ##  - target estimand: ATT (focal = "1")
    ##  - covariates: age, education, married, race, nodegree, re74, re75

Using summary() on the resulting object produces a balance table for each
treatment level against the target sample (in this case, treatment level “1”).
To view balance statistics between pairs of treatment levels, the contrast
argument can be used to specify for which pair of levels balance statistics
should be displayed. (When using URI, only the two groups involved in the
contrast supplied to lmw() will be compared.)

    summary(lmw_multi_out)
    ##
    ## Call:
    ## lmw(formula = ~treat_multi + age + education + married + race +
    ##     nodegree + re74 + re75, data = lalonde, estimand = "ATT",
    ##     method = "MRI", treat = "treat_multi", focal = "1")
    ##
    ## Summary of Balance for Unweighted Data:
    ##              TSMD 1 TSMD 2 TSMD 3 TKS 1 TKS 2 TKS 3
    ## age               0  1.791  0.963     0 0.538 0.312
    ## education         0 -0.210  1.499     0 0.164 0.573
    ## married           0  1.679  1.757     0 0.658 0.688
    ## raceblack         0 -0.912 -2.037     0 0.332 0.741
    ## racehispanic      0 -0.101 -0.121     0 0.024 0.029
    ## racewhite         0  1.200  2.596     0 0.356 0.769
    ## nodegree          0 -0.225 -1.261     0 0.102 0.573
    ## re74              0  2.350  4.226     0 0.634 0.791
    ## re75              0  3.297  6.664     0 0.630 0.857
    ##
    ## Summary of Balance for Weighted Data:
    ##              TSMD 1 TSMD 2 TSMD 3 TKS 1 TKS 2 TKS 3
    ## age               0      0      0     0 0.158 0.118
    ## education         0      0      0     0 0.028 0.086
    ## married           0      0      0     0 0.000 0.000
    ## raceblack         0      0      0     0 0.000 0.000
    ## racehispanic      0      0      0     0 0.000 0.000
    ## racewhite         0      0      0     0 0.000 0.000
    ## nodegree          0      0      0     0 0.000 0.000
    ## re74              0      0      0     0 0.484 0.677
    ## re75              0      0      0     0 0.439 0.675
    ##
    ## Effective Sample Sizes:
    ##            1      2       3
    ## All      185 901.   1589.
    ## Weighted 185 153.16  109.55

The means of each treatment group are equal to the means of the target sample
after weighting by the MRI weights, but some imbalances remain on the KS
statistics between the two control groups and the target. Just as with binary
treatments, plot() can be used on the summary() output to plot balance
statistics or on the lmw() output to view the distribution of weights and the
degree of extrapolation.

Estimating the treatment effect follows the same procedure as with binary
treatments: use lmw_est() to fit the outcome model and use summary() to
extract the treatment effect estimates and their standard errors.

lmw_multi_est <\- lmw_est(lmw_multi_out, outcome = re78)lmw_multi_est## An
lmw_est object## - outcome: re78## - standard errors: robust (HC3)## -
estimand: ATT## - method: MRI#### Use summary() to examine estimates, standard
errors, p-values,## and confidence intervals.summary(lmw_multi_est)#### Effect
estimates:## Estimate Std. Error 95% CI L 95% CI U t value Pr(>|t|)##
E[Y1-Y2|A=1] 1418.5 866.1 -279.7 3116.7 1.638 0.102## E[Y1-Y3|A=1] 757.6
1114.6 -1427.9 2943.1 0.680 0.497## E[Y2-Y3|A=1] -660.9 1129.0 -2874.6 1552.9
-0.585 0.558#### Residual standard error: 10040 on 2648 degrees of freedom####
Potential outcome means:## Estimate Std. Error 95% CI L 95% CI U t value
Pr(>|t|)## E[Y1|A=1] 6349.1 599.1 5174.5 7523.8 10.598 < 2e-16 ***## E[Y2|A=1]
4930.6 625.5 3704.2 6157.1 7.883 4.62e-15 ***## E[Y3|A=1] 5591.5 939.9 3748.6
7434.5 5.949 3.05e-09 ***## ---## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’
0.05 ’.’ 0.1 ’ ’ 1

The ATTs for the control groups are $1419 (SE: $866, 95% CI: (-280, 3117), p =
.102) and $758 (SE: $1115, 95% CI: (-1428, 2943), p = .497).

###  5.5 Two-stage least squares

Using lmw with instrumental variables is straightforward; the user must
specify the “second-stage” (i.e., outcome) model in the main model formula and
must supply the instrument separately. The lmw_iv() function is used to
compute the weights implied by the 2SLS model. Both URI and MRI are possible
with 2SLS, although URI is far more common. When MRI is used or treatment-
covariate interactions are included in the outcome model, interactions between
the covariates and the instrument will be included as additional instruments.
We will use URI here, using the simulated variable Ins as an instrument and
conditioning on age, education, race, and re74 in the 2SLS models. Specifying
the estimand does not affect estimation of the weights with URI when no
treatment-covariate interactions are included in the outcome model, but it can
be useful to determine how far the target population of the local average
treatment effect implied by the weights is from the desired target population.

    lmw_iv_out <- lmw_iv(~treat + age + education + race + re74,
                         data = lalonde, estimand = "ATT", method = "URI",
                         treat = "treat", iv = ~Ins)
    lmw_iv_out
    ## An lmw_iv object
    ##  - treatment: treat (2 levels)
    ##  - instrument: Ins
    ##  - method: URI (uni-regression imputation)
    ##  - number of obs.: 2675
    ##  - sampling weights: none
    ##  - base weights: none
    ##  - target estimand: ATT
    ##  - covariates: age, education, race, re74

As before, one can use summary() to assess balance. Variables that were not
included in the model can be added using addlvariables.

    summary(lmw_iv_out, addlvariables = ~married + nodegree + re75)
    ##
    ## Call:
    ## lmw_iv(formula = ~treat + age + education + race + re74, data = lalonde,
    ##     estimand = "ATT", method = "URI", treat = "treat", iv = ~Ins)
    ##
    ## Summary of Balance for Unweighted Data:
    ##                 SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age          -1.263        1.263            0 0.377       0.377           0
    ## education    -0.881        0.881            0 0.403       0.403           0
    ## raceblack     1.630       -1.630            0 0.593       0.593           0
    ## racehispanic  0.114       -0.114            0 0.027       0.027           0
    ## racewhite    -2.091        2.091            0 0.620       0.620           0
    ## re74         -3.547        3.547            0 0.729       0.729           0
    ## married      -1.729        1.729            0 0.677       0.677           0
    ## nodegree      0.886       -0.886            0 0.403       0.403           0
    ## re75         -5.446        5.446            0 0.774       0.774           0
    ##
    ## Summary of Balance for Weighted Data:
    ##                 SMD TSMD Control TSMD Treated    KS TKS Control TKS Treated
    ## age           0.000        0.539        0.539 0.324       0.432       0.198
    ## education     0.000       -0.218       -0.218 0.580       0.534       0.106
    ## raceblack     0.000       -0.557       -0.557 0.000       0.203       0.203
    ## racehispanic  0.000        0.390        0.390 0.000       0.092       0.092
    ## racewhite     0.000        0.373        0.373 0.000       0.110       0.110
    ## re74          0.000       -0.219       -0.219 0.704       0.725       0.103
    ## married      -1.255        1.388        0.134 0.491       0.544       0.052
    ## nodegree      1.275       -1.175        0.100 0.580       0.534       0.045
    ## re75          0.260       -0.322       -0.062 0.535       0.532       0.059
    ##
    ## Effective Sample Sizes:
    ##          Control Treated
    ## All      2490.     185.
    ## Weighted    2.25    59.6

Extrapolation can be a problem with 2SLS models due to large, negative
weights; the extrapolation plot displays the severity of this problem in this
data setting:

    plot(lmw_iv_out, type = "extrapolation",
         var = ~age + married + re74)

Finally, one can estimate the treatment effect using lmw_est(), which produces
a treatment effect estimate based on the second-stage regression model on the
treatment and covariates after adjustment by the instrument. As with other
regression models, robust standard errors are produced by default.

lmw_iv_est <\- lmw_est(lmw_iv_out, outcome = re78)lmw_iv_est## An lmw_est_iv
object## - outcome: re78## - standard errors: robust (HC3)## - estimand: ATT##
- method: URI#### Use summary() to examine estimates, standard errors,
p-values,## and confidence intervals.summary(lmw_iv_est)#### Effect
estimates:## Estimate Std. Error 95% CI L 95% CI U t value Pr(>|t|)##
E[Y1-Y0|A=1] 7802 7160 -6238 21841 1.09 0.276#### Residual standard error:
11030 on 2668 degrees of freedom

The treatment effect estimate is $7802 (SE: $7160, 95% CI: (-6238, 21841), p =
.276). Note this estimate should not be taken seriously as the instrument here
was synthetic and used solely for illustration purposes.

## References

  * Abadie et al. (2015) Abadie, A., Diamond, A., and Hainmueller, J. (2015), “Comparative politics and the synthetic control method,” American Journal of Political Science, 59, 495–510.

  * Angrist and Pischke (2008) Angrist, J. D. and Pischke, J.-S. (2008), Mostly harmless econometrics: An empiricist’s companion, Princeton university press.
  * Aronow and Samii (2016) Aronow, P. M. and Samii, C. (2016), “Does regression produce representative estimates of causal effects?” American Journal of Political Science, 60, 250–267.
  * Baiocchi et al. (2014) Baiocchi, M., Cheng, J., and Small, D. S. (2014), “Instrumental variable methods for causal inference,” Statistics in Medicine, 33, 2297–2340.
  * Ben-Michael et al. (2021) Ben-Michael, E., Feller, A., and Rothstein, J. (2021), “The augmented synthetic control method,” Journal of the American Statistical Association, 116, 1789–1803.
  * Chattopadhyay (2022) Chattopadhyay, A. (2022), “New Methods for Causal Inference in Randomized Experiments and Observational Studies,” Ph.D. thesis, Harvard University.
  * Chattopadhyay et al. (2020) Chattopadhyay, A., Hase, C. H., and Zubizarreta, J. R. (2020), “Balancing vs modeling approaches to weighting in practice,” Statistics in Medicine, 39, 3227–3254.
  * Chattopadhyay and Zubizarreta (2021) Chattopadhyay, A. and Zubizarreta, J. R. (2021), “On the implied weights of linear regression for causal inference,” arXiv preprint arXiv:2104.06581v2.
  * Chattopadhyay and Zubizarreta (2022) — (2022), “On the implied weights of linear regression for causal inference,” Biometrika, in press.
  * Cochran (1965) Cochran, W. G. (1965), “The planning of observational studies of human populations,” Journal of the Royal Statistical Society, 128, 234–266.
  * Dehejia and Wahba (1999) Dehejia, R. and Wahba, S. (1999), “Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs,” Journal of the American Statistical Association, 94, 1053–1062.
  * Dorn (1953) Dorn, H. F. (1953), “Philosophy of inferences from retrospective studies,” American Journal of Public Health and the Nations Health, 43, 677–683.
  * Gelman and Imbens (2018) Gelman, A. and Imbens, G. (2018), “Why high-order polynomials should not be used in regression discontinuity designs,” Journal of Business & Economic Statistics, 1–10.
  * Greifer and Stuart (2021) Greifer, N. and Stuart, E. A. (2021), “Choosing the estimand when matching or weighting in observational studies,” arXiv preprint arXiv:2106.10577.
  * Hernán and Robins (2016) Hernán, M. A. and Robins, J. M. (2016), “Using big data to emulate a target trial when a randomized trial is not available,” American Journal of Epidemiology, 183, 758–764.
  * Hernán and Robins (2020) Hernán, M. A. and Robins, J. M. (2020), Causal inference: What if, CRC Boca Raton.
  * Imbens (2015) Imbens, G. W. (2015), “Matching methods in practice: three examples,” Journal of Human Resources, 50, 373–419.
  * Imbens and Rubin (2015) Imbens, G. W. and Rubin, D. B. (2015), Causal inference in statistics, social, and biomedical sciences, Cambridge University Press.
  * LaLonde (1986) LaLonde, R. J. (1986), “Evaluating the econometric evaluations of training programs with experimental data,” The American Economic Review, 604–620.
  * Neyman (1923,1990) Neyman, J. (1923,1990), “On the application of probability theory to agricultural experiments,” Statistical Science, 5, 463–480.
  * Rosenbaum (2010) Rosenbaum, P. R. (2010), Design of observational studies, Springer.
  * Rosenbaum and Rubin (1983) Rosenbaum, P. R. and Rubin, D. B. (1983), “The central role of the propensity score in observational studies for causal effects,” Biometrika, 70, 41–55.
  * Rubin (1974) Rubin, D. B. (1974), “Estimating causal effects of treatments in randomized and nonrandomized studies.” Journal of Educational Psychology, 66, 688.
  * Rubin (2007) — (2007), “The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials,” Statistics in Medicine, 26, 20–36.
  * Schafer and Kang (2008) Schafer, J. L. and Kang, J. (2008), “Average causal effects from nonrandomized studies: A practical guide and simulated example,” Psychological Methods, 13, 279–313.
  * Zubizarreta (2015) Zubizarreta, J. R. (2015), “Stable weights that balance covariates for estimation with incomplete outcome data,” Journal of the American Statistical Association, 110, 910–922.