# Local Non-Cooperative Games with Principled Player Selection
for Scalable Motion Planning

Makram Chahine1, Roya Firoozi2, Wei Xiao1, Mac Schwager2 and Daniela Rus1 1
Computer Science and Artificial Intelligence Lab, Massachusetts Institute of
Technology {chahine, weixy<EMAIL_ADDRESS>Department of Aeronautics &
Astronautics, Stanford University {rfiroozi<EMAIL_ADDRESS>work
was supported in part by ONR grant N00014-18-1-2830. Toyota Research Institute
provided funds to support this work. The second author was supported on an
ASEE eFellows fellowship.

###### Abstract

Game-theoretic motion planners are a powerful tool for the control of
interactive multi-agent robot systems. Indeed, contrary to predict-then-plan
paradigms, game-theoretic planners do not ignore the interactive nature of the
problem, and simultaneously predict the behaviour of other agents while
considering change in one’s policy. This, however, comes at the expense of
computational complexity, especially as the number of agents considered grows.
In fact, planning with more than a handful of agents can quickly become
intractable, disqualifying game-theoretic planners as possible candidates for
large scale planning. In this paper, we propose a planning algorithm enabling
the use of game-theoretic planners in robot systems with a large number of
agents. Our planner is based on the reality of locality of information and
thus deploys local games with a selected subset of agents in a receding
horizon fashion to plan collision avoiding trajectories. We propose five
different principled schemes for selecting game participants and compare their
collision avoidance performance. We observe that the use of Control Barrier
Functions for priority ranking is a potent solution to the player selection
problem for motion planning.

##  I Introduction

While interactions among multiple mobile agents can be thoroughly modeled
using game theory, the complexity of game-theoretic planners grows cubically
with the number of the agents considered [1]. Thus, in large scale settings
where a large number of interactive agents are present, game-theoretic
planning is intractable [2]. The typical procedure in large scale settings is
to identify the closest neighbors and perform local game-theoretic planning
while ignoring the rest of the agents in the scene [3]. This neighbor
selection scheme is justified in structured environments such as driving,
where the geometry of the road and driving rules ensure agents closest are
generally the most important to consider. However, in congested settings,
taking the spatially closest neighbors into account (e.g. all agents in a
given radius) might still result in a computationally expensive planning
scheme. Furthermore, in unstructured environments such as quadrotor flight,
the validity of the argument for planning with the closest agents is highly
questionable.

In this work we study the problem of game-theoretic motion planning for
systems with a number of agents beyond the computational feasibility of a full
game solution. The investigation aims to provide insights into this question
on two levels. First, we aim to evaluate the capacity of receding horizon
local game-theoretic planners to achieve satisfactory trajectories in crowded
environments, when the maximum number of agents considered for planning is
constrained. Trajectories in this setting are obtained by reasoning about
agents’ equilibrium behavior in the game perceived by the ego agent. However,
in general, no two agents need to share the same pool of players during
planning. The influence of agents optimizing different games have been studied
from the point of view of cost mismatch [4]. To the best of our knowledge, the
question of games with player mismatching, or _local games_ , has not been
addressed in the literature. Second, with the nearest neighbor criterion as a
baseline, we introduce several neighbor selection schemes based on Control
Barrier Functions (CBF) [5] [6], cost sensitivity, and contribution to cost
evolution. The aim is to understand how much improvement principled player
selection methods can achieve with respect to the baseline, in order to make a
case for the use of _local games_ as a scalable solution to multi-agent motion
planning problems. The contributions of this paper are as follows:

Figure 1: In practice, robots make decisions based on local information. This
can lead to agents solving games with different subsets of agents present in
the scene. For example, based on a fixed radius criterion, agent 1 may
consider 2, 3, 4, and 9 as neighbors in its game, while agent 2 may consider
1, 3, 4, 5, 6, and 9. How do such systems behave and can local information be
leveraged to select only significantly interacting neighbors, in order to
mitigate the computational burden of considering agents with little or no
impact on one’s actions?

  * •

We investigate _local games_ (in which each agent considers only a small
number of local neighbors) for scalable decentralized trajectory planning in
congested environments.

  * •

We propose multiple neighbor selection schemes, amongst which is a novel use
of Control Barrier Functions (CBF) for priority ranking, to overcome
shortcomings of a naive nearest neighbor criterion.

  * •

We provide numerical analysis to study how game complexity (in number of
players per game) affects anticipation capacity and safety in congested
environments, and compare methods with double integrator agents in the plane
and nonlinear quadrotor agents in $\mathbb{R}^{3}$.

The paper is organized as follows: Sec. II reviews the related work. Sec. III
describes the problem statement. Sec. IV presents different proposed player
section methods. Sec. V presents the numerical results based on simulation
studies. Sec. VI makes concluding remarks and discusses the future work.

##  II Related Work

Game-theoretic planning: Game-theoretic planning provides a mathematical
formalism to model highly complex interactions among different agents.
Interactive planning approaches can be categorized in two categories of (i)
Predict-then-Plan approaches and (ii) Predict-and-Plan approaches. In Predict-
then-Plan approaches, the prediction and planning are decoupled. The ego agent
first predicts the other agents’ behavior along a horizon in the future and
then plans based on the predicted trajectories as presented in [7], and [8].
In the second category which is predict-and-plan approaches such as game-
theoretic planning, the prediction and planning steps are coupled. The
planning and prediction is performed together by finding a game theoretic
equilibrium in the joint space of trajectories of all the agents in the scene
as presented in [9], [1], [10], [11], [12], and [4]. With these methods, the
coupled interaction or mutual influence between the other agents in the scene
and the ego agent is captured. In game-theoretic planning, not only the future
actions of the other agents in the scene are predicted, but the prediction is
in a feedback loop with ego agent actions. The ego agent will be able to
reason about how its action will affect the future reactions of other agents
in the scene.

Interaction quantification and player selection: Little literature covers the
question of quantifying the interactions between motion planning robots, which
is important information for the selection of a subset of agents relevant to
the ego robot’s performance. In [3], the authors touch on this topic in the
work’s appendix. They implement an MPC game-theoretic planner limiting
interactions to 4 neighboring agents in the context of autonomous driving.
This limitation is justified by empirical tests but also in post analysis by
computing how one agent’s utility gradient depends on another agent’s inputs,
i.e. a Hessian norm in the game formulation framework. To quantify the mutual
influence of interactive agents, the authors in [13], address the influential
neighbor selection problem and propose a decentralized influential
maximization problem by selecting k neighbors rather than arbitrary agents.
The authors have introduced different types of neighbor selection schemes and
have shown information on online social networks e.g. Twitter can be
propagated efficiently by selecting neighbors with high propagation rate
rather than those with a high number of neighbors. In [14], the authors
present a reversed nearest neighbor heat map which provides influence
distribution in a two-dimensional space.

In [15], the authors propose to select neighbors to conduct communication
with, using deep reinforcement learning. In [16], the authors propose Graph
Attention Networks to choose which agents to communicate with. In [17], the
authors study a range of k Nearest Neighbor (kNN) search techniques for crowd
simulation. Finding kNN neighbors by querying different data structures
including k-d trees, BD trees, R-trees, Voronoi diagrams, hierarchical k-mean
clustering, line search and grids is studied. The query performance of these
methods are compared in terms of computation time. Also, the methods are
compared based on how they scale with increasing number of agents. For all
approaches the number of neighboring agents is assumed to be constant $k$. In
contrast, we propose to quantify the influence of other agents and take that
into account to select the most influential agents as the ego agent neighbors
and not a constant number of neighbors.

##  III Problem Statement

We design and compare player selection methodologies to enable scalable
receding horizon multi-agent motion planning where each robot repeatedly
solves Generalized Nash Equilibrium Problems (GNEPs), updating their pool of
opponents at each iteration. We thus present the GNEP framework in its general
formulation. Also, a novel contribution of the work is the use of Control
Barrier Functions as an interaction quantification method. The CBF framework
is thus also presented in this section.

###  III-A Generalized Nash Equilibrium Problem

A given GNEP will involve $N$ players $i\in\\{1,\dots,N\\}$ over a horizon of
$T$ time steps. An agent $i$’s state at time step index $k$ is denoted
$\mathbf{x}_{k}^{i}\in\mathbb{R}^{n^{i}}$ and control input
$\mathbf{u}_{k}^{i}\in\mathbb{R}^{m^{i}}$, with dimensions of agent $i$’s
state and control $n^{i}$ and $m^{i}$. Let
$\mathbf{x}_{k}=[\mathbf{x}_{k}^{1,\top},\dots,\mathbf{x}_{k}^{N,\top}]^{\top}\in\mathbb{R}^{n}$
denote the joint state and
$\mathbf{u}_{k}=[\mathbf{u}_{k}^{1,\top},\dots,\mathbf{u}_{k}^{N,\top}]^{\top}\in\mathbb{R}^{m}$
denote the joint control of all agents at time $k$, with joint dimensions
$n=\sum_{i}n^{i}$ and $m=\sum_{i}m^{i}$. We define player $i$’s policy as
$\pi^{i}=[\mathbf{u}_{1}^{1,\top},\dots,\mathbf{u}_{T-1}^{1,\top}]^{\top}\in\mathbb{R}^{\tilde{m}^{i}}$
where $\tilde{m}^{i}=m^{i}(T-1)$ denotes the dimension of the entire
trajectory of agent $i$’s control inputs. The notation $\neg i$ indicates all
agents except $i$, for instance $\pi^{\neg i}$ represents the vector of the
agents’ policies except that of $i$. Also, let
$X=[\mathbf{x}_{2}^{\top},\dots,\mathbf{x}_{N}^{\top}]^{\top}\in\mathbb{R}^{\tilde{n}}$,
with $\tilde{n}=n(T-1)$, denote the trajectory of joint state variables
resulting from the application of the joint control inputs to the dynamical
system defined by
$f:\mathbb{R}^{n}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{n}$ such that,

$\displaystyle\mathbf{x}_{k+1}=f(\mathbf{x}_{k},\mathbf{u}_{k}).$ (1)

Over the whole trajectory we can express the above kinodynamic constraints
with $\tilde{n}$ equality constraints,

$\displaystyle D(X,\pi^{1},\dots,\pi^{N})=D(X,\pi)=0\in\mathbb{R}^{\tilde{n}}$
(2)

The cost function of each player $i$ depends on its policy $\pi^{i}$ as well
as on the joint state trajectory $X$, which is common to all players, such
that $\forall i\in\\{1,\dots,N\\}$,

$\displaystyle\mathcal{C}^{i}(X,\pi^{i})=c_{T}^{i}(\mathbf{x}_{N})+\sum_{k=1}^{T-1}c_{k}^{i}(\mathbf{x}_{k},\mathbf{u}_{k}^{i}).$
(3)

Notice that as player $i$ minimizes $\mathcal{C}^{i}$ with respect to $X$ and
$\pi^{i}$, the selection of $X$ is constrained by the other players’
strategies $\pi^{\neg i}$ and the dynamics of the joint system via (2). In
addition, the strategy $\pi^{i}$ could be required to satisfy (safety)
constraints that depend on the joint state trajectory $X$ as well as on the
other players strategies $\pi^{\neg i}$ . This can be expressed with a set of
$l$ inequality constraints,

$\displaystyle C(X,\pi)\leq 0\in\mathbb{R}^{l}$ (4)

where
$C:\mathbb{R}^{\tilde{n}}\times\mathbb{R}^{m(T-1)}\rightarrow\mathbb{R}^{l}$.
The GNEP we form is the problem of minimizing (3) for all players
$i\in\\{1,\dots,N\\}$ with respect to (2) and (4). More specifically,

$\displaystyle\underset{X,\pi^{i}}{\min}\ $
$\displaystyle\mathcal{C}^{i}(X,\pi^{i})\qquad\forall i\in\\{1,\dots,N\\}$
subject to $\displaystyle D(X,\pi)=0$ (5) $\displaystyle C(X,\pi)\leq 0.$

The solution to such a dynamic game is a generalized Nash equilibrium, i.e. a
policy $\hat{\pi}$ such that, $\forall i\in\\{1,\dots,N\\}$, $\hat{\pi}^{i}$
is a solution to (5) with the other players’ policies given by
$\hat{\pi}^{\neg i}$ that is also solved by (5) for all $\neg i$. As a
consequence, at a Nash equilibrium point solution, no player can improve their
strategy by unilaterally modifying their policy.

###  III-B Control Barrier Functions

Control Barrier Functions (CBFs) [5] are a popular control synthesis tool for
safety-critical systems. CBFs can transform safety constraints of control-
affine systems into state-feedback constraints that are linear in controls.
With such control constraints, one can formulate convex optimization problems
(CBF-based optimization) whose solutions (optimal controls) guarantee system
safety. Consider a safety constraint $h_{i,j}(X)\geq 0$ between two agents $i$
and $j$, where $h_{i,j}:\mathbb{R}^{\tilde{n}}\rightarrow\mathbb{R}$ (recall
$\tilde{n}$ is the dimension of $X$) is continuously differentiable. Following
the CBF method, we have that $h_{i,j}(X)\geq 0$ is guaranteed for all times if
$h_{i,j}(X(t_{0}))\geq 0$ and the following CBF constraint is satisfied:

$\dot{h}_{i,j}(X)+\alpha_{1}(h_{i,j}(X))\geq 0,$ (6)

where $\alpha_{1}$ is a class $\mathcal{K}$ function (strictly increasing
function that passes through the origin). If the relative degree (number of
times we need to take the derivative of $h_{i,j}(X)$ along dynamics until any
control shows in the corresponding derivative) of $h_{i,j}(X)$ is one, then we
would have control $\pi$ in $\dot{h}_{i,j}(X)$, i.e., we rewrite
$\dot{h}_{i,j}(X)$ as $\dot{h}_{i,j}(X,\pi)$, and (6) can be rewritten as:

$\dot{h}_{i,j}(X,\pi)+\kappa\alpha_{1}(h_{i,j}(X))\geq 0,$ (7)

where $\kappa>0$ is a scalar that we added to the class $\mathcal{K}$
function. The magnitude of $\kappa$ will determine the size of the activation
zone of the CBF; with empirical evidence suggesting that it grows larger for
smaller values of $\kappa$.

If the relative degree of $h_{i,j}(X)$ is $m\in\mathbb{N}$ that is larger than
one, then the control terms would not appear in (7). In this case involving
high-relative-degree safety constraints, high order CBF (HOCBF) methods [6]
are required, by defining,

$\psi_{k}(X):=\dot{\psi}_{k-1}(X)+\alpha_{k}(\psi_{k-1}(X)),\quad
k\in\\{1,\dots,m\\},$ (8)

where $\psi_{0}(X)=h_{i,j}(X)$, $\alpha_{k},k\in\\{1,\dots,m\\}$ are class
$\mathcal{K}$ functions, and $\psi_{m}(X,\pi)\equiv\psi_{m}(X)\geq 0$ implies
$h_{i,j}(X)\geq 0$ and therefore safety. Again, $\kappa>0$ can be added to
each of the class $\mathcal{K}$ functions above to regulate the size of the
HOCBF activation zone.

##  IV Proposed player selection methods

In this section, we develop the multiple ranking techniques that aim to sort
agents in terms of the significance of their interactions. As the resolution
of a dynamic game over some time horizon $T$, is performed on a fixed set of
agents (both in number and identity), we aim to use the ranking systems to
update the subset of players the ego agent takes into consideration for
planning. We propose 5 methods to rank agents and to be compared along with
the naive approach of considering the Euclidean nearest neighbors.

For all applications presented in this paper, we explicitly provide the
structure of the cost at time step $k<T$ of a given agent $i$, appearing in
(3),

$\displaystyle c_{k}^{i}(\mathbf{x}_{k},\mathbf{u}_{k}^{i})$
$\displaystyle=(\mathbf{x}_{k}^{i}-\mathbf{x}_{f}^{i})^{\top}Q_{i}(\mathbf{x}_{k}^{i}-\mathbf{x}_{f}^{i})+{\mathbf{u}_{k}^{i}}^{\top}R_{i}\mathbf{u}_{k}^{i}$
$\displaystyle\hskip 10.00002pt+\sum_{j=1,j\neq
i}^{N}\frac{\mu}{2}\max(0,R-\left\lVert\mathbf{x}_{k}^{i}-\mathbf{x}_{k}^{j}\right\rVert)^{2}$
(9)

where $Q_{i}$ is the state quadratic cost matrix, $R_{i}$ the control
quadratic cost matrix, $\mu$ the multiplicative constant associated with the
collision avoidance cost, and $R$ the repulsion radius. We mention that the
collision avoidance term only contributes to the cost when agents are within
the distance $R$. To obtain a global ranking of agents independent of the
repulsion radius we can use a proxy for the collision cost by looking at the
quantity,

$\displaystyle\mathcal{C}^{i}_{col}(k)=\sum_{j=1,j\neq
i}^{N}\frac{\mu}{\left\lVert\mathbf{x}_{k}^{i}-\mathbf{x}_{k}^{j}\right\rVert^{2}},$
(10)

where $\mathcal{C}^{i}_{col}$ represent the portion of the cost associated
with collision avoidance. Indeed, in equation (9), the maximum operator sets
the pairwise collision cost to zero as long as two agents are farther than the
selected repulsion radius. Hence, two agents could be indistinguishable in
terms of collision cost (i.e. both with zero contribution), while presenting
very disparate collision risks with the ego agent. For instance, it is clear
that an agent entering the repulsion radius at high speed in the direction of
the ego agent should score much higher in terms of interaction significance
compared to another that is both distant and moving away from the ego agent.
The collision cost structure in (9) is oblivious to this, but the proposed
proxy (10), continuous in space, can capture this difference.

###  IV-A Jacobian/Hessian

The sensitivity of an agent $i$ to another agent $j$’s actions can be defined
as the derivative of the cost of agent $i$ with respect to agent $j$’s
actions. In general, this metric distinguishes whether one agent has
sensitivity towards another but without the other way around needing to hold.
We compute it first by noticing that only the collision cost incorporates a
direct short term dependence on the actions of agent $j$.

Agent i can compute a pairwise utility Jacobian with respect to agent $j$ as
follows,

$\displaystyle
J_{i,j}=\frac{\partial\mathcal{C}^{i}_{col}}{\partial\mathbf{u}^{j}}.$ (11)

Each scalar term in the Jacobian can be approximated by the finite elements
method, perturbing the controls from the previous state and forward
propagating the dynamics to obtain perturbed costs. At a given time, we can
evaluate the norm of this Jacobian. This will give agent $i$ information about
the sensitivity of its objective to changes in $j$’s control. Larger norms
will indicate higher interaction and we thus propose a pairwise scoring
function $f^{J}_{i,j}(X,\mu)$ such that,

$\displaystyle f^{J}_{i,j}(X,\mu)=\left\lVert J_{i,j}(X,\mu)\right\rVert.$
(12)

Furthermore, the Hessian of that accumulated reward is the derivative of the
sensitivity of agent $i$ to $j$ with respect to agent $i$’s actions. In other
words, it expresses how much agent $i$’s actions will change if agent $j$’s
actions change. Agent $i$ can compute a pairwise utility Hessian with respect
to agent $j$ as follows,

$\displaystyle
H_{i,j}=\frac{\partial^{2}\mathcal{C}_{col}^{i}}{\partial\mathbf{u}^{i}\partial\mathbf{u}^{j}}.$
(13)

Again, we can evaluate the norm of this Hessian using finite elements. This
can be an even more intricate indicator, since the cost might be sensitive to
changes in $j$’s actions without $i$ being able to react to it. The Hessian
norm will remain small in such a case. In contrast, larger norms will indicate
higher interaction and we thus propose a pairwise scoring function
$f^{H}_{i,j}(X,\mu)$ such that,

$\displaystyle f^{H}_{i,j}(X,\mu)=\left\lVert H_{i,j}(X,\mu)\right\rVert,$
(14)

where we use the Frobenius norm.

###  IV-B Cost evolution

Another approach to quantify interaction can be based on the cost impact
agents have on each other. Accordingly, greedy ranking can be used to limit
the impact of the highest contributing agents. Statically, the highest cost is
associated to the nearest neighbor. However, dynamically we can consider
looking at which agent has had the biggest recent impact on cost increase.
This can be achieved based on position observations only, although we
implicitly take into account dynamics as we estimate the additional
contribution of each agent over the past time step. Indeed, we rank agents by
how fast their contribution increases the ego collision avoidance cost, i.e.
by evaluating the pairwise quantity,

$\displaystyle
f^{CE}_{i,j}(X)=\frac{\mu}{\left\lVert\mathbf{x}_{k}^{i}-\mathbf{x}_{k}^{j}\right\rVert^{2}}-\frac{\mu}{\left\lVert\mathbf{x}_{k-1}^{i}-\mathbf{x}_{k-1}^{j}\right\rVert^{2}}.$
(15)

Negative contributions indicate the other agent is getting farther away from
the ego agent, and can more easily be ignored in planning. Another desirable
feature of this metric is that for equal distance reductions, agents that are
closer will contribute more to the cost increase, and will thus be higher up
the interaction ranking.

###  IV-C Control Barrier Function

In this work, we use CBFs as a risk metric evaluation tool that can be used to
select the most safety-threatening agents in local games. This tool is useful
since CBFs usually have activation zones around the unsafe sets, and these
activation zones are larger than the unsafe sets (the size depends on the CBF
parameters), which means that CBFs can help to predict collisions before they
may happen, as previously discussed in Sec. III. Between each pair of agents
$i$ and $j$, given the collision radius to avoid violating, we can define a
zeroth order barrier function, capturing the pairwise collision avoidance
constraint, as follows,

$\displaystyle h_{i,j}(X)$
$\displaystyle=\left\lVert\mathbf{x}^{i}-\mathbf{x}^{j}\right\rVert^{2}-R^{2}$
(16)

its first Lie derivative (along the trajectory) is with respect to time, thus,

$\displaystyle\dot{h}_{i,j}(X)$
$\displaystyle=2(\mathbf{x}^{i}-\mathbf{x}^{j})^{\top}(\mathbf{v}^{i}-\mathbf{v}^{j})$
(17)

With the choice of linear class $\mathcal{K}$ functions in (7) and (8), a
first order pairwise barrier function can thus be obtained,

$\displaystyle f^{BF}_{i,j}(X,\kappa)=\dot{h}_{i,j}(X)+\kappa h_{i,j}(X)$ (18)

with $\kappa>0$. We can repeat the process to obtain a higher order barrier
function in which the control input appears. For motion dynamics considered in
this paper (both double intergrator and quadrotor), the order required is 2.
We thus derive the second derivative,

$\displaystyle\ddot{h}_{i,j}(X,\pi)=2[(\mathbf{v}^{i}-\mathbf{v}^{j})^{\top}(\mathbf{v}^{i}-\mathbf{v}^{j})+(\mathbf{x}^{i}-\mathbf{x}^{j})^{\top}(\mathbf{a}^{i}-\mathbf{a}^{j})]$
(19)

where the acceleration vector of a given agent $i$, denoted above by
$\mathbf{a}^{i}$, is a known function of its control input. Hence, we obtain a
pairwise Control Barrier Function,

$\displaystyle
f^{CBF}_{i,j}(X,\pi,\kappa)=\ddot{h}_{i,j}(X,\pi)+2\kappa\dot{h}_{i,j}(X)+\kappa^{2}h_{i,j}(X)$
(20)

with $\kappa>0$. Both the BF and CBF constraints requires $f^{BF}_{i,j}(X)\geq
0$ and $f^{CBF}_{i,j}(X,\pi)\geq 0$ for the $(i,j)$ pair of agents to avoid
collision. Thus, larger values will correspond to pairs of agents who are far
from activating the collision constraint and that are subsequently unlikely to
have significant interaction. The barrier functions thus constitute a natural
ranking method for planning priority.

###  IV-D Methods summary

We have thus proposed 5 methods in addition to the naive nearest neighbor
approach, two of which are based on the sensitivity analysis of the cost
function with respect to other agents’ control (Jacobian and Hessian),
another, adopting a greedy perspective consisting in dealing with agents that
contribute to a cost increase at the biggest rates (Cost Evolution), and,
finally, two more inspired by control theory and specifically barrier
functions (BF and CBF). The computational requirements and characteristics of
each method are presented in Table I.

TABLE I: Characteristics of considered player selection methods. Ticks/crosses indicate whether or not the method requires the associated information/computation. We consider access to positions (pos), velocities (vel), control inputs and dynamics (c&d), in addition to need for forward integration of dynamics in time (f-i) and tuning (tun). | pos | vel | c&d | f-i | tun
---|---|---|---|---|---
Nearest Neighbor | ✓ | ✗ | ✗ | ✗ | ✗
Cost Evolution | ✓ | ✗ | ✗ | ✗ | ✗
Jacobian | ✓ | ✓ | ✓ | ✓ | ✗
Hessian | ✓ | ✓ | ✓ | ✓ | ✗
BF | ✓ | ✓ | ✗ | ✗ | ✓
CBF | ✓ | ✓ | ✓ | ✗ | ✓

##  V Simulations

We compare our proposed player selection criteria in three scenarios with
increasing complexity. All tests consist in having the agents navigate from a
start position to a random final position on a grid, minimizing their control
input and simultaneously attempting to avoid other agents. This spatial
architecture is interesting as agents have to deal with potential collisions
from all sides, at least on the inside of the grid. The crucial
computationally limiting factor is $p$, the number of players considered by
the ego agent at each planning step, in other words the size of the
$(p+1)$-player game considered. We aim to understand how the increase in game
complexity, by incorporating additional players, impacts an agents capacity to
safely navigate the environment. We assume all agents plan using the receding
horizon game theoretic planner with the same $p$ and the same player selection
method for each run. The performance metric we select to differentiate the
methods is the minimum distance between any pair of agents along the joint
trajectory.

###  V-A Simulations setup

The dynamic games solver we use is ALGAMES [1], which is an open-source solver
for game theoretic planning problems with multiple actors and general
nonlinear state and input constraints. First, we test the methods for double
integrator dynamics in the plane, considering a 3x3 grid of agents. This
smaller setup, though less congested, will allow us to compare the methods’
performance in avoiding collisions against the full game implementation, i.e.
one with all other agents considered during planning. We extend this
experiment to a larger 5x5 grid, increasing the congestion and number of
interactions occurring during navigation. We finally test on an even larger
cubic grid of 3x3x3 agents following 12 dimensional quadrotor dynamics to
exhibit the advantages of our method in real world flight applications. The
tuning of $\kappa$, appearing in the barrier functions is done separately for
the BF and CBF functions as well as for the chosen dynamics. However in all 4
cases we converge to a close value and thus fix $\kappa=5.0$ in all
experiments.

###  V-B Double integrator

We first consider agents evolving in the 2D plane according to double
integrator dynamics. The state of a vehicle comprises of its 2D positions and
velocities and the control input comprises of the 2D accelerations.

####  V-B1 3x3 grid

The setup is small enough to permit the complete resolution of the game in
reasonable time offline, i.e. taking all other 8 agents into account when
planning at each receding horizon time step. Thus, we can obtain a ground
truth baseline against which to compare the performance of local games with
different player selection criteria. We repeat the experiment 20 times,
randomizing the desired final positions of agents on the grid, for
$p=1,\dots,4$, and for all 6 player selection methods, as well as solving the
entire 9-player game. The minimum pairwise distances are averaged over the
runs, the results are presented in Fig. 2.

Figure 2: 3x3 grid: Mean normalized minimum distances between any pair of
agents with $p$ other participants using the Jacobian, Hessian, Nearest
neighbor, Cost evolution, BF, and CBF schemes. The dotted black line is the
value obtained by solving the full 9-player game (also averaged over runs).
The normalization factor is the constraint activation radius defined in (9).

First, we notice that both methods (Jacobian and Hessian) based on the partial
differentiation of the ego agent’s cost with respect to controls perform even
worse than the nearest neighbor approach. This could be due to the absence of
dependence on the agents’ trajectories and their evolution in time, taking
only into account a short term relationship to a change in controls. We will
abandon the further analysis of these methods in the subsequent experiments.
Also, we notice that both barrier function methods and the cost evolution
criterion perform similarly well and improve performance against the nearest
neighbor method, especially for games restrained to 1 or 2 opponents.
Elsewhere, the limited congestion due to the small number of agents in the
scene ensures that collision avoidance performance levels comparable to those
for the full game can be achieved by considering 3 or more participants.

We can also analyse how non-static methods anticipate the future collision
threat of agents in the scene and the effect of such anticipation on the
safety of planned trajectories. Indeed, as depicted in Fig. 3, we can see that
the myopic nearest neighbor approach can easily lead to more dangerous
situations.

Figure 3: Nearest neighbor vs CBF player selection for $p=2$: Trajectory
screenshots at 4 equally spaced time steps, the solid markers depicting
instantaneous positions of the agents in the plane and the lines their past
trajectories. The ego agent is in blue, agents used for planning are in red,
and agents ignored in green.

At time $t=\delta t$, the ego agent switches attention from the agent on the
top left (benignly passing behind) to the one coming from the bottom right
with whom it is on a collision course. By the time the bottom right agent is
considered using the nearest neighbor approach at $t=2\delta t$, a maneuvre is
required to avoid it. In the meantime, using the CBF ranking system, the
bottom left agent is already included in planning as the ego agent notices
they can neglect the centre agent which is near immobile. At $t=3\delta t$,
the ego is in the clear using CBF, whereas it has to deal with another short
notice collision avoidance maneuvre with the bottom left agent with the
nearest neighbor selection method.

####  V-B2 5x5 grid

The more crowded setup provides a platform to emphasize the differences
between player selection methods. Indeed, the sample trajectory depicted in
Fig. 4 shows the large number of multi-directional robots an agent finds
themselves surrounded by while navigating towards their desired positions.

Figure 4: Sample run depicting the congested environment agents have to
negotiate their way through to their target positions. Solid circles depict
agents’ instantaneous position, lines represent the agents trace of positions
over the last 10 time steps. Time increases from left to right, top to bottom.

Again, we repeat the experiment 20 times, randomizing the desired final
positions of agents on the grid, for $p=1,\dots,4$, and for the nearest
neighbor, cost evolution, BF, and CBF methods. The minimum pairwise distances
are averaged over the runs, the results are presented in Fig. 5.

Figure 5: 5x5 grid: Mean normalized minimum distances between any pair of
agents with $p$ other participants using the Nearest neighbor, Cost evolution,
BF, and CBF schemes. The normalization factor is the constraint activation
radius defined in (9).

It is clear that the increased number of agents makes this experiment more
challenging than the previous. Indeed, we notice that performance levels are,
as expected, globally lower. More interestingly, this experiments exhibits the
limits of the myopic nearest neighbor approach, which performs poorly in
avoiding other agents, even as the number of participants considered grows.
Moreover, a hierarchy between the remaining methods manifests itself in
crowded maneuvering. Ranking agents based on their contribution in increasing
the value of an ego agent’s cost function over the last time step performs
consistently worse than the barrier function based methods. Indeed, this
disparity is clearest for pairwise game planning ($p=1$), where the repulsion
radius is violated on average by over 60% with the cost evolution method,
while the CBF criterion limits violation to under 40%. We mention that the
effect of the difference in cost is even more exacerbated as the dependency is
quadratic as per (9). Although this gap decreases as $p$ grows, it remains
significant enough to consider the cost evolution criterion inferior to the
barrier function methods. In fact, the latter two offer similar performance
for $p\geq 2$, with the added information of CBF seamingly giving it a slight
advantage. A nice advantage of the CBF metric, is that it seems to suffer less
from fluctuation in performance with respect to the number of agents
considered than the simpler BF, and presents consistent improvement as $p$
grows.

###  V-C Quadrotor dynamics

Next, we retain the Cost Evolution and CBF methods and test them against the
Nearest Neighbor baseline with agents evolving according to quadrotor dynamics
randomly exchanging positions on a 3x3x3 grid. The state of an agent comprises
of the Euclidean positions $\mathbf{x}=[x,y,z]^{\top}$, Euler angles
$\mathbf{q}=[\phi,\theta,\psi]^{\top}$, translation velocities
$\mathbf{v}=[v_{x},v_{y},v_{z}]^{\top}$, and angular rates
$\mathbf{\omega}=[\omega_{\phi},\omega_{\theta},\omega_{\psi}]^{\top}$
constituting a 12 dimensional vector
$[\mathbf{x}^{\top},\mathbf{q}^{\top},\mathbf{v}^{\top},\mathbf{\omega}^{\top}]^{\top}$.
The control vector is the power input to each of the 4 motors
$\mathbf{u}=[w_{1},w_{2},w_{3},w_{4}]$. The dynamics are given by,

$\displaystyle\mathbf{\dot{x}}$ $\displaystyle=\mathbf{v},\quad$
$\displaystyle\mathbf{\dot{q}}=R(\phi,\theta,\psi)\mathbf{\omega},$
$\displaystyle\mathbf{\dot{v}}$ $\displaystyle=\frac{\mathbf{F}}{m},\quad$
$\displaystyle\mathbf{\dot{\omega}}=I^{-1}(\mathbf{\tau}-\mathbf{\omega}\times(I\mathbf{\omega})),$
(21)

with $R(\phi,\theta,\psi)$ the rotation matrix between the drone body and
world frames, $m$ the mass, and $I$ the inertia of the drone [18]. The force
$\mathbf{F}$ and moment $\mathbf{\tau}$ vectors are given by,

$\displaystyle\mathbf{F}$
$\displaystyle=m\mathbf{g}+R(\phi,\theta,\psi)[0,0,k_{f}(w_{1}+w_{2}+w_{3}+w_{4})],$
$\displaystyle\mathbf{\tau}$
$\displaystyle=[Lk_{f}(w_{2}-w_{4}),Lk_{f}(w_{3}-w_{1}),k_{m}(w_{1}-w_{2}+w_{3}-w_{4})]$

with $\mathbf{g}=[0,0,-g]$ the gravity vector, $k_{f}$ and $k_{m}$ the motor
force and torque constants, and $L$ the distance between the motors.

Figure 6: Sample run depicting 3x3x3 grid swap with quadrotor dynamics. Solid
circles depict agents’ instantaneous position, lines represent the agents
trace of positions over the last 10 time steps. Time increases from left to
right, top to bottom.

We run 20 batches of simulations, each with a different random final state
configuration over the 3x3x3 grid. For each, we compare the Nearest Neighbor
approach with the player selection schemes based on the Cost Evolution
Criterion and the CBF metric. Each experiment is repeated for games of fixed
sizes 2 to 4. For every configuration, the minimum pairwise distances over all
trajectories are computed, averaged over random final positions on the grid
and presented in Figure 7.

Figure 7: Mean normalized minimum distances between any pair of quadrotors
versus $p$. Nearest neighbor, Cost evolution and CBF schemes. The
normalization factor is the constraint activation radius defined in (9).

Surprisingly, the nearest neighbor and cost evolution schemes perform
consistently as the number of agents per game increases. Also, both methods
seem to do better than the BF selection criterion. The latter, however,
behaves as expected; improving performance as the game size grows, although
its performance is substantially worse on smaller games. The CBF method also
consistently improves collision avoidance as the games grow in size. Again, it
outperforms the other methods, and does so significantly for $p\geq 2$.

##  VI Conclusion

We explored the concept of local game-theoretic motion planning that scales
with the number of agents in the scene. To maintain satisfactory collision
avoidance performance while ensuring scalability, we designed local planners
selecting agents with the strongest interaction as neighbors. To this end we
proposed different player selection schemes and quantified the interactions of
the ego agent with others in the scene. We compared various methods including
Jacobian, Hessian, Nearest Neighbor, Cost Evolution, Barrier Function, and
Control Barrier Function. Based on the numerical results, Control Barrier
Functions consistently outperform the rest of the player selection methods,
and prove to be the strongest candidate for principled player selection for
local games.

## References

  * [1] S. Le Cleac’h, M. Schwager, and Z. Manchester, “Algames: a fast augmented lagrangian solver for constrained dynamic games,” _Autonomous Robots_ , vol. 46, no. 1, pp. 201–215, 2022.
  * [2] C. Daskalakis, P. W. Goldberg, and C. H. Papadimitriou, “The complexity of computing a Nash equilibrium,” _SIAM Journal on Computing_ , vol. 39, no. 1, pp. 195–259, 2009. [Online]. Available: https://doi.org/10.1137/070699652
  * [3] W. Schwarting, A. Pierson, J. Alonso-Mora, S. Karaman, and D. Rus, “Social behavior for autonomous vehicles,” _Proceedings of the National Academy of Sciences_ , vol. 116, p. 201820676, 11 2019.
  * [4] M. Chahine, R. Firoozi, W. Xiao, M. Schwager, and D. Rus, “Intention communication and hypothesis likelihood in game-theoretic motion planning,” _IEEE Robotics and Automation Letters_ , vol. 8, no. 3, pp. 1223–1230, 2023.
  * [5] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs for safety critical systems,” _IEEE Transactions on Automatic Control_ , vol. 62, no. 8, pp. 3861–3876, 2017.
  * [6] W. Xiao and C. Belta, “Control barrier functions for systems with high relative degree,” in _Proceedings of the 58th IEEE Conference on Decision and Control_ , Nice, France, 2019, pp. 474–479.
  * [7] F. Rey, Z. Pan, A. Hauswirth, and J. Lygeros, “Fully decentralized admm for coordination and collision avoidance,” in _2018 European Control Conference (ECC)_ , 2018, pp. 825–830.
  * [8] L. Ferranti, R. R. Negenborn, T. Keviczky, and J. Alonso-Mora, “Coordination of multiple vessels via distributed nonlinear model predictive control,” in _2018 European Control Conference (ECC)_ , 2018, pp. 2523–2528.
  * [9] R. Spica, E. Cristofalo, Z. Wang, E. Montijano, and M. Schwager, “A real-time game theoretic planner for autonomous two-player drone racing,” _IEEE Transactions on Robotics_ , vol. 36, no. 5, pp. 1389–1403, 2020.
  * [10] D. Fridovich-Keil, E. Ratner, L. Peters, A. D. Dragan, and C. J. Tomlin, “Efficient iterative linear-quadratic approximations for nonlinear multi-player general-sum differential games,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_ , 2020, pp. 1475–1481.
  * [11] J. F. Fisac, E. Bronstein, E. Stefansson, D. Sadigh, S. S. Sastry, and A. D. Dragan, “Hierarchical game-theoretic planning for autonomous vehicles,” _2019 International Conference on Robotics and Automation (ICRA)_ , 2018.
  * [12] D. Sadigh, S. S. Sastry, S. A. Seshia, and A. D. Dragan, “Planning for autonomous cars that leverage effects on human actions,” in _Robotics: Science and Systems_ , 2016.
  * [13] H. Kim, K. Beznosov, E. Yoneki, and E. Yoneki, “A study on the influential neighbors to maximize information diffusion in online social networks,” _Computational Social Networks_ , vol. 2, 2015. [Online]. Available: https://doi.org/10.1186/s40649-015-0013-8
  * [14] Y. Sun, R. Zhang, A. Y. Xue, J. Qi, and X. Du, “Reverse nearest neighbor heat maps: A tool for influence exploration,” in _2016 IEEE 32nd International Conference on Data Engineering (ICDE)_ , 2016, pp. 966–977.
  * [15] Z. Ma, Y. Luo, and J. Pan, “Learning selective communication for multi-agent path finding,” _IEEE Robotics and Automation Letters_ , vol. 7, pp. 1455–1462, 2021.
  * [16] Y. Niu, R. Paleja, and M. Gombolay, “Multi-agent graph-attention communication and teaming,” in _Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems_ , ser. AAMAS ’21. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems, 2021, p. 964–973.
  * [17] J. L. Vermeulen, A. Hillebrand, and R. Geraerts, “A comparative study of k-nearest neighbour techniques in crowd simulation,” _Computer Animation and Virtual Worlds_ , vol. 28, no. 3-4, p. e1775, 2017, e1775 cav.1775.
  * [18] G. Hoffmann, H. Huang, S. Waslander, and C. Tomlin, “Quadrotor helicopter flight dynamics and control: Theory and experiment,” in _AIAA Guidance, Navigation and Control Conference and Exhibit_ , 2007. [Online]. Available: https://arc.aiaa.org/doi/abs/10.2514/6.2007-6461


$\displaystyle-\big{(}\frac{1}{N}\sum_{i=1}^{N}\upsilon(X_{i})\big{)}^{-2}\frac{1}{N}\sum_{i=1}^{N}[\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}]^{2}\Big{\|}$
$\displaystyle\leq\Big{\|}\big{\\{}\mathbb{E}\hat{\upsilon}(X_{i})\big{\\}}^{-2}\mathbb{E}\big{[}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\big{]}^{2}-\big{\\{}\mathbb{E}\upsilon(X_{i})\big{\\}}^{-2}\mathbb{E}\big{[}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\big{]}^{2}\Big{\|}+o_{P}(1)=o_{P}(1),$

where the second inequality comes from 3(a), and the third equality comes from
the consistency of $\hat{\theta}$ such that
$\hat{\upsilon}(X_{i})=\upsilon(X_{i})+o_{P}(1)$ and the similar proofs as
$A_{1N}$.

For $C_{3N}$, by the Law of Large Numbers, we have

$\displaystyle\big{(}\frac{1}{N}\sum_{i=1}^{N}\upsilon(X_{i})\big{)}^{-2}\frac{1}{N}\sum_{i=1}^{N}\Big{[}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}$
$\displaystyle=\big{(}\mathbb{E}\upsilon(X_{i})\big{)}^{-2}\mathbb{E}\Big{[}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}+o_{P}(1)$
$\displaystyle=\sigma_{\omega}^{2}+o_{P}(1).$

In sum, we have shown
$\widehat{\sigma}_{\omega}^{2}-\sigma_{\omega}^{2}=o_{P}(1)$.

(b) We write

$\displaystyle\hat{\sigma}^{2}-\sigma^{2}=\frac{1}{N}\sum_{i=1}^{N}\Big{[}\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\hat{\pi}(X_{i})\big{)}\widehat{U}_{i}\Big{]}^{2}-\sigma^{2}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\Big{[}\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\hat{\pi}(X_{i})\big{)}\widehat{U}_{i}\Big{]}^{2}-\frac{1}{N}\sum_{i=1}^{N}\Big{[}\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}$
$\displaystyle+\frac{1}{N}\sum_{i=1}^{N}\Big{[}\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}-\frac{1}{N}\sum_{i=1}^{N}\Big{[}\upsilon(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}$
$\displaystyle+\frac{1}{N}\sum_{i=1}^{N}[\upsilon(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}]^{2}-\sigma^{2}=D_{1N}+D_{2N}+D_{3N}.$

For $D_{1N}$, we have:

$\displaystyle\|D_{1N}\|=$
$\displaystyle\Big{\|}\frac{1}{N}\sum_{i=1}^{N}\big{[}\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\hat{\pi}(X_{i})\big{)}\widehat{U}_{i}\big{]}^{2}-\frac{1}{N}\sum_{i=1}^{N}\big{[}\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\big{]}^{2}\Big{\|}$
$\displaystyle\leq$ $\displaystyle
O(1)\Big{\|}\frac{1}{N}\sum_{i=1}^{N}\big{[}\big{(}D_{i}-\hat{\pi}(X_{i})\big{)}\widehat{U}_{i}\big{]}^{2}-\frac{1}{N}\sum_{i=1}^{N}\big{[}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\big{]}^{2}\Big{\|}\leq
o_{P}(1),$

where the first inequality comes from the boundness of $\upsilon(\cdot)$, and
the second equality comes from the similar proofs as $C_{1N}$.

For $D_{2N}$, we have:

$\displaystyle\|D_{2N}\|$
$\displaystyle=\Big{\|}\frac{1}{N}\sum_{i=1}^{N}[\hat{\upsilon}(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}]^{2}-\frac{1}{N}\sum_{i=1}^{N}[\upsilon(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}]^{2}\Big{\|}$
$\displaystyle=\Big{\|}\frac{1}{N}\sum_{i=1}^{N}[\hat{\upsilon}(X_{i})^{-1}-\upsilon(X_{i})^{-1}][\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}]^{2}\Big{\|}$
$\displaystyle\leq\Big{\|}\mathbb{E}[\hat{\upsilon}(X_{i})^{-1}-\upsilon(X_{i})^{-1}][\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}]^{2}\Big{\|}+o_{P}(1)$
$\displaystyle\leq\Big{\\{}\mathbb{E}\big{\|}\hat{\upsilon}(X_{i})^{-1}-\upsilon(X_{i})^{-1}\big{\|}^{2}\mathbb{E}\big{\|}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\big{\|}^{4}\Big{\\}}^{1/2}+o_{P}(1)=o_{P}(1),$

where the third inequality comes from 3(a), the fourth inequality comes from
Cauchy-Schwarz inequality, and the last inequality comes from the consistency
of $\hat{\theta}$ such that $\hat{\upsilon}(X_{i})=\upsilon(X_{i})+o_{P}(1)$
and the similar proofs as $C_{2N}$.

For $D_{3N}$, by the Law of Large Numbers, we have:

$\displaystyle\frac{1}{N}\sum_{i=1}^{N}\Big{[}\upsilon(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}$
$\displaystyle=\mathbb{E}\Big{[}\upsilon(X_{i})^{-1}\big{(}D_{i}-\pi(X_{i})\big{)}U_{i}\Big{]}^{2}+o_{P}(1)$
$\displaystyle=\sigma^{2}+o_{P}(1).$

In sum, we have shown $\widehat{\sigma}^{2}-\sigma^{2}=o_{P}(1)$.

∎

Appendix D: Additional Simulation Results

Appendix D.1: Simulation Study: Estimating the Propensity Score Function

We first conduct a Monte Carlo study to investigate the finite sample
properties of our semiparametric estimator proposed in Section 3.1 for the
propensity score. We consider the following four functional forms (DGPs) with
a vector of regressors $X=(X_{1},X_{2})^{\prime}\sim N(0,I_{2})$:

  * 1A. $g_{0}(X^{\prime}\theta_{0})=\sin(X^{\prime}\theta_{0})$, where $\theta_{0}=(0.8,-0.6)^{\prime}$.

  * 1B. $g_{0}(X^{\prime}\theta_{0})=0.5\\{(X^{\prime}\theta_{0})^{3}-(X^{\prime}\theta_{0})\\}$, where $\theta_{0}$ is the same as in Setting 1A.

  * 2A. $g_{0}(X^{\prime}\theta_{0})=\sin(X^{\prime}\theta_{0})$, where $\theta_{0}=(0.5,-0.5)^{\prime}$.

  * 2B. $g_{0}(X^{\prime}\theta_{0})=0.5\\{(X^{\prime}\theta_{0})^{3}-(X^{\prime}\theta_{0})\\}$, where $\theta_{0}$ is the same as in Setting 1B.

The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$. The DGPs in Settings 1A
and 1B satisfy the $\|\theta_{0}\|=1$ for identification purpose, while the
DGPs in Settings 2A and 2B violate $\|\theta_{0}\|=1$.

Let $\hat{\theta}=(\hat{\theta}_{1},\hat{\theta}_{2})^{\prime}$ be the
estimator of $\theta_{0}$. We evaluate the estimators in terms of bias,
standard deviation (std) and root mean-squared error (RMSE). We summarize the
simulation results in Table 4. For the DGPs in Settings 1A and 1B that satisfy
$\|\theta_{0}\|=1$, the simulation results show that our estimator performs
well. Moreover, by focusing on the bias and RMSE separately, it is clear that
both terms vanish quickly as the sample size increases. The asymptotic normal
approximation is accurate as the standard deviations become smaller as the
sample size grows. Our estimation approach is valid for the polynomial single
index structure or more complicated periodic single index form. More
importantly, the convergence speed is $O(1/N)$ along the direction of true
parameter $\theta_{0}$, consistent with the results in Lemma C.2.

Table 4:  Simulation Results for the Estimators of Propensity Score Function of Settings 1A to 2B. | Setting | $N$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ |  | Setting | $N$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$
---|---|---|---|---|---|---|---|---|---
Bias |  |  |  |  |  |  |  |  |
| 1A | 400 | -0.0030 | 0.0019 |  | 2A | 400 | -0.0114 | -0.0094
|  | 800 | 0.0011 | -0.0010 |  |  | 800 | -0.0051 | 0.0050
|  | 1600 | 0.0003 | 0.0003 |  |  | 1600 | 0.0019 | 0.0015
| 1B | 400 | -0.0010 | -0.0015 |  | 2B | 400 | 0.0108 | -0.0081
|  | 800 | -0.0003 | 0.0005 |  |  | 800 | -0.0052 | 0.0043
|  | 1600 | 0.0001 | 0.0003 |  |  | 1600 | -0.0009 | 0.0009
Std |  |  |  |  |  |  |  |  |
| 1A | 400 | 0.0356 | 0.0482 |  | 2A | 400 | 0.0439 | 0.0449
|  | 800 | 0.0232 | 0.0309 |  |  | 800 | 0.0305 | 0.0314
|  | 1600 | 0.0195 | 0.0249 |  |  | 1600 | 0.0215 | 0.0217
| 1B | 400 | 0.0222 | 0.0349 |  | 2B | 400 | 0.0471 | 0.0482
|  | 800 | 0.0138 | 0.0184 |  |  | 800 | 0.0317 | 0.0334
|  | 1600 | 0.0136 | 0.0167 |  |  | 1600 | 0.0268 | 0.0282

For the DGPs in Settings 2A and 2B that violate $\|\theta_{0}\|=1$, the
estimated value $\hat{\theta}_{1}$ and $\hat{\theta}_{2}$ converge to
$1/\sqrt{2}$ and -$1/\sqrt{2}$, respectively. For all sample sizes we studied,
the estimators have relative small biases. The standard deviations of
$\hat{\theta}_{1}$ and $\hat{\theta}_{2}$ become smaller as the sample size
increases. The asymptotic normal approximation is accurate and the convergence
speed is $O(1/\sqrt{N})$ along all other direction orthogonal to the true
value $\theta_{0}$, in accordance with the results in Lemma C.2.

We further conduct the simulation study to investigate the performance of our
estimators for six dimensions of covariates. We consider the following four
DGPs for the propensity score with the vector of regressors
$X=(X_{1},X_{2},X_{3},X_{4},X_{5},X_{6})^{\prime}\sim N(0,I_{6})$:

  * •

3A. $g_{0}(X^{\prime}\theta_{0})=\sin(X^{\prime}\theta_{0})$, where
$\theta_{0}=(\sqrt{0.2},\sqrt{0.3},\sqrt{0.25},-\sqrt{0.1},\sqrt{0.08},-\sqrt{0.07})^{\prime}$.

  * •

3B.
$g_{0}(X^{\prime}\theta_{0})=0.5\\{(X^{\prime}\theta_{0})^{3}-(X^{\prime}\theta_{0})\\}$,
where $\theta_{0}$ is the same as in Setting 3A.

  * •

4A. $g_{0}(X^{\prime}\theta_{0})=\sin(X^{\prime}\theta_{0})$, where
$\theta_{0}=(\sqrt{0.2},\sqrt{0.4},\sqrt{0.6},-\sqrt{0.25},\sqrt{0.1},-\sqrt{0.45})^{\prime}$.

  * •

4B.
$g_{0}(X^{\prime}\theta_{0})=0.5\\{(X^{\prime}\theta_{0})^{3}-(X^{\prime}\theta_{0})\\}$,
where $\theta_{0}$ is the same as in Setting 4A.

The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$. The DGPs in Settings 3A
and 3B satisfy $\|\theta_{0}\|=1$, while the DGPs in Settings 4A and 4B
violate $\|\theta_{0}\|=1$.

We show the simulation results for Settings 3A and 3B in Table 5 and the
results for Settings 4A and 4B in Table 6, respectively. Our estimation
approach is valid for the six dimensions of covariates we considered. All
estimators have small biases for the three sample sizes we considered. The
biases vanish quickly when the sample size increases and RMSEs generally
decrease as the sample size increases. The asymptotic normal approximation is
accurate as the standard deviations become smaller as the sample size grows.
The convergence speed is consistent with the results in Lemma C.2.

Table 5: Simulation Results for the Estimators of Propensity Score Function of Settings 3A to 3B. | Setting | $N$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta_{3}}$ | $\widehat{\theta_{4}}$ | $\widehat{\theta_{5}}$ | $\widehat{\theta_{6}}$
---|---|---|---|---|---|---|---|---
Bias |  |  |  |  |  |  |  |
| 3A | 400 | -0.0059 | -0.0047 | -0.0076 | 0.0040 | -0.0047 | 0.0042
|  | 800 | 0.0025 | 0.0024 | -0.0028 | -0.0017 | 0.0016 | -0.0017
|  | 1600 | 0.0011 | -0.0013 | 0.0011 | -0.0009 | -0.0005 | 0.0011
| 3B | 400 | 0.0018 | 0.0036 | 0.0024 | 0.0031 | -0.0014 | 0.0043
|  | 800 | -0.0008 | 0.0010 | -0.0009 | 0.0006 | -0.0004 | 0.0009
|  | 1600 | 0.0001 | -0.0001 | -0.0001 | 0.0001 | 0.0002 | 0.0001
Std |  |  |  |  |  |  |  |
| 3A | 400 | 0.0524 | 0.0542 | 0.0629 | 0.0476 | 0.0384 | 0.0641
|  | 800 | 0.0340 | 0.0351 | 0.0392 | 0.0293 | 0.0230 | 0.0381
|  | 1600 | 0.0236 | 0.0244 | 0.0266 | 0.0199 | 0.0152 | 0.0253
| 3B | 400 | 0.0326 | 0.0387 | 0.0387 | 0.0339 | 0.0244 | 0.0511
|  | 800 | 0.0203 | 0.0210 | 0.0232 | 0.0173 | 0.0137 | 0.0267
|  | 1600 | 0.0142 | 0.0147 | 0.0161 | 0.0120 | 0.0100 | 0.0165

Table 6: Simulation Results for the Estimators of Propensity Score Function of Settings 4A to 4B. | Setting | $N$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta_{3}}$ | $\widehat{\theta_{4}}$ | $\widehat{\theta_{5}}$ | $\widehat{\theta_{6}}$
---|---|---|---|---|---|---|---|---
Bias |  |  |  |  |  |  |  |
| 4A | 400 | 0.0134 | 0.0219 | -0.0234 | -0.0165 | -0.0092 | 0.0199
|  | 800 | 0.0123 | 0.0137 | -0.0214 | -0.0146 | 0.0086 | -0.0142
|  | 1600 | 0.0029 | 0.0114 | -0.0116 | -0.0103 | -0.0030 | -0.0114
| 4B | 400 | 0.0125 | -0.0265 | -0.0134 | 0.0195 | 0.0071 | 0.0189
|  | 800 | 0.0114 | -0.0055 | 0.0125 | 0.0056 | -0.0068 | 0.0070
|  | 1600 | 0.0033 | 0.0070 | -0.0064 | -0.0032 | -0.0046 | -0.0014
Std |  |  |  |  |  |  |  |
| 4A | 400 | 0.0601 | 0.0467 | 0.0557 | 0.0526 | 0.0537 | 0.0677
|  | 800 | 0.0420 | 0.0323 | 0.0395 | 0.0371 | 0.0378 | 0.0472
|  | 1600 | 0.0293 | 0.0226 | 0.0273 | 0.0260 | 0.0268 | 0.0332
| 4B | 400 | 0.0432 | 0.0470 | 0.0487 | 0.0496 | 0.0426 | 0.0624
|  | 800 | 0.0337 | 0.0380 | 0.0374 | 0.0405 | 0.0319 | 0.0525
|  | 1600 | 0.0298 | 0.0271 | 0.0324 | 0.0287 | 0.0236 | 0.0375

We further conduct a Monte Carlo study to investigate the finite sample
properties of our semiparametric estimator proposed in Section 3.2, and
examine the average treatment effect under potential misspecification of
propensity score function and the relaxation of $\|\theta_{0}\|=1$ in Equation
3. We consider the following eight DGPs for both the propensity score and
potential outcome.

Setting 7A: $g_{0}(X^{\prime}\theta_{0})=\sin(X^{\prime}\theta_{0})$, where
$\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.5,-0.5)^{\prime}$, the
vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from standard
normal distributions $N(0,1)$. The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the outcome model is
generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$
and $\beta_{d}=1$.

Setting 7B:
$g_{0}(X^{\prime}\theta_{0})=0.5\\{(X^{\prime}\theta_{0})^{3}-(X^{\prime}\theta_{0})\\}$,
where $\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.5,-0.5)^{\prime}$,
the vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from
standard normal distributions $N(0,1)$. The propensity score function is
defined as $D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the
outcome model is generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where
$\epsilon\sim N(0,1)$ and $\beta_{d}=1$.

Setting 8A: $D=1\\{X^{\prime}\theta_{0}+\varepsilon>0\\}$, where
$\varepsilon\sim N(0,1)$ is independent of $X$,
$\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$, the
vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from standard
normal distributions $N(0,1)$. The outcome model is generated as
$Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$ and
$\beta_{d}=1$.

Setting 8B:
$D=1\\{(X^{\prime}\theta_{0})^{3}+(X^{\prime}\theta_{0})^{2}+X^{\prime}\theta_{0}+\varepsilon>0\\}$,
where $\varepsilon\sim N(0,1)$ is independent of $X$,
$\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$, the
vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from standard
normal distributions $N(0,1)$. The outcome model is generated as
$Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$ and
$\beta_{d}=1$.

Setting 9A: $g_{0}(X^{\prime}\theta_{0})=\sin(X^{\prime}\theta_{0})$, where
$\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$ and the
vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from standard
normal distributions $N(0,1)$. The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the outcome model is
generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon$ is the chi-
square with 1 degree of freedom minus its median and $\beta_{d}=1$.

Setting 9B:
$g_{0}(X^{\prime}\theta_{0})=0.5\\{(X^{\prime}\theta_{0})^{3}-(X^{\prime}\theta_{0})\\}$,
where $\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$
and the vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from the
standard normal distribution $N(0,1)$. The propensity score function is
defined as $D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the
outcome model is generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where
$\epsilon$ is Cauchy distribution and $\beta_{d}=1$.

Setting 10A: $g_{0}(X^{\prime}\theta_{0})=2X_{1}X_{2}$, where the vector of
covariates $X=(X_{1},X_{2})^{\prime}$ are generated from standard normal
distributions $N(0,1)$. The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the outcome model is
generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$
and $\beta_{d}=1$.

Setting 10B: $D=1\\{2X_{1}X_{2}+\varepsilon>0\\}$, where $\varepsilon\sim
N(0,1)$ is independent of $X$ and the vector of covariates
$X=(X_{1},X_{2})^{\prime}$ are generated from standard normal distributions
$N(0,1)$. The outcome model is generated as
$Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$ and
$\beta_{d}=1$.

The DGPs in Settings 7A and 7B violate $\|\theta_{0}\|=1$ in Equation 3, the
DGPs in Settings 8A and 8B have probit propensity score function and are
supposed to be handled by our semiparametric single-index models, DGPs in
Settings 9A and 9B have heavy-tail error distribution of the outcome
equations, and DGPs in Settings 10A and 10B impose additional misspecification
of propensity score function with respect to the semiparametric single-index
model.

Table 7: Simulation Results the Estimators of Average Treatment Effect of
Settings 7A to 10B.

| Setting | $N$ | $\beta_{d}$ | Setting | $N$ | $\beta_{d}$ | Setting | $N$ | $\beta_{d}$ | Setting | $N$ | $\beta_{d}$
---|---|---|---|---|---|---|---|---|---|---|---|---
Bias |  |  |  |  |  |  |  |  |  |  |  |
| 7A | 400 | 0.0044 | 8A | 400 | 0.0046 | 9A | 400 | 0.0039 | 10A | 400 | -0.0059
|  | 800 | -0.0021 |  | 800 | -0.0017 |  | 800 | -0.0022 |  | 800 | 0.0027
|  | 1600 | 0.0002 |  | 1600 | -0.0007 |  | 1600 | -0.0002 |  | 1600 | 0.0002
| 7B | 400 | 0.0028 | 8B | 400 | -0.0052 | 9B | 400 | -0.0026 | 10B | 400 | 0.0046
|  | 800 | -0.0011 |  | 800 | 0.0035 |  | 800 | 0.0011 |  | 800 | -0.0019
|  | 1600 | -0.0003 |  | 1600 | -0.0007 |  | 1600 | -0.0003 |  | 1600 | -0.0005
Std |  |  |  |  |  |  |  |  |  |  |  |
| 7A | 400 | 0.0402 | 8A | 400 | 0.0442 | 9A | 400 | 0.0370 | 10A | 400 | 0.0502
|  | 800 | 0.0277 |  | 800 | 0.0307 |  | 800 | 0.0255 |  | 800 | 0.0346
|  | 1600 | 0.0192 |  | 1600 | 0.0216 |  | 1600 | 0.0175 |  | 1600 | 0.0224
| 7B | 400 | 0.0366 | 8B | 400 | 0.0483 | 9B | 400 | 0.0337 | 10B | 400 | 0.0457
|  | 800 | 0.0253 |  | 800 | 0.0345 |  | 800 | 0.0234 |  | 800 | 0.0303
|  | 1600 | 0.0177 |  | 1600 | 0.0239 |  | 1600 | 0.0163 |  | 1600 | 0.0208

We summarize the simulation results in Table 7. Our estimator performs well
under the relaxation of $\|\theta_{0}\|=1$ and misspecification of propensity
score and outcome function. All estimators have relatively small biases,
standard deviations, and RMSEs for the three sample sizes we studied, and all
three terms generally decrease as the sample size increases. Our simulation
results show that our estimators are valid for certain degree of
misspecification in propensity score and outcome equation.

Appendix D.2: Comparison with other estimators

In this section, we compare the finite sample performance of our approach with
two alternative approaches in Liu et al. (2018) and Sun et al. (2021). Both
papers assume the boundedness of the link function and the boundedness of
function support, imposing additional constraints on the asymptotic theory and
numerical performance. We use the following four DGPs with unbounded link
function and unbounded support for propensity score functions.

Setting 11A: $g_{0}(X^{\prime}\theta_{0})=10\exp(X^{\prime}\theta_{0})$, where
$\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$ and the
vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from independent
standard normal distributions $N(0,1)$. The propensity score function is
defined as $D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the
outcome model is generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where
$\epsilon\sim N(0,1)$ and $\beta_{d}=1$.

Setting 11B:
$g_{0}(X^{\prime}\theta_{0})=10\\{(X^{\prime}\theta_{0})^{5}-(X^{\prime}\theta_{0})^{3}\\}+10\exp(X^{\prime}\theta_{0})$,
where $\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$
and the vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from
independent standard normal distributions $N(0,1)$. The propensity score
function is defined as $D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$
and the outcome model is generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$,
where $\epsilon\sim N(0,1)$ and $\beta_{d}=1$.

Setting 12A: $g_{0}(X^{\prime}\theta_{0})=10\exp(X^{\prime}\theta_{0})$, where
$\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$ and the
vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from independent
Cauchy distribution. The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the outcome model is
generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$
and $\beta_{d}=1$.

Setting 12B:
$g_{0}(X^{\prime}\theta_{0})=10\\{(X^{\prime}\theta_{0})^{5}-(X^{\prime}\theta_{0})^{3}\\}+10\exp(X^{\prime}\theta_{0})$,
where $\theta_{0}=(\theta_{0,1},\theta_{0,2})^{\prime}=(0.8,-0.6)^{\prime}$
and the vector of covariates $X=(X_{1},X_{2})^{\prime}$ are generated from
independent Cauchy distribution. The propensity score function is defined as
$D=\Lambda\big{(}g_{0}(X^{\prime}\theta_{0})\big{)}$ and the outcome model is
generated as $Y=\beta_{d}D+X_{1}+X_{2}+\epsilon$, where $\epsilon\sim N(0,1)$
and $\beta_{d}=1$.

Table 8:  Simulation Results for the Comparison between Different Estimators
of Average Treatment Effect of Settings 11A to 12B.

| Setting | $N$ | SP | Efficient | NP | Setting | $N$ | SP | Efficient | NP
---|---|---|---|---|---|---|---|---|---|---
Bias |  |  |  |  |  |  |  |  |  |
| 11A | 400 | -0.0445 | 0.1003 | -0.0899 | 12A | 400 | -0.0517 | 0.0816 | -0.0922
|  | 800 | 0.0237 | 0.0444 | 0.0480 |  | 800 | 0.0229 | -0.0436 | 0.0408
|  | 1600 | -0.0072 | 0.0168 | 0.0145 |  | 1600 | -0.0087 | 0.0131 | -0.0155
| 11B | 400 | 0.0384 | 0.0944 | -0.0776 | 12B | 400 | 0.0486 | 0.0705 | 0.0867
|  | 800 | 0.0204 | -0.0457 | 0.0411 |  | 800 | 0.0235 | -0.0374 | 0.0420
|  | 1600 | -0.0045 | -0.0081 | -0.0090 |  | 1600 | -0.0042 | -0.0082 | -0.0075
RMSE |  |  |  |  |  |  |  |  |  |
| 11A | 400 | 0.0625 | 0.1060 | 0.0934 | 12A | 400 | 0.0646 | 0.0838 | 0.0769
|  | 800 | 0.0410 | 0.0710 | 0.0613 |  | 800 | 0.0432 | 0.0550 | 0.0515
|  | 1600 | 0.0330 | 0.0546 | 0.0493 |  | 1600 | 0.0332 | 0.0443 | 0.0396
| 11B | 400 | 0.0672 | 0.1142 | 0.1004 | 12B | 400 | 0.0695 | 0.0902 | 0.0828
|  | 800 | 0.0467 | 0.0773 | 0.0697 |  | 800 | 0.0471 | 0.0626 | 0.0560
|  | 1600 | 0.0394 | 0.0654 | 0.0589 |  | 1600 | 0.0398 | 0.0529 | 0.0474
Std |  |  |  |  |  |  |  |  |  |
| 11A | 400 | 0.0684 | 0.1362 | 0.1176 | 12A | 400 | 0.0726 | 0.1038 | 0.1024
|  | 800 | 0.0427 | 0.0769 | 0.0682 |  | 800 | 0.0448 | 0.0607 | 0.0565
|  | 1600 | 0.0332 | 0.0554 | 0.0499 |  | 1600 | 0.0335 | 0.0448 | 0.0403
| 11B | 400 | 0.0716 | 0.1409 | 0.1185 | 12B | 400 | 0.0766 | 0.1051 | 0.1054
|  | 800 | 0.0479 | 0.0836 | 0.0748 |  | 800 | 0.0487 | 0.0668 | 0.0613
|  | 1600 | 0.0395 | 0.0656 | 0.0591 |  | 1600 | 0.0399 | 0.0531 | 0.0476

In Table 8, we show the finite sample performance of these three approaches,
where we denote our estimator as SP, the locally efficient estimator as
Efficient (Liu et al., 2018), and the nonparametric estimator using cross-
validation least squares to select bandwidth as NP (Sun et al., 2021). First,
all three estimators are nearly unbiased in all cases, while our estimator has
relatively smaller bias when either boundedness of the link function or
boundedness of function support does not hold. Second, our estimator has
smaller standard deviation and RMSE than the estimators by Liu et al. (2018)
and Sun et al. (2021) for all sample sizes we considered. Overall, our
proposed method dominates the existing methods for the data with unbounded
link function and unbounded support.

Appendix D.3: Additional simulation with various choices of truncation
parameter

In addition, we also show some extra simulation results for the finite sample
performance of the estimation approach with various choices of truncation
parameter $k$. Tables 9 and 10 show the simulation results of the propensity
score estimation with the choices of truncation parameter $k$ ranging from
$k=2$ to $k=6$. Tables 11 and 12 show the simulation results of the average
therapeutic effect with the choices of truncation parameter $k$ ranging from
$k=2$ to $k=6$. Overall, the estimators are valid against different choices of
truncation parameters.

Table 9: Simulation Results for the Estimators of Propensity Score Function
with Different Truncation Parameters for Setting 1A and 1B.

|  |  | $k=2$ | $k=3$ | $k=4$ | $k=5$ | $k=6$
---|---|---|---|---|---|---|---
| Setting | $N$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$
Bias |  |  |  |  |  |  |  |  |  |  |  |
| 1A | 400 | 0.0125 | 0.0077 | -0.0121 | 0.0075 | -0.0127 | 0.0078 | -0.0131 | 0.0080 | 0.0134 | 0.0083
|  | 800 | 0.0044 | 0.0042 | -0.0043 | -0.0041 | -0.0045 | -0.0043 | -0.0046 | 0.0044 | 0.0048 | 0.0045
|  | 1600 | -0.0006 | 0.0006 | 0.0006 | 0.0006 | -0.0006 | 0.0006 | 0.0006 | -0.0006 | -0.0007 | -0.0007
| 1B | 400 | -0.0040 | 0.0062 | -0.0039 | -0.0060 | -0.0041 | 0.0062 | -0.0042 | -0.0064 | -0.0043 | -0.0066
|  | 800 | -0.0011 | 0.0020 | -0.0011 | 0.0019 | -0.0012 | 0.0020 | -0.0012 | 0.0020 | 0.0012 | 0.0021
|  | 1600 | 0.0003 | 0.0006 | 0.0003 | 0.0006 | 0.0003 | 0.0006 | 0.0003 | 0.0006 | 0.0003 | 0.0007
Std |  |  |  |  |  |  |  |  |  |  |  |
| 1A | 400 | 0.0411 | 0.0553 | 0.0403 | 0.0542 | 0.0420 | 0.0564 | 0.0428 | 0.0575 | 0.0437 | 0.0587
|  | 800 | 0.0266 | 0.0353 | 0.0261 | 0.0347 | 0.0271 | 0.0361 | 0.0277 | 0.0368 | 0.0282 | 0.0375
|  | 1600 | 0.0223 | 0.0284 | 0.0228 | 0.0290 | 0.0219 | 0.0279 | 0.0232 | 0.0296 | 0.0237 | 0.0302
| 1B | 400 | 0.0254 | 0.0399 | 0.0249 | 0.0391 | 0.0259 | 0.0407 | 0.0265 | 0.0416 | 0.0270 | 0.0424
|  | 800 | 0.0158 | 0.0210 | 0.0154 | 0.0206 | 0.0161 | 0.0214 | 0.0164 | 0.0219 | 0.0167 | 0.0223
|  | 1600 | 0.0155 | 0.0191 | 0.0158 | 0.0195 | 0.0152 | 0.0187 | 0.0161 | 0.0199 | 0.0164 | 0.0203

Table 10: Simulation Results for the Estimators of Propensity Score Function
with Different Truncation Parameters for Setting 2A and 2B.

|  |  | $k=2$ | $k=3$ | $k=4$ | $k=5$ | $k=6$
---|---|---|---|---|---|---|---
| Setting | $N$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$ | $\widehat{\theta}_{1}$ | $\widehat{\theta}_{2}$
Bias |  |  |  |  |  |  |  |  |  |  |  |
| 2A | 400 | -0.0512 | 0.0418 | -0.0473 | 0.0387 | 0.0502 | -0.0410 | 0.0492 | 0.0402 | -0.0482 | -0.0394
|  | 800 | 0.0226 | -0.0223 | 0.0209 | 0.0206 | -0.0222 | 0.0219 | 0.0218 | -0.0215 | 0.0213 | -0.0211
|  | 1600 | -0.0086 | 0.0067 | -0.0079 | -0.0062 | 0.0084 | -0.0066 | -0.0082 | -0.0065 | -0.0081 | 0.0063
| 2B | 400 | 0.0481 | 0.0361 | 0.0445 | -0.0334 | -0.0472 | -0.0354 | -0.0463 | -0.0347 | -0.0453 | -0.0341
|  | 800 | 0.0233 | -0.0192 | 0.0215 | -0.0177 | 0.0228 | 0.0188 | 0.0224 | 0.0184 | -0.0220 | 0.0181
|  | 1600 | -0.0042 | -0.0042 | -0.0038 | 0.0039 | -0.0041 | 0.0041 | -0.0040 | 0.0040 | 0.0039 | 0.0040
Std |  |  |  |  |  |  |  |  |  |  |  |
| 2A | 400 | 0.0182 | 0.0159 | 0.0164 | 0.0145 | 0.0177 | 0.0155 | 0.0173 | 0.0152 | 0.0317 | 0.0300
|  | 800 | 0.0085 | 0.0085 | 0.0078 | 0.0078 | 0.0083 | 0.0083 | 0.0081 | 0.0081 | 0.0179 | 0.0180
|  | 1600 | 0.0055 | 0.0057 | 0.0052 | 0.0054 | 0.0055 | 0.0057 | 0.0054 | 0.0056 | 0.0129 | 0.0135
| 2B | 400 | 0.0181 | 0.0153 | 0.0164 | 0.0141 | 0.0176 | 0.0150 | 0.0172 | 0.0147 | 0.0328 | 0.0308
|  | 800 | 0.0092 | 0.0090 | 0.0085 | 0.0084 | 0.0090 | 0.0089 | 0.0088 | 0.0087 | 0.0194 | 0.0199
|  | 1600 | 0.0064 | 0.0067 | 0.0060 | 0.0064 | 0.0063 | 0.0066 | 0.0062 | 0.0065 | 0.0153 | 0.0161

Table 11: Simulation Results the Estimators of Average Treatment Effect with Truncation Parameters for Setting 5A to 5D. |  |  | $k=2$ | $k=3$ | $k=4$ | $k=5$ | $k=6$
---|---|---|---|---|---|---|---
| Setting | $N$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$
Bias |  |  |  |  |  |  |
| 5A | 400 | 0.0052 | 0.0049 | 0.0050 | 0.0051 | 0.0051
|  | 800 | -0.0003 | 0.0003 | -0.0003 | -0.0003 | -0.0003
|  | 1600 | 0.0001 | -0.0001 | 0.0001 | 0.0001 | -0.0001
| 5B | 400 | 0.0065 | 0.0062 | 0.0063 | 0.0063 | 0.0064
|  | 800 | -0.0006 | 0.0005 | -0.0006 | 0.0006 | -0.0006
|  | 1600 | -0.0001 | -0.0001 | 0.0001 | -0.0001 | 0.0001
Std |  |  |  |  |  |  |
| 5A | 400 | 0.0527 | 0.0487 | 0.0497 | 0.0507 | 0.0517
|  | 800 | 0.0365 | 0.0337 | 0.0344 | 0.0351 | 0.0358
|  | 1600 | 0.0284 | 0.0262 | 0.0268 | 0.0273 | 0.0278
| 5B | 400 | 0.0616 | 0.0569 | 0.0580 | 0.0592 | 0.0604
|  | 800 | 0.0445 | 0.0411 | 0.0420 | 0.0428 | 0.0436
|  | 1600 | 0.0327 | 0.0302 | 0.0308 | 0.0314 | 0.0320
Bias |  |  |  |  |  |  |
| 5C | 400 | 0.0056 | -0.0054 | -0.0054 | -0.0055 | -0.0056
|  | 800 | 0.0003 | -0.0002 | 0.0002 | -0.0002 | 0.0002
|  | 1600 | -0.0002 | 0.0002 | -0.0002 | 0.0002 | -0.0002
| 5D | 400 | 0.0029 | -0.0028 | 0.0028 | -0.0029 | 0.0029
|  | 800 | 0.0012 | 0.0011 | 0.0012 | 0.0012 | -0.0012
|  | 1600 | -0.0003 | 0.0003 | -0.0003 | -0.0003 | -0.0003
Std |  |  |  |  |  |  |
| 5C | 400 | 0.0508 | 0.0470 | 0.0479 | 0.0489 | 0.0499
|  | 800 | 0.0350 | 0.0323 | 0.0330 | 0.0337 | 0.0343
|  | 1600 | 0.0227 | 0.0210 | 0.0214 | 0.0218 | 0.0222
| 5D | 400 | 0.0463 | 0.0427 | 0.0436 | 0.0445 | 0.0454
|  | 800 | 0.0307 | 0.0284 | 0.0289 | 0.0295 | 0.0301
|  | 1600 | 0.0210 | 0.0194 | 0.0198 | 0.0202 | 0.0206

Table 12: Simulation Results the Estimators of Average Treatment Effect with Truncation Parameters for Setting 7A to 8B. |  |  | $k=2$ | $k=3$ | $k=4$ | $k=5$ | $k=6$
---|---|---|---|---|---|---|---
| Setting | $N$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$ | $\widehat{\beta}_{d}$
Bias |  |  |  |  |  |  |
| 7A | 400 | -0.0058 | 0.0054 | 0.0055 | -0.0056 | 0.0057
|  | 800 | 0.0026 | -0.0024 | -0.0025 | 0.0025 | 0.0026
|  | 1600 | 0.0002 | -0.0002 | -0.0002 | -0.0002 | -0.0002
| 7B | 400 | 0.0030 | -0.0028 | 0.0029 | 0.0029 | 0.0030
|  | 800 | -0.0012 | 0.0011 | -0.0012 | 0.0012 | 0.0012
|  | 1600 | -0.0004 | 0.0003 | 0.0003 | -0.0003 | -0.0004
Std |  |  |  |  |  |  |
| 7A | 400 | 0.0497 | 0.0470 | 0.0476 | 0.0483 | 0.0490
|  | 800 | 0.0342 | 0.0324 | 0.0328 | 0.0333 | 0.0337
|  | 1600 | 0.0222 | 0.0210 | 0.0212 | 0.0215 | 0.0218
| 7B | 400 | 0.0452 | 0.0427 | 0.0433 | 0.0439 | 0.0446
|  | 800 | 0.0300 | 0.0284 | 0.0287 | 0.0292 | 0.0296
|  | 1600 | 0.0205 | 0.0194 | 0.0197 | 0.0200 | 0.0203
Bias |  |  |  |  |  |  |
| 8A | 400 | 0.0094 | -0.0086 | 0.0088 | -0.0090 | 0.0092
|  | 800 | -0.0019 | 0.0017 | -0.0018 | -0.0018 | 0.0019
|  | 1600 | -0.0008 | 0.0007 | -0.0008 | 0.0008 | -0.0008
| 8B | 400 | -0.0165 | -0.0152 | 0.0155 | 0.0159 | -0.0162
|  | 800 | 0.0092 | -0.0085 | -0.0087 | -0.0089 | 0.0091
|  | 1600 | 0.0008 | -0.0007 | -0.0008 | -0.0008 | -0.0008
Std |  |  |  |  |  |  |
| 8A | 400 | 0.0549 | 0.0519 | 0.0526 | 0.0534 | 0.0541
|  | 800 | 0.0317 | 0.0299 | 0.0304 | 0.0308 | 0.0312
|  | 1600 | 0.0211 | 0.0200 | 0.0203 | 0.0206 | 0.0209
| 8B | 400 | 0.0606 | 0.0573 | 0.0581 | 0.0589 | 0.0598
|  | 800 | 0.0394 | 0.0372 | 0.0377 | 0.0383 | 0.0388
|  | 1600 | 0.0211 | 0.0200 | 0.0203 | 0.0206 | 0.0209

Appendix E: Additional Empirical Results

We apply our semiparametric methods to study the effect of job training on
future earnings. Our dataset is based on the classic National Supported Work
(NSW) demonstration dataset, as analyzed by LaLonde (1986) and reconstructed
by Dehejia and Wahba (1999). The NSW experiment took place from March 1975
until June 1977 and randomly assigned participants to the treatment group who
received a guaranteed job for 9 to 18 months and frequent counselor meetings
or control groups who were left in the labor market by themselves.

The outcome variable $Y$ is the participant’s earnings in 1978. The binary
treatment variable $D$ is the job training status ($D=1$ indicates participant
is assigned training, $D=0$ indicates participant is not assigned training).
The covariates $X$ include the education, ethnicity, age, and employment
variables before treatment, including earnings in 1974 and 1975. Following the
Imbens and Rubin (2015), we augment this dataset with observations from the
Current Population Survey (CPS). We focus on estimating the average treatment
effect of the program for participants using the comparison group from the
CPS. Due to data availability for the randomly assigned control group, we can
assess whether the non-experimental estimators are accurate. We provide the
summary statistics for these two datasets in Table 13 and see much larger
differences between the two groups, suggesting that it is important to
carefully adjust for these differences in estimating the average treatment
effects.

Table 13: Summary Statistics for Job Training data. | Treatment Group |  | Experimental
---
Control Group
| Nonexperimental
---
Control Group
| $N=185$ | $N=260$ | $N=15992$
| Mean | Std | Mean | Std | Mean | Std
Age | 25.82 | 7.16 | 25.05 | 7.06 | 33.23 | 11.05
Education | 10.35 | 2.01 | 10.09 | 1.61 | 12.03 | 2.87
Black | 0.84 | 0.36 | 0.83 | 0.38 | 0.07 | 0.26
Hispanic | 0.06 | 0.24 | 0.11 | 0.31 | 0.07 | 0.26
Married | 0.19 | 0.39 | 0.15 | 0.36 | 0.71 | 0.45
Nodegree | 0.71 | 0.46 | 0.83 | 0.37 | 0.3 | 0.46
Earnings (1974) | 2095.57 | 4886.62 | 2107.03 | 5687.91 | 14016.8 | 9569.8
Earnings (1975) | 1532.06 | 3219.25 | 1266.91 | 3102.98 | 13650.8 | 9270.4
Earnings (1978) | 6349.14 | 7867.4 | 4554.8 | 5483.84 | 14846.66 | 9647.39

As the dataset has 7,657 observations in the control group with propensity
scores less than 1.00e-05, the propensity score-related estimators may not be
readily applied in this case. We compare the results of average treatment
effect estimation based on bias-corrected matching (MBC) estimator (Abadie and
Imbens, 2006, 2011) and regression adjustment (RA) estimator (Lane and Nelder,
1982). We summarize the estimation results for the average treatment effect of
job training on earnings in 1978 in Table 14 along with the mean and standard
deviation. We first show the average difference between treatment group and
experiment control group is 1794.34 and statistically significant. The RA
estimator suggests that the job training increases the further earning by
3680.60 but is statistically insignificant. The MBC estimator shows that the
average effect of 6293.11 is statistically significant but quite different
from the experimental effect of 1794.34. Our ATE estimator is 4778.24 while
variance-weighted ATE estimator is 2914.72. Both estimators are statistical
significant, and the estimate of variance-weighted ATE is closest to the
difference between treatment and experiment control group among all estimators
we considered, suggesting that the our estimators may provide relatively
accurate estimation and inference for the dataset with limited overlap.

Table 14: Average Treatment Effect Estimation for the Job Training Data.

| ATE | Std | Z-statistics |  $p$-value | 95% CI
---|---|---|---|---|---
Difference | 1794.34 | 671.00 | 2.67 | 0.01 | 474.01 | 3114.67
MBC | 6293.11 | 1532.69 | 4.11 | 0.00 | 328.00 | 9297.12
RA | 3680.60 | 2622.33 | 1.40 | 0.16 | -1459.07 | 8820.27
Our Estimator ($\Delta^{ATE}$) | 4778.24 | 631.95 | 7.56 | 0.00 | 3539.62 | 6016.86
Our Estimator ($\Delta_{\omega}^{ATE}$) | 2914.72 | 391.81 | 7.44 | 0.00 | 2146.77 | 3682.67

Figure 2: Estimated nonparametric link function $g(\omega)$ of the propensity
score for the job training data.

We plot the estimated link function $\hat{g}$ of propensity score using the
wild bootstrap simulation with 500 bootstrap repetitions in Figure 2. The
estimated link function $\hat{g}$ is shown in a solid line, the corresponding
95% confidence interval is drawn in dashed lines. We calculate the optimal
truncation parameter $k$ based on the minimization of the prediction mean of
squares. In this data example, the optimal truncation parameter $k=2$. We
estimate the approximated second-order polynomial fitted function using the
ordinary least squares estimation approach as a reference as following (with
standard errors in bracket):

$\widehat{g}_{k}(\omega_{i})=\underset{(0.098)}{1.34}-\underset{(0.072)}{0.47}\omega_{i}+\underset{(0.008)}{0.12}\omega_{i}^{2},$

where $\omega_{i}$ is the $x_{i}^{\prime}\hat{\theta}$ for $1\leq i\leq N$.

For reference, we also plot the approximated second-order polynomial fitted
function using the ordinary least squares estimation approach. As shown, the
estimated link functions $\hat{g}$ in the data sample of job training program
are quite far away from the identity function, meaning that the logistic
parametric assumption for propensity score may not be valid. One reason is
that in the job training program data sample, there are 185 observations in
treated group, 15,992 observations in control group, and 7,657 observations in
the control group with propensity scores less than 1.00e-05. Therefore, most
of the observations have the zero value for propensity score, making the
regular parametric assumptions not feasible in this particular data sample.
The estimated link function is very close to the approximated second-order
polynomial fitted function, showing our model and method work well in this
data sample.


Background and Methodology

Definitions Fact-Driven Questions We are particularly interested in
determining how LLMs model human answers to fact-driven questions on social
media. By fact-driven questions, we mean posts which contain a question with
factually-grounded answers. We consider each post on Reddit’s r/Ask{Topic}
communities to either be fact-driven, or subjective (non-fact-driven), as
exemplified in Table tab:examples. While this binary formulation of fact-
driven is a simplification of a complex idea, classifying posts in this manner
enables us to filter out questions which would likely require human experience
to answer.

Answer Score Each comment is assigned, via Reddit, a score based on peer
engagement and human preference. This peer-assigned score is a measure of the
upvote to downvote ratio normalized by comment
age.https://www.reddit.com/wiki/faq/https://www.reddit.com/wiki/faq/We use
this measure to proxy peer perception. Highly-rated answers are deemed more
valuable or helpful, while low-rated ones are seen as less helpful, aligning
with prior research on social media engagement sharma2020engagement,
trunfio2021conceptualising.

Perplexity To determine whether an LLM models a human answer well, we use
perplexity jelinek1977perplexity. Intuitively, perplexity measures how
“surprised” the LLM is by an answer. If the perplexity is low, the LLM is
expecting to see this answer, meaning the LLM can model the autoregressive
linguistic properties of the answer. This metric is commonly used to measure
performance of language models in encoding linguistic phenomena
belinkov2019analysis. We use perplexity to measure how well an LLM models
human answers to fact-driven questions asked on Reddit’s r/Ask{Topic}
subreddits. Given a human answer $a$ comprised of a sequence of English tokens
$a=[a_{1},\cdots,a_{|a|}]$, an LLM $M$ assigns autoregressive conditional
token probabilities $p_{M}(a_{i}|a_{1}\cdots a_{i-1})$ to each token $a_{i}$
in answer $a$. The perplexity of an answer $a$ under language model $M$ is
defined as follows.

PPL_M(a) = exp{ - 1—a— ∑_i^—a— logp_M(a_i — a_1 ⋯a_i-1 ) } Perplexity is the
exponentiated average negative log-likelihood of tokens in a sequence, and can
be thought of as a measure of LLM modeling error over a sequence.

##  Relevant Data Collection and Processing

Some of the largest communities on Reddit are the r/Ask{Topic} communities,
including millions of members who engage in topic-specific discussions with
peers, e.g., r/AskWomen has 5.5 million members. We collect fact-driven
question-answers from these communities using the following steps. First, 15
r/Ask{Topic} communities across three categories are selected for
investigation, as seen in Table . We select these popular subreddits to
capture diverse content, enabling us to investigate fact-driven questions with
high peer engagement (captured as numbers of upvotes and comments). Second, a
random sample of 50% of all posts and comments, dating back to 2011, are
collected from each of these communities. This represents several million such
posts and comments. We filter to exclude deleted or removed posts and
comments, or exceptionally long posts and comments ($>10,000$ characters)
since they often contain personal narratives rather than fact-driven
questions. Third, posts with less than three top-level comments are then
excluded, as we wish to examine questions with high peer engagement. We
consider top-level comments to be answers to the primary question in each
post.

Category | Type | #Q | #A | %F
---|---|---|---|---
5*Profession | Historians | 35 | 164 | 42%
| Academia | 13 | 72 | 37%
| Engineers | 27 | 394 | 42%
| Culinary | 42 | 302 | 45%
| Photography | 21 | 168 | 44%
5*Identity | Men | 15 | 332 | 50%
| Women | 100 | 2925 | 47%
| MenOver30 | 7 | 180 | 32%
| WomenOver30 | 9 | 268 | 31%
| OldPeople | 7 | 245 | 58%
5*Geographic | UK | 67 | 1684 | 48%
| NYC | 49 | 392 | 51%
| France | 7 | 273 | 47%
| Singapore | 8 | 132 | 47%
| Argentina | 2 | 11 | 50%
Table : Final dataset statistics of the questions and answers gathered from 15
r/Ask{Topic} subreddits across 3 topic categories. Here, #Q, #A, and %F
indicate the number of fact-driven questions, the number of answers to those
questions, and the percentage of filtered questions which were deem fact-
driven.

[width=1.8]figures/main.png

Figure : LLM perplexity of answers to fact-driven questions posed on 15 of
Reddit’s r/Ask{Topic} communities, compared with the peer-assigned score of
answers. The perplexities of the top row are calculated by the vanilla SL LLM,
and the bottom row are calculated the fine-tuned SLFT LLM. For each graph, the
X and Y axes indicate peer-assigned scores and LLM’s perplexity in log scale,
respectively.

Fourth, we then exclude posts whose comments all have the same rating. As we
wish to explore LLM modeling capability as it correlates with human
perception, we are primarily interested in questions whose answers have
varying human ratings. When top-level comments all have the same rating, this
typically indicates that the post received less peer engagement and is
therefore less suitable for our study. The filtering thus far results in a
sample of 903 questions (posts) and 15,711 answers (top-level comments).
Finally, we filter to consider only fact-driven questions. We do this by
passing each question to the large, open-source LLaMA-2-70B model
[touvron2023llama], asking whether the post contains a question with a
factually-grounded answer. Filtering social media posts using LLMs is common
in social media analysis [kikkisetti2024using, gatto2024scope]. To verify that
LLaMA-2-70B performs this fact-driven question binary classification task
intuitively, we select a sample of 100 posts and perform inter-annotator
agreement, with each post being labeled by two annotators. Each annotator is
familiar with Reddit and has vast experience in social computing and
factuality assessment tasks. Two human annotators agree with each other on 83%
of posts. Both human annotators agree with LLaMA-2-70B on 41% of posts, and at
least one agreed on 58% of posts. We attribute these inter-annotator agreement
scores to the limitation of treating questions as either fully fact-driven or
fully subjective, when in reality there exists a range of subjectivity.
However, we deem this classification to be a better alternative to considering
all posts, as such posts vary widely in context and scope. This process
results in a dataset of 409 questions (posts containing fact-driven questions)
from 15 r/Ask{Topic} across 3 categories, and 7,534 answers (top-level
comments containing answers) to those questions. We release all filtered
questions and answers, including those that are not deemed fact-driven by
LLaMA-2-70B, and leave further analysis of this filtering for future work.

##  Models

To evaluate LLM modeling of answers to questions posed on Reddit’s
r/Ask{Topic} subreddits, we utilize Sheared LLaMA, a light-weight version of
the open-source LLaMA2-7B LLM which is pruned to only 1.3 billion parameters
using target structured pruning [xia2023sheared]. We employ Sheared Llama as
opposed to it’s larger counterpart Llama2-7B as the later requires significant
computational resources to fine-tune. Thus, using Sheared Llama allows us to
(i) run experiments where we fine-tune an LLM on in-domain data and (ii) share
a model that is more useful to those in the community with limited access to
computational resources. To compare two common inference scenarios with LLMs,
we consider both settings: (i) SL: The out-of-the-box pre-trained Sheared
LLaMA 1.3B model, and (ii) SLFT: A version of the Sheared LLaMA 1.3B model,
fine-tuned on a curated dataset of 100,000 comments from r/AskReddit
111https://tinyurl.com/y6r68hkv. This model is fine-tuned using the
Huggingface Transformers library222https://huggingface.co/ on an A100 GPU for
1 epoch, using a max token length of 128, a learning rate of $2e^{-5}$, and a
batch size of 6.