# A Dual-Connection based Handover Scheme for Ultra-Dense Millimeter-Wave
Cellular Networks

Seongjoon Kang, Siyoung Choi, Goodsol Lee, and Saewoong Bahk  Department of
Electrical and Computer Engineering and INMC, Seoul National University,
Seoul, South Korea
Email: {sjkang, sychoi, gslee2<EMAIL_ADDRESS>
###### Abstract

Mobile users in an ultra-dense millimeter-wave cellular network experience
handover events more frequently than in conventional networks, which results
in increased service interruption time and performance degradation due to
blockages. Multi-connectivity has been proposed to resolve this, and it also
extends the coverage of millimeter-wave communications. In this paper, we
propose a dual-connection based handover scheme for mobile UEs in an
environment where they are connected simultaneously with two millimeter-wave
cells to overcome frequent handover problems. This scheme allows a mobile UE
to choose its serving link between the two mmWave connections according to the
measured SINRs and then the corresponding base stations may forward duplicate
packets to the UE. We compare our dual-connection based scheme with a
conventional single-connection based scheme through ns-3 simulation. The
simulation results show that the proposed scheme significantly reduces
handover rate and delay. Therefore, we argue that the dual-connection based
scheme helps mobile users achieve performance goals they require in ultra-
dense cellular environments.

###### Index Terms:

multi-connectivity, ultra-dense networks, millimeter-wave communication,
secondary cell handover, ns-3 simulation

##  I Introduction

The increasing growth in mobile users and traffic demand requires cellular
network operators to provide high network capacity [1]. As a promising
technology in the fifth-generation (5G), millimeter-wave (mmWave)
communication provides much higher data rate owing to its new spectrum bands
with a wider bandwidth. In particular, mmWave communication is aggressively
pushed ahead by exploiting 28 GHz band first and then higher band [2].
However, mmWave signals have a short transmission range and are vulnerable to
blockages due to the characteristics of mmWave propagation [3]. Specifically,
blockages due to buildings, forests, and people, etc. incur Non-line-of-sight
(NLOS) connections (i.e., multi-paths) between a mmWave base station (BS) and
a mobile user, which greatly lower its signal-to-noise-ratio (SNR) and degrade
quality-of-service (QoS) of the user. Hence, LOS connections are highly
desirable in mmWave communications.

To overcome blockage effects, multi-connectivity can be exploited where a
mobile user is simultaneously connected to multiple BSs, which clearly
increases the chance of LOS connections [4]. However, multi-connections of a
user to multiple BSs cause higher handover rate and consequently higher
network overhead [5, 6]. Specifically, the authors in [7] proposed a mmWave
secondary cell handover (SCH) scheme in an LTE-mmWave dual-connectivity
network, which basically uses a single mmWave connection and an LTE
connection. In this scheme, a mmWave connection is switched to another mmWave
cell that provides a larger signal-to-interference-plus-noise-ratio (SINR)
after time-to-trigger (TTT). However, even when dynamic TTT is adopted, if the
BS density is very high, a TTT based handover scheme increases handover rate
to maintain connectivity of the user with better SINR when frequent connection
changes occur.

In this paper, we propose a mmWave dual-connectivity scheme that a user
equipment (UE) is connected with two mmWave BSs and an LTE BS. By utilizing
the two mmWave links in an ultra-dense network, we exploit spatial diversity
of the two links, which provides more robust and stable connectivity in a
dynamic wireless environment, especially due to the characteristics of mmWave
channels. Our proposal helps to decrease handover rate in an ultra-dense
cellular network, which contributes to a significant reduction in network
control overhead. To support UE mobility under mmWave dual-connectivity, we
enhance the SCH scheme by triggering handover according to the channel state
between the UE and mmWave BSs.

We compare our dual-connection based SCH scheme with the single-connection
based one through ns-3 simulation. Simulation results show that our scheme
outperforms the single-connection based scheme in terms of QoS, network
overhead, spectral efficiency, and TCP performance. To the best of our
knowledge, we are the first to evaluate the performance of a mobile UE under
mmWave dual-connectivity, and propose a handover triggering algorithm suitable
for a dynamic mmWave cellular network.

The rest of the paper is organized as follows. Section II summarizes the
related work. Section III explains the considered network, channel and
propagation model. Section IV presents our proposed SCH scheme under mmWave
dual-connectivity environments. Section V shows simulation results and
explains the advantages of the proposed scheme. We conclude the paper in
Section VI.

##  II Related Work

An ultra-dense network with very short inter-BS distance and mmWave band
operation has been investigated as a promising deployment solution to meet 5G
requirements [8, 9]. Multi-connectivity in the ultra-dense mmWave network
achieves low session drop probability so that it provides robust communication
even when there exist heavy traffic and many blockages [10]. The authors in
[11] presented that multi-connectivity in mmWave cellular networks improves
overall network throughput compared to single-connectivity.

Through extensive ray tracing simulation using 3D urban building data, they
analyzed cell coverage and robustness in mmWave urban cellular network, and
revealed the necessity of a multi-frequency heterogeneous mmWave network
system [12].

A mobility handling scheme with multi-connectivity would be quite different
from conventional schemes with single- connectivity. Many recent studies
mathematically analyzed handover performance in ultra-dense networks[6, 13,
14]. In a network with separate control/user plane, the handover probability
was investigated in [6] and the analytic model for handover signalling
according to the UE’s velocity was proposed in [13]. In [14], the handover
rate in a multi-tier heterogeneous network was analyzed.

These works focused only on link level analysis and did not take into account
the whole network level performance. The authors in [7] conducted several
network level simulations using ns-3 and evaluated the performance of the SCH
scheme under a single-connection between a mmWave BS and a UE. However, the
single-connection in mmWave communication cannot guarantee reliability and
delay constraints because its link quality can be severely degraded due to
blockages and UE mobility.

##  III System Model

In this section, we describe the system model that we consider.

###  III-A Network Model

We take account of a downlink scenario where a UE is connected with one master
node (MN) and two secondary nodes (SNs) simultaneously. Each BS operates as an
MN or SN according to its radio characteristics such as carrier frequency as
shown in Fig. 1. The MN is responsible for forwarding data traffic from the
core network to the two SNs via wired links such as X2 interface and sends
control messages toward a UE. On the other hand, the SN works for sending data
traffic to a UE with the high data rate. Hence, we assume that the MN uses the
radio access technology (RAT) of low carrier frequency such as LTE where its
coverage is much larger than that of an SN. The SN uses the RAT of the high
carrier frequency of mmWave whose data rate is higher than that of the MN.

We assume that the MN covers the whole area we are interested in. SNs are
placed at regular intervals, where the distance between the two neighboring
SNs depends on the network density. We assume that the two SNs providing dual-
connectivity of a UE have different carrier frequencies to avoid inter-cell
interference. The UE moves along the path among SNs at a constant speed,
maintaining its connectivity with two neighbors SNs. Rectangular-shaped
obstacles are scattered randomly in a 2-dimensional plane to create blockages.

###  III-B Channel and Propagation Model

We use the mmWave channel model given in [15] for a link between an SN and a
UE, and the LTE channel model for a link between an MN and a UE. In [15], the
authors provide a realistic assessment, including long-term and short-term
fading, for mmWave micro- and pico-cellular networks in a densely deployed
environment. We assume that mmWave channel matrix, small scale fading and
beamforming gain models are the same as in [16].

In this paper, we use the following pathloss model

$PL(d)\,[dB]=\alpha+\beta 10\log_{10}(d)+\xi\,,\quad\xi\sim N(0,\sigma^{2})$
(1)

where $d$ is the distance between receiver and transmitter, $\sigma^{2}$ is
the lognormal shadowing variance, and $\alpha$ and $\beta$ are the parameters
given in [17].

We measure channel quality in terms of SINR for serving BS $j$ and UE, which
is computed as

$SINR_{j,UE}=\frac{\frac{P_{j,TX}}{PL_{j,UE}}G_{j,UE}}{\sum_{k\neq
j}G_{k,UE}\frac{P_{k,TX}}{PL_{k,UE}}+W_{tot}\times N_{0}},$ (2)

where $P_{j,TX}$ is the transmit power of BS $j$, $G_{j,UE}$ is the
beamforming gain, $PL_{j,UE}$ is the pathloss between BS $j$ and UE,
$PL_{k,UE}$ is pathloss between BS $k$ and UE, and $W_{tot}\times N_{0}$ is
the thermal noise. SINR values can be classified into three categories: LOS,
NLOS, and outage. The classified SINR is used as a criterion for handover
decision or path switching in the proposed scheme.

Figure 1: An illustration of the system model. An MN covers the whole area and
SNs are placed at regular intervals. A UE moving at a constant speed has a
single-connection to an MN and connections to two SNs simultaneously, and
moves along the street among SNs. Blockages are randomly scattered on 2-D
plane.

##  IV Secondary Cell Handover Scheme for Multi-Connectivity

Multi-connectivity in an ultra-dense network plays an important role in
improving the performance of mobile users in terms of throughput, handover
rate, and network overhead [6]. We consider mmWave dual-connectivity, where a
UE is connected with two mmWave SNs, while keeping a connection to one MN. One
is a serving SN, denoted as $SN_{serve}$, and the other is an idle SN, denoted
as $SN_{idle}$. We use only $SN_{serve}$ for data transmission which has
higher SINR value enough to meet service requirements, and the other mmWave
link plays a role for supporting the serving link intermittently. This leads
us to avoid discontinuity of data flow resulting from mmWave channel dynamics
and ensure a reliable connection. Swapping the serving link happens when its
SINR value is very low.

In our scheme, an MN forwards data traffic from the core network to two SNs,
and manages uplink or downlink control signals toward a UE. The two SNs
receive a sounding reference signal (SRS) periodically from the UE that they
serve, and estimate their downlink channel state under the assumption that
channel reciprocity holds [16]. An MN controls data traffic and makes handover
decision according to the channel state information.

Figure 2: SN handover procedures.

There are some benefits of using dual mmWave connections. First, connecting
with the two mmWave SNs enables fast intra-RAT switching so that the MN can
choose a mmWave link to serve the UE between the two links without incurring
signalling overhead.

When the link of $SN_{serve}$ is in NLOS or outage, $SN_{idle}$ serves the UE
on behalf of $SN_{serve}$ by switching the data path (not handover) without
generating control messages. This reduces the frequency of handover between
the SNs and service interruption time. Second, it makes $SN_{serve}$ avoid
suffering the buffer overflow problem or data loss in the Radio Link Control
(RLC) layer by forwarding RLC data to $SN_{idle}$ via X2 interface.

In other words, when the SINR value of $SN_{serve}$ is abruptly dropped by
blockages, it may incur many packet losses and retransmissions. If
$SN_{serve}$ quickly perceives its SINR dropping and forwards its buffered
data through X2 interface to $SN_{idle}$ with better link quality, the
duration that the link quality of the UE is bad will be reduced, which results
in preventing a lot of packet losses. Lastly, we apply a packet duplication
(PD) scheme for dual-connections, which makes an MN forward the same data to
both SNs to exploit diversity gain. On the condition that the two mmWave links
have lower SINR values than a certain threshold value or they are unstable, we
would get the link diversity gain from the PD scheme.

Since conventional handover schemes are based on single-connectivity between
the UE and a mmWave BS, and they do not consider the frequent handover problem
in ultra-dense networks. Our dual-connection based handover scheme aims to
achieve high reliability and reduce service interruption time. Dual-connection
based handover procedures are shown in Fig. 2, where Radio Resource Control
(RRC) reconfiguration messages are exchanged between the UE and MN to exploit
wide coverage of the MN unlike in [7].

Assuming that the MN obtains all channel information between the UE and SNs by
periodically receiving the link state information from each SN, the MN makes
all control decisions such as handover, path switching, and PD. Algorithm 1
presents the decision-making procedures at the MN for handover and path
switching, which uses the SINR values ($SINR_{serve}$, $SINR_{idle}$) reported
by the $SN_{serve}$ and $SN_{idle}$.

Algorithm 1 Handover decision algorithm

1: while There is at least one connectable $SN$ do

2: if $SINR_{serve}\leq SINR_{th}$ and $SINR_{idle}\leq SINR_{th}$ then

3: Send the same packets to two SNs during HO

4: if $SINR_{serve}^{target}\leq SINR_{idle}^{target}$ then

5: Handover to $SN_{idle}^{target}$

6: Forward the buffered data to $SN_{idle}^{target}$

7: else

8: Handover to $SN_{serve}^{target}$

9: end if

10: else if $SINR_{serve}\leq SINR_{th}$ and $SINR_{idle}>SINR_{th}$ then

11: Switch data forwarding path to $SN_{idle}$

12: Forward the buffered data to $SN_{idle}$

13: else if $SINR_{serve}>SINR_{th}$ and $SINR_{idle}>SINR_{th}$ then

14: if $SINR_{idle}>SINR_{serve}$ then

15: Switch data forwarding path to $SN_{idle}$

16: end if

17: end if

18: end while

  * •

The first case (lines 2 - 9) happens when the SINR values of the two SNs are
lower than a certain SINR threshold value, $SINR_{th}$. The MN makes handover
decision if there is a target SN that has a greater SINR value than the
threshold. The MN knows the SINRs of the two candidates SNs
($SN_{serve}^{target}$, $SN_{idle}^{target}$) from SRS of neighbor SNs. The MN
compares the two target SINR values ($SINR_{serve}^{target}$,
$SINR_{idle}^{target}$) and selects the higher one. If the selected one is
$SN_{idle}^{target}$, $SN_{serve}$ forwards the buffered data to
$SN_{idle}^{target}$ through X2 interface. So only one SN from the two
connected ones will be changed. Because the SINR values of the two connected
links are expected to be very low, the packet delay would be quite long during
the handover event. So, whenever a handover occurs, the MN sends the same data
to both SNs via X2 interfaces until all the handover procedures are completed
to avoid the long service interruption time.

  * •

The second case (lines 10 - 12) deals with the case when NLOS or outage occurs
in the transmission link and the other idle link has a higher SINR. In this
case, the MN knows the channel states and changes the data forwarding path.
The MN sends a control message to $SN_{serve}$ to trigger the buffer
forwarding to $SN_{idle}$ through X2 interface.

  * •

The last case (lines 13 - 16) occurs when the SINR values of the two connected
SNs are high enough. The MN selects one SN with a higher SINR and forwards
data to it via X2 interface.

In the above three cases, handover occurs only when the SINR values of the two
connected SNs are smaller than the $SINR_{th}$, which depends on QoS
requirements of the UE. Only one mmWave link is used for data transmission
except when handover occurs. Provided that the density of SNs is very high,
the handover frequency in our scheme would be decreased, compared with that in
a single-connection based one. If the two links are in the outage and there is
no candidate link to support handover, the UE falls back to the MN, which is
the same as in [7].

Another important consideration in designing the handover scheme is the
selection of the TTT value that a UE should wait until handover starts. If the
mmWave signal is highly attenuated during the TTT time, too many packet losses
occur, resulting in retransmissions. The TTT value affects the frequency of
handover a lot in single-connection based schemes, whereas in our scheme it
has limited impact on handover rate because handover occurs according to the
state of the two links, not a single one. That is, even when the TTT value is
very low, unless both the SINR values of the two links are less than
$SINR_{th}$, handover does not occur.

##  V Simulation and Analysis

###  V-A Simulation Settings and Scenario

We modified the uplink based initial access scheme (IA) of NYU simulator [15]
to make a UE simultaneous connections with one MN and two SNs. The two SNs
connected with the UE use different mmWave carrier frequencies to avoid inter-
cell interference. While the UE is moving with dual connections, it
periodically tries to find its optimal beam direction toward SNs. To this end,
we developed a beam alignment scheme suitable for dual connections. Fig. 3
shows the user-plane protocol model in ns-3 that supports dual connections. We
added all the control procedures related to handover and data path switching
into RRC classes in MN and UE. The network device of the UE is equipped with
the protocol stacks corresponding to two SNs and one MN, and all data traffic
received from each RLC layer is aggregated at the Packet Data Convergence
Protocol (PDCP) class in the UE.

Figure 3: User-plane protocol model in ns-3 simulator supporting dual-
connectivity.

The parameters we used in the simulation are based on realistic system design
and summarized in Table I. Low carrier frequency is employed at the MN while
ultra high carrier frequency at SNs because the MN’s coverage is much larger
than that of an SN, ensuring stable signalling message exchange with the UE.
Specifically, LTE and mmWave protocols are installed at the MN and SNs,
respectively. We consider UDP and RLC AM protocols to remove unnecessary delay
and to ensure transmission reliability. Considering the coverage of a mmWave
BS [12], we emulate the deployment of mmWave BSs in a dense urban region,
where six BSs are located in an area of $100\times 100$ m2, setting the inter-
BS distance as 50 m. The UE moves along the street between mmWave BSs at a
constant speed of 10 m/s. Small blockages indicating humans, trees, vehicles,
etc., are randomly scattered to make mmWave channels more dynamic. They have
rectangular shapes with their widths uniformly distributed between 1.0 and 2.0
m.

TABLE I: Simulation parameters Parameter | Value
---|---
mmWave BS TX power (dBm) | 30
mmWave bandwidth (MHz) | 1000
mmWave BS antenna configuration |  $8\times 8$ ULA
UE antenna configuration |  $4\times 4$ ULA
Inter-BS distance (m) | 50
RLC mode | RLC AM
RLC buffer size (MB) | 100
LTE bandwidth (MHz) | 20
UE speed (m/s) | 10
LTE downlink carrier frequency (GHz) | 2.1
X2 link delay (ms) | 1.0
file size for downloading (MB) | 0.1, 1, 10, 100, 200
file transmission interval (ms) | 120
blockage size (m) | x, y dimensions $\in(1.0,~{}2.0)$
blockage density (blockages/km2) | 1000, 2000, 4000, 6000
TTT (ms) | 20
$SINR_{th}$ (dB) | 20

For comparison, we consider the single-connection based scheme that has
connections to one MN and one SN in [7]. To evaluate the benefits of using
dual-connection over single-connection, we simulate a file download scenario
and use performance metrics of file download completion time, handover rate,
file download failure ratio, and TCP throughput. In the simulation, the
serving SN periodically sends a fixed sized file to the UE with the interval
of 120 ms.

###  V-B Simulation Results and Discussions

We measured file download completion time according to the file size as shown
in Fig. 4. We observe that the dual-connection based scheme achieves smaller
download completion time by reducing the number of outlier points, which can
be interpreted as service interruption events. From these results, we noticed
that utilizing an additional mmWave link improves QoS performance. In
particular, when the file size is very large as much as 100 or 200 Mbytes, the
file download time using single-connection is much longer than that of using
dual-connection, which means that large file transmission on single-connection
is unlikely to guarantee QoS. In the dual-connection based scheme, there are a
few service interruption events. The reason is that channel state information
reported from SNs may not always help to estimate link state perfectly. Note
that received SINR values may change a lot after SRS reporting since the
wireless environment is dynamic.

We evaluate handover rate and transmission reliability under different
blockage densities. We define the handover rate as the total number of
handover trials divided by the whole simulation time. Although the BS density
is high, under high blockage density, it would be difficult to maintain a
stable connection between the UE and BS due to frequent handovers, resulting
in falling back to the MN. If we have an extra link, we can exploit spatial
diversity and reduce control signalling overhead caused by handover or falling
back. Fig. 5 shows the handover rate for single- and dual-connection. Our
dual-connection based scheme remarkably reduces the number of handovers and
accordingly overall network overhead. We observe that the UE does not
experience handover when the blockage density is below 1000 blockages/km2,
i.e., very low. In this case, switching the data path between the two
connected links is enough.

Figure 4: File download completion time with 4,000 blockages/km2.

Figure 5: Handover rate with the transmission file size of 1 Mbyte.

Figure 6: File download failure ratio.

(a) CWND variation

(b) Throughput

Figure 7: TCP performance comparison

To see transmission reliability, we take into account the file download
failure ratio, which is defined as the number of file download failures
divided by that of total file transmissions. A delay constraint is given to
assess whether a file is successfully downloaded or not. If the file does not
arrive within the delay constraint, it is regarded as a transmission failure.
Fig. 6 shows the file download failure ratio when blockage density varies, and
indicates that the dual-connection scheme achieves reliable file transmission.
However, like the single connection scheme, the dual-connection scheme doesn’t
guarantee high reliability either when the blockage density is high. This is
because, even if we utilize two mmWave connections, under high blockage
density, handover or falling back events occur frequently.

Lastly, we assess TCP (NewReno) performance in the same scenario and
simulation settings; the UE moves up and down with constant speed under the
blockage density of 4000 blockages/km2. A remote server sends data to the UE
with rate 200 Mbps during the simulation time, and the PDCP reordering
function is enabled for aggregating packets from the two paths. Figs.
LABEL:sub@subfig:tcp_cwnd and LABEL:sub@subfig:tcp_tput show that our dual-
connection based scheme outperforms the single-connection based one in terms
of TCP throughput and CWND (congestion window) variation. This is because the
proposed scheme successfully reduces both service interruption time and packet
losses caused by frequent handover events.

##  VI Conclusion

In this paper, we proposed a handover scheme that considers dual-connectivity
in the mmWave band, and evaluated its performance through ns-3 simulation. For
doing this, we modified the mmWave ns-3 modules that have been implemented by
NYU. Our handover scheme considers the case that a UE is connected to two
mmWave BSs and one LTE BS. In the simulation, we compared the performance of
the dual-connection based scheme with that of the single-connection based one,
in terms of file download time, hand over rate, file download failure ratio,
and TCP throughput.

When it comes to the use of radio resource, the dual-connection based scheme
utilizes twice as many controlling signals as a single-connection based scheme
to keep constant connections with a UE. However, our scheme achieves a more
robust and stable performance compared to conventional single-connection based
handover scheme although there are overheads of keeping the dual-connection
with a UE.

Our scheme shows more reduced and less fluctuating download completion time
according to the file size, compared to the single connection based scheme. By
varying the blockage density, we observed the handover rate and download
failure ratio. Our scheme shows significantly lowered handover rate and
download failure ratio, resulting in much better reliability. Lastly, we
observed our handover scheme achieves very much improved TCP performance in
terms of throughput and CWND variation.

## Acknowledgment

This research was supported by Basic Science Research Program through the
National Research Foundation of Korea (NRF) funded by the Ministry of Science,
ICT and Future Planning (No. 2017R1E1A1A01074358)

## References

  * [1] C. V. N. Index, “Cisco visual networking index: global mobile data traffic forecast update, 2014–2019,” _Tech. Rep_ , 2015.
  * [2] 3GPP TR 38.901, “Study on channel model for frequencies from 0.5 to 100 GHz,” ver. 14.0.0, Dec. 2017.
  * [3] S. Rangan, T. S. Rappaport, and E. Erkip, “Millimeter-wave cellular wireless networks: Potentials and challenges,” _Proc. IEEE_ , vol. 102, no. 3, pp. 366–385, 2014.
  * [4] A. K. Gupta, J. G. Andrews, and R. W. Heath, “Macrodiversity in cellular networks with random blockages,” _IEEE Trans. Wireless Commun._ , vol. 17, no. 2, pp. 996–1010, 2018.
  * [5] W. Bao and B. Liang, “Stochastic geometric analysis of handoffs in user-centric cooperative wireless networks.” in _Proc. IEEE INFOCOM_ , Apr. 2016, pp. 1–9.
  * [6] H. Zhang and W. Huang, “Tractable Mobility Model for Multi-Connectivity in 5G User-Centric Ultra-Dense Networks,” _IEEE Access_ , vol. 6, pp. 43 100–43 112, 2018.
  * [7] M. Polese, M. Giordani, M. Mezzavilla, S. Rangan, and M. Zorzi, “Improved handover through dual connectivity in 5G mmWave mobile networks,” _IEEE J. Sel. Areas Commun._ , vol. 35, no. 9, pp. 2069–2084, 2017.
  * [8] X. Ge, S. Tu, G. Mao, C.-X. Wang, and T. Han, “5G ultra-dense cellular networks,” _IEEE Wireless Commun._ , vol. 23, no. 1, pp. 72–79, 2016.
  * [9] R. Baldemair, T. Irnich, K. Balachandran, E. Dahlman, G. Mildh, Y. Selén, S. Parkvall, M. Meyer, and A. Osseiran, “Ultra-dense networks in millimeter-wave frequencies,” _IEEE Commun. Mag._ , vol. 53, no. 1, pp. 202–208, 2015.
  * [10] V. Petrov _et al._ , “Dynamic multi-connectivity performance in ultra-dense urban mmWave deployments,” _IEEE J. Sel. Areas Commun._ , vol. 35, no. 9, pp. 2038–2055, 2017.
  * [11] C. Tatino, I. Malanchini, N. Pappas, and D. Yuan, “Maximum throughput scheduling for multi-connectivity in millimeter-wave networks,” in _Proc. IEEE WiOpt_ , May 2018, pp. 1–6.
  * [12] L. Simic, S. Panda, J. Riihijarvi, and P. Mahonen, “Coverage and robustness of mm-wave urban cellular networks: multi-frequency HetNets are the 5G future,” in _Proc. IEEE SECON_ , Jun. 2017, pp. 1–9.
  * [13] A. Taufique, A. Mohamed, H. Farooq, A. Imran, and R. Tafazolli, “Analytical Modelling for Mobility Signalling in Ultra-Dense HetNets,” _IEEE Trans. Veh. Technol._ , 2018.
  * [14] S. Sadr and R. S. Adve, “Handoff rate and coverage analysis in multi-tier heterogeneous networks,” _IEEE Trans. Wireless Commun._ , vol. 14, no. 5, pp. 2626–2638, 2015.
  * [15] M. Mezzavilla, M. Zhang, M. Polese, R. Ford, S. Dutta, S. Rangan, and M. Zorzi, “End-to-end simulation of 5G mmwave networks,” _IEEE Commun. Surveys Tuts._ , 2018.
  * [16] M. Giordani, M. Mezzavilla, S. Rangan, and M. Zorzi, “Uplink-based framework for control plane applications in 5G mmWave cellular networks,” _IEEE Trans. Wireless Commun._ , submitted. [Online]. Available: http://arxiv.org/abs/1610.04836.
  * [17] M. R. Akdeniz, Y. Liu, M. K. Samimi, S. Sun, S. Rangan, T. S. Rappaport, and E. Erkip, “Millimeter wave channel modeling and cellular capacity evaluation,” _IEEE J. Sel. Areas Commun._ , vol. 32, no. 6, pp. 1164–1179, 2014.


# A Note on the Topology of the First Stage of 2SLS with Many Instruments

Guy Tchuente
University of Kent School of Economics and MaGHiC, Email:
<EMAIL_ADDRESS>The author thanks anonymous referees and the Associate
Editor for very thoughtful comments that have improved the current state of
the paper. This paper is currently ongoing an in-depth revision for Reject and
Resubmit at the JBES, please e-mail the author for comments and suggestions.

( July 2021)

###### Abstract

Finite sample properties of estimators are usually understood or approximated
using asymptotic theories. Two main asymptotic constructions have been used to
characterize the presence of many instruments. The first assumes that the
number of instruments increases with the sample size. I demonstrate that in
this case, one of the key assumptions used in the asymptotic construction may
imply that the number of “effective" instruments should be finite, resulting
in an internal contradiction. The second asymptotic representation considers
that the number of instrumental variables (IVs) may be finite, infinite or
even a continuum. The number does not change with the sample size. In this
scenario, the regularized estimator obtained depends on the topology imposed
on the set of instruments as well as on a regularization parameter. These
restrictions may induce a bias or restrict the set of admissible instruments.
However, the assumptions are internally coherent. The limitations of many IVs
asymptotic assumptions provide support for finite sample distributional
studies to better understand the behavior of many IV estimators.
Keywords: High-dimensional models, 2SLS, Many instruments, Regularization
methods.
JEL classification: C13, C31.

##  1 Introduction

This paper discusses implications of using many IVs (IVs) in an asymptotic
setting. I illustrate the many IVs asymptotic framework with the following
simple linear model:

$\displaystyle\displaystyle\underset{N\times 1}{y}$
$\displaystyle\displaystyle=$ $\displaystyle\displaystyle\underset{N\times
G}{X}\underset{G\times 1}{\delta}+\underset{N\times 1}{u},$ (1)
$\displaystyle\displaystyle X$ $\displaystyle\displaystyle=$
$\displaystyle\displaystyle\underset{N\times G}{f(Z)}+\underset{N\times
G}{V}.$ (2)

Empirical researchers are usually interested in the effect of a set of
variables, $\displaystyle X$, on an outcome, $\displaystyle y$. I assume
endogeneity of $\displaystyle X$ with respect to $\displaystyle\delta$ (i.e.
$\displaystyle X$ is correlated with the error term $\displaystyle u$). To
solve the endogeneity problem, some IVs ($\displaystyle Z$) are identified.
Having many instruments may be desirable as it could increase the fit of
Equation (2). However, the precision improvement comes at the expense of bias
in the estimation of $\displaystyle\delta$ using the IVs (see Bekker (1994)).
The behavior of the estimator in a finite sample is usually approximated using
asymptotic theories. An important question in theoretical econometrics is how
to formalize the presence of many instruments in an asymptotic construction.
In the many IVs literature, there are two main branches, differentiated by the
characterization of the IVs set.

The first and most important part of the theoretical econometrics literature
assumes that the number of instruments, represented by the number of columns
of $\displaystyle Z$ ($\displaystyle K$), grows with sample size
$\displaystyle N$. The asymptotic construction is done à la Bekker (1994),
i.e. with $\displaystyle\frac{K}{N}\longrightarrow\kappa$ or with a moderate
number of instruments $\displaystyle\frac{K^{d}}{N}\longrightarrow 0$, where
$\displaystyle d=2$ or $\displaystyle 3$. The value of $\displaystyle d$
depends on whether asymptotic convergence in probability or in distribution is
being investigated; see Andrews and Stock (2007), van Hasselt (2010), and
Anatolyev and Gospodinov (2011). In all these cases, the number of instruments
increases with the sample size.

A second strand of the literature considers the number of instruments as given
and unchanging with sample size. It proposes a framework in which estimation
can be done with fixed, infinite or a continuum of IVs. Seminal papers are
Carrasco and Florens (2000) for generalized method of moments (GMM) with a
continuum of moments condition, and Carrasco (2012) with regularized two-stage
least squares (2SLS).

In this paper, I use Bekker’s asymptotic approximation to refer to the case
where the number of IVs grows with the sample size and Carrasco’s asymptotic
approximation to refer to the case where there could be many, infinite or a
continuum of IVs. The aim of this paper is to understand the implications of
each approach’s key assumptions.

Using the 2SLS estimators (the regularized and the classical 2SLS), I present
an intuitive description of the challenges in using both types of many IVs in
an asymptotic environment. Indeed, the 2SLS estimator is the result of two
optimization processes. The first-stage optimization problem is to find
$\displaystyle K$ parameters to minimize the sum of the quadratic error in
Equation (2). The second stage replaces $\displaystyle X$ with its predicted
value from estimates obtained in the first stage.

In Bekker’s style of asymptotic approximation, the number of instruments
increases with the sample size. This means that the parameter space of the
first-stage estimation changes with more observations as the sample becomes
closer to the population size. This leads to changes in the first-stage
estimation that are not comparable, as the numbers of parameters
$\displaystyle K$ are different when the sample size increases. What are the
consequences of assuming a growing first stage on the optimization process
leading to the IV estimators? I show that certain assumptions proposed in the
literature resolve the problem described above by implying implicitly that the
number of effective IVs is finite.

In Carrasco’s asymptotic approximation, as the sample size approaches the
population size, the first-stage optimization problem’s dimension stays the
same. However, the problem may be infinite-dimensional, in the sense that the
object of interest in the first stage belongs to a space with infinite
dimensions. The infinite dimensionality of the problem implies that some
assumptions should be imposed on the set of instruments, and regularization
methods are needed to solve the problem. Carrasco (2012) imposes an assumption
about the compactness of the IVs covariance operator. Thus, the existence of a
regularized estimator depends on the specific topology imposed on the space
spanned by the IVs. Nevertheless, when the instruments are correlated, the
assumption is plausible and the set of assumptions are internally coherent.

I show that the compactness of the covariance operator assumption implies a
key assumption of the behavior of instruments in the Bekker’s asymptotic
approximation. In addition, the compactness assumption does not hold for
orthonormal IVs. In conclusion, the drawbacks associated with these two
asymptotic constructions suggests that finite sample distributional
investigation may be better for many IVs, consistent with Harding, Hausman,
and Palmer (2016) and Bun and Windmeijer (2011).

##  2 Many Instruments Asymptotic Approximation

This section presents the assumptions commonly used in the many IVs
literature. I first discuss the assumptions used in Bekker’s asymptotic
approximation method, followed by the assumptions in Carrasco’s many IVs
method. My main focus is the restrictions they imply as well as possible links
between the two types of many IVs.

In Bekker’s many instruments method, the asymptotic behavior of the estimator
of $\displaystyle\delta$ is usually obtained under the following assumptions:

Assumption 1: $\displaystyle{y_{i},X_{i},Z_{i},u_{i},V_{i}}$,
$\displaystyle{i=1,...,N}$ are $\displaystyle iid$. $\displaystyle u_{i}$ and
$\displaystyle V_{i}$ have a mean of zero and finite fourth moments, and the
variance of $\displaystyle(u_{i},V_{i})$ is non-singular.

Let us define the following quantities
$\displaystyle\sigma_{Vu}=E(V_{i}^{\prime}u_{i}),$
$\displaystyle\sigma_{u}^{2}=E(u_{i}^{2}),$
$\displaystyle\gamma=\sigma_{Vu}/\sigma_{u}^{2}$ and
$\displaystyle\tilde{V}=V-u\gamma^{\prime}$.

Assumption 2:
($\displaystyle i$) $\displaystyle E(u_{i}|\tilde{V},Z_{i})=0$, $\displaystyle
E(u_{i}^{2}|\tilde{V},Z_{i})=\sigma_{u}^{2}$ and for some $\displaystyle p>2$,
$\displaystyle E(\mid u_{i}\mid^{p}|\tilde{V},Z_{i})$ is bounded; and
($\displaystyle ii$) $\displaystyle Z^{\prime}Z$ is non-singular;
$\displaystyle f(Z)=Z\pi$.

Assumptions 1 and 2 (i) are common to both Bekker’s and Carrasco’s approaches.
Assumption 2 (ii) has an infinite-dimensional counterpart.

Assumption 3:
($\displaystyle i$) $\displaystyle\frac{K}{N}\longrightarrow\kappa$, with
$\displaystyle 0\leq\kappa<1$;
($\displaystyle ii$) $\displaystyle\pi^{\prime}Z^{\prime}Z\pi/N\longrightarrow
Q$, where $\displaystyle Q$ is a positive definite matrix; and
($\displaystyle iii$) $\displaystyle max_{i\leq N}\parallel\pi
Z^{\prime}_{i}\parallel/\sqrt{N}\longrightarrow 0$.

Assumption 3 creates the gap between the two strands of the literature on many
instruments. The crucial regularity assumption is that
$\displaystyle\pi^{\prime}Z^{\prime}Z\pi/N\longrightarrow Q$, where
$\displaystyle Q$ is a positive definite matrix.

The 2SLS estimator is often employed in the presence of many instruments. It
will be used to show the importance of these assumptions in the derivation of
their asymptotic behavior. The 2SLS estimator of $\displaystyle\delta$ given
by:

$\hat{\delta}=(X^{\prime}P_{Z}X)^{-1}X^{\prime}P_{Z}y,$ (3)

where $\displaystyle P_{Z}=Z(Z^{\prime}Z)^{-1}Z^{\prime}$ is the projection
matrix on the space spanned by the instrument.

The next subsection investigates the implications of Assumption 3 on the
asymptotic behavior of the 2SLS estimator.

###  2.1 Implications of Bekker’s Asymptotic Approximation Assumptions

Using Equation (1), I have

$\displaystyle\hat{\delta}-\delta=(X^{\prime}P_{Z}X)^{-1}X^{\prime}P_{Z}u=[X^{\prime}Z(Z^{\prime}Z)^{-1}Z^{\prime}X]^{-1}X^{\prime}Z(Z^{\prime}Z)^{-1}Z^{\prime}u.$

To prove the consistency of 2SLS with a fixed number of IVs, the law of large
numbers implies that
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}Z_{i}^{\prime}Z_{i}\rightarrow^{p}E(Z_{i}^{\prime}Z_{i})$,
where $\displaystyle E(Z_{i}^{\prime}Z_{i})$ is finite and nonsingular by
assumption. If I assume that $\displaystyle K$ goes to infinity,
$\displaystyle E(Z_{i}^{\prime}Z_{i})$ will become infinite dimensional. To
avoid dealing with this infinite-dimensional object, I use a sequential
asymptotic approach. I let the sample size and then $\displaystyle K$ each go
to infinity. The sequential asymptotic approximation is applied under
Assumption 3. Indeed, Stock and Yogo (2005) show, using Phillips and Moon
(1999), that the sequential asymptotic approximation is equivalent to the
joint asymptotic approximation (i.e. $\displaystyle K,N\rightarrow\infty$).

Assumption 3 ($\displaystyle ii$) says that
$\displaystyle\pi^{\prime}Z^{\prime}Z\pi/N\longrightarrow Q$, where
$\displaystyle Q$ is a positive definite matrix. This is a strong assumption
when joint asymptotic approximation is considered. For Assumption 3 to be
coherent with joint asymptotic approximation, the same value for
$\displaystyle Q$ should be obtained regardless of whether the convergence is
sequential.

Indeed, in such cases,
$\displaystyle\pi^{\prime}Z^{\prime}Z\pi/N=\pi^{\prime}\frac{Z^{\prime}Z}{N}\pi$
when $\displaystyle N$ and $\displaystyle K$ go to infinity simultaneously,
and $\displaystyle\pi$ and $\displaystyle\frac{Z^{\prime}Z}{N}$ become
infinite dimensional objects.

The law of large numbers implies that $\displaystyle\frac{Z^{\prime}Z}{N}$
converges to an infinite dimensional matrix $\displaystyle Q_{\infty}$, the
convergence norm needs to be specified.111As I have infinite dimensions, the
topology or the distance used for convergence is important.

Assuming that the quadratic product converges to $\displaystyle Q$ regardless
of the topology or the norm used can be a problem, because in infinite
dimensions all norms are not equivalent.

Van Hasselt (2010) notes that this assumption implicitly imposes on
$\displaystyle Z$; however, he does not discuss the implication of the
infinite dimensional nature of the imposed structure and the necessity of
discussing the specific norm used.

For a better understanding of the consequences of different strands assumption
I introduce the following definition.

Definition: effective instrument Let consider $\displaystyle Z_{K_{a}}$ the
$\displaystyle K^{th}$ instrument vector and, $\displaystyle\pi_{K_{a}}$ the
unobserved coefficient associated to the instrument $\displaystyle Z_{K_{a}}$
in the linear representation of the reduced form in Equation (2),
$\displaystyle|\pi_{K_{a}}|=O(c)$. If $\displaystyle\pi_{K_{a}}=0$, then the
instrument $\displaystyle Z_{K_{a}}$ is irrelevant. If $\displaystyle
c>\frac{1}{\sqrt{N}}$, the instrument is "effective".

###### Proposition 1

Under Assumptions 1 to 3, the existence of $\displaystyle Q$ within a joint
asymptotic framework implies that the number of effective IVs is finite.

Proof of Proposition 1.
The following proof uses a contradiction argument to show that the existence
of $\displaystyle Q$ implies that the number of instruments is finite. The
steps are the following:

  1. 1.

Assume that the number of effective instruments is infinite.

  2. 2.

Assume that the joint and sequential convergences have the same limit.

  3. 3.

Show that a sequence of matrices indexed by $\displaystyle K$, to obtain
Assumption 3, cannot be a Cauchy sequence, and thus cannot be convergent.222 A
sequence ($\displaystyle a_{K}$), $\displaystyle K>0$ is a Cauchy sequence, in
a space with norm $\displaystyle\|\|$, if for all arbitrary small positive
real number $\displaystyle\varepsilon$ there exist an integer $\displaystyle
K_{\varepsilon}$ such that for any $\displaystyle n,m>K_{\varepsilon}$,
$\displaystyle\|a_{n}-a_{m}\|<\varepsilon$.

Since $\displaystyle\pi$ is supposed to be known for any value of
$\displaystyle K$ as it goes to infinity, I can denote
$\displaystyle\pi_{\infty}$ as its limit value. Based on the equivalence
between joint and sequential convergence, $\displaystyle Q$ exists if and only
if

$\displaystyle\displaystyle Q$ $\displaystyle\displaystyle=$
$\displaystyle\displaystyle\pi_{\infty}^{\prime}M_{\infty}\pi_{\infty}$ (4)
$\displaystyle\displaystyle=$ $\displaystyle\displaystyle
plim_{K,N\rightarrow\infty}\pi_{K.}^{\prime}M_{K}\pi_{K.}$ (5)

where $\displaystyle M_{K}=plim_{N\rightarrow\infty}\frac{Z^{\prime}Z}{N}$ is
a $\displaystyle K\times K$ matrix and
$\displaystyle\pi_{K.}=(\pi_{1},\pi_{2},....,\pi_{K})^{\prime}$ a
$\displaystyle K\times G$ matrix.

The quantity $\displaystyle
plim_{K,N\rightarrow\infty}\pi_{K.}^{\prime}M_{K}\pi_{K.}$ is obtained using a
joint asymptotic approximation. It should converge to
$\displaystyle\pi_{\infty}^{\prime}M_{\infty}\pi_{\infty}$, where
$\displaystyle\pi_{\infty}$ is an infinite-dimensional vector and
$\displaystyle Q_{\infty}$ is an infinite-dimensional matrix.

For all values of $\displaystyle K$, the matrix
$\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$ belongs to space of
$\displaystyle G\times G$ matrices, $\displaystyle\mathbb{M}(G)$.
$\displaystyle(\mathbb{M}(G),\|.\|)$, where $\displaystyle\|.\|$ is the
Frobenius norm, is a complete finite-dimensional space. Thus, all Cauchy
sequences are convergent and vice versa.

The discussion below shows that $\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$
cannot be a Cauchy sequence in $\displaystyle K$ unless the number of relevant
instruments is fixed. Thus, the convergence of
$\displaystyle\pi^{\prime}\frac{Z^{\prime}Z}{N}\pi$ can generally only occur
with a fixed number of instruments.

Step 1: The number of effective instruments is assumed to be infinite.
This implies that for any $\displaystyle K_{0}>0$, there exists a
$\displaystyle K_{1}>K_{0}$ such that $\displaystyle\pi_{K_{1}}\neq 0$.

Step 2: Joint and sequential convergences have the same limit.
As a consequence, $\displaystyle
plim_{K\rightarrow\infty}\pi_{K.}^{\prime}M_{K}\pi_{K.}$ is convergent and has
to be a Cauchy sequence.

Step 3: Contradiction: I show that $\displaystyle
plim_{K\rightarrow\infty}\pi_{K.}^{\prime}M_{K}\pi_{K.}$ diverges.
Indeed, I prove that the sequence
$\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$ is not a Cauchy sequence.

Consider any integer $\displaystyle K_{L}$. If I take $\displaystyle K>K_{L}$,
by assumption, there exists an infinite number of $\displaystyle K_{j}>K$ such
that $\displaystyle\pi_{K_{j}}\neq 0$ with $\displaystyle j>1$, and for at
least one $\displaystyle K_{j}$, I can show that
$\displaystyle\|\pi_{K_{j}.}^{\prime}M_{K_{j}}\pi_{K_{j}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|$
does not neighbor zero. In other words, I can find an arbitrary small number
$\displaystyle\varepsilon_{K_{M}}$ such that
$\displaystyle\|\pi_{K_{j}.}^{\prime}M_{K_{j}}\pi_{K_{j}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|>\varepsilon_{K_{M}}$.

Indeed, let us consider $\displaystyle K_{2}$ as the minimum of $\displaystyle
K_{j}>K$ such that $\displaystyle|\pi_{K_{j}}|\geq\frac{1}{\sqrt{N}}$.

Consider the block matrix notation of $\displaystyle
M_{K_{2}}=\left(\begin{array}[]{cc}M_{K}&S_{K,K_{2}}^{\prime}\\\
S_{K,K_{2}}&M_{K,K_{2}}\\\ \end{array}\right),$ and
$\displaystyle\pi_{K_{2}.}=(\pi_{K.}^{\prime},\pi_{K+1},...,\pi_{K_{2}})^{\prime}$.

$\displaystyle\displaystyle\pi_{K_{2}.}^{\prime}Q_{K_{2}}\pi_{K_{2}.}$
$\displaystyle\displaystyle=$
$\displaystyle\displaystyle(\pi_{K.}^{\prime},\pi_{K+1},...,\pi_{K_{2}})\left(\begin{array}[]{cc}M_{K}&S_{K,K_{2}}^{\prime}\\\
S_{K,K_{2}}&M_{K,K_{2}}\\\
\end{array}\right)(\pi_{K.}^{\prime},\pi_{K+1},...,\pi_{K_{2}})^{\prime}$
$\displaystyle\displaystyle=$
$\displaystyle\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}+\pi_{K,K_{2}}^{\prime}S_{K,K_{2}}\pi_{K.}+(\pi_{K,K_{2}}^{\prime}S_{K,K_{2}}\pi_{K.})^{\prime}+\pi_{K,K_{2}}^{\prime}M_{K,K_{2}}\pi_{K,K_{2}}$

with $\displaystyle\pi_{K,K_{2}}=(\pi_{K+1},...,\pi_{K_{2}})^{\prime}$ and
$\displaystyle M_{K,K_{2}}$ is a square matrix
$\displaystyle(K_{2}-K)\times(K_{2}-K)$.

I can therefore conclude that

$\displaystyle\displaystyle\|\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|$
$\displaystyle\displaystyle=$
$\displaystyle\displaystyle\|\pi_{K,K_{2}}^{\prime}S_{K,K_{2}}\pi_{K_{2}.}+(\pi_{K,K_{2}}^{\prime}S_{K,K_{2}}\pi_{K_{2}.})^{\prime}+\pi_{K,K_{2}}^{\prime}M_{K,K_{2}}\pi_{K,K_{2}}\|$
(9) $\displaystyle\displaystyle=$
$\displaystyle\displaystyle\|\pi_{K_{2}.}^{\prime}\left(\begin{array}[]{cc}0&S_{K,K_{2}}^{\prime}\\\
S_{K,K_{2}}&M_{K,K_{2}}\\\ \end{array}\right)\pi_{K_{2}.}\|.$

Let us assume that the IVs are orthogonal and non-constant. Then,
$\displaystyle S_{K,K_{2}}=0$, which implies that

$\displaystyle\|\pi_{K.}^{\prime}M_{K}\pi_{K.}-\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}\|\geq
O_{p}(\frac{1}{N})>0.$

The above inequality is true because $\displaystyle M_{K,K_{2}}$ is a
symmetric positive semi-definite matrix and
$\displaystyle|\pi_{K_{2}}|\geq\sqrt{N}$. In this case,
$\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$ cannot be a Cauchy sequence.

Similarly, in the case where the new IVs ($\displaystyle K_{2}-K$ IVs) are
orthogonal to the $\displaystyle K$ first instrument, then $\displaystyle
S_{K,K_{2}}=0$ and thus

$\displaystyle\|\pi_{K.}^{\prime}M_{K}\pi_{K.}-\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}\|\geq
O_{p}(\frac{1}{N})>0,$

leading to a similar conclusion.

For any set of IVs, I can define

$\displaystyle A_{K_{2}}=\left(\begin{array}[]{cc}0&S_{K,K_{2}}^{\prime}\\\
S_{K,K_{2}}&M_{K,K_{2}}\\\ \end{array}\right).$

$\displaystyle A_{K_{2}}$ is a symmetric matrix. Therefore, from the Spectral
Theorem, there is a spectral decomposition $\displaystyle
A_{K_{2}}=ODO^{\prime}$ where $\displaystyle D$ is a diagonal matrix of
eigenvalues and $\displaystyle O$ is a matrix of orthonormal eigenvectors.
Thus from Equation (9) and the representation of $\displaystyle A_{K_{2}}$
above,

$\displaystyle\|\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|=\|\pi_{K_{2}.}^{\prime}ODO^{\prime}\pi_{K_{2}.}\|$

which is equal to zero if and only if $\displaystyle
D^{\frac{1}{2}}O^{\prime}\pi_{K_{2}.}=0$ (i.e. $\displaystyle\pi_{K_{2}.}=0$,
since $\displaystyle OD^{-\frac{1}{2}}D^{\frac{1}{2}}O^{\prime}=I$). Also,
because $\displaystyle\pi_{K_{2}}\neq O(\frac{1}{\sqrt{N}})$,
$\displaystyle\|\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|\geq
O_{p}(\frac{1}{N})>0$.

Because of the density of the space of real number, showing that
$\displaystyle\|\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|\geq
O_{p}(\frac{1}{N})>0$ implies that there exist a real number
$\displaystyle\varepsilon_{K_{L}}$ such that

$\displaystyle\|\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|\geq
O_{p}(\frac{1}{N})>\varepsilon_{K_{L}}>0.$

for all $\displaystyle K$ and $\displaystyle K_{2}$.

This means that $\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$ is not a Cauchy
sequence. Indeed, for any given integer $\displaystyle K_{L}$, I can find a
small real number $\displaystyle\varepsilon_{K_{L}}$, such that there are
$\displaystyle K,K_{2}>K_{L}$ with
$\displaystyle\|\pi_{K_{2}.}^{\prime}M_{K_{2}}\pi_{K_{2}.}-\pi_{K.}^{\prime}M_{K}\pi_{K.}\|>\varepsilon_{K_{L}}.$

In conclusion, if for any $\displaystyle K_{0}>0$ there exists a
$\displaystyle K_{1}>K_{0}$ such that $\displaystyle\pi_{K_{1}}\geq
O_{p}(\frac{1}{N})$; then, $\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$
cannot be a Cauchy sequence. It follows that there needs to be a fixed integer
$\displaystyle K_{L}$ such that for all $\displaystyle K>K_{L}$,
$\displaystyle|\pi_{K}|<O(\frac{1}{N})$.

This ends the proof of Proposition 1.

I have shown that Assumption 3 does not hold if joint asymptotic approximation
is applied. However, it is a reasonable assumption under sequential asymptotic
approximation. The equivalence between the two modes of convergence does not
hold in this case because the probability spaces of $\displaystyle Q$ and
$\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$ are different.333 $\displaystyle
Q$ is the limit in probability of a quantity, and the probability used in that
limit calculation is different from that induced by the random variables used
in the calculation of $\displaystyle\pi_{K.}^{\prime}M_{K}\pi_{K.}$.
Therefore, Lemma 6 of Phillips and Moon (1999) cannot be applied. Indeed, the
difference in the probability spaces reflects the fact that the first-stage
regression parameter dimension changes as the sample size increases. As the
sample size goes to infinity, it becomes difficult to establish the
consistency of the estimator of $\displaystyle\pi$ without further
assumptions. Indeed, it can be noted that if $\displaystyle\pi_{K_{2}}$ go to
zero with $\displaystyle N$ the term in Equation (9) will also go to zero.
This will be possible in the context of Staiger and Stock (1997) weak
instruments where $\displaystyle\pi=\frac{1}{\sqrt{N}}(1,...,1,...)$, or in
the situation of Belloni, Chen, Chernozhukov, and Hansen (2012) where the
first-stage is assumed to be sparse.

Assumption 3 ($\displaystyle ii$) implies that the number of instruments
should be fixed (even if large). Intuitively, this result means that as the
sample size increases, there is a point at which a full knowledge of the
first-stage equation is achieved and no more IVs are needed. However, in a
finite sample the number of instruments can be close to the sample size. This
will lead to the many instruments bias problem, which is usually solved by
introducing bias correction terms to obtain a bias-corrected 2SLS estimator.
Assumption 3, under which many Bekker’s type asymptotic approximation of the
estimators are constructed, suggests a fixed number of instruments. Therefore,
it is Likely that these asymptotic estimators may not be very good
approximation of finite sample behavior of the 2SLS estimator, as pointed out
by Harding, Hausman, and Palmer (2016), who use a finite sample approximation
as a solution to the many IVs problem. However, alternative asymptotic
estimators could be helpful for inference in large samples. One candidate is
Carrasco’s asymptotic approximation, which does not assume an increasing
number of IVs. The next subsection discusses the assumptions imposed by
Carrasco’s asymptotic approximation.

###  2.2 Implications of the Carrasco’s Asymptotic Approximation Assumptions

In some empirical applications, the structural equation can suggest a form of
Equation (1) implying the use of many IVs. Examples of such models include the
spatial model and dynamic panel. In these frameworks, the first-stage equation
results in an infinite sum. Equation (2) becomes

$\displaystyle X=\underset{N\times
G}{\sum_{j=1}^{\infty}Z(j)\pi_{j}}+\underset{N\times G}{V}.$

The present case contains infinite potential IVs, so the 2SLS estimator
proposed above cannot be used. Indeed, the number of moment conditions
generated by the model is infinite. In general, if the instruments are assumed
to be strictly exogenous, (following Carrasco (2012), this means that for
$\displaystyle E(u|z)=0$), any function of $\displaystyle z$ can serve as an
instrument. For instance, the set of $\displaystyle
Z_{i}(\tau)=exp(i\tau^{\prime}z_{i})$ with
$\displaystyle\tau\in\mathbb{R}^{b}$ and $\displaystyle z_{i}$ a vector of
exogenous variables from $\displaystyle\mathbb{R}^{b}$ can be considered as a
set of instruments. Carrasco (2012) proposes a regularized 2SLS estimator for
this type of many instruments problem. She uses a unifying framework in which
fixed, infinite and a continuum of moment conditions can be examined
simultaneously. To account for the high dimensional nature of the problem in
the first-stage equation, she proposes four regularization schemes: Tikhonov
(ridge), Landweber-Fridman, principal components and spectral cut-off. This
section uses the Tikhonov regularization to investigate the implications of
the assumptions proposed in this alternative asymptotic approximation
framework.

To use regularization methods, the matrix $\displaystyle Z^{\prime}Z/N$ is
replaced by the covariance operator $\displaystyle\mathbf{\Upsilon}$. In
particular, for a continuum of moment conditions, the covariance operator is
defined as:

$\mathbf{\Upsilon}:L^{2}(\mu)\hskip 2.168pt\rightarrow\hskip
2.168ptL^{2}(\mu)$ (10) $(\mathbf{\Upsilon}g)(\tau_{1})=\int
E(Z_{i}(\tau_{1})\overline{Z_{i}(\tau_{2})})g(\tau_{2})\mu(\tau_{2})d\tau_{2}$

where $\displaystyle\overline{Z_{i}(\tau_{2})}$ denotes the complex conjugate
of $\displaystyle Z_{i}(\tau_{2})$ and $\displaystyle\pi$ is a probability
density function. Details on the regularized 2SLS estimator are in Carrasco
(2012), and its value is given by

$\displaystyle\hat{\delta}_{R}=(X^{\prime}P^{\alpha}X)^{-1}X^{\prime}P^{{\alpha}}y$

where $\displaystyle P^{\alpha}$ is the infinite-dimensional analogue of the
projection matrix on the space spanned by IVs, and $\displaystyle\alpha$ is
the regularization parameter. I consider that $\displaystyle f$ function from
the space of exogenous variables to $\displaystyle\mathbb{R}^{G}$. The
asymptotic behavior of $\displaystyle\hat{\delta}_{R}$ is obtained under the
following assumptions:

Assumption 4: $\displaystyle E[f(z_{i})f(z_{i})^{\prime}]$ exists and is non-
singular, and $\displaystyle f(z_{i})$ belongs to the closure of the linear
span of {$\displaystyle Z_{i}(\tau)$ with $\displaystyle\tau\in\mathbb{R}$}.

Assumption 4 is the infinite-dimensional counterpart of Assumption 2
($\displaystyle ii$).

Assumption 5: $\displaystyle\mathbf{\Upsilon}$ is a compact or nuclear
operator.444Let $\displaystyle(X,\big{<}.,.\big{>}_{X})$ and
$\displaystyle(Y,\big{<}.,.\big{>}_{Y})$ be separable Hilbert spaces. An
operator T:$\displaystyle X\rightarrow Y$ is nuclear if it can be represented
by $\displaystyle Tx=\sum_{j=1}^{\infty}a_{j}\big{<}b_{j},x\big{>}$ for all
$\displaystyle x\in X$, where the sequences $\displaystyle\\{a_{j}\\}\subset
X$ and $\displaystyle\\{b_{j}\\}\subset Y$ such that
$\displaystyle\sum_{j=1}^{\infty}\|a_{j}\|_{X}\|b_{j}\|_{Y}<\infty$.

The new assumption introduced in this case is that the operator
$\displaystyle\mathbf{\Upsilon}$ is compact (see Carrasco (2012)) or nuclear
(see Carrasco and Tchuente (2016)).

###### Proposition 2

Assume that Assumptions 1, 2 ($\displaystyle i$) and 4 hold. Then, Assumption
5 implies Assumption 3 ($\displaystyle ii$).

Proof of Proposition 2.
I explore different numbers of instruments to prove the proposition in each
case. The case of finite instruments is trivial: both assumptions 3
($\displaystyle ii$) and 5 are implied by Assumptions 1, 2, and 4.

In the infinite-dimensional cases, the compactness assumption implies that the
spectrum of the operator is discrete. In other words, the operator can be
approximated by a finite-dimensional operator. In practice, this assumption is
crucial as it allows for a finite sum to represent the operator.

($\displaystyle i$) Finite number of instruments

Assumption 4 implies that $\displaystyle f(Z)=Z\pi_{0}$. The covariance
operator in this case is compact, so Assumptions 3 (ii) and 5 are trivially
satisfied.

($\displaystyle ii$) Infinite countable number of instruments
I consider that for each individual $\displaystyle i$, $\displaystyle Z_{i}$
belongs to the space of squared summable sequence $\displaystyle l^{2}$. Let
$\displaystyle x$ and $\displaystyle y$ be two elements of $\displaystyle
l^{2}$. The scalar product
$\displaystyle<x,y>=\sum_{n=1}^{\infty}x_{n}\bar{y}_{n}$ where
$\displaystyle\bar{y}_{n}$ is the conjugate of $\displaystyle y_{n}$. I denote
the norm associated with this scalar product by $\displaystyle\|.\|_{2}$, and
c ($\displaystyle l^{2},\|.\|_{2}$) is a Hilbert space. $\displaystyle Z_{i}$
are $\displaystyle iid$ elements of $\displaystyle l^{2}$ and $\displaystyle
E\left(\|Z_{i}()\|_{2}\right)<\infty.$

The application of Assumption 4 leads to

$\displaystyle f(Z)=\sum_{j=1}^{\infty}Z(j)\pi_{j}.$

The set of instruments is infinite and can be written as $\displaystyle Z$. It
can be shown that

$\frac{f(Z)^{\prime}f(Z)}{N}=\frac{[\sum_{j=1}^{\infty}Z(j)\pi_{j}]^{\prime}[\sum_{j=1}^{\infty}Z(j)\pi_{j}]}{N}=\pi^{\prime}\frac{Z^{\prime}Z}{N}\pi$
(11)

with $\displaystyle\pi$ a vector of $\displaystyle G$ sequence in
$\displaystyle l^{2}$. Thus,

$\pi^{\prime}\frac{Z^{\prime}Z}{N}\pi=\frac{\sum_{i=1}^{N}\pi^{\prime}Z_{i}^{\prime}Z_{i}\pi}{N}.$
(12)

Note that
$\displaystyle\pi_{g}Z_{i}^{\prime}Z_{i}\pi_{g}=<Z_{i}\pi_{g},Z_{i}\pi_{g}>=\|Z_{i}\pi_{g}\|_{2}$,
where $\displaystyle\pi_{g}$ is an element of $\displaystyle l^{2}$,
$\displaystyle g=1,..,G$. The covariance operator for this sequence is

$\mathbf{\Upsilon}:l^{2}\hskip 2.168pt\rightarrow\hskip 2.168ptl^{2}$ (13)
$(\mathbf{\Upsilon}\Lambda)_{k}=\sum_{n=1}^{\infty}E(Z_{i}(k)\overline{Z_{i}(n)})\Lambda(n)$

Applying Assumption 5 to $\displaystyle\mathbf{\Upsilon}_{l}$ means that
$\displaystyle\sum_{n=1}^{\infty}E(Z_{in}\overline{Z_{in}}\pi_{gn}^{2})<\infty$,
and thus $\displaystyle E(\|Z_{i}\pi_{g}\|_{2})<\infty.$ The application of
the weak law of large numbers gives that
$\displaystyle\frac{\sum_{i=1}^{N}\pi^{\prime}Z_{i}^{\prime}Z_{i}\pi}{N}$
converges in probability $\displaystyle Q$, where $\displaystyle Q$ is a
finite positive definite matrix.

This shows that Assumption 5 implies Assumption 3 ($\displaystyle ii$).

($\displaystyle iii$) Continuum of instruments

The presence of a continuum of moment conditions means that the first-stage
representation does not directly have a linear representation. The
corresponding representation of Assumption 3 ($\displaystyle ii$) is that
$\displaystyle\frac{f^{\prime}(Z)f(Z)}{N}\rightarrow Q$.

$\displaystyle f(Z)$ belongs to the closure of the linear span of
{$\displaystyle Z(\tau)$ with $\displaystyle\tau\in\mathbb{R}$} from
Assumption 4. Thus, there exists $\displaystyle p(\tau)$ with
$\displaystyle\tau\in\mathbb{R}$ such that

$f(Z_{i})=\int Z_{i}(\tau)p(\tau)d\tau.$ (14)

Using this general representation, it can be seen that

$\frac{f(Z)^{\prime}f(Z)}{N}=\int\frac{p^{\prime}(\tau_{1})Z^{\prime}(\tau_{1})Z(\tau)p(\tau)}{N}d\tau
d\tau_{1}.$ (15)

Assumption 5 claims that $\displaystyle\mathbf{\Upsilon}$ is a compact
operator also, I consider $\displaystyle Z_{i}(\tau)\in L^{2}(\mu)$ and are
$\displaystyle iid$ with $\displaystyle E(\|Z_{i}(.)\|)<\infty$ and I apply
the law of large numbers on the quantify
$\displaystyle\frac{p^{\prime}(\tau_{1})Z^{\prime}(\tau_{1})Z(\tau)p(\tau)}{N}$.
Assumption 5 implies that

$\displaystyle\int\frac{p^{\prime}(\tau_{1})Z^{\prime}(\tau_{1})Z(\tau)p(\tau)}{N}d\tau
d\tau_{1}\rightarrow Q.$

Thus, Assumption 5 implies Assumption 3 (ii). This ends the proof of
Proposition 2.

The above proposition suggests that a key assumption of the Carrasco
asymptotic approximation implies a controversial part of Assumption 3, which
is popular in the Bekker asymptotic approximation. The restriction of
compactness of the covariance operator ensures some ideal properties of the
set of instruments and the first stage equation exist.

For Assumption 3 to be coherent, additional restrictions should be imposed on
the parameters of the first equation. Note that $\displaystyle\pi$ needs to be
estimated, so a researcher should impose reasonable restrictions on the set of
instruments, rather than on the coefficient to be estimated.

Assumption 5 is defined in a specific topological environment (that is, the
norm used is clearly defined). This means that in Carrasco’s asymptotic
approximation, the results obtained depend on the norm used on the space
spanned by instruments, which is restrictive. In the estimation, the infinite
dimensions of the first stage implies that problem needs to be regularized to
obtain a solution. The choice of the regularization parameter is an additional
challenge.

To illustrate the use of a regularized estimator, consider the ridge
regularization applied to a model with multicollinearity or near-
multicollinearity. I assume that there are many IVs and that the same IVs are
used for the 2SLS estimator.

The regularized counterpart of the projection matrix is $\displaystyle
P^{\alpha}=Z([(Z^{\prime}Z)^{2}+\alpha I_{K}]^{-1}[Z^{\prime}Z])Z^{\prime}$,
with $\displaystyle\alpha>0$.

Given that $\displaystyle Z^{\prime}Z$ is a positive semi-definite matrix, it
can have the spectral decomposition $\displaystyle
Z^{\prime}Z=U^{\prime}\Lambda U$, where $\displaystyle\Lambda$ is a diagonal
matrix of eigenvalues and $\displaystyle U$ is a matrix orthonormal
eigenvector; thus, $\displaystyle P^{\alpha}=ZU^{\prime}([\Lambda^{2}+\alpha
I_{K}]^{-1}\Lambda)UZ^{\prime}$. The regularized 2SLS estimator proposed by
Carrasco (2012) is

$\displaystyle\hat{\delta}_{R}=\left(X^{\prime}U^{\prime}\Lambda_{\alpha}UX\right)^{-1}X^{\prime}U^{\prime}\Lambda_{\alpha}Uy$

with $\displaystyle\Lambda_{\alpha}=[\Lambda^{2}+\alpha I_{K}]^{-1}\Lambda$.

The regularized estimator depends on Assumption 5 via
$\displaystyle\Lambda_{\alpha}$ and $\displaystyle U$, and the compactness
assumption ensures that these quantities exist asymptotically. Under classical
assumptions, this is always the case in finite dimensions. However, in
infinite dimensions, the choice of the appropriate norm is crucial to ensure
compactness. For instance, if $\displaystyle Z^{\prime}Z$ approximates an
infinite sum such that $\displaystyle K>N$, the choice of norm becomes
crucial. Moreover, the presence of a regularization parameter in
$\displaystyle\hat{\delta}_{R}$ is a potential source of bias. However, its
choice can be based on data-driven optimal minimization of the approximated
mean-squared error (MSE). In Carrasco (2012), such procedures achieve smaller
MSEs than standard 2SLS estimators.

While the compactness of the covariance operator enables the development of a
coherent many instruments asymptotic framework, it has some limitations, such
as the type of instruments that can be used. The following proposition
presents the example of an inadmissible IV set.

###### Proposition 3

If I consider the set of instrumental variables (IVs) that are orthonormal,
Assumption 5 implies that the number of effective IVs is finite.

Proof of Proposition 3
Let us assume that I have an infinite number of instruments. This will
correspond to two cases: $\displaystyle(ii)$ infinite and countable or
$\displaystyle(iii)$ continuum of instrumental variables. For each individual,
the set of $\displaystyle\\{Z_{i}()\\}$ belongs to an Hilbert space. In this
case, $\displaystyle\mathbf{\Upsilon}$ the covariance operators as define in
(10) and (13) would be the identity operator. It can be verified that the
identity operator is not compact. Leading to a contradiction. This ends the
proof of Proposition 3.

The compactness assumption may also fail to hold if all instruments are
orthogonal with $\displaystyle L^{2}(\mu)$ or $\displaystyle l^{2}$ norm.
However, in the case of finite number of instrumental variables, the
covariance operator is compact as a finite-dimensional matrix.

The result in Proposition 3 suggests that regularization methods that rely on
the Assumption 5 for asymptotic approximation regularities should in practice
perform better if the IVs are correlated.

##  3 Conclusion

Many IVs are regularly used in empirical studies (see Hansen, Hausman, and
Newey (2008) for some examples). In finite sample estimation, the popular 2SLS
estimator has a bias that increases with the number of instruments. This note
shows that a crucial assumption of Bekker’s asymptotic approximation may have
strong limiting consequences. Indeed, in the Bekker’s asymptotic approach, the
number of effective IVs is should be finite. This restricts its application of
this asymptotic framework to the case of many weak instruments or sparse set
of instrumental variables. This may explain the poor behavior of the many IVs
asymptotic approaches noted by Harding, Hausman, and Palmer (2016).

The investigation of Carrasco’s asymptotic framework shows the importance of
the covariance operator compactness assumption. The covariance compactness
assumption implies the controversial crucial part of Bekker’s asymptotic
framework. Moreover, this assumption restricts the IV set, as it depends on a
specific topology imposed on the space spanned by instruments. The norm used
should be clearly specified, because in infinite dimensions, all norms are not
equivalent. The compactness assumption does not support orthonormal IV sets
unless the number of IVs is finite. Future research should consider the
extension of finite sample approximation, following Harding, Hausman, and
Palmer (2016), to regularized k-class estimators.

## References

  * (1)
  * Anatolyev and Gospodinov (2011) Anatolyev, S., and N. Gospodinov (2011): “Specification testing in models with many instruments,” _Econometric Theory_ , 27(2), 427–441.
  * Andrews and Stock (2007) Andrews, D. W., and J. H. Stock (2007): “Testing with many weak instruments,” _Journal of Econometrics_ , 138(1), 24–46.
  * Bekker (1994) Bekker, P. A. (1994): “Alternative approximations to the distributions of instrumental variable estimators,” _Econometrica_ , 62(3), 657–81.
  * Belloni, Chen, Chernozhukov, and Hansen (2012) Belloni, A., D. Chen, V. Chernozhukov, and C. Hansen (2012): “Sparse models and methods for optimal instruments with an application to eminent domain,” _Econometrica_ , 80(6), 2369–2429.
  * Bun and Windmeijer (2011) Bun, M. J., and F. Windmeijer (2011): “A comparison of bias approximations for the two-stage least squares (2SLS) estimator,” _Economics Letters_ , 113(1), 76–79.
  * Carrasco (2012) Carrasco, M. (2012): “A regularization approach to the many instruments problem,” _Journal of Econometrics_ , 170(2), 383 – 398.
  * Carrasco and Florens (2000) Carrasco, M., and J.-P. Florens (2000): “Generalization Of GMM to a continuum Of moment conditions,” _Econometric Theory_ , 16(06), 797–834.
  * Carrasco and Tchuente (2016) Carrasco, M., and G. Tchuente (2016): “Efficient estimation with many weak instruments using regularization techniques,” _Econometric Reviews_ , 35(8-10), 1609–1637.
  * Hansen, Hausman, and Newey (2008) Hansen, C., J. Hausman, and W. Newey (2008): “Estimation with many instrumental variables,” _Journal of Business & Economic Statistics_, 26(4), 398–422.
  * Harding, Hausman, and Palmer (2016) Harding, M., J. Hausman, and C. J. Palmer (2016): “Finite sample bias corrected IV estimation for weak and many instruments,” in _Essays in Honor of Aman Ullah_ , pp. 245–273. Emerald Group Publishing Limited.
  * Phillips and Moon (1999) Phillips, P. C., and H. R. Moon (1999): “Linear regression limit theory for nonstationary panel data,” _Econometrica_ , 67(5), 1057–1111.
  * Staiger and Stock (1997) Staiger, D., and J. H. Stock (1997): “Instrumental Variables Regression with Weak Instruments,” _Econometrica_ , 65(3), 557–586.
  * Stock and Yogo (2005) Stock, J., and M. Yogo (2005): “Asymptotic distributions of instrumental variables statistics with many instruments,” _Identification and inference for econometric models: Essays in honor of Thomas Rothenberg_ , pp. 109–120.
  * van Hasselt (2010) van Hasselt, M. (2010): “Many instruments asymptotic approximations under nonnormal error distributions,” _Econometric Theory_ , pp. 633–645.