# Mathematically Modeling the Lexicon Entropy of Emergent Language

Brendon Boldt, David Mortensen
Language Technologies Institute
Carnegie Mellon University
Pittsburgh, PA 15213, USA
<EMAIL_ADDRESS>

###### Abstract

We formulate a stochastic process, FiLex, as a mathematical model of lexicon
entropy in deep learning-based emergent language systems. Defining a model
mathematically allows it to generate clear predictions which can be directly
and decisively tested. We empirically verify across four different
environments that FiLex predicts the correct correlation between
hyperparameters (training steps, lexicon size, learning rate, rollout buffer
size, and Gumbel-Softmax temperature) and the emergent language’s entropy in
$20$ out of $20$ environment-hyperparameter combinations. Furthermore, our
experiments reveal that different environments show diverse relationships
between their hyperparameters and entropy which demonstrates the need for a
model which can make well-defined predictions at a precise level of
granularity.

##  1 Introduction

The methods of deep learning-based emergent language provide a uniquely
powerful way to study the nature of language and language change. In
addressing these topics, some papers hypothesize general principles describing
emergent language. For example, Resnick et al. (2020) hypothesize a
predictable relationship exists between compositionality and neural network
capacity, and Kharitonov et al. (2020) hypothesize a general entropy
minimization pressure in deep learning-based emergent language. In many cases,
these hypotheses are derived from intuitions and stated in natural language;
this can lead to ambiguous interpretation, inadequate experiments, and _ad
hoc_ explanations. To this end, we study a general principle of emergent
language by proposing a mathematical model which generates a testable
hypothesis which can be directly evaluated through the empirical studies, akin
to what we find prototypically in natural science.

We formulate a stochastic process, FiLex, as a mathematical model of lexicon
entropy in deep learning-based emergent language systems111 _Emergent language
system_ or ELS refers to the combination of agents (neural networks), the
environment, and the training procedure used as part of an emergent language
experiment. (ELS). We empirically verify across four different environments
that FiLex predicts the correct correlation between hyperparameters (training
steps, lexicon size, learning rate, rollout buffer size, and Gumbel-Softmax
temperature) and the emergent language’s entropy in $20$ out of $20$
environment-hyperparameter combinations.

There are three primary reasons for using an explicitly defined model for
studying a topic like emergent language: clarity, testability, and
extensibility. A mathematical model yields a _clear_ , unambiguous
interpretation since its components have precise meanings; this is especially
important when conveying such concepts in writing. It is easier to _test_ a
model than a hypothesis articulated in natural language because the model
yields clear predictions which can be shown to be accurate or inaccurate; as a
result, models can also be directly compared to one another. Our experiments
reveal that different environments show diverse relationships between their
hyperparameters and entropy which demonstrates the need for such clarity in
making well-defined predictions at a precise level of granularity. Finally,
mathematical models hypothesize a _mechanism_ for an observed effect and not
simply the effect itself (with a possibly _ad hoc_ explanation). This is what
facilitates their _extensibility_ since a multitude of hypotheses can be
derived from these mechanisms; furthermore, this “mechanical” nature allows
future work to build directly on top of the model.

As mathematical models are seldom used to their full potential in studying
emergent language, this paper is meant to serve as a reference and starting
point for entire methodology of developing and testing such models. We
articulate our contributions as follows:

  * •

Defining a mathematical model of lexicon entropy in emergent language systems
which we demonstrate to be accurate in predicting hyperparameter-entropy
correlations.

  * •

Presenting a case study of defining and empirically evaluating a mathematical
model in emergent language.

  * •

Provide a direct, intuitive comparison of the effects of hyperparameters on
lexicon entropy across different environments.

We briefly discuss related work in Section 2. In Section 3, we introduce the
mathematical model, FiLex, as well as the ELSs. Empirical evaluation is
presented in Section 4 and discussed in Section 5, concluding with Section 6.
Code is available at https://example.com/reponame (in supplemental material
while under review).

##  2 Related work

For a survey of deep learning-based emergent language work, please see
Lazaridou and Baroni (2020). Contemporary deep learning-based emergent
language research often aims at establishing and refining general principles
about emergent language. In large part, these principles can be expressed as
relationships between certain characteristics of the environment or agents
(e.g., model capacity (Resnick et al., 2020), population size (Rita et al.,
2022)) and properties of the emergent language (e.g., compositionality
(Resnick et al., 2020; Rodríguez Luna et al., 2020), entropy (Kharitonov et
al., 2020; Chaabouni et al., 2021; Rita et al., 2022), and generalizability
(Chaabouni et al., 2020; Guo et al., 2021; Słowik et al., 2020)). Some of
these works (Kharitonov et al., 2020; Khomtchouk and Sudhakaran, 2018; Resnick
et al., 2020) make use of mathematical models to describe parts of the
hypotheses and/or experiments, but these fall short of establishing a clear
model which generates a testable hypothesis which is then evaluated through
the empirical studies.

Pre-deep learning emergent language research frequently relied on mathematical
models (Skyrms, 2010; Kirby et al., 2015; Brighton et al., 2005), but such
models played a different role. Whereas these models were meant to account for
some property of language observed in human language, the model presented in
this paper is accounting for emergent language directly (and human language
only indirectly). Thus, this paper presents a (mathematical) model of a
(computational) model which, in the future, will be used to more directly
study human language.

##  3 Methods

###  3.1 Model

FiLex (“fixed lexicon stochastic process”) is a mathematical model developed
from the Chinese restaurant process (Blei, 2007; Aldous, 1985), a stochastic
process where each element in the sequence is a stochastic distribution over
the positive integers (i.e., a distribution over distributions). The analogy
for the Chinese restaurant process is a restaurant with tables indexed by the
natural numbers; as each customer walks in, they sit at a random table with a
probability proportional to the number of people already at that table. The
key property here is that the process is _self-reinforcing_ ; tables with many
people are likely to get even more. By analogy to language, the more a word is
used the more likely it is to continue to be used. For example, speakers may
develop a cognitive preference for it, or it gets passed along to subsequent
generations as a higher rate (Francis et al., 2021).

#### Formulation

FiLex is defined as a sequence of stochastic vectors indexed by
$N\in\mathbb{N}^{+}$ given by:

$\displaystyle\textsc{FiLex}{}(\alpha,\beta,S,N)$
$\displaystyle=\frac{\boldsymbol{w}^{(N)}}{\|\boldsymbol{w}^{(N)}\|_{1}}$ (1)
$\displaystyle\boldsymbol{w}^{(n+1)}$
$\displaystyle=\boldsymbol{w}^{(n)}+\alpha\frac{\boldsymbol{x}^{(n)}}{\beta}$
(2) $\displaystyle\boldsymbol{x}^{(n)}$
$\displaystyle\sim\text{Multi}\\!\left(\beta,\frac{\boldsymbol{w}^{(n)}}{\|\boldsymbol{w}^{(n)}\|_{1}}\right)$
(3) $\displaystyle\boldsymbol{w}^{(1)}$
$\displaystyle=\frac{1}{S}\cdot(1,1,\ldots,1)\in\mathbb{R}^{S}$ (4)

where $\boldsymbol{w}^{(n)}$ is a vector of weights,
$\alpha\in\mathbb{R}_{>0}$ controls the weight update magnitude,
$\beta\in\mathbb{N}^{+}$ controls the variance of the updates,
$S\in\mathbb{N}^{+}$ is the size of the weight vector (i.e., lexicon), and
$\text{Multi}(k,\boldsymbol{p})$ is a $k$-trial multinomial distribution with
probabilities $\boldsymbol{p}\in\mathbb{R}^{S}$. The pseudocode describing
FiLex is given in Algorithm 1. Conceptually, the process starts with an
$S$-element array of weights initialized to $1/S$. At each iteration we draw
from a $\beta$-trial multinomial distribution parameterized by the normalized
weights.222The $\beta$-trial multinomial sample is written as $\beta$ i.i.d.
samples from a categorical distribution to draw parallels to PPO in Algorithm
2. This multinomial sample is multiplied by $\alpha/\beta$ and added to the
weights so that the update magnitude is $\alpha$. This proceeds $N$ times.
Since the sequence elements are the _normalized_ weights, the elements are
themselves probability distributions; thus, FiLex is technically a sequence of
distributions over distributions.

⬇

1alpha: float > 0

2beta: int > 0

3N: int > 0

4S: int > 0

5

6weights = array(size=S)

7weights.fill(1 / S)

8for _ in range(N):

9 W += sample_multinomial(W / sum(W), beta) / beta

10 w_copy = weights.copy()

11 for _ in range(beta): # equivalent to normalized multinomial

12 i = sample_categorical(w_copy / sum(w_copy))

13 weights[i] += alpha / beta

14return weights / sum(weights)

Algorithm 1 FiLex pseudocode

The two key differences between FiLex and the Chinese restaurant process are
the hyperparameters $S$ and $\beta$.333Note that $\alpha$ in FiLex is actually
equivalent to the _inverse_ of $\alpha$ in the Chinese restaurant process.
FiLex has a fixed number of parameters so as to match the fact that the agents
in the ELS have a fixed-size bottleneck layer, that is, a fixed lexicon.
Secondly, $\beta$ is introduced to modulate the smoothness of parameter
updates. It is closely connected to the fact that certain RL algorithms like
PPO accumulate a buffer of data points from the environment with the same
parameters before performing gradient descent.

###  3.2 Environments

To evaluate FiLex, we use four different reinforcement learning environments
in our experiments. These are inhabited by two deep learning-based agents: (1)
a sender agent which receives an observation and produces a message and (2) a
receiver agent which receives a message (and possibly additional observation)
and takes an action. The agent architecture and optimization are detailed
Section 3.3.

#### NoDyn

The “no dynamics” environment is a proof-of-concept environment which is not
intended to be realistic but rather to match as closely as possible the
simplifying assumptions which FiLex makes while keeping the same neural
architecture in the environments below. As the name suggests, the primary
simplification in this environment is that there are trivial dynamics, that
is, every episode immediately ends with reward of $1$ no matter what the
sender or receiver do. The sender input and receiver output are identical to
those of Nav, defined below. Just as FiLex assumes that every instance of word
use is reinforced, this process reinforces every message which the sender
produces.

#### Recon

The reconstruction game (Chaabouni et al., 2020), in the general case, mimics
a discrete autoencoder: the input value is translated into a discrete message
by the sender, and the receiver tries to output the original input based on
the message. For a given episode, the sender observes $x\sim\mathcal{U}(-1,1)$
and produces a message; the receiver’s action is a real number $\hat{x}$,
yielding a reward $(x-\hat{x})^{2}$.

#### Sig

The signaling game environment comes from Lewis (1970) and has been frequently
used in the literature (Lazaridou et al., 2017; Bouchacourt and Baroni, 2018).
In this setup, the data is partitioned into a fixed number of discrete
classes. The sender observes a datum from one of the classes and produces a
message; the receivers observes this message, the sender’s datum, and data
points from other classes (i.e., “distractors”). The reward for the
environment is $1$ if the receiver correctly identifies the sender’s datum
among the distractors and $0$ otherwise.

To eliminate the potential confounding factors from using natural inputs
(e.g., image embeddings (Lazaridou et al., 2017)), we use a synthetic dataset.
For an $n$-dimensional signaling game, we have $2^{n}$ classes. Each class is
represented by an isotropic multivariate normal distribution with mean
$(\mu_{1},\mu_{2},\dots,\mu_{n})$ where $\mu_{i}\in\\{-3,3\\}$. Observations
of a given are samples of its corresponding distribution. For example, in the
$2$-dimensional game, the $4$ classes would be represented by the
distributions: $\mathcal{N}((-3,-3),I_{2})$, $\mathcal{N}((3,-3),I_{2})$,
$\mathcal{N}((-3,3),I_{2})$, and $\mathcal{N}((3,3),I_{2})$ (we use a
$5$-dimensional signaling game for our experiments with $32$ classes). The
motivation for this setup is minimal need for feature extraction while still
using real-valued, stochastic inputs.

#### Nav

For a multi-step environment, we use a $2$-dimensional, obstacle-free
navigation task. The sender agent observes the $(x,y)$ position of a receiver
and produces a message; the receiver moves by producing an $(x,y)$ vector. For
a given episode, the receiver is initialized uniformly at random within a
circle and must navigate towards a smaller circular goal region at the center.
The agents are rewarded for both reaching the goal and for moving towards the
center. An illustration is provided in Appendix A. The receiver’s location and
action are continuous variables.

###  3.3 Agents

#### Architecture

Our architecture comprises two agents, conceptually speaking, but in practice,
they are a single neural network. The sender and receiver are randomly
initialized at the start of training, are trained together, and are tested
together. The sender itself is a $2$-layer perceptron with tanh activations.
The sender’s input is environment-dependent. The output of the second layer is
passed to a Gumbel-Softmax bottleneck layer (Maddison et al., 2017; Jang et
al., 2017) which enables learning a discrete, one-hot representation.444Using
a Gumbel-Softmax bottleneck layer allows for end-to-end backpropagation,
making optimization faster and more consistent than using a backpropagation-
free method like REINFORCE (Kharitonov et al., 2020; Williams, 1992).
Nevertheless, future work may want to use REINFORCE for its more realistic
assumptions about communication. The activations of this layer can be thought
of as the words forming the lexicon of the emergent language. Messages consist
only of a single one-hot vector (word) passed from sender to receiver. At
evaluation time, the bottleneck layer functions deterministically as an argmax
layer, emitting one-hot vectors. The receiver is a $1$-layer perceptron which
takes the output of the Gumbel-Softmax layer as input. The receiver’s output
is environment-dependent. An illustration and precise specification are
provided in Appendices A and B.

#### Optimization

Although only our Nav environment involves multi-step episodes, using a full
reinforcement learning algorithm across all environments benefits
comparability and extensibility in future work. Specifically, we use proximal
policy optimization (PPO) (Schulman et al., 2017) paired with Adam (Kingma and
Ba, 2015) to optimize the neural networks. PPO is widely used RL algorithm
which selected primarily for its stability (e.g., training almost always
converges, minimal hyperparameter tuning); attempts to train with “vanilla”
advantage actor critic did not consistently converge. We use the PPO
implementation of Stable Baselines 3 (MIT license) built on PyTorch (BSD
license) (Raffin et al., 2019; Paszke et al., 2019).

⬇

1n_updates: int >= 0

2buffer_size: int > 0

3

4for _ in range(n_updates): # outer loop

5 rollout_buffer = []

6 for _ in range(buffer_size): # inner loop

7 episode = run_episode(model, environment)

8 rollout_buffer.append(episode)

9 update_parameters(model, rollout_buffer)

Algorithm 2 PPO pseudocode

One relevant characteristic of PPO and similar algorithms is that in their
training they contain an inner and outer loop analogous to FiLex (Algorithm
1); this is illustrated in Algorithm 2. The (main) outer loop consists of two
steps: the inner loop which populates a rollout buffer with “experience” from
the environment and the updating of parameters based on that buffer. What is
important to note is that the buffer is populated with data from the same
model parameters, and it is not until after this that model parameters change.

###  3.4 Hypothesis

Here we state the hypothesis used to evaluate FiLex. The sign of
hyperparameter-entropy correlation observed in FiLex will be the same as what
we observe for a corresponding hyperparameter in the ELSs. We can state this
more formally as: for each pair of corresponding hyperparameters
$(h,h^{\prime})$ in FiLex and an ELS respectively,

$\displaystyle\text{sgn}(\text{corr}(D))$
$\displaystyle=\text{sgn}(\text{corr}(D^{\prime}))$ (5) $\displaystyle D$
$\displaystyle=\left\\{(x,H(\boldsymbol{y}))\mid x\in
X_{h},\,\boldsymbol{y}\sim\textsc{FiLex}{}_{h=x}\right\\}$ (6) $\displaystyle
D^{\prime}$ $\displaystyle=\left\\{(x,H(\boldsymbol{y}))\mid x\in
X_{h^{\prime}},\,\boldsymbol{y}\sim\text{ELS}_{h^{\prime}=x}\right\\}$ (7)
$\displaystyle H(\boldsymbol{y})$
$\displaystyle=-\sum_{i=1}^{S}y_{i}\log_{2}y_{i}$ (8)

where $\text{corr}(\cdot)$ is the Kendall rank correlation coefficient
($\tau$) (Kendall, 1938), $\textsc{FiLex}{}_{h=x}$ is the distribution over
frequency vectors yielded by the model for hyperparameter $h$ set to $x$
(assume likewise for $\text{ELS}_{h^{\prime}=x}$), $H$ is Shannon entropy, and
$X_{h}$ is the set of experimental values for hyperparameter $h$. A “sample”
from an ELS consists of training the agents in the environment, and estimating
word frequencies by collecting the sender’s messages over a random sample of
inputs. Accordingly, our null hypothesis is that FiLex does not meaningfully
correspond to the ELSs, and thus the signs of correlation would be expected to
match with a probability $0.5$.

We intentionally formulate our hypothesis at this level of granularity:
equality of direction (sign) of correlation rather stronger claims such as raw
correlation: $|\text{corr}(D)-\text{corr}(D^{\prime})|<\epsilon$ or mean
squared error: $1/|X|\cdot\sum_{x\in X}(D(x)-D^{\prime}(x))^{2}$. We select
this level of direction of correlation for a few reasons. The level of
simplicity of FiLex compared to the ELSs means that the unaccounted for
factors would make supporting stronger hypotheses too difficult; furthermore,
even if the hypothesis were defended, it would be less widely applicable for
the same reasons. Additionally, the current literature tends to speak of the
general principles of emergent language at the level of “relationships” and
“effects” rather than exact numeric approximations (Kharitonov et al., 2020;
Resnick et al., 2020).

Table 1: Corresponding hyperparameters in the ELSs and FiLex. ELS | FiLex
---|---
Time steps | $N$
Lexicon size | $S$
Learning rate | $\alpha$
Buffer size | $\beta$
Temperature | $\beta$

#### Corresponding Hyperparameters

A key component of the hypothesis is the correspondence of hyperparameters of
the ELSs with those of FiLex. These correspondences are the foundation for
applying reasoning about FiLex to the ELSs; accordingly, they also determine
how the model will be empirically tested. We present five pairs of
corresponding environment-agnostic hyperparameters in Table 1. Although
environment-specific hyperparameters can easily correspond with those of FiLex
we chose the agnostic for ease of experimentation and comparison.

To identify these correspondences, it is important to understand the intuitive
similarities between the ELSs and FiLex. Firstly, the weights of FiLex
correspond the learned likelihood with which a given bottleneck unit is used
in the ELS; in turn, both of these correspond to the frequency with which a
word is used in a language. Each iteration of FiLex’s outer loop is analogous
to a whole cycle in the ELS of simulating episodes in the environment,
receiving the rewards, and performing gradient descent with respect to the
rewards (compare Algorithms 1 and 2).

Based on this analogy, we can explain the corresponding hyperparameters as
follows. $N$ corresponds the number of parameter updates taken throughout the
course of training the ELS (i.e., the outer loop of PPO). $S$ corresponds the
size of the bottleneck layer in the ELS. $\alpha$ corresponds to the learning
rate (i.e., magnitude of parameter updates) in the ELS. The ELS has two
analogs of $\beta$. First, $\beta$ corresponds to the rollout buffer size of
PPO because both control the number of iterations of the inner loop of
training where episodes are collected before updating the weights. Second,
$\beta$, more generally, control how smooth the updates to FiLex’s weights are
which makes it analogous to the temperature of the Gumbel-Softmax distribution
in the ELS since a higher temperature results in smoother updates to the
bottleneck’s parameters.

##  4 Experiments

Our experiments consist of comparing the correlation between the
hyperparameters of FiLex and the ELSs and the Shannon entropy of lexicon at
the end of training. The entropy for the ELSs is calculated based on the
bottleneck unit (word) frequencies gathered by sampling from the sender’s
input distribution. To gather data for FiLex, we run a Rust implementation of
a sampling algorithm. Each experiment consists of a logarithmic sweep of a
hyperparameter plotted against the entropy yielded by those hyperparameters
(see Appendix B for details).

Time Steps

Lexicon Size

Learning Rate

Buffer Size

Temperature

FiLex

NoDyn

Recon

Sig

Nav

Figure 1: Plots of hyperparameters ($x$-axis, log scale) vs. entropy ($y$-axis) . Each row corresponds to a particular environment. Each column corresponds to a particular hyperparameter. All $y$-axes are on the same scale with the dashed lines representing min/max entropy. The points are individual runs and the lines are a Gaussian convolution of the points.  Table 2:  Kendall’s $\tau$’s for various configurations. All values have a significance of $p\leq 0.01$.  Environment | Time Steps | Lexicon Size | Learning Rate | Buffer Size | Temperature
---|---|---|---|---|---
FiLex | $-0.53$ | $+0.67$ | $-0.87$ | $+0.93$ | $+0.93$
NoDyn | $-0.81$ | $+0.12$ | $-0.74$ | $+0.07$ | $+0.58$
Recon | $-0.17$ | $+0.93$ | $-0.35$ | $+0.84$ | $+0.68$
Sig | $-0.49$ | $+0.15$ | $-0.16$ | $+0.30$ | $+0.49$
Nav | $-0.81$ | $+0.36$ | $-0.84$ | $+0.20$ | $+0.68$

Each point in the resulting scatter plots corresponds to an independent run of
the model or ELS with the hyperparameter on the $x$-axis and entropy on the
$y$-axis. The plots also include a Gaussian convolution of the data points
(the solid line) to better illustrate the general trend of the data. The plots
are presented in Figure 1 with the rank correlation coefficients in Table 2.

##  5 Discussion

###  5.1 Model evaluation

Looking at the signs of correlations shows that FiLex makes the correct
prediction $20$ out of $20$ times. Given a simple one-sided binomial test, the
empirical data rejects the null hypothesis at $p<0.001$. Although this number
drops to $15$ out of $20$ if we require $|\tau|\geq 0.2$, the binomial test
rejects the null hypothesis with $p=0.02$ for this stronger hypothesis.

Though the directions of correlations predicted by FiLex are correct, looking
at the plots show that ELSs do not always demonstrate the monotonicity
predicted by the model. This is especially evident in _Time Steps_ for Recon:
moving left-to-right, the plot follows a similar path to the other environment
and FiLex at first but then diverges halfway through with increasing entropy.
A possible explanation of this is that Recon allows learning new, useful words
more easily than Sig or Nav, meaning that additional training can lead to
further improvement. The conclusion we draw from these plots is that FiLex
correctly predicts a sort of baseline correlation between the hyperparameters
and entropy. Other works, Kharitonov et al. (2020); Chaabouni et al. (2021)
for example, find similar correlations between entropy and bottleneck
temperature. Nevertheless, this correlation can be overridden by the specifics
of the environment.

###  5.2 Environment variability

When looking beyond just the direction of correlation at the slopes and shapes
of the curves, the four ELSs all present unique set of relationships between
entropy their hyperparameters. This implies that none of these environments
are reducible to each other, that is, we cannot make observations about one
environment and automatically assume they apply to other environments.
Certainly this makes an researcher’s task harder as learning general
principles would not be possible from a single environment. Furthermore, there
is a sensitivity to hyperparameters _within_ a given environment, which would
imply that discovering general principles within single environment could not
be done with just a single set of hyperparameters.

Although this diversity in behavior makes modeling it more difficult, it also
shows the importance of precision we get from a mathematical model. For
example, say Recon has not been empirically tested and we wanted to predict
the lexicon size-entropy relationship in Recon. It is the case that we could
simply observe the positive correlations in the other environments and predict
the same Recon, but we could easily over-extrapolate and predict a relatively
shallow slope when Recon’s slope is relatively steep. What this paper’s model,
hypothesis, and evaluation offer in this situation is not a more detailed
prediction but a “prepackaged” prediction which is precisely stated and
supported by data.

###  5.3 Applications to future work

There are two primary ways in which FiLex can be applied in future research.
First, the model can be applied to and tested against further phenomena in
emergent language (i.e., it is _extensible_). The fact that it is formulated
mathematically means that it does not just predict correlations but
_mechanisms_ which account for the correlations. For example, FiLex’s $\beta$
hyperparameter was designed to account for _Buffer Size_ and the _Temperature_
experiment was conducted after the fact. The fact that FiLex describes both
_Buffer Size_ and _Temperature_ with the same hyperparameter suggests that
similar mechanisms account for their positive correlations with entropy. This
statement about similar mechanisms, on the other hand, is not present set of
one-off hypotheses about hyperparameter-entropy correlations derived from
intuition. Second, FiLex and accompanying experiments provide an easy way for
future research to discover confounding factors in their experiments. For
example, an experiment might show that entropy decreases as rewards are scaled
up, yet FiLex would suggest that this might be equivalent to simply increasing
the learning rate rather than being its own unique cause of the effect on
entropy.

###  5.4 Methodological difficulties

The greatest challenge in the methodology of this work is not the formulation
of the model but rather evaluating the quality of the model. In part, this is
on account of a lack of established baseline model—comparative analysis
(“which is better?”) is significantly easier than absolute analysis (“how good
is this?”) yet requires an adequate baseline to compare against. But more
significantly, the granularity of experimentation is a design decision with no
obvious answer.

For example, merely comparing the signs of rank correlations is very coarse-
grained as it makes minimal assumptions about the data (e.g., linearity,
absence of outliers) and captures very little information about the data.
Naturally, it is easier to apply such an analysis, and as mentioned before,
researcher typically phrase hypotheses in terms of such correlations, but it
can only offer minimal support for applicability of the model to the actual
system. On the other hand, evaluating the model’s ability to predict exact
behavior of the system (e.g., measuring mean squared error of the model’s
predictions) can establish a more precise link between model and system but
might miss more general but important similarities. For example, _Lexicon
Size_ for FiLex and Nav might show similar trends, but be different by a
constant, yielding a high mean squared error.

A subtle but significant methodological difficulty is the selection of
hyperparameters. In Recon’s _Time Steps_ plot, it is easy to see that changing
the range of hyperparameters could easily yield either a positive or a
negative correlation when in reality there are both. To a certain extent, this
can be resolved be choosing a “reasonable” range of hyperparameters based on
values are typically, but this is of little help to selection of FiLex’s
hyperparameters as there is no “typical usage.” For example, FiLex for
$\beta=1$ and $\beta=100$ yield significantly different distributions, but
there is no obvious _a priori_ reason to say that one value of $\beta$ should
be preferred over the other for comparing to the ELSs. Although additional
hyperparameters increase the range of phenomena which the model can account
for, the additional degrees of freedom can weaken the model’s predictions by
introducing confounding variables (cf. overparameterization).

One of the primary contributions of this work is to serve as a case study and
example of working with explicitly defined models in studying deep learning-
based emergent language. Thus, this paper is starting point for future work to
improve upon. One of the most important improvements would be finding a more
rigorous way to select “reasonable” experimental hyperparameters.
Additionally, it would be better to develop the hypothesis and experimental in
full before performing any evaluation; the process was somewhat iterative in
this paper.

##  6 Conclusion

We have presented FiLex as a mathematical model of lexicon entropy in deep
learning-based emergent language systems and demonstrated that, at the level
of correlations, it accurately predicts the behavior of our emergent language
environments. Opting for a mathematical model possesses the benefits of having
a clear interpretation, making testable predictions, and being reused for new
predictions in future studies. Although the model’s hypothesis was testable,
the process is not free from non-trivial design decisions which affect the
quality of evaluation. Nevertheless, this paper serves as starting point and
example of how more rigorous models can be applied to the study of emergent
language.

## Acknowledgments and Disclosure of Funding

This material is based on research sponsored in part by the Air Force Research
Laboratory under agreement number FA8750-19-2-0200. The U.S. Government is
authorized to reproduce and distribute reprints for Governmental purposes
notwithstanding any copyright notation thereon. The views and conclusions
contained herein are those of the authors and should not be interpreted as
necessarily representing the official policies or endorsements, either
expressed or implied, of the Air Force Research Laboratory or the U.S.
Government.

## References

  * Aldous (1985) David J. Aldous.  Exchangeability and related topics.  In P. L. Hennequin, editor, _École d’Été de Probabilités de Saint-Flour XIII — 1983_ , pages 1–198, Berlin, Heidelberg, 1985. Springer Berlin Heidelberg.  ISBN 978-3-540-39316-0.
  * Blei (2007) David Blei.  The chinese restaurant process, 2007.  URL https://www.cs.princeton.edu/courses/archive/fall07/cos597C/scribe/20070921.pdf.
  * Bouchacourt and Baroni (2018) Diane Bouchacourt and Marco Baroni.  How agents see things: On visual representations in an emergent language game.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , page 981–985, Brussels, Belgium, Oct 2018. Association for Computational Linguistics.  doi: 10.18653/v1/D18-1119.  URL https://aclanthology.org/D18-1119.
  * Brighton et al. (2005) Henry Brighton, Kenny Smith, and Simon Kirby.  Language as an evolutionary system.  _Physics of Life Reviews_ , 2:177–226, 2005.
  * Chaabouni et al. (2020) Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni.  Compositionality and generalization in emergent languages.  _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020.  doi: 10.18653/v1/2020.acl-main.407.  URL http://dx.doi.org/10.18653/v1/2020.acl-main.407.
  * Chaabouni et al. (2021) Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni.  Communicating artificial neural networks develop efficient color-naming systems.  _Proceedings of the National Academy of Sciences_ , 118(12), Mar 2021.  ISSN 0027-8424, 1091-6490.  doi: 10.1073/pnas.2016569118.  URL https://www.pnas.org/content/118/12/e2016569118.
  * Francis et al. (2021) David Francis, Ella Rabinovich, Farhan Samir, David Mortensen, and Suzanne Stevenson.  Quantifying Cognitive Factors in Lexical Decline.  _Transactions of the Association for Computational Linguistics_ , 9:1529–1545, 12 2021.  ISSN 2307-387X.  doi: 10.1162/tacl_a_00441.  URL https://doi.org/10.1162/tacl_a_00441.
  * Guo et al. (2021) Shangmin Guo, Yi Ren, Simon Kirby, Kenny Smith, Kory Wallace Mathewson, and Stefano V. Albrecht.  Expressivity of Emergent Languages is a Trade-off between Contextual Complexity and Unpredictability.  September 2021.  URL https://openreview.net/forum?id=WxuE_JWxjkW.
  * Jang et al. (2017) Eric Jang, Shixian Gu, and Ben Poole.  Categorical reparameterization with gumbel-softmax.  In _Proceedings of the 2017 International Conference on Learning Representations (ICLR)_ , 2017.  URL https://openreview.net/forum?id=rkE3y85ee.
  * Kendall (1938) M. G. Kendall.  A new measure of rank correlation.  _Biometrika_ , 30(1-2):81–93, 06 1938.  ISSN 0006-3444.  doi: 10.1093/biomet/30.1-2.81.  URL https://doi.org/10.1093/biomet/30.1-2.81.
  * Kharitonov et al. (2020) Eugene Kharitonov, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni.  Entropy minimization in emergent languages.  In Hal Daumé III and Aarti Singh, editors, _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 5220–5230. PMLR, 13–18 Jul 2020.  URL http://proceedings.mlr.press/v119/kharitonov20a.html.
  * Khomtchouk and Sudhakaran (2018) Bohdan Khomtchouk and Shyam Sudhakaran.  Modeling natural language emergence with integral transform theory and reinforcement learning.  _arXiv:1812.01431 [cs]_ , Nov 2018.  URL http://arxiv.org/abs/1812.01431.  arXiv: 1812.01431.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _ICLR (Poster)_ , 2015.  URL http://arxiv.org/abs/1412.6980.
  * Kirby et al. (2015) Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith.  Compression and communication in the cultural evolution of linguistic structure.  _Cognition_ , 141:87–102, 2015.  ISSN 0010-0277.  doi: https://doi.org/10.1016/j.cognition.2015.03.016.
  * Lazaridou and Baroni (2020) Angeliki Lazaridou and Marco Baroni.  Emergent multi-agent communication in the deep learning era.  _arXiv:2006.02419 [cs]_ , Jul 2020.  URL http://arxiv.org/abs/2006.02419.  arXiv: 2006.02419.
  * Lazaridou et al. (2017) Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni.  Multi-agent cooperation and the emergence of (natural) language, 2017\.  URL https://openreview.net/forum?id=Hk8N3Sclg.
  * Lewis (1970) David Lewis.  Convention: A philosophical study.  1970\.
  * Maddison et al. (2017) Chris J. Maddison, Andriy Mnih, and Yee Whye Teh.  The concrete distribution: A continuous relaxation of discrete random variables.  In _Proceedings of the 2017 International Conference on Learning Representations (ICLR)_ , 2017.  URL https://openreview.net/forum?id=S1jE5L5gl.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, _Advances in Neural Information Processing Systems 32_ , pages 8024–8035. Curran Associates, Inc., 2019.  URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
  * Raffin et al. (2019) Antonin Raffin, Ashley Hill, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, and Noah Dormann.  Stable baselines3.  https://github.com/DLR-RM/stable-baselines3, 2019.
  * Resnick et al. (2020) Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M. Dai, and Kyunghyun Cho.  Capacity, Bandwidth, and Compositionality in Emergent Language Learning.  _International Conference on Autonomous Agents and Multi-Agent Systems_ , April 2020.  URL http://arxiv.org/abs/1910.11424.
  * Rita et al. (2022) Mathieu Rita, Florian Strub, Jean-Bastien Grill, Olivier Pietquin, and Emmanuel Dupoux.  On the role of population heterogeneity in emergent communication.  In _International Conference on Learning Representations_ , 2022.  URL https://openreview.net/forum?id=5Qkd7-bZfI.
  * Rodríguez Luna et al. (2020) Diana Rodríguez Luna, Edoardo Maria Ponti, Dieuwke Hupkes, and Elia Bruni.  Internal and external pressures on language emergence: least effort, object constancy and frequency.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , page 4428–4437, Online, 2020. Association for Computational Linguistics.  doi: 10.18653/v1/2020.findings-emnlp.397.  URL https://www.aclweb.org/anthology/2020.findings-emnlp.397.
  * Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ , 2017.
  * Skyrms (2010) Brian Skyrms.  _Signals: Evolution, learning, and information_.  OUP Oxford, 2010.
  * Słowik et al. (2020) Agnieszka Słowik, Abhinav Gupta, William L. Hamilton, M. Jamnik, S. Holden, and C. Pal.  Exploring Structural Inductive Biases in Emergent Communication.  _undefined_ , 2020.  URL https://www.semanticscholar.org/paper/Exploring-Structural-Inductive-Biases-in-Emergent-S%C5%82owik-Gupta/29d1adb458d5b5a0fc837d37af01a6673efd531c.
  * Williams (1992) Ronald J Williams.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  _Machine learning_ , 8(3):229–256, 1992.

##  Appendix A Emergent language system illustration

(a) The receiver (pictured) is rewarded for moving towards the goal region at
the center in the Nav environment.

(b) The agent architecture for Nav.

Figure 2:

##  Appendix B Experiment parameters

Each experiment uses a logarithmic sweep across hyperparameters; the sweep is
defined by Equation 9, where $x$ and $y$ are the inclusive upper and lower
bounds respectively and $n$ is the number steps to divide the interval into.
The floor function is applied if the elements must be integers.

$\text{LS}(x,y,n)=\left\\{x\cdot\left(\frac{y}{x}\right)^{\frac{i}{n-1}}\,\middle|\,i\in\\{0,1,\dots,n-1\\}\right\\}$
(9)

Hyperparameter | Default | Low | High | Steps
---|---|---|---|---
$N$ | $10^{3}$ | $10^{0}$ | $10^{3}$ | $1000$
$S$ | $2^{6}$ | $2^{3}$ | $2^{8}$ | $1000$
$\alpha$ | $1$ | $10^{-3}$ | $10^{3}$ | $1000$
$\beta$ | $8$ | $10^{0}$ | $10^{3}$ | $1000$
Table 3: Hyperparameters for the empirical evaluation of FiLex. “Low” and “High” refer to the logarithmic sweep used for that experiment; default values used for all other experiments.  Hyperparameter | Default | Low | High | Steps
---|---|---|---|---
Time steps | $2\cdot 10^{5}$ | $10^{2}$ | $10^{6}$ | $600$
Bottleneck size | $2^{6}$ | $2^{3}$ | $2^{8}$ | $600$
Learning rate | $3\cdot 10^{-3}$ | $10^{-4}$ | $10^{-1}$ | $600$
Buffer size | $2^{8}$ | $2^{3}$ | $2^{10}$ | $600$
Temperature | $1.5$ | $10^{-1}$ | $10^{1}$ | $600$
Table 4: Hyperparameters for the empirical evaluation of FiLex. “Low” and
“High” refer to the logarithmic sweep used for that experiment; default values
used for all other experiments. Please see code for further details and
default values.


# E2E Segmentation in a Two-Pass Cascaded Encoder ASR Model

###### Abstract

We explore unifying a neural segmenter with two-pass cascaded encoder ASR into
a single model. A key challenge is allowing the segmenter (which runs in real-
time, synchronously with the decoder) to finalize the 2nd pass (which runs 900
ms behind real-time) without introducing user-perceived latency or deletion
errors during inference. We propose a design where the neural segmenter is
integrated with the causal 1st pass decoder to emit a end-of-segment (EOS)
signal in real-time. The EOS signal is then used to finalize the non-causal
2nd pass. We experiment with different ways to finalize the 2nd pass, and find
that a novel dummy frame injection strategy allows for simultaneous high
quality 2nd pass results and low finalization latency. On a real-world long-
form captioning task (YouTube), we achieve 2.4% relative WER and 140 ms EOS
latency gains over a baseline VAD-based segmenter with the same cascaded
encoder.

Index Terms—  ASR, segmentation, decoding algorithms

##  1 Introduction

Two-pass cascaded encoder models [1] are used in streaming ASR systems to
provide real-time, causal results to the user from the 1st pass, while also
providing superior, high-quality, non-causal results from a 2nd pass that lags
the 1st pass by, e.g., 900 ms in order to access 900 ms of right-context
frames. Running in parallel allows the 2nd pass to, in particular, operate on
long-form utterances (i.e., those longer than a few minutes) prevalent in
captioning and dictation tasks.

Another common practice when decoding long-form utterances is to segment the
utterance into shorter lengths, and to deliver the final 2nd pass result at
each end-of-segment (EOS). Many downstream tasks (e.g. machine translation)
can incrementally consume partial recognition results, so quick and frequent
segmentation is beneficial for overall user experience [2]. In streaming
settings, the segmenter runs synchronously with beam search and decides where
to end each segment in real time. In particular, it emits a EOS token, which
resets the beam search, discarding all hypotheses except for the top
hypothesis, whose state is passed to the next segment. This clears room for
new hypotheses to enter the beam, introducing more diversity. More critically,
the EOS token finalizes the top hypothesis by discarding all other possible
contenders, guaranteeing that the top hypothesis will no longer change, e.g.,
by later evidence causing a non-top hypothesis to be promoted to the top.
Finalization is prerequisite for delivering the 2nd pass result to downstream
tasks because downstream tasks typically require the hypothesis to be fixed.
On the other hand, a poorly placed EOS can lead to the wrong hypothesis being
finalized, hurting ASR quality. Consequently, the segmenter’s job of deciding
when to emit EOS is foundationally important to both WER and user-perceived
latency in long-form tasks.

Conventionally, EOS is determined by the VAD, a classifier predicting whether
each frame is speech or silence [3, 4]. A hand-coded state machine keeps track
of the predictions, does some smoothing, and emits an EOS signal when, e.g.
200 ms in our case, of consecutive silence frames are detected. There are two
disadvantages to the VAD segmenter: (1) it requires a state machine with hand-
crafted hyperparameters which adds complexity, and (2) the VAD conditions on
acoustic features only (speech/silence). The E2E segmenter [5] tackles these
issues by predicting the EOS signal directly from the decoder of a streaming
E2E ASR model. Specifically it adds an additional RNNT joint layer trained to
predict the EOS token via a weakly supervised EOS annotation method [6]. This
simplifies the system by unifying ASR and segmentation into a single model. It
also improves quality and EOS latency by leveraging both acoustic and
linguistic context available to the E2E ASR model. That said, E2E segmentation
has only been demonstrated in single-pass models [5].

Unifying E2E segmentation with the higher-quality two-pass cascaded encoder
model is the next logical step toward building a low-latency and state-of-the-
art streaming ASR system for long-form tasks. In this work, we discuss the
challenges to unifying ASR and segmentation and present our solution which led
to a single cascaded-encoder model that predicts EOS as shown in Figure 1a,
with gains of 2.4% WER relative, 7.3% oracle WER relative, and 140 ms EOS
latency on a YouTube long-form testset.

###  1.1 Related work

Previous works have sought to tune the 2nd-pass latency in two-pass ASR. [7]
modifies the conformer layers to have dynamic right context length and
combines it with 2nd-pass rescoring to allow for tunable finalization latency
in a two-pass model. Similarly, [8] leverages two non-causal encoders with
variable input context sizes to allow for fast and slow outputs from the 2nd
pass.

Other works have sought to unify additional tasks with two-pass ASR into a
single model. E2E prefetching [9] lowers fulfillment latency by querying the
downstream response based on 1st-pass results and delivering that response if
the 1st pass matches the eventual 2nd-pass finalized result. E2E endpointing
[10, 11] lowers microphone closing latency by predicting end-of-utterance with
the 1st pass and using that signal to finalize the 2nd pass. Unifying
prefetching and endpointing with two-pass ASR is important for applications
where the utterances are short and require quick 2nd-pass results, such as the
voice search. In a similar spirit, we for the first time unify segmenting with
two-pass ASR for applications where the utterances are long and yet still
require quick 2nd-pass finalization, such as live captioning, dictation, or
conversation agents.

Fig. 1: Schematic of model architecture (a) and inference procedure
immediately before (b) and after EOS emission (c).

##  2 Unifying E2E Segmentation with Cascaded Encoder

In contrast to conventional two-pass models where the 2nd pass must wait till
the end-of-segment before decoding [12], the cascaded encoder’s 2nd pass
decodes in parallel with the 1st pass, but in a time-staggered fashion so that
it can access right-context audio frames. Specifically, the 2nd pass decoder
lags real-time by 900 ms so that the cascaded encoder can condition on 900 ms
worth of frames to the right of its center frame [13, 14], as illustrated in
Figure 1a. While E2E segmentation has been successfully demonstrated for
single-pass models, certain challenges stand in the way of integrating it into
the two-pass cascaded encoder setup. We now discuss these challenges and our
solutions. Note that we’ll refer to “frame” as the representation at a
particular time, but the representation could correspond to the input, causal
encoder output, or cascaded encoder output, depending on the discussion
context.

###  2.1 1st-pass EOS prediction, 2nd-pass finalization

The EOS signal should be determined in real-time since any delay directly
affects user-perceived latency. This is easy to implement with a VAD
segmenter, since the VAD is run upstream of the decoder. In a E2E segmenter,
however, the EOS is determined by the decoder itself. If EOS was determined by
the 2nd pass decoder, its timestamp would always be at least 900 ms behind
real-time, which is a prohibitive amount of latency. We propose the following
inference strategy:

  1. 1.

The 1st pass predicts EOS in real-time

  2. 2.

When the EOS signal is emitted, finalize the 1st pass and route the EOS
timestamp to the 2nd pass

  3. 3.

When the 2nd pass receives the EOS timestamp, inject dummy frames and finalize
(discussed further in Section 2.2)

Figure 1a illustrates this architecture.

###  2.2 2nd-pass finalization strategy

Note importantly that the procedures discussed in the following section
applies to inference only, where latency is critical. The training the
procedure is discussed in Section 3.2. Now consider as illustration Figure 1b.
When the time is $T$, “hello world” has been spoken. At this time, the causal
1st pass decoder (not shown) decides to emit an EOS (red frame at time $T$)
and it immediately routes the EOS’s timestamp, i.e. $T$, to the 2nd pass. At
that time, the cascaded encoder’s center frame is at $T-3$, or 3 frames (900
ms in reality) behind the real-time frame at $T$, so the 2nd pass decoder has
yet to decode the word, “world”. When the 2nd pass receives the EOS timestamp,
there are several possible responses:

Table 1: Comparison of 2nd-pass EOS handling strategies

|  | 2nd pass final. latency
---|---|---
2nd pass EOS handling | WER | Algorithmic | Computational
B1: Finalize at EOS emission | 17.99 | 0 |  $\sim$ 0
B2: Finalize at EOS timestamp | 17.31 | 900 |  $\geq$900
E1: B2 + dummy frames (zeros) | 18.65 | 0 |  208 [13]
E2: B2 + dummy frames (last) | 15.86 | 0 |  208 [13]

  1. 1.

Finalize immediately. This would result in the last 3 frames of the segment
not being decoded, resulting in the erroneous deletion of “world”, since the
2nd pass center frame lags that of the 1st pass by 3 frames and is still
processing $T-3$ when 1st pass emits EOS. See Figure 1b.

  2. 2.

Finalize when 2nd pass reaches EOS timestamp. This would mean waiting until
the center frame of the 2nd pass decoder (currently at $T-3$) to reach $T$, or
equivalently, for the center frame of the 1st pass decoder or causal encoder
(currently at $T$) reaches $T+3$. This can be visualized in Figure 1b as
sliding the blue fprop context windows 3 frames rightward. By then, “world” is
correctly decoded by the 2nd pass decoder, avoiding the deletion error from
Option 1 above. However, this requires waiting for 3 additional frames to be
generated because they are needed by the cascaded encoder as right-context.
Incurring 3 frames of latency (900 ms in reality) can be prohibitive for user
experience.

  3. 3.

Inject 900 ms of dummy right-context frames, then finalize when 2nd pass
reaches EOS timestamp. In this best-of-both-worlds alternative, we still allow
the 2nd pass to decode until the EOS timestamp to avoid deletion errors, but
we prevent waiting for 3 right-context frames by immediately injecting 3 dummy
right-context frames, as illustrated in Figure 1c. Since the dummy frames are
injected all at once, the cascaded encoder can fprop them all in parallel, as
illustrated by the three fprop context windows in Figure 1c. And they are
injected at the input to the cascaded encoder, so we save some compute by not
fpropping anything through the causal encoder. All this can occur at time $T$
before the arrival of the next frame; thus the algorithmic latency of the
process is zero. The computational latency (time for the cascaded encoder to
fprop and decode the dummy frames), on the other hand, is a relatively low 208
ms based on measurements with a Pixel 6 phone on a similar architecture [13].
Unlike Option 2, which has an algorithmically lower-bounded latency of 900 ms,
208 ms is an acceptable amount of user-perceived latency and can be improved
with better hardware.

We experiment with two types of dummy frames:

     * •

Zero: all dummy frames are zero vectors

     * •

Last-frame: all dummy frames are identical to last frame outputted by the
causal encoder. The rationale is that the last frame occurs during the brief
silence after the user speaks and is a better encoder representation of
silence than zeros.

Table 1 summarizes the WER and algorithmic and computational latency of 2nd
pass finalization for all the strategies discussed in this section. We pick as
our final strategy the one which gives the best WER and finalization latency,
namely dummy last-frame injection (E2). Again, we apply this strategy only
during inference.

##  3 Experimental Settings

###  3.1 Model and dataset

Our base cascaded encoder model is identical to that of [14] in vocabulary,
architecture, optimizer setup, training data, codebase, and training hardware.
Briefly, the causal encoder, cascaded encoder, and separate decoders have 47M,
60M, and 4.4M parameters, respectively. It’s trained on 400k hours of English
anonymized audio-text pairs from multiple domains, such as YouTube and
anonymized voice search traffic. Our data handling abides by Google AI
Principles [15].

###  3.2 EOS joint layer

Our EOS joint layer is identical in size to, and initialized from the weights
of, the fully trained ASR joint layer. It consumes the outputs of the encoder
and prediction network, allowing it to see both acoustic and linguistic
context, as shown in Figure 1a. It is fine-tuned on the same dataset but with
the target transcripts augmented by EOS tokens. The EOS annotation scheme is
identical to that of [5, 6]. Briefly, EOS tokens are inserted at the end of
the transcript for short-query utterances, such as voice search. They’re also
inserted between two words for other utterances, such as YouTube, when there
is a long silence preceded by no hesitation or an even longer silence preceded
by hesitation. During inference, if the EOS token negative log-probability
cost goes below a certain threshold, the 1st-pass beam search will immediately
finalize and the EOS timestamp will be routed to the 2nd pass for 2nd-pass
finalization.

###  3.3 Beam search

We use a frame-synchronous beam search for both passes. The 1st and 2nd passes
have beam sizes of 4 and 8, respectively, with pruning threshold of 5 for
both. At each frame, we apply breadth-first search for possible expansions
similar to [16], ignoring any expansion with a negative log posterior of 5 or
greater, and limiting the search depth to 10 expansions.

###  3.4 Voice activity detector

Our pipeline contains a lightweight voice activity detector [17] upstream of
the E2E model. In addition to being used as a segmenter in our VAD segmenter
experiments, it also functions as a frame filterer; consecutive frames
classified as silence are filtered out after the initial 200 ms of silence.
Frame filtering is turned on for all experiments to match edge deployment
conditions.

###  3.5 Evaluation

YouTube videos are multi-domain (commentary, lectures, meetings, etc.) and are
often long-form [18], making captioning of YouTube audio an ideal task for our
long-form study. We evaluate on YT_LONG, a standard YouTube testset used in
[19, 20, 21] whose videos are sampled from YouTube video-on-demand. YT_LONG
has 77 total utterances made of 22.2 hours and 207191 words, with median
utterance length of 14.8 minutes.

##  4 Results

In this section, we report the 50th and 90th percentile segment lengths in
seconds (SL50/90) and EOS latency in milliseconds (EOS50/90), along with 1st
and 2nd pass WER. Segment lengths provide a gauge of how aggressive the
segmenter is. A reasonable length for segments is 10-20 seconds. EOS latency
is measured as the delay from the end-of-speech (obtained via forced
alignment) and the EOS timestamp. For consistency, we only keep the EOS
latencies for the last segment of each utterance in order to de-confound the
fact that different segmenters have different number of segments. 1st pass WER
is important for live-streaming accurate results to the user as they speak,
while 2nd pass WER is important for sending accurate results to the downstream
task.

Table 2: Segment lengths (SL), EOS latencies (EOS), and WERs of various
segmenters on YT_LONG

|  |  |  |  | 2nd pass | 1st pass
---|---|---|---|---|---|---
Segmenter | SL50 | SL90 | EOS50 | EOS90 | WER | WER
B3: Fixed-3s | 3.0 | 3.0 | - | - | 19.60 | 19.20
B4: Fixed-5s | 5.0 | 5.0 | - | - | 17.78 | 19.39
B5: Fixed-10s | 10.0 | 10.0 | - | - | 16.82 | 19.16
B6: VAD | 3.3 | 14.0 | 380 | 490 | 16.23 | 19.13
E3: E2E | 6.9 | 20.8 | 240 | 480 | 15.86 | 18.72
$\Rightarrow$ E3 vs. B6 | - | - | -140 | -10 | -2.4% | -2.1%

Fig. 2: Distributions of latencies and segment lengths.

###  4.1 Main results

Table 2 shows our main results: we evaluate the E2E segmenter (E3) against the
VAD (B6) and a few fixed-length segmenters (B3-B5) as baselines. Note that
henceforth we will use E2E and VAD to refer to their respective segmenters.
The fixed-length segmenters fare worse than the VAD and E2E in WER as
expected, despite having segment lengths within the same range, indicating
that segment length alone does not have a strong effect on WER. The E2E
segmenter achieves a median EOS latency gain of 140 ms, and a small 90th
percentile gain of 10 ms over the VAD. Further, it achieves a WER gains of
2.4% and 2.1% on the 2nd and 1st pass, respectively. While the WER gains are
modest, the median latency gain of 140 ms is significant for user experience.

The distributions of EOS latency are shown in Figure 2 (top) for E2E and VAD.
Note here we include the EOS latencies across all segments in the testset (as
opposed to only the last segment) to get more statistics. E2E has
significantly more variance in its EOS latency distribution, perhaps
reflecting the diversity of acousto-linguistic situations under which it must
make a decision. For example, some EOS may be easy to predict from the
linguistic context, while others may be ambiguous and require more acoustic
context. Note that the VAD has no EOS latency under 200 ms as expected due to
its design of requiring 200 ms of detected silence before activating (Section
1). The distributions of segment length are shown in Figure 2 (bottom). The
biggest difference is that E2E has much fewer segments lengths near zero than
the VAD, reflecting the fact that during very long silences, the VAD—which
only cares whether the past 200 ms is silence—can emit more than one EOS
signal within the same span of silence.

###  4.2 Ablation study

There are two hyperparameters that tune how aggressively EOS signals are
emitted during inference: (1) EOS threshold, the negative-log-probability
threshold under which EOS is emitted during inference, and (2) Silence length
threshold, the minimum length of silence between two words in a training
utterance such that a EOS token will be injected between those two words in
its target transcript. We provide an ablation study against both these
parameters in Figure 3 (left), where we plot the WER against the EOS threshold
(x-axis) for multiple silence length thresholds (legend). Each curve has a
sweet spot where the WER is optimal, and the EOS threshold at which this
occurs gets smaller (tighter) as the silence length threshold gets smaller
(looser). It makes sense: if the model is trained to emit EOS more
aggressively (smaller silence length threshold), then it needs to be more
restricted during inference (smaller EOS threshold). This study shows the
silence length threshold hyperparameter is rather insensitive during training
as we can compensate for it during inference. In Figure 3 (right), we plot the
same data but against the median segment length (x-axis), which is tuned via
the EOS threshold. This shows that the changing the silence length threshold
does not affect the optimal segment lengths during inference, suggesting that
the testset has an underlying sweet spot for how long segments should be. We
choose as our operating point the point that gives best WER, 600 ms silence
length threshold and 3.7 EOS threshold, and use this operating point for the
results in Table 2 and Figure 2.

Fig. 3: Ablation study of 2nd pass WER as a function of various levels of EOS
emission aggressiveness during training and inference.

###  4.3 Oracle WER

Since we use a fixed ASR model, measuring the oracle WER (OWER) can give some
insight into how much improvement can come from a better beam search decoding.
In Table 3, we report WER and OWER for baseline (B7-B9) and E2E segmenter (E4)
on a 15-utterance subset of YT_LONG. This was the subset on which we were able
to successfully obtain forced alignments on the transcripts, which are
required for computing the oracle WER on long utterances without hitting
memory limits. In general, more frequent segmentation increases diversity in
the beam search, but can also lead to poorer hypotheses. To rule out this
confounding factor, we do a controlled experiment where we pick a EOS
threshold such that the median segment length is the same as that of the VAD.
At this setting, the E2E achieves a 5.6% OWER gain over the VAD (Standard
column in Table 3). Next, we repeat the comparison under a high lattice
diversity scenario by applying 2-gram-based path merging during beam search
[22] (Path-merging column). We find that the OWER improves by 25% relative to
standard beam search due to path merging, and under these conditions, the E2E
achieves an even bigger OWER gain of 7.3% over the VAD. These findings suggest
that segmentation improves ASR quality not by creating more lattice diversity,
but by improving the overall accuracy of the hypotheses in the lattice.

Table 3: Oracle WER from a standard and 2-gram-based path-merging beam search
for various segmenters

|  | Standard | Path-merging
---|---|---|---
Segmenter | SL50 | WER | OWER | WER | OWER
B7: Fixed-3s | 3.0 | 11.6 | 10.3 | 11.4 | 7.2
B8: Fixed-5s | 5.0 | 9.6 | 7.8 | 9.4 | 6.0
B9: VAD | 4.4 | 7.8 | 5.4 | 7.9 | 4.1
E4: E2E | 4.4 | 7.8 | 5.1 | 7.9 | 3.8
$\Rightarrow$ E4 vs. B9 | - | - | -5.6% | - | -7.3%

###  4.4 Error analysis

Lastly, we investigate the decoding of some single utterances with the E2E and
VAD segmenters to better understand where the wins and losses are coming from.
Table 4 shows snippets of three long utterance decodings for the E2E and VAD.
EOS locations are shown in blue and deletions/insertions relative to ground
truth are shown in red/green. In Table 4a, the VAD substitutes “up” for
“just”, but E2E does not make that mistake. The only difference was that the
VAD emitted an EOS immediately after “up”, causing the prior (erroneous)
hypothesis to be finalized, whereas E2E had the opportunity to correct the top
hypothesis given later context. A similar story applies to Table 4b, where the
VAD prematurely finalizes the deletion of “you can put the” by emitting an EOS
immediately afterward. The examples from Table 4a and 4b highlight a more
general trend that the WER wins from E2E tend to be due to smarter EOS
boundaries, preventing finalization of the wrong hypotheses. We were unable to
find any trends that favored the VAD over E2E, and the VAD wins tended to be
less systematic. For example, Table 4c shows a loss for E2E, where “over
stretch” is substituted by “overstretch”. Since both VAD and E2E shared the
same ASR model and differ only in the segmenters, this error can only be
attributed to the two having different decoder states and beam search lattices
caused by earlier differences in EOS emission. It’s unlikely that such non-
systematic errors favor one segmenter over the other.

Table 4: Error analysis.

| VAD Segmenter | E2E Segmenter
---|---|---
(a) Win |  …<EOS>the inch <EOS> |  …<EOS>the inch <EOS>
| increments here all my line | increments here all my line
|  <EOS>just up <EOS> |  <EOS>just
|  so we get the same <EOS>… |  so we get the same <EOS>…
(b) Win | …you can go around like i was | …you can go around like i was
|  doing before you can put |  doing before <EOS>you can put
|  the <EOS>edge every time |  the today edge every time
| right up to your … | right up to your …
(c) Loss |  …<EOS>you don’t have to go |  …<EOS>you don’t have to go
| back and forth and | back and forth and
| over stretch |  over stretch overstretch
| or i’m sorry over shrink or | or i’m sorry over shrink or
| over stretch |  over stretch overstretch
| depending what you’re doing and | depending what you’re doing and
|  cause an issue <EOS>… |  cause an issue <EOS>…

##  5 Conclusion

We have unified the E2E segmenter with the two-pass cascaded encoder model, an
important step toward low latency and high quality long-form speech
recognition in streaming mode. Advantages of this unification include
simplicity (removing additional VAD module), EOS latency reduction, and WER
improvements on both the 1st and 2nd pass.

## References

  * [1] Arun Narayanan, Tara N Sainath, Ruoming Pang, Jiahui Yu, Chung-Cheng Chiu, Rohit Prabhavalkar, Ehsan Variani, and Trevor Strohman,  “Cascaded encoders for unifying streaming and non-streaming asr,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 5629–5633.
  * [2] Yuan Shangguan, Rohit Prabhavalkar, Hang Su, Jay Mahadeokar, Yangyang Shi, Jiatong Zhou, Chunyang Wu, Duc Le, Ozlem Kalinli, Christian Fuegen, et al.,  “Dissecting user-perceived latency of on-device e2e speech recognition,”  arXiv preprint arXiv:2104.02207, 2021.
  * [3] Javier Ramirez, Juan Manuel Górriz, and José Carlos Segura,  “Voice activity detection. fundamentals and speech recognition system robustness,”  Robust speech recognition and understanding, vol. 6, no. 9, pp. 1–22, 2007.
  * [4] Takenori Yoshimura, Tomoki Hayashi, Kazuya Takeda, and Shinji Watanabe,  “End-to-end automatic speech recognition integrated with ctc-based voice activity detection,”  in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 6999–7003.
  * [5] W Ronny Huang, Shuo-yiin Chang, David Rybach, Rohit Prabhavalkar, Tara N Sainath, Cyril Allauzen, Cal Peyser, and Zhiyun Lu,  “E2e segmenter: Joint segmenting and decoding for long-form asr,”  arXiv preprint arXiv:2204.10749, 2022.
  * [6] Shuo-yiin Chang, Bo Li, Tara N Sainath, Chao Zhang, Trevor Strohman, Qiao Liang, and Yanzhang He,  “Turn-taking prediction for natural conversational speech,”  arXiv preprint arXiv:2208.13321, 2022.
  * [7] Binbin Zhang, Di Wu, Zhuoyuan Yao, Xiong Wang, Fan Yu, Chao Yang, Liyong Guo, Yaguang Hu, Lei Xie, and Xin Lei,  “Unified streaming and non-streaming two-pass end-to-end model for speech recognition,”  arXiv preprint arXiv:2012.05481, 2020.
  * [8] Jay Mahadeokar, Yangyang Shi, Ke Li, Duc Le, Jiedan Zhu, Vikas Chandra, Ozlem Kalinli, and Michael L Seltzer,  “Streaming parallel transducer beam search with fast-slow cascaded encoders,”  arXiv preprint arXiv:2203.15773, 2022.
  * [9] Shuo-Yiin Chang, Bo Li, David Rybach, et al.,  “Low latency speech recognition using end-to-end prefetching,”  in Proc. Interspeech, 2020.
  * [10] Bo Li, Anmol Gulati, Jiahui Yu, Tara N Sainath, Chung-Cheng Chiu, Arun Narayanan, Shuo-Yiin Chang, Ruoming Pang, Yanzhang He, James Qin, et al.,  “A better and faster end-to-end model for streaming asr,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 5634–5638.
  * [11] Bo Li Tara Sainath Chao Zhang Yanzhang He Shaan Bijwadia, Shuo-yiin Chang,  “Toward Domain-Invariant Speech Recognition via Large Scale Training,”  in Proc. SLT, 2018.
  * [12] Tara N Sainath, Ruoming Pang, David Rybach, Yanzhang He, Rohit Prabhavalkar, Wei Li, Mirkó Visontai, Qiao Liang, Trevor Strohman, Yonghui Wu, et al.,  “Two-pass end-to-end speech recognition,”  Proc. Interspeech 2019, pp. 2773–2777, 2019.
  * [13] Tara N Sainath, Yanzhang He, Arun Narayanan, Rami Botros, Weiran Wang, David Qiu, Chung-Cheng Chiu, Rohit Prabhavalkar, Alexander Gruenstein, Anmol Gulati, et al.,  “Improving the latency and quality of cascaded encoders,”  in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8112–8116.
  * [14] Shaojin Ding, Weiran Wang, Ding Zhao, Tara N Sainath, Yanzhang He, Robert David, Rami Botros, Xin Wang, Rina Panigrahy, Qiao Liang, et al.,  “A unified cascaded encoder asr model for dynamic model sizes,”  arXiv preprint arXiv:2204.06164, 2022.
  * [15] “Artificial intelligence at google: Our principles.,” https://ai.google/principles/,  Accessed: 2022-07-20.
  * [16] Anshuman Tripathi, Han Lu, Hasim Sak, and Hagen Soltau,  “Monotonic recurrent neural network transducer and decoding strategies,”  in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2019, pp. 944–948.
  * [17] Rubén Zazo Candil, Tara N Sainath, Gabor Simko, and Carolina Parada,  “Feature learning with raw-waveform cldnns for voice activity detection,”  2016\.
  * [18] Arun Narayanan, Rohit Prabhavalkar, Chung-Cheng Chiu, David Rybach, Tara N Sainath, and Trevor Strohman,  “Recognizing long-form speech using streaming end-to-end models,”  in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2019, pp. 920–927.
  * [19] H. Soltau, H. Liao, and H. Sak,  “Reducing the Computational Complexity for Whole Word Models,”  in Proc. ASRU, 2017.
  * [20] C.-C. Chiu, W. Han, Y. Zhang, et al.,  “A comparison of end-to-end models for long-form speech recognition,”  in Proc. ASRU, 2019.
  * [21] Chung-Cheng Chiu, Arun Narayanan, Wei Han, Rohit Prabhavalkar, Yu Zhang, Navdeep Jaitly, Ruoming Pang, Tara N Sainath, Patrick Nguyen, Liangliang Cao, et al.,  “Rnn-t models fail to generalize to out-of-domain audio: Causes and solutions,”  in 2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 873–880.
  * [22] Rohit Prabhavalkar, Yanzhang He, David Rybach, Sean Campbell, Arun Narayanan, Trevor Strohman, and Tara N Sainath,  “Less is more: Improved rnn-t decoding using limited label context and path merging,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 5659–5663.