††thanks: ECS Active member

Also at: ]Lomonosov Moscow State University, Research Computing Center, 119991
Moscow, Russia

# Analytical impedance of oxygen transport in the channel and gas diffusion
layer of a PEM fuel cell

Andrei Kulikovsky<EMAIL_ADDRESS>Forschungszentrum Jülich GmbH
Theory and Computation of Energy Materials (IEK–13)
Institute of Energy and Climate Research,
D–52425 Jülich, Germany  [

###### Abstract

Analytical model for impedance of oxygen transport in the gas–diffusion layer
(GDL) and cathode channel of a PEM fuel cell is developed. The model is based
on transient oxygen mass conservation equations coupled to the proton current
conservation equation in the catalyst layer. Analytical formula for the
“GDL+channel” impedance is derived assuming that the oxygen and proton
transport in the cathode catalyst layer (CCL) are fast. In the Nyquist plot,
the resulting impedance consists of two arcs describing oxygen transport in
the air channel (low–frequency arc) and in the GDL. The characteristic
frequency of GDL arc depends on the CCL thickness: large CCL thickness
strongly lowers this frequency. At small CCL thickness, the high–frequency
feature on the arc shape forms. This effect is important for identification of
peaks in distribution of relaxation times spectra of low–Pt PEMFCs.

PEM fuel cell, impedance, GDL, modeling

##  I Introduction

Electrochemical impedance spectroscopy (EIS) provides invaluable information
on transport properties of PEM fuel cell in a current production mode, without
interruption of cell functioning[1]. Not surprisingly, EIS of PEM fuel cells
is a rapidly growing field[2]. Understanding impedance spectra requires
modeling. Strong criticism of equivalent circuit approach has been published
by Macdonald in his seminal paper[3] and in recent years, physics–based models
for PEMFC impedance tend to replace equivalent circuit modeling (see recent
reviews of Tang et al. [2] and Huang et al. [4]).

Every transport and kinetic process in a PEM fuel cell has its own resonance
frequency. If these frequencies do not overlap, one could identify them using
the distribution of relaxation times (DRT) technique[5, 6, 7]. In addition,
DRT analysis of impedance spectra returns the contribution of every process
into the total differential resistance of the cell. However, correct
identification of DRT peaks is a non–trivial task requiring modeling and
experimental work. Analytical models predicting characteristic frequencies of
oxygen transport processes in the cell could be very helpful in this respect.

Oxygen reduction reaction (ORR) is usually responsible for a large part of
potential loss in a PEMFC. Oxygen is transported to the catalyst sites through
the cathode channel and gas–diffusion layer; both the transport processes have
their signatures in the EIS spectra. After pioneering experimental work of
Schneider et al. [8, 9], Kulikovsky and Shamardina[10, 11], Maranzana et al.
[12] and Chevalier et al. [13] developed numerical and analytical models
incorporating channel impedance. Formulas for pure channel impedance have been
obtained[13, 14] assuming fast oxygen transport through the GDL and cathode
catalyst layer (CCL). However, the coupling between the channel and GDL
impedance remained poorly understood. Recently, Cruz–Manzo and Greenwood
reported analytical model for the GDL+channel impedance[15]. However, their
result missing important effect of double layer charging on this impedance, as
discussed below.

In this work, we develop analytical model for the GDL+channel impedance
$Z_{gdlc}$ in a PEM fuel cell. Assuming fast oxygen and proton transport in
the CCL, analytical expression for $Z_{gdlc}$ is derived. We show that for
typical PEMFC parameters, the Nyquist spectrum of $Z_{gdlc}$ consists of two
arcs corresponding to oxygen transport in the channel and GDL. GDL impedance
differs from the Warburg finite–length impedance due to “non–Warburg” factor
depending on the superficial double layer capacitance of the electrode. For
typical PEMFC parameters, the characteristic frequency $f_{gdl}$ of the GDL
impedance $Z_{gdl}$ is close to the Warburg finite–length frequency; however,
for larger CCL thickness typical for non–Pt cells, the non–Warburg factor
strongly lowers $f_{gdl}$. In the opposite limit of small catalyst layer
thickness, a high–frequency feature on the shape of imaginary part of
$Z_{gdl}$ vs frequency forms. Analysis shows that the channel impedance also
depends on the double layer capacitance meaning that impedance of all oxygen
transport medias in the PEMFC “feel” double layer capacitance of the electrode
where the oxygen is transported to. The goal of this paper is to clarify the
situation with oxygen transport impedance in the GDL and channel after
interesting and useful, but incomplete and rather difficult for understanding
work of Cruz-Manzo and Greenwood[15].

##  II Model

Schematic of the cell with the straight cathode channel is shown in Figure 1.
Our main goal here is analysis of impedance of the GDL+channel oxygen
transport system. To simplify the model and to separate the GDL+channel
impedance from impedance of oxygen transport in the CCL, we will assume that
the latter transport is fast. In addition, for simplicity we will assume that
the proton transport in the CCL is also fast. The characteristic frequency of
proton transport in the CCL is much higher than the other frequencies in the
system and hence this assumption does not affect low– and medium–frequency
impedances which are of primary interest in this work.

Figure 1: Schematic of the segmented cell cathode side with the straight
channel and the system of coordinates. $Z$ stands for local impedance
depending on the distance along the channel coordinate $z$.

###  II.1 Proton charge conservation equation

The proton charge conservation equation reads

$C_{dl}\dfrac{\partial{\eta}}{\partial{t}}+\dfrac{\partial{j}}{\partial{x}}=-i_{*}\left(\dfrac{c}{c_{h}^{in}}\right)\exp\left(\dfrac{\eta}{b}\right)$
(1)

where $C_{dl}$ is the volumetric double layer capacitance (F cm-3), $\eta$ is
the positive by convention ORR overpotential, $j$ is the local proton current
density in the CCL, $x$ is coordinate through the cell cathode counted from
the membrane, $i_{*}$ is the volumetric exchange current density (A cm-3), $c$
is the local oxygen concentration, $c_{h}^{in}$ is the reference oxygen
concentration, and $b$ is the ORR Tafel slope.

Since proton and oxygen transport in the CCL are fast, $\eta$ and $c$ are
nearly independent of $x$. Integrating Eq.(1) over $x$ from 0 to the CCL
thickness $l_{t}$ we come to

$C_{dl}l_{t}\dfrac{\partial{\eta_{0}}}{\partial{t}}-j_{0}=-i_{*}l_{t}\left(\dfrac{c_{1}}{c_{h}^{in}}\right)\exp\left(\dfrac{\eta_{0}}{b}\right)$
(2)

where $\eta_{0}$ is the ORR overpotential at the membrane surface, $j_{0}$ is
the local cell current density, and $c_{1}$ is the oxygen concentration at the
CCL/GDL interface.

To simplify calculations, we introduce dimensionless variables

$\tilde{t}=\dfrac{t}{t_{*}},\quad\tilde{x}=\dfrac{x}{l_{t}},\quad\tilde{z}=\dfrac{z}{L},\quad\tilde{c}=\dfrac{c}{c_{h}^{in}},\quad\tilde{j}=\dfrac{j}{i_{*}l_{t}},\\\
\tilde{\eta}=\dfrac{\eta}{b},\quad\tilde{D}_{b}=\dfrac{4FD_{b}c_{h}^{in}}{i_{*}l_{t}^{2}},\quad\tilde{l}_{b}=\dfrac{l_{b}}{l_{t}},\\\
\tilde{\omega}=\omega t_{*},\quad\tilde{Z}=\dfrac{Zi_{*}l_{t}}{b}$ (3)

where

$t_{*}=\dfrac{C_{dl}b}{i_{*}},$ (4)

is the characteristic time of double layer charging, $z$ is the coordinate
along the cathode channel, $L$ is the channel length, $D_{b}$ is the oxygen
diffusion coefficient in the GDL, $l_{b}$ is the GDL thickness, $\omega$ is
the angular frequency of the applied AC signal, and $Z$ is the local
impedance.

With the dimensionless variables (3), Eq.(2) takes the form

$\dfrac{\partial{\tilde{\eta}_{0}}}{\partial{\tilde{t}}}-\tilde{j}_{0}=-\tilde{c}_{1}{{\rm
e}^{\tilde{\eta}_{0}}}$ (5)

Substituting Fourier–transforms

$\begin{split}&\tilde{\eta}_{0}=\tilde{\eta}^{0}+\tilde{\eta}^{1}(\tilde{\omega})\exp({\rm
i}\tilde{\omega}\tilde{t})\\\
&\tilde{j}_{0}=\tilde{j}^{0}+\tilde{j}^{1}(\tilde{\omega})\exp({\rm
i}\tilde{\omega}\tilde{t})\\\
&\tilde{c}_{1}=\tilde{c}_{1}^{0}+\tilde{c}_{1}^{1}(\tilde{\omega})\exp({\rm
i}\tilde{\omega}\tilde{t})\end{split}$ (6)

into Eq.(5), expanding exponent in Taylor series, neglecting term with the
perturbations product, and subtracting the static equation, we get equation
relating the small perturbation amplitudes $\tilde{\eta}^{1}$, $\tilde{j}^{1}$
and $\tilde{c}_{1}^{1}$ in the $\tilde{\omega}$–space:

$\tilde{j}^{1}={\rm i}\tilde{\omega}\tilde{\eta}^{1}+{{\rm
e}^{\tilde{\eta}_{0}}}\left(\tilde{c}_{1}^{1}+\tilde{c}_{1}^{0}\tilde{\eta}^{1}\right).$
(7)

Here, the superscripts 0 and 1 mark the static variables and the small
perturbation amplitudes, respectively. Local cathode side impedance at a
distance $\tilde{z}$ from the channel inlet is given by

$\tilde{Z}=\dfrac{\tilde{\eta}^{1}}{\tilde{j}^{1}}$ (8)

Dividing Eq.(7) by $\tilde{j}^{1}$, we obtain equation for $\tilde{Z}$:

$1=\left({\rm i}\tilde{\omega}+\tilde{c}_{1}^{0}{{\rm
e}^{\tilde{\eta}_{0}}}\right)\tilde{Z}+{{\rm
e}^{\tilde{\eta}_{0}}}\dfrac{\tilde{c}_{1}^{1}}{\tilde{j}^{1}}$ (9)

Suppose that the perturbation of oxygen concentration is zero:
$\tilde{c}_{1}^{1}=0$. Physically, this is equivalent to fast oxygen transport
in all transport medias of the cathode side, including channel. Solution to
Eq.(9) is then

$\tilde{Z}_{f}=\dfrac{1}{{\rm i}\tilde{\omega}+\tilde{c}_{1}^{0}{{\rm
e}^{\tilde{\eta}_{0}}}},$ (10)

which is impedance of a parallel $RC$–circuit, where the term ${\rm
i}\tilde{\omega}$ in denominator describes contribution of the double layer
capacitance and $\tilde{c}_{1}^{0}{{\rm e}^{\tilde{\eta}_{0}}}$ is the inverse
faradaic resistivity of the catalyst layer.

The static oxygen concentration $\tilde{c}_{1}^{0}$ is related to the channel
concentration $\tilde{c}_{h}^{0}$ as

$\tilde{c}_{1}^{0}=\tilde{c}_{h}^{0}\left(1-\dfrac{\tilde{j}^{0}}{\tilde{j}_{\lim}}\right)$
(11)

where

$j_{\lim}=\dfrac{4FD_{b}c_{h}^{0}}{l_{b}}$ (12)

is the local limiting current density due to oxygen transport in the GDL.
Eq.(11) is solution of the static version of equation for oxygen transport in
the GDL, see Eq.(18) below. The concentration $\tilde{c}_{h}^{0}$ varies along
the channel; numerical model[16] shows that unless the mean cell current
density is small, this variation to a good approximation is linear:

$\tilde{c}_{h}^{0}\simeq 1-\dfrac{\tilde{z}}{\lambda}$ (13)

Note that Eq.(13) is an approximation 111At low cell currents, the shape of
local oxygen concentration is exponential[18]:
$\tilde{c}_{h}^{0}=\left(1-\dfrac{1}{\lambda}\right)^{\tilde{z}}$ The model
can be reformulated using the above equation for $\tilde{c}_{h}^{0}$. However,
this equation is valid at the low cell currents only, while Eq.(13) works
better at higher cell currents, which are of practical interest.  the exact
shape of $\tilde{c}_{h}^{0}(\tilde{z})$ should be calculated using a numerical
model[17, 16].

To simplify calculations, we will ignore the factor
$1-{\tilde{j}^{0}}/{\tilde{j}_{\lim}}$ in Eq.(11), assuming that $j_{\lim}$ is
large and the variation of static oxygen concentration across the GDL is
negligible. With this, Eqs.(7), (9) transform to

$\tilde{j}^{1}={\rm i}\tilde{\omega}\tilde{\eta}^{1}+{{\rm
e}^{\tilde{\eta}_{0}}}\left(\tilde{c}_{1}^{1}+\left(1-\dfrac{\tilde{z}}{\lambda}\right)\tilde{\eta}^{1}\right)$
(14) $1=\left({\rm i}\tilde{\omega}+{{\rm
e}^{\tilde{\eta}_{0}}}\left(1-\dfrac{\tilde{z}}{\lambda}\right)\right)\tilde{Z}+{{\rm
e}^{\tilde{\eta}_{0}}}\dfrac{\tilde{c}_{1}^{1}}{\tilde{j}^{1}}$ (15)

As discussed above, the non–faradaic oxygen transport contributions to local
impedance gives the term with $\tilde{c}_{1}^{1}$. In order to calculate
$\tilde{c}_{1}^{1}$, we need to consecutively solve equations for oxygen
transport in the GDL and channel, as discussed in the next section.

For further references we need a total faradaic impedance $\tilde{Z}_{f,tot}$
of the cathode, taking into account variation of oxygen concentration along
the channel. Suppose that the cell is divided into $N\to\infty$ virtual
segments. Local current in each segment flows in the through–plane direction,
hence local faradaic impedances are connected in parallel. Thus,
$\tilde{Z}_{f,tot}$ is given by

$\tilde{Z}_{f,tot}=\left(\int_{0}^{1}\dfrac{d\tilde{z}}{\tilde{Z}_{f}}\right)^{-1}$
(16)

Electron conductivity of the cell components is assumed to be large and hence
$\tilde{\eta}^{0}$ is independent of the coordinate $\tilde{z}$. Substituting
Eq.(13) into Eq.(10), we get the local faradaic impedance
$\tilde{Z}_{f}(\tilde{z})$. Calculating integral, from Eq.(16) we find

$\tilde{Z}_{f,tot}=\dfrac{1}{{\rm
i}\tilde{\omega}+\left(1-\dfrac{1}{2\lambda}\right){{\rm
e}^{\tilde{\eta}_{0}}}}$ (17)

###  II.2 Oxygen transport in the GDL

Oxygen transport in the gas–diffusion layer is described by the diffusion
equation

$\dfrac{\partial{c_{b}}}{\partial{t}}-D_{b}\dfrac{\partial^{2}{c_{b}}}{\partial{x}^{2}}=0,\quad\left.D_{b}\dfrac{\partial{c_{b}}}{\partial{x}}\right|_{x=l_{t}+}=\dfrac{j_{0}}{4F},\\\
c_{b}(l_{t}+l_{b})=c_{h},$ (18)

where $c_{b}$ is the oxygen concentration in the GDL and $c_{h}$ is the oxygen
concentration in channel. The left boundary condition for Eq.(18) means that
the oxygen flux on the GDL side of the CCL/GDL interface equals the local
current density in the cell. This condition agrees with the assumption of fast
oxygen transport in the CCL.

With the dimensionless variables Eq.(3), Eq.(18) takes the form

$\mu^{2}\dfrac{\partial{\tilde{c}_{b}}}{\partial{\tilde{t}}}-\tilde{D}_{b}\dfrac{\partial^{2}{\tilde{c}_{b}}}{\partial{\tilde{x}}^{2}}=0,\quad\left.\tilde{D}_{b}\dfrac{\partial{\tilde{c}_{b}}}{\partial{\tilde{x}}}\right|_{\tilde{x}=1+}=\tilde{j}_{0},\\\
\tilde{c}_{b}(1+\tilde{l}_{b})=\tilde{c}_{h},$ (19)

where the dimensionless parameter $\mu$ is given by

$\mu=\sqrt{\dfrac{4Fc_{h}^{in}}{C_{dl}b}}.$ (20)

Eq.(19) is linear and hence the equation for the small perturbation amplitude
$\tilde{c}_{b}^{1}$ is

$\tilde{D}_{b}\dfrac{\partial^{2}{\tilde{c}_{b}^{1}}}{\partial{\tilde{x}}^{2}}={\rm
i}\tilde{\omega}\mu^{2}\tilde{c}_{b}^{1},\quad\left.\tilde{D}_{b}\dfrac{\partial{\tilde{c}_{b}^{1}}}{\partial{\tilde{x}}}\right|_{\tilde{x}=1+}=\tilde{j}^{1},\\\
\tilde{c}_{b}^{1}\left(1+\tilde{l}_{b}\right)=\tilde{c}_{h}^{1}$ (21)

where $\tilde{c}_{h}^{1}$ is the oxygen perturbation in channel (see below).

Solution to Eq.(21) is

$\tilde{c}_{b}^{1}=-\dfrac{\tilde{j}^{1}\sinh\left(\mu\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}\,\left(1+\tilde{l}_{b}-\tilde{x}\right)\right)}{\mu\sqrt{{\rm
i}\tilde{\omega}\tilde{D}_{b}}\cosh\left(\mu\tilde{l}_{b}\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}\right)}\\\
+\dfrac{\tilde{c}_{h}^{1}\cosh\left(\,\mu\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}(1-\tilde{x})\right)}{\cosh\left(\mu\tilde{l}_{b}\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}\right)}$ (22)

Setting here $\tilde{x}=1$, we get the perturbation amplitude of oxygen
concentration at the CCL/GDL interface
$\tilde{c}_{b}^{1}(1)=\tilde{c}_{1}^{1}$

$\tilde{c}_{1}^{1}=-\dfrac{\tilde{j}^{1}\tanh\left(\phi\right)}{\psi}+\dfrac{\tilde{c}_{h}^{1}}{\cosh\left(\phi\right)}$
(23)

where $\phi$ and $\psi$ are auxiliary dimensonless parameters

$\begin{split}&\phi=\mu\tilde{l}_{b}\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}\\\ &\psi=\mu\sqrt{{\rm
i}\tilde{\omega}\tilde{D}_{b}}\end{split}$ (24)

###  II.3 Oxygen transport in channel

To a good approximation, oxygen mass transport in the channel can be described
by the 1d + 1d plug flow equation:

$\dfrac{\partial{c_{h}}}{\partial{t}}+v\dfrac{\partial{c_{h}}}{\partial{z}}=-\left.\dfrac{D_{b}}{h}\dfrac{\partial{c_{b}}}{\partial{x}}\right|_{x=l_{t}+l_{b}}$
(25)

where $h$ is the channel depth. The right side of Eq.(25) is the oxygen
diffusive flux in the GDL at the channel/GDL interface representing oxygen
“sink” from the channel.

With the dimensionless variables (3), Eq.(25) reads

$\xi^{2}\dfrac{\partial{\tilde{c}_{h}}}{\partial{\tilde{t}}}+\lambda\tilde{J}\dfrac{\partial{\tilde{c}_{h}}}{\partial{\tilde{z}}}=-\left.\tilde{D}_{b}\dfrac{\partial{\tilde{c}_{b}}}{\partial{\tilde{x}}}\right|_{\tilde{x}=1+\tilde{l}_{b}}$
(26)

where $\tilde{J}$ is the mean current density in the cell

$\tilde{J}=\int_{0}^{1}\tilde{j}_{0}d\tilde{z},$ (27)

$\xi$ is the dimensionless parameter, and $\lambda$ is the stoichiometry of
air flow

$\xi=\sqrt{\dfrac{4Fhc_{h}^{in}}{C_{dl}l_{t}b}},\quad\lambda=\dfrac{4Fhvc_{h}^{in}}{LJ}.$
(28)

Eq.(26) is linear and we can immediately write down equation for the
perturbation amplitude $\tilde{c}_{h}^{1}$:

$\lambda\tilde{J}\dfrac{\partial{\tilde{c}_{h}^{1}}}{\partial{\tilde{z}}}=-{\rm
i}\tilde{\omega}\xi^{2}\tilde{c}_{h}^{1}-\left.\tilde{D}_{b}\dfrac{\partial{\tilde{c}_{b}^{1}}}{\partial{\tilde{x}}}\right|_{\tilde{x}=1+\tilde{l}_{b}}$
(29)

Differentiating Eq.(22), we find the flux
$\tilde{D}_{b}{\partial{\tilde{c}_{b}^{1}}}/{\partial{\tilde{x}}}|_{\tilde{x}=1+\tilde{l}_{b}}$
and Eq.(29) takes the form

$\lambda\tilde{J}\dfrac{\partial{\tilde{c}_{h}^{1}}}{\partial{\tilde{z}}}=-{\rm
i}\tilde{\omega}\xi^{2}\tilde{c}_{h}^{1}-\tilde{j}^{1}\cosh\left(\phi\right)^{-1}-\tilde{c}_{h}^{1}\psi\,\tanh\left(\phi\right),\\\
\tilde{c}_{h}^{1}(0)=0$ (30)

where $\tilde{j}^{1}$ is a function of coordinate $\tilde{z}$ and of
$\tilde{c}_{h}^{1}$. To find the explicit dependence
$\tilde{j}^{1}(\tilde{z})$ we substitute (23) into Eq.(14); solving the
resulting equation for $\tilde{j}^{1}$ we get

$\tilde{j}^{1}=\left(\dfrac{{{\rm
e}^{\tilde{\eta}_{0}}}\tanh\left(\phi\right)}{\psi}+1\right)^{-1}\\\
\times\left(\left({\rm i}\tilde{\omega}+{{\rm
e}^{\tilde{\eta}_{0}}}\left(1-\dfrac{\tilde{z}}{\lambda}\right)\right)\tilde{\eta}^{1}+\dfrac{{{\rm
e}^{\tilde{\eta}_{0}}}\tilde{c}_{h}^{1}}{\cosh(\phi)}\right)$ (31)

###  II.4 Solution procedure and total cathode impedance

Substituting $\tilde{j}^{1}$, Eq.(31), into Eq.(30) and solving the resulting
equation we find the amplitude of oxygen concentration perturbation along the
channel

$\tilde{c}_{h}^{1}(\tilde{z})=\dfrac{\tilde{\eta}^{1}}{B}\left(\left(\dfrac{A\lambda\tilde{J}}{B}+C\right)\left(\exp\left(\dfrac{B\tilde{z}}{\lambda\tilde{J}}\right)-1\right)-A\tilde{z}\right)$
(32)

where the independent of $\tilde{z}$ coefficients $A$, $B$ and $C$ are given
by

$A=\dfrac{\psi{{\rm e}^{\tilde{\eta}_{0}}}}{\lambda\cosh(\phi)\left(\psi+{{\rm
e}^{\tilde{\eta}_{0}}}\tanh(\phi)\right)}$ (33) $B=-{\rm
i}\tilde{\omega}\xi^{2}-\psi\tanh(\phi)-\dfrac{\lambda A}{\cosh(\phi)}$ (34)
$C=-\dfrac{\psi\left({\rm i}\tilde{\omega}+{{\rm
e}^{\tilde{\eta}_{0}}}\right)}{\cosh(\phi)\left(\psi+{{\rm
e}^{\tilde{\eta}_{0}}}\tanh(\phi)\right)}$ (35)

Setting in Eq.(32) $\tilde{\eta}^{1}=\tilde{Z}\tilde{j}^{1}$ and using the
result in Eq.(23) we get a formula for $\tilde{c}_{1}^{1}$, which linearly
depends on $\tilde{Z}$. Substituting this $\tilde{c}_{1}^{1}$ into Eq.(15), we
obtain a linear algebraic equation for local impedance $\tilde{Z}$. Solving
this equation, we find local impedance, which includes faradaic and oxygen
transport in the channel and GDL processes

$\tilde{Z}(\tilde{z})=\dfrac{B^{2}}{D_{loc}}\left(\cosh(\phi)+\dfrac{{{\rm
e}^{\tilde{\eta}_{0}}}\sinh(\phi)}{\psi}\right)$ (36)

where

$D_{loc}={{\rm
e}^{\tilde{\eta}_{0}}}\left(\lambda\tilde{J}A+BC\right)\left(\exp\left(\dfrac{B\tilde{z}}{\lambda\tilde{J}}\right)-1\right)\\\
+B^{2}\left({\rm i}\tilde{\omega}+{{\rm
e}^{\tilde{\eta}_{0}}}\left(1-\dfrac{\tilde{z}}{\lambda}\right)\right)\cosh(\phi)-{{\rm
e}^{\tilde{\eta}_{0}}}BA\tilde{z}.$ (37)

Note that $D_{loc}$ contains complex exponent
$\exp\left({B\tilde{z}}/({\lambda\tilde{J}})\right)$ leading to oscillations
of $\tilde{Z}$ along the channel coordinate $\tilde{z}$ (Ref.[11]).

The total cathode side impedance is given by

$\tilde{Z}_{tot}=\left(\int_{0}^{1}\dfrac{d\tilde{z}}{\tilde{Z}}\right)^{-1}$
(38)

Calculation of integral leads to

$\tilde{Z}_{tot}=\dfrac{\lambda B^{3}}{D_{tot}}\left(\cosh(\phi)+\dfrac{{{\rm
e}^{\tilde{\eta}_{0}}}\sinh(\phi)}{\psi}\right)$ (39)

where

$D_{tot}=\lambda^{2}\tilde{J}{{\rm
e}^{\tilde{\eta}_{0}}}\left(\lambda\tilde{J}A+BC\right)\left(\exp\left(\dfrac{B}{\lambda\tilde{J}}\right)-1\right)\\\
+\left(({\rm i}\tilde{\omega}+{{\rm e}^{\tilde{\eta}_{0}}})\lambda-{{\rm
e}^{\tilde{\eta}_{0}}}/2\right)B^{3}\cosh(\phi)\\\ -\lambda{{\rm
e}^{\tilde{\eta}_{0}}}B\left(\lambda\tilde{J}A+B(A/2+C)\right).$ (40)

The closed form of integral in Eq.(38) is a key point leading to analytical
formula for $\tilde{Z}_{tot}$. At high frequencies of the AC signal,
$\tilde{c}_{h}^{1}$ and local impedance $\tilde{Z}$ are rapidly oscillating
functions of $\tilde{z}$, which requires a lot of steps in numerical solution
of Eq.(29) and makes it difficult numerical calculation of Eq.(38). Analytical
result for integral in Eq.(38) solves the problem.

##  III Results and Discussion

Equations of the previous Section contain mean current density in the cell
$\tilde{J}$ and the total static potential loss (overpotential)
$\tilde{\eta}^{0}$. These parameters are related by the polarization curve,
which could be obtained from solution of the static version of equations (1),
(18) and (25). The solution is Tafel–like equation corrected for the finite
flow stoichiometry $\lambda$ (Ref.[19]):

$-\lambda\ln\left(1-\dfrac{1}{\lambda}\right)\tilde{J}={{\rm
e}^{\tilde{\eta}_{0}}}$ (41)

Note that Eq.(41) does not include the effect of cell ohmic resistivity. If
experimental polarization curve is available, an IR–corrected numerical
relation between $\tilde{\eta}^{0}$ and $\tilde{J}$ could be used instead of
Eq.(41). A more accurate numerical approximation for the polarization curve
could be obtained using the model[17, 16].

Eq.(41) allows us to eliminate ${{\rm e}^{\tilde{\eta}_{0}}}$ from Eq.(39).
Nyquist spectra of total cathode side impedance $Z_{tot}$ for the parameters
listed in Table 1, the mean cell current density $J=0.5$ A cm-2 and several
stoichiometries of the air flow are shown in Figure 2a. Figure 2b shows the
frequency dependence of $\operatorname{Im}\left(Z_{tot}\right)$. As can be
seen, the impedance consists of three arcs, of which the low–frequency (LF)
one strongly depends on $\lambda$ (Figure 2a). In the pioneering experiments
of Schneider et al. this arc has been associated with the oxygen transport in
channel[8, 9]. The LF arc vanishes as $\lambda\to\infty$ (Figures 2a,b).

Figure 2: (a) Nyquist spectra of the total cathode side impedance, Eq.(39),
for the indicated air flow stoichiometries $\lambda$. The cell current density
is 0.5 A cm-2. (b) Frequency dependence of imaginary part of impedance in (a).

The medium–frequency (MF) arc in Figure 2a represents oxygen transport in the
GDL. This arc is not fully seen due to masking effect of the third,
high–frequency (HF) faradaic arc (Figure 2). To emphasize the GDL arc, we
subtract the total faradaic impedance (17) from the total cathode impedance,
Eq.(39). This leads to the GDL+channel impedance $\tilde{Z}_{gdlc}$:

$\tilde{Z}_{gdlc}=\tilde{Z}_{tot}-\dfrac{1}{{\rm
i}\tilde{\omega}+\left(1-\dfrac{1}{2\lambda}\right){{\rm
e}^{\tilde{\eta}_{0}}}}$ (42)

Nyquist spectra of Eq.(42) for the same parameters exhibit two arcs; now the
left, GDL arc is fully resolved (Figure 3a). Frequency dependence of
$\operatorname{Im}\left(Z_{gdlc}\right)$ is shown in Figure 3b. Two peaks in
Figure 3b correspond to the channel and GDL arcs in Figure 3a. At low
stochiometry of the air flow, the GDL arc is located at the right “wing” of
the channel arc; the latter contributes to imaginary part of the GDL impedance
(Figure 3b). However, as $\lambda$ increases, the channel arc vanishes and
only GDL arc is left (the curves for $\lambda=1000$ in Figures 3a,b).

Figure 3: (a) Nyquist spectra of the GDL+channel impedance, Eq.(42), for the indicated air flow stoichiometries $\lambda$. The cell current density is $J=0.5$ A cm-2. KS marks the spectrum of Eq.(44). (b) Frequency dependence of imaginary part of impedance in (a).  GDL thickness $l_{b}$, cm | 0.023
---|---
Catalyst layer thickness $l_{t}$, cm |  $10\cdot 10^{-4}$ (10 $\mu$m)
ORR Tafel slope $b$, mV | 30
Double layer capacitance $C_{dl}$, F cm-3 | 20
GDL oxygen diffusivity $D_{b}$, cm2 s-1 | 0.01
Cell current density $j_{0}$, A cm-2 | 0.1
Pressure | Standard
Cell temperature $T$, K | 273 + 80
Aur flow stoichiometry $\lambda$ | 2.0
Table 1: The base–case cell parameters used in calculations.

Equation for GDL impedance in the limit of infinite air flow stoichiometry has
been derived by Kulikovsky and Shamardina[11]. In the notations of this work,
Eq.(36) of Ref.[11] is

$\tilde{Z}_{gdl}=\dfrac{\tanh\left(\mu\tilde{l}_{b}\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}\right)}{\mu\sqrt{{\rm
i}\tilde{\omega}\tilde{D}_{b}}\left(1-\tilde{J}\tilde{l}_{b}/\tilde{D}_{b}\right)\left(1+{\rm
i}\tilde{\omega}/\tilde{J}\right)}$ (43)

The frequency–independent factor $(1-\tilde{J}\tilde{l}_{b}/\tilde{D}_{b})$ in
denominator describes the growth of static resistivity upon approaching the
limiting current density $\tilde{D}_{b}/\tilde{l}_{b}$ due to oxygen transport
in the GDL. In this work, we assume that the limiting current is large and the
factor $(1-\tilde{J}\tilde{l}_{b}/\tilde{D}_{b})$ can be replaced by unity.
With this, Eq.(43) simplifies to

$\tilde{Z}_{gdl}=\dfrac{\tanh\left(\mu\tilde{l}_{b}\sqrt{{\rm
i}\tilde{\omega}/\tilde{D}_{b}}\right)}{\mu\sqrt{{\rm
i}\tilde{\omega}\tilde{D}_{b}}\left(1+{\rm i}\tilde{\omega}/\tilde{J}\right)}$
(44)

Eq.(44) can be directly obtained from Eq.(42) by passing to the limit
$\lambda\to\infty$. The spectrum of Eq.(44) (KS–spectrum) is shown in Figures
3a,b by small blue dots. As expected, the spectrum of Eq.(42) for
$\lambda=1000$ (red open circles, Figure 3a,b) is practically
indistinguishable with the spectrum of Eq.(44).

Of particular interest is the “non–Warburg” factor $(1+{\rm
i}\tilde{\omega}/\tilde{J})$ in denominator of Eq.(44). Without this factor,
Eq.(44) is equivalent to the Warburg finite–length impedance[1]. “Non–Warburg”
shape of the GDL Nyquist arc (HF part of this arc looks like an elephant’s
trunk, Figure 3a) is due to the effect of charging double layer capacitance in
the CCL[11]. In other words, the GDL impedance “feels” the double layer
capacitance of the attached CCL, since CCL provides a boundary condition for
oxygen transport through the GDL. Similar capacitive correction to the classic
Warburg impedance has been discussed by Barbero[20] in the context of
Poisson–Nernst–Plank model for the planar electrode. It is worth noting a
detailed study of the effect of non–capacitive boundary conditions on Warburg
impedance published recently by Huang[21].

The non–Warburg factor $(1+{\rm i}\tilde{\omega}/\tilde{J})$ in Eq.(44) leads
to important effects. In the dimension form, ${\rm
i}\tilde{\omega}/\tilde{J}={\rm i}\omega C_{dl}l_{t}b/J$, i.e., this factor
includes the superficial double layer capacitance $C_{dl}l_{t}$ (F cm-2) of
the CCL. Under constant volumetric DL capacitance $C_{dl}$ (F cm-3), this
results in dependence of the GDL spectrum on the catalyst layer thickness
$l_{t}$. Figure 4 shows that with typical for Pt/C–based PEMFCs CCL thickness
in the range of 10 $\mu$m to 1 $\mu$m, the effect of non-Warburg factor on the
characteristic frequency $f_{gdl}$ of GDL spectrum is small and $f_{gdl}$ is
close to the Warburg finite–length frequency $f_{W}$:

$f_{W}=\dfrac{2.54D_{b}}{2\pi l_{b}^{2}}$ (45)

However, for CCL thickness on the order of 100 $\mu$m typical for cells with
non–Pt catalysts[22], the non–Warburg factor strongly shifts the frequency
$f_{gdl}$ to lower values (Figure 4). This may lead to partial overlapping of
the GDL and channel peaks of the respective imaginary parts (cf. Figure 3b,
the curve for $\lambda=5$). Figure 4 also shows that with the decrease in CCL
thickness, the width of the GDL peak increases. Moreover, at low $l_{t}$, the
non–Warburg factor leads to formation of a distinct high–frequency feature in
the spectrum (Figure 4). The effects in Figure 4 are particularly important
for identification of DRT peaks in the high– and low–Pt PEM fuel cell spectra.
The thickness of a low–Pt CCL is typically three to four times less than the
high–Pt CCL thickness, while the volumetric $C_{dl}$ in both types of cells is
the same.

Figure 4: Frequency dependence of imaginary part of the GDL impedance,
Eq.(44), for the indicated values of CCL thickness $l_{t}$. The cell current
density is $0.1$ A cm-2. The volumetric double layer capacitance $C_{dl}$ is
assumed to be constant for all the curves.

Finally, a formula for “pure” channel impedance $\tilde{Z}_{chan}$ can be
obtained from Eqs.(42), (39) by setting GDL thickness to zero:

$\tilde{Z}_{chan}=-\dfrac{4\lambda{{\rm
e}^{\tilde{\eta}_{0}}}N_{c}}{\bigl{(}2\lambda{\rm
i}\tilde{\omega}+(2\lambda-1){{\rm e}^{\tilde{\eta}_{0}}}\bigr{)}D_{c}}$ (46)

where $N_{c}$ and $D_{c}$ are given in Appendix. The characteristic frequency
$f_{chan}$ of channel impedance depends almost linearly of the cell current
density and on the air flow stoichiometry (Figure 5). It is interesting to
note that the slope of $f_{chan}(J)$ increases with the growth of
stoichiometry $\lambda$ (Figure 5a), although the magnitude of
$\tilde{Z}_{chan}$ rapidly decreases with $\lambda$.

Setting in Eq.(46) $\tilde{\omega}=0$, we find the static resistivity due to
channel; in the dimension form this resistivity reads

$R_{chan}=\dfrac{b}{J}\left(\dfrac{2}{(2\lambda-1)\ln(1-1/\lambda)}-\lambda\ln\left(1-\dfrac{1}{\lambda}\right)\right)$
(47)

Eq.(47) is independent of parameter $\xi$, Eq.(28); however, $\xi$ affects the
shape of impedance (46). As $\xi$ contains the product $C_{dl}l_{t}$, the
shape of $\tilde{Z}_{chan}$ appears to be dependent of the CCL thickness.
Though for typical PEMFC parameters this dependence is weak, we should stress
that impedance of oxygen transport layers (GDL and channel) depend on the
double layer capacitance in the electrode.

Figure 5: (a) The characteristic frequency of channel impedance $f_{chan}$ vs
cell current density for the indicated air flow stoichiometries. (b) The
frequency $f_{chan}$ vs air flow stoichiometry for the indicated cell current
densities.

##  IV Conclusions

An analytical model for impedance of oxygen transport in the GDL and channel
of a PEM fuel cell cathode is developed. The model is based on 1d+1d–coupled
oxygen mass transport equations along the channel and through the GDL. The
oxygen and proton transport through the cathode catalyst layer are assumed to
be fast. Analytical solution for the GDL+channel impedance $Z_{gdlc}$ is
obtained. The Nyquist spectrum of $Z_{gdlc}$ consists of two arcs, of which
the left arcs corresponds to oxygen transport in the GDL and the right (LF)
arc is due to oxygen transport in the channel. The characteristic frequency of
the channel arc depends linearly on the oxygen stoichiometry. The shape of the
GDL arc differs from the Warburg finite–length impedance due to effect of
double layer charging in the catalyst layer attached to the GDL. For typical
PEMFC parameters, the characteristic frequency of the GDL arc $f_{gdl}$ is
close to the Warburg finite–length frequency. However, for larger CCL
thickness the non–Warburg factor strongly shifts $f_{gdl}$ to lower values. At
low catalyst layer thickness, the high–frequency feature in the spectrum of
imaginary part of the GDL impedance forms. This effect is particularly
important for identification of DRT peaks in low–Pt cell spectra, as low–Pt
cells typically differ from the high–Pt cells by the catalyst layer thickness
only.

##  Appendix A Equations for the factors $N_{c}$ and $D_{c}$ in Eq.(46)

$N_{c}=\lambda^{2}\tilde{J}\left({{\rm
e}^{2\tilde{\eta}_{0}}}+\left(\tilde{J}+{\rm
i}\tilde{\omega}(1+\xi^{2})\right){{\rm
e}^{\tilde{\eta}_{0}}}-\xi^{2}\tilde{\omega}^{2}\right)\\\
\times\exp\left(\dfrac{-{{\rm e}^{\tilde{\eta}_{0}}}-{\rm
i}\tilde{\omega}\xi^{2}}{\lambda\tilde{J}}\right)+\lambda\left({{\rm
e}^{\tilde{\eta}_{0}}}-\lambda\tilde{J}+{\rm i}\tilde{\omega}\xi^{2}\right)\\\
\times\left({{\rm e}^{2\tilde{\eta}_{0}}}+\left(\tilde{J}+{\rm
i}\tilde{\omega}(\xi^{2}+1)\right){{\rm
e}^{\tilde{\eta}_{0}}}-\xi^{2}\tilde{\omega}^{2}\right)\\\ -\left({{\rm
e}^{\tilde{\eta}_{0}}}+{\rm i}\tilde{\omega}\xi^{2}\right)^{2}{{\rm
e}^{\tilde{\eta}_{0}}}/2$ (48)

$D_{c}=2\lambda^{2}\tilde{J}\left({{\rm
e}^{2\tilde{\eta}_{0}}}+\left(\tilde{J}+{\rm
i}\tilde{\omega}(1+\xi^{2})\right){{\rm
e}^{\tilde{\eta}_{0}}}-\xi^{2}\tilde{\omega}^{2}\right)\\\
\times\exp\left(\dfrac{-{{\rm e}^{\tilde{\eta}_{0}}}-{\rm
i}\tilde{\omega}\xi^{2}}{\lambda\tilde{J}}\right){{\rm
e}^{\tilde{\eta}_{0}}}\\\ -2\lambda\xi^{6}\tilde{\omega}^{4}+{\rm
i}\left(2\lambda\xi^{2}-\xi^{2}+4\lambda\right)\xi^{4}{{\rm
e}^{\tilde{\eta}_{0}}}\,\tilde{\omega}^{3}\\\
+2\xi^{2}\left(\left(2\lambda\xi^{2}-\xi^{2}+\lambda\right){{\rm
e}^{\tilde{\eta}_{0}}}+\lambda^{2}\tilde{J}\right){{\rm
e}^{\tilde{\eta}_{0}}}\tilde{\omega}^{2}\\\ -{\rm
i}\left(\left(2\lambda-1\right)\xi^{2}{{\rm
e}^{\tilde{\eta}_{0}}}+2\lambda\tilde{J}\left(\lambda\xi^{2}-\xi^{2}+\lambda\right)\right){{\rm
e}^{2\tilde{\eta}_{0}}}\tilde{\omega}\\\
-2\lambda\tilde{J}\left(\left(\lambda-1\right){{\rm
e}^{\tilde{\eta}_{0}}}+\lambda\tilde{J}\right){{\rm e}^{2\tilde{\eta}_{0}}}$
(49)

## References

  * Lasia [2014] A. Lasia.  _Electrochemical Impedance Spectroscopy and its Applications_.  Springer, New York, 2014.
  * Tang et al. [2020] Zh. Tang, Q.-A. Huang, Y.-J. Wang, F. Zhang, W. Li, A. Li, L. Zhang, and JJ Zhang.  Recent progress in the use of electrochemical impedance spectroscopy for the measurement, monitoring, diagnosis and optimization of proton exchange membrane fuel cell performance.  _J. Power Sources_ , 468:228361, 2020.  doi: 10.1016/j.jpowsour.2020.228361.
  * Macdonald [2006] D. Macdonald.  Reflections on the history of electrochemical impedance spectroscopy.  _Electrochim. Acta_ , 51:1376–1388, 2006.  doi: 10.1016/j.electacta.2005.02.107.
  * Huang et al. [2020] J. Huang, Y. Gao, J. Luo, S. Wang, C. Li, S. Chen, and J. Zhang.  Editors’ choice–review–impedance response of porous electrodes: Theoretical framework, physical models and applications.  _J. Electrochem. Soc._ , 167:166503, 2020.  doi: 10.1149/1945-7111/abc655.
  * Fuoss and Kirkwood [1941] R.M. Fuoss and J.G. Kirkwood.  Electrical properties of solids. viii. Dipole moments in polyvinyl chloride-diphenyl systems.  _J. Am. Chem. Soc._ , 63:385–394, 1941.  doi: 10.1021/ja01847a013.
  * Schichlein et al. [2002] H. Schichlein, A. C. Müller, M. Voigts, A. Krügel, and E. Ivers-Tiffée.  Deconvolution of electrochemical impedance spectra for the identification of electrode reaction mechanisms in solid oxide fuel cells.  _J. Appl. Electrochem._ , 32:875–882, 2002.  doi: 10.1023/A:1020599525160.
  * Effendy et al. [2020] S. Effendy, J. Song, and M. Z. Bazant.  Analysis, design, and generalization of electrochemical impedance spectroscopy (EIS) inversion algorithms.  _J. Electrochem. Soc._ , 167:106508, 2020.  doi: 10.1149/1945-7111/ab9c82.
  * Schneider et al. [2007a] I. A. Schneider, S. A. Freunberger, D. Kramer, A. Wokaun, and G. G. Scherer.  Oscillations in gas channels. Part I. The forgotten player in impedance spectroscopy in PEFCs.  _J. Electrochem. Soc._ , 154:B383–B388, 2007a.  doi: 10.1149/1.2435706.
  * Schneider et al. [2007b] I. A. Schneider, D. Kramer, A. Wokaun, and G. G. Scherer.  Oscillations in gas channels. II. Unraveling the characteristics of the low–frequency loop in air–fed PEFC impedance spectra.  _J. Electrochem. Soc._ , 154:B770–B3782, 2007b.  doi: 10.1149/1.2742291.
  * Kulikovsky [2012] A. A. Kulikovsky.  A model for local impedance of the cathode side of PEM fuel cell with segmented electrodes.  _J. Electrochem. Soc._ , 159:F294–F300, 2012.  doi: 10.1149/2.066207jes.
  * Kulikovsky and Shamardina [2015] A. Kulikovsky and O. Shamardina.  A model for PEM fuel cell impedance: Oxygen flow in the channel triggers spatial and frequency oscillations of the local impedance.  _J. Electrochem. Soc._ , 162:F1068–F1077, 2015.  doi: 10.1149/2.0911509jes.
  * Maranzana et al. [2012] G. Maranzana, J. Mainka, O. Lottin, J. Dillet, A. Lamibrac, A. Thomas, and S. Didierjean.  A proton exchange membrane fuel cell impedance model taking into account convection along the air channel: On the bias between the low frequency limit of the impedance and the slope of the polarization curve.  _Electrochim. Acta_ , 83:13–27, 2012.
  * Chevalier et al. [2016] S. Chevalier, C. Josset, A. Bazylak, and B. Auvity.  Measurements of air velocities in polymer electrolyte membrane fuel cell channels using electrochemical impedance spectroscopy.  _J. Electrochem. Soc._ , 163:F816–F823, 2016.  doi: 10.1149/2.0481608jes.
  * Kulikovsky [2019] A. Kulikovsky.  Analytical impedance of oxygen transport in a PEM fuel cell channel.  _J. Electrochem. Soc._ , 166:F306–F311, 2019.  doi: 10.1149/2.0951904jes.
  * Cruz-Manzo and Greenwood [2021] S. Cruz-Manzo and P. Greenwood.  Analytical Warburg impedance model for EIS analysis of the gas diffusion layer with oxygen depletion in the air channel of a PEMFC.  _J. Electrochem. Soc._ , 168:074502, 2021.  doi: 10.1149/1945-7111/ac1031.
  * Reshetenko and Kulikovsky [2019] T. Reshetenko and A. Kulikovsky.  On the distribution of local current density along the PEM fuel cell cathode channel.  _Electrochem. Comm._ , 101:35–38, 2019.  doi: 10.1016/j.elecom.2019.02.005.
  * Chevalier et al. [2018] S. Chevalier, C. Josset, and B. Auvity.  Analytical solutions and dimensional analysis of pseudo 2D current density distribution model in PEM fuel cells.  _Renew. Energy_ , 125:738–746, 2018.  doi: 10.1016/j.renene.2018.02.120.
  * Kulikovsky [2003] A. A. Kulikovsky.  The voltage current curve of a PEM fuel cell: Analytical and numerical modeling.  In M. Laudon and B. Romanowicz, editors, _Techn. Proc. of the 2003 Nanotechnology Conf. and Trade Show_ , volume 3, pages 467–470, San–Francisco, CA, USA, February 23–27 2003. Computational Publications.
  * Kulikovsky [2004] A. A. Kulikovsky.  The effect of stoichiometric ratio $\lambda$ on the performance of a polymer electrolyte fuel cell.  _Electrochim. Acta_ , 49(4):617–625, 2004.
  * Barbero [2016] G. Barbero.  Warburg’s impedance revisited.  _Phys. Chem. Chem. Phys._ , 18:29537–29542, 2016.  doi: 10.1039/c6cp05049b.
  * Huang [2018] J. Huang.  Diffusion impedance of electroactive materials, electrolytic solutions and porous electrodes: Warburg impedance and beyond.  _Electrochim. Acta_ , 281:170–188, 2018.  doi: 10.1016/j.electacta.2018.05.136.
  * Reshetenko et al. [2020] T. Reshetenko, G. Randolf, M. Odgaard, B. Zulevi, A. Serov, and A. Kulikovsky.  The effect of proton conductivity of Fe-N-C–based cathode on PEM fuel cell performance.  _J. Electrochem. Soc._ , 167:084501, 2020.  doi: 10.1149/1945-7111/ab8825.

Nomenclature

$\tilde{}$ | Marks dimensionless variables
---|---
$b$ | ORR Tafel slope, V
$C_{dl}$ |  Double layer volumetric capacitance, F cm-3
$c_{1}$ | Oxygen molar concentration
|  at the CCL/GDL interface, mol cm-3
$c_{b}$ |  Oxygen molar concentration in the GDL, mol cm-3
$c_{h}$ |  Oxygen molar concentration in the channel, mol cm-3
$c_{h}^{in}$ |  Reference (inlet) oxygen concentration, mol cm-3
$D_{b}$ |  Oxygen diffusion coefficient in the GDL, cm2 s-1
$F$ |  Faraday constant, C mol-1
$f$ | Characteristic frequency, Hz
$i_{*}$ |  ORR volumetric exchange current density, A cm-3
${\rm i}$ | Imaginary unit
$j$ |  Local proton current density along the CCL, A cm-2
$j_{\lim}$ | Limiting current density
|  due to oxygen transport in the GDL, Eq.(12) A cm-2
$j_{0}$ |  Local cell current density, A cm-2
$l_{b}$ | GDL thickness, cm
$l_{t}$ | CCL thickness, cm
$t$ | Time, s
$t_{*}$ |  Characteristic time, s, Eq.(4)
$x$ | Coordinate through the cell, cm
$Z$ |  Local impedance, Ohm cm2
$Z_{gdlc}$ |  GDL+channel impedance, Ohm cm2
$Z_{tot}$ | Total cathode side impedance, including
|  faradaic one, Ohm cm2
$z$ | Coordinate along the cathode channel, cm

Subscripts:

$0$ | Membrane/CCL interface
---|---
$1$ | CCL/GDL interface
$b$ | In the GDL
$gdl$ | GDL
$gdlc$ | GDL+channel
$f$ | faradaic
$h$ | Air channel
$W$ | Warburg

Superscripts:

$0$ | Steady–state value
---|---
$1$ | Small–amplitude perturbation

Greek:

$\eta$ | ORR overpotential, positive by convention, V
---|---
$\lambda$ |  Air flow stoichiometry, Eq.(28)
$\mu$ |  Dimensionless parameter, Eq.(20)
$\xi$ |  Dimensionless parameter, Eq.(28)
$\phi$ |  Dimensionless parameter. Eq.(24)
$\psi$ |  Dimensionless parameter. Eq.(24)
$\omega$ |  Angular frequency of the AC signal, s-1


# Evaluating Prompts Across Multiple Choice Tasks In a Zero-Shot Setting

Gabriel Orlanski
<EMAIL_ADDRESS>

###### Abstract

Large language models have shown that impressive zero-shot performance can be
achieved through natural language prompts Radford et al. (2019); Brown et al.
(2020); Sanh et al. (2021). Creating an effective prompt, however, requires
significant trial and error. That prompts the question: how do the qualities
of a prompt effects its performance? To this end, we collect and standardize
prompts from a diverse range of tasks for use with tasks they were not
designed for. We then evaluate these prompts across fixed multiple choice
datasets for a quantitative analysis of how certain attributes of a prompt
affect performance. We find that including the choices and using prompts not
used during pre-training provide significant improvements. All experiments and
code can be found https://github.com/gabeorlanski/zero-shot-cross-task.

##  1 Introduction

Recent work has shown that using a natural language (NL) prompt with pre-
trained language models (LM) significantly improves performance in few-shot
and zero-shot settings Brown et al. (2020); Schick and Schütze (2021b), to the
point where they can be worth 100s of data points Scao and Rush (2021).
Further, T5 Raffel et al. (2020) showed that simple prompts and reformulating
NLP tasks as text-to-text performs well on a wide range of tasks. Recent
models such as FLAN Wei et al. (2021b) and T0 Sanh et al. (2021) demonstrate
that multi-task training a large LM with prompts results in improved zero-shot
performance on a wide range of tasks. However, manually designing a prompt is
a non-trivial task due to the trial-and-error nature of the task Jiang et al.
(2020); Shin et al. (2021). Some works focus on “prompt programming”– best
practices for designing prompts Reynolds and McDonell (2021); Liu et al.
(2021a). While others have looked towards continuous “soft-prompts”– random
vectors added to the input sequence, which are then fine-tuned Zhong et al.
(2021b); Qin and Eisner (2021). However, recent work has revealed that these
prompts are susceptible to minor perturbations Mishra et al. (2021).

Motivated by this, we aim to conduct a quantitative analysis of what affects a
prompt’s performance. We evaluate T0-3B Sanh et al. (2021) on generalized
prompts from a wide range of tasks with eight datasets to provide a
quantitative analysis of how the qualitative aspects of a prompt effect its
performance.

We collected 95 prompts across 20 tasks and evaluated each on eight datasets.
We find that using a prompt performs better for every evaluation task than not
using a prompt. Further, we find that the set of prompts with the highest
performance is not the ones designed for the specific task for seven of the
eight datasets. In our ablations, we find that adding a small amount of task-
specific NL to the generalized prompt increases performance by a median of
4.65%. Finally, we find that prompts with the choices present outperform those
that do not and that presenting the options as multiple distinct choices
further improves the results. Additionally, the longer a prompt is, the worse
it performs.

##  2 Methodology

Figure 1: Overview of the approach. For a given example from a dataset
(leftmost box) we use a prompt from a different task to perform zero shot
predictions with T0-3B Sanh et al. (2021). The bolded text in the example
represents its choices.

Our overall approach is detailed in Figure 1. Given a downstream multiple-
choice task
$\mathcal{T}_{\textrm{o}}=\\{(\mathbf{x}_{1},\mathbf{y}_{1}),\ldots,(\mathbf{x}_{n},\mathbf{y}_{n})\\}$,
evaluate how well T0 Sanh et al. (2021) performs when using a prompt
$\mathcal{P}_{\textrm{d}}$ designed for a different task
$\mathcal{T}_{\textrm{d}}$.

###  2.1 Fixed Choice Tasks

For the purposes of this paper, we limit the scope of the tasks we look at to
be only multiple choice tasks whose choices are constant across all examples.
Thus, for
$\mathcal{T}_{\textrm{o}}=\\{(\mathbf{x}_{1},\mathbf{y}_{1}),\ldots,(\mathbf{x}_{n},\mathbf{y}_{n})\\}$,
every $\mathbf{y}_{i}\in\mathcal{C}_{\textrm{o}}$ where
$\mathcal{C}_{\textrm{o}}=\\{\mathcal{C}_{1},\ldots,\mathcal{C}_{c}\\}$ with
lengths $\ell=\\{\ell_{1},\ldots,\ell_{c}\\}$. To make a prediction for the
example point $\mathbf{x}_{i}$, we follow Holtzman et al. (2021) and use rank
scoring: taking the choice with the highest probability as defined by

$\operatorname*{argmax}_{\mathcal{C}_{j}\in\mathcal{C}_{\textrm{o}}}\prod_{k=1}^{\ell_{j}}P(\mathcal{C}_{j}^{k}|\mathbf{x}_{i},\mathcal{C}_{j}^{1}\ldots\mathcal{C}_{j}^{k-1})$
(1)

However, the lengths in $\ell$ are not guaranteed to be the same and thus
Equation 1 will unintentionally penalize longer choices Brown et al. (2020);
Holtzman et al. (2021). We thus follow prior work and take the choice with the
highest Average Log-Likelihood

$\operatorname*{argmax}_{\mathcal{C}_{j}\in\mathcal{C}_{\textrm{o}}}\frac{\sum_{k=1}^{\ell_{j}}\log[P(\mathcal{C}_{j}^{k}|\mathbf{x}_{i},\mathcal{C}_{j}^{1}\ldots\mathcal{C}_{j}^{k-1})]}{\ell_{j}}$
(2)

###  2.2 Generalized Prompts

Task |  Prompt
---|---
Original |  Sentence A: {{sentence1}} Sentence B: {{sentence2}} "{{word}}" has a similar meaning in sentences A and B. True or False?
Generalized |  Sentence A: {{premise}} Sentence B: {{hypothesis}} "{{domain}}" has a similar meaning in sentences A and B. {{ choice_string }}?
Example |  Sentence A: What is 2+2? Sentence B: Choices are: $\infty$, -10, fish, 4, $\sqrt{2}$ "math problem" has a similar meaning in sentences A and B. "A", "B", "C", "D" or "E"?
Table 1: Sample prompt from WordsInContext Task Pilehvar and osé Camacho-
Collados (2018) and its generalized form. Each {{ }} represents an input from
the dataset. The colors are the alignment of inputs.

We use the PromptSource111https://github.com/bigscience-workshop/promptsource
framework proposed by Sanh et al. (2021) for templates as it provides a
standardized format for managing prompts. We standardize the input fields and
answer formats across all tasks such that they fell into three general
categories: Classification, Entailment, and Question Answering (QA). Table 1
displays an example of what the generalization would look like for an
Entailment prompt. To use a prompt with a task that does not have the same
number of inputs, we add additional task specific NL to better align the
inputs. To use the example prompt from Table 1 with a sentiment classification
task, we would map the input text from the task to the premise  field and pass
“what is the sentiment” as the hypothesis. In prompts where answers choices
are present, we replace them with an additional input field for the choice
string(i.e. “yes, no, or maybe”) to hold how the choices are presented
constant across all prompts. A detailed breakdown of the tasks used for the
generalized prompts can be found in Table 3.

##  3 Experimental Setup

###  3.1 Datasets

For all datasets, we use the most recent version on HuggingFace Wolf et al.
(2020). Every evaluation is done using the validation split as per Sanh et al.
(2021). For the evaluation datasets, we again follow Sanh et al. (2021) and
use: Adversarial NLI (ANLI) Nie et al. (2020), CommitmentBank (CB) De Marneff
et al. (2019), Recognizing Textural Entailment (RTE) Dagan et al. (2005); Bar-
Haim et al. (2006); Giampiccolo et al. (2007); Bentivogli et al. (2009), and
Words In Context (WiC) Pilehvar and osé Camacho-Collados (2018).
Two additional tasks are used to evaluate T0’s performance on complex tasks in
unseen domains: Algebra Question Answering (AQuA) Ling et al. (2017) and
CraigslistBargains He et al. (2018). Descriptions for these two tasks can be
found in Appendix B.
For the generalized prompts, we collect 86 prompts across 19 distinct tasks.
In addition to these, we also include 12 prompts with no additional NL for
each of the three categories for 4 different ablations. More details can be
found in Appendix A.

###  3.2 Model and Metrics

We evaluate the performance of the 3B parameter T0, and T5 Raffel et al.
(2020) models with the HuggingFace implementation Wolf et al. (2020) as we
were limited to a single RTX Titan 24GB card. Following prior works Sanh et
al. (2021); Brown et al. (2020); Holtzman et al. (2021), we report the
accuracy and F1 scores on each of the eight datasets.

As each task will have different mean metrics, we cannot compare the raw
accuracy and F1 scores across tasks. For a given prompt, we calculate the
median accuracy and F1 ranks compared to the 95 other prompts for all
evaluation tasks. As the rank is ascending, lower values for median accuracy
rank (MAR) and median F1 rank (MFR) indicate a better performing prompt.

##  4 Results

###  4.1 Baselines On New Tasks

As we evaluate T0-3B on two new tasks, we first want to gauge how the model
performs as shown in both Figure 2 and Table 4. We follow Sanh et al. (2021)
in using rank scoring without length normalization as defined by Equation 1
and find that for both AQuA and CraigslistBargains, the base T5 model performs
better than T0. However, T0’s unweighted multi-class F1 is better than that of
T5 on both tasks, indicating that T5 achieves a higher score due to only
predicting a subset of the choices. Figure 3 provides further evidence for
this hypothesis. In both AQuA and CraigslistBargains, T0 more evenly
distributes its predictions across the possible choices where as T5 heavily
favors a subset of the choices. Thus, the disparity in the accuracy is likely
a result of class imbalance in the evaluation datasets in which T5 is ’lucky’
in heavily predicting the class that was more populous. We consider this to be
’luck’ as T5 outperforming T0 only occurs in only two of the datasets we
examined.

###  4.2 Generalized Prompts

ANLI R1 ANLI R2 ANLI R3 AQuA CB Craigslist RTE WiC Rank No Prompt 34.15 33.35
33.42 26.77 24.11 16.83 59.57 50.24 46.25 Unseen Prompts ANLI 37.60 34.70
34.08 25.95 32.14 21.44 64.62 50.16 24.50 AQuA 36.10 33.40 35.42 17.32 33.93
23.45 71.12 51.57 18.25 COPA 39.30 34.40 34.00 20.47 26.79 16.58 69.31 50.63
21.25 Craigslist 31.40 31.30 32.83 25.79 8.04 26.72 49.82 50.16 71.25 MathQA
37.30 33.50 34.25 19.29 26.79 16.25 73.29 51.10 24.50 RTE 36.10 33.20 33.58
22.05 23.21 20.27 61.37 50.47 43.25 SemEval2010 33.10 32.00 32.58 27.56 14.29
25.63 55.23 50.47 66.50 WiC 31.75 33.45 32.33 26.57 13.39 18.01 55.05 50.47
64.25 Training Prompts AppReviews 34.20 33.10 33.62 27.17 19.64 33.17 61.55
50.31 33.50 IMDB 33.00 32.20 33.08 26.38 12.50 14.57 55.23 50.16 71.25 Yelp
33.25 32.35 33.04 26.77 12.50 24.29 62.27 51.57 41.75

Table 2: Median Accuracy when using modified prompts for cross task zero-shot
evaluation. Bolded entries are prompts for the original task.  Green Cells and
Red Cells are the best and worst performing tasks for a column respectively.
Rank is the median rank of prompts from this task out of 95 total prompts.
ANLI and CB both use the same prompts for their original task prompts per
PromptSource. Some tasks are left out for clarity. The full table can be found
in Table 6.

We report the results of the cross-task evaluation in Table 2. We find that
the only task in which the original222By original we are referring to the
prompts specifically designed for a task. prompt performs best is ANLI R2 with
an accuracy of 34.70. Conversely, the worst performing prompts for the AQuA
task were its original prompts with an accuracy of 17.32. Furthermore, we find
that there is no task out of the eight used for evaluation where not using a
prompt has the best performance.

We report how the added NL discussed in subsection 2.2 effects the zero-shot
performance in Table 5. Across the eight evaluation tasks, adding some task
specific text leads to an average increase of $4.65\%$ in the accuracy and a
$2.34\%$ increase to the unweighted multiclass F1. However, there was also a
decrease of $4.21\%$ and $13.90\%$ to minimum accuracy and unweighted
multiclass F1 respectively. This implies that the added extra text helps to
better amplify the negative and positive elements of a prompt.

###  4.3 Qualitative Analysis of Prompts

Table 5 displays the rank statistics across multiple ablations and Table 7
displays the correlations of a prompt’s qualities with their rank. In prompts
which have choices, the MAR is 33.12 compared to 52.25 when the choices are
left out. However, the range as indicated by the Q1 and Q3 MARs is
significantly larger when the choices are included, implying that adding the
choice string causes high variance.

We also find that when the choices are presented as multiple distinct
choices333Presented with a clear delimiters/separation. For example A) yes B)
no C) maybe the MAR is further improves to 22.25. In comparison, the prompts
with choices that are not in this format have a MAR of 36.00. These results
provide further evidence to the findings from Wei et al. (2021a) that clearly
distinguishing the options in a prompt improves performance.

Next, we find that the median rank of prompts used in training is 50.25
compared to 42.00 of the unseen prompts. The F1 scores display a similar
pattern with training prompts having a MFR of 55.75 while unseen prompts have
a median of 36.50. Although this implies that prompts that share tokens with
those used for training will perform worse, we find that there is no
significant correlation between the number of tokens a prompt $\mathcal{P}$
shares with those used in training and its rank.

Finally, we find that longer prompts have a slightly negative impact on the
performance of a prompt. Figure 4 shows that, with the 95 prompts used across
eight tasks, the best performing prompts are those whose length is in the
range $[14,21)$ as their MAR is 28.50 and MFR is 36.50. The Q1 values are
18.00 and 15.38, respectively. In comparison, we find that prompts with
lengths with lengths $\geq 25$ have a median MAR of 50.25 and MFR of 72.00,
indicating a negative impact on performance. Surprisingly, we find that
prompts whose lengths are $<14$ have a median MAR of 47.75 and MFR of 49.00.
While this is a negative impact on performance, it is not as large as that in
longer prompts.

##  5 Related Works

Pre-trained Language Models In the past few years, large pre-trained models
have rose to prominence due to their strong performance on a wide range of NLP
tasks Radford et al. (2019); Brown et al. (2020); Lewis et al. (2020). In
particular, T5 Raffel et al. (2020) explored transferred learning for large
LMs by transforming all NLP problems to a text-to-text format. Beyond natural
language, these models have also excelled at tasks in other modalities such as
code generation Austin et al. (2021); Chen et al. (2021); Orlanski and Gittens
(2021), in-context learning Min et al. (2021); Zhong et al. (2021a), and
semantic parsing Rongali et al. (2020). One aspect of large LMs is that they
perform well in zero-shot settings Radford et al. (2019); Brown et al. (2020);
Vu et al. (2020).

NL Prompting A drawback of these large LMs is that their size makes it costly
to fine-tune them. This lead to the rise of the “pre-train, prompt, and
predict” paradigm in which a downstream tasks are modified to resemble those
used in training through the use of NL prompts Liu et al. (2021b). These
prompts have improved few-shot and zero-shot performance across a vast number
of models and tasks Brown et al. (2020); Schick and Schütze (2021b, a); Mishra
et al. (2021); Scao and Rush (2021); Shin et al. (2020). Recent models such as
FLAN Wei et al. (2021b) and T0 Sanh et al. (2021) have shown that even better
zero-shot performance can be achieved through using a multi-task pre-training
objectives with a diverse set of prompts.

##  6 Conclusion

In this paper, we examined T0’s performance on a range of fixed multiple-
choice tasks. We find that T0 does worse than T5 on two unseen complex tasks.
Next, we evaluated how the performance of a prompt transfers between tasks.
Our results show that using a prompt performs consistently better than not
using any prompt. We conclude with a quantitative analysis of what aspects of
a prompt affect its performance. We find that prompts with the choices in them
are 66.82% better than those that leave the choices out. Next, we find that
prompts not used in pre-training are 19.64% better than those that were.
Finally, we find that prompts whose length are between 14 and 24 perform
better than both longer and shorter prompts. Further work should examine
prompt transfer with larger models while also expanding the number of prompts
and tasks used.

## References

  * Amini et al. (2019) Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019.  MathQA: Towards interpretable math word problem solving with operation-based formalisms.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2357–2367, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Austin et al. (2021) Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021.  Program synthesis with large language models.  _ArXiv_ , abs/2108.07732.
  * Bar-Haim et al. (2006) Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006.  The second pascal recognising textual entailment challenge.  In _Proceedings of the second PASCAL challenges workshop on recognising textual entailment_ , volume 6, pages 6–4. Venice.
  * Bartolo et al. (2020) Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, and Pontus Stenetorp. 2020.  Beat the ai: Investigating adversarial human annotation for reading comprehension.  _Transactions of the Association for Computational Linguistics_ , 8:662–678.
  * Bentivogli et al. (2009) Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. 2009.  The fifth pascal recognizing textual entailment challenge.  In _TAC_.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, David W. Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William H. Guss, Alex Nichol, Igor Babuschkin, S. Arun Balaji, Shantanu Jain, Andrew Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew M. Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021.  Evaluating large language models trained on code.  _ArXiv_ , abs/2107.03374.
  * Dagan et al. (2005) Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005.  The pascal recognising textual entailment challenge.  In _Machine Learning Challenges Workshop_ , pages 177–190. Springer.
  * De Marneff et al. (2019) Marie-Catherine De Marneff, Mandy Simons, and Judith Tonhauser. 2019.  The commitmentbank: Investigating projection in naturally occurring discourse.  _proceedings of Sinn und Bedeutung 23_.
  * Giampiccolo et al. (2007) Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007.  The third pascal recognizing textual entailment challenge.  In _Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing_ , pages 1–9. Association for Computational Linguistics.
  * Grano et al. (2017) Giovanni Grano, Andrea Di Sorbo, Francesco Mercaldo, Corrado A. Visaggio, Gerardo Canfora, and Sebastiano Panichella. 2017.  Android apps and user feedback: A dataset for software evolution and quality improvement.  In _Proceedings of the 2nd ACM SIGSOFT International Workshop on App Market Analytics_ , WAMA 2017, page 8–11, New York, NY, USA. Association for Computing Machinery.
  * He et al. (2018) He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018.  Decoupling strategy and generation in negotiation dialogues.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2333–2343, Brussels, Belgium. Association for Computational Linguistics.
  * Hendrickx et al. (2010) Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid ’O S’eaghdha, Sebastian Pad’o, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2010.  SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals.  In _Proceedings of the 5th International Workshop on Semantic Evaluation_ , pages 33–38, Uppsala, Sweden. Association for Computational Linguistics.
  * Holtzman et al. (2021) Ari Holtzman, Peter West, Vered Schwartz, Yejin Choi, and Luke Zettlemoyer. 2021\.  Surface form competition: Why the highest probability answer isn’t always right.  In _EMNLP_.
  * Jiang et al. (2020) Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020.  How can we know what language models know?
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lin et al. (2020a) Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. 2020a.  Birds have four legs?! numersense: Probing numerical commonsense knowledge of pre-trained language models.  In _Proceedings of EMNLP_.  To appear.
  * Lin et al. (2020b) Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020b.  CommonGen: A constrained text generation challenge for generative commonsense reasoning.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1823–1840, Online. Association for Computational Linguistics.
  * Ling et al. (2017) Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017.  Program induction by rationale generation: Learning to solve and explain algebraic word problems.  _ACL_.
  * Liu et al. (2021a) Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021a.  What makes good in-context examples for gpt-$3$?
  * Liu et al. (2021b) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021b.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _ArXiv_ , abs/2107.13586.
  * Lu et al. (2020) Yao Lu, Yue Dong, and Laurent Charlin. 2020.  Multi-xscience: A large-scale dataset for extreme multi-document summarization of scientific articles.  _arXiv preprint arXiv:2010.14235_.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * Malo et al. (2014) P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala. 2014.  Good debt or bad debt: Detecting semantic orientations in economic texts.  _Journal of the Association for Information Science and Technology_ , 65.
  * Min et al. (2021) Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2021.  Metaicl: Learning to learn in context.  _ArXiv_ , abs/2110.15943.
  * Mishra et al. (2021) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hanna Hajishirzi. 2021.  Cross-task generalization via natural language crowdsourcing instructions.
  * Narayan et al. (2018) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  _ArXiv_ , abs/1808.08745.
  * Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020.  Adversarial nli: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
  * Orlanski and Gittens (2021) Gabriel Orlanski and Alex Gittens. 2021.  Reading StackOverflow encourages cheating: Adding question text improves extractive code generation.  In _Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021)_ , pages 65–76, Online. Association for Computational Linguistics.
  * Paperno et al. (2016) Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez. 2016.  The LAMBADA dataset: Word prediction requiring a broad discourse context.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1525–1534, Berlin, Germany. Association for Computational Linguistics.
  * Pilehvar and osé Camacho-Collados (2018) Mohammad Taher Pilehvar and osé Camacho-Collados. 2018.  Wic: 10, 000 example pairs for evaluating context-sensitive representations.  _CoRR_ , abs/1808.09121.
  * Qin and Eisner (2021) Guanghui Qin and Jason Eisner. 2021.  Learning how to ask: Querying lms with mixtures of soft prompts.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Reynolds and McDonell (2021) Laria Reynolds and Kyle McDonell. 2021.  Prompt programming for large language models: Beyond the few-shot paradigm.
  * Roemmele et al. (2011) Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. 2011.  Choice of plausible alternatives: An evaluation of commonsense causal reasoning.  In _2011 AAAI Spring Symposium Series_.
  * Rongali et al. (2020) Subendhu Rongali, Luca Soldaini, Emilio Monti, and Wael Hamza. 2020.  Don’t parse, generate! a sequence to sequence architecture for task-oriented semantic parsing.  _Proceedings of The Web Conference 2020_.
  * Sanh et al. (2021) Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. 2021.  Multitask prompted training enables zero-shot task generalization.
  * Scao and Rush (2021) Teven Le Scao and Alexander M. Rush. 2021.  How many data points is a prompt worth?  In _NAACL_.
  * Schick and Schütze (2021a) Timo Schick and Hinrich Schütze. 2021a.  Exploiting cloze-questions for few-shot text classification and natural language inference.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 255–269, Online. Association for Computational Linguistics.
  * Schick and Schütze (2021b) Timo Schick and Hinrich Schütze. 2021b.  It’s not just size that matters: Small language models are also few-shot learners.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2339–2352, Online. Association for Computational Linguistics.
  * Shin et al. (2021) Richard Shin, Christopher H. Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme. 2021.  Constrained language models yield few-shot semantic parsers.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020.  AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4222–4235, Online. Association for Computational Linguistics.
  * Vu et al. (2020) Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, and Mohit Iyyer. 2020.  Exploring and predicting transferability across NLP tasks.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7882–7926, Online. Association for Computational Linguistics.
  * Wei et al. (2021a) Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2021a.  Finetuned language models are zero-shot learners.  _ArXiv_ , abs/2109.01652.
  * Wei et al. (2021b) Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021b.  Finetuned language models are zero-shot learners.  _arXiv preprint arXiv:2109.01652_.
  * Weller et al. (2020) Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E. Peters. 2020.  Learning from task descriptions.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1361–1375, Online. Association for Computational Linguistics.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Zhong et al. (2021a) Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. 2021a.  Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections.  In _EMNLP_.
  * Zhong et al. (2021b) Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021b.  Factual probing is [mask]: Learning vs. learning to recall.

##  Appendix A Generalized Prompts

The tasks we took prompts from that were not used to train T0 are:

  * •

COPA Roemmele et al. (2011)

  * •

FinancialNews Malo et al. (2014)

  * •

LAMBADA Paperno et al. (2016)

  * •

MathQA Amini et al. (2019)

  * •

MultiXSci Lu et al. (2020)

  * •

NumerSense Lin et al. (2020a)

  * •

SemEval2010 Hendrickx et al. (2010)

  * •

ZEST Weller et al. (2020)

The tasks we took prompts from that were used to train T0 are:

  * •

AppReviews Grano et al. (2017)

  * •

Adversarial QA Bartolo et al. (2020)

  * •

CommonGen Lin et al. (2020b)

  * •

IMDB Maas et al. (2011)

  * •

XSum Narayan et al. (2018)

Task Name | # | Type | MCQ
---|---|---|---
Text | 6 | All Three | ✓
Text+Choices | 6 | All Three | ✓
ANLI | 9 | Entailment |
AQuA | 5 | Classification | ✓
COPA | 3 | Entailment |
Craigslist | 4 | Classification | ✓
FinancialNews | 4 | Classification |
LAMBADA | 3 | Classification |
MathQA | 5 | Classification | ✓
MultiXSci | 4 | Classification |
NumerSense | 5 | Classification |
RTE | 5 | Entailment |
SemEval2010 | 3 | Classification |
WiC | 4 | Entailment |
ZEST | 4 | QA |
AppReviews | 2 | Classification |
AdversarialQA | 4 | QA |
CommonGen | 2 | Classification |
IMDB | 5 | Classification |
XSum | 4 | Classification |
Yelp | 4 | Classification |
Table 3: Number of prompts used by task for the generalized prompts. # is the
number of prompts used from this task. MCQ indicates if the task had prompts
that are formatted as an multiple choice question with choice letters.

##  Appendix B Complex Task Datasets

The two complex task used to evaluate T0’s performance are:

Algebra Question Answering (AQuA) Dataset of multiple choice algebraic word
problems. The choices for this task are {A,B,C,D,E} and each letter maps to a
potential mathematical answer Ling et al. (2017).
CraigslistBargains A collection of dialogues involving two-parties negotiating
the price of an item for sale on Craigslist. For the scope of this paper, we
use the task of classifying who won the negotiation He et al. (2018).

##  Appendix C Additional Results

This section is for results that could not be included in the main body of the
paper due to the page limits.

Figure 2: Median Accuracy with interquartile range for three models: GPT-3,  T5, and  T0. Darker indicates larger model. Results for GPT-3 Model are from Brown et al. (2020). Results for the 11B T0 and T5 models are taken from Sanh et al. (2021) Task | Model | Accuracy | F1
---|---|---|---
ANLI R1 | T0 | 34.30 | 26.17
T5 | 33.40 | 20.28
ANLI R2 | T0 | 33.40 | 23.70
T5 | 33.50 | 21.25
ANLI R3 | T0 | 33.42 | 21.82
T5 | 33.33 | 24.84
AQuA | T0 | 18.90 | 15.04
T5 | 24.41 | 12.74
CB | T0 | 55.36 | 38.62
T5 | 21.43 | 19.41
CraigslistBargains | T0 | 33.42 | 18.45
T5 | 44.89 | 16.47
RTE | T0 | 59.21 | 69.56
T5 | 52.89 | 36.08
WiC | T0 | 50.86 | 8.81
T5 | 50.16 | 5.44
Table 4: Median accuracy and F1 for the corresponding Figure 2. Figure 3: Distribution of choices for T0 and T5 on the AQuA and CraigslistBargains.  | Accuracy | F1
---|---|---
Ablation | Mean | Median | Q1 | Q3 | Mean | Median | Q1 | Q3
Training Prompts | 51.46 | 50.25 | 45.25 | 63.00 | 54.18 | 55.75 | 38.00 | 66.00
Unseen Prompts | 42.72 | 42.00 | 23.50 | 60.75 | 42.46 | 36.50 | 22.00 | 62.50
With Choices | 39.44 | 33.12 | 20.19 | 58.62 | 39.37 | 31.00 | 19.12 | 61.75
No Choices | 51.73 | 52.25 | 44.75 | 60.50 | 55.93 | 53.50 | 43.00 | 66.00
Is MCQ | 25.80 | 22.25 | 16.50 | 26.00 | 23.14 | 16.50 | 13.75 | 25.25
Not MCQ | 43.28 | 36.00 | 26.38 | 62.62 | 43.95 | 36.50 | 22.00 | 65.50
Extra Text | 44.99 | 46.75 | 28.81 | 60.31 | 46.44 | 46.50 | 27.75 | 66.00
No Extra Text | 44.41 | 48.00 | 32.75 | 57.62 | 45.43 | 44.50 | 26.62 | 62.25
Table 5: Accuracy and F1 ranks for different ablations. It is calculated by
taking the median rank of a given prompt across all 8 tasks then taking the
Mean, Median, Q1, and Q3 of that. Lower is better. Q1 and Q3 are the first and
third quartile.

ANLI R1 ANLI R2 ANLI R3 AQuA CB Craigslist RTE WiC Rank No Prompt 34.15 33.35
33.42 26.77 24.11 16.83 59.57 50.24 46.25 Unseen Prompts ANLI 37.60 34.70
34.08 25.95 32.14 21.44 64.62 50.16 24.50 AQuA 36.10 33.40 35.42 17.32 33.93
23.45 71.12 51.57 18.25 COPA 39.30 34.40 34.00 20.47 26.79 16.58 69.31 50.63
21.25 Craigslist 31.40 31.30 32.83 25.79 8.04 26.72 49.82 50.16 71.25 FinNews
33.05 31.65 32.83 25.95 18.75 19.68 55.78 50.31 64.00 LAMBADA 34.00 32.40
32.50 26.77 19.64 16.08 57.76 50.78 58.50 MathQA 37.30 33.50 34.25 19.29 26.79
16.25 73.29 51.10 24.50 Multi-XSci 34.20 32.70 32.75 27.17 19.64 19.43 58.84
50.31 54.75 NumerSense 37.70 33.30 33.17 25.20 25.00 15.75 65.70 50.63 40.50
RTE 36.10 33.20 33.58 22.05 23.21 20.27 61.37 50.47 43.25 SemEval2010 33.10
32.00 32.58 27.56 14.29 25.63 55.23 50.47 66.50 WiC 31.75 33.45 32.33 26.57
13.39 18.01 55.05 50.47 64.25 ZEST 35.20 32.65 33.38 26.77 23.21 17.76 66.79
50.71 38.25 Training Prompts AppReviews 34.20 33.10 33.62 27.17 19.64 33.17
61.55 50.31 33.50 CommonGen 33.75 33.35 32.50 25.39 13.39 23.62 51.81 51.18
58.75 IMDB 33.00 32.20 33.08 26.38 12.50 14.57 55.23 50.16 71.25 XSum 33.50
32.00 33.00 26.97 10.71 19.26 57.22 50.86 58.50 Yelp 33.25 32.35 33.04 26.77
12.50 24.29 62.27 51.57 41.75

Table 6: Median Accuracy when using modified prompts for cross task zero-shot evaluation. Bolded entries are prompts for the original task.  Green Cells and  Red Cells are the best and worst performing tasks for a column respectively. Rank is the median rank of prompts from this task out of 95 total prompts. ANLI and CB both use the same prompts for their original task prompts per PromptSource. | Accuracy | F1
---|---|---
Has Choices | -0.14 | -0.27
Is MCQ | -0.16 | -0.25
Training Prompt | 0.07 | 0.13
Length | 0.14 | 0.18
Table 7: Correlations with metric rank for a given prompt quality. Per the
definition of rank, a lower score is better and therefore a negative
correlation indicates a quality improves performance. Length is measured as
the raw number of tokens in a prompt. Figure 4: Accuracy and F1 rank compared
with the number of tokens in the prompt. The tick value is the lower bound of
the range. p=The number of prompts that fall into that respective range.