Retrieval-Augmented Large Language Models (LLMs), which incorporate the non-parametric knowledge from external knowledge bases into LLMs, have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA). However, even though there are various approaches dealing with queries of different complexities, they either handle simple queries with unnecessary computational overhead or fail to adequately address complex multi-step queries; yet, not all user requests fall into only one of the simple or complex categories. In this work, we propose a novel adaptive QA framework that can dynamically select the most suitable strategy for (retrieval-augmented) LLMs from the simplest to the most sophisticated ones based on the query complexity. Also, this selection process is operationalized with a classifier, which is a smaller LM trained to predict the complexity level of incoming queries with automatically collected labels, obtained from actual predicted outcomes of models and inherent inductive biases in datasets. This approach offers a balanced strategy, seamlessly adapting between the iterative and single-step retrieval-augmented LLMs, as well as the no-retrieval methods, in response to a range of query complexities. We validate our model on a set of open-domain QA datasets, covering multiple query complexities, and show that ours enhances the overall efficiency and accuracy of QA systems, compared to relevant baselines including the adaptive retrieval approaches. Code is available at: <https://github.com/starsuzi/Adaptive-RAG>.

§ INTRODUCTION

Recent Large Language Models (LLMs) <cit.> have shown overwhelming performances across diverse tasks, including question-answering (QA) <cit.>. However, they still generate factually incorrect answers since their knowledge solely relies on their parametric memory <cit.>. Meanwhile, memorizing all the (ever-changing) world knowledge may not be possible. To address this problem, retrieval-augmented LLMs <cit.>, which incorporate non-parametric knowledge into LLMs with additional retrieval modules, have gained much increasing attention. Specifically, these models access a knowledge base, which serves as an extensive repository of information across various subjects and disciplines, to retrieve information relevant to the given input, and then incorporate the retrieved information into LLMs, which enables them to stay accurate and current with the world knowledge.

A particularly salient application of retrieval-augmented LLMs is to handling QA tasks, whose goal is to provide correct answers in response to user queries, especially those of high complexity. Early work on retrieval-augmented LLMs focuses primarily on single-hop queries <cit.>, whose answers are typically found within a single document; therefore, this approach involves retrieving a relevant document based on the query and subsequently integrating this information into QA models to formulate a response. However, unlike this single-hop QA, some queries require connecting and aggregating multiple documents, which are, furthermore, often not answerable through a single-step process of retrieval-and-response. An example query is `When did the people who captured Malakoff come to the region where Philipsburg is located?', which requires four reasoning steps to solve. Therefore, to effectively handle such complex queries, recent studies have concentrated largely on multi-step and multi-reasoning QA, which requires iterative accesses to both LLMs and retrievers multiple times <cit.>, at the cost of heavy computational overheads.

Yet, we should rethink: In a real-world scenario, are all the requests from users complex? Instead, users might often ask simple and straightforward questions, while only occasionally asking complex ones. Specifically, a query such as `Paris is the capital of what?' is likely to be asked more frequently, compared to the aforementioned multi-step query, and this simpler query might also be easily answered by the LLMs themselves, without accessing external knowledge. In other words, a multi-step QA approach could give rise to unnecessary computational overhead for simple queries, even though it would be vital for complex queries (see Figure <ref> (A)). On the other hand, handling complex queries with single-step-retrieval or even non-retrieval strategies would be largely insufficient (Figure <ref> (B)). This suggests the need for an adaptive QA system, which can dynamically adjust the operational strategies of retrieval-augmented LLMs based on the query complexity. While some recent approaches are capable of doing this based on the frequency of entities in queries <cit.> or on the generated outputs from models for multi-step QA <cit.>, they are still suboptimal: the former methods are overly simplistic, failing to consider multi-hop queries; meanwhile, the latter are excessively complex, terminating answer solving steps after several rounds of module access.

In this work, considering diverse complexity levels of real-world queries, we argue that previous one-size-fits-all approaches might be inadequate to cover all of them. Instead, we propose to select the most suitable strategy from a range of (retrieval-augmented) LLMs, each of which is tailored to the specific complexity of the input query. Notably, a critical step in this process is pre-defining the query complexity, which is instrumental in determining the most fitting model to it. In this work, we operationalize this process with a novel classifier, which is a smaller model trained to predict the complexity level of incoming queries (see Figure <ref> (c)). Moreover, we automatically collect its training datasets without human labeling, by leveraging the predicted outcomes (i.e., which models accurately respond to which queries) as well as by capitalizing on the inherent biases in existing datasets (i.e., samples in the datasets are designed either for single-step or for multi-step QA scenarios). This proposed method can offer a robust middle ground among the iterative LLM augmentation methods for complex queries, single-step methods for simpler queries, and even no-retrieval-augmented methods for the most straightforward queries (answerable by LLMs themselves), thus significantly enhancing the overall efficiency and accuracy, as shown in Figure <ref>. We refer to our framework as Adaptive Retrieval-Augmented Generation (Adaptive-RAG).

We validate Adaptive-RAG using benchmark open-domain QA datasets, covering a wide range of query complexity from single-hop <cit.> to multi-hop <cit.> queries. The experimental results show that ours significantly improves the overall accuracy and efficiency, compared to the prior adaptive strategies, on multiple LLMs, such as GPT-3.5 <cit.> and FLAN-T5 series <cit.>.

Our contributions and findings are threefold:

* We point out the realistic scenario of queries of varying complexities, and find out that existing retrieval-augmented generation approaches tend to be overly simple or complex.

* We adapt retrieval-augmented LLMs to the query complexity assessed by the classifier, which enables the utilization of the most suitable approach tailored to each query.

* We show that our Adaptive-RAG is highly effective and efficient, balancing between the complexity and the simplicity for diverse queries.

§ RELATED WORK

Open-domain QA

Open-domain QA is the task of accurately answering a query by sourcing for query-relevant documents, and then interpreting them to provide answers <cit.>, which, thus, generally involves two modules: a retriever <cit.> and a reader <cit.>.
Along with the emergence of LLMs with superior reasoning capabilities thanks to their billion-sized parameters <cit.>, a synergy between LLMs and retrievers has led to significant advancements <cit.>. Specifically, this integration has been shown to enhance Open-domain QA by mitigating the hallucination problem from LLMs through strengthened reasoning abilities of the reader, as well as utilizing the retrieved, external documents <cit.>. Despite these advancements for single-hop retrieval-augmented LLMs, however, the complexity of some queries needs a more complex strategy.

Multi-hop QA
Multi-hop QA is an extension of conventional Open-domain QA, which additionally requires the system to comprehensively gather and contextualize information from multiple documents (often iteratively), to answer more complex queries <cit.>. In the realm of multi-hop QA, the approach to iteratively access both LLMs and the retrieval module is generally employed. Specifically, <cit.>, <cit.>, <cit.> and <cit.> proposed to first decompose the multi-hop queries into simpler single-hop queries, repeatedly access the LLMs and retriever to solve these sub-queries, and merge their solutions to formulate a complete answer. In contrast to this decomposition-based approach, other recent studies, such as <cit.> and <cit.>, explored the interleaving of Chain-of-Thought reasoning <cit.> — a method where a logical sequence of thoughts is generated — with document retrieval, repeatedly applying this process until the reasoning chain generates the answer. In addition, <cit.> introduced an approach to repeatedly retrieving new documents if the tokens within generated sentences have low confidence. However, the aforementioned methods overlooked the fact that, in real-world scenarios, queries are of a wide variety of complexities. Therefore, it would be largely inefficient to iteratively access LLMs and retrievers for every query, which might be simple enough with a single retrieval step or even only with an LLM itself.

Adaptive Retrieval
To handle queries of varying complexities, the adaptive retrieval strategy aims to dynamically decide whether to retrieve documents or not, based on each query's complexity.
In this vein, <cit.> proposed to decide the query's complexity level based on the frequency of its entities and suggested using the retrieval modules only when the frequency falls below a certain threshold. However, this approach, focusing solely on the binary decision of whether to retrieve or not, may not be sufficient for more complex queries that require multiple reasoning steps.
Additionally, <cit.> proposed an approach that performs a fixed set of operations (retrieving, reading, and reranking) multiple times until the answer is derived for the given query, which is built upon traditional BERT-like LMs. However, unlike our Adaptive-RAG which pre-determines the query complexity and adapts the operational behavior of any off-the-shelf LLMs accordingly, this approach applies the same fixed operations to every query regardless of its complexity but also necessitates additional specific training to LMs. Concurrent to our work, <cit.> suggested training a sophisticated model to dynamically retrieve, critique, and generate the text. Nevertheless, we argue that all the aforementioned adaptive retrieval methods that rely on a single model might be suboptimal in handling a variety of queries of a range of different complexities since they tend to be either overly simple or complex for all the input queries, which demands a new approach that can select the most suitable strategy of retrieval-augmented LLMs tailored to the query complexity.

§ METHOD

In this section, we describe our approach to adapting retrieval-augmented LLMs, by pre-determining the query complexity and then selecting the most fitting strategies for retrieval-augmented LLMs.

§.§ Preliminaries

We begin with preliminaries, formally introducing different strategies of retrieval-augmented LLMs.

Non Retrieval for QA
Let us first define an LLM as a model $\texttt{LLM}$, which takes a sequence of tokens $\vx = [x_1, x_2, ..., x_n]$ as an input and then generates a sequence of tokens $\vy = [y_1, y_2, ..., y_n]$ as an output, which is formalized as follows: $\vy = \texttt{LLM}(\vx)$. Then, in our problem setup for QA, $\vx$ and $\vy$ become the input query ($\vq$) from the user and the generated answer ($\bar{\va}$) from the LLM, respectively: $\vq = \vx$ and $\bar{\va} = \vy$. Also, subsequently, the most naïve LLM-powered QA model can be represented as follows: $\bar{\va} = \texttt{LLM}(\vq)$. Ideally, $\bar{\va}$ should match the actual correct answer $\va$. This non-retrieval-based QA method is highly efficient and could be a somewhat promising approach to handling easy queries, as the size of LLMs becomes extremely large with its effect on storing a large amount of knowledge. However, this approach is largely problematic on queries that require precise or concurrent knowledge of specific people, events, or any subjects beyond the LLMs' internal knowledge.

Single-step Approach for QA
To address the aforementioned scenarios where $\texttt{LLM}$ may struggle with queries that are not answerable by $\texttt{LLM}$ itself, we can utilize the external knowledge $\vd$, which includes useful information for queries, retrieved from the external knowledge source $\mathcal{D}$ that could be an encyclopedia (e.g., Wikipedia) consisting of millions of documents. Specifically, to obtain such $\vd$ from $\mathcal{D}$, a specific retrieval model is necessary, which returns documents based on their relevance with the given query. This process can be formulated as follows: $\vd = \texttt{Retriever}(\vq; D)$, where $\texttt{Retriever}$ is the retrieval model, with $\vd \in \mathcal{D}$. Here, we can use any off-the-shelf retriever <cit.>.

After the retrieval step is done, we now have a pair of query $\vq$ and its relevant documents $\vd$. Then, in order to augment LLMs with this retrieved external knowledge, we can incorporate it into the input of LLMs, represented as follows: $\bar{\va} = \texttt{LLM}(\vq, \vd)$. This process allows LLMs to gain access to external information contained in $\vd$, which can provide the supplementary context that the internal knowledge of $\texttt{LLM}$ lacks, which can subsequently improve the accuracy and concurrency of LLMs for QA.

Multi-step Approach for QA
Even though the aforementioned single-step approach offers significant improvements over non-retrieval for $\vq$ that requires external knowledge, it encounters notable limitations, particularly when dealing with complex queries that necessitate synthesizing information from multiple source documents and reasoning over them. This is where a multi-step approach and reasoning for QA become essential.

In this multi-step approach, $\texttt{LLM}$ interacts with  in several rounds, progressively refining its understanding of $\vq$, until it formulates the final answer from findings accumulated across these multiple steps. Specifically, the process begins with the initial query $\vq$, and at every retrieval step $i$, new documents $\vd_i$ are retrieved from $\mathcal{D}$ and then incorporated into the input of LLMs, as follows: $\bar{\va}_i = \texttt{LLM}(\vq, \vd_i, \vc_i)$, where the additional context $\vc_i$ can be composed of previous documents and outcomes $(\vd_1, \vd_2, ..., \vd_{i-1}, \bar{\va}_1, \bar{\va}_2, ..., \bar{\va}_{i-1})$, and $\vd_i = \texttt{Retriever}(\vq, \vc_i; D)$[It is worth noting that implementations of the LLM and retriever vary across different multi-step retrieval-augmented LLM approaches <cit.>; therefore, the context $\vc_i$ may incorporate none, some, or all of the previous documents and answers.]. We would like to note that this iterative, multi-step process enables $\texttt{LLM}$ to construct a more comprehensive and extensive foundation to solve queries effectively, specifically adept at complex multi-hop queries where answers depend on interconnected pieces of information. However, it is important to recognize that this multi-step approach can be resource-intensive due to the repeated accesses to $\texttt{Retriever}$ and $\texttt{LLM}$, which entail substantial computational costs.

§.§ Adaptive-RAG: Adaptive Retrieval-Augmented Generation

We now introduce our adaptive retrieval-augmented LLMs, which are built upon three different strategies described in the previous section, and which are designed to select the most suitable strategy according to the complexity of queries.

Adapting Retrieval-Augmented LLMs
Note that in real-world scenarios, not all $\vq$ from users have the same level of complexity, necessitating tailored strategies for handling each query. In other words, employing the most basic, non-retrieval-based approach $\texttt{LLM}(\vq)$ to respond to the complex query $\vq$ would be also ineffective (Figure <ref>, A); conversely, using a more elaborate multi-step approach $ \texttt{LLM}(\vq, \vd, \vc)$ for simple $\vq$ would be inefficient (Figure <ref>, B). Therefore, our adaptive framework is designed to dynamically adjust the query-handling strategy of retrieval-augmented LLMs, which is achieved by determining the complexity of each query before attempting a solution. Notably, this framework can offer a robust middle ground with a range of solutions, from the simplest approach for the most straightforward queries, to the one-step approach for moderate queries, and up to the most comprehensive and rigorous approach for complex queries. In addition, since the operations of $\texttt{LLM}$ and $\texttt{Retriever}$ remain consistent regardless of inputs to them, our method can seeminglessly go back and forth across queries of different complexities, without changing the internal model architecture or parameters during adaption.

Query Complexity Assessment
To operationalize our adaptive retrieval-augmented LLM framework, we should determine the query complexity, and to achieve this, we propose to model a complexity classifier, whose goal is to return the appropriate complexity level of the given query. Specifically, given the query $\vq$, our classifier can be formulated as follows: $o = \texttt{Classifier}(\vq)$, where $\texttt{Classifier}$ is a smaller Language Model that is trained to classify one of three different complexity levels and $o$ is its corresponding class label. In our classifier design, there are three class labels: `A', `B', and `C', where `A' indicates that $\vq$ is straightforward and answerable by $\texttt{LLM}(\vq)$ itself, `B' indicates that $\vq$ has the moderate complexity where at least a single-step approach $\texttt{LLM}(\vq, \vd)$ is needed, and `C' indicates that $\vq$ is complex, requiring the most extensive solution $\texttt{LLM}(\vq, \vd, \vc)$[We consider three levels of query complexity, and leave the exploration of more fine-grained complexities as future work.].

Training Strategy
The remaining step is to train the smaller Language Model for $\texttt{Classifier}$, to accurately predict its complexity $o$ in response to the given query $\vq$. Yet, there is no annotated dataset available for query-complexity pairs. Hence, we propose to automatically construct the training dataset with two particular strategies.

To be specific, we first aim at labeling the query complexity based on the results from three different retrieval-augmented LLM strategies, in order to determine the label by its needs. For example, if the simplest non-retrieval-based approach correctly generates the answer, the label for its corresponding query is assigned `A'. Also, to break the tie between different models in providing the label to the query, we provide a higher priority to a simpler model. In other words, if both single-step and multi-step approaches produce the same correct answer while the non-retrieval-based approach fails, we assign label `B' to its corresponding query.

However, this labeling strategy has a limitation in that not all the queries are assigned labels, since the three retrieval-augmented approaches may all fail to generate the correct answer. On the other hand, the benchmark datasets may already have meaningful inductive biases about the most appropriate retrieval-augmented LLM strategies for their queries, considering the ways they are created (e.g., QA datasets that require sequential reasoning usually necessitate a multi-step approach; while queries of those with labeled single documents can be ideally answerable with the single-step approach). Therefore, for those queries that remain unlabeled after the first labeling step, we assign `B' to queries in single-hop datasets and `C' to queries in multi-hop datasets. Finally, we train $\texttt{Classifier}$ with these automatically-collected query-complexity pairs[As we automatically assign classifier labels, there might be errors in labeling and might be more advanced strategies to automatically assign labels, which we leave as future work.], by using a cross-entropy loss. Then, at inference, we can determine the complexity of the query, which is one of {`A', `B', `C'}, by forwarding it to $\texttt{Classifier}$: $o = \texttt{Classifier}(\vq)$.

§ EXPERIMENTAL SETUPS

In this section, we explain datasets, models, metrics, and implementation details. We provide additional details in Appendix <ref>.

§.§ Datasets

In order to simulate a realistic scenario, where different queries have varying complexities, we use both the single-hop and multi-hop QA datasets simultaneously, in the unified experimental setting.

Single-hop QA
For simpler queries, we use three benchmark single-hop QA datasets, which consist of queries and their associated documents containing answers, namely 1) SQuAD v1.1 <cit.>, 2) Natural Questions <cit.>, and 3) TriviaQA <cit.>.

Multi-hop QA
To consider more complex query scenarios, we use three benchmark multi-hop QA datasets, which require sequential reasoning over multiple documents, namely 1) MuSiQue <cit.>, 2) HotpotQA <cit.>, and 3) 2WikiMultiHopQA <cit.>.

§.§ Models

We compare our Adaptive-RAG against relevant models, including three retrieval-augmented LLM strategies (in Section <ref>) and the adaptive retrieval approaches <cit.>, which can be grouped into one of three categories: Simple, Adaptive, and Complex. Specifically, Simple approaches include the 1) No Retrieval and 2) Single-step Approach-based methods. Adaptive approaches include the 3) Adaptive Retrieval <cit.>, 4) Self-RAG <cit.>, and our 5) Adaptive-RAG, which can adaptively perform retrieval based on the question complexity. For the 6) Multi-step Approach, we use the most sophisticated state-of-the-art method <cit.>, iteratively accessing both the retriever and LLM with Chain-of-Thought reasoning <cit.>, for every query. Note that models across different categories are not directly comparable. Yet, in the ideal setting, Adaptive approaches should be more effective than those in the Simple category while simultaneously being more efficient than the Complex one. Therefore, we also report the performance in an ideal scenario, 7) Adaptive-RAG w/ Oracle, using the oracle classifier with our Adaptive-RAG.

§.§ Evaluation Metrics

When it comes to evaluating adaptive models, it is essential to simultaneously consider both the task performance and efficiency along with their trade-offs. Thus, we report the results with five metrics, where three of them measure the effectiveness and the other two measure the efficiency. In particular, for effectiveness, we use F1, EM, and Accuracy (Acc), following the standard evaluation protocol <cit.>, where F1 measures the number of overlapping words between the predicted answer and the ground truth, EM measures whether they are the same, and Acc measures whether the predicted answer contains the ground-truth answer.
For efficiency, we measure the number of retrieval-and-generate steps and the average time for answering each query relative to the one-step approach.

§.§ Implementation Details

For a fair comparison and following <cit.> and <cit.>, we use the same retriever, a term-based sparse retrieval model known as BM25 <cit.>, across all different models. For the external document corpus, we use different sources depending on the dataset type: the Wikipedia corpus preprocessed by <cit.> for single-hop datasets, and the preprocessed corpus by <cit.> for multi-hop datasets. Regarding the LLMs that are used to generate answers, we use the FLAN-T5 series models <cit.> of XL with 3B parameters and XXL with 11B parameters, and the GPT-3.5 model (gpt-3.5-turbo-instruct). For the retrieval-augmented LLM design, we follow the implementation details from <cit.>, which include input prompts, instructions, and the number of test samples for evaluation (e.g., 500 samples per dataset). In our Adaptive-RAG, for the query-complexity classifier, we use and train the T5-Large model <cit.>. Specifically, the classifier is trained using the epoch that shows the best performance until 100 training iterations from the validation set, with the learning rate of 3e-5 and the AdamW <cit.> as an optimizer. Regarding its training data, we sample and annotate 400 queries from 6 datasets based on its inductive bias (single-hop for one-step approach and multi-hop for multi-step). In addition, we use predicted outcomes of three different strategies over 400 queries sampled from each dataset. Note that those queries used for classifier training do not overlap with the testing queries for QA.

§ EXPERIMENTAL RESULTS AND ANALYSES

In this section, we show the overall experimental results and offer in-depth analyses of our method.

Main Results
First of all, Table <ref> shows our main results averaged over all considered datasets, which corroborate our hypothesis that simple retrieval-augmented strategies are less effective than the complex strategy, while the complex one is significantly more expensive than the simple ones. In addition, we report the more granular results with FLAN-T5-XL on each of the single-hop and multi-hop datasets in Table <ref> (and more with different LLMs in Table <ref> and Table <ref> of Appendix), which are consistent with the results observed in Table <ref>.

However, in a real-world scenario, not all users ask queries with the same level of complexity, which emphasizes the importance of the need for adaptive strategies. Note that among the adaptive strategies, our Adaptive-RAG shows remarkable effectiveness over the competitors (Table <ref>). This indicates that merely focusing on the decision of whether to retrieve or not is suboptimal. Also, as shown in Table <ref>, such simple adaptive strategies are particularly inadequate for handling complex queries in multi-hop datasets, which require aggregated information and reasoning over multiple documents. Meanwhile, our approach can consider a more fine-grained query handling strategy by further incorporating an iterative module for complex queries.

Furthermore, in a realistic setting, we should take into account not only effectiveness but also efficiency. As shown in Table <ref>, compared to the complex multi-step strategy, our proposed adaptive strategy is significantly more efficient across all model sizes. This is meaningful in this era of LLMs, where the cost of accessing them is a critical factor for practical applications and scalability.

Finally, to see the upper bound of our Adaptive-RAG, we report its performances with the oracle classifier where the classification performance is perfect. As shown in Table <ref> and Table <ref>, we observe that it achieves the best performance while being much more efficient than our Adaptive-RAG without the oracle classifier. These results support the validity and significance of our proposal for adapting retrieval-augmented LLM strategies based on query complexity, and further suggest the direction to develop more improved classifiers to achieve optimal performance.

Classifier Performance
To understand how the proposed classifier works, we analyze its performance across different complexity labels. As Figure <ref> (Left and Center) shows, the classification accuracy of our Adaptive-RAG is better than those of the other adaptive retrieval baselines, which leads to overall QA performance improvements. In other words, this result indicates that our Adaptive-RAG is capable of more accurately classifying the complexity levels with various granularities, which include not performing retrieval, performing retrieval only once, and performing retrieval multiple times. In addition to the true positive performance of our classifier averaged over all those three labels in Figure <ref> (Left and Center), we further report its confusion matrix in Figure <ref> (Right). We note that the confusion matrix reveals some notable trends: `C (Multi)' is sometimes misclassified as `B (One)' (about 31%) and `B (One)' as `C (Multi)' (about 23%); `A (No)' is misclassified often as `B (One)' (about 47%) and less frequently as `C (Multi)' (about 22%). While the overall results in Figure <ref> show that our classifier effectively categorizes the three labels, further refining it based on such misclassification would be a meaningful direction for future work.

Analyses on Efficiency for Classifier
While Table <ref> shows the relative elapsed time for each of the three different RAG strategies, we further provide the exact elapsed time per query for our Adaptive-RAG and the distribution for predicted labels from our query-complexity classifier in Table <ref>. Similar to the results of the elapsed time in Table <ref> (relative time), Table <ref> (exact time) shows that efficiency can be substantially improved by identifying simple or straightforward queries.

Analyses on Training Data for Classifier
We have shown that the classifier plays an important role in adaptive retrieval. Here, we further analyze the different strategies for training the classifier by ablating our full training strategy, which includes two approaches: generating silver data from predicted outcomes of models and utilizing inductive bias in datasets (see Section <ref>). As Table <ref> shows, compared to the training strategy relying solely on the data derived from inductive bias, ours is significantly more efficient. This efficiency is partly because ours also takes into account the case that does not consider any documents at all, as also implied by the classification accuracy; meanwhile, queries in the existing datasets do not capture the information on whether the retrieval is required or not. On the other hand, in the case of only using the silver data annotated from the correct predictions, while its overall classification accuracy is high, the overall QA performance implies that relying on the silver data may not be optimal. This may be because this silver data does not cover complexity labels over incorrectly predicted queries, which leads to lower generalization effect on queries relevant to them. Meanwhile, by also incorporating complexity labels from dataset bias (single-hop vs multi-hop), the classifier becomes more accurate in predicting multi-hop queries, leading to the better performance. It is worth noting that our automatic labeling strategies are two particular instantiations for training the classifier, and that there could be other instantiations, which we leave as future work.

Analyses on Classifier Size
To investigate the sensitivity of our classifier according to its varying sizes, we conducted further experiments. As shown in Table <ref>, we observe no significant performance differences among classifiers of various sizes, even with reduced complexity and fewer parameters in smaller classifiers. This indicates that our proposed classifier can contribute to resource-efficient settings in real-use cases with smaller sizes without compromising the performance.

Case Study
We conduct a case study to qualitatively compare our Adaptive-RAG against Adaptive Retrieval. Table <ref> shows the classified complexity and the query handling patterns for both simple and complex questions.
First, for the simple single-hop question, our Adaptive-RAG identifies that it is answerable by only using the LLM's parametric knowledge about `Google'. By contrast, Adaptive Retrieval fetches additional documents, leading to longer processing times and occasionally producing incorrect responses due to the inclusion of partially irrelevant information about `Microsoft'.
Meanwhile, faced with a complex question, Adaptive-RAG seeks out relevant information, including details like `a son of John Cabot', which may not have been stored in LLMs, while Adaptive Retrieval fails to request such information from external sources, resulting in inaccurate answers.

§ CONCLUSION

In this work, we proposed the Adaptive Retrieval-Augmented Generation framework, referred to as Adaptive-RAG, to handle queries of various complexities. Specifically, Adaptive-RAG is designed to dynamically adjust its query handling strategies in the unified retrieval-augmented LLM based on the complexity of queries that they encounter, which spans across a spectrum of the non-retrieval-based approach for the most straightforward queries, to the single-step approach for the queries of moderate complexity, and finally to the multi-step approach for the complex queries. The core step of our Adaptive-RAG lies in determining the complexity of the given query, which is instrumental in selecting the most suitable strategy for its answer. To operationalize this process, we trained a smaller Language Model with query-complexity pairs, which are automatically annotated from the predicted outcomes and the inductive biases in datasets. We validated our Adaptive-RAG on a collection of open-domain QA datasets, covering the multiple query complexities including both the single- and multi-hop questions. The results demonstrate that our Adaptive-RAG enhances the overall accuracy and efficiency of QA systems, allocating more resources to handle complex queries while efficiently handling simpler queries, compared to the existing one-size-fits-all approaches that tend to be either minimalist or maximalist over varying query complexities.

§ LIMITATIONS

While our Adaptive-RAG shows clear advantages in effectiveness and efficiency by determining the query complexity and then leveraging the most suitable approach for tackling it, it is important to recognize that there still exist potential avenues for improving the classifier from the perspectives of its training datasets and architecture. Specifically, as there are no available datasets for training the query-complexity classifier, we automatically create new data based on the model prediction outcomes and the inductive dataset biases. However, our labeling process is one specific instantiation of labeling the query complexity, and it may have the potential to label queries incorrectly despite its effectiveness. Therefore, future work may create new datasets that are annotated with a diverse range of query complexities, in addition to the labels of question-answer pairs. Also, as the performance gap between the ideal classifier in Table <ref> and the current classifier in Figure <ref> indicates, there is still room to improve the effectiveness of the classifier. In other words, our classifier design based on the smaller LM is the initial, simplest instantiation for classifying the query complexity, and based upon it, future work may improve the classifier architecture and its performance, which will positively contribute to the overall QA performance.

§ ETHICS STATEMENT

The experimental results on Adaptive-RAG validate its applicability in realistic scenarios, where a wide range of diverse user queries exist. Nonetheless, given the potential diversity of real-world user inputs, it is crucial to also consider scenarios where these inputs might be offensive or harmful. We should be aware that such inputs could lead to the retrieval of offensive documents and the generation of inappropriate responses by the retrieval-augmented LLMs. To address this challenge, developing methods to detect and manage offensive or inappropriate content in both user inputs and retrieved documents within the retrieval-augmented framework is essential. We believe that this is a critical area for future work.

§ ACKNOWLEDGEMENTS

This work was supported by Institute for Information and communications Technology Promotion (IITP) grant funded by the Korea government (No. 2018-0-00582, Prediction and augmentation of the credibility distribution via linguistic analysis and automated evidence document collection), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (RS-2023-00275747), and the Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT (MSIT, Korea) & Gwangju Metropolitan City.

§ ADDITIONAL EXPERIMENTAL SETUPS

§.§ Datasets

We use publicly open datasets for both single-hop and multi-hop QA datasets, referring to as <cit.> and <cit.>, respectively. We describe the characteristics of each dataset:

1) SQuAD v1.1 <cit.> is created through a process where annotators write questions based on the documents they read.

2) Natural Questions <cit.> is constructed by real user queries on Google Search.

3) TriviaQA <cit.> comprises trivia questions sourced from various quiz websites.

4) MuSiQue <cit.> is collected by compositing multiple single-hop queries, to form queries spanning 2-4 hops.

5) HotpotQA <cit.> is constructed by having annotators create questions that link multiple Wikipedia articles.

6) 2WikiMultiHopQA <cit.> is derived from Wikipedia and its associated knowledge graph path, needing 2-hops.

§.§ Models

We describe the details of models as follows:

1) No Retrieval. This approach uses only the LLM itself, to generate the answer to the given query.

2) Single-step Approach. This approach first retrieves the relevant knowledge with the given query from the external knowledge sources and then augments the LLM with this retrieved knowledge to generate the answer, which iterates only once.

3) Adaptive Retrieval. This baseline <cit.> adaptively augments the LLM with the retrieval module, only when the entities appearing in queries are less popular. To extract entities, we use the available entity-linking method <cit.>, namely BLINK, for questions.

4) Self-RAG. This baseline <cit.> trains the LLM to adaptively perform retrieval and generation, where the retrieval is conducted once it predicts the special retrieval token above a certain threshold, and the answer generation follows.

5) Adaptive-RAG. This is our model that adaptively selects the retrieval-augmented generation strategy, smoothly oscillating between the non-retrieval, single-step approach, and multi-step approaches[For the multi-step approach, we use the state-of-the-art question answering strategy from IRCoT <cit.>.] without architectural changes, based on the query complexity assessed by the classifier.

6) Multi-step Approach. This approach <cit.> is the multi-step retrieval-augmented LLM, which iteratively accesses both the retriever and LLM with interleaved Chain-of-Thought reasoning <cit.> repeatedly until it derives the solution or reaches the maximum step number.

7) Adaptive-RAG w/ Oracle This is an ideal scenario of our Adaptive-RAG equipped with an oracle classifier that perfectly categorizes the query complexity.

§.§ Implementation Details

For computing resources, we use A100 GPUs with 80GB memory. In addition, due to the significant costs associated with evaluating retrieval-augmented generation models, we perform experiments with a single run. Finally, we implemented models using PyTorch <cit.> and Transformers library <cit.>.

§ ADDITIONAL EXPERIMENTAL RESULTS

Performance vs Time
We further provide a comparison of different retrieval-augmented generation approaches with FLAN-T5-XL and FLAN-T5-XXL models in Figure <ref> and Figure <ref>, respectively, in the context of performance and efficiency trade-offs. Similar to the observation made from the GPT-3.5 model in Figure <ref>, our proposed Adaptive-RAG is significantly more effective as well as efficient.

Performance per Dataset
In addition to detailing the performance of each dataset with the FLAN-T5-XL model, as shown in Table <ref>, we also present the results for each dataset with the FLAN-T5-XXL and GPT-3.5 models in Table <ref> and Table <ref>, respectively. The experimental results show that our Adaptive-RAG consistently balances between efficiency and accuracy. It is worth noting that while the GPT-3.5 model performs effectively in addressing straightforward queries even without document retrieval, it benefits significantly from our Adaptive-RAG in terms of effectiveness when solving complex multi-hop queries.


# A Superconducting Nanowire Binary Shift Register

Reed A. Foster<EMAIL_ADDRESS>Matteo Castellani  Alessandro Buzzi  Owen
Medeiros  Marco Colangelo  Karl K. Berggren  Department of Electrical
Engineering and Computer Science, Massachusetts Institute of Technology,
Cambridge, Massachusetts 02139, USA

###### Abstract

We present a design for a superconducting nanowire binary shift register,
which stores digital states in the form of circulating supercurrents in high-
kinetic-inductance loops. Adjacent superconducting loops are connected with
nanocryotrons, three terminal electrothermal switches, and fed with an
alternating two-phase clock to synchronously transfer the digital state
between the loops. A two-loop serial-input shift register was fabricated with
thin-film NbN and achieved a bit error rate less than 10-4, operating at a
maximum clock frequency of ${83}\,\mathrm{MHz}$ and in an out-of-plane
magnetic field up to ${6}\,\mathrm{mT}$. A shift register based on this
technology offers an integrated solution for low-power readout of
superconducting nanowire single photon detector arrays, and is capable of
interfacing directly with room-temperature electronics and operating
unshielded in high magnetic field environments.

††preprint: AIP/123-QED

Superconducting nanowires are interesting candidates for cryogenic data
processing and storage, particularly for readout of superconducting nanowire
single photon detector (SNSPD) arrays. The high kinetic inductance of thin
film superconductors allows them to store data in compact loops,Zhao  _et al._
(2018) and the existence of nanocryotrons (nTrons), three-terminal
electrothermal switches, McCaughan and Berggren (2014) enables the creation of
low-power digital logic and memory elements.Buzzi  _et al._ (2022) In
addition, superconducting nanowires can operate in harsh environments. NbN is
radiation hard,Gregshammer  _et al._ (1988) and SNSPDs have been shown to
operate under high magnetic fields: both in-plane up to ${5}\,\mathrm{T}$ and
out-of-plane up to ${500}\,\mathrm{mT}$.Polakovic  _et al._ (2020) This makes
nanowires an interesting candidate for applications in which SNSPD readout
electronics must be able to withstand strong ambient magnetic fields or
radiation, such as high energy physics and space exploration. Furthermore, the
shared technology platform with SNSPDs and ability to drive high-impedance
loadsMcCaughan and Berggren (2014) is a strong motivator for direct
integration of nanowire electronics with SNSPD arrays. Dedicated readout
electronics are necessary to address the thermal and mechanical challenges of
scaling SNSPD imagers beyond 1 kilopixel, Steinhauer, Gyger, and Zwiller
(2021) and low-power electronic devices that operate in extreme environments
and can be fabricated adjacent to superconducting detectors are an attractive
choice over Josephson junction logic and CMOS. Previous work Zhao  _et al._
(2017); McCaughan  _et al._ (2022) has used the high kinetic inductance of
superconducting nanowires to make analog delay-line imagers, which offer high
pixel counts and preserve the picosecond timing resolution of the SNSPDs. Row
column multiplexing has also been shown as an effective technique for reducing
cable counts, Wollman  _et al._ (2019) however a more aggressive reduction in
cable count will be required for megapixel arrays. Inspired by the operation
of a semiconductor CCD, serial readout of SNSPD arrays could be performed by a
superconducting nanowire binary shift register. Serial readout may enable
higher count rates than delay-line techniques by shortening dead-time, but
most importantly, it simplifies the interface to conventional CMOS readout
electronics by removing the need for high resolution, low jitter time-to-
digital converters.

In this work, we demonstrate a proof-of-concept for a superconducting nanowire
binary shift register, which encodes digital states with the presence or
absence of circulating current in superconducting loops. As shown by the
sequence in Fig. 1, a two-phase clock is used to synchronously transfer states
between adjacent loops. When there is a circulating current present in a loop,
it adds with the clock pulse and exceeds the critical current density in the
nTron channel, causing it to switch from superconductive to resistive,
diverting the clock pulse into the next loop. In comparison to the original
nTron design,McCaughan and Berggren (2014) which acts like an amplifier, the
loops are connected with wide-gate nTrons, where the width of the gate
constriction is comparable or equal to that of the channel constriction. A
wide-gate nTron is crucial for the shift register: because the output of one
nTron becomes the input of another, the current levels for the input and
output should be equal. The additional readout nTron shown in Fig. 1f uses a
standard nTron with a small choke. It terminates the final loop of the shift
register to destroy any circulating current present at the end of each clock
cycle. The readout nTron serves two purposes: (1) to reset the final loop of
the shift register, and (2) to generate an output voltage signal, which can be
sent to off-chip readout electronics or cascaded through a resistor to other
nTron logic.

Figure 1:  Principle of operation of the shift register, which uses the
presence or absence of circulating current to encode digital states. A two
phase clock ($\phi_{1}$, $\phi_{2}$) is used to transfer the digital state
between adjacent loops. (a) Shows a shift register with an initial circulating
current in the loop formed between nTrons U1 and U2. In (b), the first clock
phase $\phi_{1}$ is applied, which, when combined with the circulating
current, exceeds the critical current of U2’s channel. This forms a hotspot in
U2’s channel, diverting the clock through the kinetic inductor between U2 and
U3 as shown in (c). By the time the clock is turned off in (d), the channel of
U2 has healed and a circulating current forms in the loop between U2 and U3.
The process continues in (e) when the second clock phase $\phi_{2}$ is
applied. Two clock phases are needed to ensure a zero resistance path to
ground for the diverted clock, for example, the path through U3 as shown in
(c). The readout nTron U${}_{\text{ro}}$ in (f) is used to reset the state of
the final loop and generate an output voltage conditional on the presence of a
circulating current.  Figure 2:  (a) and (b) show electron micrographs of
fabricated wide nTron and two loop shift register. The large meanders in (b)
are ${100}\,\mathrm{nH}$ kinetic inductors. (c) Is an equivalent circuit model
of the experimental circuit. The current pulses are provided by a voltage
source in series with ${2}\,\mathrm{k\Omega}$ resistors mounted off-chip on a
printed circuit board.

The shift register was fabricated on a ${16}\,\mathrm{nm}$-thick layer of NbN,
deposited with an AJA sputtering system onto an Si wafer with
${300}\,\mathrm{nm}$-thick SiO2 thermal oxide. The circuit geometry was
patterned on the NbN layer with electron-beam lithography using ZEP530A resist
and CF4 reactive ion etching. The wide-gate nTron channel constriction widths
were designed to be ${270}\,\mathrm{nm}$ (with an equal-sized gate choke), and
the readout nTron channel width was designed to be ${240}\,\mathrm{nm}$, with
a gate choke width of ${40}\,\mathrm{nm}$. Figure 2a shows an electron
micrograph of a wide-gate nTron patterned on thin-film NbN. Figure 2b is an
electron micrograph of the experimental two-loop shift register circuit, and
the equivalent circuit model is shown in Fig. 2c. The loop kinetic inductors
were designed to be ${100}\,\mathrm{nH}$; the estimated inductance came out to
${60}\,\mathrm{nH}$ (${30}\,\mathrm{pH}$ per square) based on a room
temperature sheet resistance measurement of ${194}\,\mathrm{\Omega}$ per
square. The finished chip was wirebonded to a printed circuit board with off-
chip current bias and shunt resistors, which was mounted to a custom dip
probeButters (2022) and cooled to ${4.2}\,\mathrm{K}$ in a dewar of liquid
helium. The ${2}\,\mathrm{k\Omega}$ bias resistors were used as approximate
current sources to convert an applied voltage to a current through the
nanowire. The hotspot resistance of the switching nTron is small compared to
${2}\,\mathrm{k\Omega}$, so the amount of current through the nanowire given
some applied voltage stays roughly constant regardless of the nanowire state.
The nTron dimensions, inductor sizes and resistor values were selected through
LTSpice simulation using electrical models of nTron devices.Berggren  _et al._
(2018) Eight different shift register circuits were fabricated on a single
${1}\,\mathrm{cm^{2}}$ chip. Two circuits were tested: the circuit presented
in this letter, which used a wide-gate nTron to connect adjacent loops, and a
shift register with a different switch geometry. The alternative design used
current summation into a single two-terminal constriction as a switch, which
performed worse than the design based on the wide-gate nTron, likely due to
leakage current that could flow between loops unimpeded regardless of the
switch state. The results presented in this letter are from the circuit which
used wide-gate nTrons.

The circuit was characterized with clock rates from ${10}\,\mathrm{M}$Hz to
${100}\,\mathrm{MHz}$ and under magnetic fields from ${\pm 1}\,\mathrm{mT}$ to
${\pm 6}\,\mathrm{mT}$, applied orthogonal to the chip surface by a
superconducting magnet mounted on the end of the dip probe. A Keysight PXIe
M3202A (arbitrary waveform generator) and M3102A (digitizer) were used to
verify correct operation of the shift register over a range of signal
amplitudes. This was done by generating multiple ${10}\,\mathrm{kbit}$-long
pseudorandom binary sequences of voltage pulses and measuring the circuit
response. The data and clock input signals encoded digital “1”s with low-duty-
cycle ${2}\,\mathrm{ns}$ FWHM voltage pulses, as can be seen in the top panel
of Fig. 3c. The PXIe chassis controller swept the amplitude of the shift and
readout clock pulses and measured the bit error rate in near real time for
each set of clock amplitudes by comparing the device output with the
${10}\,\mathrm{kbit}$ input sequence. Each spike of the output waveform was
thresholded and digitized, and the result was compared with a copy of the
input signal delayed by a clock period — for each instance where the input and
digitized output differed, the total error count was incremented. A sample
waveform used to calculate the bit error rate is shown in Fig. 3c.

Figure 3:  (a) and (b) are bias margin plots, which show the bit error rate of
the shift register (number of errors out of a ${10}\,\mathrm{kbit}$ random bit
sequence) as a function of shift and readout clock amplitude. The black
regions represent correct operation, with a bit error rate below $10^{-4}$.
The input clock amplitude was fixed at a level that gave optimal margins. (c)
Shows an example trace of the transient response of the circuit with a
${10}\,\mathrm{MHz}$ clock. The voltage spikes on V${}_{\text{shunt1}}$
indicate the storage of a circulating current in the first loop, and spikes on
V${}_{\text{shunt2}}$ indicate transfer of state between adjacent loops.
Traces are vertically offset for clarity.

The plots in Fig. 3a are bias margin plots, which show the bit error rate as a
function of clock pulse amplitude for various clock rates. The dark regions
indicate no measured errors for the ${10}\,\mathrm{kbit}$ sequence, and the
width of the dark regions give the bias margins, defined as the amount of
variation in clock amplitude that is acceptable before the circuit begins to
function incorrectly. The device performed correctly up to a maximum clock
rate of ${83}\,\mathrm{MHz}$, with the bias margins steadily shrinking for
increasing clock frequency. The bias margins of the shift clock were
$\pm$${24}\,\mathrm{\%}$ at $f_{\text{clk}}=10$${}\,\mathrm{MHz}$, but only
$\pm$${7}\,\mathrm{\%}$ for $f_{\text{clk}}=83$${}\,\mathrm{MHz}$. Margins for
the readout clock shrank even more, from $>\pm$${45}\,\mathrm{\%}$ at
$f_{\text{clk}}=10$${}\,\mathrm{MHz}$ to $\pm$${5}\,\mathrm{\%}$ at
$f_{\text{clk}}=83$${}\,\mathrm{MHz}$. As shown in Fig. 3b, the introduction
of a $\pm$${1}\,\mathrm{mT}$ field did not dramatically hurt the margins of
the shift clock: $\pm$${25}\,\mathrm{\%}$ for +${1}\,\mathrm{mT}$ and
$\pm$${20}\,\mathrm{\%}$ for -${1}\,\mathrm{mT}$. The readout clock margins
were unimpacted. However, introduction of a +${6}\,\mathrm{mT}$ field reduced
the margins of the shift clock to $\pm$${4}\,\mathrm{\%}$, and a
-${6}\,\mathrm{mT}$ field (not shown) prevented the device from working with a
bit error rate below 10-3.

The lower half of each bias margin plot exhibits a downwards slope due to the
transfer characteristics of the readout nTron: for a larger gate current, the
required channel current to switch the nTron is lower. Therefore, for a larger
readout clock, the required loop current (and thus shift clock amplitude) is
lower. The abrupt change in bit error rate for readout clock amplitudes below
${30}\,\mathrm{\mu A}$ occurred because the readout clock was not strong
enough to switch the readout nTron. If the final loop current is left
circulating, it prevents the middle nTron from switching again when a shift
clock is applied. The optimal bias region slopes upwards for high readout
clock currents, possibly because of current injection from the readout clock,
which would create a reverse circulating current in the final shift register
loop. This would require the amplitude of the shift clock to be larger to
leave a net-forward circulating current in the final loop that was large
enough for the readout nTron to switch when clocked.

As the frequency of the clock increased, the bias margins for the shift clock
shrank from both sides, and the maximum acceptable readout clock amplitude
decreased dramatically. The $L/R$ time constant to charge a loop with a
circulating current depends on the loop kinetic inductance and the total shunt
resistance. It is plausible that, for higher clock frequencies, the
circulating current does not reach a stable level in the half-period between
the two clock phases, thus producing incorrect behavior. Further
characterization with various shunt resistor and kinetic inductor sizes should
be performed to verify that the decrease in margins is due to this electrical
time constant, and not a thermal process or some other unconsidered effect.
One possible explanation for the large decrease in the bias margins of the
readout clock could be slow thermal reset of the readout nTron gate choke. The
designed critical current of the choke was only ${30}\,\mathrm{\mu A}$, and
overdriving the readout clock significantly above that (e.g.,
${100}\,\mathrm{\mu A}$) would generate a considerable amount of heat.
Residual heat from a readout clock with phase $\phi_{1}$ would suppress the
critical current of the channel, potentially causing the readout nTron to
switch on phase $\phi_{2}$ if it had not cooled sufficiently. Shunting the
gate with a small resistor could limit the heating of the choke, potentially
restoring the bias margin range of the readout clock for high clock
frequencies.

The observed shift in bias margins of ${15}\,\mathrm{\mu A/mT}$ due to the
external magnetic field (Fig. 3b) agrees with the expected loop current
induced by the Meissner effect. However, enhancement of current crowding
around constrictions (such as the sharp corners in the nTron channel as can be
seen in Fig. 2a) due to the Lorentz force is also a plausible explanation, so
further work must be done to understand the mechanism of the external field on
the bias margins of the circuit. If the Meissner effect is the dominant
mechanism, reducing the size of the loop inductor may help improve resilience
against out-of-plane magnetic fields. Instead, if the mechanism is current
crowding enhanced by the Lorentz force, then the nTron geometry would need to
be modified to mitigate this effect.

The energy consumption of the shift register is primarily dictated by the
switching current of the nTrons and kinetic inductance of the superconducting
loops. The total energy consumption of the shift register is estimated to be
${50}\,\mathrm{fJ/bit}$, and is set by the energy stored in each
superconducting loop and the energy dissipated by the clock bias resistors.
When the shift register stores a “1”, approximately ${300}\,\mathrm{aJ}$ of
energy is stored (${100}\,\mathrm{\mu A}$ in a ${60}\,\mathrm{nH}$ loop). Each
shifting operation destroys this circulating current, dissipating the stored
energy through the resistive hotspot in the nTron channel. The energy of this
shift register is dominated by clocking, which dissipates on the order of
${50}\,\mathrm{fJ/bit}$ in the bias resistors (${100}\,\mathrm{\mu A}$ through
${2}\,\mathrm{k\Omega}$ for ${2}\,\mathrm{ns}$). Lowering the operating
current by decreasing the size of the nTrons would reduce the energy
consumption of the shift register substantially.

Decreasing the size of the loop inductor will enable faster, more compact
shift registers due to a reduced $L/R$ loop current time constant. The speed
of the device is fundamentally limited by the hotspot thermal relaxation time,
since the nTron channel must cool between the two clock phases, otherwise
there will not be a superconducting path for the diverted clock if the
previous shift register stage switches (Fig. 1c). An nTron fabricated with NbN
on SiO2 thermal oxide has achieved a thermally-limited switching speed of
${615.4}\,\mathrm{MHz}$, with an estimated thermal relaxation time of
${130}\,\mathrm{ps}$.Zheng  _et al._ (2019) Based on this, a conservative
estimate for the thermal-reset-limited clock frequency of the shift register
is about ${1}\,\mathrm{ns}$, allowing for a ${500}\,\mathrm{MHz}$ two-phase
clock. At this clock rate, a 1 kilopixel array could be read out on two wires
at a maximum photon count rate on the order of ${1}\,\mathrm{Gcps}$. More
thermally conductive substrates can speed up thermal relaxation, Zhang  _et
al._ (2018) potentially offering further speed improvements to nanowire logic.

Because the device we fabricated only accepts serial inputs, it would provide
little practical benefit for large SNSPD arrays, as it is incapable of
reducing wire count. However, modifications to the circuit design can be
incorporated to load data from an entire row of pixels in parallel into the
shift register, as shown in Fig. 4. This proposed modification was designed
and simulated in LTSpice. A simple pixel and destructive-readout memory can be
implemented with an inductively-shunted SNSPD and nTron. A second nTron is
used to store a current in the shift register when the pixel is read out,
conditional on the presence of a current in the pixel inductor. Using this
technique, data from all pixels could be loaded simultaneously into the shift
register. Since the readout of the pixels is destructive, the bias current
through the SNSPD is restored, so the pixels can still detect photons after
the pixel data is loaded into the shift register. There is still per-pixel
dead time set by the frame rate of the imager, since each pixel can only
detect a single photon before it is reset again, but there is no imager-wide
dead time like in a delay-line readout approach.

Figure 4:  Proposed parallel readout scheme. Modifications to the original
shift register (gray) are shown in black. Each pixel consists of an
inductively-shunted SNSPD which is read out with an nTron. When the bias
$I_{\text{bias}}$ is enabled, photon arrivals divert the bias current into the
right branch of the pixel. An additional bias current is applied to the
$\phi_{1}$ clock input of each shift register stage. If the SNSPD bias current
is diverted, a pulse of current $I_{\text{load}}$ applied to the gate of nTron
U1 will cause it to switch, sending a pulse of current to the gate of nTron
U2. The additional bias on the $\phi_{1}$ input will be diverted, forming a
circulating current in the shift register.

In addition to performing detector readout, the simplicity of a shift register
makes it a useful test structure, which could be used to characterize process
yield, as has been done in the past with SFQ logic to evaluate yield for
Josephson junction processes. Nagasawa  _et al._ (2009) More generally, the
inherent ability of shift registers to serialize and deserialize data makes
them a critical function of any large-scale digital system. A superconducting
shift register could help increase the capacity of links between room
temperature and superconducting electronics, and with the introduction of
digital logic, push even more computing into the fridge and enable larger
scale superconducting systems based on nanowires.

The initial stages of this work were sponsored by the Army Research Office
(ARO) under Cooperative Agreement Number W911NF-21-2-0041. The views and
conclusions contained in this document are those of the authors and should not
be interpreted as representing the official policies, either expressed or
implied, of the Army Research Office or the U.S. Government. The U.S.
Government is authorized to reproduce and distribute reprints for Government
purposes notwithstanding any copyright notation herein. The completion of the
data analysis and presentation was funded by the DOE under the National
Laboratory LAB 21-2491 Microelectronics grant. The authors would like to thank
Kyle Richards and Teja Kothamasu for assistance with setting up and using the
Keysight PXIe system. The data that support the findings of this study are
available from the corresponding author upon reasonable request. The authors
have no conflicts of interest to report.

## References

  * Zhao  _et al._ (2018) Q.-Y. Zhao, E. A. Toomey, B. A. Butters, A. N. McCaughan, A. E. Dane, S.-W. Nam, and K. K. Berggren, “A compact superconducting nanowire memory element operated by nanowire cryotrons,” Superconductor Science and Technology 31, 035009 (2018).
  * McCaughan and Berggren (2014) A. N. McCaughan and K. K. Berggren, “A superconducting-nanowire three-terminal electrothermal device,” Nano Letters 14, 5748–5753 (2014).
  * Buzzi  _et al._ (2022) A. Buzzi, M. Castellani, R. A. Foster, O. Medeiros, M. Colangelo, and K. K. Berggren, “A nanocryotron memory and logic family,”  (2022).
  * Gregshammer  _et al._ (1988) P. Gregshammer, H. W. Weber, R. T. Kampwirth, and K. E. Gray, “The effects of high-fluence neutron irradiation on the superconducting properties of magnetron sputtered nbn films,” Journal of Applied Physics 64, 1301–1306 (1988), https://doi.org/10.1063/1.341850 .
  * Polakovic  _et al._ (2020) T. Polakovic, W. Armstrong, V. Yefremenko, J. E. Pearson, K. Hafidi, G. Karapetrov, Z.-E. Meziani, and V. Novosad, “Superconducting nanowires as high-rate photon detectors in strong magnetic fields,” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 959, 163543 (2020).
  * Steinhauer, Gyger, and Zwiller (2021) S. Steinhauer, S. Gyger, and V. Zwiller, “Progress on large-scale superconducting nanowire single-photon detectors,” Applied Physics Letters 118, 100501 (2021), https://doi.org/10.1063/5.0044057 .
  * Zhao  _et al._ (2017) Q.-Y. Zhao, D. Zhu, N. Calandri, A. E. Dane, A. N. McCaughan, F. Bellei, H.-Z. Wang, D. F. Santavicca, and K. K. Berggren, “Single-photon imager based on a superconducting nanowire delay line,” Nature Photonics 11, 247–251 (2017).
  * McCaughan  _et al._ (2022) A. N. McCaughan, Y. Zhai, B. Korzh, J. P. Allmaras, B. G. Oripov, M. D. Shaw, and S. W. Nam, “The thermally coupled imager: A scalable readout architecture for superconducting nanowire single photon detectors,” Applied Physics Letters 121, 102602 (2022), https://doi.org/10.1063/5.0102154 .
  * Wollman  _et al._ (2019) E. E. Wollman, V. B. Verma, A. E. Lita, W. H. Farr, M. D. Shaw, R. P. Mirin, and S. W. Nam, “Kilopixel array of superconducting nanowire single-photon detectors,” Opt. Express 27, 35279–35289 (2019).
  * Butters (2022) B. A. Butters,  _Digital and Microwave Superconducting Electronics and Experimental Apparatus_ , Ph.D. thesis, Massachusetts Institute of Technology, Cambridge MA (2022).
  * Berggren  _et al._ (2018) K. K. Berggren, Q.-Y. Zhao, N. Abebe, M. Chen, P. Ravindran, A. McCaughan, and J. C. Bardin, “A superconducting nanowire can be modeled by using spice,” Superconductor Science and Technology 31, 055010 (2018).
  * Zheng  _et al._ (2019) K. Zheng, Q.-Y. Zhao, L.-D. Kong, S. Chen, H.-Y.-B. Lu, X.-C. Tu, L.-B. Zhang, X.-Q. Jia, J. Chen, L. Kang, and P.-H. Wu, “Characterize the switching performance of a superconducting nanowire cryotron for reading superconducting nanowire single photon detectors,” Scientific Reports 9, 16345 (2019).
  * Zhang  _et al._ (2018) L. Zhang, L. You, X. Yang, J. Wu, C. Lv, Q. Guo, W. Zhang, H. Li, W. Peng, Z. Wang, and X. Xie, “Hotspot relaxation time of nbn superconducting nanowire single-photon detectors on various substrates,” Scientific Reports 8, 1486 (2018).
  * Nagasawa  _et al._ (2009) S. Nagasawa, T. Satoh, K. Hinode, Y. Kitagawa, M. Hidaka, H. Akaike, A. Fujimaki, K. Takagi, N. Takagi, and N. Yoshikawa, “New nb multi-layer fabrication process for large-scale sfq circuits,” Physica C: Superconductivity 469, 1578–1584 (2009), proceedings of the 21st International Symposium on Superconductivity (ISS 2008).