RAG
RAG
**Retriever-Augmented Generation**, or **RAG**, is a type of language generation model that combines pre-trained parametric and non-parametric memory for language generation. Specifically, the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever.  For query $x$, Maximum Inner Product Search (MIPS) is used to find the top-K documents $z\_{i}$. For final prediction $y$, we treat $z$ as a latent variable and marginalize over seq2seq predictions given different documents.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
https://paperswithcode.com/paper/retrieval-augmented-generation-for-knowledge
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks