# Precision spectroscopy of the hydrogen molecular ions: present status of
theory and experiment

V.I.Korobov
Bogolyubov Laboratory of Theoretical Physics,
Joint Institute for Nuclear Research, Dubna, 141980, Russia

###### Abstract

We review recent experiments on HD+ spectroscopy. The reduced proton-deuteron
mass to electron mass ratio, $\mu_{pd}/m_{e}$, is inferred from the comparison
of theory and experiment. Theoretical issues related to the calculations of
the spin structure, which are currently the main limiting factor of
theoretical accuracy, are discussed.

Contribution to the Proceedings of the Conference on Precision Physics and
Fundamental Physical Constants, Slovakia, 2021.

##  1 Introduction

In recent years substantial progress in the precision spectroscopy of the HD +
molecular ion through Doppler-free spectroscopy of ultracold trapped ions in
the Lamb-Dicke regime [1, 2, 3] has been achieved.

This conference proceedings contribution has a threefold interest: we want to
present an analysis of the obtained experimental data and to infer from it the
new value of the reduced mass of the two nuclei: $m_{p}m_{d}/(m_{p}+m_{d})$
normalized to the electron mass.

Then we give an independent derivation of the pure rotational spin-averaged
transition frequency based on the experimental data of the ($G_{1}=1,G_{2}=2$)
manifold of spin states. We keep here the notation for the spin coupling
similar to Ref. [1]: $\mathbf{G}_{1}=\mathbf{s}_{e}\\!+\\!\mathbf{I}_{p}$,
$\mathbf{G}_{2}\\!=\\!\mathbf{G}_{1}\\!+\\!\mathbf{I}_{d}$, and
$\mathbf{F}\\!=\\!\mathbf{G}_{2}\\!+\\!\mathbf{N}$, where $\mathbf{N}$ is the
total orbital angular momentum operator.

Finally we consider the spin interaction HFS Hamiltonian and will show that
the contributions from the _nuclear spin – electron spin – nuclear spin_
interactions of $m\alpha^{6}$ order are by far lower then the present
theoretical and experimental uncertainty.

In our analysis, we will rely on the theoretical results obtained by our group
in [4, 5].

##  2 Nuclear $\mu_{pd}$ reduced mass from the HD+ spectroscopy.

Table 1: Calculated sensitivity coefficients $A_{i}$ for Eq. (2). transition | $(2cR_{\infty})A_{i}\\!\times\\!(\mu/\nu_{0,i})$
---|---
$\nu_{(00)\to(10)}$ |  $-$0.99
$\nu_{(00)\to(11)}$ |  $-$0.49
$\nu_{(30)\to(39)}$ |  $-$0.35
Table 2: Theoretical and experimental spin-averaged transition frequencies (in kHz). CODATA18 values of fundamental constants were used in the calculations. $(N,v)\to(N^{\prime},v^{\prime})$ | theory | experiment
---|---|---
$(0,0)\to(1,0)$ | 1 314 925 752.932(19) | 1 314 925 752.910(17)
$(0,0)\to(1,1)$ | 58 605 052 163.9(0.5) | 58 605 052 164.24(86)
$(3,0)\to(3,9)$ | 415 264 925 502.8(3.3) | 415 264 925 501.8(1.3)

First, we want to make an independent calculation of the reduced mass of the
nuclei in the HD+ molecular ion from the experimental data. In fact, the
spectral transitions in the hydrogen molecular ions may be obtained using the
adiabatic approximation with precision of about five significant digits, that
means that the spectral line frequencies as functions of the fundamental
constants depend mainly on the reduced mass and only very few on other
configurations of the mass parameters. More precisely, we perform numerical
calculations of the nonrelativistic three-body problem using atomic units, and
thus the parameter

$\mu=\frac{\mu_{pd}}{m_{e}}=\frac{m_{p}m_{d}}{(m_{p}+m_{d})m_{e}}$ (1)

and the Rydberg constant which is known with relative precision $1.2\times
10^{-12}$ from the last CODATA adjustment [6] and the only parameters
determining the transition frequencies in HD+. The latter may be used in our
analysis as an exact constant. The uncertainty in other fundamental constants
like $r_{p}$ and $r_{d}$ contribute to the relative level much below
$10^{-12}$.

In order to get the experimental value of $\mu$ we linearize the dependence of
the frequency of the transition line as a function of the fitted parameters:

$\nu_{{\rm
exp},i}(R_{\infty},\mu)=\nu_{0,i}+\frac{dR_{\infty}}{R_{\infty}}\,\nu_{0,i}+(2cR_{\infty})A_{i}\,d\mu.$
(2)

The coefficients $A_{i}$ are obtained numerically (see Table 1) from the
nonrelativistic Schrödinger three-body equation using the Hellman-Feynman
theorem [7]. As is seen from the Table the properly normalized coefficients
correspond to the molecular spectroscopy picture: a rotational transition is
proportional to $1/\mu$ and vibrational transition to $\sqrt{1/\mu}$. Only in
the case of the $(3,0)\\!\to\\!(3,9)$ transition, the coefficient becomes
smaller than 0.5 since the upper state is already in that part of the
potential well where anharmonicity is important.

Results of recent experiments and corresponding theoretical values are
presented in Table 2. Next Table (Table 3) shows our results for the reduced
mass taken from three individual experimental data and the last line is the
statistical average of the three experiments. Relative uncertainty of the
obtained value is $u_{r}(\mu)=1.4\times 10^{-11}$.

Table 3: Reduced mass $\mu$ inferred from the HD+ ion spectroscopy | $\mu$
---|---
CODATA18 | $1223.899\,228\,722(51)$
$(0,0)\to(0,1)$ | $1223.899\,228\,743(16)_{\rm exp}(17)_{\rm th}$
$(0,0)\to(1,1)$ | $1223.899\,228\,707(17)_{\rm exp}(17)_{\rm th}$
$(3,0)\to(3,9)$ | $1223.899\,228\,730(04)_{\rm exp}(17)_{\rm th}$
| $1223.899\,228\,730(04)_{\rm exp}(17)_{\rm th}$

If we combine this result with the high precision mass spectrometry experiment
carried out by Myers’ group at Florida State University [8]:

$m_{d}/m_{p}=1.999\,007\,501\,274(38),$

we arrive at new values for the proton-electron and deuteron-electron mass
ratios:

$m_{p}/m_{e}=1836.152\,673\,476(44),\qquad
m_{d}/m_{e}=3670.482\,967\,763(88).$

##  3 Yet another determination of the spin-averaged frequency for the
$(N\\!=\\!0,v\\!=\\!0)\to(1,0)$ transition

In the first experiment at Düsseldorf [1] six HFS transitions were measured.
Three of them belong to the manifold of the same spin structure and different
coupling with orbital angular momentum $N$. Thus the weighted sum of the three
states

$\nu_{\rm spin}=\frac{1}{15}\sum_{F=1}^{3}(2F+1)f^{\rm exp}(122\to 12F)-f^{\rm
exp}_{\rm spin-av}$ (3)

should not depend on the HFS couplings with $\mathbf{N}$ and may be calculated
from theory [9] using the spin-spin coupling constants $E_{4}$ and $E_{5}$
only, which are accurate up to six significant digits. Thus the corresponding
theoretical value for this shift is

$\nu_{\rm spin}^{\rm theor}=1971.759(2),$

and the spin-averaged frequency for the $(N\\!=\\!0,v\\!=\\!0)\to(1,0)$
transition deduced from this analysis is equal to

$f^{\rm exp}_{\rm spin-av}=1\,314\,925\,752.892(25)_{\rm exp}(2)_{\rm
theor,spin}\mbox{ kHz},$ (4)

which is in a good agreement with the previous _composite frequency_ result.
On the other hand, this trick eliminates the critical inaccuracy in the HFS
spin-orbit coupling coefficients.

##  4 The nuclear spin-spin interaction

The nuclear spin-spin interaction,

$\delta\left(\mathbf{I}_{p}\\!\cdot\\!\mathbf{I}_{d}\right),$ (5)

via the second-order electron coupled interactions has been considered by
Ramsey in [10]. As was discussed in [10], the largest contribution comes from
the _electron spin–nuclear spin_ interaction. In our case it may be expressed
as

$\Delta E_{s_{N}s_{N}}^{(0)}=2\left\langle
H_{ss;p}^{(0)}Q(E_{0}-H_{0})^{-1}QH_{ss;d}^{(0)}\right\rangle,$ (6)

where

$H_{ss;a}^{(0)}=-\frac{16\pi}{3}\mu_{e}\frac{\mu_{a}}{I_{a}}\mu_{0}\mu_{N}(\mathbf{s}_{e}\\!\cdot\\!\mathbf{I}_{a}).$

Then using

$(\boldsymbol{\sigma}_{e}\\!\cdot\\!\mathbf{I}_{p})(\boldsymbol{\sigma}_{e}\\!\cdot\\!\mathbf{I}_{d})=(\mathbf{I}_{p}\\!\cdot\\!\mathbf{I}_{d})+i\boldsymbol{\sigma}_{e}[\mathbf{I_{p}}\\!\times\\!\mathbf{I_{d}}]$

one may get a new term into the effective HFS Hamiltonian, where the nuclear
spin-spin interaction appears explicitly. However, in our case it is more
convenient to keep the interaction in the form:

$H_{N\\!N}=\delta_{N\\!N}\bigl{[}(\mathbf{s}_{e}\mathbf{I}_{p})(\mathbf{s}_{e}\mathbf{I}_{d})+(\mathbf{s}_{e}\mathbf{I}_{d})(\mathbf{s}_{e}\mathbf{I}_{p})\bigr{]}/2.$
(7)

Then performing the numerical calculations, which average the spatial part of
Eq. (6), we obtain for the $(N\\!=\\!1,v\\!=\\!0)$ state of HD+ ion that the
coefficient

$\delta_{N\\!N}=117\mbox{ Hz}.$

This leads to a shift in the energy of an individual HFS state to less than 27
Hz. In its turn, for the measured HFS transition energies of the pure
rotational transition [1], the corresponding correction is less than 2 Hz,
which may be neglected at present.

##  5 Conclusion

In conclusion we want to state that the new precision spectroscopy data may be
used for determination of the proton-electron and deuteron-electron mass
ratio, which provides a three and two-fold improvement with respect to
CODATA18 values. Next we have shown that inclusion of the nuclear spin-spin
interaction into the HFS Hamiltonian does not solve the problem of some
discrepancy between experiment and theory in the hyperfine structure of the
HD+ ion.

## Acknowledgements

This work was done in collaboration with Laurent Hilico, Jean-Philippe Karr,
Mohammad Haidar (LKB), and Zhen-Xiang Zhong (Wuhan, WIPM CAS) that is
gratefully acknowledged.

## References

  * [1] S. Alighanbari, G.S. Giri, F.L. Constantin, V.I. Korobov, and S. Schiller, Precise test of quantum electrodynamics and determination of fundamental constants with HD+ ions. Nature 581, 152 (2020).
  * [2] S. Patra, M. Germann, J.-Ph. Karr, M. Haidar, L. Hilico, V.I. Korobov, F.M.J. Cozijn, K.S.E. Eikema, W. Ubachs, and J.C.J. Koelemeij, Proton-electron mass ratio from laser spectroscopy of HD+ at the part-per-trillion level. Science 369, 1238 (2020).
  * [3] I. Kortunov, S. Alighanbari, M.G. Hansen, G.S. Giri, S. Schiller, and V.I. Korobov, Proton-electron mass ratio by high-resolution optical spectroscopy of ion ensemble in the resolved-carrier regime. Nature Phys. 17, 569 (2021).
  * [4] V.I. Korobov, L. Hilico, and J.-Ph. Karr, Fundamental transitions and ionization energies of the hydrogen molecular ions with few ppt uncertainty. Phys. Rev. Lett. 118, 233001 (2017).
  * [5] V.I. Korobov and J.-P. Karr, Rovibrational spin-averaged transitions in the hydrogen molecular ions. Phys. Rev. A 104, 032806 (2021).
  * [6] E. Tiesinga, P.J. Mohr, D.B. Newell, and B.N. Taylor, CODATA recommended values of the fundamental physical constants: 2018. Rev. Mod. Phys. 93, 025010 (2021).
  * [7] S. Weinberg, _Lectures on Quantum Mechanics_. Cambridge University Press, Cambridge UK, 2013.
  * [8] D.J. Fink and E.G. Myers, Deuteron-to-Proton Mass Ratio from the Cyclotron Frequency Ratio of H${}_{2}^{+}$ to D+ with H${}_{2}^{+}$ in a Resolved Vibrational State. Phys. Rev. Lett. 124, 013001 (2020).
  * [9] J.-Ph. Karr, M. Haidar, L. Hilico, Zhen-Xiang Zhong, and V.I. Korobov, Higher-order corrections to spin-spin scalar interactions in HD+ and H${}_{2}^{+}$. Phy. Rev. A 102, 052827 (2020).
  * [10] N.F. Ramsey, Electron Coupled Interactions between Nuclear Spins in Molecules. Phys. Rev. 91, 303 (1953).


Serving Deep Learning Model in Relational Databases

Alexandre Eichenberger^c, Qi Lin^a, Saif Masood^a, Hong Min^c, Alexander Sim^e,
Jie Wang^b, Yida Wang^b, Kesheng Wu^e, Binhang Yuan^d, Lixi Zhou^a, Jia Zou^a Authors are ordered alphabetically; Jia Zou<EMAIL_ADDRESS>is the corresponding author.

$^a$ Arizona State University, $^b$ Amazon, $^c$ IBM T. J. Watson Research Center,
$^d$ Hong Kong University of Science and Technology, $^e$ Lawrence Berkeley National Lab

Serving deep learning (DL) models on relational data has become a critical requirement across diverse commercial and scientific domains, sparking growing interest recently. In this visionary paper, we embark on a comprehensive exploration of representative architectures to address the requirement. We highlight three pivotal paradigms: The state-of-the-art DL-Centric architecture offloads DL computations to dedicated DL frameworks. The potential UDF-Centric architecture encapsulates one or more tensor computations into User Defined Functions (UDFs) within the database system. The potential Relation-Centric architecture aims to represent a large-scale tensor computation through relational operators. While each of these architectures demonstrates promise in specific use scenarios, we identify urgent requirements for seamless integration of these architectures and the middle ground in-between these architectures. We delve into the gaps that impede the integration and explore innovative strategies to close them. We present a pathway to establish a novel database system for enabling a broad class of data-intensive DL inference applications.

<concept_desc>Do Not Use This Code, Generate the Correct Terms for Your Paper</concept_desc>
<concept_desc>Do Not Use This Code, Generate the Correct Terms for Your Paper</concept_desc>
<concept_desc>Do Not Use This Code, Generate the Correct Terms for Your Paper</concept_desc>
<concept_desc>Do Not Use This Code, Generate the Correct Terms for Your Paper</concept_desc>

§ INTRODUCTION

Recently, key applications emerged from the desire to nest database queries with deep learning inferences, including but not limited to the following categories:

$\bullet$ Commercial applications such as credit card fraud detection, personalized recommendation, customer service chatbots, and anti-money laundering (AML). These use cases increasingly rely on deep learning <cit.>, and are latency-critical. In these cases, transaction data, order data, and customer profiles are usually managed by a relational data management system (RDBMS or DB), and subject to SQL queries for updates and analytics.

$\bullet$ Scientific applications also benefit from the intriguing capabilities of integrating deep learning and data management for automatic optimization, efficient surrogate models, and self-driving lab experiments<cit.>. For example, real-time control coupled with material modeling requires fast access to databases of pre-computed results to trigger model inferences. <cit.>.

$\bullet$ AI-enhanced autonomous database systems increasingly use DL <cit.>. In order to minimize the runtime overheads brought by AI techniques, it is critical to reduce the latency of DL inferences for these techniques.

The DL models in these workloads range from simple models such as fully connected neural networks (FCNN) and convolutional neural networks (CNN), to complex models such as Transformer (e.g., BERT <cit.>) and recommendation models (e.g., deep learning recommendation model (DLRM) <cit.>), as well as the foundation models <cit.>, including large language models (LLMs) <cit.>.

The state-of-the-art architecture for inferences over database data offloads the inference computations to decoupled DL runtimes, as illustrated in Fig. <ref>a. For example, Amazon Redshift offloads the inference to SageMaker, Microsoft Raven <cit.> offloads the inference from Microsoft SQLServer to the ONNX runtime <cit.>, and Google BigQuery offloads the inference to TensorFlow <cit.>. In addition, PostgresML <cit.> offloads the inference of language models to HuggingFace <cit.>. This is the state-of-the-art architecture.

However, there are two potential architectures as alternatives, though they may not have gained enough attention yet.

$\bullet$ The potential UDF-Centric architecture encapsulates inference logic as UDFs within RDBMS, as illustrated in Fig. <ref>b.  However, existing systems such as VerticaML <cit.> and PostgresML <cit.> mostly use UDF-encapsulation for traditional ML such as XGBoost and logistic regression. There are few existing implementations of UDF-centric systems for DL, although DL offers diverse libraries on CPU/GPU <cit.> for implementing the UDFs.

$\bullet$ The potential Relation-Centric architecture, as illustrated in Fig. <ref>c, represents a model parameter tensor as a relation, i.e., a collection of tensor blocks, and breaks down a tensor operator into multiple relational operators that nest with fine-grained UDFs. This approach has been advocated by SystemsML <cit.> and Tensor Relational Algebra  <cit.> for scaling to massive tensor operations. However, this architecture hasn't been widely adopted for serving state-of-the-art DL models.

Overview of the three pivotal architectures and our vision

In Sec.<ref>, we ascertain that none of the paradigms can entirely dominate all applications, as each possesses unique strengths catering to distinct use scenarios. In addition, there are various hybrid solutions, such as wrapping some of the linear algebra operators into UDFs and breaking down some into relational operators, which haven't been discussed by existing literature. Hence, we advocate for a new in-database inference system, as depicted in Fig.<ref>d, to harmoniously encompass these architectures and middle grounds.
However, during our investigation, we observed that existing database systems lack adequate support for this envisioned system, resulting in multiple critical performance gaps:

$\bullet$ Existing RDBMS has inadequate serving capabilities. Popular DL serving techniques, such as dynamic batching <cit.>, retrieval of inference results/contexts through nearest neighbor indexes and vector databases <cit.>, and pipelining of model layers <cit.>, play a vital role in serving DL models effectively. The absence of these functionalities may significantly limit the UDF-Centric and Relation-Centric approaches. However, not all serving techniques are compliant with RDBMS. (Sec. <ref>)

$\bullet$ Despite the presence of existing works in co-optimizing linear algebra and relational algebra computations, such as Raven IR <cit.>, LingoDB <cit.>, Daphne IR <cit.>, and TRA <cit.>, none of these approaches can dynamically unify the DL-Centric, UDF-Centric, Relation-Centric, and the middle grounds in-between for an inference query or a subgraph of the query. There also lacks a unified resource management framework, as DBs, DL runtimes, and DL libraries all have their own parallelism mechanisms, which may conflict. (Sec. <ref>)

$\bullet$ A unified storage management framework for both relational data and tensor data is currently missing. These data types exhibit disparate characteristics, including varying error tolerance levels, distinct access patterns, and different preferences for data locations (e.g., CPU vs. GPU), indexing, and caching schemes. (Sec. <ref>)

Position: In this work, we advocate for a novel database system that bridges the identified gaps, providing support and seamless integration for the DL-Centric, UDF-Centric, and Relation-Centric architectures as well as various middle grounds in between these architectures.

The key contributions include:

(1) We identify the significance of integrating diverse architectures and the middle grounds in-between (or hybrid solutions) into a novel database system with native deep learning model serving and inference query processing.

(2) We pinpoint the performance gaps in legacy RDBMS design along multiple dimensions such as query compilation and optimization, resource management, storage management, and the integration of model serving techniques, which we believe to be the foundation for enabling high-performance in-database inference. We delve into the associated challenges and opportunities for addressing these gaps.

(3) We also discuss other potential components that could play a role in the ecosystem, broadening the scope of possibilities.

$\bullet$ Identify the Challenges and Opportunities  to close the gaps. We examined both the legacy design and emerging solutions along the following aspects:

[enhanced jigsaw,breakable,
title after break=,

(1) Query Compilation (Sec. <ref>). We discussed the adaptive representation of deep learning inference computations, efficient and automatic optimization for queries that involve large-scale models, dynamic models, and similar models, and the integration of code generation and vectorization.

(2) Query Execution (Sec. <ref>). We discussed executing UDFs that invoke external libraries with unified threading and memory management and implementing popular model-serving techniques such as caching of inference results, dynamic batching, and model partitioning within the RDBMS.

(3) Storage and Memory Management (Sec. <ref>). We discussed the unified storage and caching of relational data, tensor data, and vector data for new accuracy-aware query optimizations, e.g., approximation of the tensor/vector data for compression.

(4) Transaction Management (Sec. <ref>). We discussed the dynamic batching of transactions that involve inference queries.

* Representation of Deep Learning Inference Computations in RDBMS. (Sec. <ref>) There are abundant works representing linear algebra computations in RDBMS, falling into two categories: (a) Relation-centric representation <cit.>, such as Tensor Relational Algebra <cit.>, and (b) a straight forward UDF-centric representation. These two representations are illustrated in Fig. <ref>b and Fig. <ref>c respectively. In Sec. <ref>, we provide a comparative analysis of these two approaches as well as the aforementioned decoupled  approach for inference computations.  The results demonstrated that none of the approaches can work well for DNN inference at all scales, which suggests a potential hybrid and adaptive solution.

* Unified Compilation of  Deep Learning and Relational Processing. Recent works, such as Daphne <cit.> and LingoDB <cit.>, propose to encapsulate the relational processing into a Multi-Level Intermediate Representation (MLIR) dialect <cit.>, so that, by leveraging abundant existing MLIR dialects for deep learning <cit.>, they can unify relational and deep learning processing in the MLIR compilation framework. However, such design is based on the assumption that the deep learning computation is the performance bottleneck, which is not the case for many inference scenarios <cit.>. In this work, we will compare the MLIR-based approach to other unified compiler designs, which are more closely integrated with existing RDBMS and vectorized operators (e.g., operators from libraries such as Eigen, InterlMKL, CuBlas, etc.) for more efficient relational processing and lower query compilation overheads.

* Inference Query Execution Runtime (Sec. <ref>). Existing query execution engine is ill-suited for running inference queries. First, existing systems lack unified resource management for UDFs and relational processing. For example, for ML processing, the UDFs may invoke external libraries such as Eigen <cit.>, which relies on underlying libraries such as OpenMP <cit.> for parallelization. In this case, if the RDBMS is unaware of how many threads the UDF will start, it would be hard to avoid the context-switching overheads. Second, existing systems tend to incur a significant amount of repeated processing across inference queries. For example, similar inference queries that involve the same DNN models can share the execution plan and even the intermediate datasets with each other. We discussed opportunities for addressing these problems.

* Serving in RDBMS (Sec. <ref>).
Existing high-performance model serving systems, such as Clipper <cit.>, Nexus <cit.>, Pretzel <cit.>, TensorFlow Serving <cit.>, TorchServe <cit.>, etc, adopted a broad class of serving techniques, such as inference result caching <cit.>, dynamic batching <cit.>, and partitioning and packing models to GPUs <cit.>. We discussed the challenges and opportunities to implement these techniques in RDBMS. For example, we may leverage the nearest neighbor indexing (as widely supported in vector databases <cit.>) to accelerate the retrieving of inference results from a cache. Dynamic batching can be implemented on top of vectorized query processing <cit.>. Model partitioning and packing require adapting the database query optimizer and scheduler.

* Storage and Memory Management (Sec. <ref>). Different from classical RDBMS, which are designed for tabular data, to support native inference queries, the RDBMS also needs to manage features, embedding vectors, model parameter tensors. Building indexing for these data could be time-consuming. Storing all such datasets and their indexes could incur significant overheads and complicate the buffer pool management. Therefore, it is important to design new compression, indexing, and caching mechanisms, see our prior work <cit.>.

* Transaction Processing and ACID (Sec. <ref>). Involving inference functions within transactions poses new challenges
for transaction management. It is important to batch inference requests for serving the same model in order to fully utilize the hardware resources and maintain transaction throughput. Therefore, buffering transaction requests that involve the same model, within the response time SLA bound, is highly desired. However, such buffering may lead to re-ordering of transaction requests and cause fairness issues. Therefore, it is promising to design a look-ahead scheduling mechanism to execute those inference operations asynchronously with the processing of their corresponding transactions, while ensuring ACID and fairness.

We believe our comprehensive and in-depth discussion along above key aspects of DNN inference query processing is helpful for building the next generation database systems that serves data-centric AI processing by bringing the computations closer to data. Our work may trigger various future research that requires low-latency/data-intensive intelligent applications.

§ GAP IDENTIFICATION AND VISION

Performance Gap Identification.

None of the existing pivotal architectures perform optimally across all scenarios.

Regarding latency, the DL-Centric architecture incurs additional latency for transferring data from/to the RDBMS. For example: a Tier 1 bank in the US running its core system on an IBM zSystems platform has deployed an off-platform AI-based fraud detection system. Due to latency issues, the bank runs only very high-risk transactions through the AI system. Most transactions are run through rules-based scoring with lower accuracy <cit.>. As illustrated in Fig. <ref> and Tab <ref>, for workloads with simple models such as small convolutional networks (e.g., DeepBench-Conv1), or large size of inference data (e.g., Amazon-14K-FC requires $594,540$ features), the data transfer overheads dominate. Conversely, the Relation-Centric and UDF-Centric architectures eliminate such data transfer. However, their inference latency for complex models could be worse than the DL-Centric counterpart. In Table <ref>, if we deduct the data transfer latency, as illustrated in Fig. <ref>, from the end-to-end latency for DL-Centric platforms, the DL-Centric platforms outperform UDF-Centric and Relation-Centric in most of the cases except those Out-of-Memory (OOM) cases. One reason is that RDBMS lacks the serving techniques, as discussed in Sec. <ref>.

In terms of scalability, the linear algebra computations in the Relation-Centric architecture can effortlessly scale up to accommodate massive tensors that exceed memory limitations. This capability stems from its ability to split a single tensor into blocks and spill partial tensor blocks to disk, leveraging the RDBMS buffer pool management. In contrast, most of today's DL runtimes and libraries encounter out-of-memory (OOM) errors in such situations, as illustrated in Tab. <ref>. That's because these libraries consider a tensor to be the smallest unit of objects and there is no data management layer to split a tensor into blocks and run tensor operations on top of a collection of blocks with only a portion of the blocks residing in memory and the rest of blocks in memory.

Fully Connected (FC) Models (one hidden layer)

Model     Number of Features/Number of Neurons (hidden layer size)/Outputs

Keras-Bert-Encoder-FC   $76/3,072/768$

Amazon-14k-FC   $597,540/1,024/14,588$

Convolutional Models

Model     Image/Input Shape     Kernel Shape

DeepBench-CONV1   $112\times112\times64$   $64\times64\times1\times1$

LandCover-DeepBench-CONV3   $2500\times2500\times3$   $2048\times3\times1\times1$

End-to-End Latency Comparison (unit: seconds), UDF-Centric and Relation-Centric representations are implemented on netsDB  <cit.>$^{\ref{footnote:netsdb}}$. We compare these to the DL-Centric counterparts using Pytorch-2.0 and TensorFlow-2.0. Experiments were run on an Amazon EC2 r4.2xlarge instance with eight CPU cores and $62$ GB memory. The architecture of these models or kernels can be found in Tab. <ref> and Tab. <ref>. The state-of-the-art ConnectorX <cit.> was used as the Postgres connector.

Model     Batch Size     Relation-Centric     UDF-Centric     DL-Centric (Postgres+ TF)     DL-Centric (Postgres+ Pytorch)

2*Keras-BERT-Encoder-FC    $1000$    $12.6$    $0.5$   $\textbf{0.4}$   $\textbf{0.4}$

$8000$   $13.4$   $\textbf{0.6}$   $1.5$   $1.0$

2*Amazon-14k-FC    $1000$   $58.6$   $60.4$   $34.6$   $\textbf{22.6}$

$8000$   $\textbf{407.2}$   OOM   OOM   OOM

5*DeepBench-CONV1    $64$   $8.9$   $\textbf{0.4}$   $2.8$   $2.8$

$128$   $10.8$   $\textbf{0.8}$   $5.2$   $5.3$

$256$   $14.8$   $\textbf{1.6}$   $10.0$   $10.2$

$512$   $24.4$   $\textbf{3.3}$   $18.9$   $19.4$

$1024$   $42.5$   $\textbf{6.5}$   $38.3$   $38.9$

$1$   $36.8$    OOM     $\textbf{9.9}$     OOM

$2$   $\textbf{45.2}$    OOM     OOM     OOM

$4$   $\textbf{46.6}$    OOM     OOM     OOM

$8$   $\textbf{49.6}$    OOM     OOM     OOM

$16$   $\textbf{76.9}$    OOM     OOM     OOM

[1]NetsDB (https://github.com/asu-cactus/netsdb) is developed on top of PlinyCompute <cit.>, which is an object-oriented relational database system.

Breakdown of Normalized Latency of the Model Inference Process into Data Transfer Latency (Blue) and Inference Latency (Green) for different types of DL models. All experiments ran models or kernels from popular models on an Amazon EC2 r4.2xlarge instance with eight CPU cores and $62$ GB memory. The architecture of these models or kernels can be found in Tab. <ref> and Tab. <ref>. The data transfer latency is measured as the latency of reading one batch of data from Postgres using ConnectorX <cit.>. We used Pytorch 2.0 and TensorFlow 2.0. The numbers following the model name indicate the inference data batch sizes.

Middle Grounds in-between Pivotal Architectures. We also identified a hybrid representation may achieve significantly better results. Taking Amazon-14k-FC for example, the most memory-intensive operation is the matrix multiplication between the input batch (of shape $batch\_size \times feature\_size$) and the model weight matrix of the first layer (of shape $feature\_size \times hidden\_layer\_size$).

Other operations are simple and involve significantly smaller tensors. Using Relation-Centric representation for small matrix multiplication or addition will introduce multiple joins, aggregations, and matrix reblocking operations, which are pipeline breakers. Such heterogeneous characteristics among operators motivate a hybrid representation: only apply the Relation-Centric representation to this large-scale matrix multiplication operation, while the rest of the computations are represented in UDF-Centric representation.

We've found the hybrid representation achieved more than $2.4\times$ speedup compared to the pure Relation-Centric representation and $2.5\times$ speedup compared to the pure UDF-Centric, reducing the latency from $58.6$ seconds to $23.9$, when batch size is $1,000$. (UDF-Centric representation failed when batch size increased to $8,000$.)

A summary of the specific use scenarios for each of the three pivotal paradigms can be found as follows:

[enhanced jigsaw,breakable,
title after break=,

$\bullet$ DL-Centric Architecture: Ideal for scenarios where leveraging unique techniques in DL systems is crucial. It is particularly effective for applications bounded by inference computations with insignificant data transfer overheads.

$\bullet$ UDF-Centric Architecture: Well-suited for scenarios that require inferences over large batches of data and/or high-dimensional data with relatively simple models. By "simple models," we refer to models that do not require non-compliant serving capabilities within RDBMS (Sec. <ref>). Additionally, it is essential that each operator in such models can fit into memory.

$\bullet$ Relation-Centric Architecture: Excels in performing large-scale tensor computations with cost constraints, e.g., handling tensors that exceed memory within a single CPU node. In addition, through experiments, we also found that Relation-Centric architecture could outperform UDF-Centric architecture for large-scale matrix multiplications that fit memory because it reduces memory footprint and cache misses by allowing each thread to process a few blocks each time.

Furthermore, inference applications exhibit diverse natures. As illustrated in Fig. <ref>, different batch sizes and models lead to divergent inference/data-transfer ratios. In addition, as illustrated in Tab. <ref>, not only do different applications have disparate characteristics, but also the same application may experience fluctuations in event frequency, necessitating different batch sizes and models at different time points. Given such variability, we advocate for seamlessly integrating these architectures into one cohesive system.

Diverse Requirements of Example Applications

Application     Event Frequency     Desired Latency     DL Models     Batch Size     Functionalities

Customer chatbot <cit.>     10-10K per sec     <300ms     Jasper, Bert, Tacotron2, etc.     1-1K     context retrieving, security & privacy

Fraud detection & AML <cit.>     1K-1M per sec     <100 ms     LSTM, CNN, GNN, etc.     100-100K     transaction, security & privacy, trigger

Recommendation <cit.>     1K-100K per sec     <50ms     DLRM, DeepFM, NCF, etc.     50-5K     security & privacy

Vision: Enabling a Novel Database System. As shown in Fig. <ref>d, the novel database connects to DL runtimes, linear algebra libraries, and DNN libraries simultaneously.  It consists of four components: (1) A collection of in-DB model serving techniques; (2) A unified cost-based optimizer so that any inference computations may be offloaded to DL runtime, encapsulated in UDFs, or represented as relational operators, capitalizing on the flexibility brought by a unique IR design. (3) Resource management for the threading and memory allocation across DB operations, DL runtime operations, DNN library operations, and linear algebra operations. (4) Storage management for tensor blocking and co-optimization (e.g., co-partitioning) with relational data. These components seamlessly integrate and streamline efficiency across all architectural paradigms, including the middle grounds.

Expected Benefits. The proposed novel database design has numerous advantages over state-of-the-art architecture and other potential architectures. First, it enhances the productivity of developing applications that desire DB functionalities such as transaction management and access control and shortens the time to market. Second, it delivers superior performance for a broad class of inference applications at different scales through unified query optimization. Third, it ensures overall storage efficiency through unified storage management. Last but not the least, it effectively coordinates resources across the database, the DL runtime, the DL libraries, and the linear algebra libraries, reducing application deployment costs.

Inference Latency Comparison (unit: seconds)
model type     #inputs     netsDB-rel     netsDB-UDF     TF     PyTorch
2*1.5cmKeras-BERT-Encoder-FC   $1000$   $12.6$   $0.4$   $0.3$   $\textbf{0.1}$

$8000$   $13.4$   $0.6$   $1.2$   $\textbf{0.3}$
2*1.5cmAmazon-14k   $1000$   $48.1$   $7.8$   $15.9$   $\textbf{4.7}$

$8000$   $\textbf{407.2}$   failed   failed   failed

$64$   $8.9$   $0.4$   $0.3$   $\textbf{0.2}$

$128$   $10.8$   $0.8$   $0.5$   $\textbf{0.3}$

$256$   $14.8$   $1.6$   $1.0$   $\textbf{0.7}$

$512$   $24.4$   $3.3$   $1.9$   $\textbf{1.3}$

$1024$   $42.5$   $6.5$   $3.7$   $\textbf{2.6}$
3*1.5cmLandCover-CONV3   $2$   $\textbf{45.2}$    failed     failed     failed

$4$   $\textbf{46.6}$    failed     failed     failed

$8$   $\textbf{49.6}$    failed     failed     failed

$16$   $\textbf{76.9}$    failed     failed     failed

§.§ Fully Connected Layer

As illustrated in Fig. <ref>, at a dense layer, the weight tensor is multiplied with the input feature vector (or a tensor that represents a batch of inputs). The output is then added with the bias vector. The output tensor is applied with an activation function ($\sigma$), such as  and . Then the final output is sent to the next layer, which will be computed similarly.

Illustration of Dense (Fully-Connected) Layers

§.§.§ Relation-Centric Representation

As illustrated in Fig. <ref>, a dense layer consists of a matrix multiplication, a matrix addition, and an activation. As discussed and demonstrated in our previous works <cit.>, in RDBMS, a matrix multiplication can be computed as a  query followed by an  query. A matrix addition can be computed as a . An activation can be computed as a .

§.§.§ UDF-Centric Representation
All computations, such as matrix multiplication, matrix addition, and activation, are encapsulated in one user defined function. A  operator will apply the function to each feature vector in the input set.

§.§.§ Evaluation Methodology
We first determine a range of the depth (e.g., number of layers) and width (e.g., number of neurons at each layer) of dense layers  in different real-world workloads, including DLRM, DeepSpeech, BERT, VGG$16$, ResNet, and other small scale models that we found in Kaggle. Then, we compare various representations in netsDB, as well as our implementations in TensorFlow and PyTorch with GPU acceleration.

§.§.§ Results

The results are illustrated in Tab. <ref>.

Representations for Fully Connected Layers (FC) in one AWS r4xlarge instance (Unit:Seconds)
FC-type   UDF   Relation   PyTorch   TensorFlow
DLRM-RMC3/DenseFC   $\textbf{3.2}$   $55$   0.1   0.2
VGG-19/FC   $\textbf{12.8}$   $56$   0.3    0.8
keras-BERT/EncoderFC   $\textbf{4.2}$   $55$   0.1   0.3
MLP-mixer-Huge-channel-mixing   $\textbf{6.2}$   $37$   0.2   0.6
Amazon-14k   Failed   $\textbf{208}$   Failed   Failed

§.§ Convolutional Layer

As illustrated in Fig. <ref>, at a convolutional layer, a filter shifts across the input feature tensor. At each shifting step, the filter is mapped to a restricted area in the input tensor, called as a receptive field, then the sum of all elements of a Hadamard product of the receptive field and the filter is computed and added to a bias value. The result becomes a value in the corresponding position of an output feature map.

Illustration of the Convolutional Layer

§.§.§ Relation-Centric Representation

As illustrated in Fig <ref>, a
convolutional layer operation involves computing a set of Hadamard products. As shown in [reference to memfusion paper], this operation can be converted to a large matrix multiplication operation. In the Relation-Centric approach, this large matrix is stored as a set of matrix blocks, and multiplication is performed on the matrix blocks using a  followed by an .

§.§.§ UDF-Centric Representation

The convolution operation using Eigen spatial convolution or call to a library method such as Aten (PyTorch) can be encapsulated in one user defined function.

§.§.§ Evaluation Methodology

We took widely used models considered in DeepBench CNN inference benchmark. Then, we compare various representations in netsDB, as well as our implementations in TensorFlow and PyTorch with GPU acceleration.

Size of CNN in popular models
model type     Image/Input Shape     Kernel Shape
DeepBench-CONV1   $112\times112\times64$   $64\times64\times1\times1$
DeepBench-CONV2   $56\times56\times256$   $128\times256\times1\times1$
DeepBench-CONV3   $7\times7\times512$   $2048\times512\times1\times1$
DeepBench-CONV4   $341\times79\times32\times4$   $32\times32\times5\times10$
RetinaMNIST-DeepBench-CONV3   $1\times1736\times1824\times3$   $2048\times3\times1\times1$
LandCover-DeepBench-CONV3   $1\times2500\times2500\times3$   $2048\times3\times1\times1$

§.§.§ Results

The results are illustrated in Tab. <ref>.

Representations for Convolution (CNN) in one AWS r4xlarge instance (Unit:Seconds)
CNN-type   UDF   Relation   Decoupled-PyTorch   Decoupled-TensorFlow
DeepBench-Conv1   $\textbf{3.6}$   $30$   $0.1$   $0.5$
DeepBench-Conv2   $\textbf{3.6}$   $30$   $0.1$   $0.5$
DeepBench-Conv3   $\textbf{3.7}$   $35$   $0.1$   $0.5$
DeepBench-Conv4   $\textbf{3.6}$   $41$   $0.2$   $0.7$
RetinaMNIST-DeepBench-CONV3   Fail   $\textbf{64}$   Fail   Fail
LandCover-DeepBench-CONV3   Fail   $\textbf{459}$   Fail   Fail

§ DL SERVING TECHNIQUES IN RDBMS

Existing works in model serving systems fall into two categories:

1. Black-box approaches.

Popular model serving systems such as TensorFlow Serving <cit.>, Amazon SageMaker <cit.>,  Microsoft's Azure ML <cit.>, and Rafiki <cit.>,  deploy ML models as black boxes with knobs for tuning.
For example, many of those systems <cit.>
dynamically optimize the batch sizes to balance the accuracy and the latency.

2. White-box approaches.
Nexus <cit.> proposes a bin-packing algorithm to place ML layers to devices and tune batch sizes for each layer. DeepSpeed <cit.> and FlexGen <cit.> offload certain weights to the CPU and even the disk. Other techniques are not limited to early prediction based on cached (intermediate) inference results <cit.>, sharing operators across models <cit.>, and approximate inference  <cit.>.

To discuss the following challenge, we assess the RDBMS compliance with several challenging techniques.

Challenge 1. Which techniques are compliant with and should be incorporated into RDBMS to accelerate the Relation-Centric and UDF-Centric approaches, and how to incorporate these techniques?

3-1. Caching of Inference Results and Contexts. One option is to leverage database indexing to facilitate an inference result cache. The idea is to create a table that records the features or intermediate representations (e.g., embedding vectors) of frequent inference requests and the corresponding prediction results or contexts that are auxiliary to the prediction. We then construct nearest neighbor indexing over the features so that an inference query may efficiently retrieve results or contexts from the cache.

Recently, there emerged vector databases, such as Pinecone <cit.>, Milvus <cit.>, Faiss <cit.>, Qdrant <cit.>, Vespa <cit.>, which store vectors such as embedding vectors for images, documents, and other objects, and build various indexing to facilitate nearest neighbor search, including hierarchical navigable small world (HNSW) <cit.>, locality sensitive hashing (LSH) <cit.>, inverted file indexing (IVF) <cit.>, and product quantization <cit.>.

It is promising to leverage those nearest neighbor indexing within RDBMS to facilitate the fast retrieval of inference results. Such integration also triggers many research opportunities. For example, the query optimizer and cost models must be enhanced to decide whether to fetch the result/context from the cache or directly run the query. In addition, the buffer pool page replacement policy also needs to be improved to consider GPU and coordinate the disparate access patterns of the vector data, the relational data, and various indexes.

In our preliminary experiments, we've found using Faiss' HNSW as indexing, we can reduce the inference latency for a simple convolutional neural network that has two convolutional layers and two fully connected layers on the MNIST dataset by $10.3\times$ speedup, with accuracy dropped from $98.75\%$ to $93.65\%$. This result is promising in providing more latency-accuracy trade-off options for DL applications.

3-2. Model Partition and Pipelining. In DL serving systems, if a model exceeds available memory, it will be partitioned into multiple operators/layers, which will be dispatched to multiple devices based on the costs of the operators and the available resources of the devices <cit.>. Those devices work in parallel, composing a pipeline. A pipeline stage at each device works in a streaming style. It continuously accepts inputs from its upstreaming stage, processes the inputs, and passes the output to the downstream stage.

However, although the "pipelining" concepts in RDBMS also exist, it mostly refers to operator fusion, and the query processing in RDBMS  mainly relies on a different parallelism model. A query will first be optimized logically and physically into a graph of pipeline stages. The last operator in each pipeline stage must be a pipeline breaker, e.g., an exchange/shuffling/materialization operator, such as join hash partitioning, the shuffling at the end of the first phase in aggregation, and writing to a file. In each resource container, at a time, only one pipeline stage is scheduled to run using all resources at the container.  All threads at that stage run the same processing logic on different input splits <cit.>. The assumption is that there are many input splits, and the split size is controlled so that the collection of concurrently processed splits fits the available resources <cit.> in the container. At the end of the pipeline stage, all threads materialize results by writing the output to a file or shuffling the output, which will be processed by the next stage.

These two approaches represent different trade-offs. The pipelining in DL frameworks is subject to the operator size limitation, where an operator or a layer must fit the resource on the corresponding device. The Relation-Centric processing in RDBMS has no such restriction because a large-scale tensor is represented as a collection of blocks that can spill to disks or be distributed to multiple devices. However, the pipelining in DL frameworks tends to be more efficient than the RDBMS parallel processing. First, there is no need to shuffle data globally as required by database join and aggregation. Second, computations can be reused across queries without re-compilation/re-scheduling overheads.

Implementing the DL pipelining mechanism in the UDF-Centric approach is feasible by breaking the model UDF into multiple operator UDFs and deploying those UDFs on different devices following the stream processing paradigm. However, it is challenging to accommodate both batch and stream processing in one system <cit.>.

Therefore, offloading large-scale models to DL frameworks may be reasonable as long as each operator fits memory and the data transfer between the RDBMS and the DL framework is insignificant.

3-3. Dynamic Batching. Batching of model inference requests could balance the latency, throughput, and resource utilization. Each deep learning operation (e.g., a kernel) may desire a different batch size <cit.>. However, in most RDBMS, all vectorized operators fused in one pipeline stage  (i.e., a job stage in SparkSQL) usually have the same batch size.

To address the problem, on the one hand, vectorization and pipelining logic should be enhanced so that each operator can use a different batch size, and the batch size could change over time based on the variability of the available system resources. On the other hand, the RDBMS should provide a mechanism to automatically determine the batch size for each kernel/layer, which may be integrated as part of the query optimization logic or the hyper-parameter tuning component.

Storage-related techniques, such as model compression and offloading tensors to disks, are discussed in Challenge 4 (Sec. <ref>).

§ CO-OPTIMIZATION OF SQL AND TENSOR OPERATIONS

This section discusses the challenges and opportunities in query optimization and resource management for inference queries.

§.§ Query Compilation and Optimization

Challenge 2.  How to design a query compilation framework to dynamically generate the DL-Centric, UDF-Centric, and Relation-Centric architectures for each operator and facilitate co-optimization of relational processing and DL inference computations?

We discuss two options, which represent different opportunities:

Option 1: Multi-level Intermediate Representation (IR).

Multi-Level Intermediate Representation (MLIR) <cit.> is developed by Google and LLVM for integrating multiple levels of IR dialects for machine learning processing.
Recently, LingoDB <cit.> and Daphne IR <cit.> provide an MLIR dialect for relational processing, which can serve as a nice foundation for integrating with existing dialects for AI/ML computations developed by the MLIR community, including high-dimensional tensor abstraction dialects such as Tensor, Arith, Linalg, and high-dimensional vector abstraction such as Vector and SCF, to facilitate cross-optimizations within the MLIR framework. However, a challenging problem is how to co-optimize the DB dialect and ML dialects, and how to glue MLIR and RDBMS runtime for parallel query execution, caching, and indexing <cit.>. These are not discussed in existing works.

Option 2: Adaptive in-database IR. An alternative is to extend existing relational algebra IR to represent linear algebra computations as UDFs or relational operators adaptively. This approach may seamlessly integrate with the RDBMS runtime. For example, we consider an IR, where each node is a traditional relational operator or a model UDF operator. The latter corresponds to a DL model inference computation. Such computation can be lowered to a graph IR, where each node represents a linear algebra operator such as matrix multiply, matrix add, relu, softmax, conv2d, etc.

Both options should support flexible IR transformations that capitalize on the flexibility provided by the proposed system. First, each linear algebra operator can be flexibly converted into relational operators when a tensor exceeds available memory. For example, a large-scale matrix multiply can be replaced by a join followed by an aggregation as illustrated in Fig. <ref>c <cit.>. Second, one or more operators can be offloaded to external DL runtimes, e.g., for pipelining across multiple GPU devices. Third, multiple simple linear algebra operators can be merged into one coarse-grained UDF by invoking libraries, such as CuDNN <cit.>, and IBM zDNN <cit.>, or using code generation <cit.>, to reduce data transfer latency. Most importantly, it is promising to explore novel co-optimization opportunities between SQL query processing and deep learning models. For example, consider a simple end-to-end pipeline that first joins (i.e., concatnates) two separate wide feature datasets $D_1$ and $D_2$ to obtain the entire features and run a DL model on top of the features. If the first layer of the model is an embedding layer or a fully connected layer that will reduce the feature dimensions significantly, we may decompose the corresponding weight matrix $W$ into two submatrices $W_1$ and $W_2$, each of which only processes the features belonging to $D_1$ or $D_2$ respectively, so that $W \times concat(D_1, D_2) = W_1 \times D_1 + W_2\times D_2$. Then, a co-optimization rule is to push down the submatrix multiplication $W_1 \times D_1$ and $W_2\times D_2$ before we perform the join to run the concatenate (i.e., apply a matrix addition to both outputs). It will reduce the intermediate data size if the overall width (total number of features) in the outputs from $W_1\times D_1$ and $W_2 \times D_2$ is significantly smaller than the total number of features in $D_1$ and $D_2$.

Considering the tremendous flexibility, optimizing for complicated inference queries could be expensive. In addition to employing standard techniques such as learning-based optimization <cit.>,  Bayesian optimization <cit.>, and query graph partitioning <cit.>, a promising technique is ahead-of-time (AoT) compilation <cit.>. For example, DietCode <cit.> optimizes for a dynamic model by generating multiple execution plans at compilation time and selecting the best plan at runtime. The RDBMS should be able to identify frequent patterns of dynamic inference queries and leverage AoT compilation to reduce runtime compilation overheads.

§.§ Resource Management

Challenge 3. How to unify the resource management for RDBMS, DL runtimes, and DL libraries invoked from UDFs?

Coordinating the resources between the database runtime and external DL runtimes/libraries that co-locate on the same machine is important and challenging. For example,
while configuring the RDBMS buffer pool sizes, we shall also consider the memory requirements of the DL runtimes. Similarly, in the UDF-Centric approach, the UDF, e.g., invoking OpenBLAS libraries, may rely on OpenMP for parallelism. We must carefully configure the number of threads for the database query processing and OpenMP. Otherwise, significant context switch overheads may occur.  For example, multiple database threads execute the same pipeline stage that fetches a batch of tuples and processes the batch using a sequence of operators, with some operators running simple filtering and projection, and some executing linear algebra operations. Each linear algebra operator may run with a different number of threads leveraging OpenMP. Our experiments found that tuning these threads can improve the performance by up to $20$ times. Existing database hyper-parameter tuning works <cit.> and AutoML hyper-parameter tuning works <cit.> did not consider the heterogeneous nature of these threading configurations of UDF-based linear algebra operators. For example, we may configure the OpenBLAS threading for each linear algebra operator. However, we can only configure the database threading for each pipeline stage that fused many operators. Such heterogeneity made the formulation of the hyper-parameter co-optimization problem challenging. In addition, it is challenging to establish a valid cost model, because it is hard to predict how the operators in one database pipeline thread will overlap with those from other pipeline threads at runtime. Moreover, hyper-parameter search usually requires either significant search latency at the online stage or significant overheads for training surrogate models, reinforcement learning models, or other types of models at the offline stage. Therefore, the data-efficient hyper-parameter search, such as meta-learning, zero-shot, and few-shot learning with generative models and external knowledge, are all promising future directions.

In addition, well-known resource management frameworks for cloud databases, such as Kubernetes <cit.>, Yarn <cit.>, and Mesos <cit.>, were designed for MapReduce. The programming and resource models need to be reconsidered in these frameworks to negotiate resources between DB and the DL runtimes. The recent Apache Submarine project <cit.> targets providing a unified resource management framework for a machine learning workflow. However, these frameworks do not have many automatic configuration capabilities, which should be enhanced to alleviate human efforts in handling complicated end-to-end ML inference pipelines.

§ RELATIONAL DATA AND TENSOR DATA STORAGE CO-OPTIMIZATION

Challenge 4. How to manage the storage and caching of relational, tensor, and vector data in a unified way?

Tensor/vector data possesses unique characteristics. First, different from relational data that must be stored exactly, some errors/approximations in the tensor/vector data may not significantly affect the downstream application accuracy. In addition, similar tensor/vector data may be deduplicated approximately to reduce storage costs and memory footprint <cit.>. Second, such data requires nearest neighbor indexing for fast retrieval. Third, processing such data may benefit from hardware accelerators such as GPU. (However, we also found that for many inference scenarios that involve simple models and small datasets, the overheads of moving data from host memory to GPU memory could outweigh the performance benefits brought by GPU acceleration.)

Therefore, the caching of model data should facilitate error-bounded compression and deduplication, nearest neighbor indexing, and hardware acceleration. For example, the weights that are approximately shared by multiple tensors or the embedding vectors that are frequently accessed should be easily identified and prioritized in the caching hierarchy that considers GPU memory.

Moreover, the storage of the model data should facilitate accuracy-aware query optimization. For example, the storage optimizer may automatically employ compression, such as pruning and quantization, to create multiple versions of the same model with different size, efficiency, and accuracy trade-offs. Then, the query optimizer may dynamically select one version of the model for runtime query processing based on the latency and accuracy requirements defined in the Service Level Agreement (SLA). Introducing accuracy as a new dimension for query optimization may trigger abundant opportunities in multi-objective query optimization. In addition, managing DL models in RDBMS will facilitate the binding of each model and its training datasets (e.g., through the catalog component), which will facilitate model selection based on data similarity <cit.>.

Furthermore,  it is important to consolidate existing tensor partitioning, distributing, and offloading in existing DL libraries <cit.> with the RDBMS storage management for tensor relations (i.e., as in the Relation-Centric approach). For example, the automatic tiling of the tensors in TVM <cit.> and the distributing of the tensor blocks in DISTAL <cit.> could be merged with the physical database design <cit.>. The offloading tensors to disk in FlexGen <cit.> could be integrated with the RDBMS buffer pool management.

Existing vector databases <cit.>, do not have many of the aforementioned features. In addition, it is important to unify the management of both data types for queries involving both tensor and relational data, which requires a new hybrid storage component beyond the vector databases.

§ FURTHER DISCUSSION

The ecosystem extends beyond the components discussed in this vision paper. For instance, ConnectorX <cit.> is a high-performance database connection library designed to reduce data transfer overhead for the DL-Centric architecture. Additionally, AI-Centric database <cit.> converts relational processing to tensor processing and executes queries in deep learning frameworks, leveraging hardware acceleration and compilation/optimization support for tensor computations. TFX <cit.> extends the ML platform with basic data processing capabilities. Although these works are relevant, it is essential to identify and consolidate redundant components to avoid duplication of functionalities, a task that remains for future work.

§ CONCLUSIONS

In this work, we discuss how the RDBMS should be upgraded to support various model-serving applications that rely on RDBMS for data management. First, we identify three pivotal architectures: DL-Centric, UDF-Centric, and Relation-Centric, and we argue that these approaches as well as the middle ground among them should be unified to facilitate the serving of deep learning models on relational data at different scales. We further identify challenges and opportunities in existing RDBMS systems for achieving unified optimization. In summary, we argue for:

$\bullet$ Integrating compliant model serving techniques such as caching of inference results and dynamic batching into RDBMS.

$\bullet$ Adaptive co-optimization of inference computations and relational computations at a fine-grained level, so that any subgraph in the inference query IR has the flexibility to be scheduled as DL-Centric, UDF-Centric, or Relation-Centric.

$\bullet$ Coordination of computational and memory resources for DB, DL runtimes, and lower-level runtimes that support DL libraries to unify the resource management for the future RDBMS ecosystem.

$\bullet$ Renovating the storage, caching, and indexing of tensors/vectors to reduce storage costs, consider data placement in GPU, and facilitate accuracy-aware query optimization.

The work is sponsored by the National Science Foundation (NSF) CAREER Award (Number 2144923), the IBM Academic Research Award, the Amazon Research Award, and a U.S. Department of Homeland Security (DHS) Award (Number 17STQAC00001-03-03).

#1     #1#1#1     #1     #1     #1     #1#1          #1#1

titleIntroduction to Conversational AI.

titleIBM Z Deep Neural Network Library (zDNN).

titleChatbots in consumer finance.

titlePinecone: Vector Database for Vector Search.

titleVespa: The big data serving engine.

[Abadi et al(2015)]
authorpersonMartín Abadi, personAshish Agarwal, personPaul Barham, personEugene Brevdo, personZhifeng Chen, personCraig Citro, personGreg S. Corrado, personAndy Davis, personJeffrey Dean, personMatthieu Devin, personSanjay Ghemawat, personIan Goodfellow, personAndrew Harp, personGeoffrey Irving, personMichael Isard, personYangqing Jia, personRafal Jozefowicz, personLukasz Kaiser, personManjunath Kudlur, personJosh Levenberg, personDandelion Mané, personRajat Monga, personSherry Moore, personDerek Murray, personChris Olah, personMike Schuster, personJonathon Shlens, personBenoit Steiner, personIlya Sutskever, personKunal Talwar,
personPaul Tucker, personVincent Vanhoucke, personVijay Vasudevan, personFernanda Viégas, personOriol Vinyals, personPete Warden, personMartin Wattenberg, personMartin Wicke, personYuan Yu, and personXiaoqiang Zheng. year2015.
titleTensorFlow: Large-Scale Machine Learning on Heterogeneous Systems.

Software available from tensorflow.org.

[Agrawal et al(2000)]
authorpersonSanjay Agrawal, personSurajit Chaudhuri, and personVivek R Narasayya. year2000.
Automated selection of materialized views and indexes in SQL databases. In booktitleVLDB, Vol. volume2000. pages496–505.

[Ali et al(2020)]
authorpersonAhsan Ali, personRiccardo Pinciroli, personFeng Yan, and personEvgenia Smirni. year2020.
Batch: Machine learning inference serving on serverless platforms with adaptive batching. In booktitleSC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, pages1–15.

[Ali et al(2022)]
authorpersonAhsan Ali, personRiccardo Pinciroli, personFeng Yan, and personEvgenia Smirni. year2022.
Optimizing inference serving on serverless platforms.
journalProceedings of the VLDB Endowment volume15, number10 (year2022).

[Aminabadi et al(2022)]
authorpersonReza Yazdani Aminabadi, personSamyam Rajbhandari, personAmmar Ahmad Awan, personCheng Li, personDu Li, personElton Zheng, personOlatunji Ruwase, personShaden Smith, personMinjia Zhang, personJeff Rasley, et al year2022.
DeepSpeed-inference: enabling efficient inference of transformer models at unprecedented scale. In booktitleSC22: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, pages1–15.

[Ardestani et al(2022)]
authorpersonEhsan K Ardestani, personChangkyu Kim, personSeung Jae Lee, personLuoshang Pan, personJens Axboe, personValmiki Rampersad, personBanit Agrawal, personFuxun Yu, personAnsha Yu, personTrung Le, et al year2022.
Supporting massive DLRM inference through software defined memory. In booktitle2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS). IEEE, pages302–312.

[Balasubramanian et al(2021)]
authorpersonArjun Balasubramanian, personAdarsh Kumar, personYuhan Liu, personHan Cao, personShivaram Venkataraman, and personAditya Akella. year2021.
Accelerating deep learning inference via learned caches.
journalarXiv preprint arXiv:2101.07344 (year2021).

[Barga et al(2015)]
authorpersonRoger Barga, personValentine Fontama, personWee Hyong Tok, and personLuis Cabrera-Cordon. year2015.
booktitlePredictive analytics with Microsoft Azure machine learning.

[Baylor et al(2017)]
authorpersonDenis Baylor, personEric Breck, personHeng-Tze Cheng, personNoah Fiedel, personChuan Yu Foo, personZakaria Haque, personSalem Haykal, personMustafa Ispir, personVihan Jain, personLevent Koc, et al year2017.
Tfx: A tensorflow-based production-scale machine learning platform. In booktitleProceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pages1387–1395.

[Boehm et al(2016)]
authorpersonMatthias Boehm, personMichael W Dusenberry, personDeron Eriksson, personAlexandre V Evfimievski, personFaraz Makari Manshadi, personNiketan Pansare, personBerthold Reinwald, personFrederick R Reiss, personPrithviraj Sen, personArvind C Surve, et al year2016.
Systemml: Declarative machine learning on spark.
journalProceedings of the VLDB Endowment volume9, number13 (year2016), pages1425–1436.

[Bommasani et al(2021)]
authorpersonRishi Bommasani, personDrew A Hudson, personEhsan Adeli, personRuss Altman, personSimran Arora, personSydney von Arx, personMichael S Bernstein, personJeannette Bohg, personAntoine Bosselut, personEmma Brunskill, et al year2021.
On the opportunities and risks of foundation models.
journalarXiv preprint arXiv:2108.07258 (year2021).

[Breslow et al(2017)]
authorpersonStuart Breslow, personMikael Hagstroem, personDaniel Mikkelsen, and personKate Robu. year2017.
The new frontier in anti–money laundering.
journalMcKinsey & Company, November (year2017).

[Butrovich et al(2022)]
authorpersonMatthew Butrovich, personWan Shen Lim, personLin Ma, personJohn Rollinson, personWilliam Zhang, personYu Xia, and personAndrew Pavlo. year2022.
Tastes Great! Less Filling! High Performance and Accurate Training Data Collection for Self-Driving Database Management Systems. ACM| Proceedings of the 2022 International Conference on Management of Data.

[Cai et al(2019)]
authorpersonShaofeng Cai, personGang Chen, personBeng Chin Ooi, and personJinyang Gao. year2019.
Model slicing for supporting complex analytics with elastic inference cost and resource constraints.
journalarXiv preprint arXiv:1904.01831 (year2019).

[Chen et al(2022)]
authorpersonKai-Hsun Chen, personHuan-Ping Su, personWei-Chiu Chuang, personHung-Chang Hsiao, personWangda Tan, personZhankun Tang, personXun Liu, personYanbo Liang, personWen-Chih Lo, personWanqiang Ji, et al year2022.
Apache submarine: a unified machine learning platform made simple. In booktitleProceedings of the 2nd European Workshop on Machine Learning and Systems. pages101–108.

[Chen et al(2018)]
authorpersonTianqi Chen, personThierry Moreau, personZiheng Jiang, personLianmin Zheng, personEddie Yan, personHaichen Shen, personMeghan Cowan, personLeyuan Wang, personYuwei Hu, personLuis Ceze, et al year2018.
$\{$TVM$\}$: An automated end-to-end optimizing compiler for deep learning. In booktitle13th $\{$USENIX$\}$ Symposium on Operating Systems Design and Implementation ($\{$OSDI$\}$ 18). pages578–594.

[Chen et al(2021)]
authorpersonZhi Chen, personCody Hao Yu, personTrevor Morris, personJorn Tuyls, personYi-Hsiang Lai, personJared Roesch, personElliott Delaye, personVin Sharma, and personYida Wang. year2021.
Bring your own codegen to deep learning compiler.
journalarXiv preprint arXiv:2105.03215 (year2021).

[Cheng et al(2020)]
authorpersonDawei Cheng, personSheng Xiang, personChencheng Shang, personYiyi Zhang, personFangzhou Yang, and personLiqing Zhang. year2020.
Spatio-temporal attention-based neural network for credit card fraud detection. In booktitleProceedings of the AAAI conference on artificial intelligence, Vol. volume34. pages362–369.

[Crankshaw et al(2017)]
authorpersonDaniel Crankshaw, personXin Wang, personGuilio Zhou, personMichael J Franklin, personJoseph E Gonzalez, and personIon Stoica. year2017.
Clipper: A low-latency online prediction serving system. In booktitle14th $\{$USENIX$\}$ Symposium on Networked Systems Design and Implementation ($\{$NSDI$\}$ 17). pages613–627.

[Damme et al(2022)]
authorpersonPatrick Damme, personMarius Birkenbach, personConstantinos Bitsakos, personMatthias Boehm, personPhilippe Bonnet, personFlorina Ciorba, personMark Dokter, personPawl Dowgiallo, personAhmed Eleliemy, personChristian Faerber, et al year2022.
DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines. In booktitleConference on Innovative Data Systems Research.

[Das et al(2014)]
authorpersonTathagata Das, personYuan Zhong, personIon Stoica, and personScott Shenker. year2014.
Adaptive stream processing using dynamic batch sizing. In booktitleProceedings of the ACM Symposium on Cloud Computing. pages1–13.

authorpersonONNX Runtime developers. year2021.
titleOpen neural network exchange format (ONNX).

[Eksombatchai et al(2018)]
authorpersonChantat Eksombatchai, personPranav Jindal, personJerry Zitao Liu, personYuchen Liu, personRahul Sharma, personCharles Sugnet, personMark Ulrich, and personJure Leskovec. year2018.
Pixie: A system for recommending 3+ billion items to 200+ million users in real-time. In booktitleProceedings of the 2018 world wide web conference. pages1775–1784.

[Fard et al(2020)]
authorpersonArash Fard, personAnh Le, personGeorge Larionov, personWaqas Dhillon, and personChuck Bear. year2020.
Vertica-ml: Distributed machine learning in vertica database. In booktitleProceedings of the 2020 ACM SIGMOD International Conference on Management of Data. pages755–768.

authorpersonPeter I Frazier. year2018.
A tutorial on Bayesian optimization.
journalarXiv preprint arXiv:1807.02811 (year2018).

[Gandhi et al(2022)]
authorpersonApurva Gandhi, personYuki Asada, personVictor Fu, personAdvitya Gemawat, personLihao Zhang, personRathijit Sen, personCarlo Curino, personJesús Camacho-Rodríguez, and personMatteo Interlandi. year2022.
The Tensor Data Platform: Towards an AI-centric Database System.
journalarXiv preprint arXiv:2211.02753 (year2022).

[Gao et al(2018)]
authorpersonJianfeng Gao, personMichel Galley, and personLihong Li. year2018.
Neural approaches to conversational AI. In booktitleThe 41st international ACM SIGIR conference on research & development in information retrieval. pages1371–1374.

[Gupta et al(2020)]
authorpersonUdit Gupta, personSamuel Hsia, personVikram Saraph, personXiaodong Wang, personBrandon Reagen, personGu-Yeon Wei, personHsien-Hsin S Lee, personDavid Brooks, and personCarole-Jean Wu. year2020.
Deeprecsys: A system for optimizing end-to-end at-scale neural recommendation inference. In booktitle2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, pages982–995.

[Hindman et al(2011)]
authorpersonBenjamin Hindman, personAndy Konwinski, personMatei Zaharia, personAli Ghodsi, personAnthony D Joseph, personRandy H Katz, personScott Shenker, and personIon Stoica. year2011.
Mesos: A platform for fine-grained resource sharing in the data center.. In booktitleNSDI, Vol. volume11. pages22–22.

[Huang and et al.(2010)]
authorpersonS. Huang and personet al. year2010.
The HiBench benchmark suite: Characterization of the MapReduce-based data analysis. In booktitleICDEW. pages41–51.

authorpersonAlexa Jacky. year2020.
titleConvert Batch Anti Money Laundering Watch List Management to Real-Time.

[Jankov et al(2019)]
authorpersonDimitrije Jankov, personShangyu Luo, personBinhang Yuan, personZhuhua Cai, personJia Zou, personChris Jermaine, and personZekai J Gao. year2019.
Declarative Recursive Computation on an RDBMS.
journalProceedings of the VLDB Endowment volume12, number7 (year2019).

[Jegou et al(2010)]
authorpersonHerve Jegou, personMatthijs Douze, and personCordelia Schmid. year2010.
Product quantization for nearest neighbor search.
journalIEEE transactions on pattern analysis and machine intelligence volume33, number1 (year2010), pages117–128.

[Johnson et al(2019)]
authorpersonJeff Johnson, personMatthijs Douze, and personHervé Jégou. year2019.
Billion-scale similarity search with GPUs.
journalIEEE Transactions on Big Data volume7, number3 (year2019), pages535–547.

[Jungmair et al(2022)]
authorpersonMichael Jungmair, personAndré Kohn, and personJana Giceva. year2022.
Designing an open framework for query optimization and compilation.
journalProceedings of the VLDB Endowment volume15, number11 (year2022), pages2389–2401.

[Kang et al([n. d.])]
authorpersonDaniel Kang, personPeter Bailis, and personMatei Zaharia. year[n. d.].
BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics.
journalProceedings of the VLDB Endowment volume13, number4 (year[n. d.]).

[Kang et al(2017)]
authorpersonDaniel Kang, personJohn Emmons, personFiras Abuzaid, personPeter Bailis, and personMatei Zaharia. year2017.
NoScope: Optimizing Deep CNN-Based Queries over Video Streams at Scale.
journalProc. VLDB Endow. volume10, number11 (year2017), pages1586–1597.

[Kang et al(2020)]
authorpersonDaniel Kang, personAnkit Mathur, personTeja Veeramacheneni, personPeter Bailis, and personMatei Zaharia. year2020.
Jointly optimizing preprocessing and inference for DNN-based visual analytics.
journalarXiv preprint arXiv:2007.13005 (year2020).

[Karanasos et al(2019)]
authorpersonKonstantinos Karanasos, personMatteo Interlandi, personDoris Xin, personFotis Psallidas, personRathijit Sen, personKwanghyun Park, personIvan Popivanov, personSupun Nakandal, personSubru Krishnan, personMarkus Weimer, et al year2019.
Extending relational query processing with ML inference.
journalarXiv preprint arXiv:1911.00231 (year2019).

authorpersonNeil Katkov. year2022.
titleOPERATIONALIZING FRAUD PREVENTION ON IBM Z16.

[Kim et al(2022)]
authorpersonKyoungmin Kim, personJisung Jung, personIn Seo, personWook-Shin Han, personKangwoo Choi, and personJaehyok Chong. year2022.
Learned cardinality estimation: An in-depth study. In booktitleProceedings of the 2022 International Conference on Management of Data. pages1214–1227.

[Kipf et al(2018)]
authorpersonAndreas Kipf, personThomas Kipf, personBernhard Radke, personViktor Leis, personPeter Boncz, and personAlfons Kemper. year2018.
Learned cardinalities: Estimating correlated joins with deep learning.
journalarXiv preprint arXiv:1809.00677 (year2018).

[Koutsoukos et al(2021)]
authorpersonDimitrios Koutsoukos, personSupun Nakandala, personKonstantinos Karanasos, personKarla Saur, personGustavo Alonso, and personMatteo Interlandi. year2021.
Tensors: An abstraction for general data processing.
journalProc. VLDB Endow. volume14, number10 (year2021), pages1797–1804.

[Kumar et al(2019)]
authorpersonAdarsh Kumar, personArjun Balasubramanian, personShivaram Venkataraman, and personAditya Akella. year2019.
Accelerating Deep Learning Inference via Freezing.. In booktitleHotCloud.

[Lattner et al(2021a)]
authorpersonChris Lattner, personMehdi Amini, personUday Bondhugula, personAlbert Cohen, personAndy Davis, personJacques Pienaar, personRiver Riddle, personTatiana Shpeisman, personNicolas Vasilache, and personOleksandr Zinenko. year2021a.
MLIR: Scaling compiler infrastructure for domain specific computation. In booktitle2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). IEEE, pages2–14.

[Lattner et al(2021b)]
authorpersonChris Lattner, personMehdi Amini, personUday Bondhugula, personAlbert Cohen, personAndy Davis, personJacques Pienaar, personRiver Riddle, personTatiana Shpeisman, personNicolas Vasilache, and personOleksandr Zinenko. year2021b.
Mlir: Scaling compiler infrastructure for domain specific computation. In booktitle2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). IEEE, pages2–14.

[Lee et al(2018)]
authorpersonYunseong Lee, personAlberto Scolari, personByung-Gon Chun, personMarco Domenico Santambrogio, personMarkus Weimer, and personMatteo Interlandi. year2018.
$\{$PRETZEL$\}$: Opening the Black Box of Machine Learning Prediction Serving Systems. In booktitle13th $\{$USENIX$\}$ Symposium on Operating Systems Design and Implementation ($\{$OSDI$\}$ 18). pages611–626.

[Li et al(2019)]
authorpersonGuoliang Li, personXuanhe Zhou, personShifu Li, and personBo Gao. year2019.
Qtune: A query-aware database tuning system with deep reinforcement learning.
journalProceedings of the VLDB Endowment volume12, number12 (year2019), pages2118–2130.

[Li et al(2017)]
authorpersonLisha Li, personKevin Jamieson, personGiulia DeSalvo, personAfshin Rostamizadeh, and personAmeet Talwalkar. year2017.
Hyperband: A novel bandit-based approach to hyperparameter optimization.
journalThe journal of machine learning research volume18, number1 (year2017), pages6765–6816.

[Liberty et al(2020)]
authorpersonEdo Liberty, personZohar Karnin, personBing Xiang, personLaurence Rouesnel, personBaris Coskun, personRamesh Nallapati, personJulio Delgado, personAmir Sadoughi, personYury Astashonok, personPiali Das, et al year2020.
Elastic machine learning algorithms in amazon sagemaker. In booktitleProceedings of the 2020 ACM SIGMOD International Conference on Management of Data. pages731–737.

authorpersonMarko Luksa. year2017.
booktitleKubernetes in action.
publisherSimon and Schuster.

[Luo et al(2018)]
authorpersonShangyu Luo, personZekai J Gao, personMichael Gubanov, personLuis L Perez, and personChristopher Jermaine. year2018.
Scalable linear algebra on a relational database system.
journalIEEE Transactions on Knowledge and Data Engineering volume31, number7 (year2018), pages1224–1238.

[Ma et al(2018)]
authorpersonLin Ma, personDana Van Aken, personAhmed Hefny, personGustavo Mezerhane, personAndrew Pavlo, and personGeoffrey J Gordon. year2018.
Query-based workload forecasting for self-driving database management systems. In booktitleProceedings of the 2018 International Conference on Management of Data. pages631–645.

[MacLeod et al(2020)]
authorpersonBenjamin P MacLeod, personFraser GL Parlane, personThomas D Morrissey, personFlorian Häse, personLoïc M Roch, personKevan E Dettelbach, personRaphaell Moreira, personLars PE Yunker, personMichael B Rooney, personJoseph R Deeth, et al year2020.
Self-driving laboratory for accelerated discovery of thin-film materials.
journalScience Advances volume6, number20 (year2020), pageseaaz8867.

[Malkov and Yashunin(2018)]
authorpersonYu A Malkov and personDmitry A Yashunin. year2018.
Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.
journalIEEE transactions on pattern analysis and machine intelligence volume42, number4 (year2018), pages824–836.

[Maxwell and Bingler(2023)]
authorpersonTim Maxwell and personLiz Bingler. year2023.
titleHow major credit card networks protect customers against fraud.

[Nakandala et al(2019)]
authorpersonSupun Nakandala, personArun Kumar, and personYannis Papakonstantinou. year2019.
Incremental and approximate inference for faster occlusion-based deep cnn explanations. In booktitleProceedings of the 2019 International Conference on Management of Data. pages1589–1606.

[Naumov et al(2019)]
authorpersonMaxim Naumov, personDheevatsa Mudigere, personHao-Jun Michael Shi, personJianyu Huang, personNarayanan Sundaraman, personJongsoo Park, personXiaodong Wang, personUdit Gupta, personCarole-Jean Wu, personAlisson G Azzolini, et al year2019.
Deep learning recommendation model for personalization and recommendation systems.
journalarXiv preprint arXiv:1906.00091 (year2019).

authorpersonThomas Neumann. year2011.
Efficiently compiling efficient query plans for modern hardware.
journalProceedings of the VLDB Endowment volume4, number9 (year2011), pages539–550.

authorpersonThomas Neumann. year2021.
Evolution of a compiling query engine.
journalProceedings of the VLDB Endowment volume14, number12 (year2021), pages3207–3210.

authorpersonNVIDIA. year2014.
booktitleNVIDIA cuDNN.
typeTechnical Report. institutionNVIDIA Corporation, addressSanta Clara, California, USA.

[Olston et al(2017)]
authorpersonChristopher Olston, personNoah Fiedel, personKiril Gorovoy, personJeremiah Harmsen, personLi Lao, personFangwei Li, personVinu Rajashekhar, personSukriti Ramesh, and personJordan Soyke. year2017.
Tensorflow-serving: Flexible, high-performance ml serving.
journalarXiv preprint arXiv:1712.06139 (year2017).

[Park et al(2022)]
authorpersonKwanghyun Park, personKarla Saur, personDalitso Banda, personRathijit Sen, personMatteo Interlandi, and personKonstantinos Karanasos. year2022.
End-to-end Optimization of Machine Learning Prediction Queries. In booktitleProceedings of the 2022 International Conference on Management of Data. pages587–601.

[Peng et al(2023)]
authorpersonBaolin Peng, personChunyuan Li, personPengcheng He, personMichel Galley, and personJianfeng Gao. year2023.
Instruction tuning with gpt-4.
journalarXiv preprint arXiv:2304.03277 (year2023).

[Ragan-Kelley et al(2013)]
authorpersonJonathan Ragan-Kelley, personConnelly Barnes, personAndrew Adams, personSylvain Paris, personFrédo Durand, and personSaman Amarasinghe. year2013.
Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines.
journalAcm Sigplan Notices volume48, number6 (year2013), pages519–530.

[Rao et al(2002)]
authorpersonJun Rao, personChun Zhang, personNimrod Megiddo, and personGuy Lohman. year2002.
Automating physical database design in a parallel database. In booktitleProceedings of the 2002 ACM SIGMOD international conference on Management of data. ACM, pages558–569.

[Ratner et al(2019)]
authorpersonDaniel Ratner, personBobby Sumpter, personFrank Alexander, personJay Jay Billings, personRyan Coffee, personSarah Cousineau, personPeter Denes, personMathieu Doucet, personIan Foster, personAlex Hexemer, et al year2019.
booktitle[Office of Basic Energy Sciences (BES)] Roundtable on Producing and Managing Large Scientific Data with Artificial Intelligence and Machine Learning.
typeTechnical Report. institutionDOESC Office of Basic Energy Sciences.

[Schleich et al(2023)]
authorpersonMaximilian Schleich, personAmir Shaikhha, and personDan Suciu. year2023.
Optimizing Tensor Programs on Flexible Storage.
journalProceedings of the ACM on Management of Data volume1, number1 (year2023), pages1–27.

[Shen et al(2019)]
authorpersonHaichen Shen, personLequn Chen, personYuchen Jin, personLiangyu Zhao, personBingyu Kong, personMatthai Philipose, personArvind Krishnamurthy, and personRavi Sundaram. year2019.
Nexus: a GPU cluster engine for accelerating DNN-based video analysis. In booktitleProceedings of the 27th ACM Symposium on Operating Systems Principles. pages322–337.

[Sheng et al(2023)]
authorpersonYing Sheng, personLianmin Zheng, personBinhang Yuan, personZhuohan Li, personMax Ryabinin, personDaniel Y Fu, personZhiqiang Xie, personBeidi Chen, personClark Barrett, personJoseph E Gonzalez, et al year2023.
High-throughput generative inference of large language models with a single gpu.
journalarXiv preprint arXiv:2303.06865 (year2023).

[Shi et al(2014)]
authorpersonJuwei Shi, personJia Zou, personJiaheng Lu, personZhao Cao, personShiqiang Li, and personChen Wang. year2014.
MRTuner: a toolkit to enable holistic optimization for mapreduce jobs.
journalProceedings of the VLDB Endowment volume7, number13 (year2014), pages1319–1330.

[Sivic and Zisserman(2003)]
authorpersonSivic and personZisserman. year2003.
Video Google: A text retrieval approach to object matching in videos. In booktitleProceedings ninth IEEE international conference on computer vision. IEEE, pages1470–1477.

[Snoek et al(2012)]
authorpersonJasper Snoek, personHugo Larochelle, and personRyan P Adams. year2012.
Practical bayesian optimization of machine learning algorithms.
journalAdvances in neural information processing systems  volume25 (year2012).

[Tan et al(2023)]
authorpersonWang-Chiew Tan, personYuliang Li, personPedro Rodriguez, personRichard James, personXi Victoria Lin, personAlon Halevy, and personScott Yih. year2023.
Reimagining Retrieval Augmented Language Models for Answering Queries.
journalarXiv preprint arXiv:2306.01061 (year2023).

[Vasilache et al(2018)]
authorpersonNicolas Vasilache, personOleksandr Zinenko, personTheodoros Theodoridis, personPriya Goyal, personZachary DeVito, personWilliam S Moses, personSven Verdoolaege, personAndrew Adams, and personAlbert Cohen. year2018.
Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions.
journalarXiv preprint arXiv:1802.04730 (year2018).

[Vavilapalli et al(2013)]
authorpersonVinod Kumar Vavilapalli, personArun C Murthy, personChris Douglas, personSharad Agarwal, personMahadev Konar, personRobert Evans, personThomas Graves, personJason Lowe, personHitesh Shah, personSiddharth Seth, et al year2013.
Apache hadoop yarn: Yet another resource negotiator. In booktitleProceedings of the 4th annual Symposium on Cloud Computing. pages1–16.

[Wang et al(2021b)]
authorpersonJianguo Wang, personXiaomeng Yi, personRentong Guo, personHai Jin, personPeng Xu, personShengjun Li, personXiangyu Wang, personXiangzhou Guo, personChengming Li, personXiaohai Xu, et al year2021b.
Milvus: A Purpose-Built Vector Data Management System. In booktitleProceedings of the 2021 International Conference on Management of Data. pages2614–2627.

[Wang et al(2018)]
authorpersonWei Wang, personSheng Wang, personJinyang Gao, personMeihui Zhang, personGang Chen, personTeck Khim Ng, and personBeng Chin Ooi. year2018.
Rafiki: machine learning as an analytics service system.
journalarXiv preprint arXiv:1804.06087 (year2018).

[Wang et al(2021a)]
authorpersonXuhong Wang, personDing Lyu, personMengjian Li, personYang Xia, personQi Yang, personXinwen Wang, personXinguang Wang, personPing Cui, personYupu Yang, personBowen Sun, et al year2021a.
Apan: Asynchronous propagation attention network for real-time temporal graph embedding. In booktitleProceedings of the 2021 international conference on management of data. pages2628–2638.

[Wang et al(2022)]
authorpersonXiaoying Wang, personWeiyuan Wu, personJinze Wu, personYizhou Chen, personNick Zrymiak, personChangbo Qu, personLampros Flokas, personGeorge Chow, personJiannan Wang, personTianzheng Wang, et al year2022.
ConnectorX: accelerating data loading from databases to dataframes.
journalProceedings of the VLDB Endowment volume15, number11 (year2022), pages2994–3003.

[Wolf et al(2020)]
authorpersonThomas Wolf, personLysandre Debut, personVictor Sanh, personJulien Chaumond, personClement Delangue, personAnthony Moi, personPierric Cistac, personTim Rault, personRémi Louf, personMorgan Funtowicz, personJoe Davison, personSam Shleifer, personPatrick von Platen, personClara Ma, personYacine Jernite, personJulien Plu, personCanwen Xu, personTeven Le Scao, personSylvain Gugger, personMariama Drame, personQuentin Lhoest, and personAlexander M. Rush. year2020.
Transformers: State-of-the-Art Natural Language Processing. In booktitleProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. publisherAssociation for Computational Linguistics, addressOnline, pages38–45.

[Xiang et al(2023)]
authorpersonSheng Xiang, personMingzhi Zhu, personDawei Cheng, personEnxia Li, personRuihui Zhao, personYi Ouyang, personLing Chen, and personYefeng Zheng. year2023.
Semi-supervised Credit Card Fraud Detection via Attribute-Driven Graph Representation. In booktitleProceedings of the AAAI Conference on Artificial Intelligence, Vol. volume37. pages14557–14565.

[Yadav et al(2022)]
authorpersonRohan Yadav, personAlex Aiken, and personFredrik Kjolstad. year2022.
DISTAL: the distributed tensor algebra compiler. In booktitleProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation. pages286–300.

[Yuan et al(2021)]
authorpersonBinhang Yuan, personDimitrije Jankov, personJia Zou, personYuxin Tang, personDaniel Bourgeois, and personChris Jermaine. year2021.
Tensor Relational Algebra for Distributed Machine Learning System Design.
journalProc. VLDB Endow. volume14, number8 (year2021), pages1338–1350.

[Yuan et al(2020)]
authorpersonHaitao Yuan, personGuoliang Li, personLing Feng, personJi Sun, and personYue Han. year2020.
Automatic view generation with deep learning and reinforcement learning. In booktitle2020 IEEE 36th International Conference on Data Engineering (ICDE). IEEE, pages1501–1512.

[Zhang et al(2019)]
authorpersonJi Zhang, personYu Liu, personKe Zhou, personGuoliang Li, personZhili Xiao, personBin Cheng, personJiashu Xing, personYangtao Wang, personTianheng Cheng, personLi Liu, et al year2019.
An end-to-end automatic cloud database tuning system using deep reinforcement learning. In booktitleProceedings of the 2019 International Conference on Management of Data. pages415–432.

[Zhang et al(2021)]
authorpersonWangda Zhang, personJunyoung Kim, personKenneth A Ross, personEric Sedlar, and personLukas Stadler. year2021.
Adaptive code generation for data-intensive analytics.
journalProceedings of the VLDB Endowment volume14, number6 (year2021), pages929–942.

[Zheng et al(2022)]
authorpersonBojian Zheng, personZiheng Jiang, personCody Hao Yu, personHaichen Shen, personJoshua Fromm, personYizhi Liu, personYida Wang, personLuis Ceze, personTianqi Chen, and personGennady Pekhimenko. year2022.
DietCode: Automatic optimization for dynamic tensor programs.
journalProceedings of Machine Learning and Systems  volume4 (year2022), pages848–863.

[Zhou et al(2023)]
authorpersonLixi Zhou, personK Selçuk Candan, and personJia Zou. year2023.
DeepMapping: The Case for Learned Data Mapping for Compression and Efficient Query Processing.
journalarXiv preprint arXiv:2307.05861 (year2023).

[Zhou et al(2022a)]
authorpersonLixi Zhou, personJiaqing Chen, personAmitabh Das, personHong Min, personLei Yu, personMing Zhao, and personJia Zou. year2022a.
Serving Deep Learning Models with Deduplication from Relational Databases.
journalProc. VLDB Endow. volume15, number10 (year2022), pages2230–2243.

[Zhou et al(2022b)]
authorpersonLixi Zhou, personArindam Jain, personZijie Wang, personAmitabh Das, personYingzhen Yang, and personJia Zou. year2022b.
Benchmark of DNN Model Search at Deployment Time. In booktitleProceedings of the 34th International Conference on Scientific and Statistical Database Management. pages1–12.

[Zhou et al(2020)]
authorpersonXuanhe Zhou, personChengliang Chai, personGuoliang Li, and personJi Sun. year2020.
Database meets artificial intelligence: A survey.
journalIEEE Transactions on Knowledge and Data Engineering volume34, number3 (year2020), pages1096–1116.

[Zhu et al(2016)]
authorpersonErkang Zhu, personFatemeh Nargesian, personKen Q Pu, and personRenée J Miller. year2016.
LSH ensemble: Internet-scale domain search.
journalarXiv preprint arXiv:1603.07410 (year2016).

[Zou et al(2018)]
authorpersonJia Zou, personR Matthew Barnett, personTania Lorido-Botran, personShangyu Luo, personCarlos Monroy, personSourav Sikdar, personKia Teymourian, personBinhang Yuan, and personChris Jermaine. year2018.
PlinyCompute: A platform for high-performance, distributed, data-intensive tool development. In booktitleProceedings of the 2018 International Conference on Management of Data. pages1189–1204.

[Zou et al(2021a)]
authorpersonJia Zou, personAmitabh Das, personPratik Barhate, personArun Iyengar, personBinhang Yuan, personDimitrije Jankov, and personChris Jermaine. year2021a.
Lachesis: Automated Partitioning for UDF-Centric Analytics.
journalProc. VLDB Endow. volume14, number8 (year2021), pages1262–1275.

[Zou et al(2021b)]
authorpersonJia Zou, personAmitabh Das, personPratik Barhate, personArun Iyengar, personBinhang Yuan, personDimitrije Jankov, and personChris Jermaine. year2021b.
Lachesis: automatic partitioning for UDF-centric analytics.
journalProceedings of the VLDB Endowment volume14, number8 (year2021), pages1262–1275.

[Zou et al(2019)]
authorpersonJia Zou, personArun Iyengar, and personChris Jermaine. year2019.
Pangea: monolithic distributed storage for data analytics.
journalProceedings of the VLDB Endowment volume12, number6 (year2019), pages681–694.

[Zou et al(2020a)]
authorpersonJia Zou, personArun Iyengar, and personChris Jermaine. year2020a.
Architecture of a distributed storage that combines file system, memory and computation in a single layer.
journalThe VLDB Journal (year2020), pages1–25.

[Zou et al(2020b)]
authorpersonJia Zou, personMing Zhao, personJuwei Shi, and personChen Wang. year2020b.
Watson: A workflow-based data storage optimizer for analytics. In booktitle36th Intl. Conf. on Massive Storage Systems and Technology.