# HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion
Models

Pei Lin1 Sihang Xu1 Hongdi Yang1 Yiran Liu1 Xin Chen2
Jingya Wang1 Jingyi Yu1 Lan Xu1
1ShanghaiTech University 2Tencent PCG
https://handdiffuse.github.io/

###### Abstract

Existing hands datasets are largely short-range and the interaction is weak
due to the self-occlusion and self-similarity of hands, which can not yet fit
the need for interacting hands motion generation. To rescue the data scarcity,
we propose HandDiffuse12.5M, a novel dataset that consists of temporal
sequences with strong two-hand interactions. HandDiffuse12.5M has the largest
scale and richest interactions among the existing two-hand datasets. We
further present a strong baseline method HandDiffuse for the controllable
motion generation of interacting hands using various controllers.
Specifically, we apply the diffusion model as the backbone and design two
motion representations for different controllers. To reduce artifacts, we also
propose Interaction Loss which explicitly quantifies the dynamic interaction
process. Our HandDiffuse enables various applications with vivid two-hand
interactions, i.e., motion in-betweening and trajectory control. Experiments
show that our method outperforms the state-of-the-art techniques in motion
generation and can also contribute to data augmentation for other datasets.
Our dataset, corresponding codes, and pre-trained models will be disseminated
to the community for future research towards two-hand interaction modeling.

Figure 1: Overview: The proposed HandDiffuse12.5M benchmark dataset consists
of temporal sequences with strong interaction. Based on HandDiffuse12.5M, we
propose HandDiffuse, a strong baseline for the motion generation of
interacting hands. HandDiffuse enables three motion generation tasks,
including Unconstrained Generation, Motion In-betweening, and Trajectory
Control.

##  1 Introduction

The tightly interacting hands play a crucial role in human emotional
expression and communication, serving as a vital component for physical
interaction with oneself. Controllable hand motion generation not only
enhances the experience of augmented reality/virtual reality (AR/VR) but also
help robotics and avatars to express their emotion in another dimension.

So far, lots of works have emerged in the field of human body motion
generation [51, 59, 4, 22], but the generation of temporal hands motions in
strong interaction remains blank as the lack of strong interacting hands
datasets. However, the dataset acquisition is time-consuming and expensive due
to the interacting hands present self-occlusion, self-similarity, and complex
articulations. There are only a few datasets [53, 36, 64, 28, 37] focusing on
hands with strong interactions. Most of them, i,e, InterHand2.6M [36], Two-
hand 500K [64] and RenderIH [28], provides large-scale discrete frames but the
available temporal sequences remain sparse. The severe lack of sufficient
temporal motions makes them unsuitable for the motion generation task of two-
hand interactions. Only recently, the concurrent work Re:InterHand [37]
provides large-scale interaction data, yet it focuses on the motion capture of
interacted hands under diverse lighting conditions.

To break the data scarcity, in this paper, we first contribute a comprehensive
dataset, HandDiffuse12.5M, which focuses on capturing diverse temporal
sequences with strong two-hand interactions. Our dataset consists of various
modalities, including synchronous 50-view videos, 3D key points, and hand
poses. It covers interacting hand motions in random and complex poses, daily
communications, finger dances, and Chinese daily sign language with 250k
temporal frames, resulting in roughly 12.5 million pictures. To address the
issue of occlusion, which is unavoidable in hand interactions, we manually
annotate the key points in 240k images. Note that our HandDiffuse12.5M dataset
owns the largest scale not only in the number of temporal frames but also in
the number of pictures among the existing available interacting hands
datasets. Thus, it not only opens up the research for temporally consistent
two-hand motion generation but also brings huge potential for future
exploration in hand modeling.

Based on our novel dataset, we further provide a strong baseline algorithm
named HandDiffuse, which tailors the diffusion model [21] to generate vivid
interacting hands motions under diverse explicit control. Note that hand
motion is an abstract form of emotional expression and ten fingers always
gather together during interactions. Therefore, it is difficult to describe
the motions with concrete text prompts and the control for motion generation
needs to be explicit. To this end, we adopt two kinds of explicit controllers
as the conditions for our diffusion models, including the discrete keyframes
and the trajectories of wrists. These conditionings enable various downstream
applications, ranging from hand motion inbetweening to trajectory-aided hand
motion completion which connects hand motions with body motions (see Figure
1). In HandDiffuse, we design two specific motion representations for
different controllers, as well as a novel Interaction Loss for the dynamic
process of hand interaction, which helps to reduce the artifacts. Finally, we
provide a thorough evaluation of our approach as well as a comparison with
state-of-the-art motion generation methods [51, 31]. We also provide results
to show that HandDiffuse can contribute to data augmentation for other
datasets. Our preliminary outcomes indicate that controllable hand motion
generation remains a challenging problem, and our dataset will consistently
benefit further investigations of this new research direction. To summarize,
our main contributions include:

  * •

We contribute a large-scale dataset, which provides accurate and temporally
consistent tracking of human hands under diverse and strong two-hand
interactions.

  * •

We propose a strong baseline for generating interacting hand motions from
various explicit controllers, enabling vivid motion in-betweening and
trajectory conditioning.

  * •

We introduce two motion representations as well as an Interaction Loss to
significantly improve the generation results of two-hand interactions.

  * •

Our dataset, related codes, and pre-trained models will be made publicly
available to stimulate further research.

##  2 Related Works

Dataset | Temporal Sequence | Source | # Views | Manually Annotation | Appearance
---|---|---|---|---|---
RenderIH [28] | No | Synthesis | - | No | Natural
Two-hand 500K [64] | Partial | IMU+Synthesis | - | No | -
InterHand2.6M [36] | Partial | RGB | 80 | Yes | Lab
Re:InterHand [37] | Yes | RGB | 26 | No | Relighting
HandDiffuse12.5M(Ours) | Yes | RGB | 50 | Yes | Lab

Table 1: Dataset Comparisons. We compare our HandDiffuse12.5M dataset with
other existing publicly available interacting-hand datasets. Our
HandDiffuse12.5M dataset contains both strong interaction and temporal
sequences, and has the largest scale.

#### Hand Dataset.

The establishment of realistic hand datasets is a challenging and time-
consuming process. However, these datasets have significantly accelerated the
development of VR/AR, embodied AI and human-object interaction. Many
researchers have utilized various sensors, including multiple RGB cameras,
depth sensors and maker-base cameras, to collect realistic data in hand-object
interaction or dual-hand interaction [36, 37, 17, 15, 5, 58, 48, 27, 3, 9, 47,
53, 40, 18, 57, 13, 11, 34].

Obtaining ground truth for specific dual-hand interaction motion from real-
world captured data is challenging [36, 28]. Moon et al. [36] have attempted
to combine manual and automated annotations, but manual annotations are labor-
intensive. Recently, Moon et al. [37] have improved the detection accuracy of
2D keypoints and reconstructed hands to facilitate fully automated annotation
schemes, but these methods are still time-consuming.

As a result, some works have established synthetic hand datasets to enhance
annotation precision [64, 28, 63, 32, 10]. Among them, RenderIH [28] increases
the diversity of data through re-lighting, while DART [10] provides a wide
range of hand accessories to bridge the gap between synthetic and real data.
However, synthetic datasets fail to balance the diversity of motion and the
continuity of sequences. Some works [28, 64] sample single-frame actions for
the left and right hands separately from real datasets and solve the occlusion
problem between the hands through optimization. However, such optimization
methods are not effective for temporal sequences. Therefore, very few papers
[53, 36, 37] have collected temporal interacting hand datasets [28].

#### Hands Capture & Reconstruction.

The reconstruction of both hands is a significant challenge in the area of
human motion capture [64]. The release of the InterHand2.6M [36] has motivated
many regression-based methods [45, 1, 30, 16, 6, 8, 26, 19, 35, 35]. These
regression-based methods are widely applied in VTuber and VR/AR devices due to
their balance between real-time performance and accuracy. Some works [30, 38]
proposed a Transformer-based network with the cross-attention between right
and left hands. Recently, Zuo et al. [64] constructed a learning-based prior
to capture dual-hands which achieved great results. Building on the impressive
performance of diffusion models in image generation [46, 49, 21], DiffHand
[29] combines hand reconstruction with diffusion models.

#### Human Motion generation.

In recent years, there has been a growing interest in the field of human
motion generation, with many studies exploring the generation of human motions
based on conditioning signals. These conditioning signals are used to guide
the generation of specific types of motions, such as motion in-betweening [25,
20, 42], trajectory control [60, 43, 24, 54] and unconstrained motion
synthesis [39, 55, 62]. Leading by the pioneering work of MDM [52], human
motion generation models based on diffusion architecture [31, 61, 7] have been
shown to gain better motion diversity and expressiveness compared to prior
works based on GAN [33] or VAE [2, 41]. However, most existing works in this
area have primarily focused on generating whole-body motions, neglecting the
generation of dual-hand motions, which plays a crucial role in many
interactive scenarios. Utilizing the capabilities of the diffusion models, we
propose a novel method HandDiffuse to enable interacting hands motion
generation tasks with control signals such as key frames and trajectory.

##  3 HandDiffuse1.2M Dataset

Figure 2: Overview of our HandDiffuse12.5M. HandDiffuse12.5M is featured with
RGB images, multi-view 2D keypoints, reconstructed 3D keypoints, and MANO [44]
parameters. Figure 3: t-SNE visualization of HandDiffuse12.5M(Ours),
InterHand2.6M and Re:InterHand. For InterHand2.6M, we only choose its temporal
frames. The result indicates the diversity and richness of our
HandDiffuse12.5M. Figure 4: (Left) HandDiffuse Training procedure overview.
Given HandDiffuse Motion Representation $\mathbf{x_{t}^{1:N}}$ of length $N$
together with a timestep embedding $t$ as input, we use a transformer Encoder
to predict the final hand motion $\mathbf{\hat{x}_{0}^{1:N}}$. The proposed
Interaction Loss is utilized to regulate the generated motion. (Right)
Downsteam Applications overview. For Unconstrained Generation, we sample pure
Gaussian noise $x_{t}$ and predict the final motion $\hat{x}_{0}$. For Motion
In-betweening, we sample the final motion $\hat{x}_{0}$ given
$\mathbf{x_{t}^{1:5}}$ and $\mathbf{x_{t}^{-5:N}}$. For Trajectory Control, we
sample the final motion $\hat{x}_{0}$ given the root angular velocity and
linear velocity.

###  3.1 Capture system

Existing hand motion capture solutions include IMU gloves, elastic sensor
gloves, marker-based systems, and multi-view camera capture. However, due to
the severe occlusion in hand interactions, marker-based systems are not
suitable; IMU gloves and elastic sensor-based capture solutions theoretically
address the issue of visual occlusion, but these gloves require specific hand
shapes from the wearer and are difficult to calibrate in each instance of use.
Therefore, we have opted for a multi-view RGB approach combined with manual
annotation, similar to InterHand2.6M. As shown in Figure 3, our capture system
consists of 50 mounted video cameras (Zcam E2) recording interaction sequences
at a frame rate of 30 frames per second (fps). We adjust the resolution to
3840 × 2160 to allocate more pixels for capturing hand movements. To ensure
uniform illumination and minimize motion blur, we utilize 35 directional LED
panel lights and set the shutter speed to 1/500. We design four categories of
hand interaction motions for recording: random and complex hand interactions,
hand movements during daily communication, finger tutting dance, and Chinese
daily sign language. Each motion sequence is controlled to last approximately
one minute.

###  3.2 2/3D joint coordinates & MANO fitting

Upon securing multi-view video sequences, Mediapipe [12] is utilized for 2D
joint detection with a confidence threshold set at 0.95 to eliminate poorly
captured points. The subsequent triangulation of these 2D keypoints yields 3D
joint positions in world coordinates, which are then integrated into the MANO
3D hand model [44] which is differentiable and can generate 3D hand mesh from
3D pose and shape inputs. It transforms shape parameters
$\beta\in\mathbb{R}^{B}$, global pose $\theta_{g}\in\mathbb{R}^{3}$, and local
pose $\theta_{l}\in\mathbb{R}^{45}$ into
vertices$\mathcal{V}_{\text{MANO}}\in\mathbb{R}^{D_{V}\times 3}$ and
joints$\mathcal{J}_{\text{MANO}}\in\mathbb{R}^{D_{J}\times 3}$. Here, $B$
denotes the dimensionality of shape parameters, $D_{V}$ the number of
vertices, and $D_{J}$ the number of hand joints.

To address the computational challenges posed by the full MANO model, which
operates with a complexity of $O(BD_{V}D_{J})$. We developed a compressed MANO
model. This model reduces the computational complexity to $O(BD_{J})$ by
focusing only on the joints’ positions. The efficiency is achieved by
preprocessing MANO’s shape basis to formulate a ”joint shape basis”, enabling
accurate hand pose depiction with fewer computations.

In the optimization phase, we aim to precisely adjust the MANO parameters
($\beta,\theta_{g},\theta_{l}$) for alignment with the captured 3D joint data.
The joint positions as determined by the MANO model are represented as
$J_{\text{mano}}(\beta,\theta_{g},\theta_{l})$, while the ground-truth
positions are denoted as $J_{\text{gt}}$. The optimization encompasses two
primary components:

$\displaystyle L_{\text{joint}}$ $\displaystyle=\lVert
J_{\text{gt}}-J_{\text{mano}}(\beta,\theta_{g},\theta_{l})\rVert_{2},$
$\displaystyle L_{\text{pose}}$
$\displaystyle=\lVert\mathbf{\theta}_{l}\rVert^{2},$ $\displaystyle
L_{\text{total}}$ $\displaystyle=L_{\text{joint}}+\lambda L_{\text{pose}},$

In this formulation, $\lambda>0$ is a hyperparameter that helps balance joint
positioning accuracy with pose regularization.

Despite these steps, errors caused by hand occlusion are still inevitable. We
manually verify each frame and annotate erroneous frames for accuracy. For
each frame, we choose the six least occluded viewpoints for precise joint
position annotation, totaling 240,000 pictures manually annotated. Further
information about the construction of the HandDiffuse dataset and the
application of the compressed MANO model for efficient parameter fitting is
detailed in the Appendix.

###  3.3 Quantitative Evaluation for our Dataset.

Table 1 presents a comparison with two existing 3D hand datasets. Our dual-
hand dataset focuses on close-hand interactions and temporal sequences. Our
HandDiffuse dataset has the highest number of frames, with each sequence
containing at least 200 frames. Our HandDiffuse dataset has more than 250,000
temporal frames, making it the largest known dataset of continuous temporal
frames for dual-hand interaction data. We anticipate that this dataset with
long sequences of close hand interactions will assist in fine-grained hands
motion generation. More details are provided in the Appendix.

We evaluate the superiority of our dataset in terms of interaction data
diversity and effectiveness. To this end, we compare our dataset with
InterHand2.6M [36] and Re:InterHand [37], both of which also contain hand
interaction motion sequences. Figure 3 illustrates that our dataset exhibits a
greater variety of interaction poses compared to InterHand2.6M and
Re:InterHand. This is attributed to our dataset’s explicit focus on hand
interaction pose sequences for generation tasks.

##  4 Method

An overview of our method is shown in Figure 4. With our dataset, the goal is
to synthesize two hands motion $\mathbf{x^{1:N}}$ without or with two designed
controllers $c$. These two selected controllers are common in human motion
generation and are also able to dictate the synthesis of two hand motions,
such as initial and end frames (motion in-betweening) and the positions and
rotations of both wrists (trajectory control). In contrast to single-person
human motion generation, the task of generating dual-hand motion requires the
network to learn not only the local motion of each hand but also the dynamic
process of interaction. Therefore, we propose two specific hand motion
representations for different controllers 4.1 and Interaction Loss 4.2 to
model this dynamic interaction process.

Inspired by recent work, we adopt the diffusion model as the backbone which
has shown remarkable generative ability in image generation and human motion
generation. Diffuse is modeled as a Markov noising process and the ground
truth $\mathbf{x_{0}^{1:N}}$ is diffused to $\mathbf{x_{t}^{1:N}}$ by adding
t-step independent Gaussian noise $\epsilon\sim\mathcal{N}(0,I)$. This process
can be formulated as

$q(\mathbf{x_{t}|x_{0}})=\sqrt{\overline{\alpha}_{t}}\mathbf{x_{0}}+(1-\overline{\alpha}_{t}),$

where $\overline{\alpha}_{t}=\prod_{s=0}^{t}\alpha_{s}$ and
$\alpha_{t}=1-\beta_{t}$. $\beta_{t}$ is the cosine noise variance schedule
and $q(\mathbf{x_{t}})$ is near $\mathcal{N}(0,I)$ when t is big enough. Then,
$\mathbf{x_{t}}$ is sent to a denoiser conditioned on timestep $t$ and
different conditions $c$ to predict $\mathbf{\hat{x}_{0}}$:

$\mathbf{\hat{x}_{0}}=\mathcal{G}(\mathbf{x_{t}},t,c).$

We use an encoder-only transformer to construct our denoiser $D$ following
MDM[51] and we concatenate the left hand and right hand together as input.

###  4.1 Motion representation for two hands.

For different tasks, we have proposed two specific motion representation
methods, which we refer to as local representation (local rep) and global
representation (global rep). The local rep is inspired by HumanML3D[14], a
widely used approach in human motion generation. In each frame, the pose of
left hand is defined by a tuple of
$(\dot{R},\dot{T},J_{local},\dot{J}_{local},\theta)$ and the right hand is
$(R_{init},T_{init},\dot{R},\dot{T},J_{local},\dot{J}_{local},\theta)$, where
$\dot{R}\in\mathbb{R}^{4}$ and $\dot{T}\in\mathbb{R}^{3}$ represent root
angular velocity in quaternion and linear velocity respectively.
$J_{local}\in\mathbb{R}^{3j}$ and $\dot{J}_{local}\in\mathbb{R}^{3j}$ denote
the local joint positions and velocities in root space, with $j=21$ represents
the number of joints. $\theta\in\mathbb{R}^{4i}$ represents the local joint
rotations in quaternion representation, following the skeleton structure of
MANO with $i=15$. Each sequence is aligned with the left wrist of the first
frame as the coordinate system origin. Both left and right hands are rotated
to align the left-hand R vector with the direction (1, 0, 0) which is shown in
Figure 4. Therefore, the right hand has $R_{init}\in\mathbb{R}^{4}$ and
$T_{init}\in\mathbb{R}^{3}$.

On the other hand, the global rep is defined as a tuple of ($J_{global}$,
$\dot{J}_{global}$, $\theta$) for both hands, where $J_{global}$ represents
the positions of hand joints in the world coordinate system and two hands are
normalized by employing the same approach as local rep.

Figure 5: Module of contact potential loss in Interaction Loss. The distance
between joints has direction. Despite the approximate similarity in length of
$\overrightarrow{d_{1}}$ and $\overrightarrow{d_{2}}$, their directions are
opposite. Therefore, it is necessary to consider both the length and direction
simultaneously to eliminate ambiguity.  Figure 6: The Downstream Application
of HandDiffuse. The generation results of downstream applications of
HandDiffuse. The generated dual-hand motion exhibits natural and realistic
hand interaction.

During the experimental process, we observed that the local rep performed
better in the task of trajectory control, where the global rotation and
translation of the hand are given. The global rep achieved better results in
the unconditional generation and motion in-betweening tasks. In Section 5.3, a
detailed comparison between the two representations is provided.

###  4.2 Interaction Loss.

The first part of the Interaction Loss is the dual-hand joint contact
potential loss. In each frame, we first calculate a $25\times 25$ distance
matrix $\mathbf{D}$ between the 21 left-hand joints and 21 right-hand joints
with 4 more joints sampled in each palm. Since the interaction between the
hands involves fine-grained movements, we should focus on situations where the
finger joints are very close to each other. Inspired by the concept of Contact
Potential Field[56], we further represent the distances using the elastic
potential energy of a spring model and it can be described as

$\mathbf{P}otential=1/2\mathbf{K}\
relu(\mathbf{\tau}-\|\mathbf{D}_{pred}\|_{2})^{2},$ (1)

where $\mathbf{K}$ represents the coefficient of elasticity and
$\mathbf{\tau}$ denotes the distance threshold. The closer the distance
between the finger joints, the larger the potential energy obtained, and the
distance will be filtered out when it is over the threshold. Moreover, by
considering the direction of the distance, we can avoid ambiguity caused by
penetration, and the final contact potential loss is described as

$\mathbf{Loss}_{potential}=|\mathbf{P}_{pred}-\mathbf{P}_{gt}|(1+\overrightarrow{\mathbf{D}_{pred}}\times\overrightarrow{\mathbf{D}_{gt}}),$
(2)

where $(\times)$ means cross product. More is shown in Figure 5.

Figure 7: Qualitative comparison for different methods.  For Unconstrained
Generation, MDM[52] tend to generate motion with hands shifting away from each
other, while InterGen[31] would generate motions with a mean pose. In terms of
Trajectory Control, MDM yields satisfactory results, whereas InterGen
generates motion without incorporating interaction.

During our testing process, we observed that MANO has strict requirements for
the proportional lengths of hand bones. Therefore, the second part of the
Interaction Loss is the Shape Loss. We ensure the rationality of hand shape by
calculating the differences in bone lengths between the left and right hands,
as well as the differences between the predicted shape and the ground truth
(GT). This is described as follows:

$\mathbf{Loss}_{shape}=|\mathbf{BL}_{left}-\mathbf{BL}_{right}|+|\mathbf{BL}_{pred}-\mathbf{BL}_{GT}|,$
(3)

where $BL$ denotes the Euclidean distance of the bones.

###  4.3 Downstream Application

For the task of unconstrained generation and motion in-betweening, the key
challenge lies in constructing the global information of the dynamic
interaction process between the two hands. Therefore, we choose to use the
global rep. In motion in-betweening task, we provide the ground truth for the
previous five frames $\mathbf{x_{0}^{1:5}}$ and the last five frames
$\mathbf{x_{0}^{-5:-1}}$ for the diffused motion $\mathbf{x_{t}^{1:N}}$.

For the task of trajectory control, the goal is to generate the reasonable
local motion given the global rotation and translation of both hands. In this
case, using the local rep allows the network to focus on the local motion
during the interaction process. Furthermore, we provide $\dot{R}^{1:N}$ and
$\dot{T}^{1:N}$ for the diffuse $\mathbf{x_{t}^{1:N}}$.

##  5 Experiment

In this section, we conduct several experiments to evaluate our dataset and
method for dual-hand motion generation tasks. We particularly evaluate (1) the
comparison of our method against previous state-of-the-art approaches, (2) the
ability of our method to augment existing datasets, and (3) the effectiveness
of our hand motion representation and Interaction Loss. The above evaluations
are conducted specifically on the three tasks mentioned in Section 4.3. The
models have been trained with $T=1000$ noising steps and a cosine noise
schedule. All of them have been trained on a single NVIDIA GeForce RTX 3080
GPU for about two days.

Methods | Unconstrained | In-betweening | Trajectory Control
---|---|---|---
FID$\downarrow$ | Diversity$\uparrow$ | FID$\downarrow$ | Diversity$\uparrow$ | FID$\downarrow$ | Diversity$\uparrow$
Real | 0.050 | 12.034 |  |  |  |
MDM[52] | 0.526 | 10.366 | 0.616 | 10.258 | 0.205 | 11.265
InterGen[31] | 0.251 | 4.928 | 0.269 | 5.385 | 0.402 | 9.102
Ours | 0.176 | 11.786 | 0.282 | 11.533 | 0.182 | 11.523

Table 2: Quantitative comparisons of different methods. We compare the FID and
Diversity for different methods. ’$\downarrow$’ indicates that results are
better with lower metrics. Bold means the best result.

###  5.1 Comparisons of different methods

To the best of our knowledge, our proposed method is the first to specifically
target hand-only interaction motion generation. To provide a comprehensive
evaluation, we compare our approach with two recent state-of-the-art human
motion generation methods, namely MDM [52] and InterGen [31]. We modify these
methods by adapting their motion representation to enable hand motion
generation. As depicted in Figure 7, both MDM and InterGen exhibit limitations
when generating hand motion. They tend to produce motion sequences with severe
joint penetration or unnatural interaction since they were not originally
designed for dual-hand motion generation. In contrast, our proposed method
showcases superior performance as demonstrated by the lowest Fréchet Inception
Distance (FID) values, as shown in Table 2. The way we adopted to calculate
FID is introduced in the Appendix.

Figure 8: Cross data Evaluation. Motion In-betweening task result on
InterHand2.6M[36]. We sample two frames in InterHand2.6M as the start frame
and end frame and generate the in-betweening motion. Our HandDiffse can help
augment the existing datasets.

###  5.2 Cross Datasets Evaluation

To evaluate our method cross datasets, we conduct experiments on two large-
scale dual-hand interaction motion datasets, InterHand2.6M[36] and
Re:InterHand [37] by sampling control signal from these two datasets. As shown
in Figure 8, given the first frame and last frame from InterHand2.6M, our
model is able to generate diverse motions. Furthermore, Table 3 shows that
after performing In-betweening and Trajectory Control tasks on the datasets,
the generated data can increase the diversity. This demonstrates the
effectiveness of our approach and HandDiffuse can contribute to data
augmentation for other datasets. More results and the details of experiments
are introduced in the Appendix.

Datasets | InterHand | Re:InterHand
---|---|---
| FID$\downarrow$ | Diversity$\uparrow$ | FID$\downarrow$ | Diversity$\uparrow$
Real | 0.113 | 8.478 | 0.034 | 10.583
In-betweening | 0.194 | 8.171 | 0.208 | 11.200
Root Control | 0.142 | 10.146 | 0.236 | 12.979

Table 3: Cross Dataset Evaluation.The FID and Diversity both increase,
indicating that we get a larger dataset from the original dataset by
performing data generation tasks.

###  5.3 Ablation Study

As we propose two kinds of hand motion representations in section 4.1, here we
demonstrate the effectiveness of the representations in different tasks by
conducting qualitative and quantitative comparisons. Figure 9 illustrates the
limitations of the model without Shape Loss and Interaction Loss. By
incorporating the Interaction Loss, we mitigate the issue of joint
penetration, resulting in improved motion generation quality. To provide a
quantitative analysis, we employ FID and the penetration metric sdf
loss(SDF)[23] for evaluation. SDF represents the Signed Distance Field, which
could be defined as:

$\phi(x,y,z)=-\min(SDF(x,y,z),0),$ (4)

where $(x,y,z)$ denotes the position of MANO vertex. According to the
definition, when the hand joints penetrate with each other, $\phi$ would take
the positive value while zero otherwise. Using the formulation, we can measure
the severity of penetration during the whole motion. Table 4 demonstrates that
our model achieves the highest performance when using the proper motion
representation and Interaction Loss.

Tasks | Unconstrained | In-betweening | Trajectory Control
---|---|---|---
| FID$\downarrow$ | SDF.$\downarrow$ | FID $\downarrow$ | FID$\downarrow$
Real | 0.050 | 1.146 | - | -
w Global rep. | 0.168 | 1.575 | 0.282 | 0.415
w Local rep. | 0.436 | 1.988 | 0.572 | 0.182
w/o Interaction Loss | 0.176 | 1.652 | 0.318 | 0.216
Complete model | 0.168 | 1.575 | 0.282 | 0.182

Table 4: Quantitative comparisons of different hand motion representations and
w/o Interaction Loss. SDF indicates the sdf loss for penetration of joints
during the whole motion. Note that the mean of sdf loss is only calculated on
frames with penetraion.

##  6 Conclusion

Figure 9: Ablation Study. In the absence of Shape Loss, the model generates
abnormal hand shapes. Similarly, the absence of Interaction Loss leads to hand
interaction motion with severe penetration.

We present HandDiffuse12.5M, the largest interacting hands dataset with strong
interaction and temporal sequences, to open up the research direction of
interacting hands motion generation. HandDiffuse12.5M consists of various
modalities, including synchronous 50-view videos, 3D keypoints and hand poses.
Based on HandDiffuse12.5M, we propose a strong baseline method HandDiffuse
which is the first interacting hands motion generation approach along with two
specific controllers: key frames control and trajectory control. Our
evaluation in multi-stage demonstrates the benefit of our approach againist
others and HandDiffuse fills the blank in human motion generation.

## References

  * Baowen et al. [2021] Zhang Baowen, Wang Yangang, Deng Xiaoming, Zhang Yinda, Tan Ping, Ma Cuixia, and Wang Hongan.  Interacting two-hand 3d pose and shape reconstruction from single color image.  In _International Conference on Computer Vision (ICCV)_ , 2021.
  * Cai et al. [2021] Yujun Cai, Yiwei Wang, Yiheng Zhu, Tat-Jen Cham, Jianfei Cai, Junsong Yuan, Jun Liu, Chuanxia Zheng, Sijie Yan, Henghui Ding, et al.  A unified 3d human motion synthesis model via conditional variational auto-encoder.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 11645–11655, 2021.
  * Chao et al. [2021] Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S. Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, Jan Kautz, and Dieter Fox.  DexYCB: A benchmark for capturing hand grasping of objects.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2021.
  * Chen et al. [2023] Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu.  Executing your commands via motion diffusion in latent space.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 18000–18010, 2023.
  * Christian et al. [2019] Zimmermann Christian, Ceylan Duygu, Yang Jimei, Russel Bryan, Argus Max, and Brox Thomas.  Freihand: A dataset for markerless capture of hand pose and shape from single rgb images.  In _IEEE International Conference on Computer Vision (ICCV)_ , 2019.
  * Di and Yu [2022] Xinhan Di and Pengqian Yu.  Lwa-hand: Lightweight attention hand for interacting hand reconstruction, 2022.
  * Du et al. [2023] Yuming Du, Robin Kips, Albert Pumarola, Sebastian Starke, Ali Thabet, and Artsiom Sanakoyeu.  Avatars grow legs: Generating smooth human motion from sparse tracking inputs with diffusion model.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 481–490, 2023.
  * Fan et al. [2021] Zicong Fan, Adrian Spurr, Muhammed Kocabas, Siyu Tang, Michael Black, and Otmar Hilliges.  Learning to disambiguate strongly interacting hands via probabilistic per-pixel part segmentation.  In _International Conference on 3D Vision (3DV)_ , 2021.
  * Fan et al. [2023] Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, and Otmar Hilliges.  ARCTIC: A dataset for dexterous bimanual hand-object manipulation.  In _Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2023.
  * Gao et al. [2022] Daiheng Gao, Yuliang Xiu, Kailin Li, Lixin Yang, Feng Wang, Peng Zhang, Bang Zhang, Cewu Lu, and Ping Tan.  DART: Articulated Hand Model with Diverse Accessories and Rich Textures.  In _Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2022.
  * Garcia-Hernando et al. [2018] Guillermo Garcia-Hernando, Shanxin Yuan, Seungryul Baek, and Tae-Kyun Kim.  First-person hand action benchmark with rgb-d videos and 3d hand pose annotations.  In _Proceedings of Computer Vision and Pattern Recognition (CVPR)_ , 2018.
  * Google [2021] Google.  Mediapipe, 2021.
  * Grauman et al. [2022] Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Ziwei Zhao, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Christian Fuegen, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, and Jitendra Malik.  Ego4d: Around the world in 3,000 hours of egocentric video, 2022.
  * Guo et al. [2022] Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng.  Generating diverse and natural 3d human motions from text.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 5152–5161, 2022.
  * Hampali et al. [2020] Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vincent Lepetit.  Honnotate: A method for 3d annotation of hand and object poses.  In _CVPR_ , 2020.
  * Hampali et al. [2022] Shreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, and Vincent Lepetit.  Keypoint transformer: Solving joint identification in challenging hands and object interactions for accurate 3d pose estimation.  In _IEEE Computer Vision and Pattern Recognition Conference_ , 2022.
  * Han et al. [2020] Shangchen Han, Beibei Liu, Randi Cabezas, Christopher Twigg, Peizhao Zhang, Jeff Petkau, Tsz-Ho Yu, Chun-Jung Tai, Muzaffer Akbay, Zheng Wang, Asaf Nitzan, Gang Dong, Yuting Ye, Lingling Tao, Chengde Wan, and Robert Wang.  Megatrack: monochrome egocentric articulated hand-tracking for virtual reality.  _ACM Transactions on Graphics_ , 39, 2020.
  * Hanbyul et al. [2015] Joo Hanbyul, Liu Hao, Tan Lei, Gui Lin, Nabbe Bart, Matthews Iain, Kanad Takeo, Nobuhara Shohei, and Sheikh Yaser.  Panoptic studio: A massively multiview system for social motion capture.  2015\.
  * Hao et al. [2022] Meng Hao, Jin Sheng, Liu Wentao, Qian Chen, Lin Mengxiang, Ouyang Wanli, and Luo Ping.  3d interacting hand pose estimation by hand de-occlusion and removal.  2022\.
  * Harvey et al. [2020] Fé lix G. Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal.  Robust motion in-betweening.  _ACM Transactions on Graphics_ , 39(4), 2020.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _Advances in Neural Information Processing Systems_ , 33:6840–6851, 2020.
  * Jiang et al. [2023] Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen.  Motiongpt: Human motion as a foreign language.  _arXiv preprint arXiv:2306.14795_ , 2023.
  * Jiang et al. [2020] Wen Jiang, Nikos Kolotouros, Georgios Pavlakos, Xiaowei Zhou, and Kostas Daniilidis.  Coherent reconstruction of multiple humans from a single image.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 5579–5588, 2020.
  * Karunratanakul et al. [2023] Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang.  Gmd: Controllable human motion synthesis via guided diffusion models.  _arXiv preprint arXiv:2305.12577_ , 2023.
  * Kaufmann et al. [2020] Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, and Otmar Hilliges.  Convolutional autoencoders for human motion infilling.  In _2020 International Conference on 3D Vision (3DV)_. IEEE, 2020.
  * Kim et al. [2021] Dong Uk Kim, Kwang In Kim, and Seungryul Baek.  End-to-end detection and pose estimation of two interacting hands.  In _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 11169–11178, 2021.
  * Kwon et al. [2021] Taein Kwon, Bugra Tekin, Jan Stühmer, Federica Bogo, and Marc Pollefeys.  H2o: Two hands manipulating objects for first person interaction recognition.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 10138–10148, 2021.
  * Li et al. [2023a] Lijun Li, Linrui Tian, Xindi Zhang, Qi Wang, Bang Zhang, Mengyuan Liu, and Chen Chen.  Renderih: A large-scale synthetic dataset for 3d interacting hand pose estimation.  _arXiv preprint arXiv:2309.09301_ , 2023a.
  * Li et al. [2023b] Lijun Li, Li’an Zhuo, Bang Zhang, Liefeng Bo, and Chen Chen.  Diffhand: End-to-end hand mesh reconstruction via diffusion models, 2023b.
  * Li et al. [2022] Mengcheng Li, Liang An, Hongwen Zhang, Lianpeng Wu, Feng Chen, Tao Yu, and Yebin Liu.  Interacting attention graph for single image two-hand reconstruction.  In _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2751–2760, 2022.
  * Liang et al. [2023] Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu.  Intergen: Diffusion-based multi-human motion generation under complex interactions.  _arXiv preprint arXiv:2304.05684_ , 2023.
  * Lin et al. [2021] Fanqing Lin, Connor Wilhelm, and Tony Martinez.  Two-hand global 3d pose estimation using monocular rgb.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , pages 2373–2381, 2021.
  * Lin and Amer [2018] Xiao Lin and Mohamed R Amer.  Human motion modeling using dvgans.  _arXiv preprint arXiv:1804.10652_ , 2018.
  * Liu et al. [2022] Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi.  Hoi4d: A 4d egocentric dataset for category-level human-object interaction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 21013–21022, 2022.
  * Moon [2023] Gyeongsik Moon.  Bringing inputs to shared domains for 3D interacting hands recovery in the wild.  In _CVPR_ , 2023.
  * Moon et al. [2020] Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee.  Interhand2.6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image.  In _European Conference on Computer Vision (ECCV)_ , 2020.
  * Moon et al. [2023] Gyeongsik Moon, Shunsuke Saito, Weipeng Xu, Rohan Joshi, Julia Buffalini, Harley Bellan, Nicholas Rosen, Jesse Richardson, Mize Mallorie, Philippe Bree, Tomas Simon, Bo Peng, Shubham Garg, Kevyn McPhail, and Takaaki Shiratori.  A dataset of relighted 3D interacting hands.  In _NeurIPS Track on Datasets and Benchmarks_ , 2023.
  * Park et al. [2023] JoonKyu Park, Daniel Sungho Jung, Gyeongsik Moon, and Kyoung Mu Lee.  Extract-and-adaptation network for 3d interacting hand mesh recovery, 2023.
  * Pavlakos et al. [2019a] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black.  Expressive body capture: 3d hands, face, and body from a single image.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019a.
  * Pavlakos et al. [2019b] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black.  Expressive body capture: 3d hands, face, and body from a single image.  In _Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)_ , 2019b.
  * Petrovich et al. [2021] Mathis Petrovich, Michael J Black, and Gül Varol.  Action-conditioned 3d human motion synthesis with transformer vae.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 10985–10995, 2021.
  * Qin et al. [2022] Jia Qin, Youyi Zheng, and Kun Zhou.  Motion in-betweening via two-stage transformers.  _ACM Transactions on Graphics (TOG)_ , 41(6):1–16, 2022.
  * Rempe et al. [2023] Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany.  Trace and pace: Controllable pedestrian animation via guided trajectory diffusion.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 13756–13766, 2023.
  * Romero et al. [2017] Javier Romero, Dimitrios Tzionas, and Michael J. Black.  Embodied hands: Modeling and capturing hands and bodies together.  _ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)_ , 36(6), 2017.
  * Rong et al. [2021] Yu Rong, Jingbo Wang, Ziwei Liu, and Chen Change Loy.  Monocular 3d reconstruction of interacting handsvia collision-aware factorized refinements.  In _International Conference on 3D Vision_ , 2021.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation, 2015.
  * Sener et al. [2022] F. Sener, D. Chatterjee, D. Shelepov, K. He, D. Singhania, R. Wang, and A. Yao.  Assembly101: A large-scale multi-view video dataset for understanding procedural activities.  _CVPR 2022_ , 2022.
  * Simon et al. [2017] Tomas Simon, Hanbyul Joo, Iain Matthews, and Yaser Sheikh.  Hand keypoint detection in single images using multiview bootstrapping.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017.
  * Song et al. [2020] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  _arXiv preprint arXiv:2010.02502_ , 2020.
  * Tevet et al. [2022a] Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or.  Motionclip: Exposing human motion generation to clip space.  _arXiv preprint arXiv:2203.08063_ , 2022a.
  * Tevet et al. [2022b] Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Amit H Bermano, and Daniel Cohen-Or.  Human motion diffusion model.  _arXiv preprint arXiv:2209.14916_ , 2022b.
  * Tevet et al. [2023] Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano.  Human motion diffusion model.  In _The Eleventh International Conference on Learning Representations_ , 2023.
  * Tzionas et al. [2016] Dimitrios Tzionas, Luca Ballan, Abhilash Srikantha, Pablo Aponte, Marc Pollefeys, and Juergen Gall.  Capturing hands in action using discriminative salient points and physics simulation.  _International Journal of Computer Vision (IJCV)_ , 2016.
  * Xie et al. [2023] Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang.  Omnicontrol: Control any joint at any time for human motion generation.  _arXiv preprint arXiv:2310.08580_ , 2023.
  * Yan et al. [2019] Sijie Yan, Zhizhong Li, Yuanjun Xiong, Huahan Yan, and Dahua Lin.  Convolutional sequence generation for skeleton-based action synthesis.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4394–4402, 2019.
  * Yang et al. [2021] Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, and Cewu Lu.  Cpf: Learning a contact potential field to model the hand-object interaction.  In _ICCV_ , 2021.
  * Yu et al. [2020] Zhixuan Yu, Jae Shin Yoon, In Kyu Lee, Prashanth Venkatesh, Jaesik Park, Jihun Yu, and Hyun Soo Park.  Humbi: A large multiview dataset of human body expressions.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020.
  * Zhang et al. [2016] Jiawei Zhang, Jianbo Jiao, Mingliang Chen, Liangqiong Qu, Xiaobin Xu, and Qingxiong Yang.  3d hand pose tracking and estimation using stereo matching, 2016.
  * Zhang et al. [2022a] Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu.  Motiondiffuse: Text-driven human motion generation with diffusion model.  _arXiv preprint arXiv:2208.15001_ , 2022a.
  * Zhang et al. [2021] Siwei Zhang, Yan Zhang, Federica Bogo, Marc Pollefeys, and Siyu Tang.  Learning motion priors for 4d human body capture in 3d scenes.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 11343–11353, 2021.
  * Zhang et al. [2022b] Siwei Zhang, Qianli Ma, Yan Zhang, Zhiyin Qian, Taein Kwon, Marc Pollefeys, Federica Bogo, and Siyu Tang.  Egobody: Human body shape and motion of interacting people from head-mounted devices.  In _European Conference on Computer Vision_ , pages 180–200. Springer, 2022b.
  * Zhao et al. [2020] Rui Zhao, Hui Su, and Qiang Ji.  Bayesian adversarial human motion synthesis.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020.
  * Zimmermann and Brox [2017] Christian Zimmermann and Thomas Brox.  Learning to estimate 3d hand pose from single rgb images.  Technical report, arXiv:1705.01389, 2017.  https://arxiv.org/abs/1705.01389.
  * Zuo et al. [2023] Binghui Zuo, Zimeng Zhao, Wenqian Sun, Wei Xie, Zhou Xue, and Yangang Wang.  Reconstructing interacting hands with interaction prior from monocular images.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2023.

##  Appendix

##  7 Experiment Details

###  7.1 Frechet Inception Distance (FID)

The Frechet Inception Distance (FID) is a metric used to quantify the
dissimilarity between the distributions of generated and ground truth data in
latent space. In our experiments, we employ FID as a crucial indicator to
measure the authenticity and plausibility of our generated data. And the
formula is

$FID=|\mu_{gt}-\mu_{gen}|^{2}+\text{Tr}(C_{gt}+C_{gen}-2\sqrt{C_{gt}C_{gen}}),$

where $\mu_{gt}$ and $\mu_{gen}$ are the mean feature vectors of ground truth
and generated motions, respectively; $C_{gt}$ and $C_{gen}$ are the covariance
feature matrices of real and generated motions, respectively; Tr$(\cdot)$
denotes the trace of the matrix. In the previous works[51], the feature is
extracted by a Inception network which is a classifier. However, in the
interacting hands datasets, it is challenging to label these complex
interactive movements, making it difficult to build a classifier. Therefore,
taking inspiration from MotionCLIP[50], we propose a self-supervised encoder-
decoder architecture for the reconstruction network, as depicted in Figure 10.

More specifically, we concatenate a $z_{tk}$ vector of equal dimensions to the
original motion representation $x^{1:T}$. This concatenated sequence
$x^{1:T+1}$ is then fed into a transformer encoder. From the encoder’s output,
we select the corresponding position $z_{x}$ as the feature representation for
the input motion segment. This obtained feature $z_{x}$ serves as the memory
for the decoder. Additionally, we use a T-length zero vector that undergoes
positional encoding as the target for reconstructing the motion in the
decoder. The reconstruction loss is computed using the L2 norm.

During experimentation, we observed that using the global representation of
the motion improves the convergence of the network. Therefore, in subsequent
experiments, we convert the motion to a global representation before
calculating the motion’s feature.

Figure 10: Feature Extractor for FID. Inspired by MotionCLIP[50], we train a
transformer with encoder-decoder architecture by using self-supervised
strategy.

###  7.2 Diversity

Diversity measures the variability in the resulting motion distribution. The
way we followed to calculate Diversity is as same as MDM[51]. We randomly
generate 1000 samples and select 200 of them for calculating.

##  8 HandDiffuse12.5M

Figure 11 has shown the diversity and complexity of our dataset
HandDiffuse12.5M. Figure 12 has shown our manual annotation tool.

Figure 11: Overview of HandDiffuse12.5M. Each image represents one frame of
hand motion from a different sequence.  Figure 12: Annotation Tool. Our
specially designed tool for hand joint annotation of severely occluded frames.

##  9 More Results

Figure 13 has shown the results of trajectory control task. In each sequence,
we adopt the wrists’ rotation and translation from InterHand2.6M[36] as our
control signal and we can generate different motion based on this trajectory.

Figure 13: Results of Trajectory Control by Sampling from InterHand2.6M. Note
that only Root orientation and position is given, the local motions for hand
joints are generated from our model.


# Progress on the union-closed conjecture and offsprings in winter 2022-2023

Stijn Cambie Extremal Combinatorics and Probability Group (ECOPRO), Institute
for Basic Science (IBS), Daejeon, South Korea, supported by the Institute for
Basic Science (IBS-R029-C4), E-mail<EMAIL_ADDRESS>
Mathematicians had little idea whether the easy-to-state union-closed
conjecture was true or false even after $40$ years. However, last winter saw a
surge of interest in the conjecture and its variants, initiated by the
contribution of a researcher at Google. Justin Gilmer made a significant
breakthrough by discovering a first constant lower bound for the proportion of
the most common element in a union-closed family.

##  1 Introduction of the Union-Closed conjecture

The union-closed conjecture is due to Peter Frankl111See also
https://en.wikipedia.org/wiki/P\%C3\%A9ter_Frankl and
https://www.nrc.nl/nieuws/2023/01/20/na-wiskundige-opwinding-op-sociale-media-
is-het-probleem-van-de-kleurige-knikkers-bijna-opgelost-2-a4154586., who
constructed the elegant statement in $1979$ after observing many implications
of the statement. Before fully stating it, we need to define crucial concepts
from set theory.

The ground set is generally denoted with $[n]=\\{1,2,\ldots,n\\}$, where
$n\in\mathbb{N}$ is a finite number. A subset $A\subseteq[n]$ is nothing more
than a set containing integers between $1$ and $n$, e.g.,
$A=\\{2,4,6\\}\subset[7]$.

A family $\mathcal{F}\subseteq 2^{[n]}$ is a collection of subsets of $[n]$.
Here $2^{[n]}$ contains all $2^{n}$ possible subsets of $[n]$, which includes
the empty set $\emptyset$ as well.

A family $\mathcal{F}$ is called union-closed if for every
$A,B\in\mathcal{F}$, the union $A\cup B$ belongs to $\mathcal{F}$. This can be
written as $\mathcal{F}=\mathcal{F}\cup\mathcal{F}$, where the latter equals
exactly $\\{A\cup B\mid A,B\in\mathcal{F}\\}.$ An example of such a family is
presented in Figure 1. An other example, for every $m\in\mathbb{N}$, is the
family $\mathcal{F}_{m}=\\{A\mid A\subseteq[m]\vee A=[k]\mbox{ for some
}m+1\leq k\leq m^{2}\\}$ which consists of the $2^{m}$ subsets of $[m]$, as
well as $m^{2}-m$ intervals consisting of the first $k$ natural numbers.

$\\{1,2\\}$$\\{1,2,3\\}$$\\{1,2,3,4\\}$$\\{1,3\\}$$\\{2\\}$$\\{1\\}$$\\{2,3\\}$
Figure 1: Example of union-closed family

The Union-closed conjecture can now be formally stated as follows.

######  Conjecture 1 (Union-closed conjecture).

If $\mathcal{F}\neq\\{\emptyset\\}$ is a union-closed family with ground set
$[n]$, then there exists an element $i\in[n]$ such that at least half of the
sets in $\mathcal{F}$ contain $i$.

Considering our previous example $\mathcal{F}_{m}$ for large $m$, one can
verify that it might be that only a small fraction of the elements of the
ground set are abundant (belong to at least half of the sets) and their
average proportion of sets to which they belong can tend to zero. Note that
this conjecture would be (arguably) false when taking an infinite ground set
$\mathbb{N},$ e.g. by considering the (union-closed) family of finite subsets
of $\mathbb{N}$.

This conjecture can also be formulated in many different ways. For example,
one can consider bitstrings in $\\{0,1\\}^{n}$ with the element-wise
$OR$-operation. For instance, when $n=4$ and
$\mathcal{F}=\\{0011,1100,1111\\}$, we note that $0011+1100=1111$. This family
is closed under the $OR$-operation, which corresponds to being union-closed in
the initial formulation.

Taking the complements of the set, one obtains the Intersection-closed sets
conjecture, which states that an intersection-closed family has an element in
its ground set appearing in at most half of the sets. In [3, Sec. 3], one can
also find a lattice-, graph-, and Salzborn-formulation.

On November 17, 2022, Justin Gilmer [10], a researcher at Google working in
machine learning, made a breakthrough by proving a first constant fraction for
Conjecture 1. Soon thereafter, as fast as a few days, his result made others
put improvements and related results on the preprint server Arxiv. In this
note, we summarize the contributions and progress that was made in the winter
of 2022–2023. We explain the main ideas of Gilmer’s approach (Section 2),
mention the forthcoming extensions of his method (Sections 3 and 4), as well
as an unsuccessful attempt (Section 5) and discuss other work related to the
Union-closed conjecture (Section 6).

##  2 The observations and key elements in the proof by Gilmer

A first elementary observation by Gilmer is that one can always prove a
statement by proving the contrapositive of that statement. Since the statement
of the union-closed conjecture is that simple already, it might be no one
considered that before. The contraposition of Conjecture 1 can be stated as
follows. If a non-empty family $\mathcal{F}$ has no element appearing in at
least half of the sets of $\mathcal{F}$, then $\mathcal{F}$ is not a union-
closed family. By remarking that $A\cup A=A$ for every set $A$, one knows that
$\mathcal{F}\subseteq\mathcal{F}\cup\mathcal{F},$ and thus
$\lvert\mathcal{F}\cup\mathcal{F}\rvert>\lvert\mathcal{F}\rvert$ whenever
$\mathcal{F}$ is not a union-closed family. While posing related questions and
studying counterexamples to variants of Conjecture 1 similar to the ones in
[8], Gilmer noted that the entropy of a family might play a role.222More
details on his journey/ thought process can be found in
https://www.youtube.com/watch?v=AZaP0EwjR_I&t The entropy $H(X)$ of a discrete
random variable $X$ equals the Shannon entropy of its probability
distribution. The latter can be purely presented with a formula. If each
possible outcome $x$ belongs to a (finite) set $A$, and has probability
$p_{x},$ then

$H(X)=-\sum_{x\in A}p_{x}\log_{2}p_{x}.$

When sampling uniformly at random from $\mathcal{F}$, the entropy will equal
$\log_{2}{\lvert\mathcal{F}\rvert}$ and no higher entropy is possible. If one
can sample from $\mathcal{F}\cup\mathcal{F}$ in such a way that the entropy is
larger than $\log_{2}{\lvert\mathcal{F}\rvert},$ then one can conclude that
$\lvert\mathcal{F}\cup\mathcal{F}\rvert>\lvert\mathcal{F}\rvert.$ This is
exactly the core of Gilmer’s approach.

More precisely, he proved the following statement.

######  Theorem 2.

Let $A$ and $B$ denote independent and identically distributed random
variables that sample from a common distribution over subsets of $[n]$. Assume
that for all $i\in[n]$, $\,\mathbb{P}[i\in A]\leq 0.01$. Then $H(A\cup B)\geq
1.26H(A)$.

As a corollary, by taking the uniform distribution over the subsets of $[n]$,
one knows that if $\mathcal{F}\subset 2^{[n]}$ is a family for which every
element is contained in no more than $1\%$ of the sets, then
$\lvert\mathcal{F}\cup\mathcal{F}\rvert\geq\lvert\mathcal{F}\rvert^{1.26}$.333As
a corollary of later work by Sawin, this is at least
$\lvert\mathcal{F}\rvert^{1.74}$ This implies that whenever
$\lvert\mathcal{F}\rvert\geq 2$, either
$\lvert\mathcal{F}\cup\mathcal{F}\rvert>\lvert\mathcal{F}\rvert$ (and so the
family is not union-closed) or there is an element appearing in at least a
$0.01$ fraction of the sets in $\mathcal{F}.$ From this, one can conclude that
Conjecture 1 is true for a half replaced by $0.01.$

######  example 3.

Let $\mathcal{F}=\\{\\{1\\},\\{2\\}\\}$ and thus
$\mathcal{F}\cup\mathcal{F}=\\{\\{1\\},\\{2\\},\\{1,2\\}\\}.$ Let $A$ and $B$
be i.i.d. random variables that output a set of $\mathcal{F}$ uniformly at
random. Then $\,\mathbb{P}(A=\\{1\\})=\,\mathbb{P}(A=\\{2\\})$ and analogously
for $B$, which implies

$\,\mathbb{P}(A\cup B=\\{1\\})=\,\mathbb{P}(A\cup B=\\{2\\})=\frac{1}{4}\mbox{
and }\,\mathbb{P}(A\cup B=\\{1,2\\})=\frac{1}{2}.$

Now $H(A)=2\cdot\frac{1}{2}\log_{2}2=1$ and $H(A\cup
B)=2\cdot\frac{1}{4}\log_{2}4+\frac{1}{2}\log 2=\frac{3}{2}(<\log_{2}3).$
Since $\log_{2}(2)<H(A\cup B),$ we conclude that it is impossible that $A\cup
B$ takes values in a family with only $2$ elements and thus
$\lvert\mathcal{F}\cup\mathcal{F}\rvert>\lvert\mathcal{F}\rvert,$ i.e.
Gilmer’s method verifies that $\mathcal{F}$ is not union-closed.

######  example 4.

Let $\mathcal{F}=\binom{[3]}{\leq 2}$ and thus
$\mathcal{F}\cup\mathcal{F}=2^{[3]}.$ Note that $\lvert\mathcal{F}\rvert=7$
and every $1\leq i\leq 3$ appears in exactly $3$ sets and thus in a
$\frac{3}{7}$ fraction. Let $A,B$ be i.i.d. random variables that output a set
of $\mathcal{F}$ uniformly at random. Then

$\displaystyle\,\mathbb{P}(A\cup B=\emptyset)$ $\displaystyle=\frac{1}{49}$
$\displaystyle\,\mathbb{P}(\lvert A\cup B\rvert=1)$
$\displaystyle=\frac{3}{49}$ $\displaystyle\,\mathbb{P}(\lvert A\cup
B\rvert=2)$ $\displaystyle=\frac{9}{49}$ $\displaystyle\,\mathbb{P}(A\cup
B=[3])$ $\displaystyle=\frac{12}{49}$

Now

$\displaystyle H(A)=$ $\displaystyle 7\frac{1}{7}\log_{2}(7)=\log_{2}(7)$
$\displaystyle\sim$ $\displaystyle 2.81$ $\displaystyle H(A\cup B)=$
$\displaystyle\frac{1}{49}\log_{2}(49)+3\frac{3}{49}\log_{2}(49/3)$
$\displaystyle+3\frac{9}{49}\log_{2}(49/9)+\frac{12}{49}\log_{2}(49/12)$
$\displaystyle\sim$ $\displaystyle 2.70$

and thus $H(A)>H(A\cup B)$. We conclude that this is an example for which
Gilmer’s method does not provide evidence that the family is not union-closed,
even while the maximum fraction of occurence of an element is $\frac{3}{7}.$

Note: Analogously, when $\mathcal{F}=\binom{[5]}{\leq 3}$, one can verify that
$H(A)=\log_{2}(26)\sim 4.7$ and $H(A\cup B)\sim 4.54.$ Every element appears
in a $\frac{11}{26}$ fraction in this case.

##  3 Quick refinement of Gilmer’s idea

The binary entropy function $h(p)=-(p\log_{2}p+(1-p)\log_{2}(1-p))$ plays a
role in the computations in the work of Gilmer. Noting that $h(p)\leq
h(2p-p^{2})$ whenever $p\leq\psi:=\frac{3-\sqrt{5}}{2}$, Gilmer claimed that
his ideas could be extended to prove a fraction equal to $\psi.$ The authors
of [1, 5, 18, 15] quickly implemented this approach. All four of these papers
essentially reduced Conjecture 1 for the constant $\psi$ to the following key
lemma, an inequality in one variable.

######  Lemma 5.

Let $\phi=\frac{\sqrt{5}+1}{2}$ and $0\leq x\leq 1,$ then $h(x^{2})\geq\phi
xh(x).$

The validity of this lemma was established in two different ways by [1] and
Sawin [18]. The former used accurate computer calculations and applied
interval arithmetic on three intervals, while the latter utilized a purely
calculus-based approach. Thanks to some communication between the authors of
[1] and [5], in [5] a reference to the formal proof of [1] was added. In [15]
the lemma was split in two parts without formal proof, but both can be
verified easily.

A short and more elegant proof for Lemma 5 was given later by Boppana [2],
even while the proof itself would originate from $1989$. This proof relies on
the following extension of the classical Rolle’s theorem, which follows from
observations in e.g. [12].

######  Theorem 6.

Let $f$ be a differentiable function on a interval $I$. Let $m(f)$ be the sum
of multiplicities of the roots of $f$ in $I$. Then $m(f^{\prime})\geq m(f)-1.$

By iterating the theorem three times, one finds $m(f)\leq
m(f^{\prime\prime\prime})+3$. Applying this result on the function
$f(x)=h(x^{2})-\phi xh(x)$ and counting the multiplicities of the roots
$0,\frac{1}{\phi}$ and $1$ of $f$, the conclusion that $f$ is nonnegative on
$[0,1]$ follows quickly. Once Lemma 5 is derived, the proof for Conjecture 1
for constant $\psi$ (instead of $0.5$) is rather short in each of the papers
[1, 5, 15, 18], indicated e.g. by the total length of the paper by Chase and
Lovett [5]. Their work has three steps. First, they extended the analytic
claim (Lemma 5) to the two-variate function
$f(x,y):=\frac{h(xy)}{h(x)y+h(y)x}$. Next they prove a strengthened inequality
between the entropy of $A\cup B$ and the one of $A$ and $B$, for random
variables $A$ and $B$ (not necessarily identical) on $\\{0,1\\}^{n}$ for which
every bit is $1$ with a bounded probability. Finally, they finish the proof of
their slightly more general statement that holds for approximate union-closed
families. The latter being families for which the union of two random drawn
sets belong to the family with a high probability.

One example which certifies the sharpness of their proof can be derived from
$\mathcal{F}_{1}+\mathcal{F}_{2}=\\{A\mid A\in\mathcal{F}_{1}\vee
A\in\mathcal{F}_{2}\\}$ where $\mathcal{F}_{1}=\binom{[n]}{\psi n+n^{2/3}}$
and $\mathcal{F}_{2}=\binom{[n]}{\geq(1-\psi)n}$. For this, one need to note
that $\lvert\mathcal{F}_{1}\rvert>>\lvert\mathcal{F}_{2}\rvert$ and that the
union of two (iid uniform sampled) random sets from $\mathcal{F}_{1}$ belongs
with very high probability to $\mathcal{F}_{2}.$ The expected size of the
union is slightly larger (with an additional term of the order $n^{2/3}$, i.e.
$\Theta(n^{2/3})$) than $n-(1-\psi)^{2}n=(1-\psi)n$, and since the variance on
the size is $O(n^{1/2})$, the union almost surely belongs to $\mathcal{F}_{2}$
as well. The conclusion is still valid when replacing the term $n^{2/3}$ by
any function $g(n)$ for which $n>>g(n)>>n^{1/2}.$

$\mathcal{F}_{1}$$\mathcal{F}_{2}$ Figure 2: An approximate union-closed
family whose elements appear in at most a $\psi+o(1)$ fraction.

In a different direction, in his paper, Gilmer included some ideas for a full
resolution of Conjecture 1, but some of these directions were immediately
proven not to hold by Sawin and Ellis [18, 7].

##  4 Further refinements and extensions related to Gilmer’s work

Sawin [18] gave a suggestion to improve the bound further, which given the
sharpness of the form for union-closed families may be considered surprising.
Hereby the essence is in a question purely stated in terms of probability
distributions. His suggestion was worked out by Yu [20] and Cambie [4]. Yu
[20] considered the approach in a slightly more general form initially and
made a lower bound computable by restricting to the suggestion of Sawin and
applying [1, Lem. 5] and the Krein-Milman theorem [13] to bound the support
(number of values with nonzero probability) of a joint distribution by $4$. A
numerical computation then yield a bound equal to (roughly) $0.38234.$ In
parallel, Cambie [4] found an upper bound for Sawin’s approach which indicates
that the improvement is way smaller than expected and one would hope for. The
construction is a discrete probability distribution with only two values
having nonzero probability, with the values determined by a system of
equations involving the entropy function. Additionally he proved that this
value is sharp, by first reducing the support to $3$ elements, where one of
the elements equals $1.$ Finally, the conclusion is derived from the
combination of $3$-dimensional plots, a numerical minimization problem and a
more precise solution for the case where the support has exactly two elements,
one of which equals $1$.

Finally, building upon the work of [5], Yuster [21] considered families that
are almost $k$-union-closed, meaning that the union of $k$ independent uniform
random sets from $\mathcal{F}$ belongs to $\mathcal{F}$ with high probability.
He conjectured a tight version for the minimum frequency (the proportion of
sets containing the element) of some element in such families, with the
threshold for this frequency being the unique real root in $[0,1]$ of
$(1-x)^{k}=x$, denoted by $\psi_{k}$. To understand the sharpness of his
conjecture and the intuition behind the choice of $\psi_{k}$, consider the
union of $\mathcal{F}_{1}=\binom{[n]}{\psi_{k}n^{+}n^{2/3}}$ and
$\mathcal{F}_{2}=\binom{[n]}{\geq(1-\psi_{k})n}$. If at least one set from
$\mathcal{F}_{2}$ is included among the $k$ sets drawn, the union is
guaranteed to belong to $\mathcal{F}_{2}$. If all $k$ sets belong to
$\mathcal{F}_{1}$, the expected size of the union is
$n-(1-\psi_{k})^{k}n+\Theta(n^{2/3})$, and since the variance is $O(n^{1/2})$,
the union almost surely belongs to $\mathcal{F}_{2}$ as well. The conjecture
is proven to be true for $k\leq 4$, while for larger values of $k$ a weaker
bound is established.

##  5 The final Eureka moment, not yet

When Scandone [19] uploaded a preprint claiming the full resolution of the
union-closed conjecture, there arose initially excitement. However, upon
closer examination it became clear that Scandone’s proposed solution had
several issues, including a significant flaw that requires revising the
underlying construction. This was communicated to Scandone by Terence Tao, and
the details of this issue are briefly explained later in this section.

Nevertheless, Scandone’s underlying idea holds potential and is worth
mentioning for the valuable intuition it provides for Gilmer’s approach. Let
$\mathcal{F}$ be a family which is not union-closed, so
$\mathcal{F}\cup\mathcal{F}\not=\mathcal{F}$. A random variable taking values
in $\mathcal{F}$ has entropy at most $\log_{2}\lvert\mathcal{F}\rvert$ and
equality occurs only for uniform sampling from $\mathcal{F}.$ By considering
various examples, e.g. $\mathcal{F}=\\{\\{1\\},\\{2\\}\\}$, the reader can
verify that there is no strategy to choose two random variables $A,B$ which
sample sets from $\mathcal{F}$, such that $A\cup B$ samples uniformly random
from $\mathcal{F}\cup\mathcal{F}$. On the other hand, if for every set
$A\in\mathcal{F}$ the probability of obtaining it is almost equal to the
original probability and a few other sets from
$(\mathcal{F}\cup\mathcal{F})\backslash\mathcal{F}$ happen with a small
probability, the entropy can increase. The reason for this is that the
derivative of $h$ (plotted in Figure 3) is a continuously decreasing function
on the interval $(0,1)$, with $h^{\prime}(0)=+\infty$. To provide a more
explicit explanation of Scandone’s idea, we describe his proposed construction
in detail.

Let $A,B$ be independent random variables that take any set of $\mathcal{F}$
uniformly at random. Define a $\mathcal{P}([n])$-valued random variable
$A^{\delta}$ (depending on $\delta$) through the relation

$\operatorname{Pr}[A^{\delta}=X]=(1-\delta)\operatorname{Pr}[A=X]+\delta\operatorname{Pr}[A\cup
B=X]\mbox{ for every }X\subseteq[n].$

For every $X\in\mathcal{F}$,
$\operatorname{Pr}[A^{\delta}=X]\geq(1-\delta)\operatorname{Pr}[A=X]$ and thus
for $\delta$ sufficiently small, we have
$h(\operatorname{Pr}[A^{\delta}=X])-h(\operatorname{Pr}[A=X])\gtrsim\delta/\lvert\mathcal{F}\rvert
h^{\prime}(1/\lvert\mathcal{F}\rvert).$444To be precise, we assume
$\lvert\mathcal{F}\rvert\geq 3$ and
$\frac{2}{\lvert\mathcal{F}\rvert}+\delta<1.$ On the other hand, for
$X\in(\mathcal{F}\cup\mathcal{F})\backslash\mathcal{F}$, let the probability
$p:=\operatorname{Pr}[A\cup B=X]$. We have that $h(\delta p)\sim-\delta
p(\log\delta+\log p-1)$. By choosing $\delta$ to be sufficiently small such
that $-\log\delta$ is much greater than
$\frac{1}{p}h^{\prime}(1/\lvert\mathcal{F}\rvert)$, we can ensure that
$H(A^{\delta})>H(A)$ holds.

$0.2$$0.4$$0.6$$0.8$$1$$0.5$$1$$x$$h(x)$ Figure 3: Plot of the binary entropy
function $h$

Equivalently, the variable $A^{\delta}$ can be obtained by considering, in
addition to $A$ and $B$, a Bernoulli random variable of parameter $\delta$,
$Z_{\delta}$, which determines whether we take $A\cup B$ or only $A$. The flaw
in the argument is that, in the process of revealing all the digits of
$A^{\delta}$ (computed using the chain rule for the entropy), the
indeterminacy provided by $Z_{\delta}$ (and the consequent improvement of the
bounds) is lost after the first step. More precisely, there is step in the
computations in which a conditional probability distribution has been
erroneously replaced by its expected value, and this produces the
aforementioned flaw in the argument. The comment of Tao can be rephrased as
follows, “the idea of modifying the union operation by Gilmer is promising,
but a single global bit $Z_{\delta}$ is not sufficient to do the job, and a
more involved construction is needed”.

##  6 A better understanding by progress in a different direction

In this final section, we conclude with the essence of a recent paper and two
preprints on the union-closed conjecture, which consider different aspects and
angles of attack on Conjecture 1.

While Frankl’s conjecture is about the existence of one abundant element
(element that appears in at least half of the sets) in the family, it is also
natural to wonder if there are more abundant elements, assuming that all sets
in the family are sufficiently large. The following conjecture by Cui and Hu
[6] would imply Conjecture 1.

######  Conjecture 7.

If $\mathcal{F}$ is a finite union-closed family of sets whose smallest set is
of size at least $2$, then there are at least two elements such that each
belong to more than half of the sets of $\mathcal{F}$.

At the end of $2022$, the three authors of [11] considered this different
direction and proved that Conjecture [6] is not true when replacing $2$ by a
larger integer. They proved (among other results) that there are families all
of whose sets have size at least $k$, where $k$ can be arbitrary large, which
do only have $2$ abundant elements. The main construction is the family
$\mathcal{P}^{12}_{4}$. The family $\mathcal{P}^{12}_{4}$ consists of all
subsets $S$ of $\\{0,1,\dots,11\\}$ of size at least $4$ such that either
$\\{0,1\\}\subset S$, or $0\in S$ and $S\subseteq\\{0,2,\dots,10\\}$, or $1\in
S$ and $S\subseteq\\{1,3,\dots,11\\}$. The reader can verify that
$\lvert\mathcal{P}^{12}_{4}\rvert=(2^{10}-11)+2\cdot 16=1045$, while every
element $2\leq i\leq 11$ only appears $2^{9}-1+11=522$ times. One way to
increase the size of sets in families with non-abundant elements is to
duplicate an element within the sets. However, this creates blocks of size at
least $2$. A block is defined by Poonen [16] as a maximum set of elements that
all belong to the exact same sets of a family. Poonen also noted that to prove
Conjecture 1, it is sufficient to focus on families for which no block is a
singleton. Due to this, it is interesting to note that the construction of the
family $\mathcal{P}^{12}_{4}$ in [11] can be extended to such families.Let
$k\geq 3$ be a fixed integer and let $n$ be a sufficiently large even integer
as a function of $k$ ($n\geq 10k$ works). Let $E_{n}=\\{i\in[n]\mid i\equiv
0\pmod{2}\\}$ and $O_{n}=\\{i\in[n]\mid i\equiv 1\pmod{2}\\}$ be the set of
even and odd integers in $[n]$ respectively. Consider the family
$\mathcal{P}^{n}_{k}$ consisting of subsets $S$ of $[n]$ of size at least $k$,
such that either

  * •

$\\{1,2\\}\subset S$,

  * •

$S\subset E_{n}$ and $2\in S$, or

  * •

$S\subset O_{n}$ and $1\in S$.

It is clear that $1$ and $2$ are abundant elements. Now the other elements
appear all equally often (by symmetry) and by a small bijection and counting
argument, we conclude that these elements are not abundant whenever

$\binom{n-3}{k-3}<2\binom{n/2-2}{\geq k-1}.$

Since this is the case for $n$ sufficiently large, the conclusion is clear.

Another result related with union-closed families and the smallest set size,
was published early $2023$. Ellis, Ivan and Leader [9] proved that for every
$k\in\mathbb{N}$, there exists a union-closed family in which the (unique)
smallest set has size $k$, but where each element of this set has frequency
$(1+o(1))\frac{\log k}{2k}.$ As such, proving that focusing on the smallest
set cannot work in the strongest possible sense. They also proposed the
problem of verifying the union-closed conjecture for a family for which they
were unable to verify the statement. The latter was verified by Pulaj and Wood
[17]. They also proved new bounds on the least number $m$ (given $k$ and $n$)
such that every union-closed family $\mathcal{F}$ containing any
$\mathcal{A}\subseteq\binom{[n]}{k}$ with $\lvert\mathcal{A}\rvert=m$ as a
subfamily, satisfies Conjecture 1.

We can conclude that despite the progress that originates from the
breakthrough of Justin Gilmer, the exact version of Conjecture 1 is still not
proven. Mathematicians are still thinking about other directions or
modifications of the strategy and hope to resolve Conjecture 1 in the future.
Taking into account that the improvement by taking combinations suggested by
Sawin [18] turned out to be tinier than expected and hoped for, as illustrated
by the example in [4], it seems that the focus should go towards essential new
ideas. In particular, the union-closed conjecture might be a distraction of a
more general behaviour that
$\lvert\mathcal{F}\cup\mathcal{F}\rvert>\lvert\mathcal{F}\rvert^{c}$ for some
$c(\varepsilon)>1$ when every element of $[n]$ appears in less than a
$\frac{1}{2}-\varepsilon$ fraction of the sets in
$\mathcal{F}.$555communicated by Zachary Chase

Note added: In June 2023, Liu [14] improved the constant slightly with a
different method of coupling.

## Acknowledgements

We thank Zachary Chase, Justin Gilmer, Raffaele Scandone and Lei Yu for
internal communication while writing this manuscript.

## References

  * [1] R. Alweiss, B. Huang, and M. Sellke.  Improved Lower Bound for the Union-Closed Sets Conjecture.  arXiv e-prints, page arXiv:2211.11731, Nov. 2022.
  * [2] R. B. Boppana.  A Useful Inequality for the Binary Entropy Function.  arXiv e-prints, page arXiv:2301.09664, Jan. 2023.
  * [3] H. Bruhn and O. Schaudt.  The journey of the union-closed sets conjecture.  Graphs Combin., 31(6):2043–2074, 2015.
  * [4] S. Cambie.  Better bounds for the union-closed sets conjecture using the entropy approach.  arXiv e-prints, page arXiv:2212.12500, Dec. 2022.
  * [5] Z. Chase and S. Lovett.  Approximate union closed conjecture.  arXiv e-prints, page arXiv:2211.11689, Nov. 2022.
  * [6] Z. Cui and Z. Hu.  Two stronger versions of the union-closed sets conjecture.  Adv. Math. (China), 50(6):829–851, 2021.
  * [7] D. Ellis.  Note: a counterexample to a conjecture of Gilmer which would imply the union-closed conjecture.  arXiv e-prints, page arXiv:2211.12401, Nov. 2022.
  * [8] D. Ellis.  Union-closed families with small average overlap densities.  Electron. J. Combin., 29(1):Paper No. 1.11, 5, 2022.
  * [9] D. Ellis, I. Leader, and M.-R. Ivan.  Small Sets in Union-Closed Families.  Electron. J. Combin., 30(1):Paper No. 1.8–, 2023.
  * [10] J. Gilmer.  A constant lower bound for the union-closed sets conjecture.  arXiv e-prints, page arXiv:2211.09055, Nov. 2022.
  * [11] A. Kabela, M. Polák, and J. Teska.  The number of abundant elements in union-closed families without small sets.  arXiv e-prints, page arXiv:2212.09279, Dec. 2022.
  * [12] V. P. Kostov.  On arrangements of real roots of a real polynomial and its derivatives.  Serdica Math. J., 29(1):65–74, 2003.
  * [13] M. Krein and D. Milman.  On extreme points of regular convex sets.  Studia Math., 9:133–138, 1940.
  * [14] J. Liu.  Improving the Lower Bound for the Union-closed Sets Conjecture via Conditionally IID Coupling.  arXiv e-prints, page arXiv:2306.08824, June 2023.
  * [15] L. Pebody.  Extension of a Method of Gilmer.  arXiv e-prints, page arXiv:2211.13139, Nov. 2022.
  * [16] B. Poonen.  Union-closed families.  J. Combin. Theory Ser. A, 59(2):253–268, 1992.
  * [17] J. Pulaj and K. Wood.  Local Configurations in Union-Closed Families.  arXiv e-prints, page arXiv:2301.01331, Jan. 2023.
  * [18] W. Sawin.  An improved lower bound for the union-closed set conjecture.  arXiv e-prints, page arXiv:2211.11504, Nov. 2022.
  * [19] R. Scandone.  A proof of the union-closed sets conjecture.  arXiv e-prints, page arXiv:2302.03484, Feb. 2023.
  * [20] L. Yu.  Dimension-Free Bounds for the Union-Closed Sets Conjecture.  arXiv e-prints, page arXiv:2212.00658, Dec. 2022.
  * [21] R. Yuster.  Almost $k$-union closed set systems.  arXiv e-prints, page arXiv:2302.12276, Feb. 2023.