# Mobile MoCap: Retroreflector Localization On-The-Go

Gary Lvov∗, Mark Zolotas, Nathaniel Hanson, Austin Allison,
Xavier Hubbard, Michael Carvajal, Taşkin Padir Institute for Experiential
Robotics, Northeastern University, Boston, MA, USA*Corresponding author:
<EMAIL_ADDRESS>
###### Abstract

Motion capture through tracking retroreflectors obtains highly accurate pose
estimation, which is frequently used in robotics. Unlike commercial motion
capture systems, fiducial marker-based tracking methods, such as AprilTags,
can perform relative localization without requiring a static camera setup.
However, popular pose estimation methods based on fiducial markers have lower
localization accuracy than commercial motion capture systems. We propose
Mobile MoCap, a system that utilizes inexpensive near-infrared cameras for
accurate relative localization even while in motion. We present a
retroreflector feature detector that performs 6-DoF (six degrees-of-freedom)
tracking and operates with minimal camera exposure times to reduce motion
blur. To evaluate the proposed localization technique while in motion, we
mount our Mobile MoCap system, as well as an RGB camera to benchmark against
fiducial markers, onto a precision-controlled linear rail and servo. The
fiducial marker approach employs AprilTags, which are pervasively used for
localization in robotics. We evaluate the two systems at varying distances,
marker viewing angles, and relative velocities. Across all experimental
conditions, our stereo-based Mobile MoCap system obtains higher position and
orientation accuracy than the fiducial approach.

The code for Mobile MoCap is implemented in ROS 2 and made publicly available
at https://github.com/RIVeR-Lab/mobile_mocap.

##  I Introduction

Localization of people and robots is a critical need for the future of
automation. Recognizing the pose of dynamic objects enables a robot to not
only understand its current location in an environment, but also to ensure an
appropriate factor of safety. In settings where robots and humans occupy a
shared space, safety is paramount and the margin for error has to be
minimized. Artificial landmarks, such as fiducial markers, provide an easily
identifiable reference point for cameras to track objects in 3D space [1, 2,
3]. Recognizable marker geometries are used to localize fiducial markers and
to discriminate between objects. This technique has applications in virtual
reality [4], medicine [5], and robotics [6, 7, 8, 9].

Figure 1: Testing setup to evaluate a fiducial marker approach, AprilTags,
against the proposed Mobile MoCap. The central camera is in RGB mode for
AprilTag tracking. The outer two cameras are in IR mode for stereo Mobile
MoCap. All three cameras are the same model for a fair evaluation.

In robotics, relative localization of artificial landmarks or fiducials is
used to perform visual servoing [9]. This has additional applications to
autonomous vehicles, where landmarks and fiducials can be used to provide a
static reference point for simultaneous localization and mapping [10].
However, common conditions that arise in robotics applications, such as motion
blur, changes in perceived lighting, large viewing angles, and large
distances, can all decrease the accuracy of contemporary approaches. Poor
relative localization leads to erroneous robot movement when vision informs
control, resulting in potentially dangerous situations.

To address the limitations of current fiducial-based approaches, we present
Mobile MoCap, a stereo-based localization approach that uses retroreflectors,
as inspired by commercial motion capture systems [11]. Fig. 1 demonstrates an
overview of our proposed system. Retroreflectors are commercially available as
either flat, cut-to-size, textured sheets of adhesive-backed paper, or as
screw-mounted, textured spheres of plastic. Affixing three of these
retroreflectors to the same rigid target body is all that is needed to extract
said target’s 6D (six degrees-of-freedom,  _i.e._ , 6-DoF) pose by
triangulating the unique 3D positions of each of the attached retroreflectors.
These retroreflective markers reflect near infrared (NIR) light into an NIR
camera, making them easily identifiable under dim lighting conditions, or even
within scenes of complete darkness. Furthermore, unlike 2D fiducial markers,
retroreflectors can be arranged in a non-planar layout, making retroreflector-
based detection systems more robust to a wider range of IR camera viewing
angles.

Mobile MoCap has a twofold benefit over fiducial markers. First, by decreasing
the exposure time of the cameras, we inherently reduce motion blur while still
being able to isolate retroreflectors from the scene’s background. Second, by
not being constrained to planar mounting surfaces when distributing
retroreflectors over the target object’s unique geometry, we can achieve
better target tracking in 3D space.

We designed a robotic experimental testbed to autonomously benchmark our
Mobile MoCap system against AprilTags [1, 3], a popular fiducial marker
tracking approach. The robotic testbed consists of a camera assembly mounted
onto a servo and actuated linear rail, enabling each camera’s viewing angle
and distance to be controlled relative to a static target. In our experiments,
we consider different viewing angles, distances, and relative velocities to a
target. The target contains both retroreflectors and an AprilTag for the
purpose of comparing the marker localization accuracy of the two tracking
techniques. The overall setup was designed to mimic a scenario in which a
mobile robot aims to localize a static object of interest, such as a charging
station or a worker wearing a reflective safety vest.

The key contributions of this paper include:

  * •

A lightweight, inexpensive visual localization stereo system capable of real-
time 6-DoF tracking, which succeeds under dynamic conditions that are
challenging to contemporary fiducial marker-based approaches, while offering
superior accuracy.

  * •

A robotics testing platform to benchmark our proposed Mobile MoCap system
against a pervasive fiducial marker baseline, where the cameras are mounted
onto an actuated rail and perform relative localization at various distances,
viewing angles, and relative velocities.

  * •

An open-source software framework implemented atop the Robot Operating System
(ROS) 2 [12] and OpenCV [13] for easy interoperability with robotic systems
and inexpensive camera hardware.

##  II Related Work

There are numerous approaches to vision-based localization and tracking of
objects within an environment. One prominent approach is to track an object
directly, without using markers or any other landmarks. These “markerless”
approaches are attractive because they do not depend on extra hardware for
tracking [14]. For example, Region-based Convolutional Neural Networks
(R-CNNs) can perform markerless tracking by combining regional classification
with CNNs [15]. Nevertheless, learning-based markerless pose estimation
requires extensive training data and does not yet demonstrate localization
accuracy comparable to those of marker-based approaches [16].

In contrast, marker-based tracking methods rely on visually distinct landmarks
that can be easily identified by an optical camera. Active markers are a
common example, which require a power supply and supporting electronics to
function,  _e.g._ , pulsed RGB LEDs during human hand tracking [17]. These
markers excel under low-lighting conditions, but the requirement for a power
source makes them more resource-intensive than other solutions. Another
example are low-powered retroreflector tags that can be localized at larger
distances [18]. These tags are favored in high-speed vehicle applications,
despite the need for supporting hardware.

Passive markers are an alternative solution to remedy the issue of requiring
additional electronics or a power supply, thus offering a lightweight cost-
effective tracking approach. For instance, passive markers are a preferable
choice in full-body motion capture as they ensure that the participant is not
encumbered by supporting hardware [11]. One drawback of passive markers is
that they need direct line of sight with a camera, hence occlusions can
prevent effective tracking. Commercial motion capture rigs solve this issue by
leveraging multiple cameras to continuously track a single marker from all
viewpoints. However, the static configuration of cameras around the indoor
environment for commercial motion capture systems does not always translate
well to robotics, as cameras in robotics are often mounted at fixed positions
on the robot’s body and facing outwards.

Another common solution to marker-based tracking that is widely used in
robotics are fiducial markers [9]. Fiducials are visually distinct, flat,
black and white printed images,  _e.g._ , AprilTag [1] or ARTag [2]. These
fiducials are easy and quick to localize, even when using low resolution
images that are partially occluded or are not orthogonal to the camera’s line
of sight [1, 3]. Fiducial markers have a plethora of applications in robotics,
most notably in visual servoing and autonomous vehicles [10].

Passive fiducial tracking methods can be improved by incorporating
retroreflective material and multiple cameras into the setup. Monocular camera
setups are lower cost and less space obtrusive [19, 20, 21]. However, stereo
cameras offer distinct advantages, such as higher localization accuracy and
more effective acquisition of scene information [22]. Stereo motion capture
methods provide high precision localization useful for robotics [23]. In prior
work, stereo cameras with a wide field of view (FoV) and an occlusion
detection algorithm were shown to rival commercial motion capture systems,
albeit not in real-time [24]. Trinocular setups with NIR cameras have also
demonstrated the capacity to track passive markers at sub-tenth millimeter
precision [25]. Nonetheless, a multiple camera system can rapidly become
expensive and typically remains stationary.

Few existing marker-based tracking systems are mobile, while retaining the
localization accuracy of commercial motion capture technology. Previous work
in augmented reality for surgical use has utilized the stereo cameras on the
Microsoft HoloLens to track a surgical tool of known length [26]. The study’s
results were reported at sub-centimeter precision under this mobile stereo
configuration. We introduce a more general system that can have widespread use
in robotics, where stereo cameras must handle motion at significantly larger
speeds and distances than in augmented reality use-cases.

##  III Stereo-based Mobile MoCap

Given the prevalence of the ROS 2 [12] middleware, we implemented the entire
software and hardware stack of Mobile MoCap with the goals of being: 1) open-
source; 2) cross-platform; and 3) easily reproducible.

###  III-A System Hardware

A stereo NIR camera setup with an integrated light source can be constructed
from inexpensive off-the-shelf components.
ArduCam111https://www.arducam.com/product-category/uvc-usb-camera-module/usb-
uvc-cameras-night-vision/ offers an NIR USB camera that provides a camera
stream running at 30 frames per second (FPS) and 640x480p resolution. When
properly focused, the cameras have an FoV of $100^{\circ}$ horizontal and
$138^{\circ}$ diagonal. These cameras utilize a silicon-based photodetector
with broadband sensitivity. We permanently enable the IR cut filter by
removing the transition photoresistor to allow for IR tracking in well-
illuminated indoor environments, which would normally cause the camera to
autoswitch to the RGB space. IR LEDs adjacent to the camera lens provide
illumination at 850 nm, within the rated tolerance of the photodetector, but
outside human visual range. Creating a stereo NIR camera from the ArduCams
costs less than $100, including $20 in materials to fix the two cameras
together. For later benchmarking against AprilTag, a third unmodified ArduCam
of the same model is operated in the RGB space.

The monocular and stereo camera subsystems are combined in series on a laser
cut acrylic plate to enable benchmarking of AprilTags against the proposed
system. Tags are mounted on a continuous rotation servo (Dynamixel AX-12A).
The entire construction, including the AprilTag benchmarking camera, weighs
less than 250 grams and is less than 20 cm wide.

###  III-B ROS 2 Software Package

We have developed a ROS 2 package [12] to facilitate 6-DoF tracking (shown in
Fig. 2) of rigid retroreflector geometry, as well as 3D translation estimation
of individual retroreflectors. ROS 2 ensures easy interoperability with other
robotics systems and has a rich set of transform libraries that are also
suitable for geometric data processing. As detailed in Fig. 2, the core
algorithm steps are encapsulated as ROS 2 nodes, each occupying an independent
thread capable of running in parallel. Any image processing steps that exploit
OpenCV library [13] functions are developed in C++ for performance gains,
while the pose extraction and utility scripts are implemented in Python. To
the authors’ best knowledge, there is no open-source, publicly available
stereo retroreflector tracking package for ROS 2.

Figure 2: Feature detection pipeline deconstructed as a series of ROS 2 nodes.
(a) A raw camera image of four retroreflectors with exposure time minimized;
(b) Detection of contrasting edges on the retroreflectors; (c) Elimination of
nested contours within the detected region; and (d) the final detected
contours with their centroids.

###  III-C Filtering and Feature Detection

The two stereo cameras are initialized with their exposure times minimized
(500 $\mu s$). The resulting images are predominantly occupied by
retroreflectors and small sections of reflected light noise, as visualized in
Fig. 2a. When the 750 nm IR cut filter is engaged, light sources other than
the co-located IR LEDs are easily filtered out, as they appear distinctly
green in IR space. Finally, a global intensity threshold is applied to the
image, which can be tuned for the desired operating conditions.

For every incoming frame, the pixel coordinates of each potential marker
center are determined after filtering. The feature detector determines edges
in the image using a Canny Edge Detector (Fig. 2b) [27]. Multiple nested
detections are corrected for by infilling the selected contours (Fig. 2c),
resulting in the removal of redundant marker locations. Filling in nested
contours helps prevent a single marker from being detected as more than one
marker. Following the infilling process, an ellipse is fit to each contour.
The marker radius is approximated as the average of the major and minor axes
of the ellipse, with the ellipse’s center as the feature location. Fitting an
ellipse to the processed contours provides a better estimation of marker
centers when there are partial occlusions than other techniques, such as
fitting a minimum enclosing circle or applying the circle Hough Transform
[28]. Yet in situations where markers are completely occluded, maintaining
consistent tracking is difficult. This issue will be tackled in future work by
leveraging a greater number of markers per tracked object to ensure reliable
identification despite total occlusion of some markers. The final feature
detection results are shown in Fig. 2d.

###  III-D Correspondence Matching

Inputs: pixel coordinates $(x_{i},y_{i})\;\forall$ features $i\in$ left
camera; pixel coordinates $(x_{j},y_{j})\;\forall$ features $j\in$ right
camera;

Initialize: correspondences $\leftarrow$ []; assignments $\leftarrow$ [];

// find offset between one and every other feature $\forall i,j$

geomsi $\leftarrow$ $\forall$ features $i\,(x_{i},y_{i})$.$offset(\forall$
other features)

geomsj $\leftarrow$ $\forall$ features $j\,(x_{j},y_{j})$.$offset(\forall$
other features)

$sort$(geomsi); $sort$(geomsj)

shortest $\leftarrow$ $min$($length$(geomsi), $length$(geomsj))

for geometryi $\in$ geomsi do

for geometryj $\in$ geomsj do

error $\leftarrow$ $||$geometryi \- geometryj$||$

assignments.$append$(($i$, $j$), error)

end for

end for

$sort$(assignments) by increasing error

for assignment in assignments do

if features $i,j\in$ assignment not encountered then

correspondences.$append$(($i,j$))

end if

if $length$(correspondences) $=$ shortest then

return correspondences

end if

end for

return ambiguous correspondence

Algorithm 1 Geometric Correspondence Matching

Once the marker centers are determined for both frames, the 3D locations of
features are triangulated from corresponded feature pairs and the camera
intrinsics/extrinsics. A core benefit of stereo-based tracking is that the 3D
location of a feature can be triangulated from a single 2D feature pair, which
is not necessarily correlated to other features in the scene. Camera
projection matrices are determined by a stereo calibration.

An alignment method is used to match feature correspondences based on relative
geometries, as summarized in Algorithm 1. For each feature in the first frame,
the offset to every other feature is calculated and stored as a relative
geometry. For each feature in the second frame, the association that minimizes
the pixel difference for that feature’s relative geometry to the rest of the
features is then found. Only one correspondence is assumed to be correct, so
if there are several correspondences from a feature in the first frame to
features in the second frame, only the association with the lowest pixel error
is considered. This geometric matching approach is predicated on the
assumption that the cameras have roughly the same orientation, similar
extrinsics, and are not far enough apart to have vastly different views of
features. Geometric matching is used due to the sparsity of the image. Using
RANSAC [29] to estimate the homography between the two images fails to
determine correspondences, as keypoint detectors, such as ORB [30] and SURF
[31], do not locate a sufficient number of keypoints to correctly determine
correspondence in sparse low exposure time images.

Once the feature correspondence is found, it is used along with the cameras’
projection matrices and detected features to iteratively triangulate the 3D
locations of the features. This approach is akin to the Iterative-LS method
described by [32]. Triangulation allows a marker’s 3D translation to be
tracked by the cameras individually.

Extracting multiple object poses from 3D marker locations is left for future
work, as our current approach can only track a single rigid geometry comprised
of three markers arranged in a scalene triangle. This is due to the fact that
when there are multiple rigid bodies, it is non-trivial to associate the 3D
points that belong to a specific rigid body, especially when the 3D features
belonging to rigid bodies are occluded.

The points are corresponded to a known stored geometry through an exhaustive
search that correlates the longest edge between points to the longest edge in
the stored scalene triangle. Once the point correspondence is determined, the
translation and rotation are determined with a least-squares correspondence of
two 3D point sets [33].

##  IV Experiments

To evaluate our Mobile MoCap system, we designed a series of experiments using
AprilTags with a monocular camera as a benchmark. An overview of the
experimental setup is illustrated in Fig. 3.

Figure 3: Experimental setup to benchmark the performance of the linear rail
system against visual fiducial markers. Mobile MoCap cameras localize
themselves relative to the tracking target, as does an AprilTag camera.
Groundtruth is provided by the OptiTrack Trio, which determines the relative
pose of the target and cameras. Inset: Relative pose transforms from the
motion capture setup.

###  IV-A Experimental Setup

For the automated evaluation of our Mobile MoCap system, we designed a robotic
testing platform that consists of a servo-controlled camera assembly and a
linear rail (LoPro), as shown in Fig. 3. The linear rail, belt-driven with a
high-precision stepper motor, allows for a means of reliably moving the Mobile
MoCap assembly at a fixed rate with high repeatability. The rail’s
architecture is described further in [34]. To track the position of the moving
Mobile MoCap assembly relative to a static world frame, we used an OptiTrack
V120:Trio (Natural Point), which recorded data at 120 Hz during all trials and
serves as the ground truth for our experiments. Four 19 mm retroreflective
marker spheres (Natural Point) were arranged on the opposite side of the
acrylic plate to which the cameras are affixed for ground truth tracking with
the OptiTrack. The OptiTrack Trio provides sub-millimeter precision, which is
sufficient for evaluation. Mobile MoCap is intended as a compromise between
more expensive static commercial rigs, such as the OptiTrack Trio, and
inexpensive, less accurate fiducial marker approaches, like AprilTag. In order
to assess the robustness of our system, we conducted experiments where the
Mobile MoCap camera assembly was either static or moving (dynamic).

Using spherical (3D) retroreflectors instead of flat (2D) fiducial markers
would provide a non-negligible advantage to the Mobile MoCap system in
instances where the assembly’s onboard cameras point at markers from non-
orthogonal viewing angles. Therefore, we opted to test our system using only
the flat, paper-based variant of retroreflectors to allow for a fair
comparison against AprilTags.

To build a fair testbed for evaluation, we laser-cut a flat backing plate out
of acrylic. We then printed out an AprilTag Marker with a side length of 52
mm, and centered it on the acrylic plate. We cut three 30 mm circular markers
from a sheet of retroreflective paper (Qualisys) using a die-cutting machine
(Cricut) and arranged them asymmetrically on the plate, shown in Fig. 1a.
These 30 mm markers are larger than those typically used with commercial
systems. This is to ensure that the retroreflectors reflect light back into
the stereo camera at larger distances where the Mobile MoCap ArduCam’s
integrated IR LEDs are weaker than those included with commercial IR-tracking
cameras. In future work, Mobile MoCap will boast brighter IR LEDs in larger
quantities, housed in a dedicated light source. We then mounted this plate to
the distal end of the linear rail so that it was perpendicular to the table,
and could be seen from the Mobile MoCap testing rig, shown in Fig. 3.

###  IV-B Static Trials

As a base comparison point of pose estimation, we first tested the Mobile
MoCap system at various distances and angles from the marker plate. The
distances chosen were 0.9, 1.34, 1.78, and 2.23 m, and the angles chosen were
20, 10, and 0 degrees deflection from normal (yaw). The yaw rotation axis was
specifically chosen for testing, as this most aligns with the motion in which
wheeled robots operate. Each angle was recorded for 2 seconds at 30 FPS on all
three cameras (two cameras in IR for Mobile MoCap and one camera for AprilTag
in RGB). This was repeated 30 times for each angle and distance, for a total
of 120 trials.

###  IV-C Dynamic Angular Velocity Trials

Moreover, we evaluate the system’s robustness to rotational motion blur by
rotating the Mobile MoCap assembly through a 40 degree range of motion with
varying angular velocities. First, the Mobile MoCap assembly is fixed at a
distance 1 m away from the marker plate. Then, the assembly is rotated through
constant angular velocities of 0.05, 0.1, 0.2, and 0.4 rad/s. During these
trials, the three cameras on the mobile assembly are recording at 30 Hz. Each
angular velocity value is run 30 times, for a total of 120 trials.

###  IV-D Dynamic Linear Velocity Trials

Akin to the previous set of angular velocity trials, we also test the tracking
fidelity of our system while in translational motion. We moved the camera
assembly back and forth along the rail at constant velocities of 10, 20, 25,
and 30 cm/s, starting at 0.9 m and ending at 2.2 m. The overall process was
repeated five times.

##  V Results

To quantify estimation error in object tracking, we consider metrics related
to both position and orientation. Position error is calculated as the root
mean squared error (RMSE) between the OptiTrack’s observed 3D coordinate
$(x_{1},y_{1},z_{1})_{k}$ (“ground truth”) and its corresponding estimate
$(x_{2},y_{2},z_{2})_{k}$ for sample $k$. Given that the OptiTrack operates at
a significantly faster frequency than the AprilTag and Mobile MoCap tracking
methods, every sequence of estimates had to be synchronized to a common rate.
This synchronization procedure yields $n$ total samples and an RMSE metric on
3D position expressed as:

$p_{\mathrm{rmse}}=\sqrt{\sum_{k=1}^{n}\frac{(x_{1,k}-x_{2,k})^{2}+(y_{1,k}-y_{2,k})^{2}+(z_{1,k}-z_{2,k})^{2}}{n}}.$

To evaluate 3D rotation error, we consider an inner dot product of the ground
truth and estimated unit quaternions. This error function can be viewed as a
distance between orientations [35]. Let OptiTrack’s ground truth frame
orientation be $\mathbf{q}_{\mathrm{1},k}$, and the corresponding estimated
quaternion be $\mathbf{q}_{\mathrm{2},k}$, then the error in rotation for
sample $k$ is:

$q_{\mathrm{err},k}=\frac{\arccos{(|\mathbf{q}_{\mathrm{1,k}}\cdot\mathbf{q}_{\mathrm{2,k}}|)}}{n},$

where $q_{\mathrm{err},k}\in[0,\pi]$.

For each row of measurements reported in Tables I-III, the above metrics are
averaged over multiple trials conducted for the same parameter set. The values
for $x$, $y$, and $z$ are the mean positional error in each of the three axes
plus-minus the standard deviation of that axis’ relative error.

###  V-A Static

Table I: AprilTags – Static Trials Performance $d\,(\mathrm{m})$ | $a\,(\mathrm{rad})$ | $p_{\mathrm{rmse}}\,(\mathrm{cm})$ | $x\,(\mathrm{cm})$ | $y\,(\mathrm{cm})$ | $z\,(\mathrm{cm})$ | $q_{\mathrm{err}}\,(\mathrm{rad})$
---|---|---|---|---|---|---
0.9 | 0.0 | 5.11 |  -4.18 $\\!\pm\\!$ 0.1 |  7.46 $\\!\pm\\!$ 0.1 |  -2.18 $\\!\pm\\!$ 0.6 | 0.19
0.17 | 5.52 |  -4.23 $\\!\pm\\!$ 0.3 |  7.67 $\\!\pm\\!$ 0.7 |  0.54 $\\!\pm\\!$ 3.7 | 0.18
-0.34 | 7.57 |  -3.54 $\\!\pm\\!$ 2.1 |  9.02 $\\!\pm\\!$ 0.5 |  8.31 $\\!\pm\\!$ 2.0 | 0.11
1.3 | 0.0 | 8.82 |  -1.05 $\\!\pm\\!$ 0.2 |  6.50 $\\!\pm\\!$ 0.1 |  -13.67 $\\!\pm\\!$ 1.7 | 0.39
0.17 | 6.89 |  -1.67 $\\!\pm\\!$ 1.4 |  6.57 $\\!\pm\\!$ 0.8 |  -4.71 $\\!\pm\\!$ 8.5 | 0.36
-0.34 | 8.38 |  -0.93 $\\!\pm\\!$ 1.8 |  8.19 $\\!\pm\\!$ 0.8 |  9.25 $\\!\pm\\!$ 7.3 | 0.15
1.8 | 0.0 | 16.61 |  2.80 $\\!\pm\\!$ 0.9 |  6.16 $\\!\pm\\!$ 0.4 |  -26.62 $\\!\pm\\!$ 8.5 | 0.43
0.17 | 22.37 |  2.94 $\\!\pm\\!$ 3.4 |  6.91 $\\!\pm\\!$ 1.0 |  -34.23 $\\!\pm\\!$ 16.2 | 0.42
-0.34 | 7.99 |  -3.19 $\\!\pm\\!$ 2.9 |  8.54 $\\!\pm\\!$ 1.3 |  -3.84 $\\!\pm\\!$ 9.2 | 0.31
2.2 | 0.0 | 53.43 |  9.29 $\\!\pm\\!$ 1.7 |  7.96 $\\!\pm\\!$ 0.6 |  -90.40 $\\!\pm\\!$ 15.5 | 0.65
0.17 | 52.75 |  8.58 $\\!\pm\\!$ 4.1 |  8.04 $\\!\pm\\!$ 0.8 |  -87.76 $\\!\pm\\!$ 22.2 | 0.64
-0.34 | 16.48 |  1.98 $\\!\pm\\!$ 4.2 |  7.76 $\\!\pm\\!$ 1.2 |  -0.07 $\\!\pm\\!$ 27.1 | 0.27
Table II: Mobile MoCap – Static Trials Performance $d\,(\mathrm{m})$ | $a\,(\mathrm{rad})$ | $p_{\mathrm{rmse}}\,(\mathrm{cm})$ | $x\,(\mathrm{cm})$ | $y\,(\mathrm{cm})$ | $z\,(\mathrm{cm})$ | $q_{\mathrm{err}}\,(\mathrm{rad})$
---|---|---|---|---|---|---
0.9 | 0.0 | 1.91 | $-0.79\pm 0.0$ | $6.48\pm 0.0$ | $0.05\pm 0.0$ | 0.15
0.17 | 2.70 | $-0.48\pm 0.7$ | $6.76\pm 0.7$ | $1.83\pm 4.1$ | 0.15
-0.34 | 6.39 | $0.85\pm 2.2$ | $8.10\pm 0.5$ | $10.22\pm 2.3$ | 0.12
1.3 | 0.0 | 0.96 | $0.89\pm 0.1$ | $4.65\pm 0.1$ | $-2.67\pm 0.1$ | 0.17
0.17 | 1.97 | $1.05\pm 1.5$ | $5.09\pm 1.0$ | $-0.22\pm 5.8$ | 0.17
-0.34 | 5.89 | $2.01\pm 2.3$ | $6.89\pm 0.9$ | $8.76\pm 7.6$ | 0.18
1.8 | 0.0 | 0.34 | $3.05\pm 0.1$ | $3.58\pm 0.0$ | $-5.60\pm 0.07$ | 0.17
0.17 | 1.25 | $3.40\pm 1.5$ | $4.01\pm 1.1$ | $-3.67\pm 10.8$ | 0.16
-0.34 | 6.51 | $3.39\pm 1.8$ | $6.16\pm 1.2$ | $9.98\pm 7.7$ | 0.15
2.2 | 0.0 | 1.11 | $2.30\pm 0.1$ | $2.56\pm 0.1$ | $-8.20\pm 0.4$ | 0.29
0.17 | 0.60 | $2.50\pm 2.6$ | $3.56\pm 2.2$ | $-4.27\pm 10.9$ | 0.42
-0.34 | 10.08 | $-11.05\pm 8.3$ | $9.06\pm 3.5$ | $-28.25\pm 17.6$ | 0.93

The results in Tables I and II are promising for our proposed stereo
retroreflector system. Every $p_{\mathrm{rmse}}$ value for Mobile MoCap is
less than its corresponding value for AprilTags, with sub-two centimeter error
at 0.9 m from the marker plate at zero degrees. One potential outlier is a
$p_{\mathrm{rmse}}$ value of 0.6 cm at 2.2 m and 0.17 radians in Table II,
which is lower than the preceding value. Nonetheless, for every other set of
three values at each distance with increasing angular deflection, the
$p_{\mathrm{rmse}}$ value shows a positively increasing trend. At -.34 rad at
1.8 m/2.2 m, AprilTag outperforms Mobile MoCap in estimating the $z$-axis,
although with larger variance. The $p_{\mathrm{rmse}}$ for Mobile MoCap is
lower indicating better overall performance for Mobile MoCap. This anomaly in
Mobile MoCap is likely due to the current lack of handling erroneously
triangulated markers when determining pose under noisy scene conditions, such
as when the camera is tilted further from the normal to the target. Future
work will filter out erroneously triangulated markers to increase Mobile
MoCap’s effectiveness in more noisy scenarios.

###  V-B Dynamic Angular Velocity

Table III: Dynamic Angular Velocity Trials Performance $a_{vel}\,(\mathrm{rad/s})$ | AprilTag | Mobile MoCap
---|---|---
$p_{\mathrm{rmse}}\,(\mathrm{cm})$ | $q_{\mathrm{err}}\,(\mathrm{rad})$ | $p_{\mathrm{rmse}}\,(\mathrm{cm})$ | $q_{\mathrm{err}}\,(\mathrm{rad})$
0.05 | 8.86 | $0.165\pm 0.09$ | 3.88 | $0.156\pm 0.05$
0.1 | 9.07 | $0.130\pm 0.07$ | 5.02 | $0.143\pm 0.05$
0.2 | 9.16 | $0.146\pm 0.08$ | 5.53 | $0.142\pm 0.05$
0.4 | 8.90 | $0.162\pm 0.10$ | 5.04 | $0.155\pm 0.09$

The results in Table III show a significant trend in favor of Mobile MoCap. In
all of the angular rotation settings, the Mobile MoCap system demonstrates
smaller positional error than AprilTags. Although the improvement is marginal,
the results were collected at 1 m distance, where the visual recognition of
the AprilTag is still reliable. At further distances, up to the maximum range
of the system, the Mobile MoCap performance would likely be superior. The
orientation error is also notably lower for Mobile MoCap, although the
performance increase at this distance is still within a standard deviation of
the AprilTag. Across both localization systems, increasing $a_{vel}$ is
directly correlated with an increase in pose error.

###  V-C Dynamic Linear Velocity

Table IV: Dynamic Linear Velocity Trials Performance $l_{vel}\,(\mathrm{cm/s})$ | AprilTag | Mobile MoCap
---|---|---
$p_{\mathrm{rmse}}\,(\mathrm{cm})$ | $q_{\mathrm{err}}\,(\mathrm{rad})$ | $p_{\mathrm{rmse}}\,(\mathrm{cm})$ | $q_{\mathrm{err}}\,(\mathrm{rad})$
10 | 9.09 | $0.170\pm 0.10$ | 2.62 | $0.140\pm 0.03$
20 | 31.01 | $0.332\pm 0.15$ | 1.60 | $0.178\pm 0.07$
25 | 14.93 | $0.190\pm 0.12$ | 2.35 | $0.147\pm 0.06$
30 | 23.8 | $0.256\pm 0.16$ | 2.10 | $0.145\pm 0.07$

The values in Table IV show a number of distinct trends. As the constant
linear velocity values increase, the $p_{\mathrm{rmse}}$ values for AprilTags
gradually increase, while the $p_{\mathrm{rmse}}$ values for Mobile MoCap
remain more or less the same. This fruitful result suggests that the Mobile
MoCap system is robust to motion blur. Additionally, the normalized
$q_{\mathrm{rmse}}$ values associated with orientation error are on average
larger for AprilTags than for Mobile MoCap. In general, these results
demonstrate the full potential of the Mobile MoCap system as a mobile
solution, whereby minimizing motion blur poses less risk of errors in
localization. Lastly, it is worth noting that the standard deviations in
position and orientation errors are significantly less than those of
AprilTags.

##  VI Discussion

Fiducial markers, such as AprilTags [1, 3], are limited by the minimum
resolvable distance between points in an image. As AprilTags are based on a
lexographic coding system, the detection pipeline aims to detect line segments
and quads within a target area. At long distance, especially when the camera
has low resolution or poor focus, the number of pixels occupied by a quad is
small, thus decreasing the likelihood of detection. The Mobile MoCap system
overcomes this challenge by relying on simpler geometric features that are
recognized individually, rather than in the context of a larger grid encoding.
While Mobile MoCap showed similar positional tracking performance when
evaluated against AprilTags, there was a noticeable difference in dynamic
angular tracking performance. This is likely due to the vast reduction in
motion blur artifacts due to the reduced exposure time, which is a unique
feature of Mobile MoCap. These experiments were performed to evaluate the
viability of future tests involving a moving camera system and a moving
target, which will be covered in future work extending Mobile MoCap in the
muli-vehicle domain.

The Mobile MoCap system is not without limitations. Unlike AprilTags, there is
no inherent association between marker configuration and object identifier
(ID). It is possible to associate a marker geometry to a specific ID, but this
is not yet a built-in feature, unlike existing fiducial marker libraries.
Similar to other more expensive systems, our system is also subject to noise
from highly reflective surfaces, such as metallic screw heads or reflective
trim on articles of clothing. These items can be falsely detected as features
at close distances. Most indoor lighting is provided by LED or fluorescent
lights spanning the visible light spectrum, resulting in low noise from indoor
lights. However, the sun provides a broader spectrum, and incidental sunlight
can be falsely detected as a marker. The current iteration of this system is
best suited for indoor use, or outdoor use during the night. Nevertheless, our
results demonstrate that our system is functional and accurate in dynamic
environments where more expensive commerical motion tracking systems optimally
operate.

##  VII Conclusions

In this work we presented Mobile MoCap, a novel motion capture system
architecture that is both inexpensive and portable. We demonstrated the
system’s superior performance over a popular fiducial marker approach,
AprilTags, in terms of object 6-DoF localization accuracy. In future
iterations of this work, we plan to exchange the camera system for a system
capable in the Short Wave IR range, allowing for robust performance both
indoors and outdoors. Additionally, this system will be mounted to a variety
of mobile robots to explore multi-vehicle tracking. Our system and methods
empirically demonstrate motion capture systems can be made in cost-effective
formats, while still providing excellent localization accuracy in robotics
contexts.

## Acknowledgment

The authors would like to thank Alina Spectorov for aid in debugging the
Mobile MoCap software. This research has been supported in part by Defense
Advanced Research Projects Agency (DARPA) under HR0011-22-2-0004. The views
expressed are those of the authors and do not reflect the official policy or
position of the Department of Defense or the U.S. Government. Approved for
Public Release, Distribution Unlimited.

## References

  * [1] E. Olson, “Apriltag: A robust and flexible visual fiducial system,” in _IEEE International Conference on Robotics and Automation_ , 2011, pp. 3400–3407.
  * [2] M. Fiala, “Artag, a fiducial marker system using digital techniques,” in _IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , vol. 2, 2005, pp. 590–596.
  * [3] J. Wang and E. Olson, “Apriltag 2: Efficient and robust fiducial detection,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2016, pp. 4193–4198.
  * [4] M. Simonetto, S. Arena, and M. Peron, “A methodological framework to integrate motion capture system and virtual reality for assembly system 4.0 workplace design,” _Safety Science_ , vol. 146, p. 105561, 2022.
  * [5] A. C. Alarcón-Aldana, M. Callejas-Cuervo, and A. P. L. Bo, “Upper limb physical rehabilitation using serious videogames and motion capture systems: A systematic review,” _Sensors_ , vol. 20, no. 21, p. 5989, 2020.
  * [6] J. Chen, C. Sun, and A. Zhang, “Autonomous navigation for adaptive unmanned underwater vehicles using fiducial markers,” in _IEEE International Conference on Robotics and Automation_ , 2021, pp. 9298–9304.
  * [7] M. Zolotas, J. Elsdon, and Y. Demiris, “Head-mounted augmented reality for explainable robotic wheelchair assistance,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2018, pp. 1823–1829.
  * [8] B. Pfrommer and K. Daniilidis, “Tagslam: Robust slam with fiducial markers,” _arXiv:1910.00679_ , 2019.
  * [9] M. Kalaitzakis, B. Cain, S. Carroll, A. Ambrosi, C. Whitehead, and N. Vitzilaios, “Fiducial markers for pose estimation: Overview, applications and experimental comparison of the artag, apriltag, aruco and stag markers,” _Journal of Intelligent & Robotic Systems_, vol. 101, pp. 1–26, 2021.
  * [10] B. Pfrommer and K. Daniilidis, “Tagslam: Robust slam with fiducial markers,” _arXiv:1910.00679_ , 2019.
  * [11] M. Field, D. Stirling, F. Naghdy, and Z. Pan, “Motion capture in robotics review,” in _IEEE International Conference on Control and Automation_ , 2009, pp. 1697–1702.
  * [12] S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” _Science Robotics_ , vol. 7, no. 66, p. eabm6074, 2022.
  * [13] G. Bradski, “The OpenCV Library,” _Dr. Dobb’s Journal of Software Tools_ , 2000.
  * [14] J. Li, B. Wang, S. Zhu, X. Cao, F. Zhong, W. Chen, T. Li, J. Gu, and X. Qin, “Bcot: A markerless high-precision 3d object tracking benchmark,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 6697–6706.
  * [15] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , June 2014.
  * [16] N. Nakano, T. Sakura, K. Ueda, L. Omura, A. Kimura, Y. Iino, S. Fukashiro, and S. Yoshioka, “Evaluation of 3d markerless motion capture accuracy using openpose with multiple video cameras,” _Frontiers in Sports and Active Living_ , vol. 2, p. 50, 2020.
  * [17] P. Ruppel, N. Hendrich, and J. Zhang, “Low-cost multi-view pose tracking using active markers,” in _IEEE International Conference on Industrial Cyber Physical Systems_ , 2019, pp. 261–268.
  * [18] E. Soltanaghaei, A. Prabhakara, A. Balanuta, M. Anderson, J. M. Rabaey, S. Kumar, and A. Rowe, “Millimetro: mmwave retro-reflective tags for accurate, long range localization,” in _International Conference on Mobile Computing and Networking_ , 2021, pp. 69–82.
  * [19] B. P. Vagvolgyi, R. P. Jayakumar, M. S. Madhav, J. J. Knierim, and N. J. Cowan, “Wide-angle, monocular head tracking using passive markers,” _Journal of Neuroscience Methods_ , vol. 368, p. 109453, 2022.
  * [20] J. Lichtenauer and M. Pantic, “Monocular omnidirectional head motion capture in the visible light spectrum,” in _IEEE International Conference on Computer Vision Workshops_ , 2011, pp. 430–436.
  * [21] M. T. Smith, M. Livingstone, and R. Comont, “A method for low-cost, low-impact insect tracking using retroreflective tags,” _Methods in Ecology and Evolution_ , vol. 12, no. 11, pp. 2184–2195, 2021.
  * [22] P. Azad, T. Asfour, and R. Dillmann, “Stereo-based vs. monocular 6-dof pose estimation using point features: A quantitative comparison,” in _Autonome Mobile Systeme 2009: 21. Fachgespräch Karlsruhe, 3./4. Dezember 2009_. Springer, 2009, pp. 41–48.
  * [23] C. Möller, H. C. Schmidt, N. H. Shah, and J. Wollnack, “Enhanced absolute accuracy of an industrial milling robot using stereo camera system,” _Procedia Technology_ , vol. 26, pp. 389–398, 2016.
  * [24] A. Islam, M. Asikuzzaman, M. O. Khyam, M. Noor-A-Rahim, and M. R. Pickering, “Stereo vision-based 3d positioning and tracking,” _IEEE Access_ , vol. 8, pp. 138 771–138 787, 2020.
  * [25] S. Bi, Y. Gu, J. Zou, L. Wang, C. Zhai, and M. Gong, “High precision optical tracking system based on near infrared trinocular stereo vision,” _Sensors_ , vol. 21, no. 7, p. 2528, 2021.
  * [26] C. Gsaxner, J. Li, A. Pepe, D. Schmalstieg, and J. Egger, “Inside-out instrument tracking for surgical navigation in augmented reality,” in _ACM Symposium on Virtual Reality Software and Technology_ , 2021, pp. 1–11.
  * [27] J. Canny, “A computational approach to edge detection,” _IEEE Transactions on pattern analysis and machine intelligence_ , no. 6, pp. 679–698, 1986.
  * [28] P. V. Hough, “Method and means for recognizing complex patterns,” Dec. 18 1962, uS Patent 3,069,654.
  * [29] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” _Communications of the ACM_ , vol. 24, no. 6, pp. 381–395, 1981.
  * [30] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “Orb: An efficient alternative to sift or surf,” in _International conference on computer vision_ , 2011, pp. 2564–2571.
  * [31] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (surf),” _Computer vision and image understanding_ , vol. 110, no. 3, pp. 346–359, 2008.
  * [32] R. I. Hartley and P. Sturm, “Triangulation,” _Computer Vision and Image Understanding_ , vol. 68, no. 2, pp. 146–157, 1997.
  * [33] K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-d point sets,” _IEEE Transactions on pattern analysis and machine intelligence_ , no. 5, pp. 698–700, 1987.
  * [34] N. Hanson, G. Lvov, and T. Padir, “Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection,” _Frontiers in Robotics and AI_ , vol. 9, 2022.
  * [35] D. Q. Huynh, “Metrics for 3d rotations: Comparison and analysis,” _Journal of Mathematical Imaging and Vision_ , vol. 35, pp. 155–164, 2009\.