# Discovery of a 115 Day Orbital Period in the Ultraluminous X-ray Source NGC
5408 X-1

Tod E. Strohmayer  Astrophysics Science Division, NASA’s Goddard Space Flight
Center, Greenbelt, MD 20771<EMAIL_ADDRESS>
###### Abstract

We report the detection of a 115 day periodicity in SWIFT/XRT monitoring data
from the ultraluminous X-ray source (ULX) NGC 5408 X-1. Our ongoing campaign
samples its X-ray flux approximately twice weekly and has now achieved a
temporal baseline of $\approx 485$ days. Periodogram analysis reveals a
significant periodicity with a period of $115.5\pm 4$ days. The modulation is
detected with a significance of $3.2\times 10^{-4}$. The fractional modulation
amplitude decreases with increasing energy, ranging from $0.13\pm 0.02$ above
1 keV to $0.24\pm 0.02$ below 1 keV. The shape of the profile evolves as well,
becoming less sharply peaked at higher energies. The periodogram analysis is
consistent with a periodic process, however, continued monitoring is required
to confirm the coherent nature of the modulation. Spectral analysis indicates
that NGC 5408 X-1 can reach 0.3 - 10 keV luminosities of $\approx 2\times
10^{40}$ ergs s-1. We suggest that, like the 62 day period of the ULX in M82
(X41.4+60), the periodicity detected in NGC 5408 X-1 represents the orbital
period of the black hole binary containing the ULX. If this is true then the
secondary can only be a giant or supergiant star.

black hole physics - galaxies: individual: NGC 5408 - stars: oscillations -
X-rays: stars - X-rays: galaxies

##  1 Introduction

The existence of black holes in the mass range from $10^{2}-10^{4}$
$M_{\odot}$–intermediate mass black holes (IMBH)–is still not widely accepted.
It has been argued based on their extreme luminosities that some of the bright
X-ray sources found in nearby galaxies, the ultraluminous X-ray sources
(ULXs), may be IMBHs (Colbert & Mushotzky 1999), œbut it has also been
suggested that these objects may appear luminous due to beaming of their X-ray
radiation (King et al. 2001). As yet there has been no direct measurement of
the mass of a ULX.

At present the best IMBH candidates include X41.4+60, the brightest ULX in the
starburst galaxy M82 (also referred to as M82 X-1, Kaaret, Feng & Gorski 2009;
Strohmayer & Mushotzky 2003), and the ULX NGC 5408 X-1 (Strohmayer et al.
2007; Kaaret & Corbel 2009). Very recently, Farrell et al. (2009) have
reported the identification of an X-ray source (2XMM J011028.1-460421) very
near the absorption line galaxay ESO 243-49. They argue for a physical
association with this galaxy which at a redshift of $z=0.0224$ implies a
luminosity in excess of $10^{42}$ erg s-1, and suggest it is an IMBH in excess
of 500 $M_{\odot}$. If the association with ESO 243-49 is correct then this
object is the most luminous ULX currently known.

X41.4+60 is also amongst the most luminous ULXs, on occasion having a
luminosity upwards of $10^{41}$ ergs s-1 (Kaaret et al. 2009). This object
also shows a 62 day periodic modulation in its X-ray flux, which has been
proposed to be the orbital period of the binary system containing the black
hole (Kaaret, Simet & Lang 2006; Kaaret & Feng 2007). Both NGC 5408 X-1 and
X41.4+60 show quasiperiodic oscillations (QPOs) in their X-ray fluxes, and
these ULX QPOs appear at systematically lower frequencies than the analogous
QPOs observed in Galactic black hole binary systems (Strohmayer & Mushotzky
2003; Strohmayer et al. 2007). Indeed, Strohmayer & Mushotzky (2009) have
recently shown that the QPO properties in NGC 5408 X-1 correlate with the
energy spectrum and X-ray flux in a manner fully consistent with the behavior
observed for so-called Type C QPOs in Galactic systems. The Type C QPOs in
stellar mass black holes are strong (fractional rms amplitude $\sim 15\%$),
relatively coherent ($\nu_{qpo}/\Delta\nu>10$) oscillations with
characteristic frequencies of $\sim 1-10$ Hz that vary in frequency in
correlation with source flux and spectral index (see Sobczak et al 2000;
Vignarca et al. 2003; Casella, Belloni & Stella 2005). They are associated
with an energy spectral state in which approximately half or more of the flux
is carried by a power-law component with slope $\Gamma\sim 2-2.5$, commonly
referred to as the Steep Power Law state (SPL, McClintock & Remillard 2006),
or the Hard Intermediate State (HIMS, Belloni 2006). The precise origin of
these QPOs is still uncertain, but their phenomenology has been used to
estimate the masses of black holes (see Shaposhnikov & Titarchuk 2009).

The Type C QPOs in NGC 5408 X-1 identified by Strohmayer & Mushotzky (2009)
have frequencies of a few tens of mHz, a factor of 100 lower than the Galactic
black holes. Scaling the observed QPO frequencies in NGC 5408 X-1 to those
observed in Galactic systems of known mass suggests that NGC 5408 X-1 is an
IMBH with a mass greater than 1000 $M_{\odot}$ (Strohmayer & Mushotzky 2009).
Moreover, recent optical spectroscopy reported by Kaaret & Corbel (2009)
indicates that the optical counterpart to NGC 5408 X-1 is associated with
reprocessed emission from an accretion disk as well as a strongly photoionized
optical nebula. This, combined with the presence of a powerful radio nebula
also appears consistent with the presence of an IMBH (Lang et al. 2007; Soria
et al. 2006; Kaaret et al. 2003).

While it is likely that many of the ULXs are accreting black hole binary
systems, very little is known about the nature of these putative binaries. If
some of the brighter systems are indeed IMBHs accreting via Roche lobe
overflow, then their orbital periods could be as long as a few hundred days
(Portegies Zwart et al. 2004). With the exception of X41.4+60 noted above,
very few objects have been monitored with a cadence and duration that would
enable sensitive searches for X-ray periods in the range of 10s to 100s of
days. The Swift observatory uniquely provides both the X-ray imaging
sensitivity and scheduling flexibility to enable such searches. Starting with
Observing Cycle 4 Swift has been monitoring NGC 5408 X-1 several times per
week as part of an approved program. These observations have continued into
Cycle 5 and are presently ongoing. Here we present results from this campaign
which provide evidence for a 115 day periodicity in NGC 5408 X-1 which could
very likely be the orbital period of a Roche lobe overflow binary containing
an IMBH.

##  2 Data Extraction and Analysis

Monitoring of NGC 5408 X-1 with the Swift XRT began on 2008 April 9. Since
then, the source has been observed 2 - 3 times per week with a typical
exposure of 2 ksec. Aside from an $\approx 80$ day gap beginning on 2008
September 26, the observing cadence has been unbroken. Results presented here
include observations through 2009 August 5.

We began our analysis with the Level 2 cleaned XRT event files. The XRT was
operated in photon counting mode. We extracted good events using an extraction
radius of 25” around the position of NGC 5408 X-1. Most observation sequences
were made up of one to two good time intervals (GTI). In general, we combined
these to create a single count rate estimate for each observation sequence. An
important issue is to account for bad pixels and columns present on the XRT
CCD. The University of Leicester’s XRT Digest webpage (see,
http://www.swift.ac.uk/xrtdigest.shtml#analysis) provides a detailed
discussion of the problem. In order to account for the potential loss of
effective exposure due to bad pixels we constructed exposure maps for each
observation using the tool xrtexpomap. We then integrated over the extraction
region in the exposure map, weighted by the XRT point spread function (Moretti
et al. 2006). This provides an effective exposure for each observation. We
used the corrected exposures in order to estimate source counting rates for
each observation. This procedure resulted in a total of 113 count rate
estimates. The resulting light curve is shown in Figure 1. The mean rate is
0.061 s-1, and excursions from 0.01 to 0.1 s-1 are evident.

We extracted spectra from each observation using the same extraction regions
as for the count rate estimates. Background estimates were extracted from
regions of the same size nearby on the CCD. Previous observations of NGC 5408
X-1 suggest that the spectrum does not change dramatically (Kaaret et al.
2003; Strohmayer et al. 2007; Strohmayer & Mushotzky 2009), so we combined all
observations into a single, average spectrum in order to obtain an average
count rate to flux conversion. We fit the resulting spectrum with the sum of a
power-law and disk blackbody (model diskpn in XSPEC), and fixed the absorbing
column at the value measured previously (Strohmayer et al. 2007). We find a
disk temperature $kT_{max}=0.18\pm 0.02$ keV and power-law index
$\Gamma=2.6\pm 0.2$. These values are reasonably consistent with previous XMM-
Newton measurements. We find that with this spectral form an XRT count rate of
0.06 s-1 corresponds to a 0.3 - 10 keV unabsorbed flux of $4.0\times 10^{-12}$
ergs cm-2 s-1. At a distance of 4.8 Mpc this corresponds to a luminosity of
$1.1\times 10^{40}$ ergs s-1. The implied peak luminosity during these
observations was $\approx 1.9\times 10^{40}$ ergs s-1. We quote fluxes and
luminosities in the 0.3 - 10 keV range for comparison with previous
measurements, however, we note that extending the band pass down to 0.1 keV
increases these luminosity estimates by almost a factor of two.

We carried out a periodogram analysis using the methods of Scargle (1982) and
Horne & Baliunas (1986). We calculated the periodogram at 140 frequency
points, which is the number of independent frequencies estimated by the method
of Horne & Baliunas (1986), and we normalized by the total variance in the
data. The resulting power spectrum is shown in Figure 2. The highest peak in
the spectrum appears at a frequency of $1.0018\times 10^{-7}$ Hz
(corresponding to a period of 115.5 days), and has a value of 24.7. We
estimate the significance of the peak by calculating the chance probability
for obtaining this value from the $\chi^{2}$ distribution with 2 degrees of
freedom, and multiplying by the number of trials (140 frequency bins). This
yields a significance of $6.1\times 10^{-4}$, which is better than a $3\sigma$
detection. There is no evidence for significant power in any other frequency
range. The coherence $Q$, defined as the center frequency divided by the width
of the peak (measured as the FWHM), is 4.7 and is consistent with a periodic
modulation. However, the present data cover only a bit more than 4 cycles, so
further monitoring will be required to better assess the coherence of the
modulation.

We note that Kaaret & Feng (2009) have recently submitted a paper to astro-ph
(0907.5415) which reports results using some of the same data presented here.
We became aware of this work after beginning work on this manuscript. Kaaret &
Feng do not claim to detect a significant periodicity in the NGC 5408 X-1
light curve, but they do state that the highest peak they see is “near a
period of 115 days.” We believe the reason for this seeming discrepancy is
two-fold. First, we have used more data in our analysis, that is, a longer
temporal baseline, and secondly, we have included the exposure map corrections
in our light curve. Indeed, when we restrict our analysis to the time range
used by Kaaret & Feng, and ignore the exposure corrections, then we see a drop
in the peak power at 115 days to a level consistent with Kaaret & Feng. The
fact that the exposure correction increases the Fourier power is further
evidence that the 115 day modulation is a real signal.

To further explore the nature of the modulation we folded the data in several
energy bands at the measured period of 115.5 days. To do this we placed each
light curve measurement in its appropriate phase bin, and then we averaged all
measurements in a given bin. Errors were determined by averaging the
individual errors in quadrature. The resulting profiles in twelve phase bins
are shown in Figure 3. We show profiles in the full band ($E>0.2$ keV, upper
left), two soft bands ($E<1$ keV, upper right; $E<0.7$ keV, lower left), and a
hard band ($E>1$ keV, lower right), and we plot two cycles for clarity. We fit
a model to each profile that includes two Fourier components (the fundamental
and first harmonic), $I=A+B\sin 2\pi(\phi-\phi_{0})+C\sin
4\pi(\phi-\phi_{1})$. Results of these fits are summarized in Table 1. For the
hard band ($E>1$ keV) profile the harmonic term is not statistically required.
The folded profiles reveal interesting energy dependent behavior. For example,
the modulation amplitude appears to clearly decrease with increasing energy.
Defining the fractional amplitude $f_{amp}=(B^{2}+C^{2})^{1/2}/A$, we find a
significant change from $0.133\pm 0.018$ to $0.241\pm 0.017$ in going from
$E>1$ to $E<1$ keV. Indeed, recomputing the periodogram with only $E<1$ keV
photons results in an increase in the peak Fourier power to 27.33. The
detection significance then improves to $3.2\times 10^{-4}$, which includes an
additional factor of 2 for the increased number of trials. In addition to the
amplitude variations the profile appears smoother at higher energies, and the
phase of the fundamental component is significantly different above 1 keV than
below, in the sense that the harder photons appear to lag the soft photons
with a relative phase difference of $0.11\pm 0.025=12.6\pm 2.9$ days. Finally,
we computed the hardness ratio as a function of phase, where we define the
hardness ratio as the count rate for $E>1$ keV divided by the rate for $E<1$
keV. Figure 4 shows the variation of the hardness with phase, and we have also
plotted the full-band folded profile for comparison (dashed curve). The
hardness ratio shows a sharp rise to a maximum near phase 0.45, followed by a
more gradual decline. The peak of the modulation is clearly softer than the
minimum.

##  3 Discussion and Implications

A number of accreting binaries show X-ray modulations at their orbital
periods. Among black hole binaries Cyg X-1 and Cyg X-3 are well known examples
(Wen et al. 1999; Elsner 1980), and more recent detections include LMC X-3
(Boyd et al. 2001), 1E 1740.7-2942 and GRS 1758-258 (Smith et al. 2002). Many
neutron star binaries also show X-ray modulations at the binary orbital
period. The orbital modulations in accreting binaries have typically been
attributed to periodic obscuration produced by a vertically and azimuthally
structured accretion disk or scattering in an extending accretion disk corona
or stellar wind from the donor (Parmar & White 1988; Wen et al. 1999). A
recent example of an X-ray modulation at the putative orbital period in a
neutron star system is that of GX 13+1 (Corbet 2003). Thus, if ULXs are indeed
accreting black hole binaries, then it is not unexpected to detect X-ray
modulations at their orbital periods.

While their have been several claims of detection of orbital modulations in
ULXs (see, Kaaret, Simet & Lang 2006 for a brief summary), the most compelling
detection to date is that of the 62 day modulation from the M82 ULX X41.4+60
(Kaaret, Simet & Lang 2006; Kaaret & Feng 2007). The average modulation in
X41.4+60 as observed with the RXTE/PCA is roughly sinusoidal with an amplitude
of $\approx 20\%$ (Kaaret & Feng 2007). This is qualitatively consistent with
the $0.2<E<8$ keV modulation amplitude and profile that we report here for NGC
5408 X-1, and suggests that similar physical processes may produce the
observed modulation in both systems.

Superorbital periods are known in both neutron star and black hole binaries.
Well known examples include the 35 day modulation in the neutron star system
Her X-1, and the 164 day period in the putative black hole binary SS 433
(Wijers & Pringle 1999). These variations have generally been ascribed to
accretion disk precession (Ogilvie & Dubus 2001). Kaaret & Feng (2007) argued
that the 62 day period in X41.4+60 was unlikely to be a superorbital
modulation because the observed period is only consistent with the observed
superorbital periods of neutron star systems, but that the extreme luminosity
of X41.4+60 comfortably rules out an accreting neutron star (see their Figure
4 for a distribution of observed superorbital periods). Similar arguments
would appear valid for the 115 day modulation in NGC 5408 X-1. The 115 day
period is shorter than all known superorbital periods for black hole candidate
binaries, and its peak luminosity is well in excess of the Eddington limit for
a neutron star ($\approx 10^{38}$ ergs s-1).

Using X-ray timing measurements with XMM-Newton, Strohmayer et al. (2009) have
recently shown that the strong QPO observed in NGC 5408 X-1 varies in
frequency and amplitude with changes in the X-ray flux and energy spectrum in
a manner that closely mimics the correlated temporal and spectral variations
observed in stellar mass black holes in the so-called Intermediate State (also
known as the Steep Power-law State). Moreover, recent optical spectroscopy
reported by Kaaret & Corbel (2009) indicates that its optical counterpart has
a significant contribution from reprocessing of the X-ray luminosity in the
outer parts of the disk. The so-called “slim disks” that may exist at super-
Eddington accretion rates preferentially radiate more flux out along the disk
axis, and therefore have relatively less available for reprocessing
(Abramowicz et al 1988). These models thus have difficulty accounting for the
observed optical to X-ray flux ratio in NGC 5408 X-1. Further, modeling of the
observed nebular optical emission lines supports the conclusion that NGC 5408
X-1 radiates $\sim 10^{40}$ ergs s-1 in an approximately isotropic manner
(Kaaret & Corbel 2009). These observations support the notion that NGC 5408
X-1 harbors a “standard” thin, optically thick accretion disk similar to those
inferred to exist in Galactic black hole binaries.

The 115 day modulation we have found in NGC 5408 X-1 shows interesting energy
dependent effects that would appear consistent with orbital modulation. A
system which shows qualitatively similar effects to those seen in NGC 5408 X-1
is Cyg X-1. Specifically, the orbital modulation in Cyg X-1 has been modeled
in the context of absorption and scattering of X-rays in the partially ionized
wind from the companion star (Wen et al. 1999). This model predicts a decrease
in modulation amplitude with increasing energy, and a minimum in the hardness
ratio at the maximum of the modulation profile (see Wen et al. 1999, Figure
3), both of which are evident in the NGC 5408 X-1 data. While the shapes of
the modulation profiles in Cyg X-1 do not exactly match the results we find
for NGC 5408 X-1, the behavior of the $E>1$ profile (see Figure 3) appears
qualitatively similar. An important distinction would seem to be that in the
case of NGC 5408 X-1 we can directly observe the thermal disk flux whereas the
orbital modulation measurements for Cyg X-1 concern the low-hard state, when
presumably its thermal flux is weak or absent. Moreover, any disk flux appears
largely shortward of the RXTE/PCA energy band. Cyg X-1 is also a wind
accreting system, whereas wind accretion would likely be unable to account for
the high luminosity of NGC 5408 X-1.

If the 115 day period is indeed the orbital period of a Roche lobe filling
binary, then the mean density of the secondary is constrained to be
$\rho_{mean}\approx 0.2(P_{\rm days})^{-2}$ g cm-3 (Frank et al. 2002). For a
period of 115.5 days this gives $\rho_{mean}=1.50\times 10^{-5}$ g cm-3, and
the companion would have to be a giant or supergiant star. Recent observations
suggest that a substantial fraction of the optical counterpart is due to
reprocessing of the X-ray flux in an accretion disk, and while the companion
star still has not been observed directly, the recent optical measurements
suggest that it is probably a giant in the 3 - 5 $M_{\odot}$ range, and with a
spectral type of B or later (Kaaret & Corbel 2009). Binary evolution
calculations for IMBHs and massive companions appear consistent with the
notion that NGC 5408 X-1 contains a $\sim 1000M_{\odot}$ black hole with a 3 -
5 $M_{\odot}$ companion (Li 2004). Continued monitoring of NGC 5408 X-1 with
Swift is important in order to confirm the orbital nature of the modulation.

## References

  * Abramowicz et al. (1988) Abramowicz, M. A., Czerny, B., Lasota, J. P., & Szuszkiewicz, E. 1988, ApJ, 332, 646.
  * Belloni (2006) Belloni, T. 2006, Advances in Space Research, 38, 2801.
  * Boyd et al. (2001) Boyd, P. T., Smale, A. P., & Dolan, J. F. 2001, ApJ, 555, 822.
  * Casella et al. (2005) Casella, P., Belloni, T., & Stella, L. 2005, ApJ, 629, 403.
  * Colbert & Mushotzky (1999) Colbert, E. J. M. & Mushotzky, R. F. 1999, ApJ, 519, 89.
  * Corbet (2003) Corbet, R. H. D. 2003, ApJ, 595, 1086.
  * Elsner et al. (1980) Elsner, R. F., Ghosh, P., Darbro, W., Weisskopf, M. C., Sutherland, P. G., & Grindlay, J. E. 1980, ApJ, 239, 335.
  * Farrell et al. (2009) Farrell, S. A., Webb, N. A., Barret, D., Godet, O., & Rodrigues, J. M. 2009, Nature, 460, 73.
  * Frank et al. (2002) Frank, J., King, A., & Raine, D. J. 2002, Accretion Power in Astrophysics, by Juhan Frank and Andrew King and Derek Raine, pp. 398. ISBN 0521620538. Cambridge, UK: Cambridge University Press, February 2002.
  * Horne & Baliunas (1986) Horne, J. H., & Baliunas, S. L. 1986, ApJ, 302, 757.
  * Kaaret et al. (2009) Kaaret, P., Feng, H., & Gorski, M. 2009, ApJ, 692, 653.
  * Kaaret & Corbel (2009) Kaaret, P., & Corbel, S. 2009, ApJ, 697, 950.
  * Kaaret & Feng (2009) Kaaret, P., & Feng, H. 2009, ApJ, 702, 1679.
  * Kaaret & Feng (2007) Kaaret, P., & Feng, H. 2007, ApJ, 669, 106.
  * Kaaret et al. (2006) Kaaret, P., Simet, M. G., & Lang, C. C. 2006, ApJ, 646, 174.
  * Kaaret et al. (2003) Kaaret, P., Corbel, S., Prestwich, A. H., & Zezas, A. 2003, Science, 299, 365.
  * King et al. (2001) King, A. R., Davies, M. B., Ward, M. J., Fabbiano, G. & Elvis, M. 2001, ApJ, 552, L109.
  * Lang et al. (2007) Lang, C. C., Kaaret, P., Corbel, S., & Mercer, A. 2007, ApJ, 666, 79.
  * Li (2004) Li, X.-D. 2004, ApJ, 616, L119.
  * McClintock & Remillard (2006) McClintock, J. E., & Remillard, R. A. 2006, Compact stellar X-ray sources, 157.
  * Moretti et al. (2006) Moretti, A., et al. 2006, Gamma-Ray Bursts in the Swift Era, 836, 676.
  * Ogilvie & Dubus (2001) Ogilvie, G. I., & Dubus, G. 2001, MNRAS, 320, 485.
  * Parmar & White (1988) Parmar, A. N., & White, N. E. 1988, Memorie della Societa Astronomica Italiana, 59, 147.
  * Portegies Zwart et al. (2004) Portegies Zwart, S. F., Dewi, J., & Maccarone, T. 2004, MNRAS, 355, 413.
  * Scargle (1982) Scargle, J. D. 1982, ApJ, 263, 835.
  * Shaposhnikov & Titarchuk (2009) Shaposhnikov, N., & Titarchuk, L. 2009, ApJ, 699, 453.
  * Smith et al. (2002) Smith, D. M., Heindl, W. A., & Swank, J. H. 2002, ApJ, 578, L129.
  * Sobczak et al. (2000) Sobczak, G. J., McClintock, J. E., Remillard, R. A., Cui, W., Levine, A. M., Morgan, E. H., Orosz, J. A., & Bailyn, C. D. 2000, ApJ, 531, 537.
  * Soria et al. (2006) Soria, R., Fender, R. P., Hannikainen, D. C., Read, A. M., & Stevens, I. R. 2006, MNRAS, 368, 1527.
  * Strohmayer & Mushotzky (2009) Strohmayer, T. E. & Mushotzky, R. F., ApJ, in press.
  * Strohmayer et al. (2007) Strohmayer, T. E., Mushotzky, R. F., Winter, L., Soria, R., Uttley, P., & Cropper, M. 2007, ApJ, 660, 580.
  * Strohmayer & Mushotzky (2003) Strohmayer, T. E., & Mushotzky, R. F. 2003, ApJ, 586, L61.
  * Vignarca et al. (2003) Vignarca, F., Migliari, S., Belloni, T., Psaltis, D., & van der Klis, M. 2003, A&A, 397, 729.
  * Wen et al. (1999) Wen, L., Cui, W., Levine, A. M., & Bradt, H. V. 1999, ApJ, 525, 968.
  * Wijers & Pringle (1999) Wijers, R. A. M. J., & Pringle, J. E. 1999, MNRAS, 308, 207.

Figure 1: X-ray lightcurve of NGC 5408 X-1 in the 0.2 - 8 keV band derived
from Swift/XRT measurements. A count rate of 0.06 s-1 corresponds to a 0.3 -
10 keV unabsorbed flux of $4\times 10^{-12}$ ergs cm-2 s-1. The mean count
rate of 0.061 s-1 is marked by the dashed horizontal line. Time zero
corresponds to MJD 54565.85847 (UTC).

Figure 2: Lomb-Scargle periodogram computed from the Swfit/XRT light curve of
NGC 5408 X-1. There are 140 independent frequency bins plotted with a
resolution of 9.54 nanoHz. The Nyquist frequency is 1.35 $\mu$Hz
(corresponding to a period of 8.6 days). The horizontal dashed line denotes
the $3\sigma$ detection level. The peak value of 24.7 occurs at a frequency of
$1.0018\times 10^{-7}$ Hz (a period of 115.5 days).

Figure 3: X-ray flux from NGC 5408 X-1 in different energy bands folded at a
period of 115.5 days. Profiles were computed with 12 phase bins, and two
cycles are shown for clarity. Proceeding clock-wise from the upper left panel
the energy bands are, $0.2<E<8$ keV; $0.2<E<1$ keV; $1<E<8$ keV; and
$0.2<E<0.7$ keV. The best fitting model with two Fourier components is also
plotted in each panel. Note the change in profile amplitude and shape with
energy. See Table 1 for a summary of the model fits, and the text for
additional discussion.

Figure 4: Hardness ratio as a function of phase of the 115.5 day modulation in
NGC 5408 X-1. The hardness ratio is defined as the ratio of count rates for
photons with energy $>1$ keV to that for photons with energy $<1$ keV. We also
plot the full-band (0.2 - 8 keV) folded profile for comparison (dashed curve).

Table 1: Results of Pulse Profile Modeling for NGC 5408 X-111Summary of fits to energy dependent pulse profiles for NGC 5408 X-1. The fitted model is $I=A+B\sin[2\pi(\phi-\phi_{0})]+C\sin[4\pi(\phi-\phi_{1})]$.  Profile | A ($10^{-2}$) | B ($10^{-2}$) | $\phi_{0}$ | C ($10^{-2}$) | $\phi_{1}$ |  $f_{amp}$aaFractional modulation amplitude defined as $(B^{2}+C^{2})^{1/2}/A$.
---|---|---|---|---|---|---
$E>0.2$ keV | $6.23\pm 0.05$ | $1.10\pm 0.08$ | $0.915\pm 0.011$ | $0.27\pm 0.07$ | $0.46\pm 0.02$ | $0.182\pm 0.013$
$E<0.7$ keV | $2.09\pm 0.03$ | $0.48\pm 0.05$ | $0.874\pm 0.014$ | $0.14\pm 0.04$ | $0.473\pm 0.024$ | $0.238\pm 0.022$
$E<1$ keV | $3.64\pm 0.04$ | $0.83\pm 0.06$ | $0.882\pm 0.011$ | $0.27\pm 0.06$ | $0.485\pm 0.016$ | $0.241\pm 0.017$
$E>1$ keV | $2.64\pm 0.03$ | $0.34\pm 0.05$ | $0.992\pm 0.023$ | $0.096\pm 0.05$ | $0.853\pm 0.040$ | $0.133\pm 0.018$


# The Right Tool for the Job: Open-Source Auditing Tools in Machine Learning

Cherie M Poland  Complex Adaptive Systems Research
Spring, Texas, USA
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
ORCID:0000-0002-6345-649X

###### Abstract

In recent years, discussions about fairness in machine learning, AI ethics and
algorithm audits have increased. Many entities have developed framework
guidance to establish a baseline rubric for fairness and accountability.
However, in spite of increased discussions and multiple frameworks, algorithm
and data auditing still remain difficult to execute in practice. Many open-
source auditing tools are available, but users aren’t always aware of the
tools, what they are useful for, or how to access them. Model auditing and
evaluation are not frequently emphasized skills in machine learning. There are
also legal reasons for the proactive adoption of these tools that extend
beyond the desire for greater fairness in machine learning. There are positive
social issues of public perception and goodwill that matter in our highly
connected global society. Greater awareness of these tools and the reasons for
actively utilizing them may be helpful to the entire continuum of programmers,
data scientists, engineers, researchers, users and consumers of AI and machine
learning products. It is important for everyone to better understand the input
and output differentials, how they are occurring, and what can be done to
promote FATE (fairness, accountability, transparency, and ethics) in machine-
and deep learning. The ability to freely access open-source auditing tools
removes barriers to fairness assessment at the most basic levels of machine
learning. This paper aims to reinforce the urgent need to actually use these
tools and provides motivations for doing so. The exemplary tools highlighted
herein are open-source with software or code-base repositories available that
can be used immediately by anyone worldwide.

###### Index Terms:

algorithm analysis, problem complexity, auditing, fairness, accountability,
transparency, ethics, tools, open-source, machine learning, law.

##  I Introduction

As a society, we are tossed around on a sea of competing ideas, backed by
statistics, data, and algorithms. The decisions made by algorithms rely on the
data used as input and the manner in which the algorithms are configured and
programmed by humans. Ultimately, however, all decisions are human decisions,
regardless of the manner or means by which the data are filtered, manipulated
and wielded. Humans select the filters, manipulate the data, and wield the
algorithms. That means that humans can also ultimately be held responsible for
the outcome of those decisions, whether directly made by a human brain or by
extension, an algorithmic matrix. It is widely known that algorithms can
encompass, express, and unveil intentional or unconscious bias, including bias
intrinsically present in the data, and manipulation by competing interests or
those with agendas. The critical need for fairness, accountability,
transparency, and ethics (FATE) in machine learning has increased dramatically
in recent years as the output of systems has shown the disparities in data and
algorithms in every field of endeavor. This increase in awareness has also
triggered legal concerns that have evolved into rubrics for ethical use of
algorithmic tools, but real-world legal concerns are being raised and
governmental regulators around the world are being educated on remedies that
may be needed if the industry does not better police itself.

Audits are evaluations with expectations of accountability [1]. Algorithmic
auditing is not a one-size-fits all solution. Different methods of auditing
are required in order to search for and test both the data and the algorithms
for underlying or hidden bias and attributes. Methods encompassing anomaly
detection and pattern recognition are required. No single auditing method will
likely uncover all potential issues. The breadth and scope of auditing methods
necessitates creating an algorithmic auditing toolkit that one may use to
deeply examine data and algorithms with various degrees of granularity.

##  II Why Auditing is Important

Fairness is an important concern in terms of trust and reliability of the
decision-making process involving machine learning models. Each model has
specific requirements for the type of data it will operate over. These
necessitate different pre-processing techniques. However, in order to provide
a consistent comparison across models, each model must receive similar input.
Data are manipulated by pre-processing it to generate similar types of inputs
that models can handle. However, this data manipulation can also create bias
in datasets as it is a deliberate method of focusing the analytical lens on a
particular set of data. Data may also be biased because it is a collection of
human-generated information. There is an evidence-based assumption that bias
is bad. There is a societal need to be accountable, transparent, and fair,
especially because the outputs of algorithms are not always able to be
understood or interpreted by humans. Algorithmic output and decisions are also
susceptible to inaccuracies, may be adversarial to human rights or other
fundamental rights, and may present legal challenges such as data privacy and
unauthorized use or sale of data by third-parties. Auditing algorithms may not
resolve all issues of concern, but audits are part of the solution in
providing more ethical accountability for very powerful machine learning
software.

###  II-A Model Standards

The implementation of reproducible and standardized techniques has led
industry to establish guidelines for fairness, accountability, and
transparency in machine learning models [2]. However, the implementation of
these guidelines requires tools. The main objective of these tools is to
provide a mechanism by which bias and other forms of concerning algorithmic
output can be detected, measured, and reduced.

Various auditing tools and open-source libraries have been developed to
provide a high level of functionality in machine learning and deep learning
models.

Different models of analysis are used to extract different features from data.
These features, their patterns, and anomalies are then analyzed to determine
whether there are irregularities in the underlying data or in the algorithm
used that need to be explored further. The expectation of finding bias of some
type in any given generic dataset should be neutral. While none is overtly
expected, history shows that humans have both overt and unconscious bias and
that bias may be reflected in unexpected places.

Pervasive use of high performing, ’black-box’ models in sensitive areas like
healthcare[3] and criminal justice systems raises ethical concerns and a
critical need to audit these models for fairness. Recent works[4] and [5] have
revealed disturbing bias in some machine learning models against minority
groups.

One way to audit these algorithms is to look inside the inner-workings of the
algorithm, model, and datasets to understand how the underlying algorithms,
data, data pre-processing, input features, feature engineering, tuning, and
optimization are combined to arrive at the model output. However, due to
proprietary reasons, the inner workings of machine learning models may not be
publicly available. One possible solution to this dilemma is that post-hoc
interpretability techniques can serve as useful auditing tools.

###  II-B The Law

There are also growing legal reasons to be concerned about fairness in machine
learning. Legal standards differ in different countries and what may be
considered fair on one country’s legal system may not be fair in another. Much
of the law related to computer science is related to privacy and access to
data and systems. In the United States employment laws related to disparate
impact protect the public from the harm posed by biased algorithms, in some
cases, but only once that disparate impact is discovered and challenged. There
are, however, important legal distinctions. Disparate impact, by itself, is
not unlawful. Disparate treatment may be unlawful in certain situations,
particularly when individuals with a legally recognized protected status are
harmed. However, even in this circumstance, the burden of proof is very high,
is is very fact-specific, and will be limited to the specific context in which
it arises. There are anti-discrimination laws that apply to access to public
services, housing, lending, and the like, but US law is a negative limitation
on government, not the private sector.

In other areas of the world, the laws are very different. The laws of the
European Union tend to be more stringent than laws in the United States. The
General Data Protection Regulation (GDPR) protects privacy rights as well as
rights related to disparate treatment and discrimination. In Asia, some laws
related to machine learning systems are more lax than those of the United
States. The laws in many South America and African countries are limited
insofar as machine learning is concerned. Many countries have data protection
laws, but their degree of protections vary.

The lack of regulation at the present time is an opportunity for the field to
place a greater emphasis on self-policing. The more upset the global public
becomes with algorithms that show unfairness, bias, disparate impact,
disparate treatment, and differential classification that discriminates
against one group over another, the more attention will be drawn to the field
and the calls for governmental regulation will increase.

These laws and regulations may be sector-specific, such as requiring computer
vision algorithms to comply with safety standards of other vehicles driven on
public roadways. Or they may be broader in scope, such as algorithms related
to credit scoring and the manner in which they affect lending interest rates
as a surrogate of access to or denial of financial or real estate services.

Fairness in machine learning is generally related to fairness of decision
outcomes (distributive fairness), under the law. This is why the area is often
examined under the disparate impact and disparate treatment legal rubrics.
However, fairness in machine learning is also related to procedural fairness,
especially in settings where there may be a deprivation of a legally-
guaranteed right or privilege, such as a criminal, employment, or financially-
based setting. [6] This also applies to computer vision algorithms related to
biometrics, many of which simply apply Euclidean distance metrics and
classifiers. The manner in which facial recognition software operates is
important because a 1:1 identification has far less risk of discrimination or
misidentification that a 1:N or 1:many analysis where an unknown face is
compared to a large database of known faces to determine the unknown person’s
identity.

In a private setting, legal concerns about fairness in machine learning are
tied directly to concerns about risk-management and public perception. A
corporation can be sued or can easily lose brand status because of using
machine learning algorithms that are publicly considered to be unfair, even if
that unfairness does not rise to the level of unlawfulness in a civil context
or illegality in a criminal context.

Educating the public by publishing algorithmic auditing assessments and proof
of the use of auditing tools will help mitigate some of the concerns about
fairness in machine learning algorithms. Taking responsibility for the outcome
of the algorithm and its use will promote public good will.

A worst-case scenario is for industry to do nothing until a legislative body
decides to regulate machine learning technologies in a strict manner similar
to other product liability laws. This is what is slowly occurring in the
autonomous driver space. When vehicle manufacturers cannot control damage to
property and lives due to faults in their algorithms, the entities supplying
the software, the executives, and engineers responsible may be held legally
accountable.

##  III Exemplary Areas of Concern

At the 2021 Neural Information Processing Systems (NeurIPS) conference, the
organizing committee instituted an Ethics Review process for papers submitted
to the conference. According to the conference Retrospective publication, 9122
total papers were submitted and reviewed and 265 were flagged for additional
ethics review in one or more areas of concern.

Areas of concern included: Discrimination/Bias/Fairness Concerns, Inadequate
Data and Algorithm Evaluation, Inappropriate Potential Applications and Impact
(e.g. human rights concerns), Legal Compliance (e.g. GDPR, copyright, terms of
use), Privacy and Security (e.g. consent), Responsible Research Practice (e.g.
IRB, documentation, research ethics), and research integrity issues (e.g.
plagiarism) [7].

The ACL, CVPR, ICLR, and EMNLP conferences have also taken steps to establish
ethical expectations for the research community including introducing ethics
review processes [7]. Including fairness and ethics assessments in the
underlying work and research methodologies and reporting the same in the
papers, reviews, and conference presentations will boost confidence from
industry, academia, government, business, the public, and end-users who have
concerns about AI and machine learning fairness and safety.

In January 2022, the United States Internal Revenue Service (IRS) announced
that it would soon require facial recognition technology in order to access
its systems [8] and [9]. The IRS contracted with a third-party company, ID.me,
to provide the service. The company also provides services to the US
Department of Veterans Affairs and the US Social Security Administration.
However, the agency reversed its decision after large public outcries over the
practice [10].

Even more chilling data privacy concerns were uncovered following the public
outcry that showed the IRS had consented to allowing ID.me to “disclose or
share Non-Personally Identifiable Information with Third-Party Service
Providers and Affiliates” [11]. This implies that the company was or intended
to be a data broker and many data privacy advocates have become rightly
concerned.

Initially, ID.me initially stated that it was only going to be using 1:1
facial recognition, but later clarified that the company also uses 1:N
(1:many) technology, prompting privacy advocates to raise public concerns.
[12]

The CEO founder of ID.me stated that the 1:many is only used during initial
enrollment and that afterwards, 1:1 is used [13]. However, a New Jersey man
was indicted by the US Department of Justice in January 2022 for deliberately
circumventing the security measures of ID.me in order to further a scheme to
defraud the California Employment Development Department (EDD), filing at
least 78 fraudulent unemployment insurance claims with EED seeking Pandemic
Unemployment Assistance and other benefits under the federal CARES Act. EDD
has partnered with ID.me since at least October 2021 [13].

How much better off would ID.me be if they could show that they or a neutral
third-party entity has conducted an algorithm audit of their systems and
processes and be able to provide proof of the same? In situations where the
algorithmic model and data are proprietary, the use of something like zero-
knowledge proofs may be a way to affirmatively show the results of an
algorithmic and data audit without actually opening the proprietary system for
analysis to the public or in a public forum.

As issues of fairness in machine learning and AI ethics continue to expand, it
becomes imperative for the community to be aware of the potential risk and
legal implication and the need to be familiar with open-source and free
auditing tools that can be used to insure integrity of data and algorithms
used.

##  IV Techniques

In recent years several machine learning interpretability techniques have
emerged that can be used to explain the machine learning model results in
terms of individual input feature contributions. Current popular
interpretability techniques can be categorized into post-hoc interpretability
and intrinsic interpretability [14]. Post-hoc methods provide explanations of
the model output after the model is trained. Some of the current state-of-art
methods include feature attributions (LIME, SHAP), model and data analysis
tools (What-If Tool, Fairness Indicators), gradient and concept based testing
(TCAV, GRAD-CAM) and data point inspection methods (partial dependence plots,
ablation testing). Intrinsic interpretability is achieved by using inherently
interpretable machine learning models.

In many instances, deep learning models have been shown provide a higher
degree of accuracy than the traditional machine learning models when the input
consists of large, unstructured data. Similarly, sophisticated ensemble
techniques like bagging and boosting are preferred to simpler, more
interpretable decision trees as they achieve better performance with
structured tabular data. However, there exists a trade-off between accuracy
and interpretability. The complex architecture of these models make it hard
for the end-user to interpret how the input features have contributed to the
model output prediction.

Post-hoc interpretability techniques serve as viable model auditing tools as
they enable us to understand individual feature contributions to the model
output.

In certain scenarios the explanations provided by the post-hoc methods are
unreliable. Using inherently interpretable models can overcome this drawback.

##  V Tools

Data and algorithms come in a variety of flavors. This is also true of
auditing tools. It is important to know and understand which tool is
appropriate to use at each phase of the machine learning continuum. The main
objective with these tools is to detect, measure, and reduce unwanted bias.
Some of these tools also incorporate simulations and data visualization models
as graphical representations that can be used to discern differences in
datasets in order to help identify patterns or anomalies. The open-source
framework facilitates a direct comparison of algorithms.

A sampling of a wide variety of tools and toolkits with links to papers,
models, tutorials, instructions for use, and code repositories are as follows.

###  V-A AI Fairness360

The AI Fairness 360 toolkit (AIF360) is a comprehensive Python open-source
package developed at IBM that contains over 70 fairness metrics and 10 bias
mitigation algorithms. [15] It is designed to translate algorithmic research
from the lab into the actual practice of domains as wide-ranging as finance,
human capital management, healthcare, and education. [16] In some cases, AI360
toolkit requires datasets to be transformed into Binary-Label-Dataset. The
protected attributes can the be selected for observational analysis. These
attributes partition a data population into groups that have a difference in
terms of parity received.

Prior to applying fairness metrics to the selected attributes, standard
statistical analysis can also be applied to determine the target attribute.

Code and instructions can be found at [17], [18] and [19].

###  V-B LIME

LIME (Local Interpretable Model-Agnostic Explanations) [20] is a popular
perturbation-based interpretability technique that provides local explanations
to black-box models. LIME trains a linear classifier that locally approximates
the behavior of the black-box machine learning model. A perturbed dataset is
created by feeding in variations of a single data point to the ML model and
noting the corresponding outputs. LIME then trains an interpretable model (for
example, a decision tree) on this new dataset of permuted samples, where the
samples are weighted by their proximity to the sample of interest. The local
interpretable model is then used to explain the predictions made by the
machine learning model to be audited [21].

The LIME results and explanations can serve as a helpful model auditing tool
when the inner workings of the machine learning model are not revealed. The
user can feed in data samples permuted around a sample of interest to the
black-box model and store the outputs as a new dataset to train a more
interpretable model. However, LIME suffers from a drawback of unclear
coverage. The explanations provided by LIME are local in nature and the
boundary to which these explanations apply is not well defined.

Code and instructions can be found at [22].

###  V-C ANCHORS

ANCHORS (High-Precision Model-Agnostic Explanations) is a model-agnostic
system proposed by the authors of LIME that generates a set of ’if-then’ rules
as explanations for model output predictions and overcomes the drawbacks of
LIME by providing a clear coverage of the data instances to which the
explanations apply [23].

Code and instructions can be found at [24], [25] and [26].

###  V-D What-If Tool

The ”What If Tool” (WIT) is open-source auditing tool that provides an
interactive visualization platform that allows users to probe, visualize, and
analyze their data against different classifiers [27]. Visualization of
hypothetical scenarios can be made against different subsets of inputs and
analysis can be assessed related to the importance of different data features
over different models.

WIT converts datasets to a numerical format with a graphical display. Each
data point is displayed in the graph are editable and can be set to different
values to observe counterfactual points in the data. What if tool can be
useful in identify bias using hypothetical situations against different
features using linear classifier and neural network classifier. Models of
classification can be assessed against any input features. A ground truth
feature can also assess binary categories.

WIT also has a visualization tool called Facets Overview [28] that uses
subsets of the data to compare fairness metrics and assess overall model
performance.

WIT with SHAP (SHapley Additive exPlanations)[29], as a technique to audit a
black-box machine learning model used in the healthcare domain [30].

Code and instructions can be found at [31] and [32].

###  V-E FairSearch

FairSearch is the fair open-source search API to apply fairness in ranked
search results [33]. Its ranked group fairness definition extends group
fairness using the standard notion of protected groups, based on ensuring that
the proportion of protected candidates in every prefix of the top-k ranking
remains statistically above or indistinguishable from a given minimum [34].

Code and instructions can be found at [35] and [36].

###  V-F FairTest

The FairTest tool helps to identify unwanted associations, unfair,
discriminatory, or offensive user treatment in data-driven applications. It
examines and biases between potential outcomes and sensitive attributes [37].
It has been used to address disparate impact, offensive labeling, and uneven
rates of algorithmic error in a variety of datasets.

Code and instructions can be found at [38], [39] and [40].

###  V-G Aequitas

Aequitas is an open-source Python library with many fairness metrics to detect
bias [41]. To use Aequitas one must upload their data to the tool. Some may
not be comfortable in using their data on other sites. If there are data
leakage concerns, you should use another tool. Aequitas uses a Python library
and a command line interface. Users upload their data from the system being
audited, configure bias metrics for protected attribute groups of interest as
well as reference groups, and then the tool generates bias reports. Bias
assessments can be made prior to model selection, evaluating the disparities
of the various models trained based on whatever training data was used to tune
it for its task. The audits can be performed prior to a model being
operationalized or it can be used to determine bias in an A/B testing
environment using limited trials and revising the algorithm accordingly.

Code and instructions can be found at [42], [43] and [44].

###  V-H Themis-ML

Themis-ML is a tool that makes model agnostic predictions using reject object
recognition (ROC) and discrimination aware ensemble classification (DAEC). The
concept is that it that looks for bias by altering the training data by
reweighting and relabeling the classes using sampling. Themis-ML will modify
your dataset, so be sure to save your original model and make one or more
copies to run on Themis-ML. [45]

Code and instructions can be found at [46] and [47].

###  V-I Fairness Test-Bed

The ”Fairness-Aware” Machine Learning interface, Fairness Test-Bed, is an
open-source framework that facilitates a direct comparison of multiple
algorithms [48] and [49]. The framework for the tool focuses on factors like
pre-processing if multiple algorithms are going to be compared to each other.
The Fairness Test-Bed paper shows that that a combination of class-sensitive
error rates and either DI or CV is a good minimal working set. To account for
training instability, it also shows that the performance of an algorithm in a
single training-test split appears to be insufficient. Accordingly, they
recommend reporting algorithm success and stability based on a moderate number
of randomized training-test splits. They use the standard accuracy measures of
uniform accuracy, the true positive rate (positive predictive value) and the
true negative rate (negative predictive value). They also consider the
balanced classification rate, which is a version of accuracy that is
unweighted per class.

Code and instructions can be found at [50] and [51].

###  V-J ML-Fairness Gym

ML-fairness gym is dynamic tool implemented in the environment API from OpenAI
Gym that builds simple simulations that explores potential long-run impacts of
deploying machine learning-based decision systems in social environments. [52]
and [53]. This tool is more unique than some of the other tools because the
decisions made by an algorithm at step t affect the next decision it will be
asked to make at t+1. The simulations allow exploration of censoring in the
observation mechanism, errors from the learning algorithm, and interactions
between decision policy and environment dynamics. Assessments can be made on
fairness characteristics of certain decision policies based on observed data
alone. This can assist with future data collection practices. Simulations can
also be used along with reinforcement learning algorithms. Although ML-
fairness-gym is not an officially supported Google product, its environments
implement the environment API from OpenAI Gym.

Code and instructions can be found at [54] and [55].

###  V-K RETAIN

RETAIN uses a Reverse Time Attention Mechanism to build a clinically
interpretable ML model using the EHR (Electronic Health Record) data [56]. The
algorithm uses neural attention so that, given patient records, it can make
predictions while explaining how each medical code (diagnosis codes,
medication codes, or procedure codes) at each visit contributes to the
prediction. The algorithm also treats sequence classification as a special
case, which it makes at the last timestep only. There is a potential trade-off
in that the algorithm learns to pay more attention to new information than old
information. This should be analyzed on a case-by-case basis.

Code and instructions can be found at [57].

###  V-L Knowledge Graphs with Gephi

A knowledge graph comprises a graph data model ontology of attributes and
other data that may provide information about relatedness. Knowledge graphs
model topical coherence of information based on an assumption that information
from the same context tends to belong together [58]. Data are often contextual
and knowledge graphs utilize a data interchange standard comprising tuples
where each tuple can be considered a small graph [58]. It may also comprise
any other associated attributes linked to the data. The primary purpose is
classification, clustering, and relatedness that is able to be presented in
visual form. The knowledge graph maps heterogeneous types of knowledge. The
combined approaches should result in a decrease in disambiguation errors and
result in a more discerning clustering of content and inference from the
highly heterogeneous data [59]. Gephi is a visualization and exploration
software platform for all kinds of graphs and networks. It is open-source and
free [60].

Code and instructions can be found at [61].

###  V-M Adversarial Learning with Networkx

Data may also be run on a CNN and LSTM neural network framework as a
generative adversarial network (GAN) model in order to detect anomalous
patterns, such as those as previously described by Zenati et al [62]. The
results can be visualized using T-SNE as one means of assessment.

Data preparation in graph neural network allows for the representation of
complex data in non-Euclidean space for complex relationships and inter-
dependencies. Deep learning has been extended to GNNs extract latent
representations from images. Data can be converted and manipulated as .ndjson
data files, followed by applying a series of 1-dimensional convolutions,
followed by LSTM layers, with the sum of the outputs of all LSTM steps fed
into an argmax or softmax layer to make a classification decision using known
classes as a starting point.

Networkx [63] is a useful tool with capabilities in graphing, ability to
encode attributes to nodes and edges, loading and saving graphs, and ability
to convert to other graph formats such DGL [64]. The remaining steps can be
run as a progression of models to evaluate the impact of applying additional
modeling methods on classification accuracy and similarity.

Code and instructions for GAN-based anomaly detection can be found at [65],
[66] and [67].

###  V-N IMV-LSTM

IMV-LSTM uses a mixture attention mechanism in the hidden layers of recurrent
neural networks (RNNs) to capture the input variable importance in a multi-
variate time series set up [68]. Interpretable insights into the data can be
found by utilizing RNNs trained on time series with target and exogenous
variables, in addition to accurate prediction. The goal of this process is to
capture different dynamics in multi-variable time series and distinguish the
contribution of variables to the prediction [69].

Code and instructions can be found at [70].

###  V-O Data Triangulation

Data triangulation is an older standard statistical analysis used more
frequently in the social sciences, but it also has applicability in machine
learning models, as it is used to compare multiplicative groupings of data.
Instead of utilizing binary pairings, binary pairings may be made, and then
compared to other paired metrics. For example, age (as integers or
categorically-transformed by decades) can be compared with binary sex or
categorically-transformed race enumeration. All of these factors can then be
compared to other features in the dataset and cross-correlated and compared.
Data triangulation also works well on qualitative data because of the ability
to categorically-transform the data and cross-compare the same categorical
data over different features or cross-compare it over different categories.

Machine learning and deep learning models of artificial intelligence can
incorporate data triangulation in neural networks in the form of mini-batches
of additional data to compare signal to noise ratios, even in noisy datasets
[71]. The theory being that the signals will show up consistently across the
multiplier iterations of datasets, even if some of the data are incomplete,
missing, or excessively noisy. Data triangulation compares qualitative and
quantitative data and can be used across all data types and from entirely
different sources [71]. Triangulation is a correlation process that attempts
to define relational metrics between data while accounting for noise,
overlaps, outliers, and density [71]. Neural networks can also be triangulated
concurrently with other neural networks in order to refine classification,
reduce output error, and provide a better explanation of output results
(explainable AI) [72] and [73].

##  VI Mitigation

What happens after bias is found? There are choices to be made that can impact
data cleaning (inclusion, exclusion, averaging of missing data or parts
thereof), data weighting (reweighting), feature engineering (shift, exclude,
reweight), and optimization that may disregard tails (include or exclude). No
model is an accurate representation of the real-world. It is merely a
representation of an approximation. As such, its degree of accuracy is
variable because of inputs and necessary manipulation of data in order to
compare apples-to-apples

Balanced or representative data sets are commendable, but full-disclosure if
whether the representation is synthetic or real-world is necessary. Balanced
datasets are not enough. Resampling the data can provide a way to make
assessments on representation bias and mitigate against it by retraining from
the resampled dataset.

Data are what they are. The real-world is messy. Real-world data are messy.
Providing commentary and analysis about bias noted in the underlying data or
in the models is very important, especially where the bias can’t be mitigated
without over-correcting.

###  VI-A Caution Against Over-Correcting

Just as over-fitting and under-fitting are not optimal for analysis, explicit
removal of biases and variations are not necessarily appropriate. This type of
action is tantamount to introducing new subjective bias (over-correcting) into
the data or model, directly impacting outcome. It could lead to a deliberately
false model, which would be inherently unethical. Similarly, caution is
warranted when considering introducing additional variables to offset any
biasing variables. Instead, it is recommended to provide an explanation
detailing the FATE assessment so that others may understand the limitations of
the data or the model. It is better to convey the understanding that the field
uses representative approximations and does not provide absolutes. There is
much to be gained through a better understanding of data and models that may
show bias that the data, alone, does not indicate. Similarly, all machine
learning models are based on linear algebra and differential equations, as
well as numerical rather than categorical values. An assessment of floats is
not the same as a contextual assessment of categorical data. However, it is
the best approximation that may be available at any given time.

##  VII Conclusion and Future Work

Many people have asked where to find papers and code for auditing tools. This
review is a response to those requests. The goal of this work is to share the
information as broadly and as quickly as possible, in light of the ever-
changing, challenging environment and the desire of many to want to take
proactive measures in their own work for the betterment of the field.

I have endeavored to provide multiple sources of information for each tool
including Github links, links to video tutorials, and even PowerPoint slide
presentations when the authors made them publicly available.

The tools addressed herein have been utilized on many different datasets by
multiple groups all over the world. I have personally used several of them and
have found to be reliable in finding bias among real-world and publicly
available datasets across multiple domains [74]. Many of these tools are also
model-agnostic.

Many talented researchers and programmers have taken their time to create
open-source tools for the greater good. Lets own our ethical algorithmic
responsibilities to each other. Use the tools and pay it forward.

My personal thanks to my AI ethics colleagues and to all of the hardworking
researchers who continually strive to promote fundamental fairness across
algorithmic domains.

Future work consists of continuing to seek and promote algorithmic fairness,
audits, privacy, and human rights in the field of machine learning.

## References

  * [1] Deb Raji. https://foundation.mozilla.org/en/blog/its-time-to-develop-the-tools-we-need-to-hold-algorithms-accountable/ (Last Accessed 18 June 2022).
  * [2] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, A. Galstya, A Survey on Bias and Fairness in Machine Learning, arXiv 1908.09635, 2019.
  * [3] MA Gianfrancesco, S Tamang, J Yazdany, G Schmajuk, ”Potential Biases in Machine Learning Algorithms Using Elecronic Health Record Data.” November 2018, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6347576 (Last Accessed 5 February 2022).
  * [4] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, A. Galstyan, ”A Survey on Bias and Fairness in Machine Learning,” August 2019, https://arxiv.org/abs/1908.09635 (Last Accessed 5 February 2022).
  * [5] Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan, ”Dissecting racial bias in an algorithm used to manage the health of populations” Science, vol 366, October 2019, pp. 447-453, 10.1126/science.aax2342, https://pubmed.ncbi.nlm.nih.gov/31649194/ (Last Accessed 5 February 2022).
  * [6] Grgić-Hlača, N., Zafar, M.B., Gummadi, K.P. and Weller, A. 2018. Beyond Distributive Fairness in Algorithmic Decision Making: Feature Selection for Procedurally Fair Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 32, 1 (Apr. 2018). https://ojs.aaai.org/index.php/AAAI/article/view/11296 (Last Accessed 18 June 2022).
  * [7] NeurIPS Review - A Retrospective on the NeurIPS 2021 Ethics Review Process. https://blog.neurips.cc/2021/12/03/a-retrospective-on-the-neurips-2021-ethics-review-process/ (Last accessed 3 February 2022).
  * [8] https://gizmodo.com/irs-will-require-facial-recognition-scans-to-access-you-1848387715 (Last Accessed 3 February 2022).
  * [9] https://epic.org/irs-to-require-facial-recognition-to-verify-identity-online/ (Last Accessed 3 February 2022).
  * [10] ttps://www.ntu.org/foundation/detail/irs-drops-questionable-idme-facial-scanner-program (Last Accessed 18 June 2022).
  * [11] https://www.businessinsider.com/idme-data-privacy-handling-senators-markey-menendez-wyden-concerns-2022-6 (Last Accessed 18 June 2022).
  * [12] https://www.cyberscoop.com/id-me-ceo-backtracks-on-claims-company-doesnt-use-powerful-facial-recognition-tech/ (Last Accessed 3 February 2022).
  * [13] https://www.justice.gov/usao-edca/pr/new-jersey-man-indicted-fraud-scheme-steal-california-unemployment-insurance-benefits (Last Accessed 3 February 2022).
  * [14] Christoph Molnar, ”Interpretable Machine Learning. A Guide for Making Black Box Models Explainable, 2020, https://christophm.github.io/interpretable-ml-book/ (Last Accessed 5 February 2022).
  * [15] R.K.E. Bellamy, K. Dey, M. Hind, S.C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, A. Mojsilovic, S. Nagar, K.N. Ramamurthy, J. Richards, D. Saha, P. Sattigeri, M. Singh, K.R. Varshney, and Y. Zhang, ”AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias,” arXiv 1810.01943, 2018.
  * [16] https://developer.ibm.com/articles/the-ai-360-toolkit-ai-models-explained, Last Accessed 3 February 2022).
  * [17] AIF360 IBM, https://github.com/ibm/aif360 (Last Accessed 3 February 2022).
  * [18] Tutorial: AI Fairness 360 Open Source Toolkit, IBM Research Trusted AI, https://aif360.mybluemix.net/resources (Last Accessed 3 February 2022).
  * [19] M.T. Ribeiro, S. Singh, C. Guestrin, ”Anchors: High-Precision Model-Agnostic Explanations” September 2018, pp. 241-252, https://ojs.aaai.org/index.php/AAAI/article/view/11491 (Last Accessed 3 February 2022).
  * [20] https://www.oreilly.com/content/introduction-to-local-interpretable-model-agnostic-explanations-lime/
  * [21] https://homes.cs.washington.edu/ marcotcr/blog/lime/ (Last Accessed 18 June 2022).
  * [22] https://github.com/marcotcr/lime , (Last Accessed 18 June 2022).
  * [23] https://homes.cs.washington.edu/ marcotcr/aaai18.pdf (Last Accessed 18 June 2022).
  * [24] https://github.com/marcotcr/anchor (Last Accessed 18 June 2022).
  * [25] M.T. Ribeiro, S. Singh, C. Guestrin, “Why Should I Trust You? Explaining the Predictions of Any Classifier”, August 2016, 10.1145/2939672.2939778.
  * [26] https://www.kdd.org/kdd2016/papers/files/rfp0573-ribeiroA.pdf (Last Accessed 3 February 2022).
  * [27] James Wexler, M. Pushkarna, T. Bolukbasi, M. Wattenberg, F. Viegas, and J. Wilson, ”The What-If Tool: Interactive probing of machine learning models,” IEEE transactions on visualization and computer graphics, vol 26, 1, IEEE, 2019, pp. 56-65.
  * [28] https://facets.dev (Last Accessed 18 June 2022).
  * [29] James Wexler and Andrew Zaldivar, ”FAT 2020 Hands-on Tutorial: Probing ML Models for Fairness with the What-If Tool and SHAP” 2020.
  * [30] Scott Lundberg and Su-In Lee, ”A Unified Approach to Interpreting Model Predictions” SHAP, 12/2022.
  * [31] https://pair-code.github.io/what-if-tool/fat2020.html (Last Accessed 18 June 2022).
  * [32] https://research.google/teams/brain/pair/ (Last Accessed 18 June 2022).
  * [33] M. Zehlike, T. Sühr, C. Castillo, I. Kitanovski, ”FairSearch: A Tool For Fairness in Ranked Search Results,” Companion Proceedings of the Web Conference 2020, ACM, Apr 2020, SBN9781450370240, http://dx.doi.org/10.1145/3366424.3383534.
  * [34] https://dl.acm.org/doi/10.1145/3132847.3132938 (Last Accessed 18 June 2022).
  * [35] https://arxiv.org/abs/1905.13134 (Last Accessed 18 June 2022).
  * [36] https://github.com/fair-search (Last Accessed 18 June 2022).
  * [37] l. Tramèr, V. Atlidakis, R. Geambasu, D. Hsu, J.P. Hubaux, M. Humbert, A. Juels, H. Lin, ”FairTest: Discovering Unwarranted Associations in Data-Driven Applications,” arXiv 1510.02377, 2015.
  * [38] https://github.com/columbia/fairtest (Last Accessed 18 June 2022).
  * [39] https://vatlidak.github.io/resources/eurosp17fairtest.pptx , (Last Accessed 18 June 2022).
  * [40] https://www.youtube.com/watch?v=IZIpbXtDYT4 , (Last Accessed 18 June 2022).
  * [41] https://arxiv.org/pdf/1811.05577.pdf (Last Accessed 18 June 2022).
  * [42] http://www.datasciencepublicpolicy.org/our-work/tools-guides/aequitas/ [NOTE: the site is not secure.] (Last Accessed 18 June 2022).
  * [43] https://dssg.github.io/aequitas/ (Last Accessed 18 June 2022).
  * [44] https://github.com/dssg/aequitas (Last Accessed 18 June 2022).
  * [45] Niels Bantilan, ”Themis-ml: A Fairness-Aware Machine Learning Interface for End-To-End Discrimination Discovery and Mitigation,” vol 36, 1, Journal of Technology in Human Services, Informa UK Limited, pp. 15-30, http://dx.doi.org/10.1080/15228835.2017.1416512.
  * [46] https://themis-ml.readthedocs.io/en/latest/ (Last Accessed 18 June 2022).
  * [47] https://github.com/cosmicBboy/themis-ml/blob/master/paper/main.pdf (Last Accessed 18 June 2022).
  * [48] S.A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choudhary, E.P. Hamilton, D. Roth, ”A Comparative Study of Fairness-Enhancing Interventions In Machine Learning,” Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT2019, ACM Press, 2019, http://dx.doi.org/10.1145/3287560.3287589. (Last Accessed 18 June 2022).
  * [49] https://arxiv.org/abs/1802.04422 (Last Accessed 18 June 2022).
  * [50] https://github.com/algofairness/fairness-comparison (Last Accessed 18 June 2022).
  * [51] https://paperswithcode.com/paper/a-comparative-study-of-fairness-enhancing (Last Accessed 18 June 2022).
  * [52] D’Amour, Alexander and Srinivasan, Hansa and Atwood, James and Baljekar, Pallavi and Sculley, D. and Halpern, Yoni, ”Fairness is Not Static: Deeper Understanding of Long Term Fairness via Simulation Studies”, 2020. Association for Computing Machinery.
  * [53] https://doi.org/10.1145/3351095.3372878 (Last Accessed 18 June 2022).
  * [54] https://github.com/google/ml-fairness-gym/ (Last Accessed 18 June 2022).
  * [55] https://www.youtube.com/watch?v=QdW2xieKhlE (Last Accessed 18 June 2022).
  * [56] E. Choi, T. Bahadori, A. Schuetz, W. Stewart, J. Sun, ”RETAIN: Interpretable Predictive Model in Healthcare using Reverse Time Attention Mechanis” August 2016, https://arxiv.org/abs/1608.05745 (Last Accessed 18 June 2022).
  * [57] https://github.com/mp2893/retain (Last Accessed 18 June 2022).
  * [58] Lin et al., ”Learning Entity and Relation Embeddings for Knowledge Graph Completion,” AAAI Conference on Artificial Intelligence, North America, 2015, https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9571/9523 (Last Accessed 5 February 2022).
  * [59] Huang et al., ”Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation,” arXiv:1504.07678v1. 28 Apr 2015.
  * [60] M. Bastian, S. Heymann, M. Jacomy ,Gephi: An Open Source Software for Exploring and Manipulating Networks. International AAAI Conference on Weblogs and Social Media. 2009.
  * [61] https://gephi.org/users/publications/ (Last accessed 3 February 2022).
  * [62] Zenati et al,”Efficient GAN-Based Anomaly Detection,” 2019, arxiv.org/abs/1802.06222 (Last Accessed 18 June 2022).
  * [63] A Hagberg, D Schult, P Swart, ”Exploring Network Structure, Dynamics, and Function using NetworkX”, Proceedings of the 7th Python in Science conference (SciPy 2008), G Varoquaux, T Vaught, J Millman (Eds), pp. 11-15.
  * [64] Mufei et al, ”Tutorial: Batched graph classification with Deep Graph Library,” https://docs.dgl.ai (Last Accessed 5 February 2022).
  * [65] https://github.com/houssamzenati/Efficient-GAN-Anomaly-Detection (Last Accessed 18 June 2022).
  * [66] https://paperswithcode.com/paper/efficient-gan-based-anomaly-detection (Last Accessed 18 June 2022).
  * [67] https://github.com/dmlc/dgl/tree/0.8.x (Last Accessed 18 June 2022).
  * [68] T. Guo, T. Lin, Tao, N. Antulov-Fantulin, ”Exploring Interpretable LSTM Neural Networks over Multi-Variable Data,” May 2019,
  * [69] https://arxiv.org/abs/1905.12034 (Last Accessed 18 June 2022).
  * [70] https://paperswithcode.com/paper/exploring-interpretable-lstm-neural-networks (Last Accessed 18 June 2022).
  * [71] Linda Heath, ”Triangulation Methodology”, International Encyclopedia of the Social Behavioral Sciences (Second Edition), 2015.
  * [72] https://www.sciencedirect.com/topics/computer-science/triangulation (Last Accessed 3 February 2022).
  * [73] Steven Walczak, ”Methodological Triangulation Using Neural Networks for Business Research,” Advances in Artificial Neural Systems, 2012, 12 pages, doi:10.1155/2012/517234, https://dl.acm.org/doi/abs/10.1155/2012/517234 (Last Accessed 5 February 2022).
  * [74] Aldrees, A, Poland, C, Irshad, SA (2022). Auditing Algorithms: Determining Ethical Parameters of Algorithmic Decision-Making Systems in Healthcare. Information Management and Big Data. SIMBig 2021. Communications in Computer and Information Science, vol 1577. Springer, Cham. https://doi.org/10.1007/978-3-031-04447-2_20.