# Performance of AC-LGAD strip sensor designed for the DarkSHINE experiment

Kang Liu,11footnotetext: co-first author. Mengzhao Li  Junhua Zhang  Weiyi Sun
Yunyun Fan  Zhijun Liang  Yufeng Wang,22footnotetext: co-corresponding author.
Mei Zhao  Kun Liu

###### Abstract

AC-coupled Low Gain Avalanche Detector (AC-LGAD) is a new precise detector
technology developed in recent years. Based on the standard Low Gain Avalanche
Detector (LGAD) technology, AC-LGAD sensors can provide excellent timing
performance and spatial resolution. This paper presents the design and
performance of several prototype AC-LGAD strip sensors for the DarkSHINE
tracking system, as well as the electrical characteristics and spatial
resolution of the prototype sensors from two batches of wafers with different
$n^{+}$ dose.The range of spatial resolutions of 6.5 $\mathrm{\mu m}$ $\sim$
8.2 $\mathrm{\mu m}$ and 8.8 $\mathrm{\mu m}$ $\sim$ 12.3 $\mathrm{\mu m}$ are
achieved by the AC-LGAD sensors with 100$\mu m$ pitch size.

##  1 Introduction

DarkSHINE experiment[1] is a newly proposed electron-on-fixed-target
experiment searching for dark photon produced via electron and nucleon
interaction. The dark photon then decays to a pair of dark matter candidates,
which is called invisible decay[2]. Dark matter pairs escape detection as
missing momentum and missing energy, also resulting in lower momentum and
larger recoil angle of the recoil electron. The missing momentum signature is
used to identify signal from various Standard Model background processes.
Therefore, reconstruction of position and momentum of the incident and recoil
electrons is crucial for this experiment.

Figure 1-(a) shows the sub-detector systems of DarkSHINE experiment. A
tracking system is placed in a downward magnetic field of around 1.5 T which
is provided by a superconducting magnet system. The magnetic filed direction
is defined as the $y$-direction and the electron beam direction as the
$z$-direction, hence the electron will be deflected in the $x$-direction
perpendicular to the magnetic field. As shown in Figure 1-(b), the DarkSHINE
tracking system consists of seven layers of tagging modules and six layers of
recoil modules, while a tungsten target with $0.1\mathrm{X_{0}}$ decay length
is placed in between. Each layer of tracking module consists of two silicon
sensors with length of at least 20 mm, placed at a small angle (100 mrad). The
sensors are expected to be as thin as possible, in order to avoid multi-track
events caused by the interaction between charged particles and the nucleus of
the detector material. The designed position (angle) resolution of the
tracking system is better than 10 $\mathrm{\mu m}$ (0.1%) at the direction of
the electron deflection. To achieve that, several small prototype sensors are
designed with technology of AC-coupled Low-Gain Avalanche Detectors (AC-LGADs,
also called Resistive Silicon Detectors).

(a) DarkSHINE experiment.

(b) DarkSHINE tracking system.

Figure 1:  (a) Detector sketch of the DarkSHINE experiment[1]. Along the
electron incident direction from left to right in the picture, the red
material with a blue brace is the dipole magnet. The tagging tracker is placed
at the center of it. The recoil tracker is located at the edge of the magnet.
The target is caught in the middle. ECAL is placed after the recoil tracker,
followed by HCAL. (b) Sketch of the DarkSHINE tracking system. From left to
right: tagging tracker (with seven layers of tracking module), the tungsten
target, and the recoil tracker (with six layers of tracking module). For each
layer, two strip sensors are shown in the sketch, placed at a small angle (100
mrad).

The Low-Gain Avalanche Detector (LGAD)[3] has been developed in recent years,
with a novel precise detector technology initially proposed and designed for
the precise timing measurements of the High Granularity Timing Detector of
ATLAS[4] and the Endcap Timing Layer of CMS[5] for the High-Luminosity Large
Hadron Collider. Figure 2-(a) shows a sketch of a LGAD sensor. The LGAD
sensors are fabricated on high-resistivity $p$-type substrates with thickness
of about 50 $\mathrm{\mu m}$. Based on traditional $n$-in-$p$ silicon sensor,
the LGAD sensor has an additional highly-doped $p^{+}$ region (namely “gain
layer”) under the parallel $n$-$p$ junction. When a bias voltage is applied
across the sensor, this $p^{+}$ layer will be depleted and create a strong
local electric field, therefore introducing an internal gain.

(a) LGAD sensor.

(b) AC-LGAD sensor.

Figure 2:  (a) Sketch of a section of a single-pad standard LGAD and (b) a
section of a segmented AC-LGAD sensor[6]. The sketches are not to actual
scale.

In order to achieve better spatial resolution while maintaining similar gain
and fast timing performance, the AC-coupled LGAD (AC-LGAD) technology has been
brought up where the signal are capacitively induced and shared among the
metal AC-pads. Figure 2-(b) shows a sketch of an AC-coupled LGAD sensor. The
AC-pads of the sensor for signal readout are placed on a thin dielectric layer
which is grown over the $n^{+}$ layer of the sensor. The AC-LGADs has much
less doped $n^{+}$ layer comparing to the $n^{++}$ layer in the standard
LGADs. It results in an increased inter-pad resistance[6]. A highly-doped
$n^{++}$ implant is still preserved at the edge of the active area of the
sensor, in order to have DC-connection for electron current draining. The AC-
LGAD design can easily adapt to any detector geometry since the segmentation
can be achieved by the AC-pads of any shape.

Two types of strip sensors with pitch (strip) size of 100 (50) $\mathrm{\mu
m}$, 60 (40) $\mathrm{\mu m}$, and 45 (30) $\mathrm{\mu m}$ are designed for
the DarkSHINE experiment. Two batches of wafers of AC-LAGD strip sensor
prototypes have been produced by the Institute of Microelectronics, which are
based on the AC-LAGD technology designed by the Institute of High Energy
Physics (IHEP), CAS. They are named as wafer-11 and wafer-12 sensor in the
following context. The $n^{+}$ doses of these two wafers are 0.01 P and 10 P,
where P is the unit of phosphorus dose defined for the AC-LGADs[7]. Smaller
dose leads to higher resistance of the sensor and therefore smaller leakage
current. Thus it is expected to have better performance on the position
reconstruction and spatial resolution. Figure 3 shows the design and a picture
of a fabricated AC-LGAD strip sensor prototype (size of $1\times
1~{}\mathrm{mm^{2}}$). There are three rings which are (from outside to
inside) the outer ring, the Guard Ring (GR), and the DC ring. The sensors have
a 50 $\mathrm{\mu m}$ epitaxial layer and a 725 $\mathrm{\mu m}$ substrate.

(a) Strip sensor design.

(b) Strip sensor picture.

Figure 3:  (a) Design of the AC-LGAD strip sensor for DarkSHINE. (b) picture
of the silicon strip sensor under microscope.

This paper presents performance of the AC-LGAD prototype strip sensors with
pitch (strip) size of 100 (50) $\mu m$. Section 2 introduces the electrical
characteristics of the prototype sensors. Section 3 discusses position
reconstruction and the spatial resolution measurements.

##  2 AC-LGAD sensor I-V and C-V performance test

###  2.1 Measurement setup

The sensors I-V and C-V tests are carried out in the tracking lab at Shanghai
Jiao Tong University. The wiring setups for I-V and C-V measurements are
illustrated in Figure 4. A four-channel (LC, LP, HC, HP) HV adapter is
connected in parallel between the sensor and the LCR meter to limit the
voltage within a safe range. In C-V measurement, the LCR meter works on the
“Cp-Rp” mode at a frequency of 10kHz according to the recommendation of RD50
[8]. The measurements are carried out in darkness to avoid additional light
current.

(a) I-V test.

(b) C-V test.

Figure 4:  Wiring setup for the (a) I-V tests; (b) C-V tests.

###  2.2 Current-voltage measurement

The measured I-V performance is shown in Figure 5. The breakdown voltage
($V_{BD}$) of wafer-11 (wafer-12) sensor, which is defined as the reverse bias
voltage applied when the leakage current reaches 500 nA, is around 380 V (185
V) at room temperature ( 25∘C). Wafer-11 has higher $V_{BD}$ than wafer-12 due
to higher resistance. Figure 5-(b) shows the I-V curves measured at different
temperatures: 5∘C, 15∘C, 25∘C and 30∘C. As temperature increases, the current
and $V_{\mathrm{BD}}$ also increases due to thermal motion of the electron-
hole pairs. This result suggests that working voltage of the wafer-11
(wafer-12) sensors should be set below 350 V (150 V).

(a)

(b)

Figure 5:  (a) Current-Voltage performance of two AC-LGAD sensors from
wafer-11 (blue) and wafer-12 (red). (b)Current-Voltage temperature dependency
of wafer-12 sensors.

Figure 6 shows I-V uniformity dependency. The measurement is carried out at
room temperature. The bias voltage is scanned in steps of 5 V, whilst it is 1
V scanning step close to breakdown voltage. As shown in Figure 6-(a), the
leakage current shifts a little according to the position of the sensor on the
wafer due to the non-uniformity of doping. As shown in Figure 6-(b), the I-V
curves are measured with two probe needles on different position of the GR and
the DC ring, as marked in Figure 3. The obtained I-V curves are all the same
which indicates that the $n^{+}$ doping within the active region of the
$1\times 1~{}\mathrm{mm^{2}}$ sensor can be considered as being uniform.

(a)

(b)

Figure 6:  Current-Voltage performance of AC-LGAD sensors from wafer-12. (a)
Curves with different color represents sensor from different position on the
wafer: R indicates the row and L indicates the column of the sensor. (b) probe
needles on different position of the GR and the DC ring, as marked in Figure
3.

###  2.3 Capacitance-voltage measurement

The capacitance-voltage (C-V) curves indicate performance of multiplication
layer inside the AC-LGAD sensors. The AC-LGAD sensors are tested at room
temperature, and bias voltage is scanned in steps of 1 V.

Figure 7-(a) shows the measured C-V curves of one wafer-11 sensor and three
wafer-12 sensors, while figure 7-(b) shows the corresponding
$\mathrm{1/C^{2}}$-V curves. No obvious difference in the C-V curves has seen
for sensors from different position on wafer-12. It shows that the doping
uniformity among wafer has negligible effect on the junction capacitance.
There are two plateaus on the $\mathrm{1/C^{2}}$ curves. The
$\mathrm{1/C^{2}}$ curves start rising after the first plateau at the voltage
of around 20 V (24 V) for wafer-11 (wafer-12) sensors, which voltages indicate
that their gain layers are fully depleted. After gain layer depletion, the
curves rising part of wafer-11 and wafer-12 sensors are similar, since they
have the same bulk resistivity. The wafer-11 and wafer-12 sensors both
saturate and enter a second plateau at approximately 40 V voltage value.

(a) C-V curves.

(b) $\mathrm{1/C^{2}}$ curves.

Figure 7:  Capacitance-Voltage and $\mathrm{1/C^{2}}$ curves of the prototype
strip sensors are measured at room temperature. The curve of one wafer-11
sensor is shown in blue. The curves of sensors from different position on
wafer-12 are shown in yellow, green and red respectively.

##  3 Position reconstruction and spatial resolution test

###  3.1 Measurement setup

To investigate spatial resolution of the AC-LGAD sensor, four adjacent strips
on the sensor are tested on a transient current technique (TCT) platform which
are bonded to a 4-channel readout PCB board. As shown in Figure 8-(a), a
4-channel readout PCB board is developed and is produced at IHEP with
reference to the single-channel readout board designed by University of
California Sanata Cruz[9]. A bias voltage is supplied through the back of
sensors while its guard ring is grounded. The working voltage is set to $-350$
V ($-150$ V) for wafer-11 (wafer-12) sensor according to its I-V curve
measured in Section 2.2. Only 100 $\mathrm{\mu m}$ pitch design is tested in
this paper.

(a) Sensor bonded to a readout board.

(b) TCT platform.

Figure 8:  (a) Picture of sensor bonded to a 4-channel readout PCB board using
aluminium wire; (b) Schematic diagram of the laser TCT platform.

The setup of TCT platform is illustrated in Figure 8-(b). The laser wavelength
is 1064 nm. The laser spot, with size of about 6 $\mathrm{\mu m}$, can be
moved along either $x$ or $y$ direction which is controlled by a 3-dimensional
translation platform with accuracy around 1 $\mathrm{\mu m}$. In order to
simulate single photon event, an attenuator is placed above the sensor. The
signal is triggered by a synchronous pulse from laser with a potential trigger
time shift of around 15 ps. After amplification the signal pulses readout from
four wire bonded strips are recorded by digital oscilloscope. It has a
sampling width of 20 Gs/s and bandwidth of 1 GHz.

The laser spot is moved with 2 $\mathrm{\mu m}$ step size along the $x$ plus
and minus directions perpendicular to the strips, which is illustrated as
line-1 and line-2 in Figure 9-(a). The $y$ coordinate is fixed during the
measurement. During a data-taking period the oscilloscope is triggered by the
laser around 1300 times. Four signal pulses are recorded in each triggered
event which is shown as the deepest peaks in Figure 9-(b). The signal
waveforms have typically 1 ns width. The dependence of peak amplitudes comes
from different distances between the readout strips and the laser spot. The
peaks with opposite polarity come from electron diffusion within $n^{+}$ layer
towards the edge of the sensor[6].

(a) Sensor strips.

(b) Signal waveforms.

Figure 9:  (a) Schematic diagram of sensor strips. Two lines along the $x$
coordinate indicate the paths of laser spot moving. C1 to C4 represent four
strips wire bonded for signal readout. (b) Waveforms of one triggered event.
The yellow, red, blue and green curves represent four readout channels
corresponding to C1, C2, C3 and C4.

###  3.2 Position reconstruction and spatial resolution

A simplified linear model[10] is used to reconstruct the position of signal
(i.e. the laser spot centre). This linear model assumes that signal on each
pad decreases linearly with its distance from the point of particle incidence.
For strip sensors, the position information recorded by four strip sensors has
only one dimension. As shown in figure 9, signal fraction is defined as ratio
of the lowest peak of each channel to the lowest peak of all channels. It can
be assumed that the signal fraction on each strip varies linearly with the
distance from the impact point. The relationship between the impact position
and the signal fraction of each strip can be expressed by

$x=(f_{i}-\alpha_{i})/\beta_{i},i=1,2,3,4$

where $x$ is the impact position, $f_{i}$ is the signal fraction of each
channel, $\alpha_{i}$ is the signal fraction of each channel at $x=0$, and
$\beta_{i}$ is the change rate of the signal fraction of each channel with the
impact position.

Figure 10-(a) shows the maximum amplitudes of the four channels, which is
averaged over all triggers in each step. When laser spot is blocked by the
metal strips, the collected charge or signal amplitude reduces to zero. We set
$x=0$ at centre of the gap between C1 and C4. This is derived by finding the
edges of the metal strip after deriving the signal distribution for each
channel.

(a)

(b)

Figure 10:  (a) The average maximum amplitudes with respect to the $x$
coordinate, while $x=0$ is set to be at the centre of the gap where the laser
hits. The absolute value of amplitudes after normalization are shown in the
plot. Highlighted areas are selected for data analysis. All plots are measured
with line-1 of the wafer-12 sensor. (b) Fraction of the maximum amplitude of
each readout channel, averaged over all triggered events. Linear fits are
applied to each fraction, as shown in colored dashed lines. The yellow, red,
blue and green curves represent the four readout channels C1, C2, C3 and C4 of
the oscilloscope respectively.

Figure 10-(b) shows the fraction of the maximum amplitude of each readout
channel as a function of $x$, computed by

$f_{i}=\frac{A^{i}_{max}}{A^{1}_{max}+A^{2}_{max}+A^{3}_{max}+A^{4}_{max}},i=1,2,3,4$

where $A^{i}_{max}$ is the maximum amplitude of each channel at given $x$. A
linear function is then used to fit the amplitude fraction of each channel.
Therefore, four $x$ values can be solved from the fit function for any given
event with measured amplitude fraction $(f_{1},f_{2},f_{3},f_{4})$, and the
average among $(x_{1},x_{2},x_{3},x_{4})$ is taken as the reconstructed laser
spot position.

Figure 11-(a) shows a distribution of the reconstructed $x$ coordinates for a
known test position at $x=0~{}\mu m$. From a Gaussian fit to this
distribution, its mean value -0.88 $\mu m$ is taken as the reconstructed $x$,
and its sigma value 9.63 $\mu m$ is considered as the spatial resolution.

(a)

(b)

Figure 11:  (a) Distribution of the reconstructed $x$ coordinate for actual
laser spot position at $x=0~{}\mu m$. The red curve shows a Gaussian fit to
the distribution. (b) Reconstruction results of the two sensors from wafer-11
and wafer-12. The dashed line ($y=x$) represents the actual laser spot
position. The results of wafer-12 sensor are shifted up by 30 $\mathrm{\mu m}$
for better demonstration.

Figure 11-(b) shows two sets of reconstructed positions from wafer-11 and
wafer-12 and a comparison with spot positions. The dashed lines indicate spot
positions while the dots are reconstructed positions. They are in good
agreement. Figure 12-(a) shows the corresponding spatial resolution as a
function of $x$, which varies from 6.5 $\mathrm{\mu m}$ to 8.2 $\mathrm{\mu
m}$ for wafer-11 and from 8.8 $\mathrm{\mu m}$ to 12.3 $\mathrm{\mu m}$ for
wafer-12 respectively. Better resolution of wafer-11 sensor than wafer-12 is
due to the fact that $n^{+}$ dose (0.01 P) of wafer-11 is much smaller than
wafer-12 (10 P). The spatial resolution is slightly worse in the edge of gaps
as shown in figure 12-(b). It is due to the fact that signal fraction is less
stable when laser spot closing to metal strips. Similar effect also leads to a
small overall discrepancy between the line-1 and line-2 measurements.

(a) Spatial resolution. (b) standard deviation of signal fraction.

Figure 12:  (a) Distribution of spatial resolution with respect to $x$
coordinates. The blue and red marks represent the reconstructed position for
wafer-11 and wafer-12 sensor, respectively. The circle represents measurement
from line-1, while triangle represents measurement from line-2. (b) The
distribution of the standard deviation of the signal fraction with respect to
the $x$ coordinate

##  4 Summary

This paper presents the performance of two batches of prototype AC-LGAD
sensors with pitch size of 100 $\mu m$ designed for the DarkSHINE experiment.
The range of spatial resolutions are 6.5 $\mathrm{\mu m}$ $\sim$ 8.2
$\mathrm{\mu m}$ and 8.8 $\mathrm{\mu m}$ $\sim$ 12.3 $\mathrm{\mu m}$ for
wafer-11 and wafer-12 sensors. The typical sensor response time is about 1 ns.
The wafer-11 sensor delivers better spatial resolution because of smaller
$n^{+}$ dose. Both wafer-11 and wafer-12 sensors satisfy the requirement of
the DarkSHINE tracking system.

This work has been supported by a grant from the National Natural Science
Foundation of China (Grant NO.12150006).

## References

  * [1] J. Chen, et al, Prospective study of light dark matter search with a newly proposed DarkSHINE experiment, _Sci. China Phys. Mech. Astron._ 66 (2023) 211062.
  * [2] M. Fabbrichesi, E. Gabrielli, and G. Lanfranchi, The dark photon. arXiv: 2005.01515
  * [3] H.F.W. Sadrozinski et al., Ultra-fast silicon detectors, _Nucl. Instrum. Meth._ A 730 (2013) 226.
  * [4] ATLAS collaboration, Technical Proposal: A High-Granularity Timing Detector for the ATLAS Phase-II Upgrade. CERN-LHCC-2018-023.
  * [5] CMS collaboration, Technical proposal for a MIP timing detector in the CMS experiment phase 2 upgrade, CERN-LHCC-2017-027.
  * [6] G. Giacomini, W. Chen, G. D’Amen and A. Tricoli, Fabrication and performance of AC-coupled LGADs, _JINST_ 14 (2019) P09004.
  * [7] M. Li, et al., The performance of large-pitch AC-LGAD with different N+ dose, (2022) arxiv:2212.03754.
  * [8] A. Chilingarov, RD50-Recommendations for performing measurements. Part I: IV and CV measurements in Si diodes.
  * [9] N. Cartiglia, et al., Beam test results of a 16 ps timing system based on ultra-fast silicon detectors, _Nucl. Instrum. Methods Phys. Res. A_ 850 (2017) 83-88.
  * [10] M.Tornago, R. Arcidiacono et al., Resistive AC-Coupled Silicon Detectors: _Nucl. Instrum. Methods Phys. Res. A_ 1003(2021) 165319.


Our visual interactive workspace supports the analysis of sheet music
collections through visualizations that are connected through linking and
brushing. After filtering a sheet music corpus dlgA, music analysts can
perform exploration, comparison, and detection tasks. The feature matrix dlgB
provides detailed information about low-level characteristics for musical
compositions. The metadata table dlgC enables browsing through the titles,
composers, and composition forms. An MDS projection view dlgD helps to
identify similar compositions or even clusters based on selected features. The
sheet view dlgE allows analysts to view compositions using the familiar
notation. Domain experts can use CorpusVis to confirm and generate hypotheses
and detect interesting patterns between composers and composition types.

# CorpusVis: Visual Analysis of Digital Sheet Music Collections

Matthias Miller10000-0002-6281-2173, Julius Rauscher1,0000-0003-1318-9642,
Daniel A. Keim10000-0001-7966-9740, and Mennatallah El-
Assady20000-0001-8526-2613
1University of Konstanz, Germany 2ETH AI Center, Zürich, Switzerland

###### Abstract

Manually investigating sheet music collections is challenging for music
analysts due to the magnitude and complexity of underlying features,
structures, and contextual information. However, applying sophisticated
algorithmic methods would require advanced technical expertise that analysts
do not necessarily have. Bridging this gap, we contribute CorpusVis, an
interactive visual workspace, enabling scalable and multi-faceted analysis.
Our proposed visual analytics dashboard provides access to computational
methods, generating varying perspectives on the same data. The proposed
application uses metadata including composers, type, epoch, and low-level
features, such as pitch, melody, and rhythm. To evaluate our approach, we
conducted a pair-analytics study with nine participants. The qualitative
results show that CorpusVis supports users in performing exploratory and
confirmatory analysis, leading them to new insights and findings. In addition,
based on three exemplary workflows, we demonstrate how to apply our approach
to different tasks, such as exploring musical features or comparing composers.
<ccs2012> <concept> <concept_id>10003120.10003145.10003146</concept_id>
<concept_desc>Human-centered computing Visualization techniques</concept_desc>
<concept_significance>500</concept_significance> </concept> <concept>
<concept_id>10002951.10003317.10003318.10003321</concept_id>
<concept_desc>Information systems Content analysis and feature
selection</concept_desc> <concept_significance>500</concept_significance>
</concept> <concept>
<concept_id>10002951.10003317.10003318.10003319</concept_id>
<concept_desc>Information systems Document structure</concept_desc>
<concept_significance>500</concept_significance> </concept> </ccs2012>
[500]Human-centered computing Visualization techniques [500]Human-centered
computing Visual analytics [500]Information systems Content analysis and
feature selection [500]Information systems Document structure [500]Human-
centered computing Visualization systems and tools

††volume: 41††issue: 3

##  1 Introduction

Investigating music collections are relevant for both domain experts such as
musicologists and music theorists and usual music consumers [WJVR16].
Predominantly, the audio format has been the starting point for many
applications including music classification [FLTZ11], personalized music
recommendation [BHF∗13], structure analysis [PMK10], and the generation of
music playlists [STOH21]. In contrast, we are convinced that the investigation
of sheet music collections based on digital symbolic formats such as MusicXML
[Goo01], MIDI, Humdrum [Sap15], and Lilypond [NN03] have received less
attention compared to audio.

While services such as sheet music recommendation offered by, e.g., OKTAV
[OKT21] leverage musical features and individual preferences to provide sheet
music suggestions for piano players, it remains unclear how such
recommendations are computed. To understand which musical features (e.g.,
structure or melody) reveal differences and similarities, computational
approaches for sheet music analysis have been proposed in the community,
supporting various tasks such as genre and composer classification [ANC17].
Besides close reading of sheet music as a typical task in musicology, distant
reading is also an essential task in the digital humanities [JFCS15]. Close
reading supports musicologists to investigate musical compositions on a
detailed level, abstract visualization contain the potential to provide an
overview over a large set of musical compositions within their context, thus
enabling distant reading. While approaches that solely rely on algorithmic or
computational approaches mitigate the tedium of manual analysis of a dataset
done by human analysts, they often are not accessible to a broader user group.
Still, they are limited to music analysts that have programming skills. To
address this gap, tailored visualizations combined with user interaction can
be employed to increase access to music analysis methods for a broader user
group while taking the human analyst in the analysis loop, which is a
fundamental aspect for knowledge generation in visual analytics [SSS∗14]. An
essential aspect of the understanding process is critical thinking by the
analyst about the subject, which was introduced by Bradley et al. as _slow
analytics_ [BEAC∗18].

The availability of sheet music datasets such as KernScores [Sap05] or
MuseScore [Mus11] have the potential for analysis at different scales,
benefiting from using projection techniques based on underlying features
[MCF18]. While MuseScore contains more than a million compositions, the
quality of the uploaded material varies from content that is provided by users
who ensure that provided content is faithful to the original compositions to
pieces that suffer from inaccuracy or incompleteness. Therefore, it is
challenging to maintain a high quality of the underlying data for the analysis
when considering the full dataset, requiring expensive data cleaning steps
before performing analysis tasks on it. Manual curation or selection processes
are required to set the focus on a particular subset that is either provided
by users, that only provides high-quality content. Alternatively, analysts can
manually view each composition that shall be part of a curated dataset, which
is, of course, quite a tedious process. The research field of optical music
recognition provide approaches to convert printed sheet music into symbolic
formats such as MusicXML. Yet, it needs further improvements until
musicologists trust these automatic results [SF20]. Enabling analysts to
influence the data foundation is crucial for the analysis.

Besides data quantity, increasing the quality of the data is crucial for
effective analysis. This includes a targeted selection of representative
samples. For example, a basic issue in humanities research is striving to
avoid the exclusion of marginalized positions, a typical risk of data
colonialism. For instance, famous classical composers such as Johannes
Sebastian Bach, Wolfgang Amadeus Mozart, Joseph Haydn, or Ludwig van Beethoven
are often a more prominent analysis subject in musicology than less known
composers such as Muzio Clementi, Domenico Scarlatti, or Francois-Joseph
Gossec. This marginalization of composers is reaffirmed by the vast amount and
duplicates of famous compositions in datasets such as MuseScore. To enable a
large-scale analysis of sheet music collections, a corpus-level overview is
essential. This has the power to enable analysts to audit and refine corpora,
as well as compare music sheets on numerous features. Applying abstract
visualization techniques to sheet music corpora at a larger scale has the
potential to provide insights about the work of composers or typical
differences between compositions types without the need to manually analyze
every detail of single data items.

Driven by the need for a corpus-level music analysis technique, in this work,
we address the research question: How to support music analysts to explore,
investigate, and compare sheet music collections based on metadata and low-
level features using interactive visualization? Our aim is to provide them
with a multiscale and multi-perspective bird’s-eye view on sheet music
corpora. We thus designed an interactive visual workspace that provides
multiple tailored analysis components in an inter-linked dashboard. The visual
analysis is supported by computational methods, such as clustering, to aid in
pattern finding. Analysts can openly explore the underlying corpus,
investigate a set of pre-configured use cases, or verify hypotheses through
crafting their own analysis workflow.

Contributions –  This work contributes a problem characterization addressing
visualization requirements with regard to the analysis of sheet music corpora.
We provide a list of relevant data and task characteristics, as well as a
description of target audiences. A major contribution is the Visual Analysis
Workspace for sheet music collections through a combination of multiple
components that are seamlessly connected through linking and brushing. We
conducted a qualitative evaluation to assess the applicability of our approach
and provide details about its benefits and drawbacks. Finally, we discuss open
research opportunities to inspire interdisciplinary collaboration at the
interface of visualization and musicology.

##  2 Related Work

The field of musicology is a wide research area that covers heterogeneous
research questions and challenges. The subfield _Visual Musicology_ at the
interface of musicology and visualization research as introduced by Miller et
al. illustrates the vast opportunities for which visualization could be
applied to support domain-related, scientific issues [MSK∗19]. Their framework
highlights the potential of interactive visualization to perform analysis
tasks, including information retrieval, exploration, and comparison. We use
the _visual musicology graph_ to classify the work presented in this paper
accordingly. Specifically, we focus on _Structural Features_ and _Meta-
Information_ of sheet music and the visualization tasks
_Overview/Summarization_ , _Navigation/Exploration_ , _Clustering_ ,
_Comparison_ , and _Details on Demand_ within the domains _Theory & Analysis_
and _History_.

Visual Analysis of Music Collections – The visual investigation of musical
data collections has already received attention by visualization researchers.
Khulusi et al. created a web-based interface called “musiXplora” as part of a
digitization project that enables humanities researchers to investigate
musicological data such as meta-information (e.g., gender, religious
denomination, profession, institutions) of musicians and instrument makers
[KKFJ20]. For instance, this interactive prototype facilitates information
retrieval and browsing about the life and work of different composers.
Similarly, Jänicke et al. designed an interactive visualization tool for the
interactive profiling of musicians and their relationships based on their
meta-information [JFS16]. They did not consider specific features about the
compositional work but rather focus on the contextual information about the
life, profession, and instruments they mastered. Pampalk used SOMs to
visualize and classify the genre of music data as “Islands of Music” based on
audio-extracted features such as the loudness information from audio files
[Pam01]. Chen and Putz designed an interactive UI for browsing and organizing
music collection allowing to listen and visually explore the similarity of
musical pieces based on low-level features that reflect the genre and style of
pieces from different composers [CB09]. Weiß et al. show how exact tonal
features (pitch information) and metadata from audio data help to detect style
changes over the different epochs from baroque to modern music [WMDM18].
Georges and Nguyen use a dataset of 500 classical composers to visually
analyze different epochs using dendrograms and MDS projection techniques
[GN19]. By that, they demonstrate how abstract visualization methods support
the comparison of composers [BMHC16].

Feature Extraction from Music Data – Music is available and can be stored
based on different formats such as symbolic (e.g., MusicXML [Goo01]) or audio
data (e.g., mp3). Depending on this format, different features can be
extracted that can afterwards be used for different tasks such as music
recommendation [Sch19]. Similar to extracting features from audio signal data
(e.g., spectral information) for further processing [LWSF20], it is possible
to extract low-level features from symbolic music [Cc21]. For example, music21
[Cc21] is a Python library that enables programmers to extract jSymbolic
features from different symbolic sheet music formats including MusicXML. McKay
and Fujinaga discuss how such features can be used for MIR research tasks
[MCF18] such as composer classification as done by Verma and Thickstun
leveraging convolutional networks [VT19]. McKay leverages audio and symbolic
music data sources to perform music classification and the creation of music
information retrieval tools using his own jMIR software suite that can even
consider contextual/cultural metadata [MF09]. Corrêa and Rodrigues published a
survey about music genre classification that reveals that such tasks primarily
use automatic algorithms without using advanced visualization techniques
[CR16]. Merely computational approaches do not enable music analysts to
explore low-level features, making it impossible to step into the analysis
process to get a better overview over the inner workings of the applied
algorithms.

Visualization of Collections in the Digital Humanities – Besides musicology,
there exist other scientific issues within digital humanities that have
already been addressed by information visualization researchers such as the
visual analysis of poems [MLCM16]. Jänicke et al. discuss many digital
humanities projects that require text processing methods [JFCS17] while
Kirschenbaum also confirms the essentiality of text for humanities research
ranging from close via not-reading to distant reading [Kir07]. For example,
Bludeau et al. implemented a web-based prototype to enable the visual
investigation of literature and handwritten notes from _Fontane’s
Handbibliothek_ [BBBD20]. For instance, the field of text analysis deals with
comparison tasks for plagiarism detection by identifying typical features for
specific authors [GMB∗14]. Keim and Oelke use textual features to create a
literature fingerprinting visualization that facilitates the analysis of whole
books [KO07]. We argue that existing visualization methods that have been
successfully applied to similar endeavors in related research areas enclose
the potential for methodology transfer [MSK∗19]. Consequently, we consider it
to be useful to get inspired by existing visual techniques to address unsolved
challenges such as the visual analysis of sheet music collections.

Research Gap – While there are several projects within this research field,
they are often limited to audio data only [CB09, WMDM18]. Thus, we argue that
the analysis of music based on symbolic formats has received less attention.
We argue this shift towards audio data could be due to larger user groups of
music applications such as (e.g., Spotify) which benefit from effective music
recommendation methods. We see the reason for this unbalance in the smaller
group size of sheet music consumers, which is much smaller. Liem et al.
discuss the issue of focusing on audio signal only and argue for multimodal
and user-centered analysis strategies to improve the accessibility of digital
music data [LME∗11, MGD11]. Nevertheless, the increased use of digital
symbolic music formats led to services such as OKTAV [OKT21] which provides
sheet music recommendations based on symbolic features. While automatic
solutions have their application areas, employing visual methods allows
analysts to step into the analysis process, helping them to gain new insights
or generate and confirm hypotheses, which is also known as human-in-the-loop
approaches [SSS∗14]. To our knowledge, there is no previous work that readily
supports the analysis of sheet music corpora, facilitating the visual
comparison of metadata and low-level features.

##  3 Problem Description

Our primary objective is to provide an _interactive visual workspace_ to
support the investigation and exploration of sheet music collections. The
target audience primarily comprises musicologists and music theorists. We
assume that music librarians and Music Information Retrieval (MIR) researchers
could benefit from our work as well. For this reason, we gathered information
about the data and task requirements by conducting initial expert interviews
and consulted existing literature.

Feature Characteristics – When analyzing musical compositions, it is essential
to musicologists to know the relationship to the _composers_ which allows to
include domain knowledge into the analysis process. Usually, one can assign
musical pieces to a certain _type or style_ which often correlates to a
specific genre. Considering this contextual information can help detect and
compare entities from a musical score collection. As discussed in section 2,
there exist various frameworks that allow for extracting _statistical
features_ from symbolic music. We use music21 to extract jSymbolic features
from digital sheet music [CA10] that provides the foundation for the visual
analysis workspace that we introduce in the following sections. Specifically,
these low-level features contain rhythmic, melodic, and pitch information that
provide informative attributes about single compositions or numerical
distributions when investigating a score collection. In addition, we consider
available contextual metadata including composer, composition type, and
temporal characteristics.

Task Characteristics – Content-based MIR aims at uncovering music data
relevant characteristics that can be employed for similarity computations and
retrieval tasks [GN19]. For instance, a specific retrieval task [T1]would be
the identification of all compositions from Mozart that have a time signature
of a triple meter. Besides performing inquiring based on exact information,
music analysts could also be interested in exploring similarities of pieces
within a dataset [T2]with the objective to create new hypotheses or gain new
insights about the underlying information [KKM∗20]. Additionally, the
comparison of two or even a set of compositions [T3]can be relevant during
music analysis to identify commonalities of the complete work of composers or
different versions of the same composition [ULMS10]. Not only comparison tasks
can benefit from grouping and clustering based on meta information [T4], but
also the investigation of single composers based on statistical values, they
provide a quick overview without the need to view all items separately.
Depending on the quality of the dataset, it can be crucial for analysts to
identify duplicates within a dataset [T5]to omit works from further analysis
or other issues such as detecting musical plagiarism [Cam20]. Based on our
cooperation with musicologists we focus on the following tasks that address
typical analysis challenges:
[T1] Retrieve/Filter compositions based on title, composer, or type
[T2] Exploratory analysis of a musical score collection
[T3] Comparison of scores, composers, types, and features
[T4] Clustering/grouping of pieces, composers, and types
[T5] Detection of similar entities in a given corpus

Visualization Requirements – Based on the described data and task
characteristics, we can identify essential requirements for the visualizations
that are necessary to support the analysis process. For the information
retrieval task [T1], we need a separate filter and sort functionality for each
meta feature and the low-level characteristics. Suppose a user is looking for
a specific composer or composition. In that case, we need an interactive
search bar that allows for direct querying of the underlying dataset to avoid
the tedious search for data items within the provided visualizations. To
associate the temporal aspects of composers, we need a timeline visualization
that can also be directly used to filter composers to be analyzed by the user.
The type of composition is another fundamental characteristic for music
analysts that can be used for exploration [T2], comparison [T3], or
identification of interesting pieces. To extract similar compositions within
the dataset, we can employ a dimension reduction technique such as MDS [CC08]
to project dozens of low-level features into a two-dimensional space [T5].
Then, identifying groups based on a clustering method is required to detect
multiple pieces that have related features [T4].

Used Dataset – MuseScore comprises a heterogeneous dataset with over a million
compositions. Instead of using all these files, we manually selected a subset
from certain users who upload curated files, e.g., ClassicMan [Cla21].
Typically, these compositions have high community ratings. Because of this
preselection process, we ensure that the compositions in the corpus are of
better quality.

Figure 1:  The keyword search allows to filter composers and composition
types. Users can extend the dataset by uploading custom data. Further
settings: reset the system state, save use cases, or open an interactive
introductory guide through through CorpusVis.

##  4 Visual Interactive Analysis Workspace

This section introduces the interactive workspace and describes three
exemplary analysis workflows to illustrate how to use it for analyzing sheet
music corpora. CorpusVis is available as a web-based application via
https://visual-musicology.com/corpus.

###  4.1 CorpusVis – User Interface

The analysis workspace consists of five components (see CorpusVis: Visual
Analysis of Digital Sheet Music Collections) supporting data filtering and
visual analysis following Shneiderman’s Visual Information Seeking Mantra
[Shn96]. The components are connected through animated linking and brushing,
showing relevant details during the analysis process to the user. For
instance, if the user hovers a circle (represents a composition or
aggregations) in the projection view dlgD, then the corresponding composer and
forms dlgA and matrix row(s) matrix dlgB are highlighted. The settings widget
(Figure 1, top left) enables users to switch between the German and English
and provides a guided tour for all components.

Data Import and Selection – This component enables uploading custom sheet
music datasets (_.mxl_ format) or filtering existing datasets by keywords
(composer names, title, etc.), composition types, and via composer selection
from the timeline [T1]. The user can save an interesting data selection as a
use case, or simply reset the analysis through further options provided in the
settings widget. The composer timeline shows 62 composers for which we could
directly extract their life data from the files. Similar to Weiß et al.
[WMDM18], we use four historical periods (baroque, classic, romantic, modern)
to position the composers on the timeline. We apply an animated layout that
uses the vertical axis to avoid overlapping.

Figure 2:  The feature distribution chart provides statistical characteristics
(min., median, max.) of a subset and enables comparing the subset to the rest
of the corpus regarding a selected feature.  Figure 3: This exemplary matrix
shows 14 features from the two composers Clara Schumann and Modest Petrovich
Mussorgsky. Users can visually detect low (purple) and high (yellow) values.

jSymbolic Feature Matrix – Depending on the grouping, the feature matrix
encodes statistical information extracted from single compositions or clusters
at the workspace’ center. User interaction allows sorting and filtering all
features to focus only on subsets if needed [T3]. Analysts can also inform
themselves by getting detailed information for each available feature. The
features support the identification of salient patterns, and the matrix can be
used for comparison tasks [T2]. Overall, the feature matrix can display up to
46 different jSymbolic features extracted with music21 [Cc21]. The feature
distribution chart (see Figure 2) helps analysts compare the statistics of a
feature to the whole corpus, including the minimum, median, and maximum value
of the feature for each group [T3]. This example shows the statistical
information of the feature _Note Density,_ which corresponds to the average
number of notes that are played within a second. The values show that the
corpus contains pieces with a note density above 100, while the selection is
situated has higher values at the left side of the distribution compared to
the overall corpus. The distribution chart illustrates that the feature
regarding the selection is similar to the rest of the corpus (left-skewed).
Figure 3 displays 14 features horizontally sorted by category (M, P, R) with
the rhythmic feature _Staccato Incidence_ emphasizing a few pieces with a high
amount of staccato usage.

MDS Projection View – We compute a multidimensional scaling projection based
on the selection of low-level features to show the similarity between single
compositions or clusters in a separate component [T5]. The circle labels
displayed in Figure 4 are the initials of the composer names. In this case,
the color of the circles provides epoch information, and the respective color
scale is shown in the composer timeline. Analogously, a separate color scheme
is used for the composition types. On-demand, the user can execute a DBSCAN
clustering [HK98] and manipulate its EPS parameter via the cluster slide at
the bottom to identify groups of similar entities [T4]. DBSCAN has the
advantages that it does not require a parameter for the number of clusters and
it can identify outliers. We set $\mathrm{MinPTS}=2$ to enable the
identification of duplicates. The visual grouping of the clusters uses
concaveman [Aga17], a JavaScript library that builds on an algorithm based on
work by Park and Oh [PO12] which allows to create concave hulls around given
clusters.

Figure 4:  MDS Projection View: If no grouping is applied, each circle in the
projection view represents a single composition. The color provides the
information to which epoch a piece is assigned.  Figure 5: Analysts can use
the metadata table to sort selected compositions and simply lookup interesting
pieces from the list.

Metadata Table – As a quick reference to the feature matrix, the workspace
contains a meta information table of the current selection as displayed in
Figure 5. Users can alphabetically sort a current selection based on
composers, types, titles, and opus numbers. This rather simple data
representation facilitates the connection to the rows of the more abstract
feature matrix (see Figure 3) and projection view (see Figure 4).

Sheet View – Viewing sheet music using standard visual encoding is crucial for
music analysts. Thus, the last component is the _sheet view,_ which music
analysts can use to estimate the quality of the underlying content (see Figure
6). Users can open the first page of compositions and relate it to the
features via the context menu. Hence, the analyst can better understand
abstract features by seeing the direct relation through the familiar notation.
Via the playback icons at the top left corner of each item in the sheet view,
analysts can also listen to the compositions on demand. We use the command-
line version of MuseScore [Mus11] to convert the underlying MusicXML files
into their .mp3 version. These files are also cached to reduce loading times
if multiple users want to play back the same compositions.

###  4.2 Analysis Use Cases

CorpusVis offers multiple prepared analysis use cases consisting of
preselected data and specific component configurations tailored to the
analysis of the respective topic. Users can create additional scenarios by
either manually filtering compositions from the prepared dataset compositions
or uploading their own datasets. In the remainder of this section, we will
introduce three exemplary use cases. The first two use cases represent a
hypothetical user who is interacting with the system while [UC3] has been
provided by one of our expert users, showcasing how CorpusVis covers the
introduced tasks. All use cases can be opened directly in CorpusVis via the
following links: Investigating Epoch Characteristics [UC1], Tonality vs.
Atonality [UC2], and Composer Comparison [UC3].

Investigation of Epoch Characteristics [UC1] – In this use case, the analyst
wants to explore the differences between the historical epochs [T2]. Thus,
from the list of use cases, the analyst selects _Epoch Comparison_. This use
case considers 51 heterogeneous compositions from composers like _Pachelbel_
(baroque), _Wagner_ (romantic), and _Schönberg_ (modern). Here, only a subset
of eight features of the three feature categories (M, P, R) dlgB such as
_Repeated Notes_ and _Pitch Variety_ are selected that separate the four
epochs within the projection view [T3]. While the modern and baroque
compositions are distinguishable from the all others, the pieces from the
intermediate epochs classic and romantic have more overlap dlgD. The analyst
identifies composers between the romantic and modern epoch, such as
_Stravinsky_ , _Rachmaninoff_ , _Schönberg_ , and _Webern_ , that reside
within classical and romantic composer groups. By using the column of the
feature _Size of Melodic Arcs_ to sort the feature matrix, the analyst detects
that romantic composers (yellow circles in the projection view) often used
large melodic arcs compared to artists from other epochs.

Figure 6:  The sheet view allows viewing the first page of compositions. At
the sheet level, analysts can perform harmonic and rhythmic analysis tasks
using MusicVis [MFH∗22].

Atonality versus Tonality [UC2] – An interesting aspect regarding the harmonic
development between the different epochs is the discussion of tonality and
atonality. Atonal music is lacking a tonal center which means that none of the
twelve available pitch classes is favored over the others as it is typically
the case for classical music. Moreover, atonal music comprises more dissonant
intervals (such as minor second and tritone intervals) which are available in
the pitch feature group dlgB. Specifically low values of the feature _Most
Common Pitch Class Prevalence_ are a good indicator for a composition to be
atonal. In this example, the analyst is presented with 18 compositions from
both classical composers including Mozart and Beethoven who composed rather
tonal music as well as composers like Schönberg and Webern who are well-known
by musicologists for their atonal compositions dlgA. After starting with the
analysis, the analyst uses the projection view dlgD to see which compositions
are positioned closer together to detect similar pieces [T5] (see Figure 4).
As described, the analyst knows that tonality depends more on pitch and
melodic features and not on rhythm. The configuration of the feature matrix
dlgB in this use case does not contain any rhythmic feature, which covers the
mental model of the analyst. By viewing the columns in the feature matrix, the
analyst detects as expected that the _Melodic Tritones_ feature has much
higher values for the atonal compositions confirming the hypothesis that
atonal compositions contain more dissonant intervals [T2]. By investigating
the feature distribution chart of _Number of Common Pitches_ , the analyst
discovers that the compositions subset of the configuration comprises more
pieces with zero common pitches compared to the overall distribution in the
corpus [T3]. Since the correlation matrix confirms a positive correlation of
the _Number of Common Pitches_ to _Most Common Pitch Class Prevalence_ , the
analyst is interested to see the values for the single compositions. The
analyst is able to detect that the pieces by Schönberg and Webern try to avoid
a tonal center as none of the available pitches seems to be favored compared
to the works by the classical composers [T5].

Composer Comparison [UC3] – In the first scenario, the domain expert compares
similar pieces from two different composers [T3]and selects Erlkönig[T1]by
Franz Liszt and Franz Schubert dlgA. As a first step, the analyst manually
inspects the first page using the sheet view dlgE and detects the use of
melodic octaves in the left hand of Liszt’s interpretation which appears to be
a salient difference between the compositions. Based on this observation, the
expert focuses on the features _Melodic Octaves_ and _Average Melodic
Interval_ which he assumes that these features should reflect this
characteristic. The feature matrix dlgB confirms the analyst’s initial finding
as the values for the _Melodic Octaves_ feature is 28.1% for Liszt, but only
2.05% for Schubert [T3]. Consequently, the analyst wants to explore whether
this aspect is a typical difference between the two composers [T2]and selects
all available pieces in the dataset from both composers and aggregates them
based on the composer attribute [T4]. Although the distinction decreases, the
trend is still present (6.30% - Liszt vs. 3.8% - Schubert). While the two
features selected by the user are correlated, the analyst discovers that they
represent discriminating characteristics between Liszt and Schubert. As the
last step, the analyst is eager to see any unusual pieces (outliers) and
ungroups the two clusters back into the single compositions. By activating the
clustering in the MDS projection view dlgD the analyst groups similar pieces
into groups finding many similarities on the feature level between Liszt and
Schubert [T5]. For instance, while the compositions 11\. Ecossaise D.781
(Schubert) and Vive Henri IV (Liszt) are quite similar on the feature level.
The user finds Ave Maria - Ellens dritter Gesang by Schubert to be an
interesting outlier compared to most of the other compositions. Eventually,
the analyst uses the metadata table dlgC to sort the matrix using the composer
attribute to explore differences between the two collections [T2]. The analyst
confirms the less surprising hypothesis that Liszt’s works have a much higher
Pitch Variety. Based on the feature _Number of Common Pitches_ he discovers
that while Liszt almost always has zero pitches which individually account for
at least 9% of all notes, Schubert even has compositions with five common
pitches such as _Nachtviolen (D.752)_ [T3].

##  5 Pair Analytics Study

We conducted pair analytics sessions with users of different expertise levels
to evaluate the applicability of CorpusVis introduced in section 4. Since the
number of available qualified domain experts is limited, we decided to use the
pair analytics study method [AKGF11].

Preliminary Expert Feedback – Before the actual study, we presented CorpusVis
to three domain experts to gather initial feedback in an informal pre-study.
The domain experts appreciated interacting with an exploratory visualization.
One of the experts liked the way on how music information could be retrieved
through simply entering keywords or mouse interactions on the composer
timeline or types [T1]. The interlinked components helped in understanding the
connection between low-level features and the compositions [T2]. They found it
useful to explore similarities and differences between composers and types by
grouping them together [T3]or through using the clustering in the projection
view [T4]. They appreciated the multiple components and the use of color to
visually show connections of similar entities [T5]. Especially, they could use
the feature filtering functionality to identify outliers within single feature
dimensions. They also noted that the detailed description of the features in
connection with the sheet view provides music students the opportunity to
experience new characteristics of sheet music [T2]. In their opinion, the
sheet music component is a core element to verify hypotheses and to
investigate the quality of compositions. However, they also critiqued our used
color map and the lack of a feature to upload their own music sheets. Both of
these issues were addressed in a refinement cycle before our study. In
addition, based on our observations from the interactions in the pre-study, we
added the features of interactive clustering and distribution-based grouping
and filtering.

###  5.1 Study Design

We conducted the study to gather user feedback to identify benefits and
drawbacks of the introduced analysis workspace. For the evaluation, we used a
video conferencing software to show the interactive workspace to the
participants during the study trials which enabled them to easily interact
with CorpusVis ensuring equal study conditions. Based on the elicited _task
characteristics_ described in section 3, the participants carried out
different analysis questions provided by the study director. From the study
results, we gained useful insights on our work and its potential impact.

Study Participants – In sum, nine participants with different expertise levels
partook in the study. Due to the required musical knowledge to use the
analysis application, we focused our study on the target user group of domain
experts including musicologists and music teachers. Five participants had an
experience level of at least ten years including four university professors
and a music teacher, which we further refer to as subject matter experts
($\mathbf{SME}_{\mathbf{1}}$–$\mathbf{SME}_{\mathbf{5}}$). The remaining four
participants were Ph.D. students
($\mathbf{ST}_{\mathbf{1}}$–$\mathbf{ST}_{\mathbf{4}}$) who are professionally
involved with music on a daily basis but with less experience about music
analysis compared to the SMEs.

Methodology – We applied a combined _observation_ and _pair analytics_ study
[AKGF11] to investigate the usefulness of CorpusVis. In total, we performed
nine study trials (average duration of 95 minutes) that were each led by a
visual analytics expert (VAE) of our team. Each study session was structured
into four sections.
The first part started with an _introduction_ into the topic of sheet music
collection analysis to frame the context of our work. Each participant
completed a _demographic questionnaire_ to provide information about their
age, gender, level of education, and their level of expertise regarding music
analysis. The first phase ended with an interview about the participants’
_expectations_ regarding an analysis system intended to support the
investigation of sheet music collections.

Figure 7:  We used Intro.js [Meh16] to prepare a visual explanation of all
components of our prototype to ensure a consistent introduction of the
interactive analysis workspace for all study participants.

After the general introduction, each participant received a detailed,
textually guided _walkthrough_ regarding all components of the study prototype
(see Figure 7). Then, we introduced the five analysis tasks [T1]–[T5]from the
analyzed requirements to each participant. Besides the analysis tasks, the
introduction comprised an explanation of the relevant metadata features,
including composer, type, epoch, and the low-level features extracted from the
musical compositions visualized by the different components. In addition, the
participants were informed about the origin of the used dataset (MuseScore
[Mus11]) and its content addressing the overall amount of considered composers
and musical pieces. Phase 2 closed with a simple and unguided exploration
(similar to [T2]) of the visual interactive workspace, allowing the
participants to get acquainted with the application. This introductory task
contained a preselected set of ten different compositions and five features.
As a quick entry point, we prepared a use case _feature explanation_ that can
be directly loaded from the use case selector in the interface the provides
this preselection. We chose the pieces based on contrasting (extreme) feature
values such as low and high _Most Common Pitch Prevalence_ values (3% and
65%). The participants were asked to explore the _Feature Matrix_ (see Figure
3) to sort the selected sheets according to features of their interest. Then
they used the _sheet view_ (see Figure 6) to relate the feature values to the
underlying common music notation.
During the main part of the study, we asked the participants to execute the
analysis tasks, as described in the next section. We encouraged them to
verbalize their thoughts and interactions (_think-aloud concept_). After
completing the prescribed analysis tasks, we performed an interview to receive
feedback about the participants’ experience regarding the functionality and
usability of the prototype. We closed the study trial with a final
questionnaire containing nine questions answered via a five-point Likert scale
to identify the benefits and drawbacks of CorpusVis.

Study Tasks – During the pair analytics session, all participants analyzed
sheet music collections based on three use cases introduced in subsection 4.2.
[UC1] comprised 51 compositions from various composers of all epochs. After
getting acquainted with the metadata of the selected pieces, the participants
selected all pieces from a single epoch of choice [T1] and identified at least
two discriminating features [T3]. For this task each participant used the
feature matrix for identifying similarities between the compositions [T5]. In
addition, the participants could add additional pieces through the keyword
search to extend the selection [T1]. We encouraged each participant to freely
explore the visualizations with individual compositions of interest [T2].
A second task [UC2]addressed the comparison of _atonal_ with _tonal_ pieces
(Tonal versus Atonal), which used contemporary and classical works. Similarly,
the participants were asked to detect characteristic features values for both
groups by comparing the music sheets in the projection view and the feature
matrix [T3]. The last task addressed comparing two or more composers relevant
to the participant. For this, the _Composer Timeline_ helped filtering the
respective composers [T1], followed by a grouping of all pieces of single
composers [T4]. Then, we asked the participant to visually examine the feature
differences [T5]of the composers [T2].

###  5.2 Evaluation Results

We gathered feedback from the qualitative evaluation during the different
study parts that we grouped thematically in the following.

Participants’ Expectations – During the expectation interview, the
participants stated various aspects of an application intended to support the
musical analysis of sheet music corpora. $\mathbf{ST}_{\mathbf{1}}$ remarked
that he expected the application to group a musical repertoire into genres and
composers automatically. $\mathbf{SME}_{\mathbf{1}}$ expected the analysis
system to use all information available in musical scores to perform musical
analysis, which should present differences between musical styles, since
“classical is different from pop music”.$\mathbf{SME}_{\mathbf{2}}$ mentioned
that she expects an interface that allows her to provide information about her
interests. Then, the “system would automatically provide analysis results
without the need for manual analysis”.

Exploratory Analysis – To familiarize themselves with the compositions of
[UC1], the participants started that by investigating the metadata table to
see which composers are selected. Applying the color scheme of the composer
timeline to the projection view revealed the similarity of the compositions.
$\mathbf{SME}_{\mathbf{1}}$ mentioned the separate grouping of contemporary
Jazz and Pop pieces compared to Baroque and Classical compositions, which also
were separate groups but had some overlapping. Using the lasso to select the
yellow-colored (modern) pieces, she identified that compared to the rest,
these pieces have more note repetitions and high proportion of consonant
melodic intervals and less minor/major second intervals.
$\mathbf{ST}_{\mathbf{4}}$ checked the quality of a sheet file based on
features such as pitch variety. Typically complex compositions such as etudes
and symphonies have a high pitch variety. Thus, participants could delineate
custom arrangements from more original compositions.
While analyzing the configuration of [UC2], the participants were presented
with tonal compositions from composers such as Mozart and Beethoven, which
were compared with atonal pieces from Schönberg and Webern (see Figure 4). The
participants used the projection view to compare similarities and appreciated
the sorting functionality in the feature matrix, which helped them confirm
that atonal pieces use tritone intervals more often. They identified that the
feature _Number of Common Pitches_ for the atonal compositions is always zero,
which reflects that composers objective not to favor certain pitch classes.
Especially $\mathbf{SME}_{\mathbf{2}}$ appreciated the sheet view, which
allowed her to compare the values from the feature matrix to the common music
notation. The third use case started with aggregating all compositions from
the composers displayed in the composer timeline. The participants found that
_Changes Of Meter_ is the most discriminating feature revealing that
contemporary music seldomly changes the meter signature within a composition.

Comparative Analysis – The participants used the feature matrix to identify
similarities between single compositions or composer groups.
$\mathbf{SME}_{\mathbf{4}}$ and $\mathbf{SME}_{\mathbf{5}}$ found the melodic
features useful to investigate the melodic content. In addition,
$\mathbf{SME}_{\mathbf{3}}$ and $\mathbf{ST}_{\mathbf{1}}$ appreciated the
features regarding the intervals, which they considered to be particularly
helpful to identify dominant melodic aspects. $\mathbf{SME}_{\mathbf{1}}$
mentioned that the matrix helped confirm that contemporary music such as
“Imagine Dragons and Coldplay is more consonant and has more note repetitions,
but fewer stepwise motion” compared to Schönberg (Op. 11) or Webern (Op. 4).
Using CorpusVis, $\mathbf{SME}_{\mathbf{1}}$ learned that minuets from Bach
have much lower _Pitch Class Variety_ compared to his fugues.
$\mathbf{ST}_{\mathbf{1}}$ noted that based on the available pieces,
Schönberg’s work has more chromatic motion (25%) compared to Webern (20%).
$\mathbf{ST}_{\mathbf{2}}$ and $\mathbf{ST}_{\mathbf{3}}$ emphasized that the
pitch range and density information helped them investigate the pitch material
to better understand the complexity without even considering the sheet view.
$\mathbf{ST}_{\mathbf{3}}$ could identify that Beethoven uses 20 fewer pitches
compared to Liszt, who, as he stated, uses more chromatic melodic
progressions. During the investigation, the participants found different
matrix features to be relevant. They could easily understand the separation
into the three categories rhythm, pitch, and melody, which were of different
importance depending on the task. For instance, the time signature is a
salient feature that helps to determine certain composition types. The pitch
variety helped them detect complex compositions and differentiate tonal from
atonal compositions. The chromatic motion feature supported identifying
twelve-tone pieces.

Figure 8:  The final questionnaire comprised nine questions to assess the
usability and functionality of CorpusVis for analyzing sheet music
collections. Most questions achieved good ratings, emphasizing an overall high
satisfaction level.

Closing Questionnaire – All participants completed a final questionnaire
answering nine questions to assess the usability of CorpusVis. The questions
and the results that are based on a 5-point Likert scale ranging from
“Strongly Disagree” to “Strongly Agree” are displayed in Figure 8. The
questionnaire results reveal that our participants gained new insights (Q2)
about sheet music collections by enabling them to perform groupings of the
dataset to carry out comparison tasks based on epoch, composers, and
composition type information (Q7). Most participants agreed that CorpusVis is
suitable for the visual analysis of sheet music corpora (Q1, Q9). Q6 received
the lowest scores, which illustrates that the computationally extracted
features do not sufficiently address all users’ aspects, such as dynamics,
instrumentation, or harmonic progressions. Nevertheless, the scores of Q5 show
that the computed statistical features still enable the comparison even though
they may contain all aspects the analysts would be interested in. All but one
of the participants stated that they would recommend the interactive workspace
to a colleague or use it for their studies and research (Q4). For most
participants, the currently used dataset in CorpusVis contained the
compositions they wanted to add to their analysis (Q8). Overall, the
questionnaire results convey that the participants found potential for
analyzing sheet music collections and would continue using it in the future,
especially if it is improved based on this study’s outcomes.

##  6 Discussion

We identified benefits and drawbacks from the study which showed that
CorpusVis supports specific analysis tasks such as comparing composition types
or even the entire work of composers. Many participants questioned the quality
of the underlying MuseScore dataset which contains volunteered sheet music. To
address this issue, CorpusVis enables the upload of sheet music collections,
allowing musicologists to use their own, curated datasets if desired. In this
way, we mitigate the negative influence of datasets that suffer from poor
quality. Despite the underlying dataset’s limited quality, CorpusVis
demonstrates the positive impact that tailored visualizations have on the work
of musicologists.

Musicologists controversially discuss whether and where to place clear borders
between different epochs of music history. In our approach, we simplify the
timeline of musical history into four major epochs _baroque_ , _classic_ ,
_romantic_ , and _modern_. While we assign individual colors to each epoch,
study participants appreciated that we introduce smooth color gradients to
emphasize the transitions period. During the different musical periods, the
focus on specific characteristics of music has shifted. While CorpusVis
already includes several features, there may be other music styles that would
require to extend the application. For instance, starting from the second half
in the 20th century, new musical frontiers emerged, including serial,
electronic, minimal, spectral, and experimental music. These contemporary
music styles sometimes may not be available through transcribed common western
music notation. They often rely on auditory or physical features such as
timbre or spectral data. Therefore, such music compositions may only be
available through audio files but not symbolic music. CorpusVis would either
require an audio extension or a suitable symbolic notation that can be used to
extract features that represent the underlying information.

During the implementation and design of the presented interactive workspace,
we have encountered several challenges. A typical issue in this field is
_copyright_. As MuseScore users sometimes upload compositions from
contemporary musicians or artists who are still alive, these pieces may be
deleted from MuseScore again. To avoid legal issues caused by sheet music that
should not be visible in our system, we only include compositions from artists
who died more than 70 years ago that are still available on MuseScore.
Whenever sheets are deleted from MuseScore for any reason they are also no
longer available in our system.

During the development of this system we were in close contact with multiple
domain experts from musicology, allowing us to gather early feedback on
CorpusVis. We quickly identified challenges in this inter-disciplinary
collaboration, especially in creating visualizations that were understandable
by experts while allowing them to focus on aspects relevant to their intended
analysis. In particular, we found that the field of music analysis is
extremely broad, necessitating a flexible system with a myriad of different
options. However, adding additional functionality also complicated the
system’s design, in turn making it more difficult for experts to understand
and use. As a result, in this work we focus on specific tasks and data
characteristics to limit the scope.

###  6.1 Limitations

While the study results show that CorpusVis supports different music analysis
tasks, it also surfaced some limitations. For instance, some composers of the
20th or 21st century heavily use timbral or textural elements. These features
are often not encoded using the common Western music notation at all. As a
result, CorpusVis cannot be of help in analyzing work from such contemporary
composers. Future extensions of CorpusVis could integrate auditory features
derived from recordings rather than only music sheets to capture the
expression with which musicians played the compositions to close this gap. In
its current implementation, CorpusVis does not capture the instrumentation of
a composition. Several participants
($\mathbf{SME}_{\mathbf{2}}$–$\mathbf{SME}_{\mathbf{5}}$) stated that they
would like to filter all compositions of a composer that contain piano or
stringed instrument parts. Especially, identifying how the pitch range is
distributed between different instruments would provide useful semantic
information about the role of instruments and overall timbre of a piece.

While CorpusVis contains works from several composers, the visualization only
partly scale based on the available aggregations. The projection view and
feature matrix scale towards adding additional compositions from the same
composer using aggregation based on the composers as it uses the equal visual
space. Yet, an analyst may need to focus the analysis on a subset of musical
features if other characteristics should be considered as the cognitive
complexity increases with the number of features to be analyzed. Nevertheless,
CorpusVis already uses semantic groups for melodic, rhythmic, and pitch
features to enable users to focus on a subset. To improve this, CorpusVis
could be adapted to allow users to create custom feature groups to focus the
analysis and additional features that could be easily added if required. The
application’s performance depends on the library music21, which loads every
composition once at the beginning to prepare all features for visualization.

$\mathbf{SME}_{\mathbf{2}}$ mentioned that the classification of the pieces
into their types may be ambiguous, and that enabling analysts to re-classify
compositions could increase the analyst’s influence on a dataset’s quality
besides the option of uploading custom datasets. $\mathbf{ST}_{\mathbf{1}}$
and $\mathbf{SME}_{\mathbf{1}}$ remarked that the vertical combination of
notes (simultaneously played notes is referred to as harmony) is not covered
by the available features, since the employed jSymbolic features are limited
to pitch, melodic, and rhythm characteristics. Although CorpusVis already
offers a large array of selection and filtering mechanisms, some participants
would have liked additional options for filtering a dataset by key signatures,
voicing, and lyrics.

###  6.2 Take-Home Messages

From our work, we distill three primary take-home messages.

(1) Corpus-level music analysis is challenging due to the magnitude and
complexity of underlying features. Manual analysis of sheet music collections
is a tedious process requiring profound domain knowledge. CorpusVis supports
the analysis of extensive sheet music collections, which is not feasible when
done manually. While abstract visualizations open up new perspectives, they
should not be considered a replacement for traditional analysis methods but
rather an extension. Thus, integrating domain experts during the design phase
and the requirements analysis is crucial to address their specific needs. This
enables users to combine close- (sheet view) and distant-reading (projection,
features) and get a corpus-overview.

(2) Visualization dashboards can enable multiple perspectives on the data,
supporting scalable and multi-faceted analysis. Especially when musicologists
want to explore, understand, and interpret music sheet corpora, visual
analytics helps addressing the individual needs of analysts. Automatic
computational methods do not provide visualizations to users facilitating
interactive engagement with a sheet music collection to understand its
underlying characteristics better. We allow for various interconnected
analysis workflows by tailoring the components’ visual design to their
targeted task. The availability of multiple components covering different
characteristics of the dataset enables users to set their analysis focus on
those attributes they are interested in their analysis.

(3) Tailored interactions enable a task-driven analysis of corpora using
diverse metadata and statistical features. Interactive visualization allows
investigating sheet music from different perspectives, supporting various
aspects music analysts may be interested in. In particular, for analyzing
complex questions involving multiple meta-data and features, we rely on the
interactive inter-linking of visual components, enabling an expressive yet
visually uncomplicated analytics system. Enabling analysts to import their own
curated datasets facilitates their investigation and generation and
confirmation of hypotheses.

###  6.3 Future Work and Research Opportunities

We postulate that our work only represents a starting point for future
cooperation between visualization and musicology researchers, and we want to
emphasize that it is worth “risking the drift” for such interdisciplinary
collaborations [HEAB∗17]. So far, the classification of the compositions
metadata available in CorpusVis is fixed and cannot be adjusted by the
analyst. In the future, it may be worthwhile to enable users to change meta
information to increase the dataset quality. Extending the application through
labeling would further improve the analysts’ ability to capture and
externalize analysis results.

While we have considered the temporal aspects of music, we have neglected the
geographic information of where composers have lived and worked which has been
already studied by Khulusi et al. [KKFJ20]. It would be interesting to
investigate how the location influenced the style of specific composers. While
the low-level characteristics are just statistical measures, it would be
interesting to see how they correlate with the mood and emotions of listeners.
So far, we have not considered the dynamics, instrumentation, or timbre of the
instruments that the compositions are written for, which would be worthwhile
to add support for in the future. If pianists or guitarists are looking for
new pieces to play, it would be helpful to provide specific instrument filters
that help them find relevant information. Moreover, as the dataset’s quality
plays a primary role in analysis, it would be helpful to provide
classification means for analysts to identify low-quality pieces to exclude
unwanted items. Eventually, CorpusVis focuses the analysis on a high level. We
recently published MusicVis that enables analysts to investigate single
compositions at the sheet level [MFH∗22]. We aim at connecting both approaches
to allow analysts to filter relevant compositions at a higher level using
CorpusVis which can then be further analyzed at the sheet level.

##  7 Conclusion

In this paper, we have introduced CorpusVis, a web-based interactive workspace
for the visual analysis of sheet music collections. We discussed data and task
characteristics relevant to analyzing sheet music corpora. Multiple connected
components through linking and brushing support exploration, detection, and
comparison tasks. Based on initial expert feedback, we identified benefits and
challenges of our approach. In addition to three exemplary use cases that
demonstrate the analysis workflow to compare composers and types, we conducted
a pair of analytics study with nine participants. The qualitative study
results show that music analysts benefit from interactive visualization
addressing different use case scenarios such as comparing composers, epochs,
and composition types based on low-level features including pitch, rhythm, and
melody.

## References

  * [Aga17] Agafonkin V.:  A very fast 2D concave hull algorithm in JavaScript, 2017.  Retrieved March 9, 2022, from https://github.com/mapbox/concaveman.
  * [AKGF11] Arias-Hernández R., Kaastra L. T., Green T. M., Fisher B. D.:  Pair Analytics: Capturing Reasoning Processes in Collaborative Visual Analytics.  In _Conf. on Syst. Sc._ (2011), pp. 1–10.  doi:10.1109/HICSS.2011.339.
  * [ANC17] Armentano M. G., Noni W. A. D., Cardoso H. F.:  Genre classification of symbolic pieces of music.  _J. Intell. Inf. Syst. 48_ , 3 (2017), 579–599.  doi:10.1007/s10844-016-0430-7.
  * [BBBD20] Bludau M., Brüggemann V., Busch A., Dörk M.:  Reading Traces: Scalable Exploration in Elastic Visualizations of Cultural Heritage Data.  _Comput. Graph. Forum 39_ , 3 (2020), 77–87.
  * [BEAC∗18] Bradley A. J., El-Assady M., Coles K., Alexander E., Chen M., Collins C., Jänicke S., Wrisley D. J.:  Visualization and the Digital Humanities.  _IEEE Comput. Graph. and Appl. 38_ , 6 (2018), 26–38.
  * [BHF∗13] Bogdanov D., Haro M., Fuhrmann F., Xambó A., Gómez E., Herrera P.:  Semantic audio content-based music recommendation and visualization based on user preference examples.  _Inf. Process. Manag. 49_ , 1 (2013), 13–33.  doi:10.1016/j.ipm.2012.06.004.
  * [BMHC16] Bradley A. J., Mehta H., Hancock M., Collins C.:  Visualization, Digital Humanities, and the Problem of Instrumentalism.  _Proc. of the VIS4DH at IEEE VIS 2016_ (2016).
  * [CA10] Cuthbert M. S., Ariza C.:  Music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data.  In _Proc. of ISMIR_ (2010), Int. Soc. for Music Information Retrieval, pp. 637–642.
  * [Cam20] Cameron S.:  _What Is Plagiarism and What Is Musical Plagiarism?_ Springer, 2020, pp. 1–36.  doi:10.1007/978-3-030-42109-0_1.
  * [CB09] Chen Y., Butz A.:  MusicSim: Integrating Audio Analysis and User Feedback in an Interactive Music Browsing UI.  In _Proc. of the Int. Conf. on Intelligent User Interfaces_ (2009), ACM, pp. 429–434.
  * [CC08] Cox M. A., Cox T. F.:  Multidimensional Scaling.  In _Handbook of Data Visualization_. Springer, 2008, pp. 315–347.
  * [Cc21] Cuthbert M. S., cuthbertLab:  music21.features.jSymbolic, 2021.  Retrieved July 8, 2021, from https://web.mit.edu/music21/doc/moduleReference/moduleFeaturesJSymbolic.html.
  * [Cla21] ClassicMan:  Sheet Music Collection of MuseScore user ClassicMan, 2021.  Retrieved July 29, 2021, from https://musescore.com/user/19710/sheetmusic.
  * [CR16] Corrêa D. C., Rodrigues F. A.:  A survey on symbolic data-based music genre classification.  _Expert Syst. Appl. 60_ (2016), 190–210.  doi:10.1016/j.eswa.2016.04.008.
  * [FLTZ11] Fu Z., Lu G., Ting K. M., Zhang D.:  A Survey of Audio-Based Music Classification and Annotation.  _IEEE Trans. Multim. 13_ , 2 (2011), 303–319.  doi:10.1109/TMM.2010.2098858.
  * [GMB∗14] Gipp B., Meuschke N., Breitinger C., Pitman J., Nürnberger A.:  Web-based Demonstration of Semantic Similarity Detection Using Citation Pattern Visualization for a Cross Language Plagiarism Case.  In _Proc. of the Int. Conf. on Enterprise Information Systems_ (2014), vol. 2, SciTePress, pp. 677–683.  doi:10.5220/0004985406770683.
  * [GN19] Georges P., Nguyen N.:  Visualizing music similarity: clustering and mapping 500 classical music composers.  _Scientometrics 120_ , 3 (2019), 975–1003.  doi:10.1007/s11192-019-03166-0.
  * [Goo01] Good M.:  MusicXML for Notation and Analysis.  _The Virtual Score: Representation, Retrieval, Restoration 12_ , 113-124 (2001), 160.
  * [HEAB∗17] Hinrichs U., El-Assady M., Bradley A. J., Forlini S., Collins C.:  Risk the Drift! Stretching Disciplinary Boundaries through Critical Collaborations between the Humanities and Visualization.  In _Proc. of the IEEE VIS Workshop on Vis. for the Digital Humanities_ (2017).
  * [HK98] Hinneburg A., Keim D. A.:  An Efficient Approach to Clustering in Large Multimedia Databases with Noise.  In _Proc. of the Int. Conf. on Knowledge Discovery and Data Mining_ (1998), AAAI Press, pp. 58–65.  URL: http://www.aaai.org/Library/KDD/1998/kdd98-009.php.
  * [JFCS15] Jänicke S., Franzini G., Cheema M. F., Scheuermann G.:  On Close and Distant Reading in Digital Humanities: A Survey and Future Challenges.  In _Eurographics Conf. on Vis._ (2015), pp. 83–103.
  * [JFCS17] Jänicke S., Franzini G., Cheema M. F., Scheuermann G.:  Visual Text Analysis in Digital Humanities.  _Comput. Graph. Forum 36_ , 6 (2017), 226–250.  doi:10.1111/cgf.12873.
  * [JFS16] Jänicke S., Focht J., Scheuermann G.:  Interactive Visual Profiling of Musicians.  _IEEE Trans. on Vis. and Comp. Graph. 22_ , 1 (2016), 200–209.  doi:10.1109/TVCG.2015.2467620.
  * [Kir07] Kirschenbaum M. G.:  The remaking of reading: Data mining and the digital humanities.  In _The National Sci. Found. Symp. on Next Gen. of Data Mining and Cyber-Enabled Disc. for Innovation_ (2007), vol. 134.
  * [KKFJ20] Khulusi R., Kusnick J., Focht J., Jänicke S.:  musiXplora: Visual Analysis of a Musicological Encyclopedia.  In _Proc. of the Int. Joint Conf. on Comput. Vision, Imag. and Comput. Graph. Theory and Appl._ (2020), SciTePress, pp. 76–87.  doi:10.5220/0008977100760087.
  * [KKM∗20] Khulusi R., Kusnick J., Meinecke C., Gillmann C., Focht J., Jänicke S.:  A Survey on Visualizations for Musical Data.  _Comput. Graph. Forum_ (2020).  doi:10.1111/cgf.13905.
  * [KO07] Keim D. A., Oelke D.:  Literature Fingerprinting: A New Method for Visual Literary Analysis.  IEEE Comput. Soc., pp. 115–122.
  * [LME∗11] Liem C. C., Müller M., Eck D., Tzanetakis G., Hanjalic A.:  The Need for Music Information Retrieval with User-Centered and Multimodal Strategies.  In _Proc. of the Int. ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies_ (2011), ACM Press.  doi:10.1145/2072529.2072531.
  * [LWSF20] Lenain R., Weston J., Shivkumar A., Fristed E.:  Surfboard: Audio Feature Extraction for Modern Machine Learning.  In _Interspeech, Conf. of the Int. Speech Comm. Assoc._ (2020), ISCA, pp. 2917–2921.  doi:10.21437/Interspeech.2020-2879.
  * [MCF18] McKay C., Cumming J., Fujinaga I.:  JSYMBOLIC 2.2: Extracting features from symbolic music for use in musicological and MIR research.  In _Proc. of the Int. Soc. of Music Inf. Retrieval Conf._ (2018), Gómez E., Hu X., Humphrey E., Benetos E., (Eds.), pp. 348–354.  URL: http://ismir2018.ircam.fr/doc/pdfs/26_Paper.pdf.
  * [Meh16] Mehrabani A.:  Intro.js: Better Introductions for Websites and Features with a Step-by-Step Guide for Your Projects., 2016.  Retrieved November 29, 2021, from https://introjs.com/.
  * [MF09] McKay C., Fujinaga I.:  jMIR: Tools for Automatic Music Classification.  In _Proc. of the Int. Comp. Music Conf._ (2009), Mich. Publ.  URL: http://hdl.handle.net/2027/spo.bbp2372.2009.014.
  * [MFH∗22] Miller M., Fürst D., Hauptmann H., Keim D. A., El-Assady M.:  Augmenting Digital Sheet Music through Visual Analytics.  _Computer Graphics Forum_ (1 2022).  doi:10.1111/cgf.14436.
  * [MGD11] Müller M., Goto M., Dixon S.:  Multimodal Music Processing (Dagstuhl Seminar 11041).  _Dagstuhl Reports 1_ , 1 (2011), 68–101.
  * [MLCM16] McCurdy N., Lein J., Coles K., Meyer M. D.:  Poemage: Visualizing the Sonic Topology of a Poem.  _IEEE Trans. on Vis. and Comp. Graph. 22_ , 1 (2016), 439–448.  doi:10.1109/TVCG.2015.2467811.
  * [MSK∗19] Miller M., Schäfer H., Kraus M., Leman M., Keim D. A., El-Assady M.:  Framing Visual Musicology through Methodology Transfer.  _Proc. of the IEEE VIS Workshop on VIS4DH_ (2019).
  * [Mus11] MuseScore:  MuseScore.com, 2011.  Retrieved June 29, 2021, from https://musescore.com/.
  * [NN03] Nienhuys H.-W., Nieuwenhuizen J.:  LilyPond, a System for Automated Music Engraving.  In _Proc. of the Coll. on Musical Informatics_ (2003), vol. 1, Citeseer, pp. 167–171.
  * [OKT21] OKTAV:  OKTAV, 2021.  Retrieved July 8, 2021, from https://www.oktav.com/.
  * [Pam01] Pampalk E.:  _Islands of Music Analysis, Organization, and Visualization of Music Archives_.  CiteSeer, 2001.
  * [PMK10] Paulus J., Müller M., Klapuri A.:  State of the Art Report: Audio-Based Music Structure Analysis.  In _Proc. of the Int. Soc. of Music Inf. Retrieval Conf._ (2010), pp. 625–636.  URL: http://ismir2010.ismir.net/proceedings/ismir2010-107.pdf.
  * [PO12] Park J., Oh S.:  A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets.  _J. Inf. Sci. Eng. 28_ , 3 (2012), 587–600.  URL: http://www.iis.sinica.edu.tw/page/jise/2012/201205_10.html.
  * [Sap05] Sapp C. S.:  Online Database of Scores in the Humdrum File Format.  In _Proc. of the Int. Soc. of Music Inf. Retrieval Conf._ (2005), pp. 664–665.  URL: http://ismir2005.ismir.net/proceedings/3123.pdf.
  * [Sap15] Sapp C. S.:  Graphic to Symbolic Representations of Musical Notation.  In _Proc. of the Int. Conf. on Technologies for Music Notation and Representation_ (2015), pp. 124–132.
  * [Sch19] Schedl M.:  Deep Learning in Music Recommendation Systems.  _Frontiers in Applied Mathematics and Statistics 5_ (2019), 44.
  * [SF20] Shatri E., Fazekas G.:  Optical Music Recognition: State of the Art and Major Challenges.  In _Proc. of the Int. Conf. on Technologies for Music Notation and Representation_ (2020), Hamburg University for Music and Theater, pp. 175–184.
  * [Shn96] Shneiderman B.:  The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations.  In _Proc. of the IEEE Symp. on Visual Languages_ (1996), IEEE Computer Society, pp. 336–343.
  * [SSS∗14] Sacha D., Stoffel A., Stoffel F., Kwon B. C., Ellis G. P., Keim D. A.:  Knowledge Generation Model for Visual Analytics.  _IEEE Trans. on Vis. and Comp. Graph. 20_ , 12 (2014), 1604–1613.  doi:10.1109/TVCG.2014.2346481.
  * [STOH21] Sakurai K., Togo R., Ogawa T., Haseyama M.:  Music Playlist Generation Based on Graph Exploration Using Reinforcement Learning.  In _IEEE Global Conf. on Life Sciences and Technologies_ (2021), IEEE, pp. 53–54.  doi:10.1109/LifeTech52111.2021.9391870.
  * [ULMS10] Urbano J., Lloréns J., Morato J., Sánchez-Cuadrado S.:  Melodic Similarity through Shape Similarity.  In _Exploring Music Contents - Int. Symp., CMMR 2010_ (2010), Ystad S., Aramaki M., Kronland-Martinet R., Jensen K., (Eds.), vol. 6684 of _Lecture Notes in Computer Science_ , Springer, pp. 338–355.  doi:10.1007/978-3-642-23126-1\\_21.
  * [VT19] Verma H., Thickstun J.:  Convolutional composer classification.  In _Proc. of the Int. Soc. of Music Inf. Retrieval Conf._ (2019), Flexer A., Peeters G., Urbano J., Volk A., (Eds.), pp. 549–556.  URL: http://archives.ismir.net/ismir2019/paper/000066.pdf.
  * [WJVR16] Weihs C., Jannach D., Vatolkin I., Rudolph G.:  _Music Data Analysis: Foundations and Applications_.  Chapman and Hall/CRC Press, 2016.
  * [WMDM18] Weiß C., Mauch M., Dixon S., Müller M.:  Investigating style evolution of Western classical music: A computational approach.  _Musicae Scientiae 23_ (2018), 486–507.  doi:10.1177/1029864918757595.