# Characterizing Spontaneous Ideation Contest on Social Media: Case Study on
the Name Change of Facebook to Meta

Kunihiro Miyazaki  The University of Tokyo
Tokyo, Japan
<EMAIL_ADDRESS>Takayuki Uchiba  Sugakubunka
Tokyo, Japan
<EMAIL_ADDRESS>Haewoon Kwak, Jisun An  Singapore Management
University
Singapore
{haewoon<EMAIL_ADDRESS>
###### Abstract

Collecting good ideas is vital for organizations, especially companies, to
retain their competitiveness. Social media is gathering attention as a place
to extract ideas efficiently; however, the characteristics of ideas and the
posters of ideas on social media are underexamined. Thus, this study aims to
characterize spontaneous ideation contests among social media users by taking
an event of Facebook’s name change to Meta as a case study. As a dataset, we
comprehensively collect tweets containing new acronyms of Big Tech companies,
which we treat as an “idea” in this work. In the analysis, we especially focus
on the diversity of ideas, which would be the main reason for enlisting social
media for idea generation. As the main results, we discovered that social
media users offered a wider range of ideas than those in mainstream media. The
follow-follower network of the users suggested that the users’ position on the
network is related to the preferred ideas. Additionally, we discovered a link
between the amount of user interaction on social media and the diversity of
ideas. This study would promote the use of social media as a part of open
innovation and co-creation processes in the industry.

###### Index Terms:

social media, ideation contest, twitter, co-creation, open innovation

##  I Introduction

Creating, developing, or communicating new ideas is an essential process for
every organization. Since ideas are the source of creativity and innovation,
much attention has been paid in academia and industry to developing and
integrating them within and across organizations [boeddrich2004ideas]. In an
effort to collect good ideas, industries have been using the outsourcing of
idea generation [gatzweiler2017dark], especially in the form of _ideation
contest_. The ideation contest gathers participants and asks them for new
ideas (e.g., about products or solutions in business) using crowdsourcing
services and corporate platforms [bayus2013crowdsourcing, gamber2021effort].
Companies such as Dell [bayus2013crowdsourcing], IBM
[Majchrzak2009HarnessingTP], and Starbucks [muninger2019value] have succeeded
in running the ideation contest to gather ideas and implement many of them.

The great success in ideation contests has made companies explore the leverage
of social media [piller2012social]. Social media is a place where a wide
variety of ideas are posted and discussed on a daily basis [sajid2016social].
This big data would allow organizations to solicit ideas from a more expansive
space [afuah2012crowdsourcing] and gain ideas from the types of people who
would otherwise be excluded [jeppesen2010marginality]. Due to its potential,
existing research has proposed using social media for collecting ideas
[bharati2021idea]; however, there is still a lack of understanding about how
ideas are proposed on social media [cao2022visualizing, han2020computational].

Figure 1: Example of Idea tweet.

Therefore, this study characterizes idea generation behavior on social media.
In particular, we analyze the diversity of ideas, an important concept when it
comes to extracting ideas from social media. As data, we deal with the
acronyms of the American Big Tech companies, which have been known as FANG
(Facebook, Amazon, Netflix, and Google) or FAANG (FANG + Apple)
[FAANGSto80:online]. In October 2021, the name change of Facebook to Meta
provided a unique opportunity to study a user-driven ideation contest on
social media—“What should be a new acronym for Big Tech companies?” Since the
renaming of one of the world’s top 10 companies by market capitalization was a
phenomenal event, the discussion attracted many participants. In addition, an
ideation contest about a name is referred to as a “naming contest”
[koh2019adopting], which is categorized as the most straightforward form of
ideation contests [yang2009open]. Given these factors, Facebook’s name change
to Meta can serve as a solid exemplar of ideation contests on social media,
and the empirical analysis of this case would provide helpful insights into
the ideation behaviors of users. To characterize this spontaneous ideation
contest, we exhaustively collect tweets in the discussion on new acronyms of
Big Tech companies, which we consider as an “idea” in this work (an example is
shown in Figure 1), and examine them. After a first look at collected tweets,
we conducted analyses on the diversity of ideas, the relationship between the
users and ideas, and how user interaction affects the diversity of ideas.

From the collected tweets, we found 21 candidate acronyms for referring to new
Big Tech companies. In the analyses of the diversity of ideas, we found that
users on social media proposed a greater variety of ideas than those on
mainstream media. The user’s follow-follower network indicated that network
position is an important factor in affecting their ideation behavior.
Furthermore, we found that the interactions between users on social media are
associated with a greater diversity of ideas.

Other than the analyses of the diversity of ideas, we characterized them in
terms of the various metrics (e.g., shares, likes) and temporal dynamics.
Also, by looking at participants in detail, we found that the first person to
post ideas tends to have fewer followers, while those with more followers post
ideas secondarily or share the first posts. Finally, we conducted a regression
analysis and found that the early participants of the users with more
followers and negative posts were associated with more replies.

As far as we know, this work is the first case study to characterize
spontaneous ideation contests on social media, and we obtained helpful
insights for using social media for idea generation. Data and code will be
published when accepted and published. We believe this study would promote the
use of social media as a part of open innovation, co-creation processes, and
the management of big data in the industry.

##  II Related Works

###  II-A Ideation Contest using Crowdsourcing and Corporate Platforms

To establish an efficient idea generation method, industries tried to study
the mechanisms of idea generation in individuals [deo2021idea], compare groups
and individuals to determine which produce superior ideas [girotra2010idea],
and create an organizational system that facilitates idea generation from
employees [van2020idea]. However, with the increasing speed of information
sharing about products, competition in business is intensifying every year,
and thus the demand for collecting good ideas is increasing further
[bjork2009good]. In this context, companies are increasingly outsourcing idea
generation [gatzweiler2017dark]. Incorporating external knowledge, such as
open innovation [huizingh2011open] or co-creation [ind2013meanings], is
expected to accelerate internal innovation and reduce the risk of product
failure as it helps to involve potential customers in idea generation
[von2016free]. Among the means to outsource external ideas, ideation contest
using crowdsourcing services [bayus2013crowdsourcing] and corporate ideation
platforms [gamber2021effort], which are mainly financially incentivized
[ihl2012all], is one of the leading choices [piller2012social,
gatzweiler2017dark].

Existing research summarized the types of ideation contests in addition to
naming contests, such as graphic design and creative writing contests
[yang2009open]. As for the analysis on the corporate ideation platform, Bayus
analyzed their behavior and the quality of their ideas with respect to serial
ideators and ideators with only one idea [bayus2013crowdsourcing]. Hossain et
al. analyzed the growth process of the platform and the relationship between
the total number of ideas and the number of viable ideas
[hossain2015ideation]. In contrast, we analyze spontaneous ideation contests
on social media, which have the advantage of obtaining various participants’
opinions with less bias than corporate ideation platforms
[afuah2012crowdsourcing, jeppesen2010marginality].

###  II-B Ideation Contest using Social Media

The problem with ideation contests with crowdsourcing services and corporate
platforms is that the number of participants is limited and sometimes biased
[piller2012social]. In this context, the use of social media for ideation
contests is gaining attention to supplement existing methods
[piller2012social]. Social media is rich in various kinds of ideas, and there
is a strong demand for methods to use these ideas in corporate innovation. Due
to these properties, researchers have proposed the inclusion of social media
information in the business innovation process [piller2012social] and
demonstrated its benefits theoretically [bharati2021idea]. In fact, many
companies attempt to use social media to enhance their innovation process in
some forms [roberts2016finding], and idea creators such as fashion designers
have been using social media frequently in recent years to encounter new
inspiration [banks2014social, villioti2018social].

So far, the primary use of social media in the industry has been to look up
related words or feedback for a specific product as marketing research
[cole2015social], and there has been little study on the mechanism of how
social media users generate ideas. As the close studies, Carr et al. searched
for the related words of a product from social media, analyzed their volume,
and confirmed the insights obtained the result aligned with existing research
methods [carr2015social]. Han et al. held a design challenge in which
designers were asked to design a chair based on the chair’s related words and
feelings obtained through social media searches and found that social media
information is helpful for designers [han2020computational]. Ozcan et al.
tried to make a model to classify whether a post on social media contains
ideas or not [ozcan2021social]. Our study advances this line of research and
is the first empirical analysis of an ideation contest on social media.

Other papers examined hashtags and their evolution on social media with
respect to what words are used for certain events. Cunha et al. characterized
what hashtags are used in real-world events and showed the structure of
extreme usage of some hashtags [cunha2011analyzing]. Sato et al. visualized
how hashtags evolve over time in a network with edit distance between hashtags
as edges, and showed that hashtags are evolving from seeded hashtags in
various directions over time[sato2021exploration]. On the other hand, these
studies differ from ours in that they analyzed various hashtags describing a
specific event together, which included their impressions of the event, and
not for the purpose of ideation for naming purposes. Furthermore, these papers
focus solely on hashtags, and our study further analyzes the posters of ideas
and their interaction.

###  II-C Collective Creativity and Diversity of Ideas

Idea generation is an essential component of the production process and one of
the highest leverage points for a company [dahan2002product]. For an efficient
idea generation, the exchange of ideas by many individuals is considered
effective [janis1977decision] because it stimulates further associations and
ideas [paulus2014creativity] and provides exposure to more ideas than
generating ideas alone [csikszentmihalyi1997flow]. This kind of approach to
creative activity is called collective creativity [inakage2007collective].

Collective creativity is sometimes criticized as being ineffective for idea
generation. For example, some studies show that brainstorming does not
efficiently aid in the generation of ideas [west1996innovation]. However, the
underlying argument is that the evaluation of submitted ideas on the spot
prevents the submission of the next idea [leenders2003virtuality], and it has
been reported that this drawback can be overcome by, for example, exchanging
ideas in a text-based manner instead of gathering ideas in-person
[dennis1993computer]. Furthermore, unlike brainstorming, social media has the
potential to extract more diverse ideas because it is an environment for
casual idea generation, where submitted ideas do not necessarily get feedback
from others.

In collective creativity, the key to sound idea generation is the diversity of
participants and ideas [johnson2021neglect]. Innovation is said to come from
the combination of knowledge from different fields [parjanen2010collective,
parjanen2012brokerage]; thus, the stimulus supplied to the participants in the
exchange of ideas would increase with the diversity of the participants and
the ideas, which results in the generation of more fresh ideas. It was shown
in laboratory experiments [parjanen2012brokerage] that mediating distant
people (i.e., weak connections) in a social network can facilitate the
generation of new ideas. Considering that social media has a higher
contingency for far-distant people on social networks than in the physical
world and that it is easier to mediate distant users, social media is a
promising place for generating new ideas.

Despite the potential of social media to extract diverse ideas, there is still
a lack of understanding about how to extract them efficiently. By analyzing
ideation behaviors in social media, this study confirms the potential for
utilizing them for organizational open-innovation and explores how to utilize
them.

##  III Data Collection

###  III-A Finding Candidate Acronyms

We first collect tweets to identify candidate acronyms. We use the Twitter
Academic API [Enabling63:online] to obtain tweets containing “FANG” or
“FAANG”, which have been common names for Big Tech [FAANGSto80:online], from
28 Oct 2021 to 30 Nov 2021, one month window after Facebook’s name change. The
API is case-insensitive in search. As a result, we obtained 53,975 English
tweets without retweets (RTs). Then, after making all text lowercase, we
extracted the 150 most co-occurred words with “fang” or “faang” based on the
Jaccard coefficient [niwattanakul2013using], and conducted a manual
examination to extract a set of candidate acronyms of the new Big Tech
companies. As a result, we obtained 21 candidate ideas (shown in Table I).
Note that we omitted the candidates with a single tweet, which are MAMSANG,
TMAANG, MAANAT, MANATAM, and MATANTA, from Table I.

TABLE I: The candidate ideas and their statistics.

Idea | Tweet | RT | Like | Reply | QT | UU | Follower
---|---|---|---|---|---|---|---
MAANG | 885 | 0.62 | 5.70 | 0.97 | 0.19 | 850 | 93151.9
MANGA | 793 | 1.55 | 11.28 | 0.93 | 0.22 | 759 | 18760.1
MANG | 135 | 1.01 | 5.41 | 0.93 | 0.37 | 133 | 5901.5
MAMAA | 132 | 0.55 | 2.42 | 0.61 | 0.17 | 129 | 43564.4
MAANA | 115 | 0.60 | 6.92 | 0.56 | 0.17 | 111 | 5013.6
MAGMA | 89 | 1.06 | 12.99 | 1.73 | 0.33 | 83 | 2692.2
GAMMA | 87 | 0.34 | 5.68 | 0.98 | 0.09 | 84 | 3386.5
MAGA | 44 | 1.66 | 31.45 | 1.95 | 0.52 | 44 | 3668.2
TAANG | 23 | 0.04 | 1.65 | 0.39 | 0.04 | 21 | 1337.6
MAMA | 17 | 0.29 | 3.71 | 0.35 | 0.12 | 17 | 3224.3
MAGNA | 16 | 0.00 | 1.06 | 0.38 | 0.06 | 16 | 2097.8
MAAMA | 15 | 0.73 | 10.13 | 1.27 | 0.13 | 15 | 6493.3
MAMATA | 13 | 6.92 | 58.38 | 4.46 | 1.23 | 13 | 185656.1
MAMANG | 8 | 0.13 | 1.00 | 0.75 | 0.00 | 8 | 1768.1
AMAMA | 7 | 2.29 | 6.71 | 0.43 | 0.14 | 7 | 14150.4
MAANAM | 3 | 0.33 | 8.00 | 0.33 | 0.33 | 2 | 1662.0

###  III-B Collecting Tweets with Candidate Acronyms

We use the Twitter API again to search for tweets containing those 21
candidates over the same period. As a result, we retrieved 3.55 million
English tweets, excluding RTs. However, we found that many of the candidates
had homonyms (e.g., MAMA for a music award [MnetAsia52:online], MAGA for a
political slogan [MakeAmer35:online]). Therefore, to extract relevant tweets
for this study, we focus on the tweets that match either of the following two
conditions:

  * •

Condition 1: Tweets that contain (FANG, FAANG, or Meta) AND (any one or more
of the candidate words);

  * •

Condition 2: Tweets that contain (any one or more of the candidate words) AND
(all the company names in the corresponding candidate word).

An example of condition 2 is a tweet containing ‘MANG,’ AND ‘Meta,’ ‘Apple,’
‘Netflix,’ and ‘Google.’ As some companies could represent the same acronym
(e.g., ‘Amazon’ and ‘Apple’ could be used for ‘A’ in an acronym), we used all
the combinations of companies for searching the corresponding tweets using the
possible company names listed in $\S$3.3. We consider both lowercase and
uppercase for search. Then, we checked all the tweets manually and excluded 74
tweets that had nothing to do with the context of Big Tech, which were mainly
composed of unremoved MAMA tweets [MnetAsia52:online] and hashtag hijacking
[xanthopoulos2016hashtag]. As a result, we retrieved 2,219 tweets (Idea
tweets) excluding RTs, of which 1,766 were regular tweets that were neither
replies nor quote retweets (QTs). A drastic decrease in the number of tweets
is mainly due to the massive amount of tweets related to “MAMA,” 1.6 million
tweets in the original dataset. Our filtering criteria were found to be
effective in removing irrelevant tweets; even among the tweets containing
MAANG, we filtered out 38.8% of tweets that are unrelated to Big Tech (e.g.,
MAANG seems to be a casual word used often in Indonesian users). We then
collected how other users engaged with these Idea tweets: 3,735 replies, 850
QTs, and 1,622 RTs.

We also collected the followers of (1) the authors of Idea tweets and (2)
those who share (by RT or QT) the Idea tweets. The number of users is 3,912.
Out of the number of followers of each user, the max is approx. 24.2 mil., the
minimum is 1, the mean is 18,117.0, and the median is 366.5.

##  IV A First Look at the Spontaneous Ideation Contest

###  IV-A A List of Ideas and User Engagement Statistics

The ideas (i.e., candidate acronyms), the frequency of tweets containing them,
and various engagement metrics are shown in Table I, which include the number
of posts containing them (Tweet), retweet (RT), like, reply, quote retweets
(QT), unique users (UU), and the follower counts of the authors of Tweet
(Follower). The metrics except for Tweet and UU indicate the average values
per tweet.

All the candidates are acronyms of A (Apple, Amazon, or Alphabet), G (Google),
M (Microsoft or Meta), N (Netflix), S (Salesforce), and T (Tesla or Twitter).
Interestingly, compared to the original FAANG companies, several companies are
newly included in candidate acronyms, which might reflect the dynamics of the
IT industry.

In terms of the number of tweets, MAANG and MANGA are the two most popular
ideas. MAANG is a simple replacement of F (Facebook) with M (Meta) in FAANG.
MANGA is one of the famous Japanese words that refer to Japanese comics
[MangaWik51:online]. We suppose that the popularity of MANGA is due to the
serendipitous match with the well-known term. MANG, which is also a direct
replacement for FANG, is the third most popular idea, presumably because FAANG
has been more widely used than FANG since 2017 [FANG_FAANG]. In fact, FANG
(141 tweets) was much less common than FAANG (1,766 tweets).

For other engagement metrics, different ideas are more popular than MAANG and
MANGA: MAMA, MAGA, and MAMATA for RTs and likes—more users are engaged by
clicks; MAGMA, MAGA, and MAMATA for the reply and QT—more user are engaged by
leaving a tweet with their own text; MAMATA and AMAMA for the average number
of followers—more influencers are engaged. The number of Idea tweets and their
unique users is almost the same, indicating that each user tweeted an idea
once.

We note that 150 (8.77%) out of the 2,219 Idea tweets contained multiple
ideas, with the number of simultaneous ideas where the max is 7, the minimum
is 2, the mean is 2.12, and the median is 2. MAANG and MANGA appeared
simultaneously in the most Idea tweets (60 times), while the combination of
MAANA and MANG had the highest Jaccard coefficient (0.09).

###  IV-B Time Series of the Ideation Contest

To investigate how the ideation contest rises and converges, we examine the
time series of the idea tweets. Figure 2 presents the hourly volume of tweets
with the top 5 ideas and the hourly volume of RTs of those tweets. The period
is one week, starting from the day before Facebook changes its name to Meta.
We can see that all the ideas emerge at almost the same time, which is right
after the announcement of Facebook’s name change. When looking into the
dynamics on a finer scale, we found that the tweet of a news account that
first reported about Facebook’s name change was written at 6:07 PM on October
29, 2021 (UTC+0) [Fintwito76:online] as far as we could confirm. The ideation
contest started around 16 minutes after the name change announcement.

The number of ideas gradually increased after the announcement, with
convergence and excitement repeating several times. By October 31, three days
after the announcement, the contest is almost over. When looking at the volume
of tweets for each candidate over time, it is hard to judge which one is
dominant between MAANG and MANGA at the start, and the gap is gradually
opening up. Also, in terms of RTs, MANGA is overwhelmingly large initially,
but in line with MANGA’s gradual dominance in the number of tweets, MANGA has
been overtaken by MAANG in hourly RTs at the third peak.

(a) Tweet (b) Retweet

Figure 2: Hourly frequency of tweets and retweets of top five ideas.

##  V Diversity of ideas on social media

###  V-A Comparison with Mainstream Media

As we are especially interested in how diverse ideas are proposed on social
media, an analysis of the ideas mentioned in mainstream media and media-
related accounts can provide a good reference.

To this aim, we divide the users who post Idea tweets into media-related
accounts and the rest. For this purpose, we extracted 159 users whose account
names and bio information include the words “media,” “news,” “reporter,”
“journalist” among 2,081 users we collected. Then, we manually remove accounts
that are not related to the media from them. In addition, to mark prominent
media, e.g., CNBC and Reuters, we use Twitter’s “verified” information
associated with the account [TwitterV40:online]. As a result, we obtained 93
media-related (of which verified: 17) and 1,988 non-media-related users (of
which verified: 62).

The distribution of the ideas they mention is shown in Figure 3 and 4. These
figures show the counts of Idea tweets aggregated by each idea and converted
into percentages; here, the top five ideas and the aggregation of other ideas
(Others) are presented. Comparing media-related accounts with the rest (Figure
3), we find that media-related accounts are more heavily biased toward MAANG,
while there is a much less ratio for Others, meaning a biased distribution of
ideas. The $\chi^{2}$ tests showed a significant difference in the
distribution of ideas between media-related and non-media-related accounts
($p<0.001$). Note that we conduct the $\chi^{2}$ tests on count-based
aggregation, not on ratios [ChiSquar48:online], and the same for the later
analyses. Furthermore, in Figure 4, we see that both verified and non-verified
media-related accounts exhibit skewed distributions of ideas (in both cases,
the higher ratio for MAANG and the lower ratio for Others). This suggests that
a diversity of ideas are more likely to surface on social media than in the
mainstream media, regardless of their prominence.

Figure 3:  Relationship between (non-)media-related accounts and ideas. The
numbers (aligned with colors) indicate the percentage of ideas posted by each
group of users (the sum of row numbers is 100).  Figure 4:  Relationship
between (non-)media-related accounts and ideas. The accounts are further
classified by “verified” information. The numbers (aligned with colors)
indicate the percentage of ideas posted by each group of users (the sum of row
numbers is 100).

###  V-B Comparison with Influencers

We also show the heatmap for (non-)verified accounts in Figure 5.
Interestingly, there is no significant difference between the two groups
($p=0.33$ by the $\chi^{2}$ test). Rather, the distribution of ideas seems
flatter for verified accounts (the Gini coefficients: 0.332 for the verified
group and 0.425 for the non-verified group). Intuitively, it seems that users
with less presence (less influential) may enjoy more freedom to propose
diverse ideas. To validate this intuition quantitatively, we analyze the
relationship between the number of followers of accounts and the diversity of
ideas in this subsection.

Figure 5:  Relationship between (non-)verified accounts and ideas. The numbers
(aligned with colors) indicate the percentage of ideas posted by each group of
users (the sum of row numbers is 100).

First, we divided the users into four quadrants based on the number of
followers; the first quartile is 73, the median is 303, and the third quartile
is 1,322. As the number of followers in the third quartile, 1,322, is
relatively small to be called an influencer in the Twitterverse, we further
divide the users by the number of followers by 10k (151 users) in reference to
the discussion in [Howtouse34:online].

Figure 6 and 7 show the heatmaps by the number of followers. Interestingly, in
Figure 6, as the number of followers increases, the proportion of MAANG
decreases and the proportion of Others increases. In other words, the larger
number of followers the users have, the more diverse their ideas are. This
result is somewhat unexpected because it was expected from past studies that
users are less likely to propose ideas when they are evaluated by others
[leenders2003virtuality]. We assume they have a large number of followers
because they usually post interesting ideas.

On the other hand, Figure 7 shows that users in the range from the third
quartile (1,322) to 10k, or middle influencers, generate the most diverse
ideas, rather than users with more than 10k followers (although $p=0.10$ for
the $\chi^{2}$ test of ‘1,332 –10k’ between ‘$>=$10k’). Thus, there is the
possibility that the aforementioned study [leenders2003virtuality] is only
applicable to the users with more than 10k followers: when users have a high
probability of getting feedback on their ideas, it is more difficult to
propose unique ideas because of their publicity.

Figure 6:  Relationship between followers count (by four quadrants) and ideas.
The numbers (aligned with colors) indicate the percentage of ideas posted by
each group of users (the sum of row numbers is 100).  Figure 7:  Relationship
between followers count (by the third quartile (1,322) and 10k) and ideas. The
numbers (aligned with colors) indicate the percentage of ideas posted by each
group of users (the sum of row numbers is 100).

##  VI Network and Profiles of Participants

###  VI-A User Groups and Preferences of Ideas

Figure 8:  The largest connected component of the follower network of accounts
that proposed and shared ideas. Nodes represent users and edges represent
follow-follower relationships (undirected), and colors correspond to the
clusters by Louvain method [blondel2008fast]. The representative words of bio
information and TF-IDF scores in each cluster are also displayed with the
colors corresponding to the color of clusters in the network.

Now we move on to _who_ proposes and spreads the ideas. Existing research
suggests that there is a relationship between “distance of participants” and
diversity of ideas, with more distant people having different ideas
[parjanen2012brokerage]. On social media, which is a virtual environment,
distance is often measured from a social network constructed by the users’
relationships [backstrom2012four]. In this section, we study the relationship
between the distance of users on social media and their idea preferences.

To this aim, we collect and reconstruct the following network of users who
propose and spread ideas. We found that most accounts were connected, with the
ratio of its largest connected component as 79.7% of all users. On the other
hand, the other connected components consisted of less than three users. To
examine different groups joining in the idea contest, we identified clusters
of the largest connected component by using Louvain clustering
[blondel2008fast] (with the “resolution” parameter as 1), resulting in the
five largest clusters in the network. Figure 8 visualizes the network colored
by different clusters with their representative words. The shape of the
network is formed by the ForceAtlas2 algorithm [jacomy2014forceatlas2]. We
determined the representative words for each cluster by extracting the words
with the highest TF-IDF scores in the users’ bio information of each cluster.
When calculating TF-IDF scores, we aggregated the texts in user bios into one
document for each cluster (i.e., we made five documents for the calculation).
By manually looking at the representative words, we labeled each cluster as
Business, Engineer (web), Engineer (other), Entrepreneur, and Tech news. Each
cluster accounts for 28.4%, 19.2%, 22.2%, 15.9%, and 4.8% of the largest
connected component, respectively. Seemingly, the clusters are largely
composed of tech and business-related accounts. Their high engagement in the
ideation contest makes sense because 1) Big Tech companies are deeply related
to engineers in terms of their products and careers, and 2) since these
companies are significant investment targets, they drew interest from business
and investment accounts. We note that we manually examined that the Business
cluster has many individuals (not news organizations) who had the word “news”
in their bio, but the Tech news cluster has many news organizations.

Then, how do these clusters relate to the ideas? Figure 9 is a heatmap showing
the ratios of ideas for the above-mentioned top 5 groups and the aggregation
of users in the rest of the groups (Others). The pair-wise $\chi^{2}$ tests
showed a significant difference in the distribution of ideas in all pairs (all
$p<0.001$) except for the pair of ‘Engineer (other)’ and ‘Others’ ($p=0.49$)
and the pair of ‘Entrepreneur’ and ‘Others’ ($p=0.57$). In other words, this
result provides evidence that distant users on social media prefer different
ideas, which aligns with the previous research [backstrom2012four].

Figure 9:  Relationship between user clusters and ideas. The numbers (aligned
with colors) indicate the percentage of ideas posted by each group of users
(the sum of row numbers is 100). The rows are sorted by the percentage for
MAANG.

This result can also come from the factor that people with different jobs
prefer different ideas. Given that people with similar attributes tend to
cluster close on social networks [bisgin2012study], it is natural to assume a
relationship between these occupations and distance. Also, note that the
present analysis does not determine which affects the diversity of ideas more,
distance or attributes. Nor does the result indicate a causal relationship:
the result does not indicate that distancing users or changing attributes of
users will necessarily result in a change in their preferred ideas. However,
it is possible that distance leads to a preference for different ideas, even
if the attributes are similar. This is because the present network shows that
the engineers in different locations significantly differed in the preferred
ideas (MAANG and MANGA), although their type of engineers is different.
Nonetheless, we leave such a comparison of distances and attributes and the
causal analysis to future research.

###  VI-B Who initiates the ideation contest?

When using social media for idea generation in practice, it is important to
know who to track. Therefore, understanding the roles and attributes of those
who create and spread ideas could help extract ideas from social media more
efficiently.

To distinguish those who initiate the ideas, we utilize the following network
and examine whether the authors of Idea tweets had seen other Idea tweets
before they posted theirs. Although Twitter API does not provide whether or
not a user actually ‘sees’ a certain tweet, we approximate it based on the
posted time of the Idea tweet. In other words, we assume that a user saw an
Idea tweet if any of the followee had posted or shared an Idea tweet before a
user’s Idea tweet. Then, we label the authors of Idea tweets as _Primary_ if
they have not seen any idea tweets beforehand, and _Secondary_ if their
followees have already tweeted or shared the ideas. Among the 2,081 unique
authors of Idea tweets (i.e., Primary and Secondary), the number of Primary
users was 974 (46.8%), almost half of the authors. Also, we label the users
who just shared ideas without adding new ideas as _Spreader_ (1,831 users).

When we look at the follower counts, the median follower counts of Primary,
Secondary, and Spreader were 164, 540, and 446, respectively (shown in Figure
10). Secondary has the largest number of followers on average, followed by
Spreader and Primary, and the differences are all significant ($p<$0.0001 with
Mann-Whitney $U$ test with Bonferroni correction). In other words, it can be
seen that the ideas of Primary with fewer followers are picked up by Secondary
and Spreader with more followers. As for Spreader, it has been said that
widely spread information is associated with influencers with a large number
of followers [bakshy2011everyone]. On the other hand, in the comparison
between Primary and Secondary, it is interesting to see the former has fewer
followers, indicating the difficulty in tracking the users who tend to
initiate ideation contests. Note that the mean (max) follower counts of
Primary, Secondary, and Spreader were 5,652 (1.9 mil.), 53,609.4 (24.2 mil.),
and 3288.8 (0.8 mil.), respectively.

Figure 10: Boxplot of follower counts for accounts of Primary, Secondary, and
Spreader. The stars indicate the significant differences with Mann-Whitney
test with Bonferroni correction (****: $p<$0.0001).

Figure 11 also shows the number of followees of these three groups. The median
followee counts of Primary, Secondary, and Spreader were 210, 643, and 602,
respectively Only the Primary group has a significantly lower number of
followees, indicating that they tend to lead in proposing original ideas with
fewer information sources or that they tend to ‘come up with’ ideas by
themselves. Note that the mean (max) followee counts of Primary, Secondary,
and Spreader were 406.3 (9,979), 1,370.8 (47,445), and 1414.4 (105,504),
respectively.

Figure 11: Boxplot of followee counts for accounts of Primary, Secondary, and
Spreader. The stars indicate the significant differences with the Mann-Whitney
test with Bonferroni correction (****: $p<$0.0001, ns: $p>0.05$).

##  VII Idea Exchanges among the Participants

Ideation contests on social media can be the place of ‘idea exchanges’ through
discussions. In particular, we examine how interactions between users on
social media lead to a diversity of ideas. On social media, users can see each
other’s tweets and can exchange ideas by sending replies to others. It is said
that participant interaction in idea generation promotes diversity of ideas
[janis1977decision], and we examine whether this phenomenon can be observed in
social media interactions.

Here, we analyze how ideas are exchanged and how discussions help to generate
ideas on social media by constructing reply trees. To do so, we collect all
the replies to the 1,766 regular tweets by using the conversation_id in the
Twitter API, as noted in Section $\S$3.1. As a result, we obtained 2,096
replies. The number of regular tweets that received replies is 403 (22.8%) out
of 1,766. The number of direct replies to the regular tweets varies, ranging
from 1 at the minimum and median, 3.93 at the mean, to 64 at the maximum,
indicating that some regular tweets received an extremely large number of
replies. Then, we connect the collected replies to form reply trees. Starting
from the 403 regular tweets that received replies, treating a connected chain
to the end of replies as one pattern, we form a total of 1,585 patterns of
reply trees, including branches from the same crotch. The maximum length of a
reply tree was 10, the minimum 2, the mean 2.40, and the median 2.

We first investigate how new ideas are introduced in those reply trees. We
trace the reply tree from the root and count how many unique ideas existed in
each reply in the tree. As a result, among the 1,585 reply trees, we found
that 360 trees accumulate new ideas at some point in replies. Of these, new
ideas are added in the first reply in 327 (90.8%) trees. At the maximum, a new
idea was added at a reply depth of 4 for the first time. Figure 12 summarizes
the relationship between the depth of replies and the number of ideas.

Figure 12: The relationship between the depth of reply trees and cumulative
count of unique ideas in the tree. Size of points indicates the number of
tweets which are lastly added new ideas at a depth of reply (x-axis) and the
number of ideas (y-axis). The line indicates the pattern of accumulation of
ideas through reply trees, and its width indicates the number of each pattern
of reply trees.

Next, we analyze whether there is an association between the replies and the
diversity of ideas. To this aim, we tabulated the types of ideas within each
chain pattern. The types of reply trees here were Idea tweets that received no
replies (No_reply_all), Idea tweets that received replies and all of their
replies (With_reply_all), only Idea tweets that received replies
(With_reply_root), and only the replies to the Idea tweets
(With_reply_branch). For each, we aggregated the ideas within each group of
tweets and created a heatmap of the proportions (Figure 13). The pair-wise
$\chi^{2}$ tests for all these tweet groups all showed significant differences
($p<0.001$). Concerning the presence of replies (No_reply_all v.s.
With_reply_all), the distribution of With_reply_all is flatter, and its ratio
for Others is larger than No_reply_all. In other words, a greater variety of
ideas was generated when there were replies. Furthermore, when we look in-
depth at With_reply_all, we find that MAANG is dominated in the first tweet
(With_reply_root), while MANGA is dominated in its replies
(With_reply_branch), and its ratio of Others is also larger. In other words,
it is conceivable that unordinary ideas are added to the original tweets via
replies.

Figure 13:  Relationship between type of reply trees and ideas. The numbers
(aligned with colors) indicate the percentage of ideas posted by each group of
users (the sum of row numbers is 100).

Since the diversity of ideas seems to be associated with the interaction of
users, it would be helpful to know in which conditions the interaction tends
to occur to find them efficiently or spark them deliberately. Thus, we
performed a logistic multiple regression analysis to determine which idea
tweets were more likely to receive a reply. Here, we constructed a
classification model based on whether or not a reply was received at least
once (1 if received, 0 otherwise). We considered the following features:
Engineer: whether the author is an engineer or not. In this case, we used the
binary of whether the author belongs to the two clusters of engineers
introduced in Section $\S$6\. Follower count: we used the average number of
followers of the author. Bio length: how long it describes itself or how
anonymous it is. Verified: whether or not they have public recognition on
Twitter [TwitterV40:online]. Time: how long (in seconds) it has been since
Facebook changed its name. Negative and positive: How negative or positive the
text is. We used the roBERTa-based pre-trained model [barbieri2020unified].
This model outputs three classes, including neutral, but neutral was excluded
from the regression model. Idea count: how many ideas an Idea tweet contains.
Ideas: whether a tweet includes the top 5 ideas.

Table II shows the results of the logistic regression (number of fake tweets
is N = 1,766). The p-values are computed using two-tailed z-tests. All
variance inflation factors (VIFs) [o2007caution] are less than three,
indicating that multicollinearity is negligible. Pseudo $R^{2}$ value is 0.21.
We conduct log transform to some features when needed, indicated as (log) in
Table II. The results show that engineers with more followers and more bio
length are more likely to receive replies. We also found that the faster one
posts an Idea tweet, the more likely it gets replies. Interestingly, negative
tweets (e.g., “MAANG sounds a lot less cool than FAANG”) are more likely to
receive replies, presumably because negative emotion tends to provoke
discussions than positive tweets.

TABLE II: Results of the regression analysis predicting whether or not an Idea
tweet gets at least one reply.

|  Coefficient |  Mean |  Std
---|---|---|---
const |  -2.01 |  - |  -
Engineer |  0.81∗∗∗ |  0.22 |  0.42
Follower count (log) |  0.41∗∗∗ |  5.81 |  2.34
Bio length (log) |  0.26∗∗ |  4.13 |  1.26
Verified |  -0.11 |  0.04 |  0.19
Time (log) |  -0.26∗∗∗ |  11.16 |  1.37
negative |  1.21∗∗∗ |  0.17 |  0.20
positive |  -0.29 |  0.22 |  0.24
Idea count |  -0.39 |  1.08 |  0.33
MAANG |  -0.07 |  0.45 |  0.50
MANGA |  0.22 |  0.33 |  0.47
MANG |  0.43 |  0.06 |  0.24
MAMAA |  -0.57 |  0.06 |  0.23
MAANA |  -0.78∗ |  0.05 |  0.21
Significance codes: ***$p<0.001$, **$p<0.01$, *$p<0.05$

##  VIII Discussion and Conclusion

In this study, we used Facebook’s name change event as a case study to
characterize ideation behavior on social media. We extracted a comprehensive
set of acronyms to refer to new Big Tech companies and analyzed their dynamics
and diversity. The main conclusions are as follows: 1) more diverse ideas are
appeared on social media than on mainstream media, 2) the user’s topological
position on a social network is important for the preference of ideas, and 3)
social interactions can spark diversity of ideas. We also characterize the
popularity (e.g., shares and likes) of each idea, those who take different
roles in generating ideas, and conditions that idea tweets would attract more
replies. Our study confirms the potential of social media for idea generation
and provides a guideline on how to spark diverse ideas.

###  VIII-A Limitation and Future Works

There are some limitations in our work. First, we could not track the ideas of
private accounts or deleted accounts. However, given that the new name of Tech
companies is not about a sensitive issue, we believe that their effect would
be marginal. Second, as we examined the single case of the idea contest, more
studies are required for the generality of our findings. Third, while we
examined idea generation behavior on social media, we did not investigate the
quality of ideas. As [acar2019crowdsourcing] has already pointed out, the
ideas gathered by ideation contests are a mixed bag and methods for extracting
good ideas from them are in demand. Fourth, our analysis of the relationship
between replies and diversity of ideas does not directly examine causality.
Future research could study causal inferences from observational data and
field studies involving interventions [mosleh2022field].

We also note that we loosely define an ideation contest as a situation in
which many people are exchanging ideas. In particular, we did not decide on
the winner based on the ideas submitted. By contrast, in practice, companies
may rank the ideas based on the number of RTs or likes, which would eventually
become a form of contest. Also, we have used social media data to characterize
and evaluate ideas during the ideation contest, but their impact has not been
studied. For example, MAANG and MANGA, which were the most proposed, did not
necessarily replace the original FAANG. Nonetheless, we believe this study
presents meaningful results in terms of characterizing the idea generation
mechanism and the competition between new ideas.

As for the methods for discovering Idea words, a more sophisticated one would
be useful for future research. In this study, we decided to set a strict
search query that considers the context of the Big Tech name change.
Furthermore, we do not distinguish between suggesting and mentioning an idea.
We considered tweeting using the new Big Tech name to be in itself the same as
suggesting it. However, this can only be valid for this case study. For other
events, there may be a need to extract tweets that are proposing ideas only
[ozcan2021social].

As for the volume of data, this study’s ideation competition had about 2,000
participants, which is a relatively small size of data compared to common
Twitter studies using ‘Big’ data. This is the result of strict filtering
schemes applied to our initial 3.55 million tweets containing candidate terms,
and we got excellent precision (97%: 74 / (2219+74)) from detected idea
tweets. Nonetheless, considering that it is difficult and expensive to recruit
2,000 people via a corporate platform or crowdsourcing, this study exhibited
the usefulness of social media in recruiting such a large number of
participants.


# A neural network classifier for electron identification on the DAMPE
experiment

D.Droz,11footnotetext: Corresponding author. A.Tykhonov  X.Wu  F.Alemanno
G.Ambrosi  E.Catanzani  M.Di Santo  D.Kyratzis  S.Zimmer

###### Abstract

The Dark Matter Particle Explorer (DAMPE) is a space-borne particle detector
and cosmic ray observatory in operation since 2015, designed to probe
electrons and gamma rays from a few GeV to 10 TeV in energy, as well as cosmic
protons and nuclei up to 100 TeV. Among the main scientific objectives is the
precise measurement of the cosmic electron+positron flux, which, due to the
very large proton background in orbit, requires a powerful particle
identification method. In the past decade, the field of machine learning has
provided us the needed tools. This paper presents a neural network based
approach to cosmic electron identification and proton rejection and showcases
its performance based on simulated Monte Carlo data. The neural network
reaches significantly lower background than the classical, cut-based method
for the same detection efficiency, especially at at the highest energies
probed by the detector. Good agreement between simulation and real data is
demonstrated.

##  1 Introduction

The DArk Matter Particle Explorer (DAMPE - also known as Wukong in China) is a
satellite-based cosmic ray observatory and gamma ray telescope [1]. It can
measure cosmic electrons up to an energy of 10 TeV and protons and heavier
nuclei up to a hundred TeV. It is constituted of four subdetectors, from top
to bottom: a plastic scintillator (PSD) for absolute charge measurement; a
silicon-tungsten tracker-converter (STK) for precise direction measurement and
for enabling photon pair production; a Bismuth Germanium Oxide imaging
calorimeter (BGO) of about 32 radiation lengths, made of 308 hodoscopically
arranged bars in 14 layers and used for energy measurement, particle
identification, and trigger [2]; and a neutron detector (NUD) for improving
the identification of hadronic showers [1]. The DAMPE satellite was launched
into a 500-km Sun-synchronous orbit on December 2015, and has been in stable
operations since then [3].

Figure 1: Layout of the DAMPE detector system. Figure from [4]

Among the main scientific objectives of DAMPE is the study of cosmic ray
electrons plus positrons (CREs) in the TeV range. Precise measurement of the
spectrum of CRE is crucial for understanding the mechanisms of cosmic ray
production and propagation [5, 6, 7]. CREs represent a valuable probe of the
most energetic process in the nearby Galaxy, such as supernovae explosions [8]
and may enable the observation of phenomena such as dark matter decay or
annihilation [9, 10]. The CRE spectrum was first measured by DAMPE in 2017 up
to an energy of 4.6 TeV, with the direct detection of a spectral break in the
TeV region [11]. This result was obtained using a classical, cut-based
technique for electron/proton discrimination. Said method, while powerful
enough to reject proton background up to a few TeV with acceptable
uncertainty, could not fully exploit the particle identification capability of
the detector. After more than four years of operation and thus much higher
accumulated data statistics, DAMPE is capable of measuring the CRE spectrum up
to an unprecedented energy of 10 TeV with an instrument energy resolution
better than 1.2% [11]. However the standard identification technique does not
have sufficient electron/proton discrimination power to be used at such
extreme energies. A new method is therefore required, and we propose one
coming from the ever-growing toolbox of data sciences and artificial
intelligence.

In this paper we propose to use neural networks for the problem of electron
identification on the DAMPE experiment. In section 2 we develop the electron
identification technique based on a neural network classifier. In section 3 we
demonstrate the performances of the developed classifier on Monte-Carlo
simulations and compare it with the standard cut-based technique. In section 4
we perform a validation of the developed classifier with the 4-years DAMPE
orbit data and demonstrate that the new method allows a substantial
enhancement of the electron identification, enabling the first CREs spectrum
measurement with DAMPE in the 10 TeV energy domain. Finally, the work and the
results are summarised in section 5.

##  2 Electron identification

Measuring cosmic electrons in orbit requires first to identify them among the
many sources of background [12]. Helium and heavier nuclei can be rejected by
measuring the absolute electric charge, which is done in DAMPE using the
signal deposited inside the plastic scintillator (PSD). The PSD cut is
complemented by a charge measurement in the silicon tracker (STK) allowing to
reject heavy nuclei outside of the PSD fiducial region. Gamma rays are another
source of background, however their flux is orders of magnitude weaker than
the CRE flux especially at energies higher than several hundred GeV. They are
rejected thanks to their electric charge of zero: a gamma ray penetrating
through the PSD will not leave any signal as opposed to other particle
species.

Protons have the same absolute electric charge as electrons, meaning they
cannot be rejected with these methods. Moreover, protons are the most abundant
comic ray species, with a flux orders of magnitude higher than that of
electrons, drowning the signal in a high rate of background events. The
exploitable difference between these two particles lies in the physical
processes when interacting with matter: protons produce a wide and deep
hadronic shower, while electrons produce narrower electromagnetic showers,
resulting in different signal topologies in the BGO calorimeter. Figure 2
shows a representation of electron and proton interactions within DAMPE: the
simulated electron deposits most of its energy in the first half of the
calorimeter whereas the proton shower develops both deeper and broader.

Figure 2: Display of simulated cosmic ray events inside DAMPE. The particle
comes from the top and crosses in succession the PSD, STK and BGO detectors.
Left and right panels show the X-oriented and Y-oriented views, respectively.
Black solid lines are STK tracks, green dashed line is the BGO reconstructed
trajectory. The stars in STK represent clusters of hits. Colour bars show the
energy deposited in PSD and BGO. _Top:_ Monte Carlo electron with a measured
energy of 353 GeV. _Bottom:_ Monte Carlo proton with a reconstructed energy of
340 GeV.

Classically, one can build observables that quantify the shower shape inside
the calorimeter, and use them to reject protons. The first DAMPE cosmic
electron spectrum measurement [11] is based on a single observable named
$\zeta$, quantifying the correlation between shower length and shower width
[13]. It is defined as:

$\zeta=\frac{\left(\sum_{i}^{\text{layers}}(\texttt{RMS})_{i}\right)^{4}\cdot\mathcal{F}_{\texttt{Last}}}{8\cdot
10^{6}}$ (2.1)

where RMS is the energy-weighted root-mean-square of hit positions in the
calorimeter (shower width) and $\mathcal{F}_{\texttt{Last}}$ is the fraction
of energy deposited in the last calorimeter layer with non-zero deposition
(shower depth). This definition can be compared to the event displays in
figure 2: the narrow electron shower naturally results in a $\zeta$ value much
lower than for protons, as exhibited in their respective distributions in
figure 3. Similar variables are commonly used for electron identification in
calorimetric experiments.

While $\zeta$ proved powerful, it does not use the entirety of information
available in the detector, including the possibility to exploit the strong
correlations between topological variables used to describe the shower
development in the imaging calorimeter. Furthermore its rejection power is
limited above the TeV range where the topological development of hadronic and
electromagnetic showers in the detector is less pronounced. A more powerful
method is therefore required, and is presented in this paper.

Figure 3: Distribution and Monte Carlo/data comparison of the classical
$\zeta$ variable in the energy range 500 GeV to 1 TeV, out of a cleaned-up
data to remove ions and require good shower containment (section 2.2).

###  2.1 A neural network classifier

Deep learning and deep neural networks [14, 15] are at the forefront of
current day developments in data sciences and artificial intelligence, often
representing the state-of-the-art solution to a wide variety of problems. They
have in particular found their way into both high energy physics [16, 17],
where neural networks can exploit the large size and high dimensionality of
the data, and into astrophysics [18, 19] where their pattern recognition power
allows for extraction of additional information from telescope images.
However, neural networks have only been seldom used in cosmic ray physics, and
never exploited for CRE direct detection experiments at multi-TeV energies.

The classifier we propose to use is based on a regular neural network composed
of a stack of densely connected layers (see below). Such algorithms are
sometimes named as _multilayer perceptron_ , _feedforward neural network_ , or
simply _artificial neural network_. Other techniques were also studied in the
DAMPE collaboration prior to this work: convolutional neural networks (CNN),
suited for pattern recognition and image identification, showed promising
performances. While CNN demonstrated potential for further improvement with
respect to the feedforward network, we opted for latter technique due to the
better understood systematics of this type of networks, which is reflected, in
particular, in data to Monte-Carlo agreement of a classifier score
distribution [20]. Another technique studied by the collaboration are boosted
decision trees, commonly used in high energy physics. They showed some
improvement over the classical method, though optimised for the lower energy
domain, around 10 to 100 GeV [21].

An artificial neural network is a stack of densely connected layers of so-
called neurons. A neuron is a mathematical unit that applies a non-linear
function to the linear combination of its inputs. The function output is then
used as input by all the neurons in the next layer, in the case of fully
connected networks. Mathematically, if a neuron receives as input a set
$\left\\{X_{i}\right\\}$, then its output $y$ is:

$\left\\{X_{i}\right\\}~{}~{}\longrightarrow~{}~{}y=f\left(\sum_{i}w_{i}X_{i}\right)+b$
(2.2)

where $f$ is the non-linear activation function, $w_{i}$ the weights and $b$
the bias. The activation function $f$, as well as the number of neurons and
layers, are characteristics of the model which are decided by the human
programmer (section 2.2). The values of $w_{i}$ and $b$ are set by the machine
during the training procedure: the network is exposed to a set of labelled
data (training data) where each observation/event is associated to a class
(e.g. signal/background), and tries to minimise a given error metric on the
set. The minimisation usually follows a so-called gradient descent or one of
its flavours. In the case of a classification problem, such as discrimination
between protons and electrons, the canonical error metric is the cross-entropy
[15].

An artificial neural network takes as input a one-dimensional set of variables
$\left\\{X_{i}\right\\}$. In a binary classification, it outputs a value
between 0 and 1 which can be interpreted as the probability for an electron
(signal) to produce $\left\\{X_{i}\right\\}$. This value is obtained as the
output of a sigmoid or sigmoid-like function as the activation of the very
last layer.

In-depth reviews of neural networks can be found in references [14, 15].

###  2.2 Data and model optimisation

The data available for training and testing the model are Monte Carlo (MC)
simulated events, created using the Geant4 package [22] interfaced to the
DAMPE software [23] to emulate the detector response to the various cosmic
rays species in orbit. The training data was prepared with a set of cleaning
cuts to replicate the analysis chain applied on real data: this includes
shower containment and fiducial volume selection criteria, as well as cuts to
remove gamma rays, Helium ions, and heavier nuclei [11]. In this way, the only
remaining background for electron identification are protons. The cut flow is
completed by selecting only events with $\zeta<100$, where $\zeta$ is the
classical classifier (equation 2.1). This conservative criterion eliminates
events with obvious proton-like topology while retaining 99.95% of electrons.
The motivation behind this final cut is to get rid of easily identified
events, such that the neural network can focus on more complex ones. This
preselection yields a neural network with higher discrimination power.

Out of this cleaned-up data, we built a training set of 140’000 events,
balanced 50/50 between electrons and protons. A further 120’000 events are
kept for model validation. The data is normalised by dividing each input
variable by its maximum value through the dataset, therefore scaling them to
the $[0;1]$ range.

We selected the input variables based on several criteria. First, the features
had to be descriptive enough of the shower topology to provide as much
information as possible to the neural network. Following published results on
deep learning for particle physics [16], we opted mostly for low-level
variables to maximise said information, but we added in high-level variables
as well such as the $\zeta$ classifier which we know provides an already
powerful proton rejection. The idea being to give the network our best guess
as expert, and then the low-level ingredients to improve on it. Adding $\zeta$
significantly improved the sub-TeV performances. Finally, we required the
input variables to be well simulated. We observed that depending on the set of
features, the network could produce slightly different results between MC
simulations and real data. We therefore followed an extensive campaign of
empirical testing to find and remove the quantities that resulted in such
differences, and select a set of variables that produces a satisfactory
agreement between MC and data (section 4). As a result, the features selected
include the energy deposited and its RMS distribution in 12 out of 14 layers
of the BGO calorimeter (excluding the top two), the reconstructed energy, the
angle of the trajectory, the energy deposited in one Moliere radius of a STK
track (STK cluster energy), and the classical $\zeta$ classifier. The physical
motivation behind them is that they provide direct information on the shower
topology and the characteristics of the event (incidence and energy). The STK
variable accounts for showers that start before the calorimeter. The first two
layers of BGO were excluded based on an observed offset of resulting MC proton
distribution with respect to real data (section 4). Figures 4 and 5 show the
distributions and MC/data comparison of these variables for both particle
species, excepting the reconstructed energy (power law), angle (BGO
acceptance) and $\zeta$ (figure 3). These events were fully cleaned up by the
aforementioned chain of cuts and filters. For the sake of visual comparison
only, they have been separated into electron-like events with $\zeta<12$ and
proton-like events with $\zeta>20$ (see figure 3).

Figure 4: Distributions of selected DAMPE observables used as input of the
neural network, comparing MC with real data (flight). Events are selected in
the energy range 500 GeV to 1 TeV. Electron-like flight events are selected
with $\zeta<12$ and proton-like with $\zeta>20$, where $\zeta$ is the
classical discriminator (section 2). The same cuts are applied on both MC and
data. Error bars are purely statistical.

Figure 5: Distribution of selected DAMPE observables used as input of the
neural network, comparing MC with real data (flight). Events are selected in
the energy range 500 GeV to 1 TeV. Electron-like flight events are selected
with $\zeta<12$ and proton-like with $\zeta>20$, where $\zeta$ is the
classical discriminator (section 2). The same cuts are applied on both MC and
data. Error bars are purely statistical.

Along with optimising the set of input variables we were also researching and
optimising the architecture of the neural network itself (the model). Building
a model indeed requires several parameter choices: the number of neurons and
layers, the activation function, the optimiser and regularisers, etc. The
optimisation of these hyper-parameters is somewhat of an art, and a common
practice is to conduct a random gridsearch [24]. We decided to follow this
philosophy and tested hundreds of models against each other. The criterion
used to select the best model was the area under the energy-dependent false
positive curve (figure 10). We found the optimal configuration to be a model
consisting of 4 layers with 300, 150, 75 and 1 neuron, respectively,
regularised with a 10-20% dropout (technique consisting of randomly turning
off neurons during the training) [25]. We noted that a higher dropout resulted
in a slightly higher false positive rate, of the order of 10% more for 50%
dropout. The hidden layers use the Rectified Linear Unit (ReLU) [26]
activation function, and the output layer uses the logistic sigmoid function
to map the network output to the $\left[0;1\right]$ range as is common to
binary classification problems. Finally the model is optimised using the Adam
gradient descent algorithm [27] against the cross-entropy metric. The
architecture is represented in figure 6 and table 1. The network is trained
for 50 epochs, longer training did not result in any perceptible change in
performances and data matching (see further sections).

Figure 6: Schematic of the neural network model used in this work. The hidden layers use the ReLU activation function. The logistic sigmoid in the output layer is removed after training. Layer | Neurons | Parameters
---|---|---
Dense | 300 | 9000
Dropout, 20% |  | 0
Dense | 150 | 45150
Dropout, 10% |  | 0
Dense | 75 | 11325
Dropout, 10% |  | 0
Dense | 1 | 76
|  |
Total parameters: | 65,551 |
Trainable parameters: | 65,551 |
Non-trainable parameters: | 0 |
Table 1:  Layer-per-layer summary of the neural network used.

The extensive training campaigns were conducted on the Baobab computer cluster
of the University of Geneva, using Nvidia Titan X GPUs. On the software side,
we used Nvidia cuDNN [28], Keras [29] with Theano [30] as a backend, and
Scikit-Learn [31]. Google’s Tensorflow [32] was considered as well but
internal benchmarks with our models and data showed no gain in performances,
for a longer computing time.

A feature we noticed during the early stages of our optimisation procedure is
that the neural network output values are either very close (or exactly equal
to) 0.0 or 1.0, with only very few events classified in-between. This holds
true for false positives and false negatives as well: figure 7 (left) shows
that the histogram of MC protons (background) exhibits two peaks: one at 0.0
(true negatives) and the second, much smaller, at 1.0 (false
positives)222These events are likely protons that transfer most of their
energy to one or several $\pi_{0}$, starting electromagnetic showers while the
remaining energy yields a very small hadronic contribution. They are therefore
the most difficult background to distinguish from electron-induced showers..
The overall distribution is therefore non-monotonic, which introduces
complications for the estimation of background on a cosmic ray electron
measurement. For example, this behaviour prevents any sort of baseline
background extrapolation.

The cause is a feature of neural networks for classification: the very last
operation is a logistic sigmoid function that maps the output to the $[0;1]$
range:

$f(x)=\frac{1}{1+e^{-x}}$ (2.3)

Values $x\gg 0$ are mapped to $f(x)\simeq 1.0$, effectively compressing the
output into a limited, finite space. Computer floating point accuracy also has
an influence: for a 16-bits float, $f(18)=f(20)=1.0$ exactly. Therefore the
function is not bijective anymore.

Our workaround is to remove the logistic sigmoid from the output layer. This
must be done after training since the metric being optimised (binary
crossentropy) assumes an output between 0 and 1. The result is shown on figure
7 (right): the compression and non-monotonic features are gone, resulting in a
distribution much easier to use for baseline extrapolation methods, without
altering the classification performances (figure 8). By removing post-training
the last activation function, we get a more monotonic behaviour for both
classes. Another important gain is getting rid of the seemingly irreducible
background: without the sigmoid we can select a small sample with 0%
background, by e.g. cutting away events with a score $\lesssim 10$. Whereas
without this trick there are proton events mapped to the whole space $[0;1]$,
meaning that even the tightest possible cut at 1.0 will still have a non-zero
false positive rate.

Figure 7: Histogram of the neural network output on electron and proton Monte
Carlo: (left) standard model with a sigmoid activation function at the output
of the last layer, and with a peak of false positive outcomes in the proton
Monte Carlo highlighted with a circle ; (right) the same model after removing
(post-training) the sigmoid at the last layer. Figure 8: Sample Receiver
Operating Characteristic (ROC) curve of a model with a sigmoid activation
function at its output versus a model without. The curves compare the signal
efficiency (true positive rate) versus the remaining background (false
positive rate) at varying discrimination threshold. The perfectly superimposed
curves prove that the performances are exactly the same.

##  3 Performances

Figure 9: Sample ROC curves on Monte-Carlo for the neural network classifier
and the classical $\zeta$ classifier, in 8 reconstructed energy bins from 24
GeV to 10.4 TeV, comparing the fractional surviving background $f_{B}$ with
the signal efficiency $f_{S}$. The lowest curves have the lowest background
for a fixed efficiency. The coloured band shows statistical uncertainty from
Monte Carlo sampling. Logarithmic y-scale.

Figure 10: Energy dependency of the surviving background fraction $f_{B}$ for
a fixed signal efficiency $f_{S}$ of 85% (left) and 95% (right), for the
neural networks and the classical $\zeta$ method.

We report on figure 9 several Receiver Operating Characteristics (ROC) curves
for our neural networks, in comparison with the classical $\zeta$ method,
covering the energy range from 24 GeV to 10.4 TeV. ROC curves are obtained by
computing classification metrics at various discrimination thresholds, on the
test sample. In our case we choose to plot the signal efficiency $f_{S}$
against the fractional remaining background $f_{B}$. Both quantities are
defined as the ratio of passing events over total events for electrons and
protons respectively:

$\displaystyle f_{S}$ $\displaystyle=\frac{N_{e^{-},pass}}{N_{e^{-}}}$ (3.1)
$\displaystyle f_{B}$ $\displaystyle=\frac{N_{p,pass}}{N_{p}}$ (3.2)

These two metrics have the advantage of being independent from the relative
abundance of electrons with respect to protons. A good classifier is one that
maximises the first metric and/or minimises the second. This translates into a
lower curve on figure 9: classifiers with the lowest curves have the smallest
background for a set efficiency. The image shows that the neural network
significantly outperforms the classical method in the lowest and highest
energy ranges, while the performances appear roughly comparable at
intermediate energies. Note that for both classifiers, $N_{e^{-}}$ and $N_{p}$
are taken after the $\zeta<100$ cut for a fair comparison.

The performances are thus energy-dependent. To see this dependence and to
better quantify the performances of both methods, we report on figure 10 the
$f_{B}$ value when we set the discrimination threshold such as to have a 85%
or 95% signal efficiency, as a function of the energy reconstructed from the
BGO calorimeter. The comparison involves an uncertainty due to the efficiency
of both classifiers not being perfectly equal. On the figure, the error bars
associated to $\zeta$ show the statistical uncertainty from Monte Carlo
sampling, the darker band shows the uncertainty associated to the choice of
threshold to have compatible efficiency, and the lighter band is the
combination of both. The blue band associated to the neural networks is purely
statistical. Figure 10 confirms the previous observation that the gains of
neural networks are significant on both ends of the energy range, in the high
efficiency regime. From a few hundred GeV to 2 TeV, the performances are
within uncertainty of each other. Above 5 TeV, the proton rejection is
improved by a factor at least 2.

##  4 Model validation

Figure 11: Comparison of the neural network output distribution between
simulated Monte Carlo and real data, on three energy bins from 24 GeV to 1
TeV. Left panels are linear scale, right panels are log scale.

Figure 12: Comparison of the neural network output distribution between
simulated Monte Carlo and real data, on three energy bins from 1 to 10.4 TeV.
Left panels are linear scale, right panels are log scale.

Validating the modelling of the classifier on real data is an important step
in order for its use in an analysis measurement. Namely, it allows us to
validate the performances observed in figure 10, to exclude a potential bias
of the network with respect to MC, and to use the shape for e.g. baseline
interpolation methods.

Both distributions of simulations and data are shown on figures 11 and 12 in
six logarithmatically-spaced energy bins. Both samples have been cleaned such
that contributions from other cosmic species, notably Helium nuclei, are
negligible (section 2). A sum of proton and electron Monte-Carlo templates
were fitted to the data in the following way: the proton sample was scaled to
the data in the region $[-12;-4]$, while electron MC was scaled to the data in
the region $>1$ after subtracting the proton template from the data. The scale
factor is taken as the data-to-MC ratio of the integral over the relevant
control region. This technique allows to scale each distribution
independently, on regions where the contribution from the other species is
negligible. The Monte Carlo distributions are weighted according to their
expected flux, with a spectral index of -3 for electrons and -2.7 for protons.
Changing these indices according to the observed DAMPE values [11, 33] and
variating them within the corresponding uncertainties yielded similar figures
and conclusions. Due to some residual inaccuracies of modeling the input
variables in the neural network, both Monte Carlo distributions are shifted by
0.2 to the right (less than half a bin size) in order to improve matching with
the data. The final shift value is the result of a chi-square minimisation,
iterating by steps of 0.1. Error bars show purely statistical uncertainties.
Figures 11 and 12 show a good agreement between simulations and DAMPE data,
especially around the boundaries of signal and control regions. A small
disagreement below 100 GeV is however noticeable, although thanks to the very
low proton background at such energies it results in negligible uncertainties
if applied to a CRE flux measurement. The neural network classifier is
therefore reliable, in particular thanks to the transformation described in
section 2.2 allowing the scaling of Monte Carlo shown in the figures. The
remaining small differences between data and Monte-Carlo, in particular due to
potential inherent limitations on accuracy of the hadronic models used in the
simulations, can be further quantified and assigned a corresponding systematic
uncertainty, reflected in the subsequent measurement of cosmic ray electrons.
This is however beyond the scope of this work.

##  5 Conclusions

To tackle the problem of high energy cosmic electron identification and
measurement with DAMPE, we developed a four-layers deep feed-forward neural
network classifier, the output of which we transformed to suit the needs of
particle physics analysis. On simulated data, the new classifier shows a
strong background rejection power, outperforming the more traditional cut-
based electron identification technique in the energy ranges where the latter
shows its limits, thanks to a better exploitation of the information contained
within DAMPE calorimeter and tracker. In particular, the proton rejection
improves by a factor $>$2 in the multi-TeV range at a signal efficiency of
95%, and up to a factor 3 to 4 at 10 TeV, where the gain in accuracy is the
most demanded due to the low relative event rate of cosmic electrons at such
energies. A comparison between simulated and real data settles the reliability
of the new method and paves the way towards high precision measurement of
cosmic ray electron plus positron spectrum at the highest energies accessible
by DAMPE.

## Acknowledgments

The DAMPE mission was funded by the strategic priority science and technology
projects in space science of the Chinese Academy of Sciences. In China, the
data analysis was supported in part by the National Key Research and
Development Program of China (no. 2016YFA0400200), the National Natural
Science Foundation of China (nos. 11525313, 11622327, 11722328, U1738205,
U1738207, and U1738208), the strategic priority science and technology
projects of the Chinese Academy of Sciences (no. XDA15051100), the 100 Talents
Program of Chinese Academy of Sciences, and the Young Elite Scientists
Sponsorship Program. In Europe, the activities and the data analysis were
supported by the Swiss National Science Foundation (SNSF), Switzerland,
National Institute for Nuclear Physics (INFN), Italy and European Research
Council (ERC) under the European Union’s Horizon 2020 research and innovation
programme (grant agreement No 851103).

The computations presented in this document were performed at University of
Geneva on the Baobab cluster, with significant help from computer engineer Y.
Meunier and from the HPC team. Simulations were performed on INFN CNAF and
ReCaS clusters, Italy, and on Swiss National Supercomputing Centre (CSCS) Piz
Daint (project s979).

The corresponding author would like to acknowledge SARS-CoV-2 for making this
work significantly harder than it should have been.

## References

  * [1] J. Chang et al.  The DArk Matter Particle Explorer mission.  Astropart. Phys., 95:6–24, 2017.
  * [2] Zhiyong Zhang, Yunlong Zhang, Jianing Dong, Sicheng Wen, Changqing Feng, Chi Wang, Yifeng Wei, Xiaolian Wang, Zizong Xu, and Shubin Liu.  Design of a high dynamic range photomultiplier base board for the bgo ecal of dampe.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 780:21–26, 2015\.
  * [3] G Ambrosi, Q An, R Asfandiyarov, P Azzarello, P Bernardini, MS Cai, M Caragiulo, J Chang, DY Chen, HF Chen, et al.  The on-orbit calibration of dark matter particle explorer.  Astroparticle Physics, 106:18–34, 2019.
  * [4] A Tykhonov, G Ambrosi, R Asfandiyarov, P Azzarello, P Bernardini, B Bertucci, A Bolognini, F Cadoux, A D’Amone, A De Benedittis, et al.  Internal alignment and position resolution of the silicon tracker of dampe determined with orbit data.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 893:43–56, 2018\.
  * [5] Peter Meyer.  Cosmic rays in the galaxy.  Annual review of astronomy and astrophysics, 7(1):1–38, 1969.
  * [6] Igor V Moskalenko and AW Strong.  Production and propagation of cosmic-ray positrons and electrons.  The Astrophysical Journal, 493(2):694, 1998.
  * [7] Yi-Zhong Fan, Bing Zhang, and Jin Chang.  Electron/positron excesses in the cosmic ray spectrum and possible interpretations.  International Journal of Modern Physics D, 19(13):2011–2058, 2010\.
  * [8] T Kobayashi, Y Komori, K Yoshida, and J Nishimura.  The most likely sources of high-energy cosmic-ray electrons in supernova remnants.  The Astrophysical Journal, 601(1):340, 2004.
  * [9] Gianfranco Bertone, Dan Hooper, and Joseph Silk.  Particle dark matter: Evidence, candidates and constraints.  Physics reports, 405(5-6):279–390, 2005.
  * [10] Jonathan L Feng.  Dark matter candidates from particle physics and methods of detection.  Annual Review of Astronomy and Astrophysics, 48:495–545, 2010.
  * [11] G. Ambrosi et al.  Direct detection of a break in the teraelectronvolt cosmic-ray spectrum of electrons and positrons.  Nature, 552:63–66, 2017.
  * [12] Claus Grupen.  Astroparticle physics.  Springer Science & Business Media, 2005.
  * [13] J. Chang et al.  Resolving electrons from protons in ATIC.  Adv. Space Res., 42:431–436, 2008.
  * [14] Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton.  Deep learning.  Nature, 521(7553):436–444, 2015.
  * [15] Ian Goodfellow, Yoshua Bengio, and Aaron Courville.  Deep Learning.  MIT Press, 2016.  http://www.deeplearningbook.org.
  * [16] Pierre Baldi, Peter Sadowski, and Daniel Whiteson.  Searching for Exotic Particles in High-Energy Physics with Deep Learning.  Nature Commun., 5:4308, 2014.
  * [17] Dan Guest, Kyle Cranmer, and Daniel Whiteson.  Deep learning and its application to lhc physics.  Annual Review of Nuclear and Particle Science, 68:161–181, 2018\.
  * [18] C Schaefer, M Geiger, T Kuntzer, and J-P Kneib.  Deep convolutional neural networks as strong gravitational lens detectors.  Astronomy & Astrophysics, 611:A2, 2018.
  * [19] Daniel George and EA Huerta.  Deep neural networks to enable real-time multimessenger astrophysics.  Physical Review D, 97(4):044039, 2018.
  * [20] D Droz, A Tykhonov, and X Wu.  Neural networks for electron identification with dampe.  In 36th International Cosmic Ray Conference (ICRC2019), volume 36, 2019.
  * [21] Hao Zhao, Wen-Xi Peng, Huan-Yu Wang, Rui Qiao, Dong-Ya Guo, Hong Xiao, and Zhao-Min Wang.  A machine learning method to separate cosmic ray electrons from protons from 10 to 100 gev using dampe data.  Research in Astronomy and Astrophysics, 18(6):071, 2018.
  * [22] Sea Agostinelli, John Allison, K al Amako, John Apostolakis, H Araujo, P Arce, M Asai, D Axen, S Banerjee, G 2 Barrand, et al.  Geant4—a simulation toolkit.  Nuclear instruments and methods in physics research section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 506(3):250–303, 2003.
  * [23] Chi Wang, Dong Liu, Yifeng Wei, Zhiyong Zhang, Yunlong Zhang, Xiaolian Wang, Zizong Xu, Guangshun Huang, Andrii Tykhonov, Xin Wu, et al.  Offline software for the dampe experiment.  Chinese Physics C, 41(10):106201, 2017.
  * [24] James Bergstra and Yoshua Bengio.  Random search for hyper-parameter optimization.  Journal of Machine Learning Research, 13(Feb):281–305, 2012.
  * [25] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  The journal of machine learning research, 15(1):1929–1958, 2014\.
  * [26] Vinod Nair and Geoffrey E Hinton.  Rectified linear units improve restricted boltzmann machines.  In Proceedings of the 27th international conference on machine learning (ICML-10), pages 807–814, 2010.
  * [27] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [28] Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer.  cudnn: Efficient primitives for deep learning.  arXiv preprint arXiv:1410.0759, 2014.
  * [29] F. Chollet et al.  Keras.  https://keras.io, 2015.
  * [30] Theano Development Team.  Theano: A Python framework for fast computation of mathematical expressions.  arXiv e-prints, abs/1605.02688, May 2016.
  * [31] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * [32] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.  TensorFlow: Large-scale machine learning on heterogeneous systems, 2015\.  Software available from tensorflow.org.
  * [33] Qea An, R Asfandiyarov, P Azzarello, P Bernardini, XJ Bi, MS Cai, J Chang, DY Chen, HF Chen, JL Chen, et al.  Measurement of the cosmic ray proton spectrum from 40 gev to 100 tev with the dampe satellite.  Science advances, 5(9):eaax3793, 2019.