# A Closer Look into Recent Video-based Learning Research: A Comprehensive
Review of Video Characteristics, Tools, Technologies, and Learning
Effectiveness

Evelyn Navarrete<EMAIL_ADDRESS>0000-0002-5610-9908 L3S Research Center,
Leibniz UniversityHannoverGermany ,  Andreas Nehring  Institute for Science
EducationHannoverGermany<EMAIL_ADDRESS>0000-0002-8723-5552 ,
Sascha Schanze  Institute of Science EducationHannoverGermany<EMAIL_ADDRESS>hannover.de 0000-0002-5570-4991 ,  Ralph Ewerth  L3S Research Center, Leibniz
UniversityHannoverGermany TIB – Leibniz Information Centre for Science and
TechnologyHannoverGermany<EMAIL_ADDRESS>0000-0003-0918-6297 and  Anett
Hoppe  L3S Research Center, Leibniz UniversityHannoverGermany TIB – Leibniz
Information Centre for Science and TechnologyHannoverGermany
<EMAIL_ADDRESS>0000-0002-1452-9509

(2018)

###### Abstract.

People increasingly use videos on the Web as a source for learning, be it for
daily tasks or in formal as well as informal educational settings. To support
this way of learning, researchers and developers are continuously developing
tools, proposing guidelines, analyzing data, and conducting experiments.
However, it is still not clear what characteristics a video should have to be
an effective learning medium. In this paper, we present a comprehensive review
of 257 articles on video-based learning from computer science databases for
the period from 2016 to 2021 using the PRISMA guidelines. One of the aims of
the review is to identify the video characteristics that have been explored by
previous work. Based on our analysis, we suggest a taxonomy which organizes
the video characteristics and contextual aspects into eight categories:  (1)
audio features,  (2) visual features,  (3) textual features,  (4) instructor
behavior,  (5) learners’ activities,  (6) interactive features (quizzes,
etc.),  (7) production style, and  (8) instructional design.  Also, we
identify four representative research directions:  (1) proposals of tools to
support video-based learning,  (2) studies with controlled experiments,  (3)
data analysis studies, and  (4) proposals of design guidelines for learning
videos.  We find that the most explored characteristics are textual features
followed by visual features, learner’s activities, and interactive features.
Text of transcripts, video frames, and images (figures and illustrations) are
most frequently used by tools that support learning through videos. The
learner’s activity is heavily explored through log files in data analysis
studies, and interactive features have been frequently scrutinized in
controlled experiments. We complement our review by contrasting research
findings that investigate the impact of video characteristics on the learning
effectiveness, report on tasks and technologies used to develop tools that
support learning, and summarize trends in the development of design guidelines
to produce learning videos.

Video-based Learning, web-based learning, literature review, video
characteristics, video features

††copyright: acmcopyright††journalyear: 2018††doi: XXXXXXX.XXXXXXX

##  1\. Introduction

Videos become increasingly a choice in the everyday life of a learner,
facilitated not only by formal institutional platforms but also on commercial
general-purpose video platforms such as YouTube. In May 2019 it was stated
that about 500 hours of new content was uploaded every minute (Clement, 2019)
on YouTube, where around 50% of the views have been associated with learning
purposes (Smith et al., 2018). Furthermore, it was predicted that 82% of all
IP traffic in the world will be caused by video consumption by 2022 (Cisco,
2018). Learners who turn to video sources have to rely on the technologies
offered by the platform for providing suitable learning material. Moreover,
they need to trust the expertise (if any) of the content creators and
instructors in designing and producing effective learning videos. However,
even on dedicated platforms videos are published with no guidelines or good
practices in mind (Guo et al., 2014). The reason is that, although efforts to
better understand video-based learning (VBL) have increased (Poquet et al.,
2018; Giannakos, 2013; Yousef et al., 2014), essential questions are still not
answered, for example, what are the characteristics of a video111In the
context of this review, we understand video characteristics as specific
attributes, features, or aspects of a video that might be relevant to VBL. to
support learning in the best way, according to individual needs, the learning
goals, and other contextual aspects?

Previous VBL reviews have performed descriptive analysis on the growth
tendency of the field, the research directions, the discipline addressed by
the videos, the production style, the mode of use (i.e., main or supplementary
learning medium), the context or setting (i.e., formal or informal learning),
video characteristics, and other dimensions (Poquet et al., 2018; Giannakos,
2013; Yousef et al., 2014). Nevertheless, they have put special emphasis on
experimental research, and did not dive into different research directions.

In this paper, we present a comprehensive review of 257 articles on video-
based learning from computer science databases for the period from 2016 to
2021 using the PRISMA guidelines (Page et al., 2021). Compared to other
reviews, we have collected the largest number of articles and provide a
comprehensive overview on VBL research. As a consequence, we can achieve a
more robust analysis of the state of the art in the field. In contrast to
previous reviews that focused on experimental research, we identify and
consider four main types of research articles:  (1) proposals of tools to
support video-based learning,  (2) studies with controlled experiments,  (3)
data analysis studies, and  (4) proposals of design guidelines for learning
videos.  We present a close investigation of  (a) video characteristics,  (b)
research approaches,  (c) tasks and technologies used to develop tools,
frameworks, and pipelines that support learning,  (d) guidelines to design and
create learning videos, as well as  (e) findings from controlled experiments
and data analysis.  Furthermore, we derive a taxonomy that organizes video
characteristics and contextual aspects into eight categories (and further sub-
categories):  (1) audio features,  (2) visual features,  (3) textual features,
(4) instructor behavior,  (5) learners’ activities,  (6) interactive features,
(7) production style, and  (8) instructional design.  The presented results
are relevant for different audiences:  (a) platform providers who wish to
support their users with the best possible content,  (b) content producers who
aim to develop effective learning resources,  (c) teachers who want to enhance
their teaching by adopting multimedia resources, and  (d) researchers who seek
to update on the current state of VBL, identify potential research gaps, and
bring in ideas for further research.

This review is an extended version of our previous work (Navarrete et al.,
2021), where we have presented a review of the literature on video-based
learning (VBL) for only two years (2020 and 2021). Here, we expand the
reviewed period to 2016-2021 and also include the most relevant results of the
articles by contrasting, analyzing and summarizing their findings.

The rest of the paper continues with a discussion of related reviews (Section
2), the description of the methodology (Section 3), the analysis of the field
of video-based learning, which provides an introduction to the importance of
identifying the characteristics of learning videos (Section 4), and the
results of the review (Section 5). After that, we discuss the impact of said
video characteristics on learning effectiveness (Section 6). Finally, we draw
conclusions and point out limitations and future work (Section 7).

##  2\. Related Work

Table 1. Related survey articles: reference and publication year, temporal scope of the survey, number of reviewed papers, and reviewed characteristics Reference | Review | N° papers | Reviewed dimensions |
---|---|---|---|---
| period |  |  |
(Giannakos, 2013) (2013) | 2000-2012 | 166 | 1) Type of research | 4) Technology type
|  |  | 2) Sample characteristics | 5) Style of use
|  |  | 3) Subject area |
(Yousef et al., 2014) (2014) | 2003-2014 | 76 | 1) Learning effectiveness | 3) Design
|  |  | 2) Teaching methods | 4) Reflection
(Poquet et al., 2018) (2018) | 2007-2017 | 178 | 1) Video type | 3) Sample size
|  |  | 2) Population | 5) Video characteristics
|  |  | 3) Control of prior knowledge |
(Sablić et al., 2021) (2020) | 2008-2019 | 39 | Teacher perception and reflection |

To the best of our knowledge, four survey articles on video-based learning
have been presented previously. A summary with the aspects of the VBL field
they have reviewed is displayed in Table 1. Giannakos (Giannakos, 2013)
focused increase in the number of publications on VBL, the shift of VBL
research from theoretical to empirical, and the characteristics of research
regarding the type, sample, discipline area, use of stable or mobile devices,
and the context of the video usage (main, formal, and informal). Yousef et al.
(Yousef et al., 2014) summarized the findings of experimental research on the
effectiveness of VBL, teaching methods when using videos, integration of tools
with VBL content, and the use of videos to reflect on teaching methods. Poquet
et al. (Poquet et al., 2018) also concentrated on experimental studies. The
authors present a descriptive analysis on the types of videos, the population
to which the experiment was directed, if the experiment controlled the prior
knowledge, the sample size, growth patterns in the number of publications, and
the type of research (theoretical and empirical). In addition, the authors
review the video characteristics that influence the learning effectiveness
with special emphasis on the learning outcome. The most recent work was
presented by Sablic et al. (Sablić et al., 2021), who review research on the
use of videos to support teacher’s self-reflection and professional
development. There are other reviews on learning from YouTube videos such as
the recent work by Shoufan and Mohamed (Shoufan and Mohamed, 2022). We have
not considered this study or similar studies since they did not meet our
criteria of having a computer science perspective and focusing on VBL in
general, and not only on a specific platform.

All these reviews confirm the significant growth of research articles in the
VBL field (Poquet et al., 2018; Giannakos, 2013; Yousef et al., 2014).
Giannakos (Giannakos, 2013) and Poquet et al. (Poquet et al., 2018) reported a
noticeable increase of empirical and quantitative research over theoretical
and qualitative research. Furthermore, Giannakos (Giannakos, 2013) showed a
shift in the research domain, from the social sciences to technological
domains, which was also confirmed by the results of Poquet et al. (Poquet et
al., 2018) that science, technology, engineering, and mathematics (STEM)
disciplines are currently a predominant research focus. Further important
findings from Poquet et al. (Poquet et al., 2018) suggested that the
effectiveness of VBL has frequently been evaluated through two learning
outcome metrics: recall (remembering information) and transfer (applying what
was learned to a different scenario), which was also corroborated by Yousef et
al. (Yousef et al., 2014). Other popular metrics of learning effectiveness
were motivation, cognitive load, mental effort, attention, and affect (Poquet
et al., 2018). Also, the most common sources to obtain data for analysis were
clickstreams (e.g., views, likes, dislikes) and eye-tracking data. Regarding
the video characteristics, Poquet et al. (Poquet et al., 2018) found that the
most examined characteristics were text, audio, animations, and voice over a
PowerPoint presentation. Yousef et al. (Yousef et al., 2014) found that
trending research topics were the effects of annotations and authoring tools,
collaborative VBL, and video usage for reflective teacher education.

##  3\. Methodology

Figure 1. The diagram shows the process of finding VBL-related articles in the
databases. From an initial set of 390 papers, we removed duplicates and papers
that did not fit the inclusion criteria. The final set consists of 257
articles from 2016 to 2021.

The methodology for our systematic review was inspired by the Preferred
Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines
(Page et al., 2021). They provide a guide to summarize results of reviewed
research articles in an appropriate manner.

The papers covered by this review were found at SpringerLink (spr, 2022), the
Digital Library of the Association for Computing Machinery (ACM DL) (acm,
2022), and the Digital Bibliography and Library Project (DBLP) (dbl, 2022).
These are focused in computer science (CS) domains, and therefore, fit well
our task of reviewing the technological aspects of video-based learning. In
each of these databases, we used advanced search options as a search strategy
to refine our results. In ACM DL, we applied filters to retrieve publications
that match the query terms only in the title and that fit the time frame. In
SpringerLink, we applied the same filters complemented by an additional filter
to obtain articles only in CS disciplines. In DBLP, however, we did not apply
any filter since the search engine covers only CS and already retrieves
articles that only match the query term in the title and sorts them all per
year.

Table 2 shows the set of query combinations we used, which includes all the
keywords from our previous work (Navarrete et al., 2021) plus new terms
considered for this extended review (massive open online courses MOOC and
Lecture). We followed an iterative process where the set of query terms was
updated as new terms in the field of VBL were found (please see the search
order in Table 2), and then performed a new search in the databases.

Table 2. This table shows the set of query terms used to find relevant articles. The base term in the first column was combined with each of the extensions in the second column (using logical AND as a connector). Base | Extension |  |  |
---|---|---|---|---
Video | 1.1) Teach | 1.2) Teaching | 2.1) Education | 2.2) Educational
| 3.1) Instruction | 3.2) Instructional | 4.1) Learn | 4.2) Learning
| 5) -based Learning | 6.1) Explanatory | 6.2) Explaining | 7) Tutorial
| 8) Classroom | 9) Student | 10.1) Knowledge | 10.2) Knowing
| 11) Skills | 12) University | 13) School | 14) Pupil
| 15) MOOC | 16) Lecture |  |

We included a paper in the final set only when the article:  (a) considered
videos in academic scenarios (this excludes video tutorials for everyday tasks
such as cooking instructions),  (b) focused on videos as learning resources
(this excludes the use of classroom recordings for reflective tasks such as in
teacher education), and  (c) examined specific characteristics of video-based
learning materials (this excludes, for instance, the use of videos to evaluate
their viability in classroom).  Figure 1 shows the obtained articles using the
PRISMA-guided search process. The final set of articles consists of 257
articles published in the period from 2016 to 2021.

In our data collection process, we noted the article’s title, authors, year,
source database, matching terms of our query set, research direction, research
objectives, video characteristics, and, if applicable, main findings,
discipline of the video, study sample size, video production style(s),
dependent variables, and target tasks and technologies. All the information
was assembled in a data table to facilitate efficient data access and
analysis.

##  4\. Video-based Learning

In this section, we discuss important aspects of video-based learning,
potentials, challenges, and video characteristics. This lays the groundwork
and motivates our literature review in the following section.

Potentials and Challenges: Video-based learning is a terminology that has
already been adopted in several studies (e.g., (Giannakos, 2013; Poquet et
al., 2018; Yousef et al., 2014; Sablić et al., 2021)), and can be considered
as the task of learning using video sources, that is, acquiring knowledge and
skills through video content. Videos have been implemented in classrooms as a
teaching medium (Poquet et al., 2018) and they form an important part of
students’ habits (Smith et al., 2018). The trends show that videos might
become the main learning medium in online education (Hansch et al., 2015) due
to its wide availability. Learning platforms, such as MOOCs, can disseminate
videos in simple and scalable ways (Giannakos, 2013; Guo et al., 2014).

The debate about the benefits of VBL continues. Yousef et al. (Yousef et al.,
2014), for instance, concluded from their results that learning videos are
effective because they can improve the learning outcome, motivation,
engagement, satisfaction, interaction, and communication between learners.
They further argued that videos can support various learning styles and convey
information otherwise hard to communicate by modalities such as plain text.
However, the question of how to further support different learners’ needs
remains open, since, for example, VBL usually does not allow synchronous
communication and feedback as in classic classroom settings.

Moreover, further issues have been found in the way videos are designed and
created. Guo et al. (Guo et al., 2014) found in their interviews to video
producers that “video editing was not done with any specific pedagogical
“design patterns” in mind” (Guo et al., 2014, p. 45) and “MOOC video producers
currently base their decisions on anecdotes, folk wisdom, and best practices
distilled from studies with at most dozens of subjects and hundreds of video
watching sessions” (Guo et al., 2014, p. 42). Additionally, methods from
typical classroom scenarios have been transferred to create learning videos,
however, it is unclear whether this can be successful (Guo et al., 2014).

Examination of video characteristics: Learning videos can be a complex
learning medium since audio, visual, textual, and a large set of derived
features might collide when expressing a message. This complexity leads us to
the following research question: Which of these characteristics make a video
effective for learning? The analysis and exploration of characteristics in
learning videos have been examined for different purposes, from experiments
that try to understand the impact on the learning effectiveness to the
construction of tools for supporting learning. Consequently, there is a
multitude of video characteristics that have been extracted, studied, and
manipulated. These can range from low-level characteristics (e.g., text color,
visual luminance, audio energy) to high-level characteristics such as models
of a video’s semantic content.

Video-based learning does not only involve the video and its instructors but
also the surrounding hosting system or e-learning platform. Therefore, we
analyzed characteristics related to the platform, for example, those related
to the learner’s activity, such as the usage of play and pause buttons, or
number of views and likes. Also, more complex characteristics attempt to
consider the human aspects in the video (e.g., instructor gestures and
speech). In out analysis, we focused exclusively on characteristics related to
the video content and its hosting platform while excluding learner-related
characteristics, as for instance, previous knowledge, age, gender, etc.

##  5\. Literature Review

Figure 2. The diagram shows the share of the different research directions
identified in this review. Tools and studies that perform controlled
experiments constituted the main research directions.

In this section, we present the results of our review and, briefly, the
content of the relevant papers. Based on our analysis, we suggest to
distinguish between four representative research directions:  (1) tools,  (2)
controlled experiments,  (3) data analysis, and  (4) design guidelines.  We
found that the introduction and presentation of tools was present in more than
the half of the articles, while reports about controlled experiments
represented a quarter of the articles (see Figure 2).

In the following sub-sections, we report the results for each of these
research directions, since this provides a natural way to analyze and contrast
articles to draw conclusions. Thus, video characteristics are discussed
differently in the context of each research direction. Table 3 summarizes the
characteristics and contextual aspects identified in a single taxonomy, which
we organized into eight main categories:  (1) audio features,  (2) visual
features,  (3) textual features,  (4) instructor behavior,  (5) learner’s
activities,  (6) interactive features,  (7) production style, and  (8)
instructional design.  It is noticeable that specific characteristics fall
naturally in specific categories of the taxonomy. For example, interactive
features (e.g., quizzes and annotations) have been often studied in controlled
experiments but little in data analysis studies (see Figure 3). Figure 4
depicts the structure of this section and the dimensions that were analyzed
for each research type.

Figure 3. The diagram shows the number of investigated video characteristics in the different research directions. Some video characteristics were more widely explored in certain research directions than in others. Figure 4. The diagram shows the dimensions we analyzed for every type of research. Table 3. Taxonomy of video characteristics Category |  Sub-category |  Examples
---|---|---
Audio features |  Audio records (Shi et al., 2019; Mahapatra et al., 2018a; Ohnishi et al., 2019; Husain and Meena, 2019; Soares and Barrére, 2018; Kanadje et al., 2016; Gandhi et al., 2016; Kumar et al., 2017; Zhao et al., 2017; Mahapatra et al., 2018b; Bian and Huang, 2019; Balasubramanian et al., 2016; Walk et al., 2020; Furini, 2018; Jr. et al., 2019; Lin et al., 2019; Sharma et al., 2016b; Chatbri et al., 2017; Furini, 2021; Sanchez-Cortina et al., 2016; Gopalakrishnan et al., 2017) |
|  Sound-related (Shoufan, 2019a; Ding, 2018; Shi et al., 2019; Shoufan and Mohamed, 2017; Mukherjee et al., 2019; Soares and Barrére, 2018; Kanadje et al., 2016; Ghauri et al., 2020; Furini, 2018; Imran et al., 2019; Chen et al., 2020) |  Energy, entropy, spectral, jitter, shimmer, noise, Mel-frequency cepstral coefficients, perceived quality
|  Representation characteristics (cues) (Wang et al., 2020b; Shi et al., 2019) |  Instructor-related: verbal cues
Visual features |  Video frames and images (figures and illustrations) (Zhao et al., 2018b; Mahapatra et al., 2018a; Koka et al., 2020; Husain and Meena, 2019; Ma et al., 2017; Kumar et al., 2017; Zhao et al., 2017; Yadav et al., 2016; Mahapatra et al., 2018b; Bian and Huang, 2019; Zhao et al., 2018b; Ohnishi et al., 2019; Yadav et al., 2016; Xu et al., 2019b; Cagliero et al., 2019; Balasubramanian et al., 2016; Rahman et al., 2020, 2020; Ghauri et al., 2020; Khattabi et al., 2019; Jung et al., 2019; Davila et al., 2021; Kota et al., 2020; Guan et al., 2019; Kota et al., 2019; Xu et al., 2019a; Davila and Zanibbi, 2018; Furini, 2018; Lee et al., 2017b; Garber et al., 2017; Liu et al., 2017; Ponzanelli et al., 2019; Yadid and Yahav, 2016; Zhong and Ma, 2018; Alahmadi et al., 2018; Khandwala and Guo, 2018; Che et al., 2016; Lin et al., 2019; Jung et al., 2019; Guan et al., 2019; Furini et al., 2018; Waykar and Bharathi, 2017; Ciurez et al., 2019; Xu et al., 2020; Godavarthi et al., 2020; Sharma et al., 2016b; Wang et al., 2020a; Furini, 2021; Gopalakrishnan et al., 2017; Nong et al., 2019) |
|  Representation characteristics (cues) (Wang et al., 2020b; Moon and Ryu, 2021; Pi et al., 2019; Wang et al., 2019b; Pi et al., 2017b; Sharma et al., 2016c; Mahapatra et al., 2018a; Jung et al., 2018; Sharma et al., 2016c) |  Instructor-related: pointing gestures, directed gaze. Content-related: highlighted content, arrows, moving objects
|  Quality-related (Shoufan, 2019a; Ding, 2018; Shoufan and Mohamed, 2017) |  video quality, resolution in pixels
Textual features |  transcripts (Shi et al., 2019; van der Sluis et al., 2016; Atapattu and Falkner, 2017; Mukherjee et al., 2019; Kravvaris and Kermanidis, 2019; Das and Das, 2020; Ohnishi et al., 2019; Husain and Meena, 2019; Soares and Barrére, 2019; Galanopoulos and Mezaris, 2019; Soares and Barrére, 2018; Gandhi et al., 2016; Kumar et al., 2017; Zhao et al., 2017; Yadav et al., 2016; Mahapatra et al., 2018a; Bian and Huang, 2019; Ma and Ma, 2019; Yoo et al., 2021; Vimalaksha et al., 2019; Balasubramanian et al., 2016; Kim and Kim, 2016; Ghauri et al., 2020; Liu et al., 2018b; Jo et al., 2019; Jung et al., 2019; Walk et al., 2020; Kravvaris and Kermanidis, 2019; Furini, 2018; Albahr et al., 2021; Jr. et al., 2019; Ponzanelli et al., 2019; Escobar-Avila et al., 2017; Nangi et al., 2019; Che et al., 2016; Tavakoli et al., 2020; Bleoanca et al., 2020; Schulten et al., 2020; Jordán et al., 2020; Zhao et al., 2018a; Aytekin et al., 2020; Jung et al., 2018; Tang et al., 2021; Walk et al., 2020; Borges and dos Reis, 2019; Kravvaris and Kermanidis, 2017; Basu et al., 2016b; Cooper et al., 2018; Coccoli and Vercelli, 2016; Ji et al., 2018; Fraser et al., 2019; Stoica et al., 2019; Basu et al., 2016a; Dessì et al., 2017; Chatbri et al., 2017; Wang et al., 2020a; Horovitz and Ohayon, 2020; Hernandez and Yang, 2021) |
|  Metadata (Eradze et al., 2020; Saurabh and Gautam, 2019; Shoufan, 2019a; Lee et al., 2017a; Silva and Azevedo, 2017; Hildebrand and Ahn, 2018; Sjödén et al., 2018; Meseguer-Martinez et al., 2019; Galster et al., 2018; Das and Das, 2020; Liu et al., 2018b; Kravvaris and Kermanidis, 2019; Jr. et al., 2019; Ponzanelli et al., 2019; Escobar-Avila et al., 2017; Che et al., 2016; Tavakoli et al., 2020; Jordán et al., 2020; Zhao et al., 2018a; Tang et al., 2021; Basu et al., 2016b; Cooper et al., 2018; Borges and Silveira, 2019; Fraser et al., 2019; Stepanek and Dorn, 2017; Othman et al., 2016; Sung et al., 2016; Stoica et al., 2019; Mubarak et al., 2021; Bulathwela et al., 2020; Basu et al., 2016a; Poché et al., 2017; Othman et al., 2017; Lei et al., 2019) |  Titles, keywords, tags, video length, comments, course syllabus, video url
|  On-screen text (der Zee et al., 2017; Ozdemir et al., 2016; Whitney and Dallas, 2019; Zhao et al., 2018b; Mahapatra et al., 2018a; Ohnishi et al., 2019; Husain and Meena, 2019; Das and Das, 2020; Gandhi et al., 2016; Kumar et al., 2017; Zhao et al., 2017; Mahapatra et al., 2018a; Koka et al., 2020; Bian and Huang, 2019; Xu et al., 2019b; Cagliero et al., 2019; Balasubramanian et al., 2016; Davila et al., 2021; Kota et al., 2020, 2020; Xu et al., 2019a; Davila and Zanibbi, 2018; Lee et al., 2017b; Garber et al., 2017; Ponzanelli et al., 2019; Yadid and Yahav, 2016; Zhong and Ma, 2018; Alahmadi et al., 2018; Liu et al., 2018b; Khandwala and Guo, 2018; Che et al., 2016; Furini et al., 2018; Waykar and Bharathi, 2017; Ciurez et al., 2019; Dessì et al., 2017; Nong et al., 2019) |
|  Representation characteristics (Mahapatra et al., 2018a) |  Font size, boldeness
Instructor behavior |  Body-related (Beege et al., 2020, 2019; Wang et al., 2019a; Horovitz and Mayer, 2021; Li et al., 2016; Ding, 2018; Tian and Bourguet, 2016; Chen et al., 2020; Xu et al., 2020; Godavarthi et al., 2020; Hoogerheide et al., 2018; Shoufan and Mohamed, 2017) |  Gestures: hand strokes and waving. Facial emotions, body language, pose, clothes, gender appearance
|  Speech-related (Horovitz and Mayer, 2021; Shoufan, 2019a; Ding, 2018; van der Sluis et al., 2016; Atapattu and Falkner, 2017; Shoufan and Mohamed, 2017; Patel et al., 2018; Mahapatra et al., 2018a; Soares and Barrére, 2019; Balasubramanian et al., 2016; Jo et al., 2019; Jung et al., 2019; Che et al., 2016; Jung et al., 2019; Ji et al., 2018; Bulathwela et al., 2020) |  Tone, speech rate, intonation, lexical diversity, syllable count, prosodic characteristics, readability, silence rate
|  Attitudes (Shoufan, 2019a) |  Enthusiasm
Learner’s activity |  (Eradze et al., 2020; Stöhr et al., 2019; Wynn et al., 2019; Costley et al., 2021; Angrave et al., 2020; Li, 2019; Rodríguez et al., 2018; Costley and Lange, 2017; Giannakos et al., 2016; Hu et al., 2020; Uchidiuno et al., 2018; Kovacs, 2016; van der Sluis et al., 2016; Shoufan, 2019b; Lei et al., 2017; Yan and Baxter, 2018; Dodson et al., 2018a; Atapattu and Falkner, 2017; Patel et al., 2018; He et al., 2018a; Walk et al., 2020; Choi et al., 2019; Long et al., 2018; Zhang et al., 2018a; Bothe and Meinel, 2020; Wachtler et al., 2016b; Mbouzao et al., 2019; Min et al., 2019; He et al., 2018b; Zhang, 2017; Brinton et al., 2016; Mubarak et al., 2021; Jeon and Park, 2020; Lee and Muldner, 2020; Aouifi et al., 2021; Furini et al., 2021; Mbouzao et al., 2020; Lemay and Doleck, 2020; Korösi et al., 2018; Othman et al., 2017; Lallé and Conati, 2020; Saurabh and Gautam, 2019; Shoufan, 2019a; Lei et al., 2019; Meseguer-Martinez et al., 2019; Lee et al., 2017a; Shoufan and Mohamed, 2017; Tavakoli et al., 2020; Kravvaris and Kermanidis, 2017; Zhang et al., 2018b; Tang et al., 2021; Merkt et al., 2018; Schroeder et al., 2020; Garrett, 2021; Lang et al., 2020; Bulathwela et al., 2020) |  Explicit: play, pause, speed rate, seek forward and backward. Implicit: viewing percentage, average of videos watched per week. Popularity-related: number of views, likes, dislikes, user ratings. Pace-related: automatic pauses, pauses controlled by the learner, type of pauses, fast pace.
Interactive features |  (Kleftodimos and Evangelidis, 2018; van Alten et al., 2020; Leisner et al., 2020; Keller et al., 2019; der Zee et al., 2018; Sözeri and Kert, 2021; Taskin et al., 2019; Lu et al., 2021; Kuhail and Aqel, 2020; Chen et al., 2021; Winkler et al., 2020; Wachtler et al., 2016a; Cojean and Jamet, 2017; Ku et al., 2019; Palaigeorgiou and Papadopoulou, 2019; Moos and Bonde, 2016; Shelton et al., 2016; da Graça Campos Pimentel et al., 2019; Chen, 2020; Muñoz-Carpio et al., 2020; Araiza-Alba et al., 2021; Daher and Sleem, 2021; Huang et al., 2020; Guervós et al., 2019; Ding, 2018; Granjo and Rasteiro, 2018; Fong et al., 2018; Liu et al., 2019; Herault et al., 2018; Walk et al., 2020; Lin and Chen, 2019; Malchow et al., 2018; Jung et al., 2018; Deb et al., 2017; Jo and Lim, 2017; Nguyen and Liu, 2016; Sharma et al., 2016a; Kohen-Vacs et al., 2016; Kleftodimos and Evangelidis, 2016; Bao et al., 2019; To et al., 2016; Zhu et al., 2017; Llanda, 2019a; Liu et al., 2018a; Lai et al., 2018; Singh et al., 2016; Khandwala and Guo, 2018; Robal et al., 2018; Wachtler et al., 2016b; Koumaditis and Chinello, 2019; Berns et al., 2018; Wachtler and Ebner, 2019) |  Quizzes embedded in the video, virtual reality and 360-degree characteristics, in-video annotations, feedback to the learner, games, hyperlinks, interface elements for navigation
Production style |  (Cao et al., 2018; Ohashi et al., 2019; Hew and Lo, 2020; Pi et al., 2017a; Stull et al., 2018; Srivastava et al., 2019; Davila et al., 2021; Srivastava et al., 2020; Lee and Muldner, 2020; Nugraha et al., 2020; Pérez-Navarro et al., 2021; Al-Khateeb and Alduwairi, 2020; Cookson et al., 2020; Ding et al., 2018; Stranc and Muldner, 2019; Ding, 2018; Costley and Lange, 2017; Hildebrand and Ahn, 2018; Rahim and Shamsudin, 2019; Thornton et al., 2017; Shoufan, 2019a; Lei et al., 2019; Shoufan and Mohamed, 2017; Sharma et al., 2016b; Hartholt et al., 2020; Hayashi et al., 2020) |  Khan-style, with a talking head, dialogue-style or monologue-style, slide-based, with animations, paper-based explanations, classroom-style videos,
Instructional design and principles |  (Zhang, 2021; van der Meij, 2019; Eradze et al., 2020; Shoufan, 2019a; Ou et al., 2016; Lei et al., 2019; Acuña-Soto et al., 2020; Zhang et al., 2018b; Minnes et al., 2019; Wijnker et al., 2019; Weinert et al., 2020; Ge and Li, 2020; Lei et al., 2016; Voronkin, 2019; Lange and Costley, 2020; Stephenson, 2019; Dodson et al., 2018b; Hodges, 2018; Hung et al., 2018) |  Multimedia learning principles, guidelines for effective engaging, pedagogical characteristics of didactical mathematical knowledge, according to film theory, interaction theory, limited capacity theory
Others |  (Schreiber and Dougherty, 2017; Kikusawa et al., 2016; van der Meij and van der Meij, 2016; der Zee et al., 2017; Shi et al., 2019) |  Music, sign language, inclusion of a summary, content design, visual complexity, content design: text layout, image position, slide size, slide design, text ration, image ration

###  5.1. Tools, Pipelines, Frameworks, and Methods

Figure 5. types of tools that support video-based learning identified in this
review

In this section, we give an overview of the tools that the research community
developed to assist and enhance video-based learning. We also report the
automatic tasks carried out by the tools and the technologies that supported
it.

We identified and classified the articles according to the following types of
tools:  (a) information extraction methods,  (b) interactive tools,  (c)
recommendation systems,  (d) classifiers,  (e) video navigation and
segmentation tools,  (f) video analysis tools and methods,  (g) video
summarization, and  (h) others,  where the first five types represented about
80% of all developed tools (see Figure 5). Also, please note that some
articles proposed systems consisting of two or more tools (Jung et al., 2018;
Walk et al., 2020; Wachtler et al., 2016b; Khandwala and Guo, 2018; Zhang,
2017).

Figure 6 shows the video characteristics that have been used in articles to
develop VBL tools. It can be seen that text and visual features were the most
used characteristics, mainly by recommender systems, information extraction
and summarization methods, as well as navigational tools. Regarding tasks and
technologies, we identified some low-level tasks performed as basic steps, of
which, many rely on deep learning approaches. For example, for textual video
elements, several studies employed optical character recognition (Xu et al.,
2019b; Mahapatra et al., 2018a; Liu et al., 2018b; Davila et al., 2021; Bian
and Huang, 2019; Cagliero et al., 2019; Kota et al., 2019; Husain and Meena,
2019; Xu et al., 2019a; Davila and Zanibbi, 2018; Furini et al., 2018; Furini,
2018; Lee et al., 2017b; Waykar and Bharathi, 2017; Che et al., 2016;
Balasubramanian et al., 2016; Dessì et al., 2017; Zhao et al., 2017; Jr. et
al., 2019; Ponzanelli et al., 2019; Yadid and Yahav, 2016; Zhong and Ma, 2018;
Mahapatra et al., 2018b; Nong et al., 2019; Khandwala and Guo, 2018), keyword
extraction (Hernandez and Yang, 2021; Koka et al., 2020; Kravvaris and
Kermanidis, 2019; Jo and Lim, 2017; Waykar and Bharathi, 2017; Kanadje et al.,
2016; Balasubramanian et al., 2016; Zhang, 2021; Cooper et al., 2018; Coccoli
and Vercelli, 2016; Kumar et al., 2017; Jr. et al., 2019; Escobar-Avila et
al., 2017; Ji et al., 2018; Yadav et al., 2016; Mahapatra et al., 2018b;
Schulten et al., 2020), generic natural language processing methods (e.g.,
(Cagliero et al., 2019; Kravvaris and Kermanidis, 2019, 2017; van der Sluis et
al., 2016; Poché et al., 2017)), or utilized word embeddings (e.g., (Ghauri et
al., 2020; Hernandez and Yang, 2021; Nangi et al., 2019; Galanopoulos and
Mezaris, 2019; Dessì et al., 2017; To et al., 2016)). For image-related data,
many studies suggested approaches for slide transition detection and video
segmentation (Mahapatra et al., 2018a; Khattabi et al., 2019; Schroeder et
al., 2020; Jung et al., 2018; Guan et al., 2019; Cagliero et al., 2019; Furini
et al., 2018; Furini, 2018; Garber et al., 2017; Liu et al., 2017; Che et al.,
2016; Nguyen and Liu, 2016; Zhao et al., 2017; Galanopoulos and Mezaris, 2019;
Soares and Barrére, 2018; Basu et al., 2016a; Gandhi et al., 2016; Yadid and
Yahav, 2016; Ghosh et al., 2021; Xu et al., 2019a). For audio tracks, various
studies carried out speech recognition to generate transcripts (Leisner et
al., 2020; Walk et al., 2020; Yoo et al., 2021; Bian and Huang, 2019; Ohnishi
et al., 2019; Cagliero et al., 2019; Borges and dos Reis, 2019; Husain and
Meena, 2019; Galanopoulos and Mezaris, 2019; Furini, 2018; Jo and Lim, 2017;
Sanchez-Cortina et al., 2016; Balasubramanian et al., 2016; Basu et al.,
2016b; Dessì et al., 2017; Coccoli and Vercelli, 2016; Gandhi et al., 2016;
Zhao et al., 2017; Jr. et al., 2019; Mahapatra et al., 2018b; Chatbri et al.,
2017). In the next subsections, we discuss the tasks and technologies in
detail according to each type of tool. Within each subsection, we have grouped
the tool proposals according to their purpose and the video characteristics
that they have employed.

Figure 6. The diagram displays the shares for targeted video characteristics
used to develop tools for video-based learning. Textual and visual features
were the mostly used characteristics, mainly by recommendation systems,
information extraction, summarization, and navigational tools.

####  5.1.1. Information Extraction Tools

The reviewed articles in this section focused mainly on the extraction of text
from videos (handwritten or machine printed) and metadata, as well as the
extraction of key information such as the extraction of key video segments.

Extraction of text: We found seven proposals that extracted handwritten text
(Davila et al., 2021; Kota et al., 2020, 2019; Xu et al., 2019a; Lee et al.,
2017b; Garber et al., 2017) and two papers on extracting machine-printed text
(e.g., from slide-based videos) (Zhong and Ma, 2018; Khattabi et al., 2019).
Davila et al. (Davila et al., 2021) retrieved handwriting using a
convolutional neural network (CNN). Kota et al. (Khattabi et al., 2019) used
multiscale histogram of gradients for feature representation and deep neural
networks to extract handwriting. In another article, Kota et al. (Kota et al.,
2019) detected bounding boxes with handwriting and used clustering algorithms
to group them, which could identify also content that was occluded by the
instructor when she was standing in front of the whiteboard. The handwriting
was then extracted by enclosing it in bounding boxes and binarizing it. Lee et
al. (Lee et al., 2017b) detected changes in the the handwriting pixels and
extracted them using an adaptive threshold and time-series denoising
techniques. Davila and Zanibbi (Davila and Zanibbi, 2018) extracted
handwritten formulas using a random forest classifier trained on handwritten
symbols. Xu et al. (Xu et al., 2019a) identified the main instructor’s actions
such as writing and erasing as clues to identify the video content. The
actions were detected using pose estimation technologies. In the same way,
Garber et al. (Garber et al., 2017) identified handwriting through
instructor’s actions of writing and erasing. The instructor could be removed
by masking the regions where there is a change among video frames. Zhong and
Ma (Zhong and Ma, 2018) extracted machine printed text using the maximally
stable extremal regions method to detect candidate regions and clustering
techniques were used to ensemble lines of text. Finally, Khattabi et al.
(Khattabi et al., 2019) detected parts of the video that contained text using
an AdaBoost meta-classifier.

Extraction of key information: Ghauri et al. (Ghauri et al., 2020) classified
key segments in videos using different fusion schemes and bi-long-short-term
memory network modules. Che et al. (Che et al., 2016) detected the most
important video segments using the slides structure and speech
characteristics. Kravvaris and Kermanidis (Kravvaris and Kermanidis, 2019)
identified important topics in the transcripts using Latent Dirichlet
Allocation. They contrasted them with relevant terms from the learners’
comments which were found using the Maximum Likelihood algorithm. Lin et al.
(Lin et al., 2019) classified knowledge from non-knowledge by using a CNN that
processes image and sound simultaneously. The classifier is trained on public
datasets containing audio and visual human conversations, cooking events, and
also collected video lectures; however, it was not clarified what the criteria
were to classify something as ”knowledge”. Similarly, Liu et al. (Liu et al.,
2018b) extracted medical information according to their level of knowledge
(low or high) by building a Logistic Regression classifier using the text from
transcripts. Finally, for computer programming video tutorials, Ponzanelli et
al. (Ponzanelli et al., 2019) classified segments according to the kind of
content, for example, theoretical concepts, software source code, etc. The
classifier employs temporal, structural, semantic, and code-related
characteristics.

Extraction of metadata: Jung et al. (Jung et al., 2019) extracted keywords
from transcripts using the TextRank algorithm. Tags were extracted using
singular value decomposition techniques and other relevant information was
extracted using entity recognition techniques. Escobar-Avila et al. (Escobar-
Avila et al., 2017) compared three algorithms (term frequency–inverse document
frequency, Latent Dirichlet Allocation using genetic algorithms, and the BM25F
ranking function) to find the best technique to extract tags from video
titles, description, and transcripts. Gomes et al. (Jr. et al., 2019)
extracted related topics with the help of a knowledge base, where a knowledge
graph was built measuring the semantic similarity between concepts. The
similarity was based on the number of resources (Uniform Resource Identifiers
of instances in the knowledge base) the videos share and the categories that
the resources share, and was calculated with the Sorensen-Dice coefficient.
Furini (Furini, 2018) extracted tags by measuring the most frequent words.
Ghosh et al. (Ghosh et al., 2021) detected concepts that are not familiar or
do not belong to the domain of the video content but could be relevant to
better understand the video. The concepts were linked to Wikipedia articles,
which were used to build a concept similarity network. Unfamiliar topics were
extracted using community detection techniques. Similarly, Nangi et al. (Nangi
et al., 2019) extracted unfamiliar topics, which were considered as cross-
discipline concepts, also using community analysis in a concept graph.

Other: Part of the studies presented tools to extract key-phrases (Yadid and
Yahav, 2016; Albahr et al., 2021), slides (Guan et al., 2019; Liu et al.,
2017), and software source code (from video tutorials to learn computer
programming) (Khandwala and Guo, 2018; Yadid and Yahav, 2016; Alahmadi et al.,
2018).

####  5.1.2. Interactive Tools

The development of interactive tools (tools that implement features allowing
for communication and interaction between the video and the learner) mainly
targeted in-video annotations, quizzes embedded in the video, and,
interestingly, feedback to the learner for preventing mind wandering.

In-video Annotations: These tools offered a feature that allowed the learner
to take notes while watching the video, and also were accompanied with other
supporting characteristics. Fong et al. (Fong et al., 2018) suggested a tool
in which the learner can interact not only through annotations but also
interface elements for navigation. Liu et al. (Liu et al., 2019) allowed the
learner to create annotations and highlight parts of the transcript. Malchow
et al. (Malchow et al., 2018) let the learner create annotations for each
slide presented in the video. Lai et al. (Lai et al., 2018) did not only offer
text-based annotations but also sketch-based annotations where the learner
could draw into the video frame. Similarly, Singh et al. (Singh et al., 2016)
created a tool for text-based and sketch-based annotations, and supported the
student in comment creation, color-coding, and interface elements for
navigation. Nguyen and Liu (Nguyen and Liu, 2016) automated the note taking
process by tracking eye gaze. The tracking system automatically extracted
relevant parts as candidate notes. Deb et al. (Deb et al., 2017) allowed the
viewer to create annotations but did not introduce another supporting feature.

Quizzes: Herault et al. (Herault et al., 2018) proposed a 360-degree
interactive environment with quizzes embedded in the video and feedback to
support the learner. Kohen-Vacs et al. (Kohen-Vacs et al., 2016) introduced a
tool to embed quizzes and announcements along the playback into the video.
Walk et al. (Walk et al., 2020) included quizzes in the video with the aim of
enhancing active learning. Wachtler et al. (Wachtler et al., 2016b) presented
an application that can include several types of questions such as single- and
multiple-choice questions, as well as questions to the instructor. Ma and Ma
(Ma and Ma, 2019) developed an automatic question generation method using the
video transcript and an external knowledge graph. Wachtler and Ebner (Wachtler
and Ebner, 2019) created an application to schedule quizzes and other
interactive features along the playing video.

Feedback to learner for preventing mind wandering: Lin et al. (Lin and Chen,
2019) introduced feedback in learning videos to alert the viewers in mind-
wandering states. The tool kept track of brainwave signals to detect loss of
focus and recommended to review again parts of the video. Jo and Lim (Jo and
Lim, 2017) instead used a game that identified high-frequency words. To detect
mind wandering, the game consisted in presenting a word to the learner and ask
if that word was present previously in the video. Robal et al. (Robal et al.,
2018) made use of web page, mouse, and face tracking technologies to determine
whether the viewer was paying attention. If the user was inattentive, the
system rang a bell, displayed a red frame around the video, and paused it.
Sharma et al. (Sharma et al., 2016a) employed eye gaze tracking to identify
mind wandering and displayed red marks when distraction was detected. In a
similar way, Llanda (Llanda, 2019b) offered a tool that provided feedback to
the learner, although, not with regard to mind wandering but to the sentiment
state. The tool tracked the viewers’ emotions by analyzing face patterns and
detecting negative sentiment to consequently suggest re-watching the video.

Other: Some studies suggested tools to create videos that integrate 360-degree
characteristics (Herault et al., 2018; Koumaditis and Chinello, 2019; Berns et
al., 2018). Other papers introduced clickable objects (Jung et al., 2018; To
et al., 2016), interactive concept maps (Liu et al., 2018a), feedback for the
learners adapted to their previously identified proficiency level (Zhu et al.,
2017), and tools to interact directly with the video content for practicing,
for example, image editing (Kleftodimos and Evangelidis, 2016) and computer
programming (Bao et al., 2019; Khandwala and Guo, 2018).

####  5.1.3. Recommender and Retrieval Systems

Recommender systems have mainly considered the similarity between transcripts
and metadata to suggest new videos. There were, however, other interesting
characteristics that have been exploited, for instance, video popularity
(e.g., likes and views), learning style, teaching style, etc.

Based on the similarity of transcripts and metadata: Zhao et al. (Zhao et al.,
2018a) suggested recommendations based on topic similarity, whereby the video
is represented by means of its latent topics using the Latent Dirichlet
Allocation algorithm. Waykar and Bharathi (Waykar and Bharathi, 2017)
retrieved videos based on keywords extracted from on-screen text and also the
texture of keyframes using a texture histogram. Then, the videos were arranged
with the help of Probability Extended Neighbor Classification. Bleoanca et al.
(Bleoanca et al., 2020) retrieved the top-N videos based on topic similarity
using Latent Semantic Indexing to extract them, clustering techniques to group
videos by discipline, and Wikipedia articles to augment the query context.
Basu et al. (Basu et al., 2016b) recommended videos when the learner was
navigating Wikipedia pages. They used the Latent Dirichlet Allocation
algorithm to extract relevant topics from Wikipedia content and video
transcripts. Jung et al. (Jung et al., 2019) recommended similar videos based
on keywords, tags, and the readability of the video. In a similar manner, Ji
et al. (Ji et al., 2018) generated recommendations using metrics of linguistic
difficulty and similarity of keywords identified by the TextRank algorithm.
Jordán et al. (Jordán et al., 2020) built a system that considers not only the
characteristics of the current video but also previous videos watched by the
learner. Walk et al. (Walk et al., 2020) made use of word embeddings
constructed from the video transcript and compared them against the user
search query to retrieve the most similar video segment. Tavakoli et al.
(Tavakoli et al., 2020) built a system to recommend videos for learning skills
required in the labor market. The authors use the skills that can be learned
from the video and the skills demanded by the market together with video
popularity features (e.g., likes and views) and length.

Based on ontologies and external knowledge bases: Schulten et al. (Schulten et
al., 2020) determined related concepts through external ontologies based on
Wikipedia articles and extracted keywords using the term frequency–inverse
document frequency algorithm to discover new learning resources through a web
search API. In the same way, Borges and Reis (Borges and dos Reis, 2019)
extracted concepts and their relationships from an existing knowledge base to
recommend videos. Coccoli and Vercelli (Coccoli and Vercelli, 2016) extracted
concepts and also related concepts by linking the content to Wikipedia
articles. The learners could receive recommendations of videos and also Web
pages. Borges and Silveira (Borges and Silveira, 2019) retrieved videos by
analyzing the subject-object-predicate construction of the user query and
searching keywords in a semantic graphic database.

Playlists: Aytekin et al. (Aytekin et al., 2020) assembled a set of videos for
recommendations based on prerequisite relationship detection techniques, that
is, identifying when a concept refers to another concept. Likewise, Tang et
al. (Tang et al., 2021) recommended a playlist by building a concept map that
showed the most relevant concepts in videos and how they are linked to each
other. The authors also considered the sentiment analyzed from likes,
dislikes, and comments. Furini et al. (Furini et al., 2018) recommended a
playlist composed of video segments that covered the keywords given by the
learner. For segmenting the video, silence periods and also visual transitions
were detected using histogram-based techniques. Techniques for optical
character recognition were employed to understand the video content and match
it to the user query. Cooper et al. (Cooper et al., 2018) recommended videos
based on the main topics, which were found by using the Latent Dirichlet
Allocation algorithm and inter-topic relationships from the course syllabus.
For creating the playlist, the authors used sequential pattern mining
techniques to discover relationships.

Other: Further studies presented recommender systems based on video popularity
(Kravvaris and Kermanidis, 2017; Zhang et al., 2018b), videos delivered by the
same instructor (Guan et al., 2019; Imran et al., 2019), viewer’s learning
style (Ciurez et al., 2019), the current software application that the learner
is using (Fraser et al., 2019), and multiple pedagogical criteria (Acuña-Soto
et al., 2020).

####  5.1.4. Classifiers and Predictors

Classifiers and predictors were mainly used to estimate learning outcome,
while some other noticeable tasks were discipline classification, instructor
behavior prediction, and dropout prediction.

Learning outcome prediction: Lemay and Doleck (Lemay and Doleck, 2020) used a
repeated incremental pruning algorithm to predict the learners’ grades based
on the frequency of video viewing. Mbouzao et al. (Mbouzao et al., 2020)
predicted whether a learner will pass or fail a MOOC course using attendance
rate (number of videos the student played divided by the number of videos
mandatory to watch), utilization rate (percentage of video play time activity
divided by the sum of video lengths of all videos), and watching index
(attendance rate multiplied by utilization rate) metrics in a straightforward
rule that compared a single student against the average. Mubarak et al.
(Mubarak et al., 2021) trained a long-short-term memory network to predict the
learning outcome as pass or fail based on learner’s activity data (e.g., play,
pause, speed, etc.). Furukawa et al. (Furukawa et al., 2020) used the number
of views, interval of video views (e.g., 0 to 19 days), and the first access
to videos in a multiple regression predictor to evaluate whether the learner
will pass or fail a MOOC course. Similarly, Korosi et al. (Korösi et al.,
2018) predicted whether a learner will pass or fail a MOOC course using random
forests and bagging based on characteristics related to platform usage (e.g.,
scrolling actions) and not representing the video itself. In a different way
to determine learning success, Lallé and Conati (Lallé and Conati, 2020)
clustered learners into low and high achievers using characteristics such as
the average proportion of videos watched per week. Aouifi et al. (Aouifi et
al., 2021), instead, predicted learning outcome in terms of passing or failing
the next quiz. The authors take as characteristics the sequence of previous
quiz results to feed them into a recurrent neural network. Brinton et al.
(Brinton et al., 2016) also predicted whether the learner will pass a quiz
successfully but using characteristics such as play and pause, revisiting,
etc.

Discipline classification: Stoica et al. (Stoica et al., 2019) built a support
vector machine trained on a Wikipedia dataset to classify video transcripts
and keywords into the discipline to which they belong. Dessi et al. (Dessì et
al., 2017) introduced a model based on variants of support vector machine
algorithms to classify videos into their corresponding disciplines using
transcripts, on-screen text features, and a relevance score for each feature.
Basu et al. (Basu et al., 2016a) clustered videos according to their
disciplines by analyzing the topic similarity between the transcript and
Wikipedia articles. Othman et al. (Othman et al., 2017) classified videos into
categories such as Education and Science and Technology but also to rather
different types including How-to and Style. The authors used decision trees
and a Naive Bayes classifier for classification based on a various video
metadata features. In a different approach, Chatbri et al. (Chatbri et al.,
2017) built a CNN fed with image-like representations that characterize the
transcript to then classify the video into its discipline. The images are
generated using statistical co-occurrence transformation techniques, which
arranges the transcript into an array of characters and uses ASCII codes of
the characters to create gray scale images.

Instructor behavior prediction: Chen et al. (Chen et al., 2020) built a neural
network to predict teachers’ enthusiasm using facial expressions, voice,
volume, and pose characteristics. Godavarthi (Godavarthi et al., 2020) also
used facial landmarks characteristics but to predict instructor emotions using
a CNN. Sharma et al. (Sharma et al., 2016b) predicted video liveliness
(energetic or active) using two different long-short-term memory networks fed
with audio features, visual features from the instructor actions, and
production setup (e.g., instructor sitting or standing, slide-based video or
whiteboard-based video). Xu et al. (Xu et al., 2020) predicted the
instructor’s actions (write, pick eraser, erase, etc.) based on pose
characteristics using a random forest and adaptive CNN classifiers.

Dropout prediction: Jeon and Park (Jeon and Park, 2020) used learner’s
activity data to predict weekly course dropout in a MOOC platform by embedding
click data sequences and feeding them into a gate recurrent unit. Furini et
al. (Furini et al., 2021) investigated dropout prediction in an e-learning
platform relying on characteristics related to video access, and using random
forest and K-nearest neighbor classification.

Other: Further articles proposed models to predict how engaging a video was
(Bulathwela et al., 2020), when a learner will jump back (Zhang, 2017), and to
classify learners’ comments (Poché et al., 2017).

####  5.1.5. Navigation and Segmentation Tools

Many articles proposed navigation in the style of a table of contents, but
also navigation using keywords and visual elements has been presented. The
suggested tools commonly use segmentation techniques to match the extracted
information with the corresponding video segment to allow user navigation.

Table of contents navigation: Such tools consist of a table of contents which
the user can click on and jump to the corresponding part in the video. These
approaches usually allow users to find segments where a certain sub-topic or
keyword is presented. The segments to which the sub-topic or keyword belong
are identified and indexed to allow the learner to click on them and to jump
there directly (Das and Das, 2020; Kravvaris and Kermanidis, 2019; Mahapatra
et al., 2018a; Mukherjee et al., 2019; Husain and Meena, 2019; Gandhi et al.,
2016; Kumar et al., 2017; Zhao et al., 2017; Yadav et al., 2016; Bian and
Huang, 2019). Sub-topics and keywords have been extracted with technologies
such as the Latent Dirichlet Allocation algorithm (Mukherjee et al., 2019;
Kravvaris and Kermanidis, 2019; Husain and Meena, 2019; Bian and Huang, 2019),
TextRank algorithm (Yadav et al., 2016; Mahapatra et al., 2018b), Markov chain
Monte Carlo method (Bian and Huang, 2019), and concept extraction algorithms
(Mahapatra et al., 2018b).

Keyword navigation: Another frequent kind of proposal for navigation extracted
keywords and indexed them according to specific segments of the video (Ohnishi
et al., 2019; Koka et al., 2020; Kravvaris and Kermanidis, 2019; Kanadje et
al., 2016; Kumar et al., 2017; Zhao et al., 2017).

Navigation using visual elements: This kind of approach enabled exploration by
presenting visual parts of the video, for example, main slides, figures,
tables, charts, etc., which the viewer clicked and then jumped to the
corresponding segment in the video (Zhao et al., 2018b; Ma et al., 2017; Kumar
et al., 2017; Zhao et al., 2017; Yadav et al., 2016).

Video segmentation: One of the natural ways of splitting a video is to detect
slide changes, which can be used to extract key video frames. For video
segmentation, TextTiling (Mukherjee et al., 2019; Zhao et al., 2017) and
dynamic programming (Gandhi et al., 2016) were suggested. For the detection of
slide changes, edge-based differences between video frames (Zhao et al., 2017)
and overlapping patches (Mahapatra et al., 2018a) were expoited. Also, we
found studies dedicated to exclusively find better segmentation methods for
learning videos (e.g.(Soares and Barrére, 2019; Galanopoulos and Mezaris,
2019; Soares and Barrére, 2018)). These approaches used natural language
processing for phrase extraction, named entity recognition, and part-of-
speech-tagging (Galanopoulos and Mezaris, 2019) to segment the video. Other
studies used audio features to segment according to sound characteristics
(e.g., power spectral density, fundamental frequencies, low level acoustic
characteristics) (Soares and Barrére, 2018, 2019).

####  5.1.6. Analytic Tools

Analytic tools relied heavily in the learner’s activity data to understand
video usage, and in some cases, contrasted this usage with the learners’
performance. Another small group of analytic tools used characteristics such
as metadata and eye gaze.

Based on learner’s activity: General analytic tools exploited log data of
learner activity from e-learning platforms to show how videos are used. Walk
et al. (Walk et al., 2020) developed a tool that shows the audience retention
for specific video segments, the number of times the segment was re-watched,
and how much time learners spent on it. Wachtler et al. (Wachtler et al.,
2016b) presented a tool that provided an analysis about the video (e.g.,
number of views) and also about the student (e.g., extent to which the video
was watched). Long et al. (Long et al., 2018) proposed a framework to analyze
patterns regarding video usage, for instance, usage of a single video,
specific video segments, all videos in a course, etc.

Other tools used log data to understand the learners’ behavior in relation to
their academic performance. Min et al. (Min et al., 2019) suggested an
analytic system that contrasted the log data and the student performance on
the quizzes with the watching behavior. He et al. (He et al., 2018a) analyzed
the viewing behavior using two metrics, where the first one records if the
learner watched the video and the second one records the amount of video
viewing. Later, He at al. (He et al., 2018b) showed how these metrics can be
adopted to understand the results in academic performance.

In addition, learner activity data has been used in other interesting analytic
tools for diverse purposes. Mbouzao et al. (Mbouzao et al., 2019) presented a
method to embed the viewer’s sequence of activities into a vector and found if
they characterize a specific video, this is, if a certain video elicits
particular behavior. Bothe and Meinel (Bothe and Meinel, 2020) developed a
metric to track viewers’ visits to previous MOOC videos, considering that
learners do not always strictly follow the path suggested by a platform. Choi
et al. (Choi et al., 2019) included smartwatch data in their system log data
to improve the measurement of interest and difficulty of a video.

Based on video metadata: Othman et al. (Othman et al., 2016) mined MOOC videos
by exploiting their metadata (e.g., video length, comments, title, country).
Stepanek and Dorn (Stepanek and Dorn, 2017) presented a framework to analyze
viewers’ comments to identify and predict the engagement generated by a video.
Sung et al. (Sung et al., 2016) analyzed viewers’ comments considering the
sentiment, its relevance, its type, and its topic for displaying this
information in the video timeline.

Other: Zhang et al. (Zhang et al., 2018a) analyzed learning videos using data
from eye tracking systems.

####  5.1.7. Summarization Tools

Summarization approaches aim at generating a concise and shorter
representation of the entire video. Text summarization was the most applied
approach, however, also summarization of visual elements was suggested.

Text-based summarization: This kind of summarization was achieved by sentence
segmentation, named entity recognition, extraction of key phrase extract,
subtitles, and other metadata using transcripts as a main source (Vimalaksha
et al., 2019; Balasubramanian et al., 2016; Kim and Kim, 2016) and on-screen
text in a few cases (Xu et al., 2019b; Cagliero et al., 2019). Methods that
supported this task were term frequency–inverse document frequency (Vimalaksha
et al., 2019), Naive Bayes (Balasubramanian et al., 2016), and Latent Semantic
Analysis (Yoo et al., 2021) for identifying the relevant information.

Image-based summarization: Rahman et al. (Rahman et al., 2020) proposed a
visual summary by extracting a group of relevant images (figures or
illustrations) from slide-based videos. The image was included or not in the
summary according to its importance, which was computed based on its size,
unique aspects in the image, and the duration of its occurrence in the video.

####  5.1.8. Other

Further interesting articles proposed methods to create videos automatically
(Hartholt et al., 2020; Hayashi et al., 2020; Furini, 2021), alleviate the
split attention problem (Sharma et al., 2016c; Gopalakrishnan et al., 2017),
enhance speech recognition (Hernandez and Yang, 2021; Sanchez-Cortina et al.,
2016), and find content that is similar not in other videos but in other media
such as in textbooks (Wang et al., 2020a) and books (Horovitz and Ohayon,
2020).

###  5.2. Controlled Experiments

In this section, we report general findings regarding the evolution of
experimental research, main disciplines, production style, and sample sizes.
We also present the video characteristics that have been tested in controlled
experiments and the dependent variables used to measure the impact of those
characteristics (e.g., on engagement, learning outcome, etc.).

####  5.2.1.  Trends, Disciplines, Production Style, and Sample Size

Trends: There is an increase of articles on controlled experiment for the last
three years in comparison to the previous three years. More than 60% of
articles in this category have been published in the last three years (Beege
et al., 2020; van Alten et al., 2020; Leisner et al., 2020; Chen, 2020; Keller
et al., 2019; Stranc and Muldner, 2019; Wang et al., 2020b; Schroeder et al.,
2020; Moon and Ryu, 2021; Sözeri and Kert, 2021; Muñoz-Carpio et al., 2020;
Cookson et al., 2020; Al-Khateeb and Alduwairi, 2020; Araiza-Alba et al.,
2021; Taskin et al., 2019; Daher and Sleem, 2021; Pérez-Navarro et al., 2021;
Lu et al., 2021; Nugraha et al., 2020; Wang et al., 2019a; Kuhail and Aqel,
2020; Huang et al., 2020; Garrett, 2021; Lee and Muldner, 2020; Lang et al.,
2020; Chen et al., 2021; Srivastava et al., 2020; Horovitz and Mayer, 2021;
Davila et al., 2021; Winkler et al., 2020; Pi et al., 2019; Srivastava et al.,
2019; Zhang, 2021; Hoogerheide et al., 2018; Ku et al., 2019; Guervós et al.,
2019; da Graça Campos Pimentel et al., 2019; Whitney and Dallas, 2019; Ohashi
et al., 2019; van der Meij, 2019).

Disciplines: Most of the experiments were performed using learning videos in
STEM disciplines (around 80%) (Beege et al., 2020; Merkt et al., 2018; Beege
et al., 2019; Kleftodimos and Evangelidis, 2018; Leisner et al., 2020; Keller
et al., 2019; Ding et al., 2018; Schroeder et al., 2020; Moon and Ryu, 2021;
Sözeri and Kert, 2021; Muñoz-Carpio et al., 2020; Al-Khateeb and Alduwairi,
2020; Pérez-Navarro et al., 2021; Lu et al., 2021; Nugraha et al., 2020;
Garrett, 2021; Lee and Muldner, 2020; Lang et al., 2020; Chen et al., 2021;
Srivastava et al., 2020; Horovitz and Mayer, 2021; Winkler et al., 2020; Pi et
al., 2019; Srivastava et al., 2019; Stull et al., 2018; Wang et al., 2019b; Pi
et al., 2017a, b; Schreiber and Dougherty, 2017; Li et al., 2016; Zhang, 2021;
Sharma et al., 2016c; der Zee et al., 2017; Wachtler et al., 2016a; Cojean and
Jamet, 2017; Ozdemir et al., 2016; Hoogerheide et al., 2018; Garrett, 2018; Ku
et al., 2019; Palaigeorgiou and Papadopoulou, 2019; Hew and Lo, 2020; Guervós
et al., 2019; van der Meij and van der Meij, 2016; Shelton et al., 2016; da
Graça Campos Pimentel et al., 2019; Whitney and Dallas, 2019; Ohashi et al.,
2019; Cao et al., 2018; van der Meij, 2019) and some studies experimented with
videos in STEM and non-STEM disciplines (e.g., (Beege et al., 2020)).
Experiments with STEM topics have primarily considered physics, mathematics,
and computer science subjects. In general, we identified the following STEM
disciplines:  (1) physics and mathematics (19% of controlled experiments)
(Merkt et al., 2018; Leisner et al., 2020; Pérez-Navarro et al., 2021;
Hoogerheide et al., 2018; Palaigeorgiou and Papadopoulou, 2019; Chen et al.,
2021; Horovitz and Mayer, 2021; Wachtler et al., 2016a; Ku et al., 2019; Hew
and Lo, 2020; Al-Khateeb and Alduwairi, 2020; Srivastava et al., 2020, 2019),
(2) computer science (19%) (Sözeri and Kert, 2021; Muñoz-Carpio et al., 2020;
Lee and Muldner, 2020; Lang et al., 2020; Winkler et al., 2020; Schreiber and
Dougherty, 2017; Li et al., 2016; da Graça Campos Pimentel et al., 2019;
Ohashi et al., 2019; Whitney and Dallas, 2019; Wang et al., 2019b; Nugraha et
al., 2020; Chen et al., 2021),  (3) biology, chemistry, and medicine (14%)
(Ding et al., 2018; Zhang, 2021; Srivastava et al., 2020, 2019; Pi et al.,
2019; Lu et al., 2021; der Zee et al., 2017; Stull et al., 2018; Beege et al.,
2019; Guervós et al., 2019),  (4) software use, digital media and
communication (13%) (van der Meij, 2019; van der Meij and van der Meij, 2016;
Garrett, 2021, 2018; Kleftodimos and Evangelidis, 2018; Keller et al., 2019;
Pi et al., 2017a, b; Ozdemir et al., 2016), and  (5) environmental science and
meteorology (11%) (van der Meij, 2019; van der Meij and van der Meij, 2016;
Garrett, 2021, 2018; Schroeder et al., 2020; Beege et al., 2020; Moon and Ryu,
2021; Sharma et al., 2016c; Cao et al., 2018; Cojean and Jamet, 2017; Shelton
et al., 2016; Chen et al., 2021) Experiments on non-STEM topics focused mainly
on language education (6%) (Chen, 2020; Kuhail and Aqel, 2020; Davila et al.,
2021; Huang et al., 2020), history and geography (6%) (van Alten et al., 2020;
Wang et al., 2020b; Daher and Sleem, 2021; Beege et al., 2020), and psychology
(4%) (Cookson et al., 2020; Moos and Bonde, 2016; Chen et al., 2021).

Production style: Most of the experiments reported the production style of the
videos (69% of controlled experiments). The results showed that most of them
used slide-based videos (39% of the experiments) (Leisner et al., 2020; Wang
et al., 2019a; Lee and Muldner, 2020; Srivastava et al., 2020; Horovitz and
Mayer, 2021; Pi et al., 2019; Srivastava et al., 2019; Wang et al., 2019b; Pi
et al., 2017b; Sharma et al., 2016c; Hoogerheide et al., 2018; Kikusawa et
al., 2016; Moos and Bonde, 2016; Hew and Lo, 2020; da Graça Campos Pimentel et
al., 2019; Ohashi et al., 2019; Cao et al., 2018). The presence of an
instructor was also very common among the experiments (36%) (Beege et al.,
2020, 2019; Stranc and Muldner, 2019; Ding et al., 2018; Nugraha et al., 2020;
Wang et al., 2019a; Srivastava et al., 2020; Horovitz and Mayer, 2021; Pi et
al., 2019; Stull et al., 2018; Pi et al., 2017b; Hoogerheide et al., 2018;
Kikusawa et al., 2016; Hew and Lo, 2020; da Graça Campos Pimentel et al.,
2019; Cao et al., 2018). Less frequent production styles included handwriting
(e.g., Khan-style, classroom-style videos) (16%), (Pérez-Navarro et al., 2021;
Lee and Muldner, 2020; Srivastava et al., 2020; Stull et al., 2018; Schreiber
and Dougherty, 2017; Sharma et al., 2016c; Hew and Lo, 2020), talking-head
(16%) (Pérez-Navarro et al., 2021; Davila et al., 2021; Pi et al., 2017a, b;
Li et al., 2016; Hew and Lo, 2020; Cao et al., 2018), screencasts (14%)
(Keller et al., 2019; Garrett, 2021; Ozdemir et al., 2016; Garrett, 2018; van
der Meij and van der Meij, 2016; van der Meij, 2019), productions with virtual
reality and 360-degree characteristics (14%) (Chen, 2020; Muñoz-Carpio et al.,
2020; Araiza-Alba et al., 2021; Daher and Sleem, 2021; Huang et al., 2020;
Guervós et al., 2019), and animations in videos (content created entirely with
drawings, illustrations, or computer-generated effects) (14%) (Schroeder et
al., 2020; Moon and Ryu, 2021; Cookson et al., 2020; Al-Khateeb and Alduwairi,
2020; Srivastava et al., 2020, 2019; Li et al., 2016). Interestingly, we
noticed that animations were considered more frequently in the last three
years (Moon and Ryu, 2021; Cookson et al., 2020; Al-Khateeb and Alduwairi,
2020; Srivastava et al., 2020; Schroeder et al., 2020; Srivastava et al.,
2019). Please note that several studies used more than one production style
(e.g., (Srivastava et al., 2020; Pi et al., 2017b; Hew and Lo, 2020)).

Sample size: The experiment with the least number of participants included
four learners (Kikusawa et al., 2016), the one with most of participants
included 398 learners (Shelton et al., 2016), the average was 94 participants.
There was one experiment with a very large sample size of more than 12,000
participants, which was performed on a MOOC platform (der Zee et al., 2018).

####  5.2.2. Video Characteristics in Experiments

Figure 7. The diagram shows the number of papers for each video feature type
studied in controlled experiments. More than 80% of these studies focused on
interactive features, production style, instructor behavior, and visual
features

.

Figure 7 summarizes and ranks the video characteristics studied in controlled
experiments. Specifically, we found four categories of characteristics that
have been examined the most:  (1) interactive features (e.g., quizzes) (39% of
controlled experiments) (Kleftodimos and Evangelidis, 2018; van Alten et al.,
2020; Leisner et al., 2020; Keller et al., 2019; der Zee et al., 2018; Sözeri
and Kert, 2021; Taskin et al., 2019; Lu et al., 2021; Kuhail and Aqel, 2020;
Chen et al., 2021; Winkler et al., 2020; Wachtler et al., 2016a; Cojean and
Jamet, 2017; Ku et al., 2019; Palaigeorgiou and Papadopoulou, 2019; Shelton et
al., 2016; Moos and Bonde, 2016; da Graça Campos Pimentel et al., 2019; Chen,
2020; Muñoz-Carpio et al., 2020; Araiza-Alba et al., 2021; Daher and Sleem,
2021; Huang et al., 2020; Guervós et al., 2019),  (2) production style (e.g.,
talking head) (25%) (Stranc and Muldner, 2019; Ding et al., 2018; Cookson et
al., 2020; Al-Khateeb and Alduwairi, 2020; Pérez-Navarro et al., 2021; Nugraha
et al., 2020; Lee and Muldner, 2020; Srivastava et al., 2020; Davila et al.,
2021; Srivastava et al., 2019; Stull et al., 2018; Pi et al., 2017a; Hew and
Lo, 2020; Ohashi et al., 2019; Cao et al., 2018),  (3) instructor’s behavior
(e.g., gestures) (10%) (Beege et al., 2020, 2019; Wang et al., 2019a; Horovitz
and Mayer, 2021; Li et al., 2016; Hoogerheide et al., 2018), and  (4) visual
features (10%) (Wang et al., 2020b; Moon and Ryu, 2021; Pi et al., 2019; Wang
et al., 2019b; Pi et al., 2017b; Sharma et al., 2016c).

Interactive features: These features allow the communication and interaction
between the video and the learner. The most used interactive features in
experiments were multiple-choice quizzes and questions embedded in the video
(Kleftodimos and Evangelidis, 2018; Leisner et al., 2020; Wachtler et al.,
2016a; Palaigeorgiou and Papadopoulou, 2019; da Graça Campos Pimentel et al.,
2019; Sözeri and Kert, 2021), virtual reality and 360-degree environments
(Chen, 2020; Muñoz-Carpio et al., 2020; Araiza-Alba et al., 2021; Daher and
Sleem, 2021; Huang et al., 2020; Guervós et al., 2019), in-video annotations
(feature that allow the learner to take notes while watching the video)
(Leisner et al., 2020; Taskin et al., 2019; Kuhail and Aqel, 2020; Chen et
al., 2021), and navigational elements (e.g., table of content navigation,
timelines with clickable points of interest) (Kuhail and Aqel, 2020; da Graça
Campos Pimentel et al., 2019; Cojean and Jamet, 2017). Examples of less
researched interactive features were games (e.g., crossword puzzles) (Cojean
and Jamet, 2017), human-like conversational agents (Winkler et al., 2020), and
self-regulated learning activities (van Alten et al., 2020; Moos and Bonde,
2016).

Production style: In general, the production style refers to the scenery, the
elements used to convey the information, and also the physical arrangements
considered to record the video. Examples for production style settings are
Khan-style video, classroom-style videos, with talking head, etc.
Specifically, we use the typology of video production styles of Hansch et al.
(Hansch et al., 2015). We found that more than a third of the experiments that
examined production styles addressed the instructor’s presence (Cao et al.,
2018; Hew and Lo, 2020; Pi et al., 2017a; Davila et al., 2021; Pérez-Navarro
et al., 2021). Another third of the experiments evaluated dialogue-style in
contrast to the commonly used monologue-style (Stranc and Muldner, 2019; Ding
et al., 2018; Nugraha et al., 2020; Lee and Muldner, 2020; Ohashi et al.,
2019). The last third of the experiments in this category investigated
animations (Cookson et al., 2020; Al-Khateeb and Alduwairi, 2020; Srivastava
et al., 2020, 2019).

Visual and audio features: This kind of experiments primarily addressed
characteristics that attract the attention of the viewer, here called
representation characteristics (Wang et al., 2020b; Moon and Ryu, 2021; Pi et
al., 2019; Wang et al., 2019b; Pi et al., 2017b; Sharma et al., 2016c).
Experiments on representation characteristics have clearly focused on content-
related cues that direct attention (e.g., arrows, moving elements, color cues,
gaze overlay) (Wang et al., 2020b; Moon and Ryu, 2021; Pi et al., 2019; Sharma
et al., 2016c) and instructor-related elements that direct attention (e.g.,
pointing gestures, spoken words, directed eye gaze) (Wang et al., 2020b; Moon
and Ryu, 2021; Pi et al., 2019; Wang et al., 2019b; Pi et al., 2017b; Sharma
et al., 2016c). Other experiments studied the impact of the complexity of
visual content (Guervós et al., 2019) and the gender appearance of the
instructor (Hoogerheide et al., 2018).

Instructor behavior characteristics: In controlled experiments, the behavior
of the instructors has been studied principally with respect to their gestures
(Beege et al., 2020, 2019; Wang et al., 2019a; Horovitz and Mayer, 2021; Li et
al., 2016). Less studied characteristics were related to the speech (e.g.,
tone of voice (Horovitz and Mayer, 2021)), other visual aspects such as pose
(e.g., frontal, lateral), and dressing style (Beege et al., 2019).

Other characteristics: Some articles studied the pace at which videos are
viewed (Merkt et al., 2018; Schroeder et al., 2020; Garrett, 2021; Lang et
al., 2020; Garrett, 2018). Among the less popular researched topics, we found
experiments with captioned and non-captioned videos (Ozdemir et al., 2016;
Whitney and Dallas, 2019), incorporating music (Schreiber and Dougherty,
2017), summaries at the end of video (van der Meij and van der Meij, 2016),
the inclusion of sign language (Kikusawa et al., 2016), and experiments that
test Multimedia Learning Theory such as the effect of pre-training material
and spatial contiguity between captions and visual elements (Zhang, 2021; van
der Meij, 2019).

####  5.2.3. Dependent Variables

In controlled experiments, the dependent variable measured the impact of
specific video characteristics on different learning aspects. We have reviewed
all the dependent variables and categorized them into different groups
according to their purpose. In this way, we have obtained the following
categories that are related to:  (a) self-perceived effectiveness (e.g., self-
efficacy, instructional efficiency) (Hew and Lo, 2020; Hoogerheide et al.,
2018; Keller et al., 2019; Cojean and Jamet, 2017; van der Meij and van der
Meij, 2016; da Graça Campos Pimentel et al., 2019; Schroeder et al., 2020;
Moos and Bonde, 2016; Lee and Muldner, 2020; van der Meij, 2019; Huang et al.,
2020; Sözeri and Kert, 2021),  (b) difficulty (e.g., cognitive load, time
effort) (Cao et al., 2018; Keller et al., 2019; Hoogerheide et al., 2018;
Cojean and Jamet, 2017; da Graça Campos Pimentel et al., 2019; Beege et al.,
2020; Lu et al., 2021; Srivastava et al., 2020, 2019; der Zee et al., 2017;
Moos and Bonde, 2016; Merkt et al., 2018; Beege et al., 2019; Garrett, 2021;
Lee and Muldner, 2020; Chen et al., 2021; Davila et al., 2021; Pi et al.,
2017b; Moon and Ryu, 2021; Huang et al., 2020; Pi et al., 2017a),  (c) general
learning experience (e.g., lecture experience, learning experience) (Stull et
al., 2018; Li et al., 2016; Palaigeorgiou and Papadopoulou, 2019; Shelton et
al., 2016; Kleftodimos and Evangelidis, 2018; van Alten et al., 2020; Cookson
et al., 2020; Pérez-Navarro et al., 2021; Lu et al., 2021; Wang et al., 2019a;
Huang et al., 2020; Pi et al., 2017a; Cao et al., 2018; Chen, 2020),  (d)
emotions and feelings of the viewer (e.g., satisfaction, enjoyment) (Wang et
al., 2019a; Horovitz and Mayer, 2021; Araiza-Alba et al., 2021; Hoogerheide et
al., 2018; Srivastava et al., 2020, 2019; Schreiber and Dougherty, 2017;
Stranc and Muldner, 2019; Cojean and Jamet, 2017; Guervós et al., 2019),  (e)
instructor (e.g., enthusiasm, presentation skills) (Li et al., 2016;
Hoogerheide et al., 2018; Al-Khateeb and Alduwairi, 2020; Hew and Lo, 2020;
Cojean and Jamet, 2017; Lee and Muldner, 2020; Davila et al., 2021; Pi et al.,
2017a; Wang et al., 2019b; Cao et al., 2018),  (f) video and its content
(e.g., quality, usability) (Guervós et al., 2019; Kikusawa et al., 2016;
Palaigeorgiou and Papadopoulou, 2019; Ohashi et al., 2019; Keller et al.,
2019; Daher and Sleem, 2021; Shelton et al., 2016; van der Meij and van der
Meij, 2016) (g) engagement with the video (e.g., motivation, engagement)
(Muñoz-Carpio et al., 2020; Cookson et al., 2020; Winkler et al., 2020;
Shelton et al., 2016; Leisner et al., 2020; Araiza-Alba et al., 2021; Lee and
Muldner, 2020; Srivastava et al., 2019; Wang et al., 2019b; Lu et al., 2021;
Sharma et al., 2016c; Chen, 2020; Huang et al., 2020; Horovitz and Mayer,
2021; Ozdemir et al., 2016; Keller et al., 2019),  (h) learning performance
(e.g., learning quality, learning outcome) (Keller et al., 2019; Daher and
Sleem, 2021; Cojean and Jamet, 2017; Merkt et al., 2018; Beege et al., 2019;
Kleftodimos and Evangelidis, 2018; van Alten et al., 2020; Leisner et al.,
2020; Chen, 2020; Ding et al., 2018; der Zee et al., 2018; Wang et al., 2020b;
Schroeder et al., 2020; Moon and Ryu, 2021; Sözeri and Kert, 2021; Cookson et
al., 2020; Al-Khateeb and Alduwairi, 2020; Araiza-Alba et al., 2021; Taskin et
al., 2019; Lu et al., 2021; Nugraha et al., 2020; Wang et al., 2019a; Kuhail
and Aqel, 2020; Huang et al., 2020; Garrett, 2021; Lee and Muldner, 2020; Lang
et al., 2020; Srivastava et al., 2020; Horovitz and Mayer, 2021; Davila et
al., 2021; Pi et al., 2019; Stull et al., 2018; Wang et al., 2019b; Lin et
al., 2018; Pi et al., 2017a, b; Schreiber and Dougherty, 2017; Li et al.,
2016; Zhang, 2021; Sharma et al., 2016c; der Zee et al., 2017; Wachtler et
al., 2016a; Cojean and Jamet, 2017; Ozdemir et al., 2016; Hoogerheide et al.,
2018; Garrett, 2018; Ku et al., 2019; Palaigeorgiou and Papadopoulou, 2019;
Moos and Bonde, 2016; Hew and Lo, 2020; van der Meij and van der Meij, 2016;
Shelton et al., 2016; da Graça Campos Pimentel et al., 2019; Whitney and
Dallas, 2019; Llanda, 2019a; Cao et al., 2018; van der Meij, 2019), and  (i)
other (e.g., cybersickness, perceived similarity of the learner to the
instructor) (Huang et al., 2020; Guervós et al., 2019; Cojean and Jamet, 2017;
Beege et al., 2019; Ohashi et al., 2019; Hoogerheide et al., 2018; Hew and Lo,
2020).

Specifically, within these categories, the evaluated variable in almost every
experiment was the learning outcome (86% of controlled experiments), followed
by cognitive load (20%), satisfaction (19%), mental effort (11%), social
presence (learners’ perception of their own social presence) (11%), motivation
(9%), and self-efficacy (9%). Please note that several studies measured the
effects on more than one dependent variable.

The learning outcome has been studied under different names such as learning
gain, academic achievement, etc. Lately, this metric has been measured not
only by testing how much the learner can remember (_recall_) but also to what
extent the learner can apply the gained knowledge to new problems
(_transfer_). Our results showed that transfer tests have been more frequently
used in the last four years than in the previous years (Cojean and Jamet,
2017; Merkt et al., 2018; Beege et al., 2019; van Alten et al., 2020; Ding et
al., 2018; Wang et al., 2020b; Schroeder et al., 2020; Lu et al., 2021; Kuhail
and Aqel, 2020; Garrett, 2021; Pi et al., 2019; Zhang, 2021; Garrett, 2018; Ku
et al., 2019; Hew and Lo, 2020). However, we found that most of the articles
did not report explicitly if the type of test was recall or transfer, only 38%
of experiments reported it (Cojean and Jamet, 2017; Merkt et al., 2018; Beege
et al., 2019; van Alten et al., 2020; Ding et al., 2018; Wang et al., 2020b;
Schroeder et al., 2020; Lu et al., 2021; Kuhail and Aqel, 2020; Pi et al.,
2019; Zhang, 2021; Ku et al., 2019; Hew and Lo, 2020; Kleftodimos and
Evangelidis, 2018; Araiza-Alba et al., 2021; Nugraha et al., 2020; Davila et
al., 2021; Li et al., 2016; Whitney and Dallas, 2019; Garrett, 2021, 2018).

In addition, we found that dependent variables were derived from tracking
facial expressions (Srivastava et al., 2019), neurophysiological aspects
(e.g., galvanic skin) (Winkler et al., 2020), thermal aspects (Srivastava et
al., 2019, 2020), log data (da Graça Campos Pimentel et al., 2019; Lang et
al., 2020), and eye gaze (Nugraha et al., 2020; Lee and Muldner, 2020; Stull
et al., 2018; Wang et al., 2019b; Kikusawa et al., 2016; Pi et al., 2019;
Zhang, 2021; Sharma et al., 2016c; Moon and Ryu, 2021; Pi et al., 2017b;
Srivastava et al., 2019). The last one was the most used technique, from which
popular metrics such as dwell time have been derived to measure, for example,
attention.

###  5.3. Data Analysis Studies

The majority of data analysis studies focused on the understanding of patterns
in learning behavior. Furthermore, around half of the data analysis papers
have also tried to understand the impact of those patterns on a specific
variable that measures learning effectiveness (Eradze et al., 2020; Stöhr et
al., 2019; Wynn et al., 2019; Granjo and Rasteiro, 2018; Costley et al., 2021;
Angrave et al., 2020; Li, 2019; Rodríguez et al., 2018; Costley and Lange,
2017; Brinton et al., 2016; Kovacs, 2016; Lee et al., 2017a; Hildebrand and
Ahn, 2018; Shoufan, 2019b; Galster et al., 2018; Thornton et al., 2017;
Atapattu and Falkner, 2017). In the same way as in the controlled experiment
papers, the impact of video characteristics has been studied mostly on the
learning outcome (Costley and Lange, 2017; Stöhr et al., 2019; Wynn et al.,
2019; Angrave et al., 2020; Shoufan, 2019b; Galster et al., 2018; Rodríguez et
al., 2018; Brinton et al., 2016; Kovacs, 2016; Hildebrand and Ahn, 2018;
Thornton et al., 2017; Li, 2019) and on engagement (Granjo and Rasteiro, 2018;
Li, 2019; Costley and Lange, 2017; Galster et al., 2018; Thornton et al.,
2017; Atapattu and Falkner, 2017; Hildebrand and Ahn, 2018).

Figure 8. The diagram displays the number of papers with regard to video
characteristics explored in their data analysis studies. These studies usually
made use of log data of learner’s activities.

Figure 8 sorts the characteristics studied in data analysis articles according
to their frequency. We found that data analysis studies usually made use of
learner’s activity characteristics by taking advantage of system logs (e.g.,
play, pause) (Eradze et al., 2020; Saurabh and Gautam, 2019; Shoufan, 2019a;
Stöhr et al., 2019; Wynn et al., 2019; Costley et al., 2021; Angrave et al.,
2020; Li, 2019; Rodríguez et al., 2018; Costley and Lange, 2017; Giannakos et
al., 2016; Hu et al., 2020; Uchidiuno et al., 2018; Kovacs, 2016; van der
Sluis et al., 2016; Lei et al., 2019; Shoufan, 2019b; Meseguer-Martinez et
al., 2019; Yan and Baxter, 2018; Atapattu and Falkner, 2017; Lee et al.,
2017a; Lei et al., 2017; Dodson et al., 2018a; Patel et al., 2018). Other
characteristics often analyzed were metadata and video transcripts (Eradze et
al., 2020; Saurabh and Gautam, 2019; Shoufan, 2019a; Ding, 2018; Shi et al.,
2019; Lee et al., 2017a; Silva and Azevedo, 2017; Hildebrand and Ahn, 2018;
Sjödén et al., 2018; van der Sluis et al., 2016; Meseguer-Martinez et al.,
2019; Galster et al., 2018; Atapattu and Falkner, 2017). Moreover, data
analysis studies have inspected production style characteristics (Ding, 2018;
Costley and Lange, 2017; Hildebrand and Ahn, 2018; Rahim and Shamsudin, 2019;
Thornton et al., 2017; Shoufan, 2019a; Lei et al., 2019; Shoufan and Mohamed,
2017), audio features (Shi et al., 2019; Shoufan, 2019a; Ding, 2018; Shoufan
and Mohamed, 2017) and visual features (Shoufan, 2019a; Ding, 2018; Shoufan
and Mohamed, 2017), instructor’s behavior characteristics (Ding, 2018; Tian
and Bourguet, 2016; Shoufan, 2019a; van der Sluis et al., 2016; Atapattu and
Falkner, 2017; Shoufan and Mohamed, 2017; Patel et al., 2018), design
principles and guidelines (Eradze et al., 2020; Shoufan, 2019a; Ou et al.,
2016; Lei et al., 2019), and interactive features (Ding, 2018; Granjo and
Rasteiro, 2018). The corresponding findings are presented in the next section
(Section 6).

###  5.4. Design Guidelines

General guidelines: Wijnker et al. (Wijnker et al., 2019) developed a
framework that applies a film theory perspective to learning videos. The
authors provided guidance to analyze and categorize these videos according to
their design, aim and type in film theory (Bordwell et al., 1993). Voronkin
(Voronkin, 2019) proposed universal guidelines focusing on methodological,
psychological, didactic, ergonomic, and technical aspects. Lange and Costley
(Lange and Costley, 2020) considered media delivery problems and addressed
them by proposing guidelines that aim at improving the pace, intelligibility,
quality, media diversity, and congruence. The authors grounded their
guidelines in the limited capacity theory in multimedia learning (Mayer,
2014), generative theory (Mayer, 2014), and extraneous processing research
(Leppink et al., 2013). Dodson et al. (Dodson et al., 2018b) introduced a
framework of active viewing that categorized learner’s watching behavior. The
authors stated that VBL can be designed in a way to promote this behavior.

Guidelines for specific disciplines: Minnes et al. (Minnes et al., 2019)
proposed design guidelines for producing short learning podcast videos for
computer science courses and emphasized the use of representation
characteristics, especially call-out boxes, zooming, and animations. Hodges
(Hodges, 2018) also collected several recommendations for creating videos in
CS. The authors emphasized the need of captions, descriptions of animations as
they occur, zooming in relevant code, and for video systems that allow
learners to pause, seek, and speed up or slow down the video. Stephenson
(Stephenson, 2019) concentrated on the development of programming videos
following a specific content structure. Lei et al. (Lei et al., 2016)
discussed an approach to produce videos for Humanities disciplines. They
suggested to favor short videos, use animations, highlight key concepts and
terminologies, include an introduction and conclusions, employ consistent
topography and graphic styles, and to add music in specific parts. Ge and Li
(Ge and Li, 2020) identified beneficial characteristics to learn English as a
foreign language. They recommended to take care of the speech rate, sentences
and vocabulary complexity, use of closely related examples, use of realistic
visuals when higher levels of context are demanded, include the instructor,
and use keyword subtitles instead of complete subtitles.

Guidelines for videos with interactive features Weinert et al. (Weinert et
al., 2020) introduced reference guidelines to design Interaction Theory
(Moore, 1989), Cognitive Load Theory (Sweller, 1994), and own research. The
authors recommended to include elements for interacting not only with the
content but also with other learners and the instructor. They also discussed
recommendations for the best positions to place specific types of interactive
features. Hung et al. (Hung et al., 2018) proposed guidelines to design
interactive tasks for learning video content based on literature on embodied
cognition. The authors propose that the video should include engaging,
prompting, experiencing, facilitating, demonstrating, and questioning tasks.

##  6\. Impact of video characteristics on the learning effectiveness

In this section, we contrast, analyze, and summarize the findings of studies
from controlled experiments and data analysis research. These are organized
according to the different categories and subcategories of video
characteristics.

###  6.1. Instructor Behavior Characteristics

Gestures: Study results showed that gestures do not have a significant impact
on the learning outcome (Beege et al., 2020; Wang et al., 2019a; Horovitz and
Mayer, 2021). One study even found that gestures can cause negative comments
(Ding, 2018). However, learners still care about gestures, facial expressions,
and body language (Shoufan, 2019a). Among the benefits, gestures were found to
reduce mental effort (Beege et al., 2020), increase presence (Beege et al.,
2020; Li et al., 2016) and satisfaction (Wang et al., 2019a). Moreover,
studies have obtained a positive effect on delayed learning outcome tests and
a higher impact of human gestures compared to gestures of instructors as
animations (human-like artificial agents) (Wang et al., 2019a; Horovitz and
Mayer, 2021).

Speech: Data analysis studies revealed that characteristics related to the
speech such as speaking rate, lexical diversity, etc., affected how learners
interacted with the video (Atapattu and Falkner, 2017). Speech characteristics
were also found to correlate with sound characteristics (Shi et al., 2019).
Further findings showed that speech characteristics had an impact on dwell
time (van der Sluis et al., 2016) and the level of understanding (Shoufan,
2019a). For example, whether the instructor’s language was native or not
significantly affected the level of understanding in the learner, and the
number of likes in the video (Shoufan, 2019a; Shoufan and Mohamed, 2017).
Also, pronunciation and intonation was found to affect the learner’s
attention. Based on these discoveries, it has been recommended to reduce
lexical diversity, long sentences, and speech rate (Atapattu and Falkner,
2017).

Other: Other characteristics related to instructor behavior which the learners
cared about and had an impact were the appearance, clothes, and enthusiasm
(Shoufan, 2019a; Ding, 2018). Interestingly, the gender did not have a
significant impact (Shoufan, 2019a), it did not affect the learning outcome,
the perception of self-efficacy (Hoogerheide et al., 2018), and learners
liking or disliking a video (Shoufan and Mohamed, 2017). Yet, it was observed
that learners perceived themselves more similar to the instructor when the
gender was the same (Hoogerheide et al., 2018). With regard to instructors as
animations, these were considered interesting and attractive (Ding, 2018).

###  6.2. Interactive Features

Quizzes and questions: Most of the studies that experimented with quizzes
evidenced that the learning outcome was positively affected (Kleftodimos and
Evangelidis, 2018; Leisner et al., 2020; Wachtler et al., 2016a; Palaigeorgiou
and Papadopoulou, 2019; da Graça Campos Pimentel et al., 2019). For
Palaigeorgiou and Papadopoulou (Palaigeorgiou and Papadopoulou, 2019), the
positive effect was found when quizzes were included together with other
interactive features. For Wachtler et al. (Wachtler et al., 2016a), the
positive effect was found when quizzes were placed at the right time.

Furthermore, positive impacts were found on other aspects such as satisfaction
(Kleftodimos and Evangelidis, 2018; Palaigeorgiou and Papadopoulou, 2019) and
engagement (da Graça Campos Pimentel et al., 2019). Data analysis studies have
also confirmed these findings. Kovacs (Kovacs, 2016) found that most viewers
engaged in quizzes, which seemed to reduce dropout. On the contrary, Ding
(Ding, 2018) showed that quizzes improved understanding of the content. But
they did not draw the learner’s attention in cases where the quizzes were
complex and at the end of the video.

Virtual reality and 360-degree characteristics: The studies in this categories
commonly reported an improved learning outcome, however, not to a significant
degree (Chen, 2020; Araiza-Alba et al., 2021; Daher and Sleem, 2021; Huang et
al., 2020). Huang et al. (Huang et al., 2020) found positive effects only in
certain parts of the post test. Although statistically insignificant, Araiza-
Alba et al. (Araiza-Alba et al., 2021) reported short- and long-term
improvements in learning outcome. Studies also showed differing impact of
virtual reality and 360-degree environments on learner motivation. While Huang
et al. (Huang et al., 2020) found no positive effect on the motivation, Muñoz-
Carpio et al. (Muñoz-Carpio et al., 2020) and Chen (Chen, 2020) found that
viewers felt indeed more motivated. Regarding satisfaction, several findings
confirmed an increase (Chen, 2020; Muñoz-Carpio et al., 2020; Huang et al.,
2020). Furthermore, Huang et al. (Huang et al., 2020) also found a positive
impact on self-efficacy. Our analysis over time shows that these kind of
characteristics have been investigated more frequently in the last years
(Chen, 2020; Muñoz-Carpio et al., 2020; Huang et al., 2020; Araiza-Alba et
al., 2021; Daher and Sleem, 2021). Naturally, experiments in this category
evaluated the impact in other non-common dependent variables such as sense of
”being there” (Araiza-Alba et al., 2021; Guervós et al., 2019) and
cybersickness (Guervós et al., 2019). Interestingly, most of these experiments
were conducted with non-STEM videos (Chen, 2020; Araiza-Alba et al., 2021;
Daher and Sleem, 2021; Huang et al., 2020).

Other: Most of the studies that experimented with in-video annotations found
an increase in the learning outcome (Leisner et al., 2020; Taskin et al.,
2019; Kuhail and Aqel, 2020), whereas neutral effects were found for cognitive
load (Chen et al., 2021). Regarding navigational features, experiments using
table content navigation (Kuhail and Aqel, 2020; Cojean and Jamet, 2017) and
timeslines with clickable points (da Graça Campos Pimentel et al., 2019;
Cojean and Jamet, 2017) found a positive effect on the learning outcome
(Kuhail and Aqel, 2020; da Graça Campos Pimentel et al., 2019; Cojean and
Jamet, 2017). However, this was not always significant (da Graça Campos
Pimentel et al., 2019), or depended on the combination with other features to
be more beneficial such as table content navigation together with timelines
with markers (Cojean and Jamet, 2017). According to Granjo and Rasteiro
(Granjo and Rasteiro, 2018), videos with interactive features needed to be
used with other pedagogical techniques to improve the learning outcome.

###  6.3. Visual and Audio Features

Instructor-related representational characteristics: Tian and Bourguet (Tian
and Bourguet, 2016) observed that the most common gestures that instructors
used as cues were pointing and extending (extend the arms away from the body
with palms up) cues. Results in evaluating the impact of these types of cues
have been shown to increase the learning outcome in most cases (Wang et al.,
2020b; Pi et al., 2019; Wang et al., 2019b; Pi et al., 2017b). However, no
significant effects (Sharma et al., 2016c) or even negative effects where
found when the instructor was an animation (Moon and Ryu, 2021).

Content-related representational characteristics: Examples of these
characteristics are arrows, color-coded cues, highlighted content, etc.
Experiments showed that these cues improve the learning outcome in most cases
(Wang et al., 2020b; Pi et al., 2017b; Sharma et al., 2016c), but had no
significant effect on cognitive load (Sharma et al., 2016c) or increased it
(Moon and Ryu, 2021). When comparing content-related cues with instructor-
related ones, the results were contradictory. In one case, instructor-related
cues had a higher positive impact on the learning outcome than content-related
cues when hand-pointing gestures were compared to arrows on slide-based videos
(Pi et al., 2017b). In another case, contrary results were found when
comparing hand-pointing gestures (a pencil following the handwriting) with
gaze overlay defined by an expert (explicit drawing of the path that learners
should follow with their eyes) (Sharma et al., 2016c).

Other visual and audio elements: General visual and audio features such as the
image and audio quality seemed to matter to the learners and caused positive
effects on the learning outcome (Ding, 2018; Schreiber and Dougherty, 2017).
In the experiments of Uchidiuno et al. (Uchidiuno et al., 2018), English
language students often avoided videos that did not have visual support
(figures, charts, equations, or on-screen text) but presented only an
instructor talking. Interestingly, Ding (Ding, 2018) found that music in the
background helped to keep students focused.

###  6.4. Production Style

Instructor presence: Experiments have shown that viewers preferred a video in
which the instructor appears (Pérez-Navarro et al., 2021; Davila et al.,
2021). Hew and Lo (Hew and Lo, 2020) observed that the three most widely used
styles in a sample of YouTube videos in computer science had an instructor:
(1) slide-based videos,  (2) classroom-style videos, and  (3) Khan-style
videos with talking head.  In addition, viewers voiced their preference for
videos that show the instructor’s hands (Pérez-Navarro et al., 2021). However,
although the presence of an instructor was preferred, it did not bring notable
advantages (Davila et al., 2021; Hew and Lo, 2020). Davila et al. (Davila et
al., 2021) and Hew and Lo (Hew and Lo, 2020) found that a talking head did not
show a significant effect on the learning outcome, the social presence (Hew
and Lo, 2020) and the cognitive load (Davila et al., 2021). Experiments have
also tested the effects of the instructor’s size in the video. A small size
seemed to cause higher learning gains (Pi et al., 2017a) and satisfaction (Cao
et al., 2018) when the topic was not complex. However, the size of the
instructor had no significant effect on cognitive load, mental effort (Pi et
al., 2017a; Cao et al., 2018) and social presence (Pi et al., 2017a; Cao et
al., 2018).

Dialogue-style vs. Monologue-style: The experiments did not find a significant
effect of dialogue-style over the monologue-style on learning gain (Ding et
al., 2018; Nugraha et al., 2020; Lee and Muldner, 2020). In addition, no
effect on the social presence was found (Lee and Muldner, 2020). Ohashi et al.
(Ohashi et al., 2019) conducted an experiment using videos in computer
programming topics, one using slides only, another one with slides accompanied
by two puppets in a dialogue. They found that the number of viewers of the
video including the dialogue decreased over time (throughout a compulsory
course). The authors argued that the puppets’ dialogue was effective at the
beginning, but not when the learners became increasingly familiar with the
course content. Nevertheless, the dialogue style seemed to offer benefits with
regard to self-efficacy (Lee and Muldner, 2020) and positive sentiment (Ding
et al., 2018).

Animations: Animations in videos caused higher learning outcomes when compared
to  (a) text-only videos (i.e., slide-based videos only with text, without
illustrations, figures, charts, and similar), (Srivastava et al., 2020) (b)
textbooks (Cookson et al., 2020), and  (c) traditional live classroom lectures
(Al-Khateeb and Alduwairi, 2020).  Animations also affected other aspects such
as participation (Cookson et al., 2020), cognitive load (Lee and Muldner,
2020), mental effort (Srivastava et al., 2020), and perceived difficulty
(Srivastava et al., 2019).

Video tutorials and video lectures: In several studies, learners believed that
video tutorials were more interesting, attractive, kept them engaged, and
helped them understand better (Ding, 2018; Ou et al., 2016). It was also found
that learners tended to skip theoretical discussions, which were usually found
in video lectures (Lei et al., 2019). However, other studies showed that
learners not only watched lectures significantly more often than tutorials
(Hildebrand and Ahn, 2018) but also watched them significantly more often to
completion (Thornton et al., 2017). In any case, Hildebrand and Ahn
(Hildebrand and Ahn, 2018) reported that both styles had a similar positive
correlation with the learning outcome.

Other: Other studies found that media diversity correlated positively with
learning outcome, satisfaction, engagement, and interest (Costley and Lange,
2017). Shoufan and Mohamed (Shoufan and Mohamed, 2017) reported that the most
predominant styles in CS videos were paper-based explanations, whiteboard-
based explanations, slide-based videos, and Khan-style videos.

###  6.5. Pace of the Video

Automatic pauses vs. non-automatic pauses: No significant positive effects of
automatic pauses in videos were found compared to non-automatic pauses (where
learners can pause the video, also called self-pacing) (Schroeder et al.,
2020). Indeed, systems with automatic pauses in videos were ineffective
(Garrett, 2021) or even increased cognitive load (Garrett, 2021, 2018). Merkt
et al. (Merkt et al., 2018) did not find advantages of automatic pauses even
in long lasting videos. On the contrary, viewers in the non-automatic pauses
condition were more successful and reported a lower mental effort (Garrett,
2018; Schroeder et al., 2020).

Video acceleration: The acceleration rate at which the video is watched was
found to be important for learners (Ding, 2018). Furthermore, there was
evidence that students tend to accelerate the videos at least once (Yan and
Baxter, 2018). However, the impact reported on the learning outcome has been
diverse. On the one hand, Lang et al. (Lang et al., 2020) showed that when the
learner accelerated the video, the learning outcome improved for learners in
Stanford Online (veena, 2020) (a platform of open online courses) in diverse
disciplines such as CS and mathematics. On the other hand, Garrett (Garrett,
2021) reported that learners who watched accelerated CS videos did not
outperform those who watched non-accelerated videos. However, in this
experiment the learner could not decide when to play, pause or accelerate the
video but this was given automatically by the system.

###  6.6. Beginners, Specialists, High and Low Performers

Learners show specific behaviors depending on their performance level with
respect to the learning topic. High achievers engaged in more diverse activity
patterns (e.g., play, pause seek) (Rodríguez et al., 2018), interacted more
intensely with quizzes embedded in the video (Kovacs, 2016), and watched
videos more constantly every week (Rodríguez et al., 2018). Galster et al.
(Galster et al., 2018) reported that students who were more active through in-
video annotations, also watched more videos. One study suggested that if low-
performance learners imitate this behavior, advantages could be attained
(Angrave et al., 2020). However, Galster et al. (Galster et al., 2018) did not
find differences in learning outcomes between more engaged and less engaged
learners.

Regarding the differences between beginners and specialists, the findings
showed that specialists engaged more often in different video activities than
beginners (Li, 2019) and achieved a higher learning outcome (Stöhr et al.,
2019). Nevertheless, the same level of engagement was found in both groups and
they also consumed the same amount of videos (Stöhr et al., 2019; Li, 2019).

###  6.7. Video Completion and Length

Video completion: It was reported that learners usually watched 30% to 50% of
the total video length (on a YouTube channel (Saurabh and Gautam, 2019) and on
the MOOC platform edX (Stöhr et al., 2019; edx, 2022)) and that skipping
through videos was a very common behavior (Hildebrand and Ahn, 2018). However,
studies in other learning contexts obtained different results. If the videos
were part of a university course, most of the learners watched the full video
(Yan and Baxter, 2018; Giannakos et al., 2016). Regarding the gain in the
learning outcome, studies showed that the extent to which a video was watched
impacted the student’s grades (Hildebrand and Ahn, 2018; Angrave et al., 2020)
and that viewers who watched the video to completion were more likely to watch
other videos (Wynn et al., 2019). Regarding dropping behavior, the findings
show that this occurred within the first five seconds. On the other hand, when
the learners watched the video for a longer time, dropping behavior seemed to
be reduced (Wynn et al., 2019). Wynn et al. found that when viewers surpassed
the first five minutes, they were likely to watch the video to completion
(Wynn et al., 2019). Strategies such as including in-video quizzes seemed to
reduce video dropout (Kovacs, 2016).

Video length: Several studies have concluded that short learning videos were
preferred by learners (Saurabh and Gautam, 2019; Ding, 2018; Rahim and
Shamsudin, 2019; Engeness and Nohr, 2019). Videos within five to seven minutes
were preferred in two studies (Engeness and Nohr, 2019; Ding, 2018). One of
the studies experimented with videos with talking head, PowerPoint slides,
drawings on a board, and animations for learning English as a foreign language
by undergraduate students (Ding, 2018). The other study experimented with
videos of a MOOC on digital competence that presented pedagogical aspects and
tutorials for teachers (Engeness and Nohr, 2019). Further findings in both
studies showed that although longer videos up to 10 minutes were not
preferred, they were still well received (Engeness and Nohr, 2019; Ding,
2018). Saurabh and Gautam (Saurabh and Gautam, 2019) found that the most
popular learning videos lasted 10 to 15 minutes. Nevertheless, contrary
findings presented by Shoufan (Shoufan, 2019b) showed that videos perceived as
too short were not well received. In this study, the videos were selected from
YouTube with different liking ratios (number of likes of a video divided by
the sum of likes and dislikes), different instructors, different speaking
accents, and covering different topics in digital logic design and embedded
systems targeted to students of an engineering program. For lecture styles,
Giannakos et al. (Giannakos et al., 2016) found that the preferred length of
videos in software engineering was close to the traditional 45-minute
classroom lectures for university students of a computer science degree. In
any case, the context is an important factor to consider to select an
appropriate video length (Shoufan, 2019b).

###  6.8. Video popularity

Saurabh and Gautam (Saurabh and Gautam, 2019) found that viewers tend to like
videos that helped them to learn and dislike the ones that did not. Shoufan
and Mohamed (Shoufan and Mohamed, 2017) discovered that the number of likes
for learning videos increase  (a) with higher video resolution,  (b) with
higher talking rates (words per second),  (c) when English was the native
language of the instructor,  (d) when the production style included paper-
based or a whiteboard-based explanations, and  (e) when more than one
production style was used.  Later, Shoufan (Shoufan, 2019b), found in a survey
that learners liked a video when they perceived that  (a) the explanation
seemed appropriate  (b) the presentation used animations, diverse colors,
understandable handwriting, and appropriate usage of supporting technology and
software,  (c) the video seemed interesting and easy to focus on,  (d)
practical examples were included  (e) the instructor was concise, used a
simple vocabulary, and  (f) had a confident and clear voice.  On the other
hand, in the same study, the authors (Shoufan, 2019b) found that learners
perceived that they disliked a video when the explanation was inappropriate,
there were no practical examples, the problem was solved in non-common ways,
unneeded details were included, the voice of the instructor seemed dull or
annoying, or the content was boring and difficult to focus on. Additionally,
in a different study, Shoufan (Shoufan, 2019a) analyzed a survey and found
that learners perceived that they disliked a video when they felt like they
did not understand the video content, when the technical quality of audio or
video was insufficient, when the instructor seemed incompetent, not friendly,
or not enthusiastic, or the length of the video was not appropriate.
Curiously, it was found that lengthy videos appeared to generate slightly less
dislikes (Shoufan and Mohamed, 2017).

Lee and Osop (Lee et al., 2017a) detected that popular videos have a high
amount of comments with strong negative sentiment. Moreover, Lei et al. (Lei
et al., 2019) found that popular videos had a high number of views but lower
retention rates. The authors found that popular videos usually provided a
general background, while less popular videos dived deeper into the topic,
which might be the reason why they had higher retention rates. Indeed, it was
warned that higher amount of views and comments might not necessarily entail a
good video quality (Lee et al., 2017a).

##  7\. Conclusions

Figure 9. the diagram shows all video characteristics identified in this
review

In this paper, we have reviewed 257 articles in the field of video-based
learning (2016-2021) following the PRISMA guidelines (Page et al., 2021).
Throughout the review, we  (a) identified major research directions  (b)
surveyed researched video characteristics and organized them into a taxonomy,
(c) reported on target tasks and technologies,  (d) revealed derived trends in
instructional design guidelines to produce videos, and  (e) contrasted
research findings and summarized the impact of the features on the learning
effectiveness.

Research directions: We have identified four research directions in the field
of VBL:  (1) proposals of tools,  (2) reports on controlled experiments,  (3)
data analysis studies, and  (4) design guidelines proposals.

The majority of papers proposed tools for the support of VBL, while a moderate
proportion of papers presented studies on experiments in controlled settings.
A small part of articles were dedicated to data analysis studies and design
guidelines.

Video characteristics: Table 3 presents the taxonomy we have built based on
the investigated characteristics, topics, modalities, etc. identified in the
research articles and their corresponding references. Figure 9 shows a ranking
of these video characteristics according to the number of related
publications. The most explored characteristics were textual features followed
by visual features, learner activity, interactive features. The most used
textual features were transcripts, which were mainly employed to develop
recommendation systems (see Section 5.1.3) or navigation tools (See Section
5.1.5), as a source to extract information (see Section 5.1.1), and for
summarization tools (see Section 5.1.7 and Figure 6). The most explored visual
features were video frames, essentially used to extract information from them.
Characteristics related to the learner activity took advantage of log data and
examined, for instance, play and pause activities. These were heavily explored
in data analysis articles (see Section 5.3 and Figure 3) and in some tool
proposals for developing predictors (see Section 5.1.4) and analytic tools
(see Section 5.1.6 and Figure 6). Interactive features (e.g., quizzes embedded
in the video, in-video annotations) were popularly scrutinized in experiments
to evaluate their effectivity (Kleftodimos and Evangelidis, 2018; van Alten et
al., 2020; Leisner et al., 2020; Keller et al., 2019; der Zee et al., 2018;
Sözeri and Kert, 2021; Taskin et al., 2019; Lu et al., 2021; Kuhail and Aqel,
2020; Chen et al., 2021; Winkler et al., 2020; Wachtler et al., 2016a; Cojean
and Jamet, 2017; Ku et al., 2019; Palaigeorgiou and Papadopoulou, 2019;
Shelton et al., 2016; Moos and Bonde, 2016; da Graça Campos Pimentel et al.,
2019; Chen, 2020; Muñoz-Carpio et al., 2020; Araiza-Alba et al., 2021; Daher
and Sleem, 2021; Huang et al., 2020; Guervós et al., 2019) but also were
integrated in tools for creating videos with interactive elements (see Section
5.1.2 Figure 3).

Moderately studied characteristics were related to audio features, production
style, and instructor behavior. Audio features specifically have been used to
develop tools (Mahapatra et al., 2018a; Ohnishi et al., 2019; Husain and
Meena, 2019; Soares and Barrére, 2018; Kanadje et al., 2016; Gandhi et al.,
2016; Kumar et al., 2017; Zhao et al., 2017; Mahapatra et al., 2018b; Bian and
Huang, 2019; Balasubramanian et al., 2016; Walk et al., 2020; Furini, 2018;
Jr. et al., 2019; Lin et al., 2019; Sharma et al., 2016b; Chatbri et al.,
2017; Furini, 2021; Sanchez-Cortina et al., 2016; Gopalakrishnan et al., 2017;
Mukherjee et al., 2019; Soares and Barrére, 2018; Kanadje et al., 2016; Ghauri
et al., 2020; Furini, 2018; Imran et al., 2019; Chen et al., 2020). Different
types of production style have been tested mainly in controlled experiments
(Stranc and Muldner, 2019; Ding et al., 2018; Cookson et al., 2020; Al-Khateeb
and Alduwairi, 2020; Pérez-Navarro et al., 2021; Nugraha et al., 2020; Lee and
Muldner, 2020; Srivastava et al., 2020; Davila et al., 2021; Srivastava et
al., 2019; Stull et al., 2018; Pi et al., 2017a; Hew and Lo, 2020; Ohashi et
al., 2019; Cao et al., 2018) (see Figure 3). Regarding instructor behavior,
speech-related characteristics (Horovitz and Mayer, 2021; Shoufan, 2019a;
Ding, 2018; van der Sluis et al., 2016; Atapattu and Falkner, 2017; Shoufan
and Mohamed, 2017; Patel et al., 2018; Mahapatra et al., 2018a; Soares and
Barrére, 2019; Balasubramanian et al., 2016; Jo et al., 2019; Jung et al.,
2019; Che et al., 2016; Jung et al., 2019; Ji et al., 2018; Bulathwela et al.,
2020) have been used more often than body-related characteristics (Beege et
al., 2020, 2019; Wang et al., 2019a; Horovitz and Mayer, 2021; Li et al.,
2016; Ding, 2018; Tian and Bourguet, 2016; Chen et al., 2020; Xu et al., 2020;
Godavarthi et al., 2020; Hoogerheide et al., 2018; Shoufan and Mohamed, 2017)
(see Figure 9). Less investigated characteristics were instructional design
characteristics, which were mainly treated in articles on guidelines for the
design and creation of videos (Minnes et al., 2019; Wijnker et al., 2019;
Weinert et al., 2020; Ge and Li, 2020; Lei et al., 2016; Voronkin, 2019; Lange
and Costley, 2020; Stephenson, 2019; Dodson et al., 2018b; Hodges, 2018; Hung
et al., 2018).

Tools, tasks and technologies: The main tools, pipelines, frameworks, and
methods that have been proposed for supporting VBL were related to information
extraction, integration of interactive features, recommendation systems,
classification and prediction, video navigation, and summarization (see
Section 5.1 Figure 5). Information extraction focused mainly on retrieving
text (Davila et al., 2021; Kota et al., 2020, 2019; Xu et al., 2019a; Lee et
al., 2017b; Garber et al., 2017; Zhong and Ma, 2018; Khattabi et al., 2019;
Davila et al., 2021; Khattabi et al., 2019; Kota et al., 2019; Lee et al.,
2017b; Davila and Zanibbi, 2018; Xu et al., 2019a; Garber et al., 2017; Zhong
and Ma, 2018; Khattabi et al., 2019), metadata (Jung et al., 2019; Escobar-
Avila et al., 2017; Jr. et al., 2019; Furini, 2018; Ghosh et al., 2021; Nangi
et al., 2019), and key information from videos such as key video segments and
key topics (Ghauri et al., 2020; Che et al., 2016; Kravvaris and Kermanidis,
2019; Lin et al., 2019; Liu et al., 2018b; Ponzanelli et al., 2019). Some
approaches used deep learning for information extraction (Davila et al., 2021;
Khattabi et al., 2019; Che et al., 2016; Lin et al., 2019), but also other
(shallow) machine learning methods based on hand-crafted features were still
present (Che et al., 2016; Liu et al., 2018b; Ponzanelli et al., 2019).
Knowledge bases (Jr. et al., 2019; Ghosh et al., 2021; Nangi et al., 2019) and
natural language (Jung et al., 2019; Escobar-Avila et al., 2017; Furini, 2018)
processing methods were also utilized.

The development of interactive tools mainly targeted the integration of in-
video annotations (feature that allow the learner to take notes while watching
the video) (Fong et al., 2018; Liu et al., 2019; Malchow et al., 2018; Lai et
al., 2018; Singh et al., 2016; Nguyen and Liu, 2016; Deb et al., 2017),
quizzes (Herault et al., 2018; Kohen-Vacs et al., 2016; Walk et al., 2020;
Wachtler et al., 2016b; Ma and Ma, 2019; Wachtler and Ebner, 2019), and,
interestingly, also feedback to the learner for preventing mind wandering (Lin
and Chen, 2019; Jo and Lim, 2017; Robal et al., 2018; Sharma et al., 2016a;
Llanda, 2019b). A few proposals introduced tools to create videos with
360-degree characteristics (Herault et al., 2018; Koumaditis and Chinello,
2019; Berns et al., 2018).

Recommendation systems have mainly used the similarity between transcripts and
metadata to suggest videos (Zhao et al., 2018a; Waykar and Bharathi, 2017;
Bleoanca et al., 2020; Basu et al., 2016b; Jung et al., 2019; Ji et al., 2018;
Jordán et al., 2020; Walk et al., 2020; Tavakoli et al., 2020). A popular
approach was to extract topics, concepts, and keywords to compute the
similarity to other videos. The use of ontologies and external knowledge bases
was used particularly to extract concepts (Schulten et al., 2020; Borges and
dos Reis, 2019; Coccoli and Vercelli, 2016; Borges and Silveira, 2019). Some
interesting approaches considered other characteristics such as video
readability (Jung et al., 2019), the skills demanded by the labor market
(Tavakoli et al., 2020), video popularity (Kravvaris and Kermanidis, 2017;
Zhang et al., 2018b), videos from the same instructor (Guan et al., 2019;
Imran et al., 2019), learning style (Ciurez et al., 2019), and pedagogical
characteristics (Acuña-Soto et al., 2020).

For classifiers and predictors the main task was to predict the learning
outcome using learner’s activity data (Lemay and Doleck, 2020; Mbouzao et al.,
2020; Mubarak et al., 2021; Furukawa et al., 2020; Korösi et al., 2018; Lallé
and Conati, 2020; Brinton et al., 2016). Other, but less frequent tasks were
discipline classification (Stoica et al., 2019; Dessì et al., 2017; Basu et
al., 2016a; Othman et al., 2017; Chatbri et al., 2017), instructor behavior
prediction (Chen et al., 2020; Godavarthi et al., 2020; Sharma et al., 2016b;
Xu et al., 2020), and dropout prediction (Jeon and Park, 2020; Furini et al.,
2021).

Navigation tools popularly proposed table content navigation styles (Das and
Das, 2020; Kravvaris and Kermanidis, 2019; Mahapatra et al., 2018a; Mukherjee
et al., 2019; Husain and Meena, 2019; Gandhi et al., 2016; Kumar et al., 2017;
Zhao et al., 2017; Yadav et al., 2016; Bian and Huang, 2019) but some
interesting approaches have emerged such as navigation according to keywords
(Ohnishi et al., 2019; Koka et al., 2020; Kravvaris and Kermanidis, 2019;
Kanadje et al., 2016; Kumar et al., 2017; Zhao et al., 2017) or visual
elements (Zhao et al., 2018b; Ma et al., 2017; Kumar et al., 2017; Zhao et
al., 2017; Yadav et al., 2016). These tools have relied heavily in
technologies to segment the video and to extract subtopics, concepts, and
keywords.

Analytic tools relied usually on characteristics of the learner’s activity to
understand video usage (Walk et al., 2020; Wachtler et al., 2016b; Long et
al., 2018) and contrasted it with the learners performance (Min et al., 2019;
He et al., 2018a, b). Some interesting tools analyzed the comments of videos
to understand the learners’ sentiment (Sung et al., 2016) and their engagement
(Stepanek and Dorn, 2017).

Summarization methods used commonly text transcripts (Vimalaksha et al., 2019;
Balasubramanian et al., 2016; Kim and Kim, 2016), however, there were also
approaches using (visual) frame-based video content (Rahman et al., 2020).
Relevant methods were related to the extraction of keywords, relevant
subtitles, entities, and metadata.

Design guidelines: Some of these studies proposed general guidelines (Wijnker
et al., 2019; Voronkin, 2019; Lange and Costley, 2020; Dodson et al., 2018b),
other suggested guidelines for specific disciplines (Minnes et al., 2019;
Hodges, 2018; Stephenson, 2019; Lei et al., 2016; Ge and Li, 2020), and two
cases suggested guidelines for creating interactive videos (Weinert et al.,
2020; Hung et al., 2018) . They have leveraged on theories and frameworks, for
example, film theory (Bordwell et al., 1993), limited capacity theory in
multimedia learning (Mayer, 2014), interaction theory (Moore, 1989), active
viewing framework (Dodson et al., 2018b), etc. These guidelines aimed at the
improvement of a wide variety of aspects such as pace, intelligibility,
quality, ergonomics, interaction, visuals, etc.

Experiments, data analysis, and their results on the effectiveness of video
characteristics: One of the major characteristics explored in controlled
experiments were interactive features. Other characteristics that were
experimented are related to production style, instructor behavior, and cues.
Learner’s activity characteristics have been specially inspected in data
analysis studies.

Interactive features focused mainly on quizzes, virtual reality and 360-degree
characteristics, and in-video annotations. Quizzes showed to be effective
(Kleftodimos and Evangelidis, 2018; Leisner et al., 2020; Wachtler et al.,
2016a; Palaigeorgiou and Papadopoulou, 2019; Ding, 2018; da Graça Campos
Pimentel et al., 2019; Kovacs, 2016), but not in all cases – sometimes they
were only effective when they were used together with other interactive
features and placed at the right moment (Wachtler et al., 2016a; Palaigeorgiou
and Papadopoulou, 2019; Ding, 2018). Virtual reality and 360-degree
characteristics seemed to be effective; however, the impact was not
significant (Chen, 2020; Araiza-Alba et al., 2021; Daher and Sleem, 2021;
Huang et al., 2020). In-video annotations seemed to improve the learning
outcome (Leisner et al., 2020; Taskin et al., 2019; Kuhail and Aqel, 2020) and
also navigational elements, specifically, table content navigation and
timelines with clickable points, impacted positively the learning outcome
(Kuhail and Aqel, 2020; da Graça Campos Pimentel et al., 2019; Cojean and
Jamet, 2017); however, results were not always statistically significant (da
Graça Campos Pimentel et al., 2019) or they were more beneficial when they
were together with other features (Cojean and Jamet, 2017). Indeed, further
claims state that videos with interactive features need to consider other
pedagogical techniques to improve the learning outcome (Granjo and Rasteiro,
2018).

Experiments on production styles investigated mainly the videos with
instructor’s presence, videos with animations, video tutorials vs. video
lectures, and dialogue-style vs. monologue-style. The learners preferred the
presence of an instructor, although the benefits were not always significant
(Pérez-Navarro et al., 2021; Davila et al., 2021; Hew and Lo, 2020; Pérez-
Navarro et al., 2021; Davila et al., 2021; Hew and Lo, 2020). Animations
caused higher learning outcomes when compared to text media and traditional
class (Srivastava et al., 2020; Cookson et al., 2020; Al-Khateeb and
Alduwairi, 2020). Tutorials and lecture styles correlated positively with the
learning outcome (Hildebrand and Ahn, 2018), however, preferences for one
style over the other were different between studies (Ding, 2018; Ou et al.,
2016; Lei et al., 2019; Hildebrand and Ahn, 2018; Thornton et al., 2017).
Dialogue-style instead of the typical monologue-style showed no significant
effects on learning outcome (Ding et al., 2018; Nugraha et al., 2020; Lee and
Muldner, 2020). In any case, the use of divers styles appeared to be effective
(Costley and Lange, 2017).

Studies on instructor behavior examined particularly human gestures and
speech-related characteristics, but very little other visual elements such as
pose and gender. Speech characteristics seemed to impact certain aspects of