11institutetext: Leipzig University, Faculty for Education, Leipzig, Germany
11email<EMAIL_ADDRESS>Leipzig University of
Applied Sciences (HTWK Leipzig), Leipzig, Germany 22email: andreas.thor@htwk-
leipzig.de

# Flexible Educational Software Architecture††thanks: This work was supported
by the German Federal Ministry of Education and Research for the tech4comp
project under grant No 16DHB2102.

at the example of EAs.LiT v2

Roy Meissner 11 0000-0003-4193-8209 Andreas Thor 22 0000-0003-2575-2893

###### Abstract

EAs.LiT is an e-assessment management and analysis software for which
contextual requirements and usage scenarios changed over time. Based on these
factors and further development activities, the decision was made to adopt a
microservice architecture for EAs.LiT version 2 in order to increase its
flexibility to adapt to new and changed circumstances. This architectural
style and a few adopted technologies, like RDF as a data format, enabled an
eased implementation of various use cases. Thus we consider the microservice
architecture productive and recommend it for usage in other educational
projects. The specific architecture of EAs.LiT version 2 is presented within
this article, targeting to enable other educational projects to adopt it by
using our work as a foundation or template.

###### Keywords:

Educational Software Software Architecture EAs.LiT Microservice Architecture
Semantic Data Evolving Software

##  1 Introduction

EAs.LiT is a web application handling e-assessment item and learning outcome
lifecycle, - management, - annotation, as well as automated exam creation and
exam analysis, as introduced by Thor et al. in [9]. In 2018 the German Federal
Ministry of Education and Research granted further development of the software
EAs.LiT as part of the tech4comp project. With respect to the demands of this
research project, but also as time passed by, new and adapted requirements and
usage scenarios arose for EAs.LiT. The original software was built using a
monolithic system architecture and imposed several challenges regarding
maintenance, expandability, reusability, and repurposibility as of the evolved
context, like argued by Newman in [7] for monolithic software systems in
general. Thus we decided to develop EAs.LiT version 2 with a different and
more suitable software architecture, namely the microservice architecture,
which tackles the named topics. It offers even more advantages in comparison
to a monolithic architecture, like argued in detail by Newman in [7].

We consider the microservice architecture productive and recommend it for
usage in other educational projects. Thus the focus of this paper is to
present the architecture of EAs.LiT version 2 in section 3, as well as its
accompanying technologies RDF and JSON-LD. Furthermore we showcase in section
4 the implementation of specific use cases, which were eased by the
architecture and used technologies. Our main goal is to enable educational
projects to adopt the architectural style, using our work as a template. To
achieve this goal, we additionally present in section 2 related work that
focuses on software projects which also use a microservice architecture. We
end by summarizing our approach and emphasizing unique features, as well as
presenting future development topics.

##  2 Related Work

SlideWiki, which may be considered a Github for slides in the educational
context, uses a microservice architecture, which was presented by Khalili et
al. in [5]. They mainly focus on distributed development and collaboration of
teams. In contrast to their work and software we do not focus much the social
aspects of the architecture, but on technological aspects, like the use of
semantic technologies in all architectural layers. Furthermore we showcase
novel integration scenarios, like the use of spreadsheets as a batch
processing frontend.

Kumar et al. propose in [6] to utilise a microservice architecture for
e-learning systems. They unveil that it is a solution to several downsides of
current monolithic e-learning systems and constructed a simple, but
theoretical example. Similar to our own work, they propose to use several web
technologies, but are not providing an implemented proof of concept and only
scratch the topics surface.

Fazakas et al. present in [4] a novel virtual classware technology which uses
a microservices architecture. They mainly focus on their use case and
developed software and only lightly touch the software architecture topic.
Microservices are named an enabling technology for their solution and they
used similar web technologies to the ones we used. In contrast we focus less
on the built software, but much more on the architecture itself and why using
it enlarges the flexibility of the software system.

Rad et al. propose in [8] an "innovative technology that implements problem-
based learning and projects-based learning concepts". They utilise a cloud
infrastructure for their solution and run several microservices as part of it.
No exploration of the decision was presented and they seem to use it as it
fits their needs and is supported by the utilised cloud stack.

There are much more software architecture publications available, focusing on
microservices. In contrast to the presented literature these publications
focus on different aspects of the architecture, like scalability and
incremental updates, target different use cases, like delivering large amounts
of data to end users, and are not positioned in the educational software
context. Thus we do not present these here, but wanted to mention their
existence.

##  3 Architecture of EAs.LiT version 2

EAs.LiT version 2 implements a microservice architecture, as introduced and
argued in detail by Newman and Wolff in [7, 10]. This means that the software
system is not executed as a monolithic system, like EAs.LiT version 1, but is
composed of several independent and fine grained services. These services are
built using a domain-driven design approach, described by Evans in [3]. Each
service implements a distinct and enclosed domain of the whole system. As of
the current project progress there are services for the domains: 1) item
management, 2) data format conversion and data import/export, 3) media
management, 4) the end user frontend, 5) data management (database), 6) item
annotation, 7) traffic management (a reverse proxy), and 8) item batch
editing. Within the following paragraphs we present and discuss all parts of
the new EAs.LiTs’ architecture, using a birds’-eye view depicted with figure
1.

Figure 1: Bird’s-eye view of the EAs.LiT version 2 architecture

Figure 1 shows in the upper left corner two end user applications: the main
frontend of EAs.LiT (4), as well as a spreadsheet based item batch editing
interface (8). These two services share their end user focus and usage of
other systems services, but are built using different technologies and
targeting different use cases. 4 is built as a browser application and allows
usage of EAs.LiT by non technical user groups, i.e. by educators, by providing
an easy to use and modern user interface. To get an impression of the software
we refer to figure 3 from section 4.1. As EAs.LiTs’ main functionality is
intended for educators and psychologists, the frontend is a central part of
the whole project and is used for service composition. Javascript is used as a
programming language and it was built with a modern NodeJS stack, drawing upon
Vue.js, React, Bootstrap and Webpack. Furthermore the frontend utilises a Flux
architecture and JSON-LD, which we argue about below. 8 is also an end user
frontend, but in contrast to 4 it is a spreadsheets’ plugin, built for batch
editing items. We explain this special frontend and use-case in more detail in
section 4.2.

Service 7 (traffic management) is a reverse-proxy server and used to route
incoming and outgoing traffic correctly and to handle traffic encryption, rate
limiting, authentication and other related topics. It is a mediator (like
named in figure 1) or gatekeeper between software running on our
infrastructure and software executed elsewhere. Its purpose is to free service
developers from implementing the just named topics to each service and
allowing them to focus on their specific domain only. Currently we use a NGINX
based reverse-proxy server111Reverse-Proxy:
https://hub.docker.com/r/jwilder/nginx-proxy.

Microservices 1,2,3 and 6 focus on different distinct domains, like item
management and item annotation. All of these services implement a
representational state transfer (REST) interface, utilizing OpenAPI (formerly
known as swagger) for standardized interface descriptions. A REST interface
allows to easily incorporate services from each other, from the official
fronted (4), from specialized frontends (like 8) and also from third parties,
like presented in section 4.1. Apart of the two named shared technologies, the
various services are built using different frameworks and different
programming languages. This is a general aspect of the microservice
architecture and allows to utilise the various specialized features of
different frameworks, programming languages and also special abilities of
programmers, like argued about in [7]. For instance the import/export service
is built using Java, because of its stream processing and interface
capabilities. The annotation service is built using Python, as the specialized
framework scikit-learn is available for this language. The item management-
service is implemented using Javascript because of its native JSON interface
and developer skills.

Lastly we depicted various data storage services (5) in figure 1. We currently
use two graph databases for storing data from different services. In general
the microservice architecture allows to use different data storage engines for
different services and use-cases, e.g. a relational database where it is
appropriate. We have incorporated this possibility to separate the data of
EAs.LiT from the domain specific knowledge graphs, which are introduced in
section 4.3. Thus we use different database instances, but all are graph
stores as these fit our use cases best.

As of noticeable technologies, RDF 222RDF:
https://www.w3.org/TR/rdf11-primer/, serialized to JSON-LD333JSON-LD:
https://www.w3.org/TR/2019/CR-json-ld11-20191212/, is used as the central data
format through all of the above described services. These technologies impose
extra effort for data management and processing, but ease a lot of use-cases
on different levels, like presented in section 4.3. RDF is a open world data
format and thus easily extended with new data, without running into structural
limits like introduced by entity relationship models. JSON-LD, which is one
possible serialization of RDF data, adds a context to traditional JSON
documents. This additional context may be utilised by software like the
frontend (4), but may also be ignored and thus allows to treat the data as
traditional JSON data, like done by the import/export service (2). Furthermore
JSON-LD allows to decouple services on the data level itself, as traditional
JSON identifiers are optional and may easily be exchanged by using a different
context, as well as to restructure data as of the data input process, like
shown by Veltens in https://blog.codecentric.de/2018/07/datenaustausch-mit-
json-ld/. We are not only using RDF data, but Linked-Data444Linked-Data:
https://www.w3.org/wiki/LinkedData, which allows to decouple the executed
frontend from whole EAs.LiT instances, as received data identifies its origin.

##  4 Enabled Use-Cases

Within the following subsections we present special use cases and
implementations of these, which the described architecture from section 3
enabled and eased to implement. We treat these use cases as proof of concepts,
research prototypes and examples on how the named technologies may be
efficiently applied.

###  4.1 Mentoring Workbench Integration

The Mentoring Workbench (MW) is the main software product of the tech4comp
project (to be described in future work) and provides several web-components,
that are activatable in different places of a Learning Management System
(LMS). Some components are usable by students, others are usable by educators
and supervisors. One component the MW provides is a personal exam analysis
tool for students. It links available e-assessment items to exam results from
the LMS and pulls additional data from EAs.LiT to create a detailed analysis
of the individual exam results. The component relies on item annotations, such
as levels of Blooms taxonomy about the cognitive learning domain [2], manually
added by educators. The annotated items are pulled and processed by the MW and
merged with the LMS data in order to e.g. inform students about reached
performance levels.

The architecture presented in section 3 eased this process as the MW pulls in
items via the already existing item management API (1). It might not be
obvious, but this API has naturally evolved because of the microservice
architecture and the applied separation of domains. The frontend and the item
management are two separate domains, thus realised as different services,
which communicate through a REST API. So an API for item creation, update,
removal and retrieval already exists and is reused by the MW. In comparison to
EAs.LiT version 1, either a manual data export, a new API, or direct database
access for third party software needs to be implemented to achieve the same
functionality.

###  4.2 Spreadsheet based Batch Processing

As broached in section 3 we implemented an item batch processing interface as
a plugin for Microsoft Excel, depicted in figure 2. This plugin is written in
Visual Basic for Applications and automatically included as item sets are
exported from EAs.LiT via the import/export service or via the frontend.
Nevertheless the plugin is also usable on its own when provided with an
EAs.LiT instance reference (URLs for the item management and import/export
services). The plugin allows to fetch item sets from EAs.LiT, to use the
capabilities of spreadsheets for batch editing and processing and to upload
the updated items back to their origin. Spreadsheets efficiently allow various
batch editing capabilities for items, like to increment the number of given
points, to compare and exchange answer texts, to revise item annotations, like
presented in section 4.1, and to create a lot of similar items by copy and
paste operations. In comparison to EAs.LiT version 1, several manual steps for
data export and import via the browser interface would have been needed. Even
more importantly the already existing services supported this use case without
the need to apply any changes to their API.

Figure 2: Example of the batch processing interface

###  4.3 Domain Model Integration

One goal of the tech4comp project is to create preferably complete knowledge
graphs for specific domains, like the educational discipline. For various use
cases it is desirable to link learning outcomes and items to this graph, e.g.
to provide the ability to analyse the item/domain distribution. As broached in
section 3 items and learning outcomes managed by EAs.LiT are stored as RDF
data, as is the knowledge graph. We hinted in section 3 that RDF allows to
link data sets, but does not require to merge these in order to do so. So for
example a specific item is annotated with a specific domain from the knowledge
graph by adding the URI of this domain to the item. Thus the used data format
allows a fast and straight forward implementation of this use case. Because
RDF is an open-world data format, adding new data (even on a structural level)
will not break any existing services, like the item management service (1). It
even enables to utilise this new data, i.e. to offer new abilities to API
users and eventually frontend users.

Figure 3: EAs.LiTs item/domain distribution analysis interface

Based on these facts we realised a frontend component which allows to visually
analyse the item/domain distribution, depicted in figure 3. This component
merges the received item and learning outcome data, as well as a reduced view
of a knowledge graph, received directly from the knowledge graph store. The
result is presented as an explorable canvas to end users. Because of the RDF
technology, various RDF data serializations and the microservice architecture,
we were able to skip implementing complex merge strategies or rule sets as the
different data sets just need to be appended into a single data record prior
to being displayed.

##  5 Summary & Future Work

A particular adoption of the microservice architecture for an educational
software project has been presented and explained at the example of EAs.LiT
version 2. We focused on the concrete software architecture in order to enable
third party projects to learn about the architectural advantages and to adapt
it for their own software projects. Furthermore novel use-cases have been
discussed, which were either enabled or at least eased to realise because of
the used architecture. Several technologies are accompanying the various use
cases of EAs.LiT and thus we presented RDF and JSON-LD as key data formats,
used through all the architectural layers. Characteristics of these two
technologies complement benefits of the microservice architecture and also
enable and ease some of the presented real-world use-cases and prototypes.

As mentioned in section 3 we are using different data stores for different
use-cases. Our current data store lacks the native ability for version
management (of items). Thus we plan to incorporate the quit-store, a graph
store which is based on the distributed version management software Git,
presented by Arndt et al. in [1]. This new store fits natively into the
microservice architecture and may be used whenever end users need access to
former data versions. Furthermore it allows efficient exchange of data via the
git distributed repository strategy, usable for data sharing between different
EAs.LiT instances, e.g. hosted by different universities.

##  6 Supplemental Online Material

The application EAs.LiT consists of several services, licensed under Mozilla
Public License Version 2. The source code of these services is available as
different Gitlab repositories, listed as of this filtered project overview of
the tech4comp Gitlab group at https://gitlab.com/Tech4Comp?filter=eas.

## References

  * [1] Arndt, N., Naumann, P., Radtke, N., Martin, M., Marx, E.: Decentralized collaborative knowledge management using git. Journal of Web Semantics (2018). https://doi.org/10.1016/j.websem.2018.08.002, https://arxiv.org/pdf/1805.03721
  * [2] Bloom, B.S.: Taxonomie von Lernzielen im kognitiven Bereich. Beltz-Studienbuch, Beltz, Weinheim u.a., 3. aufl. edn. (1973), https://katalog.ub.uni-leipzig.de/Record/0001760453
  * [3] Evans, E.: Domain-driven design: tackling complexity in the heart of software. Addison-Wesley Professional (2004)
  * [4] Fazakas, B.P., Iuonas, O.C., Porumb, C., Iancu, B.: Collaborative learning tools for formal and informal engineering education. In: 2017 16th RoEduNet Conference: Networking in Education and Research (RoEduNet). pp. 1–6 (Sep 2017). https://doi.org/10.1109/ROEDUNET.2017.8123745
  * [5] de Graaf, K.A., Khalili, A.: Slidewiki microservice architecture for collaborative online system development. In: 10th ACM Conference on Web Science, WebSci 2018 (5 2018)
  * [6] Kumar, S.S., Shastry, P.M.M.: Database-per-service for e-learning system with micro-service architecture. In: 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon). pp. 705–708 (Aug 2017). https://doi.org/10.1109/SmartTechCon.2017.8358462
  * [7] Newman, S.: Building microservices: designing fine-grained systems. " O’Reilly Media, Inc." (2015)
  * [8] Rad, A.M., Popa, T.H., Mihon, V., Iancu, B.: Problem-based learning and project-based learning concepts and their applications to engineering education. In: 2017 16th RoEduNet Conference: Networking in Education and Research (RoEduNet). pp. 1–6 (Sep 2017). https://doi.org/10.1109/ROEDUNET.2017.8123746
  * [9] Thor, A., Pengel, N., Wollersheim, H.W.: Digitalisierte hochschuldidaktik: Qualitätssicherung von prüfungen mit dem e-assessment-literacy-tool eas.lit. In: Igel, C., Ullrich, C., Martin, W. (eds.) Bildungsräume 2017. pp. 179–184. Gesellschaft für Informatik, Bonn (2017)
  * [10] Wolff, E.: Microservices: flexible software architecture. Addison-Wesley Professional (2017)


WGMwgm QEqe EPep PMSpms BECbec DEde

# Designing Deep Networks for Scene Recognition

Zhinan Qiao  Xiaohui Yuan  University of North Texas, 1155 Union Cir, Denton,
TX 76203

###### Abstract

Most deep learning backbones are evaluated on ImageNet. Using scenery images
as an example, we conducted extensive experiments to demonstrate the widely
accepted principles in network design may result in dramatic performance
differences when the data is altered. Exploratory experiments are engaged to
explain the underlining cause of the differences. Based on our observation,
this paper presents a novel network design methodology: data-oriented network
design. In other words, instead of designing universal backbones, the scheming
of the networks should treat the characteristics of data as a crucial
component. We further proposed a Deep-Narrow Network and Dilated Pooling
module, which improved the scene recognition performance using less than half
of the computational resources compared to the benchmark network architecture
ResNets. The source code is publicly available on https://github.com/ZN-
Qiao/Deep-Narrow-Network.

###### keywords:

Convolutional Neural Network Scene recognition Efficient Learning

##  1 Introduction

Since the development of AlexNet [1], a number of variations of deep
Convolutional Neural Networks (CNNs) emerged. Motivated by the success of
studies [2, 3, 4], the networks become deeper by adding more convolutional
layers to improve performance for the targeted problems. On the other hand,
arguments on the benefits of increasing network width gained support from
researchers. Zagoruyko et al. [5] presented wider deep residual networks that
significantly improve the performance over ResNet [4]. The wider deep network
achieved state-of-the-art performance on ImageNet [6] and CIFAR [7], showing
consistently better accuracy than its ResNet counterparts. Xie et al. [8]
proposed a multi-branch architecture called ResNeXt by widening the residual
blocks and cooperating group convolution. In contrast to ResNet, ResNeXt [8]
demonstrated a performance boost on a larger ImageNet-5K set and the COCO
object detection dataset [9]. ResNeSt [10] preserved the wider network layout
and multi-branch strategy and introduced a modulated architecture to improve
the feature learning process. The proposed ResNeSt networks further improved
the performance on the ImageNet dataset.

Despite the benefits brought by increased depth and width (i.e., number of
channels), side effects such as the requirement of large number of training
examples make it difficult to employ a large-scale network for learning from
complex scenery images. Is a deeper network more suitable for extracting
features from scenery images for better understanding complex views? Or shall
we employ more channels to achieve improved performance? To answer these
questions, we need to have a better understanding of the functions of network
layers and channels. A number of studies have been conducted in the past
years. Lu et al. [11] argued that an integration of both depth and width
provides a better understanding of the expressive power of neural networks.
Tan and Le [12] showed that it is critical to balance the network depth and
width by maintaining a constant depth/width ratio and demonstrated the
effectiveness of their approach on ResNet and MobileNet. Besides manually
designed networks, deep Neural Architecture Search were proposed to optimize
the network depth and width [13, 14]. However, most methods, if not all, were
developed and evaluated using ImageNet [6] and CIFAR [7]. These datasets
commonly depict an object near the center of the image, and the label tells
what the object is. That is, such an image is mostly dominated by one object
and, hence, is referred to as “object-centric”. On the other hand, a scene
image presents a complex view that consists of multiple objects and background
clutters. This inadvertent data bias could potentially lead to the ignorance
of the characteristics of different data. The features that are crucial for
object recognition dominantly affect the design of the CNNs.

Deep networks with more layers use a variety of receptive fields to extract
distinctive scale features whereas networks with more channels capture fine-
grained patterns [12]. However, in many applications, both types of
information are crucial for accurately recognizing an image, but the
prominence difference between them is seldom explored.

This paper attempts to bring out a new perspective on neural network design:
the data itself has the preference. Learning the overall spatial layout is
crucial to recognize the entire scene, thus scene recognition favors the
networks that can better learn the spatial information. For the object-centric
images, typical examples only consist of one single object, the spatial layout
does not contribute much to the semantic meaning of the image. As the
differences between certain object categories are subtle, the detailed
patterns and textures of objects are likely to be more representative. The
networks that emphasize learning various features can better fit the
requirement of object recognition tasks. By considering the distinct
characteristics of scene images, our hypothesis is that for scene recognition,
learning spatial-wise information improves the performance of CNNs in a more
effective manner compared to learning channel-wise information. To evaluate
this hypothesis, we conducted comprehensive experiments and our results show
scene images gain clear benefits from deepening the network, and the
performance change caused by altering the width is marginal. We further
proposed a Deep-Narrow Network, which increases the depth of the network as
well as decreased the width of the network. We design a Dilated Pooling
component and use it in our Deep-Narrow Network to extract spatial features.
Our design presents an efficient approach to scale the network based on the
data.

We summarize our key contributions as follows:

  1. 1.

We argue that the primary factor in designing neural networks should be the
data being used, rather than designing universal network architectures.
Learning the spatial layout is important for scene recognition tasks, while
learning various features is more beneficial for object recognition tasks. We
conducted experiments to validate our hypothesis.

  2. 2.

We analyze the effect of depth and width in CNNs from the perspective of image
characteristics and demonstrate that, compared to wider networks, deeper
networks are more suitable for performing scene recognition tasks. To prove
this hypothesis, we conducted extensive experiments to analyze the underlying
factors affecting the performance. Our findings suggest that spatial
information is crucial for scene recognition. Networks that can learn spatial
information effectively perform better in recognizing the overall spatial
layout of a scene. This is in contrast to object-centric images, where the
detailed patterns and textures of objects are more representative.

  3. 3.

Based on our hypothesis, we proposed the Deep-Narrow Network, an efficient
approach to scale the network by increasing its depth while decreasing its
width. This design includes a Dilated Pooling component to extract spatial
features. Our experimental results show that deepening the network provides
clear benefits for scene images, with marginal performance changes caused by
altering the width. This approach provides an efficient way to scale the
network based on the data characteristics.

##  2 Related Work

Deep Networks Network depth has played an integral role in the success of
CNNs. With the increase of depth, the network can better approximate the
target function with richer feature hierarchies, which enables the boost of
performance. The success of VGGNet [2] and Inception [15, 3, 16] on ILSVRC
competition further reinforced the significance of the depth. ResNet [17],
which is a continuance work of deeper networks, revolutionized the possible
depth of deep networks by introducing the concept of residual learning and
identity mapping into CNNs. ResNet’s effective methodology enables the network
to be extremely deep and demonstrated improved performance in image
recognition tasks.

Wide Networks Network width has also been suggested as an essential parameter
in the design of deep networks. Wide ResNet [5] introduced an additional
factor to control the width of the ResNet. The experimental results showed
that the widening of the network might provide a more effective way to improve
performance compared to making ResNet deeper. Xception [18] can be considered
as an extreme Inception architecture, which exploits the idea of depth-wise
separable convolution. Xception modified the original inception block by
making it wider. This wider structure has also demonstrated improved
performance. ResNeXt [8] introduced a new term: cardinality to increase the
width of ResNet and won the 2016 ILSVRC classification task. With the success
of ResNeXt, it is widely accepted that widening the deep network is an
effective way to boost model performance. A most recent wider network: ResNeSt
[10] preserved the wide architecture of ResNeXt and achieved superior
performance on image and object recognition tasks.

Effects of Depth and Width Although depth and width are proven to be essential
parameters in network architecture design, the effect of depth and width,
i.e., what do deep and wide networks learn remains seldom explored. Most of
the existing literature focus on the effect of width and depth separately or
the trade-off between depth and width in the network design [11]. Tan and Le
[12] claimed that deep networks can make use of a larger receptive field while
wide networks can better capture fine-grained features. Nguyen et al. [19]
explored the effects of width and depth and found a characteristic structure
named block structure. They demonstrated that for different models, the block
structure is unique, but the representations outside the block structure
trends to be similar despite the setting of depth and width. In our paper, we
analyze the effect of depth and width in CNNs from the perspective of image
characteristics.

Scene Recognition Integrating multiple networks serves as the pioneer scene
recognition study in the era of deep learning and is widely adopted in the
community [20, 21, 22]. Network outputs are concatenated or integrated
together to generate a uniform scene feature representation and demonstrated
improved performance [23, 24, 6, 25, 21, 26, 27, 22]. In these designs, pre-
trained networks are typically used as feature extractors and the output will
be sent into a classifier to make final decision. Although in some studies,
trainable integration methods have been devised [28, 29], the backbone network
architectures remain unchanged.

Besides network integration, there exist other methods that use different
algorithms and network architectures to address the scene recognition
challenge [30, 31, 32, 33]. However, in these studies, although different
scene recognition components are leveraged, the network backbones are still
used as feature extractors. To the best of our knowledge, in most recent
works, add-on modules are introduced in scene recognition and achieved
desirable results [34, 35], but the network backbone design remains seldom
explored. Apparently, there exists a gap between state-of-the-art generic deep
neural network design and scene recognition network design. Introducing the
most advanced network design theories and practices into the scene recognition
domain is imminent and imperative.

##  3 Experimental Study

###  3.1 Data Sets and Experimental Settings

To understand the impact of network structure on the dataset and ultimately
the applications, we use ImageNet 2012 [6] and Places Standard dataset [36] as
our evaluation datasets. ImageNet 2012 is the benchmark object recognition
dataset that consists of 1,000 classes and 1.28 million training images. An
image in ImageNet 2012 usually contains a single object that is highly
distinctive from the background. Place365 Standard dataset is designed for
scene recognition and contains 1.8 million training images of 365 classes. The
images in Places365 Standard datasets present more complex scenery images.

We train deep network models and compute the single-crop ($224\times 224$
pixels) top-1 and top-5 accuracy based on the application of the models to the
validation set. We train each model for 100 epochs on eight Tesla V100 GPUs
with 32 images per GPU (the batch size is 256). All models are trained using
synchronous SGD (Stochastic Gradient Descent) with a Nesterov momentum of 0.9
and a weight decay of 0.0001. The learning rate is set to 0.1 and is reduced
by a factor of 10 in every 30 epochs. In the training of ResNet and its
variants, we follow the settings in [17].

###  3.2 Comparison on Different Data Sets

We conducted our comparison study using ResNet and its variants on Places365
and ImageNet datasets. The results are reported in Tables 1 (varying network
depth) and 2 (varying network width). By increasing the network depth from 50
to 101, i.e., ResNet-50 and ResNet-101, we obtained a performance improvement
of 1.40% and 2.32% on Place365 and ImageNet datasets in terms of Top-1
accuracy. Theoretically, if the widening of the network is more effective to
improve the performance as stated in the previous literature, we should expect
more accuracy increase when we double the width of the networks. However,
doubling the width leads to a top-1 accuracy increase of 3.28% on ImageNet,
but only 0.94% on Place365. More surprisingly, for ResNeXt, which also doubled
the network width, the relative performance increase on ImageNet is 2.34% in
terms of top-1 accuracy, but the number is only 0.14% on Place365.

Table 1: Top-1 and top-5 accuracy (%) comparison by changing the network depth.  Data | Model | Top-1 | Top-5
---|---|---|---
| ResNet-18 | 54.22 | 84.63
Places365 | ResNet-50 | 55.69 | 85.80
| ResNet-101 | 56.47 | 86.25
| ResNet-18 | 70.52 | 89.56
ImageNet | ResNet50 | 76.02 | 92.80
| ResNet-101 | 77.78 | 93.72
Table 2: Top-1 and top-5 accuracy (%) comparison by changing the network width. The numbers within parenthesis are the width scaling factors.  Data | Model | Top-1 | Top-5
---|---|---|---
| ResNet-50 ($\times$ 1) | 55.69 | 85.80
| ResNet-50 ($\times$ 2) | 56.21 | 86.11
Places365 | ResNet-50 ($\times$ .5) | 55.07 | 85.12
| ResNet-50 ($\times$ .25) | 52.16 | 82.85
| ResNeXt-50 | 55.77 | 85.99
| ResNet-50 ($\times$ 1) | 76.02 | 92.80
| ResNet-50 ($\times$ 2) | 78.51 | 94.09
ImageNet | ResNet-50 ($\times$ .5) | 72.08 | 90.78
| ResNet-50 ($\times$ .25) | 64.04 | 85.76
| ResNeXt-50 | 77.80 | 94.30

The trend that the model performance on ImageNet is more sensitive to width
change compared with Place365 is also true when we decrease network depth or
narrow down the width. When the network depth decreased from 50 to 18,
ImageNet suffered a 7.23% relative top-1 accuracy decrease, and for Place365
it is 2.71%. That is, the top-1 performance drop on ImageNet is around 2.7
times of the top-1 performance drop on Places. But when we decreased the width
of ResNet-50 to half of the original size, the number changed to 4.7. The
statement that widening of the network might provide a more effective way to
improve performance is biased towards ImageNet (object-centric data) and
ignores the characteristics of scenery images.

Table 3: Top-1 and top-5 accuracy (%) comparison by changing the network depth using 100 randomly selected classes.  Data | Model | Top-1 | Top-5
---|---|---|---
Places365 | ResNet-18 | 70.63 | 94.32
| ResNet-50 | 71.14 | 94.56
ImageNet | ResNet-18 | 81.17 | 94.50
| ResNet50 | 83.19 | 95.42
Table 4: Top-1 and top-5 accuracy (%) comparison by changing the network width using 100 randomly selected classes. The numbers within parenthesis are the width scaling factors.  Data | Model | Top-1 | Top-5
---|---|---|---
| ResNet-50 ($\times$ 1) | 71.14 | 94.56
Places365 | ResNet-50 ($\times$ .5) | 71.04 | 94.21
| ResNet-50 ($\times$ .25) | 70.14 | 93.76
| ResNet-50 ($\times$ 1) | 83.19 | 95.42
ImageNet | ResNet-50 ($\times$ .5) | 80.17 | 94.06
| ResNet-50 ($\times$ .25) | 76.93 | 92.62

As Places365 has 365 classes while ImageNet has 1000 classes, to avoid the
bias caused by the different numbers of classes between the two datasets, we
randomly selected 100 classes from each dataset and conducted comparison
experiments. As we shrunk the size of the datasets, we observed under-fitting
occurred when we apply the smaller dataset to large models. So we only include
the comparison results on smaller models in Table 3 and Table 4. We observe
that for the Places dataset, switching ResNet-50 to ResNet-18 leads to a 0.72%
top-1 accuracy drop on Places and 2.49% top-1 accuracy drop on ImageNet, ie
the performance drop on ImageNet is around 3.46 times of the drop on Places.
Meanwhile, narrowing the width of ResNet-50 to 1/2 and 1/4 of the original
width leads to 0.14% and 1.41% top-1 performance drop, for ImageNet, the
accuracy decrease is 3.6% and 7.5 %. The performance drop caused by halving
the width on ImageNet is 25.7 times the performance drop on Places, which
verified that our hypothesis that altering network width has a less
significant effect on scenery data is not biased towards the number of classes
in the dataset.

##  4 Analysis of the Impact of Depth and Width of Deep Networks

###  4.1 Complexity of Images

Our first hypothesis is the performance difference is caused by the complexity
of scenery images. This hypothesis is aroused by the distinct complexity
difference between scene images and object-centric images: object-centric
images always only contain one major object which occupies a large portion of
the view, scene images always consist of multiple objects and background
clutters. Figure 1 shows two samples from benchmark object-centric dataset
(ImageNet) and benchmark scene dataset (Place365), respectively. Figure 1 (a)
is labeled as “bald eagle”, in which the eagle stands in the center of the
view and occupies a large portion of the entire image; figure 1 (b) is labeled
as “forest-broad leaf”, the entire view consists of not only a bird but also
tree branches and leaves. As the correct recognition of scenery images relies
on multiple components, a scene image is typically considered more complex
than an object-centric image.

|  |  |
---|---|---|---
(a) ImageNet | (b) Place365 | (c) CUB-200-2011 | (d) CUB-200-2011
(Bald eagle) | (Forest-broad leaf) | (Black-footed albatross) | (Sooty albatross)
Figure 1: Example images from benchmark object recognition dataset ImageNet
(a), benchmark scene recognition dataset Places365 (b), and fine-grained
dataset Caltech-UCSD Birds-200-2011 (c and d).

To evaluate this hypothesis, we introduced another dataset that is also widely
believed to be “complex”: the fine-grained image classification dataset. Fine-
grained classification is considered to be a more complex task as the classes
in the dataset can only be discriminated by local and subtle differences.
CUB-200-2011 is a dataset consisting of 200 different species of birds, which
serves as a benchmark dataset for fine-grained classification tasks. Figure 1
shows two samples in the CUB-200-2011 dataset. In Figure 1, black-footed
albatross (c) and sooty albatross (d) are considered to be two different
categories in classification. The two albatrosses are similar in appearance,
and the differentiating of them is challenging due to the subtle traits that
characterize the different species are not straightforward.

We conducted comparison experiments on two “complex” datasets (Places365 and
CUB-200-2011) and one “simple” dataset. The results are shown in Table 5.
Using the benchmark ResNet-50 as the backbones, we observed that on
CUB-200-2011, the relative top -1 accuracy increased by 1.81% when doubling
the width and dropped 4.28% when we narrowed the width to half of the
original. This performance change caused by altering the width is much acute
compared to the result on Place365 (0.94% and 1.11%, doubling and halving the
width respectively) under the same settings, which demonstrated that a wider
network is able to effectively enhance the recognition of “complex” fine-
grained features. This result does not agree with our first hypothesis: if the
performance difference is originated from the complexity of the data, we
should observe a moderate performance change on CUB-200-2011 dataset along
with the changing of network width. The results proved that the performance
variance on different dataset is not the consequence of the complexity (rich
fine-grained details) of the data.

Table 5: Top-1 accuracy (%) of ResNet-50 on Places365, ImageNet, and Caltech-UCSD Birds-200-2011 by changing network width.  Width Scaling Factor | Places | ImageNet | CUB
---|---|---|---
2 | 56.21 | 78.51 | 71.53
1 | 55.69 | 76.02 | 70.26
0.5 | 55.07 | 72.08 | 67.25
0.25 | 52.16 | 64.04 | 61.29

###  4.2 Spatial v.s. Channel

Based on the observations, we came up with the second hypothesis: for the
scene recognition task, instead of learning more fine-grained features,
learning spatial information is more crucial. As defined in [37], spatial
information refers to the spatial ordering on the feature map. Intuitively,
for images that only contain one object, the semantic meaning related to
spatial layout is limited; for scene images, the spatial structures, namely,
scene contextual information likely to contribute more to the understanding of
the scene. Thus, for scene recognition tasks, learning spatial information is
more crucial.

To verify this hypothesis, we conducted experiments by gradually passing low
and high-frequency information on Place365 and ImageNet datasets. Generally
speaking, the high-frequency information in the image refers to the regions
where the intensity of the image (brightness/gray-scale) changes drastically,
which are often called the edges or contour; the low-frequency information in
the image refers to the regions where the image intensity changes smoothly,
such as large patches of color. As shown in Figure 2, the image filtered by
low-pass filters tends to present proximate or blurred patterns of the
original image, the images filtered by high-pass filters better preserved the
spatial information.

---
Figure 2: Results on scenery image using low and high pass filters.

To understand the importance of low and high-frequency information in
different datasets, we designed the low-pass and high-pass filter based on
Fourier Transform. We transformed the testing images into the spectrum domain
using Fourier Transform and applied both low and high-pass filters to test how
low/high-frequency information can affect the model performance on different
datasets. Figure 3 shows the design of the filters: for low-pass filters, we
masked the high-frequency components; and for high-pass filters, we masked the
low-frequency components.

|  |
---|---|---
(a) Image in frequency domain | (b) Low pass filter | (c) High pass filter
Figure 3: Illustration of low and high pass filters. In Figure (b) and (c),
the mask (black region) denotes information removed by corresponding filters;
the length of the red lines denotes the corresponding filter size.
---
Figure 4: Top-1 accuracy (%) on Place365 and ImageNet datasets using low pass
filters (left) and high pass filters (right). The x-axis denotes the size of
the corresponding filters.

For a fair comparison, we randomly selected 100 classes from Place365 and
ImageNet datasets to deploy the experiments. Note that as scene recognition is
considered a harder task compared to object recognition, the classification
accuracy on Place365 is lower despite the chance is the same. The results are
shown in Figure 4. In both of the sub-figures, the x-axis denotes the size of
the corresponding low/high pass filter in the spectrum domain (the maximum
size is 224), and the y-axis denotes the top-1 accuracy. Through the
comparison, we observed enlightening phenomena: when we gradually pass the low
frequency information, the performance increase on ImageNet is steeper than on
Place365 (Figure 4 (a)). Surprisingly, when we use the low pass filter of size
33, ImageNet can achieve a top-1 accuracy of nearly 30% while the chance is
1%. This denotes the correct recognition of object-centric images heavily rely
on low frequency information. On the contrary, when we gradually pass the high
frequency information, we can observe that the model trained on scenery
dataset is more sensitive to high-frequency information (Figure 4 (b)).
Notably, when the size of the high-pass filter is around 210 to 214, the top-1
accuracy on Place365 even exceeds the top-1 accuracy on ImageNet despite the
classification accuracy on ImageNet is always much higher on Place365 across
various networks. This observation demonstrated that recognition of scenery
images is susceptible to high frequency information.

Our observation perfectly fits the experimental results we discussed in
Section 3. The high and low frequency information in images can approximately
present the learned spatial and channel-wise information in deep networks.
Wider networks have an expanded number of channels, which enables the network
to learn more fine-grained features. Deep networks have an increasing number
of layers and larger receptive fields, which enables the network to learn more
spatial information. Thus for the scene recognition tasks, deepening the
network is more efficient than increasing the width of the network. We
proposed two directions to better enhance the scene recognition networks:
deepening and narrowing the network, and decreasing the spatial information
lost.

##  5 Deep-Narrow Network

###  5.1 Deep-Narrow Structure

Figure 5: The schema of standard ResNet (left), Wide ResNet (middle), and Deep-Narrow Network (right). C denotes the number of channels.  Table 6: Top-1 and Top-5 accuracy (%) using ResNet with different depth and width. The number of parameters is in million. The numbers within parenthesis are the width scaling factors.  Data | Model | Top-1 | Top-5 | GFLOPs |  $\\#$ Parameters
---|---|---|---|---|---
| ResNet-50 ($\times$ 1) | 55.69 | 85.80 | 4.12 | 24.26
Place365 | ResNet-50 ($\times$ .5) | 55.07 | 85.12 | 1.07 | 6.27
| Deep-Narrow Network | 55.58 | 85.80 | 2.00 | 11.03
| ResNet-50 ($\times$ 1) | 76.02 | 92.80 | 4.12 | 25.56
ImageNet | ResNet-50 ($\times$ .5) | 72.08 | 90.78 | 1.07 | 6.92
| Deep-Narrow Network | 74.99 | 92.31 | 2.00 | 11.68

Based on our observation, we argued that as correctly recognizing scenery
images is susceptible to the learning of spatial information, designing the
networks with larger depth and smaller width can potentially be an effective
and efficient option. As shown in Figure 5, we proposed a Deep-Narrow
architecture. Different from the famous Wide ResNet [5] which use the factor K
to increase the width of the ResNet, we increase the number of layers in
ResNet and decreases the width of the network using width factor K. In our
designed Deep-Narrow Network, we use K equals to 2, i.e., we halve the network
width used in benchmark ResNet.

Table 6 shows the performance comparisons among the benchmark ResNet-50, i.e.,
ResNet-50 ($\times$ 1), ResNet-50 with half the width, i.e., ResNet-50
($\times$ .5)), and our Deep-Narrow Network on Place365 dataset. Deep-Narrow
Network achieves comparable evaluation scores with benchmark ResNet-50 using
only less than half of the FLOPs and parameters: the relative top-1 accuracy
dropped 0.20%. On the ImageNet, the Deep-Narrow Architecture obtained a
relative top-1 accuracy drop of 1.35%. The results re-certified that scene
recognition is highly dependent on learning spatial information. Different
from the assertions in the previous literature that widening of the network
might provide a more effective way than by making ResNet deeper to improve the
model performance, we demonstrated that data have their preference and the
network design should rely on the characteristics of data.

###  5.2 Dilated Pooling

---
(a) Down-sample component in ResNet, during the down-sampling process, 3/4 of
the spatial in-
formation are discarded and the number of channels is quadrupled.
(b) Dilated Pooling module which does not discard spatial information.
Figure 6: The schema of down-sample component in ResNet and Dilated Pooling module.  |  |
---|---|---
(a) |  | (b)
Figure 7: The schema of the down-sample component in ResNet and Pooling module
from the view of spatial dimension. (a) Down-sample component in ResNet from
the view of spatial dimension. During the down-sampling process, 3/4 of the
spatial information are discarded and the number of channels is doubled. (b)
Pooling module from the view of spatial dimension.

Besides deepening the network to better process spatial information, we also
designed a Dilated Pooling module to better preserve the spatial information
in ResNet. As shown is Figure 6 (a), in ResNet, the down-sampling process is
conducted by quadrupling the width of the network and discard 3/4 of the
features along spatial dimension. Based on our statement, this design is not
suitable for scene recognition as discarding spatial information is likely to
devastate the performance. Alternatively, we designed a Dilated Pooling module
to better preserve the spatial information (Figure 6 (b)). More details are
illustrated in Figure 7. In Dilated Pooling module, instead of directly
discarding 3/4 of the spatial information, we separate the features maps into
four sub-sections along the spatial dimension. Then we conduct convolution on
the four feature maps and merged the result together via summation operation.
By leveraging Dilated Pooling, we are able to make use of all the spatial
information without increasing the Flops and number of parameters.

Table 7: Top-1 and top-5 accuracy (%) using Deep-Narrow Network and Dilated Pooling (DP) module. The number of parameters is in million.  Data | Model | Top-1 | top-5 | GFLOPs | Params
---|---|---|---|---|---
| ResNet-50 ($\times$ 1) | 55.69 | 85.80 | 4.12 | 24.26
Place365 | Deep-Narrow Network | 55.58 | 85.80 | 2.00 | 11.03
| Deep-Narrow Network (with DP) | 55.91 | 86.12 | 2.00 | 11.03
| ResNet-50 ($\times$ 1) | 76.02 | 92.80 | 4.12 | 25.56
ImageNet | Deep-Narrow Network | 74.99 | 92.31 | 2.00 | 11.68
| Deep-Narrow Network (with DP) | 74.63 | 92.13 | 2.00 | 11.68
Table 8: Top-1 and top-5 accuracy (%) on Place365 using different backbones. The number of parameters is in million. Note that we calculate the number of parameters based on 365-class models (Place365). The best results are highlighted in bold.  Model |  Top-1 |  Top-5 |  GFLOPs |  $\\#$ Parameter
---|---|---|---|---
ResNet-50 ($\times$ 1) |  55.69 |  85.80 |  4.12 |  24.26
Deep-Narrow Network |  55.58 |  85.80 |  2.00 |  11.03
ResNet-Ave |  55.60 |  85.58 |  2.26 |  11.03
ResNet-Max |  55.55 |  85.56 |  2.26 |  11.03
ResNet-D |  55.79 |  85.93 |  2.26 |  11.03
Antialiased-CNN |  55.85 |  85.93 |  2.26 |  11.03
Deep-Narrow Network (with DP) |  55.91 |  86.12 |  2.00 |  11.03

As shown in Table 7, by integrating the Dilated Pooling module with our Deep-
Narrow Network design, we achieved better performance than the benchmark
ResNet on the Place365 dataset, using less than half of the FLOPs and number
of parameters. Specifically, adding Dilated Pooling to our Deep-Narrow Network
resulted in a relative top-1 accuracy increase of 0.59% (Table 7), while only
causing a 0.48% relative top-1 accuracy drop on ImageNet. This demonstrates
the effectiveness and efficiency of our data-oriented network design approach
and the importance of designing networks according to the characteristics of
the data.

To demonstrate the superiority of the proposed method, we compare with several
state-of-the-art approaches that also tried to minimize the information lost
caused by shrinking the spatial resolution, including ResNet-D [38] and
Antialiased-CNN [39]. ResNet-D used average pooling and convolution operation
to maintain more spatial information; Antialiased-CNN leveraged convolution-
based pooling strategies to enhance the conventional pooling. We also
implemented two baseline strategies named ResNet-Ave and ResNet-Max, which
simply use averaging or maxing operation instead of discarding 3/4
information. As shown in Table 8, our model use 0.26 less GLOPs compared to
the aforementioned methods, which is an 11.5% computational resource saving.
In short, we can observe that Deep-Narrow Network with Dilated Pooling
achieves higher accuracy than the comparison methods using less computational
resources.

##  6 Conclusion

This paper studies the impacts of scenery images in the network architecture
design using ImageNet (object recognition dataset) and Places365 (scene
recognition dataset) as examples. Carefully designed experiments showed that
the characteristics of datasets affect the performance of the models: wider
networks often achieve better performance of recognition of images with a
prominent object and have less impact on the recognition of scenery images. We
further evaluated our hypothesis by conducting comparison experiments and
demonstrated that learning spatial-wise information is more substantial in
scene recognition tasks compared to object classification tasks. This
phenomenon explained why widening the networks is less effective than
deepening the networks for scene recognition as deeper networks can better
learn the spatial information in the training examples. Thus deploying the
networks that have larger depth and smaller width, and emphasizing the spatial
information learning likely to benefit scene recognition backbone designs. Our
proposed Deep-Narrow Network and Dilated Pooling module re-certified the
effectiveness and efficiency of taking advantage of data properly. Our design
achieved a better accuracy compared to benchmark ResNet-50 using less than
half of the computation resources.

## References

  * [1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _NIPs_ , vol. 25, pp. 1097–1105, 2012\.
  * [2] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _ICLR_ , 2015.
  * [3] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in _CVPR_ , 2016, pp. 2818–2826.
  * [4] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _CVPR_ , Los Alamitos, CA, USA, Jun 2016, pp. 770–778.
  * [5] S. Zagoruyko and N. Komodakis, “Wide residual networks,” in _BMVC_. British Machine Vision Association, 2016.
  * [6] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in _CVPR_ , 2009, pp. 248–255.
  * [7] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [8] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in _CVPR_ , 2017, pp. 1492–1500.
  * [9] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _ECCV_. Springer, 2014, pp. 740–755.
  * [10] H. Zhang, C. Wu, Z. Zhang, Y. Zhu, H. Lin, Z. Zhang, Y. Sun, T. He, J. Mueller, R. Manmatha _et al._ , “Resnest: Split-attention networks,” _arXiv preprint arXiv:2004.08955_ , 2020.
  * [11] Z. Lu, H. Pu, F. Wang, Z. Hu, and L. Wang, “The expressive power of neural networks: a view from the width,” in _Nips_ , 2017, pp. 6232–6240.
  * [12] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in _ICML_. PMLR, 2019, pp. 6105–6114.
  * [13] B. Zoph and Q. V. Le, “Neural architecture search with reinforcement learning,” _ICLR_ , 2017.
  * [14] S. Guo, Y. Wang, Q. Li, and J. Yan, “Dmcp: Differentiable markov channel pruning for neural networks,” in _CVPR_ , 2020, pp. 1539–1547.
  * [15] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in _CVPR_ , 2015, pp. 1–9.
  * [16] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning,” in _AAAI_ , 2017, pp. 4278–4284.
  * [17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _CVPR_ , 2016, pp. 770–778.
  * [18] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in _CVPR_ , 2017, pp. 1251–1258.
  * [19] T. Nguyen, M. Raghu, and S. Kornblith, “Do wide and deep networks learn the same things? Uncovering how neural network representations vary with width and depth,” _ICLR_ , 2021.
  * [20] Y. Zhu, X. Deng, and S. Newsam, “Fine-grained land use classification at the city scale using ground-level images,” _IEEE Transactions on Multimedia_ , 2019.
  * [21] X. Cheng, J. Lu, J. Feng, B. Yuan, and J. Zhou, “Scene recognition with objectness,” _Pattern Recognition_ , vol. 74, pp. 474–487, 2018.
  * [22] Z. Qiao, X. Yuan, and M. Elhoseny, “Urban scene recognition via deep network integration,” in _International Conference on Urban Intelligence and Applications_. Springer, 2020, pp. 135–149.
  * [23] L. Herranz, S. Jiang, and X. Li, “Scene recognition with cnns: objects, scales and dataset bias,” in _CVPR_ , 2016.
  * [24] Z. Wang, L. Wang, Y. Wang, B. Zhang, and Y. Qiao, “Weakly supervised patchnets: Describing and aggregating local patches for scene recognition,” _IEEE Transactions on Image Processing_ , vol. 26, no. 4, pp. 2028–2041, 2017.
  * [25] B. Zhou, A. Khosla, A. Lapedriza, A. Torralba, and A. Oliva, “Places: An image database for deep scene understanding,” _Journal of Vision_ , 2016.
  * [26] S. Xia, J. Zeng, L. Leng, and X. Fu, “WS-AM: Weakly Supervised Attention Map for Scene Recognition,” _Electronics_ , vol. 8, no. 10, p. 1072, 2019.
  * [27] R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Visual explanations from deep networks via gradient-based localization,” in _ICCV_ , 2017, pp. 618–626.
  * [28] H. Seong, J. Hyun, H. Chang, S. Lee, S. Woo, and E. Kim, “Scene recognition via object-to-scene class conversion: end-to-end training,” in _IJCNN_ , 2019, pp. 1–6.
  * [29] H. Seong, J. Hyun, and E. Kim, “Fosnet: An end-to-end trainable deep neural network for scene recognition,” _arXiv preprint arXiv:1907.07570_ , 2019\.
  * [30] J. Shi, H. Zhu, S. Yu, W. Wu, and H. Shi, “Scene categorization model using deep visually sensitive features,” _IEEE Access_ , vol. 7, pp. 45 230–45 239, 2019.
  * [31] C. Wang, G. Peng, and B. De Baets, “Deep feature fusion through adaptive discriminative metric learning for scene recognition,” _Information Fusion_ , vol. 63, pp. 1–12, 2020.
  * [32] A. Rehman, S. Saleem, U. G. Khan, S. Jabeen, and M. O. Shafiq, “Scene recognition by joint learning of dnn from bag of visual words and convolutional dct features,” _Applied Artificial Intelligence_ , pp. 1–19, 2021.
  * [33] S. Gupta, K. Sharma, D. A. Dinesh, and V. Thenkanidiyoor, “Visual semantic-based representation learning using deep cnns for scene recognition,” _ACM Transactions on Multimedia Computing, Communications, and Applications_ , vol. 17, no. 2, pp. 1–24, 2021.
  * [34] Z. Qiao, X. Yuan, C. Zhuang, and A. Meyarian, “Attention pyramid module for scene recognition,” in _2020 25th International Conference on Pattern Recognition_. IEEE, 2021, pp. 7521–7528.
  * [35] X. Yuan, Z. Qiao, and A. Meyarian, “Scale attentive network for scene recognition,” _Neurocomputing_ , 2021.
  * [36] B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 40, no. 6, pp. 1452–1464, 2017\.
  * [37] Y. Fan, Y. Xian, M. M. Losch, and B. Schiele, “Analyzing the dependency of convnets on spatial information,” in _DAGM German Conference on Pattern Recognition_. Springer, 2020, pp. 101–115.
  * [38] T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li, “Bag of tricks for image classification with convolutional neural networks,” in _Computer Vision and Pattern Recognition_ , 2019, pp. 558–567.
  * [39] R. Zhang, “Making convolutional networks shift-invariant again,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 7324–7334.


# Electrodynamic spherical harmonic

A V Novitsky
Department of Theoretical Physics, Belarusian State University,
Nezavisimosti Avenue 4, 220050 Minsk, Belarus
<EMAIL_ADDRESS>
###### Abstract

Electrodynamic spherical harmonic is a second rank tensor in three-dimensional
space. It allows to separate the radial and angle variables in vector
solutions of Maxwell’s equations. Using the orthonormalization for
electrodynamic spherical harmonic, a boundary problem on a sphere can be
easily solved.

##  1 Introduction

In this paper we introduce new function — electrodynamic spherical harmonic.
It is represented as a second rank tensor in three-dimensional space. But the
function differs from tensor spherical harmonic [1, 2, 3, 4, 5]. By its
definition the electrodynamic spherical function possesses a number of
properties of usual scalar and vector harmonics and includes them as component
parts. Why electrodynamic spherical harmonic? The word “electrodynamic”
implies that the function is applied for solution of vector field problems. We
use it in electrodynamics, however one can apply the function for description
of spin fields in quantum field theory.

Electrodynamic spherical harmonic is not a simple designation of the well-
known functions. It satisfies the Maxwell equations and describes the angular
dependence of vector fields. The introduced function separates the variables
(radial coordinate and angles) in the fields. Moreover, the notation of the
fields in terms of electrodynamic spherical harmonic noticeably simplifies the
solution of a boundary problem on spherical interface. Just application of the
ortonormalization condition allows to find the coefficients of spherical
function expansion (e.g. scattering field amplitudes).

##  2 Scalar and vector spherical harmonics

Hamilton’s operator $\nabla$ in three-dimensional space contains the
derivatives on three coordinates. In spherical coordinates ($r$, $\theta$,
$\varphi$) the derivatives on radial coordinate and angles can be separated.
This is achieved by representing the unit tensor ${\bf 1}$ in 3D space as the
superposition of two projection operators:

$\nabla={\bf 1}\nabla=\left(\frac{{\bf r}\otimes{\bf r}}{r^{2}}-\frac{{\bf
r}^{\times 2}}{r^{2}}\right)\nabla=\frac{1}{r^{2}}{\bf r}({\bf
r}\nabla)-\frac{1}{r^{2}}{\bf r}^{\times}({\bf r}^{\times}\nabla),$

where ${\bf r}\otimes{\bf r}/r^{2}$ is the projector onto the direction ${\bf
e}_{r}={\bf r}/r$, $-{\bf r}^{\times 2}/r^{2}$ is the projector onto the plane
orthogonal to the unit vector ${\bf e}_{r}$. Tensor ${\bf r}^{\times}$ dual to
the vector ${\bf r}$ gives the well-known vector product when acting on a
vector ${\bf a}$: ${\bf r}^{\times}{\bf a}={\bf r}\times{\bf a}$ and ${\bf
a}{\bf r}^{\times}={\bf a}\times{\bf r}$ [6]. Introducing the vector
differential operator

${\bf L}=\frac{1}{{\rm i}}{\bf r}\times\nabla$ (1)

the equation (LABEL:nabla1) is rewritten as follows

$\nabla=\frac{{\bf r}}{r}\frac{\partial}{\partial r}-\frac{{\rm i}}{r^{2}}{\bf
r}\times{\bf L}.$ (2)

Vector ${\bf L}$ is called orbital angular momentum operator in quantum
mechanics, because it is presented as vector product of radius vector ${\bf
r}$ and momentum ${\bf p}=-{\rm i}\nabla$ operators. ${\bf L}$ includes only
derivatives on the angles $\theta$ and $\varphi$. Using the definition of
${\bf L}$ one can derive the Laplace operator

$\Delta=\nabla^{2}=\frac{1}{r^{2}}\frac{\partial}{\partial
r}\left(r^{2}\frac{\partial}{\partial r}\right)-\frac{{\bf L}^{2}}{r^{2}}$ (3)

and the following properties:

${\bf r}{\bf L}=0,\qquad({\bf L}{\bf r})=0,\qquad{\bf L}^{2}{\bf L}={\bf
L}{\bf L}^{2},\qquad{\bf L}\times{\bf L}={\rm i}{\bf L}.$ (4)

Scalar spherical harmonic $Y_{lm}(\theta,\varphi)$ is defined as the
eigenfunction of the operator ${\bf L}^{2}$:

${\bf L}^{2}Y_{lm}=l(l+1)Y_{lm},$ (5)

where $l$ and $m$ are integer numbers. Number $m$ is the eigenvalue of the
operator of projection of angular momentum onto the axis $z$, $L_{z}$:

$L_{z}Y_{lm}=mY_{lm}.$ (6)

Spherical harmonics $Y_{lm}$ are orthogonal and normalized by the unit:

$\int_{0}^{\pi}\int_{0}^{2\pi}Y^{\ast}_{l^{\prime}m^{\prime}}(\theta,\varphi)Y_{lm}(\theta,\varphi)\sin\theta{\rm
d}\theta{\rm d}\varphi=\delta_{l^{\prime}l}\delta_{m^{\prime}m}.$ (7)

where the sign ∗ denotes the complex conjugate.

If we multiply equation (5) by the vector operator ${\bf L}$ and take into
account the commutation of ${\bf L}$ and ${\bf L}^{2}$, we will obtain that
vector ${\bf L}Y_{lm}$ satisfies the same equation (5), too. The quantity
defined as

${\bf X}_{lm}=\frac{1}{\sqrt{l(l+1)}}{\bf L}Y_{lm}$ (8)

is called vector spherical harmonic. The coefficient before ${\bf L}Y_{lm}$ is
chosen so that the orthonormalization condition is of the form

$\int_{0}^{\pi}\int_{0}^{2\pi}{\bf
X}_{l^{\prime}m^{\prime}}^{\ast}(\theta,\varphi){\bf
X}_{lm}(\theta,\varphi)\sin\theta{\rm d}\theta{\rm
d}\varphi=\delta_{l^{\prime}l}\delta_{m^{\prime}m}.$ (9)

From the self-conjugacy of the angular momentum operator ${\bf L}$ and
properties (4) the orthogonality

$\displaystyle\int_{0}^{\pi}\int_{0}^{2\pi}{\bf e}_{r}({\bf
X}_{l^{\prime}m^{\prime}}^{\ast}\times{\bf X}_{lm})\sin\theta{\rm d}\theta{\rm
d}\varphi$ $\displaystyle=$
$\displaystyle\int_{0}^{\pi}\int_{0}^{2\pi}\frac{Y_{l^{\prime}m^{\prime}}^{\ast}{\bf
e}_{r}({\bf L}\times{\bf
L})Y_{lm}}{\sqrt{l(l+1)l^{\prime}(l^{\prime}+1)}}\sin\theta{\rm d}\theta{\rm
d}\varphi$ (10) $\displaystyle=$
$\displaystyle\int_{0}^{\pi}\int_{0}^{2\pi}\frac{{\rm
i}Y_{l^{\prime}m^{\prime}}^{\ast}{\bf e}_{r}{\bf
L}Y_{lm}}{\sqrt{l(l+1)l^{\prime}(l^{\prime}+1)}}\sin\theta{\rm d}\theta{\rm
d}\varphi=0.$

follows. Below we give some properties of vector spherical harmonics:

${\bf L}{\bf X}_{lm}=\sqrt{l(l+1)}Y_{lm},\qquad{\bf L}({\bf e}_{r}\times{\bf
X}_{lm})=0.$ (11)

Scalar (vector) spherical harmonics satisfy the scalar (vector) equation for
eigenfunctions of the squared orbital angular momentum operator ${\bf L}^{2}$.
The orthogonality condition for the scalar (7) and vector (9) spherical
harmonics have the same form. Therefore, one can hope to combine them into one
mathematical object.

##  3 Electrodynamic spherical harmonic: definition and properties

We define an electrodynamic spherical harmonic as a second rank tensor in
three-dimensional space

$F_{lm}=Y_{lm}{\bf e}_{r}\otimes{\bf e}_{r}+{\bf X}_{lm}\otimes{\bf
e}_{\theta}+({\bf e}_{r}\times{\bf X}_{lm})\otimes{\bf e}_{\varphi}.$ (12)

The first term of (12) determines the longitudinal part of the tensor. It is
calculated by means of the scalar spherical function. The last two summands of
(12) fix the transverse solution, in the plane ($\theta$, $\varphi$)
perpendicular to the direction ${\bf e}_{r}$. It includes vector spherical
harmonics. The left and right vectors in dyads form two sets of orthogonal
vectors: $(Y_{lm}{\bf e}_{r},{\bf X}_{lm},{\bf e}_{r}\times{\bf X}_{lm})$ and
$({\bf e}_{r},{\bf e}_{\theta},{\bf e}_{\varphi})$.

###  3.1 Orthonormalization

Multiplying the electrodynamic spherical harmonic by the Hermitian conjugate
$F_{lm}^{+}$ we get to

$F_{l^{\prime}m^{\prime}}^{+}F_{lm}=Y^{\ast}_{l^{\prime}m^{\prime}}Y_{lm}{\bf
e}_{r}\otimes{\bf e}_{r}+{\bf X}^{\ast}_{l^{\prime}m^{\prime}}{\bf
X}_{lm}({\bf e}_{\theta}\otimes{\bf e}_{\theta}+{\bf e}_{\varphi}\otimes{\bf
e}_{\varphi})+({\bf e}_{r}({\bf X}_{l^{\prime}m^{\prime}}^{\ast}\times{\bf
X}_{lm})){\bf e}_{r}^{\times}.$ (13)

The quantity before each dyad is orthogonal or normalized as (7), (9), or
(10). Therefore, the orthonormalization condition for electrodynamic spherical
harmonics is

$\int_{0}^{\pi}\int_{0}^{2\pi}F_{lm}^{+}(\theta,\varphi)F_{lm}(\theta,\varphi)\sin\theta{\rm
d}\theta{\rm d}\varphi={\bf 1}\delta_{l^{\prime}l}\delta_{m^{\prime}m}.$ (14)

Here we consider the vectors ${\bf e}_{r}$, ${\bf e}_{\theta}$, and ${\bf
e}_{\varphi}$ to be constant in dyads and dual tensor ${\bf e}_{r}^{\times}$.
If tensors $F_{lm}$ are the solutions of equations, we can always write these
solutions in components. For each component of the tensor $F_{lm}^{+}F_{lm}$
the orthonormalization is carried out. We will obtain the same, if the
dependence of the orts ${\bf e}_{r}$, ${\bf e}_{\theta}$, ${\bf e}_{\varphi}$
on angles in (13) is omitted, i.e. they are regarded as constants. However,
when substituting $F_{lm}$ in equations, we should take into account the
angular dependence of the orts.

###  3.2 Explicit form

Let us substitute the explicit expression for the vector operator ${\bf L}$

${\bf L}=-{\rm i}{\bf e}_{\varphi}\frac{\partial}{\partial\theta}+{\rm
i}\frac{{\bf e}_{\theta}}{\sin\theta}\frac{\partial}{\partial\varphi}$ (15)

into equation (12). Then the electrodynamic spherical harmonic is equal to

$F_{lm}=\left[{\bf e}_{r}\otimes{\bf e}_{r}+\frac{{\rm
i}}{\sin\theta\sqrt{l(l+1)}}\left(I\frac{\partial}{\partial\varphi}-{\bf
e}_{r}^{\times}\sin\theta\frac{\partial}{\partial\theta}\right)\right]Y_{lm},$
(16)

where $I=-{\bf e}_{r}^{\times}{\bf e}_{r}^{\times}={\bf 1}-{\bf
e}_{r}\otimes{\bf e}_{r}$ is the projection operator onto the plane ($\theta$,
$\varphi$). To calculate the derivatives one can replace them by means of
operators $L_{z}$ and $L_{\pm}=L_{x}\pm L_{y}$ as

$\frac{\partial}{\partial\varphi}={\rm
i}L_{z},\qquad\frac{\partial}{\partial\theta}=\frac{1}{2}({\rm e}^{-{\rm
i}\varphi}L_{+}-{\rm e}^{{\rm i}\varphi}L_{-}),$

because their action on the scalar spherical harmonic is well-known:

$L_{+}Y_{lm}=\sqrt{(l-m)(l+m+1)}Y_{l,m+1},\qquad
L_{-}Y_{lm}=\sqrt{(l+m)(l-m+1)}Y_{l,m-1}.$ (17)

Hence, the electrodynamic spherical harmonic can be presented as follows

$F_{lm}=\left[{\bf e}_{r}\otimes{\bf
e}_{r}-\frac{1}{\sin\theta\sqrt{l(l+1)}}\left(IL_{z}+{\bf
e}_{r}^{\times}\frac{{\rm i}\sin\theta}{2}({\rm e}^{-{\rm i}\varphi}L_{+}-{\rm
e}^{{\rm i}\varphi}L_{-})\right)\right]Y_{lm}.$ (18)

It is easy to exclude the operators from (18). The final formula for tensor
$F_{lm}$ contains scalar spherical harmonics as angle dependence. Unit vectors
${\bf e}_{r}$, ${\bf e}_{\theta}$, ${\bf e}_{\varphi}$ determine the structure
of the tensor in three-dimensional space. $F_{lm}$ is formed by three basic
tensors: ${\bf e}_{r}\otimes{\bf e}_{r}$, ${\bf e}_{r}^{\times}$, and $I$.
Hence, it commutes with each of these tensors.

###  3.3 Invariants

The first invariant of the electrodynamic spherical harmonic as tensor
quantity is its trace. The trace of the tensor (12) equals

${\rm tr}(F_{lm})=Y_{lm}+2({\bf e}_{\theta}{\bf X}_{lm}).$ (19)

The second invariant is determinant

$\det(F_{lm})=Y_{lm}{\bf X}_{lm}^{2}.$ (20)

The third invariant of three-dimensional tensor $F_{lm}$ is the trace of the
adjoint tensor $\overline{F}_{lm}$. Adjoint tensor is defined by
$\overline{F}_{lm}F_{lm}=F_{lm}\overline{F}_{lm}=\det(F_{lm}){\bf 1}$ and
equals

$\overline{F}_{lm}={\bf X}_{lm}^{2}{\bf e}_{r}\otimes{\bf e}_{r}+Y_{lm}{\bf
e}_{\theta}\otimes{\bf X_{lm}}+Y_{lm}{\bf e}_{\varphi}\otimes{{\bf
e}_{r}\times\bf X_{lm}}.$ (21)

Further the trace is easily calculated:

${\rm tr}(\overline{F}_{lm})={\bf X}_{lm}^{2}+2Y_{lm}({\bf e}_{\theta}{\bf
X_{lm}}).$ (22)

Using these three invariants one can find other ones. For example, the trace
of squared tensor is determined from equation ${\rm tr}(F_{lm}^{2})=({\rm
tr}(F_{lm}))^{2}-2{\rm tr}(\overline{F}_{lm})$.

###  3.4 Generalization of electrodynamic spherical harmonic

The main condition on electrodynamic spherical harmonic is the
orthonormalization (14). There is more general form of the second rank tensor
spherical harmonic satisfying this equation. It is

$G_{lm}=Y_{lm}{\bf e}_{r}\otimes{\bf a}+{\bf X}_{lm}\otimes{\bf b}+({\bf
e}_{r}\times{\bf X}_{lm})\otimes{\bf c}.$ (23)

Unit vectors ${\bf a}$, ${\bf b}$, and ${\bf c}$ form the orthogonal basis in
three-dimensional space. In equation (12) we have assumed ${\bf a}={\bf
e}_{r}$, ${\bf b}={\bf e}_{\theta}$, ${\bf c}={\bf e}_{\varphi}$ because of
the spherical symmetry of the function. If we will take ${\bf a}={\bf e}_{r}$,
${\bf b}={\bf e}_{\theta}$, ${\bf c}=-{\bf e}_{\varphi}$, then the
electrodynamic spherical harmonic becomes more complex function in explicit
form, however its invariants are simplified (e.g., the trace equals ${\rm
tr}(F_{lm})=Y_{lm}$).

##  4 Solution of Maxwell’s equations

In this section we will find the spherically symmetric solutions (i.e.
electric ${\bf E}$ and magnetic ${\bf H}$ fields) of the Maxwell equations

$\nabla\times{\bf E}({\bf r})={\rm i}k\mu{\bf H}({\bf
r}),\qquad\nabla\times{\bf H}({\bf r})=-{\rm i}k\varepsilon{\bf E}({\bf r})$
(24)

for the monochromatic electromagnetic waves in isotropic medium with
dielectric permittivity $\varepsilon$ and magnetic permeability $\mu$. The
quantity $k=\omega/c$ is called wavenumber in vacuum, and $\omega$ is the wave
frequency. Arbitrary time dependence can be obtained by using the linear
superposition of monochromatic waves ${\bf E}({\bf r})\exp(-{\rm i}\omega t)$.

We will search the solution in the form

${\bf E}({\bf r})=F_{lm}(\theta,\varphi){\bf E}^{l}(r).$ (25)

The components of the vector ${\bf E}^{l}$ in spherical coordinates depend
only on the radial coordinate $r$. The dependence on the angle coordinates
presents only in the basis vectors. So, the vector ${\bf E}^{l}$ looks like

${\bf E}^{l}(r)=E^{l}_{r}(r){\bf e}_{r}+E^{l}_{\theta}(r){\bf
e}_{\theta}+E^{l}_{\varphi}(r){\bf e}_{\varphi}.$ (26)

Further we should calculate ${\rm rot}{\bf E}$. By substituting Hamilton’s
operator (2) one obtains

$\nabla\times{\bf E}={\bf e}_{r}^{\times}\frac{\partial{\bf E}}{\partial
r}-\frac{{\rm i}}{r}{\bf L}({\bf e}_{r}{\buildrel\downarrow\over{{\bf
E}}})+\frac{{\rm i}}{r}{\bf e}_{r}({\bf L}{\bf E}),$ (27)

where the arrow $\downarrow$ implies that operator ${\bf L}$ acts only on
vector ${\bf E}$, but not ${\bf e}_{r}$. Let us calculate each summand of
equation (27) applying the solution (25). The first term is of the form

${\bf e}_{r}^{\times}\frac{\partial{\bf E}}{\partial r}=F_{lm}{\bf
e}_{r}^{\times}\frac{\partial{\bf E}^{l}}{\partial r},$ (28)

where the commutation relation $[F_{lm},{\bf e}_{r}^{\times}]=0$ is taken into
account. The second summand yields

${\bf L}({\bf e}_{r}{\buildrel\downarrow\over{{\bf E}}})={\bf L}({\bf
e}_{r}{\bf E})-{\bf L}({\buildrel\downarrow\over{{\bf e}_{r}}}{\bf E})={\bf
L}(Y_{lm}E^{l}_{r})-{\bf L}({\buildrel\downarrow\over{{\bf e}_{r}}}{\bf E}).$
(29)

The quantity ${\bf L}({\buildrel\downarrow\over{{\bf e}_{r}}}{\bf E})$ is
easily calculated using the explicit expression (15) of the operator ${\bf L}$
and the relationships $\partial{\bf e}_{r}/\partial\theta={\bf e}_{\theta}$
and $\partial{\bf e}_{r}/\partial\varphi={\bf e}_{\varphi}\sin\theta$:

${\bf L}({\buildrel\downarrow\over{{\bf e}_{r}}}{\bf E})=-{\rm i}{\bf
e}_{r}^{\times}{\bf E}.$ (30)

So, we get the formula

${\bf L}({\bf e}_{r}{\buildrel\downarrow\over{{\bf
E}}})=F_{lm}\left(\sqrt{l(l+1)}{\bf e}_{\theta}\otimes{\bf e}_{r}+{\rm i}{\bf
e}_{r}^{\times}\right){\bf E}^{l}.$ (31)

The third term in (27) can be rewritten using the equation (11):

${\bf e}_{r}({\bf L}{\bf E})={\bf e}_{r}\left(E^{l}_{r}{\bf L}(Y_{lm}{\bf
e}_{r})+E^{l}_{\theta}{\bf L}{\bf X}_{lm}+E^{l}_{\varphi}{\bf L}({\bf
e}_{r}\times{\bf X}_{lm})\right)=\sqrt{l(l+1)}F_{lm}({\bf e}_{r}\otimes{\bf
e}_{\theta}){\bf E}^{l}.$ (32)

Finally, the curl of the electric field vector ${\bf E}$ equals

$\nabla\times{\bf E}=F_{lm}(\theta,\varphi)\left({\bf
e}_{r}^{\times}\frac{\partial}{\partial r}+\frac{1}{r}{\bf
e}_{r}^{\times}-\frac{{\rm i}\sqrt{l(l+1)}}{r}{\bf
e}_{\varphi}^{\times}\right){\bf E}^{l}(r).$ (33)

In expression (33) we have took into account the derivatives on the angle
variables. Therefore, further the orts of spherical coordinates ${\bf e}_{r}$,
${\bf e}_{\theta}$, and ${\bf e}_{\varphi}$ can be considered as constants.
The Maxwell equations are reduced to the set of ordinary differential
equations

$\displaystyle{\bf e}_{r}^{\times}\frac{{\rm d}{\bf E}^{l}}{{\rm
d}r}+\frac{1}{r}{\bf e}_{r}^{\times}{\bf E}^{l}-\frac{{\rm
i}\sqrt{l(l+1)}}{r}{\bf e}_{\varphi}^{\times}{\bf E}^{l}={\rm i}k\mu{\bf
H}^{l},$ $\displaystyle{\bf e}_{r}^{\times}\frac{{\rm d}{\bf H}^{l}}{{\rm
d}r}+\frac{1}{r}{\bf e}_{r}^{\times}{\bf H}^{l}-\frac{{\rm
i}\sqrt{l(l+1)}}{r}{\bf e}_{\varphi}^{\times}{\bf H}^{l}=-{\rm
i}k\varepsilon{\bf E}^{l}.$ (34)

Equations (34) allow to determine the radial dependence of the fields.
Multiplying the set of equations (34) by the unit vector ${\bf e}_{r}$ we can
express the longitudinal components of the fields as follows

$E^{l}_{r}=-\frac{\sqrt{l(l+1)}}{\varepsilon kr}H^{l}_{\theta},\qquad
H^{l}_{r}=\frac{\sqrt{l(l+1)}}{\mu kr}E^{l}_{\theta}.$ (35)

The tangential field components ${\bf E}^{l}_{\rm t}=I{\bf
E}^{l}=E^{l}_{\theta}{\bf e}_{\theta}+E^{l}_{\varphi}{\bf e}_{\varphi}$ and
${\bf H}^{l}_{\rm t}=I{\bf H}^{l}$ are determined from the equations which can
be written in matrix form:

$\frac{{\rm d}(r{\bf W})}{{\rm d}r}=ikM(r{\bf W}),$ (36)

where

$\displaystyle{\bf W}=\left(\begin{array}[]{c}{\bf H}^{l}_{\rm t}\\\ {\bf
E}^{l}_{\rm t}\end{array}\right),\qquad
M=\left(\begin{array}[]{cc}0&\varepsilon A\\\ -\mu
A&0\end{array}\right),\qquad A={\bf
e}_{r}^{\times}-\frac{l(l+1)}{\varepsilon\mu k^{2}r^{2}}{\bf
e}_{\varphi}\otimes{\bf e}_{\theta}.$ (41)

Equation (36) is satisfied for inhomogeneous media $\varepsilon(r)$, $\mu(r)$,
too. Such matrix equation can be solved numerically for arbitrary medium, or
analytically for homogeneous one. Let us find tangential field components
${\bf W}$ when $\varepsilon=const$ and $\mu=const$. The simplest way is to
write the equation for projection
$W_{\theta}=(H^{l}_{\theta},E^{l}_{\theta})$:

$\frac{{\rm d}^{2}(rW_{\theta})}{{\rm
d}r^{2}}+\left(k^{2}\varepsilon\mu-\frac{l(l+1)}{r^{2}}\right)(rW_{\theta})=0,$
(42)

The solutions of the equation (42) are well-known and can be presented as

$W_{\theta}=f^{(1)}\left(\begin{array}[]{c}c_{1}\\\
c^{\prime}_{1}\end{array}\right)+f^{(2)}\left(\begin{array}[]{c}c_{2}\\\
c^{\prime}_{2}\end{array}\right)=(f^{(1)}{\bf c}_{1}+f^{(2)}{\bf
c}_{2})\left(\begin{array}[]{c}{\bf e}_{\theta}\\\ {\bf
e}_{\varphi}\end{array}\right),$ (43)

where ${\bf c}_{1}$ and ${\bf c}_{2}$ are constant vectors. The couples of
independent solutions are spherical Bessel functions $f^{(1)}=j_{l}(nkr)$,
$f^{(2)}=j_{-l-1}(nkr)$ or spherical Hankel functions of the first and second
kind $f^{(1)}=h_{l}^{(1)}(nkr)$, $f^{(2)}=h_{l}^{(2)}(nkr)$.
$n=\sqrt{\varepsilon\mu}$ is the refractive index. After determining the
$\varphi$-projections of the fields as

$W_{\varphi}=\frac{{\rm
i}}{kr\varepsilon\mu}\left(\begin{array}[]{cc}0&-\varepsilon\\\
\mu&0\end{array}\right)\frac{{\rm d}(rW_{\theta})}{{\rm d}r}$ (44)

one can write the transverse vector field

${\bf W}(r)=\left(\begin{array}[]{cc}\eta_{1}(r)&\eta_{2}(r)\\\
\zeta_{1}(r)&\zeta_{2}(r)\end{array}\right)\left(\begin{array}[]{c}{\bf
c}_{1}\\\ {\bf c}_{2}\end{array}\right),$ (45)

where

$\displaystyle\eta_{1,2}=f^{(1,2)}{\bf e}_{\theta}\otimes{\bf
e}_{\theta}-\frac{{\rm i}}{\mu kr}\frac{{\rm d}(rf^{(1,2)})}{{\rm d}r}{\bf
e}_{\varphi}\otimes{\bf e}_{\varphi},$ $\displaystyle\zeta_{1,2}=f^{(1,2)}{\bf
e}_{\theta}\otimes{\bf e}_{\varphi}+\frac{{\rm i}}{\varepsilon kr}\frac{{\rm
d}(rf^{(1,2)})}{{\rm d}r}{\bf e}_{\varphi}\otimes{\bf e}_{\theta}.$ (46)

Tangential field vector ${\bf W}$ plays an important part, because it is
continuous on the spherical surface. That is why it can be applied for the
study of electromagnetic wave diffraction by a sphere.

##  5 Conclusion

Thus, the general solution of the Maxwell equations is of the form

$\left(\begin{array}[]{cc}{\bf H}({\bf r})\\\ {\bf E}({\bf
r})\end{array}\right)=\sum_{l=0}^{\infty}\sum_{m=-l}^{l}F_{lm}(\theta,\varphi)V^{l}(r)\left(\begin{array}[]{cc}\eta^{l}_{1}(r)&\eta^{l}_{2}(r)\\\
\zeta^{l}_{1}(r)&\zeta^{l}_{2}(r)\end{array}\right)\left(\begin{array}[]{c}{\bf
c}^{l}_{1}\\\ {\bf c}^{l}_{2}\end{array}\right),$ (47)

where $V^{l}$ is the matrix that takes into account the longitudinal
components of electric and magnetic fields. This matrix can be easily
calculated from the equation (35). In each partial solution included into the
general one (47) the radial and angle variables are separated. Using the
orthonormalization (14) for the electrodynamic spherical harmonic $F_{lm}$,
each partial wave can be easily singled out:

$V^{l}(r)\left(\begin{array}[]{cc}\eta^{l}_{1}(r)&\eta^{l}_{2}(r)\\\
\zeta^{l}_{1}(r)&\zeta^{l}_{2}(r)\end{array}\right)\left(\begin{array}[]{c}{\bf
c}^{l}_{1}\\\ {\bf
c}^{l}_{2}\end{array}\right)=\int_{0}^{\pi}\int_{0}^{2\pi}F^{+}_{lm}(\theta^{\prime},\varphi^{\prime})\left(\begin{array}[]{cc}{\bf
H}(r,\theta^{\prime},\varphi^{\prime})\\\ {\bf
E}(r,\theta^{\prime},\varphi^{\prime})\end{array}\right)\sin\theta^{\prime}{\rm
d}\theta^{\prime}{\rm d}\varphi^{\prime}.$ (48)

This property of the electrodynamic spherical harmonic is very useful for the
investigation of electromagnetic wave scattering. In scattering, the boundary
condition is the single equation for tangential fields ${\bf W}$. It is easily
solved, if the orthonormalization is applied. Some attempts of investigation
of scattering in the similar manner as described above have been made in [7].

In the general solution (47) the constants ${\bf c}_{1,2}$ determined by
initial conditions are explicitly shown. Vectors ${\bf c}_{1}$ and ${\bf
c}_{2}$ set independent solutions. For instance, if ${\bf c}_{1}=0$, then the
radial dependence is determined by the function $f^{(2)}(r)$, and vice versa.
If ${\bf c}_{2}=0$ and $f^{(1)}(r)=h^{(1)}(nkr)$, then the field (47) is the
multipole expansion [8]. The amplitude of electric multipole field is equal to
$a_{E}(l,m)={\b{c}}_{1}{\bf e}_{\theta}$, the amplitude of magnetic multipole
field is equal to $a_{M}(l,m)={\bf c}_{1}{\bf e}_{\varphi}$. So, vector ${\bf
c}_{1}$ can be called vector amplitude of multipole fields.

In further investigations we will study the scattering and multipole expansion
of electromagnetic fields in details.

## References

## References

  * [1] Newman E and Penrose R 1966 Note on the Bondi-Metzner-Sachs group J. Math. Phys. 7 836–870
  * [2] James R W 1976 New Tensor Spherical Harmonics, for Application to the Partial Differential Equations of Mathematical Physics Phil. Trans. R. Soc. London. Series A 281 195–221
  * [3] Sandberg V D 1978 Tensor spherical harmonics on $S^{2}$ and $S^{3}$ as eigenvalue problems J. Math. Phys. 19 2441–2446
  * [4] Winter J 1982 Tensor spherical harmonics Letters in Mathematical Physics 6 91–96
  * [5] Kostelec P, Maslen D K, Rockmore D N, and Healy Jr D 2000 Computational harmonic analysis for tensor fields on the two-sphere J. Comput. Phys. 162 514–535
  * [6] Fedorov F I 1976 Theory of Gyrotropy (Minsk: Nauka i Tehnika)
  * [7] Novitsky A V and Barkovsky L M 2006 Evolution and impedance operators of spherically symmetric bianisotropic media J. Phys. A: Math. Gen. 39 7543-60
  * [8] Jackson J D 1998 Classical electrodynamics (New York: John Wiley & Sons)