{% extends "base.html" %} {% block title %} {{block.super}} - Install {% endblock %} {% block main %}
Edit your preferences, and watch the magic happen.


{% extends 'templates/nhs_transaction_layout.html' %} {% block pageTitle %}NHS.UK prototyping kit - 111 design{% endblock %} {% block afterHeader %} {% include "includes/service-headers/111-design.html" %} {% endblock %} {% block content %} NHS.UK 111Design prototype Week commencing Monday 7 November 2016 Design Design session cards Home care Home care 2 Home care 3 Home care 4 See a pharmacist See a pharmacist 2 See a pharmacist 3 See a pharmacist 4 See a pharmacist 5 Call 999 Call 999 2 Call 999 3 {% include "includes/prototype_frontend/right_hand_third.html" %} {% endblock %}


McLovin
Home
My Playlists
Settings
Account
Preferences
Logout
Future Playlsit
Play
Share
Rick Roll playlist
Play
Share
Drain Playlist
Play
Share


Lynn is the co-owner of one of the most premiere dog daycares in the Greater Madison area! Because Wunderdog believes in the mental and physical balance of each and every dog (and human!), we believe that daycare can play an intregal part in supporting this balance if the program is enriching, positive, and safe. Zen Dogs Center was designed soley for daycare and training; therapeutic in every way! Zen Dogs Center.
Wunderdog will be the resident training company within ZDC, so don’t worry - you will still be able to receive training with Wunderdog just as you have before! And, now you will be able to train at ZDC in group classes or personal sessions! Especially during this pandemic, we are able to offer distance (masked) sessions in one of our three large gyms or one of our four astro-turfed yards...
For more information, visit Zen Dogs Center at Zen Dogs Center!
See you soon!
With a better understanding of innate dog behavior, both owner and dog will benefit, and the Wunderdog team strives to teach this to our clients in a way that everyone can understand.
{# #}Owner and President of Wunderdog LLC, Lynn Wunderli has been training dogs for over 35 years. Her knowledge and experience with dogs began as a child as her father was a successful breeder and trainer of English Setters and Pointers for over 55 years in Illinois.
Read moreWhether you choose personalized training or a group class, Wunderdog, LLC will provide you exceptional service from nose to tail! Our team will work with you to create a custom-designed training plan specifically for you and your four-footed family member.
Learn moreWunderdog is a family business based on trust, knowledge, and integrity. Whether our clients have chosen training, in-home routine and specialize visits, or group classes, we have provided these services with instruction and insight. Check out their experiences!


{include file="common/title"} {include file="common/css"} {block name="css"}{/block}
{include file="common/head"} {block name="body"}{/block} {include file="common/foot"} {include file="common/script"} {block name="script"}{/block}


We provide pennyworth solution to all your needs and service related to education
and for your personal growth by choosing an expert that suits your budget and quality
parameters. Our mission is to provide high quality, 100% unique and accurate assignment
help online from our qualified experts at reasonable prices
Get true guidance from our expert at free of cost because they know student concerns.
No matter where you are or not our team update you regularly from beginning to
end.
We provide all platform for payment and transaction is encrypted and 100% secure.
On time or before time submission is always our first priority to improve your grade
To resolve student concern for not copy content from anywhere. We provide unique content.
We provide immediate service as per given deadline & you directly access to our expert.


Dual Viewer
A draggable UI component which reveals two panes.
IE9+ with polyfill.
To support IE<=11, you will need to add classList
polyfill.
Dependencies
Odo Window Events, Odo Helpers, Odo Draggable, Odo Object Fit.
Object.assign
should also be available.
Styles
This component has required css - css/odo-dual-viewer.css. The theme (css/odo-dual-viewer-theme.css) should also be included and customized to your design.
Horizontal With Images
By default, the dual viewer is horizontal. When the user releases the scrubber, it will animate to the start, end, or center, depending on the zones
option.
Slide right or left to reveal more of one perspective.
Markup
This markup needs to be followed closely and is dependent on the dual viewer's css. You may put anything inside odo-dual-viewer__object
. This example is full-bleed, but it doesn’t have to be. See the video demo below for one that isn’t.

Setup
var dualViewer = new OdoDualViewer(document.getElementById('regular'), {});
Scrubber Boundaries
The odo-dual-viewer__scrubber-bounds
element determines the limits of the draggable (scrubber). In this example, it is wrapped in a container
element, but that only applies to this page. The dual viewer will calculate the limits based on the distance from the left edge of odo-dual-viewer
to the left edge of odo-dual-viewer__scrubber-bounds
. The right edge is based on the width
of the scrubber bounds element.
Animating the dual viewer
You can programmatically animate the position of the scrubber in the dual viewer with the animateTo
method. The only parameter is a value between 0
and 1
, which represents the percentage to animate to. This value will be clamped to the scrubber’s limits.
document.getElementById('animate-to').addEventListener('click', function () {
var value = parseInt(document.getElementById('animate-to-value').value, 10);
dualViewer.animateTo(value / 100);
});
Vertical With Images
Slide right or left to reveal more of one perspective.
Markup
The markup for this vertical demo is identical to the horizontal demo above, except the id attribute is different.
Setup
var vertical = new OdoDualViewer(document.getElementById('vert'), {
isVertical: true
});
Horizontal With Videos
The dual viewer can hold any type of content. This one holds some html5 videos from gfycat. The first is Moon and cloudlapse and the second is Pouring Beauty Iceland.
Slide right or left to reveal more of one perspective.
Markup
The markup is nearly identical.
s have been replaced by <video>
s and the scrubber bounds element now has inline styles defining its width.

<video class="odo-dual-viewer__media" autoplay loop>
<source src="https://zippy.gfycat.com/BestReflectingJunco.webm" type="video/webm">
<source src="https://zippy.gfycat.com/BestReflectingJunco.mp4" type="video/mp4">
</video>

<video class="odo-dual-viewer__media" autoplay loop>
<source src="https://zippy.gfycat.com/EsteemedJovialHoneybadger.webm" type="video/webm">
<source src="https://zippy.gfycat.com/EsteemedJovialHoneybadger.mp4" type="video/mp4">
</video>

Slide right or left to reveal more of one perspective.

Setup
var vid = new OdoDualViewer(document.getElementById('dual-viewer-videos'), {
animationDuration: 400,
zones: [0.3, 0.4, 0.6, 0.7]
});
Zones
When the user releases the scrubber, the dual viewer uses the position of scrubber as a percentage as a zone. The default zones
option is [0.33, 0.33, 0.66, 0.66]
. The first value in the array represents the lower zone. If the scrubber position (as a percent) is less than that value, the dual viewer will animate to 0
percent (within the boundaries). The second and third values create a zone for the middle. When the scrubber is released in the middle zone, it animates to the center. The last value is the upper zone. Any value above it will animate the scrubber to 1
percent (within the boundaries).

Zones work the same way for vertical dual viewers with the lower zone on top and the upper zone at the bottom.
Like the example above, zones can overlap. The lower and upper zones take precedence over the middle.
Events
The dual viewer emits a CAME_TO_REST
event when the handle stops animating.
vid.on(OdoDualViewer.EventType.CAME_TO_REST, function(event) {
switch (event.position) {
case OdoDualViewer.Position.START:
console.log('Handle came to rest at the start');
break;
case OdoDualViewer.Position.END:
console.log('Handle came to rest at the end');
break;
case OdoDualViewer.Position.CENTER:
console.log('Handle came to rest in the center');
break;
}
});
No Zones
This example does not use zones to animate to a resting position.
Setup
var noZones = new OdoDualViewer(document.getElementById('no-zones'), {
hasZones: false,
startPosition: 0.1
});
Starting Position
The starting position of the scrubber and the amount revealed when the module is initialized can be changed. By default, it's 50%, but using the startPosition
option and a value between zero and one, you may change this behavior.


"SCOURGING "(1 Kings 12:11). Variously administered. In no case were the stripes to exceed forty (Deuteronomy 25:3; comp. 2 Corinthians 11:24). In the time of the apostles, in consequence of the passing of what was called the Porcian law, no Roman citizen could be scourged in any case(Acts 16:22-37). (See BASTINADO.) In the scourging of our Lord(Matthew 27:26; Mark 15:15) the words of prophecy (Isaiah 53:5) were fulfilled.
"SCOURGING "(1 Kings 12:11). Variously administered. In no case were the stripes to exceed forty (Deuteronomy 25:3; comp. 2 Corinthians 11:24). In the time of the apostles, in consequence of the passing of what was called the Porcian law, no Roman citizen could be scourged in any case(Acts 16:22-37). (See BASTINADO.) In the scourging of our Lord(Matthew 27:26; Mark 15:15) the words of prophecy (Isaiah 53:5) were fulfilled.


Spark Security
- Spark Security: Things You Need To Know
- Spark RPC (Communication protocol between Spark processes)
- Local Storage Encryption
- Web UI
- Configuring Ports for Network Security
- Kerberos
- Event Logging
- Persisting driver logs in client mode
Spark Security: Things You Need To Know
Security in Spark is OFF by default. This could mean you are vulnerable to attack by default. Spark supports multiple deployments types and each one supports different levels of security. Not all deployment types will be secure in all environments and none are secure by default. Be sure to evaluate your environment, what Spark supports, and take the appropriate measure to secure your Spark deployment.
There are many different types of security concerns. Spark does not necessarily protect against all things. Listed below are some of the things Spark supports. Also check the deployment documentation for the type of deployment you are using for deployment specific settings. Anything not documented, Spark does not support.
Spark RPC (Communication protocol between Spark processes)
Authentication
Spark currently supports authentication for RPC channels using a shared secret. Authentication can
be turned on by setting the spark.authenticate
configuration parameter.
The exact mechanism used to generate and distribute the shared secret is deployment-specific. Unless
specified below, the secret must be defined by setting the spark.authenticate.secret
config
option. The same secret is shared by all Spark applications and daemons in that case, which limits
the security of these deployments, especially on multi-tenant clusters.
The REST Submission Server and the MesosClusterDispatcher do not support authentication. You should ensure that all network access to the REST API & MesosClusterDispatcher (port 6066 and 7077 respectively by default) are restricted to hosts that are trusted to submit jobs.
YARN
For Spark on YARN, Spark will automatically handle generating and distributing the shared secret. Each application will use a unique shared secret. In the case of YARN, this feature relies on YARN RPC encryption being enabled for the distribution of secrets to be secure.
Kubernetes
On Kubernetes, Spark will also automatically generate an authentication secret unique to each application. The secret is propagated to executor pods using environment variables. This means that any user that can list pods in the namespace where the Spark application is running can also see their authentication secret. Access control rules should be properly set up by the Kubernetes admin to ensure that Spark authentication is secure.
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.authenticate |
false | Whether Spark authenticates its internal connections. | 1.0.0 |
spark.authenticate.secret |
None | The secret key used authentication. See above for when this configuration should be set. | 1.0.0 |
Alternatively, one can mount authentication secrets using files and Kubernetes secrets that the user mounts into their pods.
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.authenticate.secret.file |
None | Path pointing to the secret key to use for securing connections. Ensure that the contents of the file have been securely generated. This file is loaded on both the driver and the executors unless other settings override this (see below). | 3.0.0 |
spark.authenticate.secret.driver.file |
The value of spark.authenticate.secret.file |
When specified, overrides the location that the Spark driver reads to load the secret.
Useful when in client mode, when the location of the secret file may differ in the pod versus
the node the driver is running in. When this is specified,
spark.authenticate.secret.executor.file must be specified so that the driver
and the executors can both use files to load the secret key. Ensure that the contents of the file
on the driver is identical to the contents of the file on the executors.
|
3.0.0 |
spark.authenticate.secret.executor.file |
The value of spark.authenticate.secret.file |
When specified, overrides the location that the Spark executors read to load the secret.
Useful in client mode, when the location of the secret file may differ in the pod versus
the node the driver is running in. When this is specified,
spark.authenticate.secret.driver.file must be specified so that the driver
and the executors can both use files to load the secret key. Ensure that the contents of the file
on the driver is identical to the contents of the file on the executors.
|
3.0.0 |
Note that when using files, Spark will not mount these files into the containers for you. It is up you to ensure that the secret files are deployed securely into your containers and that the driver’s secret file agrees with the executors’ secret file.
Encryption
Spark supports AES-based encryption for RPC connections. For encryption to be enabled, RPC authentication must also be enabled and properly configured. AES encryption uses the Apache Commons Crypto library, and Spark’s configuration system allows access to that library’s configuration for advanced users.
There is also support for SASL-based encryption, although it should be considered deprecated. It is still required when talking to shuffle services from Spark versions older than 2.2.0.
The following table describes the different options available for configuring this feature.
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.network.crypto.enabled |
false | Enable AES-based RPC encryption, including the new authentication protocol added in 2.2.0. | 2.2.0 |
spark.network.crypto.keyLength |
128 | The length in bits of the encryption key to generate. Valid values are 128, 192 and 256. | 2.2.0 |
spark.network.crypto.keyFactoryAlgorithm |
PBKDF2WithHmacSHA1 | The key factory algorithm to use when generating encryption keys. Should be one of the algorithms supported by the javax.crypto.SecretKeyFactory class in the JRE being used. | 2.2.0 |
spark.network.crypto.config.* |
None |
Configuration values for the commons-crypto library, such as which cipher implementations to
use. The config name should be the name of commons-crypto configuration without the
commons.crypto prefix.
|
2.2.0 |
spark.network.crypto.saslFallback |
true | Whether to fall back to SASL authentication if authentication fails using Spark's internal mechanism. This is useful when the application is connecting to old shuffle services that do not support the internal Spark authentication protocol. On the shuffle service side, disabling this feature will block older clients from authenticating. | 2.2.0 |
spark.authenticate.enableSaslEncryption |
false | Enable SASL-based encrypted communication. | 2.2.0 |
spark.network.sasl.serverAlwaysEncrypt |
false | Disable unencrypted connections for ports using SASL authentication. This will deny connections from clients that have authentication enabled, but do not request SASL-based encryption. | 1.4.0 |
Local Storage Encryption
Spark supports encrypting temporary data written to local disks. This covers shuffle files, shuffle
spills and data blocks stored on disk (for both caching and broadcast variables). It does not cover
encrypting output data generated by applications with APIs such as saveAsHadoopFile
or
saveAsTable
. It also may not cover temporary files created explicitly by the user.
The following settings cover enabling encryption for data written to disk:
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.io.encryption.enabled |
false | Enable local disk I/O encryption. Currently supported by all modes except Mesos. It's strongly recommended that RPC encryption be enabled when using this feature. | 2.1.0 |
spark.io.encryption.keySizeBits |
128 | IO encryption key size in bits. Supported values are 128, 192 and 256. | 2.1.0 |
spark.io.encryption.keygen.algorithm |
HmacSHA1 | The algorithm to use when generating the IO encryption key. The supported algorithms are described in the KeyGenerator section of the Java Cryptography Architecture Standard Algorithm Name Documentation. | 2.1.0 |
spark.io.encryption.commons.config.* |
None |
Configuration values for the commons-crypto library, such as which cipher implementations to
use. The config name should be the name of commons-crypto configuration without the
commons.crypto prefix.
|
2.1.0 |
Web UI
Authentication and Authorization
Enabling authentication for the Web UIs is done using javax servlet filters. You will need a filter that implements the authentication method you want to deploy. Spark does not provide any built-in authentication filters.
Spark also supports access control to the UI when an authentication filter is present. Each application can be configured with its own separate access control lists (ACLs). Spark differentiates between “view” permissions (who is allowed to see the application’s UI), and “modify” permissions (who can do things like kill jobs in a running application).
ACLs can be configured for either users or groups. Configuration entries accept comma-separated
lists as input, meaning multiple users or groups can be given the desired privileges. This can be
used if you run on a shared cluster and have a set of administrators or developers who need to
monitor applications they may not have started themselves. A wildcard (*
) added to specific ACL
means that all users will have the respective privilege. By default, only the user submitting the
application is added to the ACLs.
Group membership is established by using a configurable group mapping provider. The mapper is
configured using the spark.user.groups.mapping
config option, described in the table
below.
The following options control the authentication of Web UIs:
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.ui.filters |
None | See the Spark UI configuration for how to configure filters. | 1.0.0 |
spark.acls.enable |
false | Whether UI ACLs should be enabled. If enabled, this checks to see if the user has access permissions to view or modify the application. Note this requires the user to be authenticated, so if no authentication filter is installed, this option does not do anything. | 1.1.0 |
spark.admin.acls |
None | Comma-separated list of users that have view and modify access to the Spark application. | 1.1.0 |
spark.admin.acls.groups |
None | Comma-separated list of groups that have view and modify access to the Spark application. | 2.0.0 |
spark.modify.acls |
None | Comma-separated list of users that have modify access to the Spark application. | 1.1.0 |
spark.modify.acls.groups |
None | Comma-separated list of groups that have modify access to the Spark application. | 2.0.0 |
spark.ui.view.acls |
None | Comma-separated list of users that have view access to the Spark application. | 1.0.0 |
spark.ui.view.acls.groups |
None | Comma-separated list of groups that have view access to the Spark application. | 2.0.0 |
spark.user.groups.mapping |
org.apache.spark.security.ShellBasedGroupsMappingProvider |
The list of groups for a user is determined by a group mapping service defined by the trait
org.apache.spark.security.GroupMappingServiceProvider , which can be configured by
this property.
By default, a Unix shell-based implementation is used, which collects this information from the host OS. Note: This implementation supports only Unix/Linux-based environments. Windows environment is currently not supported. However, a new platform/protocol can be supported by implementing the trait mentioned above. |
2.0.0 |
On YARN, the view and modify ACLs are provided to the YARN service when submitting applications, and control who has the respective privileges via YARN interfaces.
Spark History Server ACLs
Authentication for the SHS Web UI is enabled the same way as for regular applications, using servlet filters.
To enable authorization in the SHS, a few extra options are used:
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.history.ui.acls.enable |
false |
Specifies whether ACLs should be checked to authorize users viewing the applications in
the history server. If enabled, access control checks are performed regardless of what the
individual applications had set for spark.ui.acls.enable . The application owner
will always have authorization to view their own application and any users specified via
spark.ui.view.acls and groups specified via spark.ui.view.acls.groups
when the application was run will also have authorization to view that application.
If disabled, no access control checks are made for any application UIs available through
the history server.
|
1.0.1 |
spark.history.ui.admin.acls |
None | Comma separated list of users that have view access to all the Spark applications in history server. | 2.1.1 |
spark.history.ui.admin.acls.groups |
None | Comma separated list of groups that have view access to all the Spark applications in history server. | 2.1.1 |
The SHS uses the same options to configure the group mapping provider as regular applications. In this case, the group mapping provider will apply to all UIs server by the SHS, and individual application configurations will be ignored.
SSL Configuration
Configuration for SSL is organized hierarchically. The user can configure the default SSL settings which will be used for all the supported communication protocols unless they are overwritten by protocol-specific settings. This way the user can easily provide the common settings for all the protocols without disabling the ability to configure each one individually. The following table describes the SSL configuration namespaces:
| Config Namespace | Component |
|---|---|
spark.ssl |
The default SSL configuration. These values will apply to all namespaces below, unless explicitly overridden at the namespace level. |
spark.ssl.ui |
Spark application Web UI |
spark.ssl.standalone |
Standalone Master / Worker Web UI |
spark.ssl.historyServer |
History Server Web UI |
The full breakdown of available SSL options can be found below. The ${ns}
placeholder should be
replaced with one of the above namespaces.
| Property Name | Default | Meaning |
|---|---|---|
${ns}.enabled |
false | Enables SSL. When enabled, ${ns}.ssl.protocol is required. |
${ns}.port |
None |
The port where the SSL service will listen on.
The port must be defined within a specific namespace configuration. The default namespace is ignored when reading this configuration. When not set, the SSL port will be derived from the non-SSL port for the same service. A value of "0" will make the service bind to an ephemeral port. |
${ns}.enabledAlgorithms |
None |
A comma-separated list of ciphers. The specified ciphers must be supported by JVM.
The reference list of protocols can be found in the "JSSE Cipher Suite Names" section of the Java security guide. The list for Java 8 can be found at this page. Note: If not set, the default cipher suite for the JRE will be used. |
${ns}.keyPassword |
None | The password to the private key in the key store. |
${ns}.keyStore |
None | Path to the key store file. The path can be absolute or relative to the directory in which the process is started. |
${ns}.keyStorePassword |
None | Password to the key store. |
${ns}.keyStoreType |
JKS | The type of the key store. |
${ns}.protocol |
None |
TLS protocol to use. The protocol must be supported by JVM.
The reference list of protocols can be found in the "Additional JSSE Standard Names" section of the Java security guide. For Java 8, the list can be found at this page. |
${ns}.needClientAuth |
false | Whether to require client authentication. |
${ns}.trustStore |
None | Path to the trust store file. The path can be absolute or relative to the directory in which the process is started. |
${ns}.trustStorePassword |
None | Password for the trust store. |
${ns}.trustStoreType |
JKS | The type of the trust store. |
Spark also supports retrieving ${ns}.keyPassword
, ${ns}.keyStorePassword
and ${ns}.trustStorePassword
from
Hadoop Credential Providers.
User could store password into credential file and make it accessible by different components, like:
hadoop credential create spark.ssl.keyPassword -value password \
-provider jceks://hdfs@nn1.example.com:9001/user/backup/ssl.jceks
To configure the location of the credential provider, set the hadoop.security.credential.provider.path
config option in the Hadoop configuration used by Spark, like:
<property>
<name>hadoop.security.credential.provider.path</name>
<value>jceks://hdfs@nn1.example.com:9001/user/backup/ssl.jceks</value>
</property>
Or via SparkConf “spark.hadoop.hadoop.security.credential.provider.path=jceks://hdfs@nn1.example.com:9001/user/backup/ssl.jceks”.
Preparing the key stores
Key stores can be generated by keytool
program. The reference documentation for this tool for
Java 8 is here.
The most basic steps to configure the key stores and the trust store for a Spark Standalone
deployment mode is as follows:
- Generate a key pair for each node
- Export the public key of the key pair to a file on each node
- Import all exported public keys into a single trust store
- Distribute the trust store to the cluster nodes
YARN mode
To provide a local trust store or key store file to drivers running in cluster mode, they can be
distributed with the application using the --files
command line argument (or the equivalent
spark.files
configuration). The files will be placed on the driver’s working directory, so the TLS
configuration should just reference the file name with no absolute path.
Distributing local key stores this way may require the files to be staged in HDFS (or other similar distributed file system used by the cluster), so it’s recommended that the underlying file system be configured with security in mind (e.g. by enabling authentication and wire encryption).
Standalone mode
The user needs to provide key stores and configuration options for master and workers. They have to
be set by attaching appropriate Java system properties in SPARK_MASTER_OPTS
and in
SPARK_WORKER_OPTS
environment variables, or just in SPARK_DAEMON_JAVA_OPTS
.
The user may allow the executors to use the SSL settings inherited from the worker process. That
can be accomplished by setting spark.ssl.useNodeLocalConf
to true
. In that case, the settings
provided by the user on the client side are not used.
Mesos mode
Mesos 1.3.0 and newer supports Secrets
primitives as both file-based and environment based
secrets. Spark allows the specification of file-based and environment variable based secrets with
spark.mesos.driver.secret.filenames
and spark.mesos.driver.secret.envkeys
, respectively.
Depending on the secret store backend secrets can be passed by reference or by value with the
spark.mesos.driver.secret.names
and spark.mesos.driver.secret.values
configuration properties,
respectively.
Reference type secrets are served by the secret store and referred to by name, for example
/mysecret
. Value type secrets are passed on the command line and translated into their
appropriate files or environment variables.
HTTP Security Headers
Apache Spark can be configured to include HTTP headers to aid in preventing Cross Site Scripting (XSS), Cross-Frame Scripting (XFS), MIME-Sniffing, and also to enforce HTTP Strict Transport Security.
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.ui.xXssProtection |
1; mode=block |
Value for HTTP X-XSS-Protection response header. You can choose appropriate value
from below:
|
2.3.0 |
spark.ui.xContentTypeOptions.enabled |
true |
When enabled, X-Content-Type-Options HTTP response header will be set to "nosniff". | 2.3.0 |
spark.ui.strictTransportSecurity |
None |
Value for HTTP Strict Transport Security (HSTS) Response Header. You can choose appropriate
value from below and set expire-time accordingly. This option is only used when
SSL/TLS is enabled.
|
2.3.0 |
Configuring Ports for Network Security
Generally speaking, a Spark cluster and its services are not deployed on the public internet. They are generally private services, and should only be accessible within the network of the organization that deploys Spark. Access to the hosts and ports used by Spark services should be limited to origin hosts that need to access the services.
Below are the primary ports that Spark uses for its communication and how to configure those ports.
Standalone mode only
| From | To | Default Port | Purpose | Configuration Setting | Notes |
|---|---|---|---|---|---|
| Browser | Standalone Master | 8080 | Web UI | spark.master.ui.port / |
Jetty-based. Standalone mode only. |
| Browser | Standalone Worker | 8081 | Web UI | spark.worker.ui.port / |
Jetty-based. Standalone mode only. |
| Driver / Standalone Worker |
Standalone Master | 7077 | Submit job to cluster / Join cluster |
SPARK_MASTER_PORT |
Set to "0" to choose a port randomly. Standalone mode only. |
| External Service | Standalone Master | 6066 | Submit job to cluster via REST API | spark.master.rest.port |
Use spark.master.rest.enabled to enable/disable this service. Standalone mode only. |
| Standalone Master | Standalone Worker | (random) | Schedule executors | SPARK_WORKER_PORT |
Set to "0" to choose a port randomly. Standalone mode only. |
All cluster managers
| From | To | Default Port | Purpose | Configuration Setting | Notes |
|---|---|---|---|---|---|
| Browser | Application | 4040 | Web UI | spark.ui.port |
Jetty-based |
| Browser | History Server | 18080 | Web UI | spark.history.ui.port |
Jetty-based |
| Executor / Standalone Master |
Driver | (random) | Connect to application / Notify executor state changes |
spark.driver.port |
Set to "0" to choose a port randomly. |
| Executor / Driver | Executor / Driver | (random) | Block Manager port | spark.blockManager.port |
Raw socket via ServerSocketChannel |
Kerberos
Spark supports submitting applications in environments that use Kerberos for authentication.
In most cases, Spark relies on the credentials of the current logged in user when authenticating
to Kerberos-aware services. Such credentials can be obtained by logging in to the configured KDC
with tools like kinit
.
When talking to Hadoop-based services, Spark needs to obtain delegation tokens so that non-local processes can authenticate. Spark ships with support for HDFS and other Hadoop file systems, Hive and HBase.
When using a Hadoop filesystem (such HDFS or WebHDFS), Spark will acquire the relevant tokens for the service hosting the user’s home directory.
An HBase token will be obtained if HBase is in the application’s classpath, and the HBase
configuration has Kerberos authentication turned (hbase.security.authentication=kerberos
).
Similarly, a Hive token will be obtained if Hive is in the classpath, and the configuration includes
URIs for remote metastore services (hive.metastore.uris
is not empty).
If an application needs to interact with other secure Hadoop filesystems, their URIs need to be
explicitly provided to Spark at launch time. This is done by listing them in the
spark.kerberos.access.hadoopFileSystems
property, described in the configuration section below.
Spark also supports custom delegation token providers using the Java Services
mechanism (see java.util.ServiceLoader
). Implementations of
org.apache.spark.security.HadoopDelegationTokenProvider
can be made available to Spark
by listing their names in the corresponding file in the jar’s META-INF/services
directory.
Delegation token support is currently only supported in YARN and Mesos modes. Consult the deployment-specific page for more information.
The following options provides finer-grained control for this feature:
| Property Name | Default | Meaning | Since Version |
|---|---|---|---|
spark.security.credentials.${service}.enabled |
true |
Controls whether to obtain credentials for services when security is enabled. By default, credentials for all supported services are retrieved when those services are configured, but it's possible to disable that behavior if it somehow conflicts with the application being run. | 2.3.0 |
spark.kerberos.access.hadoopFileSystems |
(none) |
A comma-separated list of secure Hadoop filesystems your Spark application is going to access. For
example, spark.kerberos.access.hadoopFileSystems=hdfs://nn1.com:8032,hdfs://nn2.com:8032,
webhdfs://nn3.com:50070 . The Spark application must have access to the filesystems listed
and Kerberos must be properly configured to be able to access them (either in the same realm
or in a trusted realm). Spark acquires security tokens for each of the filesystems so that
the Spark application can access those remote Hadoop filesystems.
|
3.0.0 |
Long-Running Applications
Long-running applications may run into issues if their run time exceeds the maximum delegation token lifetime configured in services it needs to access.
This feature is not available everywhere. In particular, it’s only implemented on YARN and Kubernetes (both client and cluster modes), and on Mesos when using client mode.
Spark supports automatically creating new tokens for these applications. There are two ways to enable this functionality.
Using a Keytab
By providing Spark with a principal and keytab (e.g. using spark-submit
with --principal
and --keytab
parameters), the application will maintain a valid Kerberos login that can be
used to retrieve delegation tokens indefinitely.
Note that when using a keytab in cluster mode, it will be copied over to the machine running the Spark driver. In the case of YARN, this means using HDFS as a staging area for the keytab, so it’s strongly recommended that both YARN and HDFS be secured with encryption, at least.
Using a ticket cache
By setting spark.kerberos.renewal.credentials
to ccache
in Spark’s configuration, the local
Kerberos ticket cache will be used for authentication. Spark will keep the ticket renewed during its
renewable life, but after it expires a new ticket needs to be acquired (e.g. by running kinit
).
It’s up to the user to maintain an updated ticket cache that Spark can use.
The location of the ticket cache can be customized by setting the KRB5CCNAME
environment
variable.
Secure Interaction with Kubernetes
When talking to Hadoop-based services behind Kerberos, it was noted that Spark needs to obtain delegation tokens so that non-local processes can authenticate. These delegation tokens in Kubernetes are stored in Secrets that are shared by the Driver and its Executors. As such, there are three ways of submitting a Kerberos job:
In all cases you must define the environment variable: HADOOP_CONF_DIR
or
spark.kubernetes.hadoop.configMapName.
It also important to note that the KDC needs to be visible from inside the containers.
If a user wishes to use a remote HADOOP_CONF directory, that contains the Hadoop configuration files, this could be
achieved by setting spark.kubernetes.hadoop.configMapName
to a pre-existing ConfigMap.
 Submitting like in (3) however specifying a pre-created krb5 ConfigMap and pre-created HADOOP_CONF_DIR
ConfigMap
/opt/spark/bin/spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.HdfsTest \
--master k8s://<KUBERNETES_MASTER_ENDPOINT> \
--conf spark.executor.instances=1 \
--conf spark.app.name=spark-hdfs \
--conf spark.kubernetes.container.image=spark:latest \
--conf spark.kubernetes.kerberos.tokenSecret.name=<SECRET_TOKEN_NAME> \
--conf spark.kubernetes.kerberos.tokenSecret.itemKey=<SECRET_ITEM_KEY> \
--conf spark.kubernetes.hadoop.configMapName=<HCONF_CONFIG_MAP_NAME> \
--conf spark.kubernetes.kerberos.krb5.configMapName=<KRB_CONFIG_MAP_NAME> \
local:///opt/spark/examples/jars/spark-examples_<VERSION>.jar \
<HDFS_FILE_LOCATION>
Event Logging
If your applications are using event logging, the directory where the event logs go
(spark.eventLog.dir
) should be manually created with proper permissions. To secure the log files,
the directory permissions should be set to drwxrwxrwxt
. The owner and group of the directory
should correspond to the super user who is running the Spark History Server.
This will allow all users to write to the directory but will prevent unprivileged users from reading, removing or renaming a file unless they own it. The event log files will be created by Spark with permissions such that only the user and group have read and write access.
Persisting driver logs in client mode
If your applications persist driver logs in client mode by enabling spark.driver.log.persistToDfs.enabled
,
the directory where the driver logs go (spark.driver.log.dfsDir
) should be manually created with proper
permissions. To secure the log files, the directory permissions should be set to drwxrwxrwxt
. The owner
and group of the directory should correspond to the super user who is running the Spark History Server.
This will allow all users to write to the directory but will prevent unprivileged users from reading, removing or renaming a file unless they own it. The driver log files will be created by Spark with permissions such that only the user and group have read and write access.


This page contains examples for the automatic data binding in JavaScript.
Name:
Phone: Opinion:
Comments:
Car make:
Foods:
Languages:
Equipment:
kW:
This is some text, unrelated to the data, which is not visible if the opinion is No. The effect demonstrates the usage of binding to a chosen property.


Swisse Wellness
Swisse required an independent review of their sponsorship and ambassador program including a full set of recommendations to take the brand forward and maintain their presence as one of Australia’s pre-eminent sponsors of sport and entertainment.


Create a module that exposes methods for performing HTTP requests by given URL and headers
app.js server from server folder must be running (command "node app.js")


Show Deals
Show Slides
Upload deals and its details
Title
Description
Url
Discount string
client name
subdiscount string
product name
upload image
upload
Get all offers
Upload slide and its details
Url
text
offertext
upload image
upload
EDIT DEALS
{{offer.Description}}
DATE:{{offer.date}} {{offer._id}}