This chapter provides advanced information on securing Java EE applications.
The following topics are addressed here:
Digital certificates for GlassFish Server have already been generated and can be found in the directory domain-dir/config/
. These digital certificates are self-signed and are intended for use in a development environment; they are not intended for production purposes. For production purposes, generate your own certificates and have them signed by a Certificate Authority (CA).
To use the Secure Sockets Layer (SSL), an application or web server must have an associated certificate for each external interface, or IP address, that accepts secure connections. The theory behind this design is that a server should provide some kind of reasonable assurance that its owner is who you think it is, particularly before receiving any sensitive information. It may be useful to think of a certificate as a "digital driver's license" for an Internet address. The certificate states with which company the site is associated, along with some basic contact information about the site owner or administrator.
The digital certificate is cryptographically signed by its owner and is difficult for anyone else to forge. For sites involved in e-commerce or in any other business transaction in which authentication of identity is important, a certificate can be purchased from a well-known CA such as VeriSign or Thawte. If your server certificate is self-signed, you must install it in the GlassFish Server keystore file (keystore.jks
). If your client certificate is self-signed, you should install it in the GlassFish Server truststore file (cacerts.jks
).
Sometimes, authentication is not really a concern. For example, an administrator might simply want to ensure that data being transmitted and received by the server is private and cannot be snooped by anyone eavesdropping on the connection. In such cases, you can save the time and expense involved in obtaining a CA certificate and simply use a self-signed certificate.
SSL uses public-key cryptography, which is based on key pairs. Key pairs contain one public key and one private key. Data encrypted with one key can be decrypted only with the other key of the pair. This property is fundamental to establishing trust and privacy in transactions. For example, using SSL, the server computes a value and encrypts it by using its private key. The encrypted value is called a digital signature. The client decrypts the encrypted value by using the server's public key and compares the value to its own computed value. If the two values match, the client can trust that the signature is authentic, because only the private key could have been used to produce such a signature.
Digital certificates are used with HTTPS to authenticate web clients. The HTTPS service of most web servers will not run unless a digital certificate has been installed. Use the procedure outlined in the next section, Creating a Server Certificate, to set up a digital certificate that can be used by your application or web server to enable SSL.
One tool that can be used to set up a digital certificate is keytool
, a key and certificate management utility that ships with the JDK. This tool enables users to administer their own public/private key pairs and associated certificates for use in self-authentication, whereby the user authenticates himself or herself to other users or services, or data integrity and authentication services, using digital signatures. The tool also allows users to cache the public keys, in the form of certificates, of their communicating peers.
For a better understanding of keytool
and public-key cryptography, see Further Information about Advanced Security Topics for a link to the keytool
documentation.
A server certificate has already been created for GlassFish Server and can be found in the domain-dir/config/
directory. The server certificate is in keystore.jks
. The cacerts.jks
file contains all the trusted certificates, including client certificates.
If necessary, you can use keytool
to generate certificates. The keytool
utility stores the keys and certificates in a file termed a keystore, a repository of certificates used for identifying a client or a server. Typically, a keystore is a file that contains one client's or one server's identity. The keystore protects private keys by using a password.
If you don't specify a directory when specifying the keystore file name, the keystores are created in the directory from which the keytool
command is run. This can be the directory where the application resides, or it can be a directory common to many applications.
The general steps for creating a server certificate are as follows.
Create the keystore.
Export the certificate from the keystore.
Sign the certificate.
Import the certificate into a truststore: a repository of certificates used for verifying the certificates. A truststore typically contains more than one certificate.
The next section provides specific information on using the keytool
utility to perform these steps.
Run keytool
to generate a new key pair in the default development keystore file, keystore.jks
. This example uses the alias server-alias
to generate a new public/private key pair and wrap the public key into a self-signed certificate inside keystore.jks
. The key pair is generated by using an algorithm of type RSA, with a default password of changeit
. For more information and other examples of creating and managing keystore files, read the keytool
documentation.
|
Note: RSA is public-key encryption technology developed by RSA Data Security, Inc. |
From the directory in which you want to create the key pair, run keytool
as shown in the following steps.
Generate the server certificate.
Enter the keytool
command all on one line:
java-home/bin/keytool -genkey -alias server-alias -keyalg RSA
-keypass changeit -storepass changeit -keystore keystore.jks
When you press Enter, keytool
prompts you to enter the server name, organizational unit, organization, locality, state, and country code.
You must enter the server name in response to keytool
's first prompt, in which it asks for first and last names. For testing purposes, this can be localhost
.
Export the generated server certificate in keystore.jks
into the file server.cer
.
Enter the keytool
command all on one line:
java-home/bin/keytool -export -alias server-alias -storepass changeit
-file server.cer -keystore keystore.jks
If you want to have the certificate signed by a CA, read the example in the keytool
documentation.
To add the server certificate to the truststore file, cacerts.jks
, run keytool
from the directory where you created the keystore and server certificate.
Use the following parameters:
java-home/bin/keytool -import -v -trustcacerts -alias server-alias
-file server.cer -keystore cacerts.jks -keypass changeit
-storepass changeit
Information on the certificate, such as that shown next, will appear:
Owner: CN=localhost, OU=My Company, O=Software, L=Santa Clara, ST=CA, C=US Issuer: CN=localhost, OU=My Company, O=Software, L=Santa Clara, ST=CA, C=US Serial number: 3e932169 Valid from: Mon Nov 26 18:15:47 EST 2012 until: Sun Feb 24 18:15:47 EST 2013 Certificate fingerprints: MD5: 52:9F:49:68:ED:78:6F:39:87:F3:98:B3:6A:6B:0F:90 SHA1: EE:2E:2A:A6:9E:03:9A:3A:1C:17:4A:28:5E:97:20:78:3F: SHA256: 80:05:EC:7E:50:50:5D:AA:A3:53:F1:11:9B:19:EB:0D:20:67:C1:12: AF:42:EC:CD:66:8C:BD:99:AD:D9:76:95 Signature algorithm name: SHA256withRSA Version: 3 ... Trust this certificate? [no]:
Enter yes
, then press the Enter or Return key.
The following information appears:
Certificate was added to keystore [Storing cacerts.jks]
In the certificate
realm, user identity is set up in the GlassFish Server security context and populated with user data obtained from cryptographically verified client certificates. For step-by-step instructions for creating this type of certificate, see Working with Digital Certificates.
Follow the steps in Creating a Server Certificate to create your own server certificate, have it signed by a CA, and import the certificate into keystore.jks
.
Make sure that when you create the certificate, you follow these rules.
When you create the server certificate, keytool
prompts you to enter your first and last name. In response to this prompt, you must enter the name of your server. For testing purposes, this can be localhost
.
If you want to replace the existing keystore.jks
, you must either change your keystore's password to the default password (changeit
) or change the default password to your keystore's password.
To specify that GlassFish Server should use the new keystore for authentication and authorization decisions, you must set the JVM options for GlassFish Server so that they recognize the new keystore. To use a different keystore from the one provided for development purposes, follow these steps.
Start GlassFish Server if you haven't already done so. Information on starting the GlassFish Server can be found in Starting and Stopping GlassFish Server.
Open the GlassFish Server Administration Console in a web browser at http://localhost:4848
.
Expand Configurations, then expand server-config, then click JVM Settings.
Click the JVM Options tab.
Change the following JVM options so that they point to the location and name of the new keystore. The current settings are shown below:
-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks -Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks
If you've changed the keystore password from its default value, you need to add the password option as well:
-Djavax.net.ssl.keyStorePassword=your-new-password
Click Save, then restart GlassFish Server.
This section discusses the client authentication and mutual authentication mechanisms.
With client authentication, the web server authenticates the client by using the client's public key certificate. Client authentication is a more secure method of authentication than either basic or form-based authentication. It uses HTTP over SSL (HTTPS), in which the server authenticates the client using the client's public key certificate. SSL technology provides data encryption, server authentication, message integrity, and optional client authentication for a TCP/IP connection. You can think of a public key certificate as the digital equivalent of a passport. The certificate is issued by a trusted organization, a certificate authority (CA), and provides identification for the bearer.
Before using client authentication, make sure that the client has a valid public key certificate. For more information on creating and using public key certificates, read Working with Digital Certificates.
The following example shows how to declare client authentication in your deployment descriptor:
<login-config> <auth-method>CLIENT-CERT</auth-method> </login-config>
With mutual authentication, the server and the client authenticate each other. Mutual authentication is of two types:
Certificate-based (see Figure 50-1)
User name/password-based (see Figure 50-2)
When using certificate-based mutual authentication, the following actions occur.
A client requests access to a protected resource.
The web server presents its certificate to the client.
The client verifies the server's certificate.
If successful, the client sends its certificate to the server.
The server verifies the client's credentials.
If successful, the server grants access to the protected resource requested by the client.
Figure 50-1 shows what occurs during certificate-based mutual authentication.
Figure 50-1 Certificate-Based Mutual Authentication
In user name/password-based mutual authentication, the following actions occur.
A client requests access to a protected resource.
The web server presents its certificate to the client.
The client verifies the server's certificate.
If successful, the client sends its user name and password to the server.
The server verifies the client's credentials
If the verification is successful, the server grants access to the protected resource requested by the client.
Figure 50-2 shows what occurs during user name/password-based mutual authentication.
Figure 50-2 User Name/Password-Based Mutual Authentication
This section discusses setting up client-side authentication. Enabling both server-side and client-side authentication is called mutual, or two-way, authentication. In client authentication, clients are required to submit certificates issued by a certificate authority that you choose to accept.
There are at least two ways to enable mutual authentication over SSL.
The preferred method is to set the method of authentication in the web.xml
application deployment descriptor to CLIENT-CERT
. This enforces mutual authentication by modifying the deployment descriptor of the given application. In this way, client authentication is enabled only for a specific resource controlled by the security constraint, and the check is performed only when the application requires client authentication.
A less commonly used method is to set the clientAuth
property in the certificate
realm to true
if you want the SSL stack to require a valid certificate chain from the client before accepting a connection. A false
value (which is the default) will not require a certificate chain unless the client requests a resource protected by a security constraint that uses CLIENT-CERT
authentication. When you enable client authentication by setting the clientAuth
property to true
, client authentication will be required for all the requests going through the specified SSL port. If you turn clientAuth
on, it is on all of the time, which can severely degrade performance.
When client authentication is enabled in both of these ways, client authentication will be performed twice.
If you have a certificate signed by a trusted Certificate Authority (CA) such as Verisign, and the GlassFish Server cacerts.jks
file already contains a certificate verified by that CA, you do not need to complete this step. You need to install your certificate in the GlassFish Server certificate file only when your certificate is self-signed.
From the directory where you want to create the client certificate, run keytool
as outlined here. When you press Enter, keytool
prompts you to enter the server name, organizational unit, organization, locality, state, and country code.
You must enter the server name in response to keytool
's first prompt, in which it asks for first and last names. For testing purposes, this can be localhost
. If this example is to verify mutual authentication and you receive a runtime error stating that the HTTPS host name is wrong, re-create the client certificate, being sure to use the same host name you will use when running the example. For example, if your machine name is duke
, then enter duke
as the certificate CN or when prompted for first and last names. When accessing the application, enter a URL that points to the same location (for example, https://duke:8181/mutualauth/hello
). This is necessary because during SSL handshake, the server verifies the client certificate by comparing the certificate name to the host name from which it originates.
To create a keystore named client_keystore.jks
that contains a client certificate named client.cer
, follow these steps.
Create a backup copy of the server truststore file. To do this,
Change to the directory containing the server's keystore and truststore files, domain-dir\config
.
Copy cacerts.jks
to cacerts.backup.jks
.
Copy keystore.jks
to keystore.backup.jks
.
Do not put client certificates in the cacerts.jks
file. Any certificate you add to the cacerts
file effectively can be a trusted root for any and all certificate chains. After you have completed development, delete the development version of the cacerts
file and replace it with the original copy.
Generate the client certificate. Enter the following command from the directory where you want to generate the client certificate:
java-home\bin\keytool -genkey -alias client-alias -keyalg RSA
-keypass changeit -storepass changeit -keystore client_keystore.jks
Export the generated client certificate into the file client.cer
:
java-home\bin\keytool -export -alias client-alias -storepass changeit
-file client.cer -keystore client_keystore.jks
Add the certificate to the truststore file domain-dir/config/cacerts.jks
. Run keytool
from the directory where you created the keystore and client certificate. Use the following parameters:
java-home\bin\keytool -import -v -trustcacerts -alias client-alias -file client.cer -keystore domain-dir/config/cacerts.jks -keypass changeit -storepass changeit
The keytool
utility returns a message like this one:
Owner: CN=localhost, OU=My Company, O=Software, L=Santa Clara, ST=CA, C=US Issuer: CN=localhost, OU=My Company, O=Software, L=Santa Clara, ST=CA, C=US Serial number: 3e39e66a Valid from: Tue Nov 27 12:22:47 EST 2012 until: Mon Feb 25 12:22:47 EST 2013 Certificate fingerprints: MD5: 5A:B0:4C:88:4E:F8:EF:E9:E5:8B:53:BD:D0:AA:8E:5A SHA1:90:00:36:5B:E0:A7:A2:BD:67:DB:EA:37:B9:61:3E:26:B3:89:46:32 Signature algorithm name: SHA1withRSA Version: 3 Trust this certificate? [no]: yes Certificate was added to keystore [Storing cacerts.jks]
Restart GlassFish Server.
An authentication realm, sometimes called a security policy domain or security domain, is a scope over which an application server defines and enforces a common security policy. A realm contains a collection of users, who may or may not be assigned to a group. GlassFish Server comes preconfigured with the file, certificate, and administration realms. An administrator can also set up LDAP, JDBC, digest, or custom realms.
An application can specify in its deployment descriptor which realm to use. If the application does not specify a realm, GlassFish Server uses its default realm, the file
realm. If an application specifies that a JDBC realm is to be used for user authentication, GlassFish Server will retrieve user credentials from a database. The application server uses the database information and the enabled JDBC realm option in the configuration file.
A database provides an easy way to add, edit, or delete users at runtime and enables users to create their own accounts without any administrative assistance. Using a database has an additional benefit: providing a place to securely store any extra user information. A realm can be thought of as a database of user names and passwords that identify valid users of a web application or set of web applications with an enumeration of the list of roles associated with each valid user. Access to specific web application resources is granted to all users in a particular role, instead of enumerating a list of associated users. A user name can have any number of roles associated with it.
Two of the tutorial case studies, Chapter 58, "Duke's Tutoring Case Study Example," and Chapter 59, "Duke's Forest Case Study Example," use a JDBC realm for user authentication.
GlassFish Server enables administrators to specify a user's credentials (user name and password) in the JDBC realm instead of in the connection pool. This prevents other applications from browsing the database tables for user credentials. By default, storing passwords as clear text is not supported in the JDBC realm. Under normal circumstances, passwords should not be stored as clear text.
Create the database tables in which user credentials for the realm will be stored.
Add user credentials to the database tables you created.
Create a JDBC connection pool for the database.
You can use the Administration Console or the command line to create a connection pool.
Create a JDBC resource for the database.
You can use the Administration Console or the command line to create a JDBC resource.
Create a realm.
This step needs to associate the resource with the realm, define the tables and columns for users and groups used for authentication, and define the digest algorithm that will be used for storing passwords in the database.
You can use the Administration Console or the command line to create a realm.
Modify the deployment descriptor for your application to specify the JDBC realm.
For an enterprise application in an EAR file, modify the glassfish-application.xml
file.
For a web application in a WAR file, modify the web.xml
file.
For an enterprise bean in an EJB JAR file, modify the glassfish-ejb-jar.xml
file.
For example, for a hypothetical application, the web.xml
file could specify the jdbcRealm
realm, as follows:
<login-config>
<auth-method>FORM</auth-method>
<realm-name>jdbcRealm</realm-name>
<form-login-config>
<form-login-page>/login.xhtml</form-login-page>
<form-error-page>/login.xhtml</form-error-page>
</form-login-config>
</login-config>
<security-constraint>
<web-resource-collection>
<web-resource-name>Secure Pages</web-resource-name>
<description/>
<url-pattern>/admin/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>ADMINS</role-name>
</auth-constraint>
</security-constraint>
Form-based login is specified for all web pages under /admin
. Access to those pages will be allowed only to users in the ADMINS
role.
Assign security roles to users or groups of users in the realm.
To assign a security role to a group or to a user, add a security-role-mapping element to the application server-specific deployment descriptor, in this case glassfish-web.xml
:
<security-role-mapping> <role-name>USERS</role-name> <group-name>USERS</group-name> </security-role-mapping> <security-role-mapping> <role-name>ADMINS</role-name> <group-name>ADMINS</group-name> </security-role-mapping>
Since GlassFish Server users are assigned to groups during the user creation process, this is more efficient than mapping security roles to individual users.
When a request URI is matched by multiple constrained URL patterns, the constraints that apply to the request are those that are associated with the best matching URL pattern. The servlet matching rules defined in Chapter 12, "Mapping Requests To Servlets," in the Java Servlet 3.1 Specification, are used to determine the best matching URL pattern to the request URI. No protection requirements apply to a request URI that is not matched by a constrained URL pattern. The HTTP method of the request plays no role in selecting the best matching URL pattern for a request.
When HTTP methods are listed within a constraint definition, the protections defined by the constraint are applied to the listed methods only.
When HTTP methods are not listed within a constraint definition, the protections defined by the constraint apply to the complete set of HTTP methods, including HTTP extension methods.
When constraints with different protection requirements apply to the same combination of URL patterns and HTTP methods, the rules for combining the protection requirements are as defined in Section 13.8.1, "Combining Constraints," in the Java Servlet 3.1 Specification.
Follow these guidelines to properly secure a web application.
Do not list HTTP methods within constraint definitions. This is the simplest way to ensure that you are not leaving HTTP methods unprotected. For example:
<!-- SECURITY CONSTRAINT #1 -->
<security-constraint>
<display-name>Do not enumerate Http Methods</display-name>
<web-resource-collection>
<url-pattern>/company/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>sales</role-name>
</auth-constraint>
</security-constraint>
If you list methods in a constraint, all non-listed methods of the effectively infinite set of possible HTTP methods, including extension methods, will be unprotected. Use such a constraint only if you are certain that this is the protection scheme you intend to define. The following example shows a constraint that lists the GET method and thus defines no protection on any of the other possible HTTP methods:
<!-- SECURITY CONSTRAINT #2 -->
<security-constraint>
<display-name>
Protect GET only, leave all other methods unprotected
</display-name>
<web-resource-collection>
<url-pattern>/company/*</url-pattern>
<http-method>GET</http-method>
</web-resource-collection>
<auth-constraint>
<role-name>sales</role-name>
</auth-constraint>
</security-constraint>
If you need to apply specific types of protection to specific HTTP methods, make sure that you define constraints to cover every method that you want to permit, with or without constraint, at the corresponding URL patterns. If there are any methods that you do not want to permit, you must also create a constraint that denies access to those methods at the same patterns; for an example, see security constraint #5 in the next bullet.
For example, to permit GET and POST, where POST requires authentication and GET is permitted without constraint, you could define the following constraints:
<!-- SECURITY CONSTRAINT #3 --> <security-constraint> <display-name>Allow unprotected GET</display-name> <web-resource-collection> <url-pattern>/company/*</url-pattern> <http-method>GET</http-method> </web-resource-collection> </security-constraint> <!-- SECURITY CONSTRAINT #4 --> <security-constraint> <display-name>Require authentication for POST</display-name> <web-resource-collection> <url-pattern>/company/*</url-pattern> <http-method>POST</http-method> </web-resource-collection> <auth-constraint> <role-name>sales</role-name> </auth-constraint> </security-constraint>
The simplest way to ensure that you deny all HTTP methods except those that you want to be permitted is to use http-method-omission
elements to omit those HTTP methods from the security constraint, and also to define an auth-constraint
that names no roles. The security constraint will apply to all methods except those that were named in the omissions, and the constraint will apply only to the resources matched by the patterns in the constraint.
For example, the following constraint excludes access to all methods except GET and POST at the resources matched by the pattern /company/*
:
<!-- SECURITY CONSTRAINT #5 -->
<security-constraint>
<display-name>Deny all HTTP methods except GET and POST</display-name>
<web-resource-collection>
<url-pattern>/company/*</url-pattern>
<http-method-omission>GET</http-method-omission>
<http-method-omission>POST</http-method-omission>
</web-resource-collection>
<auth-constraint/>
</security-constraint>
If you want to extend these exclusions to the unconstrained parts of your application, also include the URL pattern /
(forward slash):
<!-- SECURITY CONSTRAINT #6 --> <security-constraint> <display-name>Deny all HTTP methods except GET and POST</display-name> <web-resource-collection> <url-pattern>/company/*</url-pattern> <url-pattern>/</url-pattern> <http-method-omission>GET</http-method-omission> <http-method-omission>POST</http-method-omission> </web-resource-collection> <auth-constraint/> </security-constraint>
If, for your web application, you do not want any resource to be accessible unless you explicitly define a constraint that permits access to it, you can define an auth-constraint
that names no roles and associate it with the URL pattern /
. The URL pattern /
is the weakest matching pattern. Do not list any HTTP methods in this constraint:
<!-- SECURITY CONSTRAINT #7 -->
<security-constraint>
<display-name>
Switch from Constraint to Permission model
(where everything is denied by default)
</display-name>
<web-resource-collection>
<url-pattern>/</url-pattern>
</web-resource-collection>
<auth-constraint/>
</security-constraint>
The Java EE authentication requirements for application clients are the same as for other Java EE components, and the same authentication techniques can be used as for other Java EE application components. No authentication is necessary when accessing unprotected web resources.
When accessing protected web resources, the usual varieties of authentication can be used: HTTP basic authentication, HTTP login-form authentication, or SSL client authentication. Specifying an Authentication Mechanism in the Deployment Descriptor describes how to specify HTTP basic authentication and HTTP login-form authentication. Client Authentication describes how to specify SSL client authentication.
Authentication is required when accessing protected enterprise beans. The authentication mechanisms for enterprise beans are discussed in Securing Enterprise Beans.
An application client makes use of an authentication service provided by the application client container for authenticating its users. The container's service can be integrated with the native platform's authentication system so that a single sign-on capability is used. The container can authenticate the user either when the application is started or when a protected resource is accessed.
An application client can provide a class, called a login module, to gather authentication data. If so, the javax.security.auth.callback.CallbackHandler
interface must be implemented, and the class name must be specified in its deployment descriptor. The application's callback handler must fully support Callback
objects specified in the javax.security.auth.callback
package.
An application client can use the Java Authentication and Authorization Service (JAAS) to create login modules for authentication. A JAAS-based application implements the javax.security.auth.callback.CallbackHandler
interface so that it can interact with users to enter specific authentication data, such as user names or passwords, or to display error and warning messages.
Applications implement the CallbackHandler
interface and pass it to the login context, which forwards it directly to the underlying login modules. A login module uses the callback handler both to gather input, such as a password or smart card PIN, from users and to supply information, such as status information, to users. Because the application specifies the callback handler, an underlying login module can remain independent of the various ways applications interact with users.
For example, the implementation of a callback handler for a GUI application might display a window to solicit user input, or the implementation of a callback handler for a command-line tool might simply prompt the user for input directly from the command line.
The login module passes an array of appropriate callbacks to the callback handler's handle
method, such as a NameCallback
for the user name and a PasswordCallback
for the password; the callback handler performs the requested user interaction and sets appropriate values in the callbacks. For example, to process a NameCallback
, the CallbackHandler
might prompt for a name, retrieve the value from the user, and call the setName
method of the NameCallback
to store the name.
For more information on using JAAS for authentication in login modules, refer to the documentation listed in Further Information about Advanced Security Topics.
In Enterprise Information Systems (EIS) applications, components request a connection to an EIS resource. As part of this connection, the EIS can require a sign-on for the requester to access the resource. The application component provider has two choices for the design of the EIS sign-on.
Container-managed sign-on: The application component lets the container take the responsibility of configuring and managing the EIS sign-on. The container determines the user name and password for establishing a connection to an EIS instance. For more information, see Container-Managed Sign-On.
Component-managed sign-on: The application component code manages EIS sign-on by including code that performs the sign-on process to an EIS. For more information, see Component-Managed Sign-On.
You can also configure security for resource adapters. See Configuring Resource Adapter Security.
In container-managed sign-on, an application component does not have to pass any sign-on security information to the getConnection()
method. The security information is supplied by the container, as shown in the following example (the method call is highlighted in bold):
// Business method in an application component
Context initctx = new InitialContext();
// Perform JNDI lookup to obtain a connection factory
javax.resource.cci.ConnectionFactory cxf =
(javax.resource.cci.ConnectionFactory)initctx.lookup(
"java:comp/env/eis/MainframeCxFactory");
// Invoke factory to obtain a connection. The security
// information is not passed in the getConnection method
javax.resource.cci.Connection cx = cxf.getConnection();
...
In component-managed sign-on, an application component is responsible for passing the needed sign-on security information for the resource to the getConnection
method. For example, security information might be a user name and password, as shown here (the method call is highlighted in bold):
// Method in an application component
Context initctx = new InitialContext();
// Perform JNDI lookup to obtain a connection factory
javax.resource.cci.ConnectionFactory cxf =
(javax.resource.cci.ConnectionFactory)initctx.lookup(
"java:comp/env/eis/MainframeCxFactory");
// Get a new ConnectionSpec
com.myeis.ConnectionSpecImpl properties = //..
// Invoke factory to obtain a connection
properties.setUserName("...");
properties.setPassword("...");
javax.resource.cci.Connection cx =
cxf.getConnection(properties);
...
A resource adapter is a system-level software component that typically implements network connectivity to an external resource manager. A resource adapter can extend the functionality of the Java EE platform either by implementing one of the Java EE standard service APIs, such as a JDBC driver, or by defining and implementing a resource adapter for a connector to an external application system. Resource adapters can also provide services that are entirely local, perhaps interacting with native resources. Resource adapters interface with the Java EE platform through the Java EE service provider interfaces (Java EE SPI). A resource adapter that uses the Java EE SPIs to attach to the Java EE platform will be able to work with all Java EE products.
To configure the security settings for a resource adapter, you need to edit the resource adapter descriptor file, ra.xml
. Here is an example of the part of an ra.xml
file that configures security properties for a resource adapter:
<authentication-mechanism> <authentication-mechanism-type> BasicPassword </authentication-mechanism-type> <credential-interface> javax.resource.spi.security.PasswordCredential </credential-interface> </authentication-mechanism> <reauthentication-support>false</reauthentication-support>
You can find out more about the options for configuring resource adapter security by reviewing as-install/lib/schemas/connector_1_7.xsd
. You can configure the following elements in the resource adapter deployment descriptor file.
Authentication mechanisms: Use the authentication-mechanism
element to specify an authentication mechanism supported by the resource adapter. This support is for the resource adapter, not for the underlying EIS instance.
There are two supported mechanism types:
BasicPassword
, which supports the following interface:
javax.resource.spi.security.PasswordCredential
Kerbv5
, which supports the following interface:
javax.resource.spi.security.GenericCredential
GlassFish Server does not currently support this mechanism type.
Reauthentication support: Use the reauthentication-support
element to specify whether the resource adapter implementation supports reauthentication of existing Managed-Connection
instances. Options are true
or false
.
Security permissions: Use the security-permission
element to specify a security permission that is required by the resource adapter code. Support for security permissions is optional and is not supported in the current release of GlassFish Server. You can, however, manually update the server.policy
file to add the relevant permissions for the resource adapter.
The security permissions listed in the deployment descriptor are different from those required by the default permission set as specified in the connector specification.
For more information on the implementation of the security permission specification, see the security policy file documentation listed in Further Information about Advanced Security Topics.
In addition to specifying resource adapter security in the ra.xml
file, you can create a security map for a connector connection pool to map an application principal or a user group to a back-end EIS principal. The security map is usually used if one or more EIS back-end principals are used to execute operations (on the EIS) initiated by various principals or user groups in the application.
When using GlassFish Server, you can use security maps to map the caller identity of the application (principal or user group) to a suitable EIS principal in container-managed transaction-based scenarios. When an application principal initiates a request to an EIS, GlassFish Server first checks for an exact principal by using the security map defined for the connector connection pool to determine the mapped back-end EIS principal. If there is no exact match, GlassFish Server uses the wildcard character specification, if any, to determine the mapped back-end EIS principal. Security maps are used when an application user needs to execute an EIS operation that requires execution as a specific identity in the EIS.
To work with security maps, use the Administration Console. From the Administration Console, follow these steps to get to the security maps page.
In the navigation tree, expand the Resources node.
Expand the Connectors node.
Select the Connector Connection Pools node.
On the Connector Connection Pools page, click the name of the connection pool for which you want to create a security map.
Click the Security Maps tab.
Click New to create a new security map for the connection pool.
Enter a name by which you will refer to the security map, as well as the other required information.
Click Help for more information on the individual options.
The recommended way to configure security in the Java EE 7 platform is with annotations. If you wish to override the security settings at deployment time, you can use security elements in the web.xml
deployment descriptor to do so. This section describes how to use the deployment descriptor to specify basic authentication and to override default principal-to-role mapping.
The elements of the deployment descriptor that add basic authentication to an example tell the server or browser to perform the following tasks.
Send a standard login dialog box to collect user name and password data.
Verify that the user is authorized to access the application.
If authorized, display the servlet to the user.
The following sample code shows the security elements for a deployment descriptor that could be used in the example of basic authentication found in the tut-install/examples/security/hello2_basicauth/
directory:
<security-constraint> <display-name>SecurityConstraint</display-name> <web-resource-collection> <web-resource-name>WRCollection</web-resource-name> <url-pattern>/greeting</url-pattern> </web-resource-collection> <auth-constraint> <role-name>TutorialUser</role-name> </auth-constraint> <user-data-constraint> <transport-guarantee>CONFIDENTIAL</transport-guarantee> </user-data-constraint> </security-constraint> <login-config> <auth-method>BASIC</auth-method> <realm-name>file</realm-name> </login-config> <security-role> <role-name>TutorialUser</role-name> </security-role>
This deployment descriptor specifies that the request URI /greeting
can be accessed only by users who have entered their user names and passwords and have been authorized to access this URL because they have been verified to be in the role TutorialUser
. The user name and password data will be sent over a protected transport in order to keep it from being read in transit.
To map a role name permitted by the application or module to principals (users) and groups defined on the server, use the security-role-mapping
element in the runtime deployment descriptor file (glassfish-application.xml
, glassfish-web.xml
, or glassfish-ejb-jar.xml
). The entry needs to declare a mapping between a security role used in the application and one or more groups or principals defined for the applicable realm of GlassFish Server. An example for the glassfish-web.xml
file is shown below:
<glassfish-web-app> <security-role-mapping> <role-name>DIRECTOR</role-name> <principal-name>schwartz</principal-name> </security-role-mapping> <security-role-mapping> <role-name>DEPT-ADMIN</role-name> <group-name>dept-admins</group-name> </security-role-mapping> </glassfish-web-app>
The role name can be mapped to either a specific principal (user), a group, or both. The principal or group names referenced must be valid principals or groups in the current default realm of GlassFish Server. The role-name
in this example must exactly match the role-name
in the security-role
element of the corresponding web.xml
file or the role name defined in the @DeclareRoles
and/or @RolesAllowed
annotations.
For more information about the security topics covered in this chapter, see
Documentation on the keytool
command:
http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/keytool.html
Java Authentication and Authorization Service (JAAS) Reference Guide:
http://docs.oracle.com/javase/7/docs/technotes/guides/security/jaas/JAASRefGuide.html
Java Authentication and Authorization Service (JAAS): LoginModule Developer's Guide:
http://docs.oracle.com/javase/7/docs/technotes/guides/security/jaas/JAASLMDevGuide.html
Documentation on security policy file syntax:
http://docs.oracle.com/javase/7/docs/technotes/guides/security/PolicyFiles.html#FileSyntax


Date: 18th Dec 2020 Day: Friday It was a nice day . Today, I worked till 12 noon in the new house .Then, I played from around 2pm to 4pm. Date: 18th Dec 2020 Day: Friday It was a nice day . Today, I worked till 12 noon in the new house .Then, I played from around 2pm to 4pm. Date: 18th Dec 2020 Day: Friday It was a nice day . Today, I worked till 12 noon in the new house .Then, I played from around 2pm to 4pm.


--- layout: post title: "Integrating Baloo File Search with Emacs" permalink: /:title/ tags: [emacs, baloo, kde] ---
I use KDE on the desktop. Although most of my userland operations are done in Emacs and Firefox, I tend to appreciate the choice when handling tasks more complex than text-based processes.  Plasma is stuffed to the brim with features and hooks, and I've exploited them before when writing my first Emacs package to ease certain operations between Emacs and my mobile devices. But one feature I've always missed has been desktop search. KDE offers the excellent Baloo, a file indexer that handles both metadata and content, and its integration with Krunner is the main reason I use the latter. As a matter of fact, despite Krunner's many features, like arithmetic and dictionary lookups, I only ever use it for finding files and launching applications. I can improve this workflow.
When opening a file, Krunner launches a new instance of the filetype's default program. This is sensible. If I open a PDF, I probably want Okular, and if I need to check an old web page, Firefox is the appropriate tool. But if I'm working in Emacs, there's tremendous value to staying in Emacs. Further, instead of opening a new instance of Emacs for text files, a within-Emacs solution lets me simply visit the file in a new buffer.1 As the piece de resistance, I get to use Counsel for completion instead of Krunner.
As I discovered when writing kdeconnect.el, Emacs and Emacs Lisp make for an interesting programming experience.
Unlike Java's extensive libraries and Python's convenience, writing Emacs Lisp makes me yearn for quality of life functions.
Fortunately, the code to use Baloo for file searches form Emacs is basic enough that Emacs does abstract away much of the nitty gritty.
Some of these functions I already knew (completing-read
to let Emacs handle completion, split-string
to take shell output and make a list, mapconcat
and shell-quote-argument
to generate safe shell commands), but others were novel to me and a massive relief to discover in the docs, like butlast
to remove the last line of output, which contains the search's time to execute, or safe-length-query
to check if a list is empty.
Here's the resulting code:
(defun baloo-search (query) "Search for files matching QUERY using Baloo." (interactive "MSearch: ") (setq query (butlast (split-string (shell-command-to-string (mapconcat 'identity (list "baloosearch"(shell-quote-argument query)) " ")) "\n" t) 1)) (if (eq (safe-length query) 0) (error "No matches")) (find-file (completing-read "Select file: " query nil t "")))
Pop that in your init.el
or evaluate it somewhere and you're good to go!
Call M-x baloo-search
, enter your search term(s), which can include an extension, and choose your file.
I've found this to be incredibly useful when I don't know quite where a file is or what it's called, and perhaps you will too.
Update 2019-05-31 Per request, should anyone be interested in modifying or extending this program, I hereby license it under the GPLv3.
This is actually possible if you use emacsclient but I don't use that functionality.


just my musings...
The one with the DBC culture
9/19/2014 :: Phase 0 Week 1
:: My Thoughts ::
Jumping right in to my second post! We were asked to view a video of Shereef B., the founder of Dev Bootcamp, speaking about DBC.
My take on the DBC experience is that it's for individuals from all walks of life with one motive. A passion to become web developers. It's an all emcompassing community that pushes everyone in every aspect of their lives and it's definitely not for those less than 100% committed to their future.
My impression has been the same and my expectations haven't changed; DBC is going to push me to my limits, both mentally and physically. Mentally, because of the hardcore coding and the unique social environment that DBC pushes you to be a part of. And physically, because there WILL be long hours on campus and I WILL be commuting almost 4 hours every day. It's going to be a love-hate relationship, but at the end, the greatest reward I'll receive isn't just the confidence that I can code and I made it through alive, but the security to know that I am not alone.
Do I see myself surviving this culture? Yeah.  I'm pretty private as a person and being active online just makes me feel like I'm putting my life on display for people to come and judge. But I can see it now.. DBC has done it's best to make this community NOT about judging, but about the willingness to help. Not about competition, but about collaboration. So I'm going to engage in this culture just like I would test the waters before jumping into a pool. I'm going to be scared. Heck, it's making me a little nervous even now just thinking about it and admitting it here online. But what.ever. FULL DISCLOSURE from here on out. I can't expect to get 100% out of this program if I don't put in my 100% to begin with. All in all, I'm scared and excited. It's like that scene on Friends:
Ross: ... I had no idea you were so excited about Paris. Uhm, I mean, you said you were scared. Rachel: Well yeah, but I mean, it was good scared though, you know? Like when I-moved-to-New-York scared. Or uhm, when I-found-out-I-was-gonna-have-Emma scared... [many thanks to: www.livesinabox.com]
So yeah, I'm "good scared." And I know this is what I want.
main home facebook twitter linkedin


Personal Resume
Hi! For those of you who don't know me, my name is Ashwin and for those who do know me, (dramatic pause) my name is still Ashwin.
2018-Current
2018, CGPA - 9.2


Home
Photo Gallery
Getting Ready
Bride Pickup
Portraits
The Ceremony
The Reception
Contact
×
get_app


--- layout: default --- {% include reviews.html %} {% include talklist.html %} {% include teaching.html %} {% include talkvideos.html %}


About Hactoberfest
Hacktoberfest is a month-long,
community-wide celebration of open source software. It was started
in 2013 and in this first year, nearly 700 contributors
participated. Over the years it has grown to a community of nearly
150,000 participants in 135 countries, who participate by sharing
their projects, making pull requests, attending events, donating and
showing their support of open source. It is run by DigitalOcean
alongside partners like GitHub and Twilio.
To participate, one must make four valid pull requests submitted to
public GitHub repositories. On top of a boost to your open-source
cred, participants are rewarded with prizes, like stickers,
T-shirts, or a new plant-a-tree alternative.


[1]
special mention to the final bill - which took some sorting out would really make this place top notch.
[2]
accommodation was happy to the final bill - which took some sorting out would really make this place top notch.


THE EX (1997) A "Turkey of the Week" film review by Justin Felix. Copyright 1998 Justin Felix.
Rating: ** (out of five)
Based on a novel by John Lutz. Screenplay by Larry Cohen and John Lutz. Directed by Mark L. Lester. Starring Yancy Butler, Suzy Amis, Nick Mancuso. Rated R (contains violence, nudity, and profanity) 87 mins.
Synopsis: Easily-angered, chainsmoking architect David encounters his homicidal first wife Diedre five years after their divorce. Diedre easily cons David's not-so-bright new wife Molly into believing she's a child psychologist so that she can influence Molly and David's quick-tempered son, Michael.
Comments: THE EX is a very bad movie. I haven't seen a turkey of this magnitude in quite a while. I kept vacillating, however, between giving THE EX one star for its sheer awfulness and three stars for its campy humor (ultimately, I decided to split the difference and give it two stars).
As summarized briefly in the synopsis, this is yet another spurned-psycho-lover-gets-her-revenge type movie. I'm not sure how many films have been produced since FATAL ATTRACTION which use this tired storyline, but it seems like 10,000,000,000. THE EX, however, though following many of the standard cliches of this thriller subgenre, differs slightly from the norm in that, at least, it doesn't take itself too seriously like so many other duds do (MALICIOUS and STALKED are two examples which come to mind). Those involved in the movie realize they're in a turkey, apparently, and turn in hammy performances which compliment the ludicrous dialogue contained within the script.
THE EX's storyline, as I said, incorporates many cliches predominant in films of its nature; however, the plot is so overwrought and unbelievable that, after a while, the viewer must accept that logic does not operate in the realm of the movie. Diedre, David's first wife, kills and kills and kills, for example, without ever having to worry about police investigations and the like (it's not as though she's bright enough not to leave fingerprints at the scene of the crime). David and his new wife Molly have an incredibly difficult time understanding why Michael, their son, has emotional problems, specifically his inability to control his anger. Though David smokes like tomorrow will never come and blows up at people every other minute in the movie, the connection between him and his son is not made. Molly is easily swayed by Diedre into believing she's a child psychologist and allows the demented woman to spend time with her son. Time and time again, things do not add up in this movie. If a viewer can't accept this fact, then he will definately not like THE EX. If, on the other hand, he can appreciate the movie for its campiness, then he'll probably like it much better.
I don't mean to necessarily suggest, however, that THE EX is a memorable exercise in camp. Even when viewed from an it's-so-bad-it's-good angle, this movie isn't all that successful. THE EX does have its moments, though, almost all of which involve Yancy Butler (the psychotic Diedre) and Nick Mancuso (David). These two actors turn in decidedly hammy performances which, oftentimes, elicit chuckles from the audience. The truly awful lines written for them help this humor along. Butler gets the larger portion of bad dialogue; every time her character kills someone, she has a punchline that's unbelievably inane. When she murders a tenant of an apartment she wishes to use to spy on David, for example, Diedre whacks the elderly lady with a crowbar and states "I'm so sorry, your lease is terminated." Or how about when Diedre takes out her therapist in another ridiculous scene? (Diedre's therapist wishes to have her recommitted, so she decides to visit Diedre, alone, in her new apartment -- never realizing, of course, that it's not her's. Therapists typically are catatonically brain-dead in these movies). Having successfully snuffed the woman out, Diedre looks at the body and says: "What was it that you always said to me at the end of each session, Dr. Jones? Oh, that's right. 'I'm sorry, but your time is up!'" This is what passes for humor in the film.  Nick Mancuso's chainsmoking, nervous-wreck character does not have the cheesy lines that Butler's character delivers, but his ridiculously overdone performance provides for some better comic relief. In between drags off his cigarette, for instance, in one of the movie's best scenes, David tries to explain to his lawyer what a creep his ex-wife is. In exaggerated exasperation, he nearly shouts "the woman is a cuckoo-bird!" (Trust me, it is funny in context.)
And so, the movie continues until its painfully obvious conclusion. I won't reveal the ending specifically (though anyone remotely familiar with thrillers of this type could probably guess the ending just from this film review), but I will say that THE EX has one of the biggest fire hazards appearing recently in film. Molly and David own a cabin that becomes engulfed by flames in approximately three seconds! Molly, at one point in the film, not-so-brilliantly observes, after discovering Diedre has moved into an apartment across from her family's to spy upon them, that "she's crazy." You'd be pretty crazy too if you'd rented this turkey. I'd avoid it unless it's on TV and you come across it.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


??=%l^
DOMINICAN
REPUBLIC
prov.
San
Juan
IbOHMOO
V
Oplismenus
hirtellus
(L.)P.Beauv.
subsp.
setarius
(Lam.)Mez
ex
Ekman
Oplismenus
setarus
(Lam.)
R.&S,
det.
Swallen
det.
confirm.
U.ScholZ
Apr.78
MUS.
BOT.
BEROL.
Terrestrial
in
deep
woods.
Plants
of
pine
woodlands.
Alt.
8500'
El-CeČeade,
Juan
Santiago
Coll.
R.
A.
&
E.
S.
Howard
9290
Oct.
3,
1946
1604400
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


|
|
TAMBIS |
|
|
|
TAMBIS was a joint research project between the
School of Biological Sciences
and the Information Management Group,
part of Computer Science
in the University
of Manchester in the UK. the project started in 1996 and has run out of
funding. These pages and the TAMBIS software are no longer supported. You will not be able to use the software.
TAMBIS aims to aid researchers in biological science by providing a single access point for biological information sources round the world. The access point will be a single interface (via the World Wide Web) which acts as a single information source. It will find appropriate sources of information for user queries and phrase the user questions for each source, returning the results in a consistent manner which will include details of the information source. Further details about TAMBIS can be found at this site. In addition, two Powerpoint presentations can be downloaded which describe in more detail why TAMBIS is required, and how TAMBIS will work. TAMBIS has been on-line for evaluation since Jan 1999. Those interested in TAMBIS should contact the TAMBIS team. A demonstrational Java applet and video examples are no longer publically accessible. TAMBIS is funded by the BBSRC/EPSRC Bioinformatics programme and Zeneca Pharmaceuticals. A version of this web site which does not use tables and images for navigation is also available. |
|
| email:
<EMAIL_ADDRESS>© University of Manchester, 1998. |


Planet
M
Continue as Guest
PlanetM
The world of Music on A Click
Login
Register
Login
Forgot Password?
Register


Cochin Shipyard hits 52-week high after bagging Rs 3,000-crore contract
The company has signed the Phase-III contract for the underconstruction indigenous aircraft carrier being built in Kochi for the Indian Navy .
by Moneycontrol NewsShares of Cochin Shipyard touched 52-week high of Rs 409.95, rising more than 5 percent intraday on November 1, after the company signed a contract worth Rs 3,000 crore.
The company had signed the Phase-III contract for construction of the indigenous aircraft carrier on October 31, a press release said.
The Phase-III contract covers the operational and harbour acceptance trials of various equipment and systems installed onboard and also the sea trials of the carrier, which is to be handed over to the Indian Navy in 2021. It will be known as INS Vikrant once commissioned.
The contract also covers post-delivery activates, including support during weapon and aviation trials.
At 0955 hours, Cochin Shipyard was quoting at Rs 406.75, up Rs 17.80, or 4.58 percent, on the BSE.Special Thursday Expiry on 10th 7th Nov
Early Closure to The Great Diwali Discount. Last Chance to get Moneycontrol PRO for a year @ Rs. 289/- only
Coupon code: DIWALI


The KendoReact Editor allows your users to edit HTML in a familiar, user-friendly way.
The Editor provides the core HTML editing engine, which includes text formatting, hyperlinks, and lists.
The component outputs identical HTML across all major browsers, follows accessibility standards,
and provides API for content manipulation.


Jack is not loaded right now. If you see this text for a long time, it means Jack is not working. Please recreate project or report it as a bug.


VSL // The Birds And The Bees
* Isabella Rossellini conceived, wrote, directed, and starred in Green Porno
today’s Very Short List:
culture 7 May 2014
Carly Ayres is the Chief Content Officer of CreativeMornings
Subscribe
* Wild Sex
* John Green of Mental Floss explains 26 strange animal mating habits in this short video
http://link.nyobserver.com/516ca68465217dc6e1b9a3a61kf3f.bg2/TvnrKi0WTdwTHwxwCa481
delightful e-mail that shares
cultural gems from a
different curator every day.
If you have any problems reading this e-mail, click here
http://link.nyobserver.com/516ca68465217dc6e1b9a3a61kf3f.bg2/UHyHCJsf7NZSKXrPC2424
Subscribe
Forward
My Account
Unsubscribe
tech 5 May 2014
Netta Samroengraja on culture and company in a start-up world.
food 2 May 2014
Alexa Jaccarino on some high-tech kitchen things.
culture 30 Apr 2014
Nahema Mehta connects some dots.
Editorial Policy
Privacy Policy
Contact Us
Press
Terms + Conditions
very fine print: ©2014
Observer VSL LLC.
321 West 44th Street, 6th Floor
New York, NY 10036 USA
All rights reserved.
Questions?<PHONE_NUMBER>.
This message was intended for<EMAIL_ADDRESS>If you believe this has been sent to you in error, please safely unsubscribe


Home
Sleeping
Eating
Vitals
Logout
Welcome
Vitals History
Keep track of your baby's height and weight!
Last 10 Vitals Entries
Date
Height
Weight
Vitals Graph


MAP --- OziExplorer .MAP
(GDAL >= 1.10)
OziExplorer MAP files.
This driver does not support file creation.
See Also:
Implemented as
gdal/frmts/map/mapdataset.cpp
.
OziExplorer Map File Format


For each key k in this
or other1
or other2
, return a resulting RDD that contains a
tuple with the list of values for that key in this
, other1
and other2
.
For each key k in this
or other
, return a resulting RDD that contains a tuple with the
list of values for that key in this
as well as other
.
For each key k in this
or other1
or other2
, return a resulting RDD that contains a
tuple with the list of values for that key in this
, other1
and other2
.
For each key k in this
or other
, return a resulting RDD that contains a tuple with the
list of values for that key in this
as well as other
.
For each key k in this
or other1
or other2
, return a resulting RDD that contains a
tuple with the list of values for that key in this
, other1
and other2
.
For each key k in this
or other
, return a resulting RDD that contains a tuple with the
list of values for that key in this
as well as other
.
Return the key-value pairs in this RDD to the master as a Map.
Simplified version of combineByKey that hash-partitions the resulting RDD using the existing partitioner/parallelism level.
Simplified version of combineByKey that hash-partitions the output RDD.
Generic function to combine the elements for each key using a custom set of aggregation functions.
Generic function to combine the elements for each key using a custom set of aggregation functions. Turns an RDD[(K, V)] into a result of type RDD[(K, C)], for a "combined type" C Note that V and C can be different -- for example, one might group an RDD of type (Int, Int) into an RDD of type (Int, Seq[Int]). Users provide three functions:
- createCombiner
, which turns a V into a C (e.g., creates a one-element list)
- mergeValue
, to merge a V into a C (e.g., adds it to the end of a list)
- mergeCombiners
, to combine two C's into a single one.
In addition, users can control the partitioning of the output RDD, and whether to perform map-side aggregation (if a mapper can produce multiple items with the same key).
Count the number of elements for each key, and return the result to the master as a Map.
(Experimental) Approximate version of countByKey that can return a partial result if it does not finish within a timeout.
Pass each value in the key-value pair RDD through a flatMap function without changing the keys; this also retains the original RDD's partitioning.
Merge the values for each key using an associative function and a neutral "zero value" which may be added to the result an arbitrary number of times, and must not change the result (e.
Merge the values for each key using an associative function and a neutral "zero value" which may be added to the result an arbitrary number of times, and must not change the result (e.g., Nil for list concatenation, 0 for addition, or 1 for multiplication.).
Merge the values for each key using an associative function and a neutral "zero value" which may be added to the result an arbitrary number of times, and must not change the result (e.
Merge the values for each key using an associative function and a neutral "zero value" which may be added to the result an arbitrary number of times, and must not change the result (e.g., Nil for list concatenation, 0 for addition, or 1 for multiplication.).
Merge the values for each key using an associative function and a neutral "zero value" which may be added to the result an arbitrary number of times, and must not change the result (e.
Merge the values for each key using an associative function and a neutral "zero value" which may be added to the result an arbitrary number of times, and must not change the result (e.g., Nil for list concatenation, 0 for addition, or 1 for multiplication.).
Group the values for each key in the RDD into a single sequence.
Group the values for each key in the RDD into a single sequence. Hash-partitions the resulting RDD with the existing partitioner/parallelism level.
Group the values for each key in the RDD into a single sequence.
Group the values for each key in the RDD into a single sequence. Hash-partitions the
resulting RDD with into numPartitions
partitions.
Group the values for each key in the RDD into a single sequence.
Group the values for each key in the RDD into a single sequence. Allows controlling the partitioning of the resulting key-value pair RDD by passing a Partitioner.
Alias for cogroup.
Alias for cogroup.
Return an RDD containing all pairs of elements with matching keys in this
and other
.
Return an RDD containing all pairs of elements with matching keys in this
and other
. Each
pair of elements will be returned as a (k, (v1, v2)) tuple, where (k, v1) is in this
and
(k, v2) is in other
. Performs a hash join across the cluster.
Return an RDD containing all pairs of elements with matching keys in this
and other
.
Return an RDD containing all pairs of elements with matching keys in this
and other
. Each
pair of elements will be returned as a (k, (v1, v2)) tuple, where (k, v1) is in this
and
(k, v2) is in other
. Performs a hash join across the cluster.
Return an RDD containing all pairs of elements with matching keys in this
and other
.
Return an RDD containing all pairs of elements with matching keys in this
and other
. Each
pair of elements will be returned as a (k, (v1, v2)) tuple, where (k, v1) is in this
and
(k, v2) is in other
. Uses the given Partitioner to partition the output RDD.
Return an RDD with the keys of each tuple.
Perform a left outer join of this
and other
.
Perform a left outer join of this
and other
. For each element (k, v) in this
, the
resulting RDD will either contain all pairs (k, (v, Some(w))) for w in other
, or the
pair (k, (v, None)) if no elements in other
have key k. Hash-partitions the output
into numPartitions
partitions.
Perform a left outer join of this
and other
.
Perform a left outer join of this
and other
. For each element (k, v) in this
, the
resulting RDD will either contain all pairs (k, (v, Some(w))) for w in other
, or the
pair (k, (v, None)) if no elements in other
have key k. Hash-partitions the output
using the existing partitioner/parallelism level.
Perform a left outer join of this
and other
.
Perform a left outer join of this
and other
. For each element (k, v) in this
, the
resulting RDD will either contain all pairs (k, (v, Some(w))) for w in other
, or the
pair (k, (v, None)) if no elements in other
have key k. Uses the given Partitioner to
partition the output RDD.
Return the list of values in the RDD for key key
.
Return the list of values in the RDD for key key
. This operation is done efficiently if the
RDD has a known partitioner by only searching the partition that the key maps to.
Pass each value in the key-value pair RDD through a map function without changing the keys; this also retains the original RDD's partitioning.
Return a copy of the RDD partitioned using the specified partitioner.
Merge the values for each key using an associative reduce function.
Merge the values for each key using an associative reduce function. This will also perform the merging locally on each mapper before sending results to a reducer, similarly to a "combiner" in MapReduce. Output will be hash-partitioned with the existing partitioner/ parallelism level.
Merge the values for each key using an associative reduce function.
Merge the values for each key using an associative reduce function. This will also perform the merging locally on each mapper before sending results to a reducer, similarly to a "combiner" in MapReduce. Output will be hash-partitioned with numPartitions partitions.
Merge the values for each key using an associative reduce function.
Merge the values for each key using an associative reduce function. This will also perform the merging locally on each mapper before sending results to a reducer, similarly to a "combiner" in MapReduce.
Merge the values for each key using an associative reduce function, but return the results immediately to the master as a Map.
Merge the values for each key using an associative reduce function, but return the results immediately to the master as a Map. This will also perform the merging locally on each mapper before sending results to a reducer, similarly to a "combiner" in MapReduce.
Alias for reduceByKeyLocally
Perform a right outer join of this
and other
.
Perform a right outer join of this
and other
. For each element (k, w) in other
, the
resulting RDD will either contain all pairs (k, (Some(v), w)) for v in this
, or the
pair (k, (None, w)) if no elements in this
have key k. Hash-partitions the resulting
RDD into the given number of partitions.
Perform a right outer join of this
and other
.
Perform a right outer join of this
and other
. For each element (k, w) in other
, the
resulting RDD will either contain all pairs (k, (Some(v), w)) for v in this
, or the
pair (k, (None, w)) if no elements in this
have key k. Hash-partitions the resulting
RDD using the existing partitioner/parallelism level.
Perform a right outer join of this
and other
.
Perform a right outer join of this
and other
. For each element (k, w) in other
, the
resulting RDD will either contain all pairs (k, (Some(v), w)) for v in this
, or the
pair (k, (None, w)) if no elements in this
have key k. Uses the given Partitioner to
partition the output RDD.
Output the RDD to any Hadoop-supported storage system, using a Hadoop JobConf object for that storage system.
Output the RDD to any Hadoop-supported storage system, using a Hadoop JobConf object for that storage system. The JobConf should set an OutputFormat and any output paths required (e.g. a table name to write to) in the same way as it would be configured for a Hadoop MapReduce job.
Output the RDD to any Hadoop-supported file system, using a Hadoop OutputFormat
class
supporting the key and value types K and V in this RDD.
Output the RDD to any Hadoop-supported file system, using a Hadoop OutputFormat
class
supporting the key and value types K and V in this RDD.
Output the RDD to any Hadoop-supported file system, using a Hadoop OutputFormat
class
supporting the key and value types K and V in this RDD. Compress with the supplied codec.
Output the RDD to any Hadoop-supported file system, using a Hadoop OutputFormat
class
supporting the key and value types K and V in this RDD.
Output the RDD to any Hadoop-supported file system, using a Hadoop OutputFormat
class
supporting the key and value types K and V in this RDD. Compress the result with the
supplied codec.
Output the RDD to any Hadoop-supported file system, using a Hadoop OutputFormat
class
supporting the key and value types K and V in this RDD.
Output the RDD to any Hadoop-supported file system, using a new Hadoop API OutputFormat
(mapreduce.
Output the RDD to any Hadoop-supported file system, using a new Hadoop API OutputFormat
(mapreduce.OutputFormat) object supporting the key and value types K and V in this RDD.
Output the RDD to any Hadoop-supported file system, using a new Hadoop API OutputFormat
(mapreduce.
Output the RDD to any Hadoop-supported file system, using a new Hadoop API OutputFormat
(mapreduce.OutputFormat) object supporting the key and value types K and V in this RDD.
Return an RDD with the pairs from this
whose keys are not in other
.
Return an RDD with the pairs from this
whose keys are not in other
.
Return an RDD with the pairs from this
whose keys are not in other
.
Return an RDD with the pairs from this
whose keys are not in other
.
Uses this
partitioner/partition size, because even if other
is huge, the resulting
RDD will be <= us.
Return an RDD with the values of each tuple.
Extra functions available on RDDs of (key, value) pairs through an implicit conversion. Import
org.apache.spark.SparkContext._
at the top of your program to use these functions.


Space Force
✻
Bow To Our Space Force. They Are The One Who Delegated Their Steem Power!
@dimimp
Sponsor
@empato365
Project Manager (Research Dept.)
@deandaniel
Project Manager
@lordjames
Project Manager
@mbj
Project Manager (Art Dept.)
@gutzygwin
Project Developer
@kilatunzii
Photography Department
@sweetim
Art Department
@joshuaetim
Customer Care Department
@jacobite
Steem Jet Representative
@sherrifakin
Newbie Department
@lovelymkylie
Research Department
@jesse12
Art And Music Department
@ninoh22
Customer Care Department
@aideedavies
Medical Department
@mrprecious
Lead Music Department
@ungest
Customer Care Department
@etemi
Music Department
@lifeofdarlington
Supervising Department
@yungchief
Supervising Department


About Us
We are a community aimed to increase diversity in the tech industry one technologist at a time.
We offer low to no-cost events, workshops, and classes to help support individuals in protected groups
enter and stay in the tech industry.
Our Mission
Increase the diversity of the Science Engineering Art and Math (STEAM) industries by offering low to no-cost skill-building events, workshops, and classes to adults in protected groups to ensure individuals know they can enter, stay, and thrive in STEAM.
Our Vision
Increase the wealth with quality of life of protected groups by creating a space to learn the technical and nontechnical skills required for roles in the tech industry.
Our Team
Valerie Phoenix
Executive DirectorEbony Smith
Social Media ManagerTracie Howard Womack
Community Advocate DirectorOur Values
Diversity
Diversity is key to having a well rounded outlook on any problem you may face. The more diverse a team is, is the more life experiences and viewpoints you will be privy to.
Empathy
We ask our members to come to us as they are, so we must remember to be open to meet them where they are.
Integrity
In order to keep our space a safe space, we have to make sure all of our decisions are directly inline with our mission and overall vision for our members' safety.
Teach
Acquiring knowledge is important, but sharing knowledge is how we grow as a group.
Intentional
To excel for our members, we must remember to put them first.
Become A Member
Join Our Newsletter
The TBC (Tech By Choice) newsletter is a monthly email we send out to share resources, tips, tricks, and funny memes with other people across our community.
Events
Join us for classes, workshops, and other fun digital events through our Eventbrite page
Become A Member
Join Tech By Choice community today! We’re always accepting applications for members, teachers, classroom TAs, and volunteers. You'll get access to our Slack, job openings from our partners, and access to our classes.
Contribute to TBC
Join us and help us build the tech behind Tech By Choice!
We need help with:
- Website development
- Workshop/Class material development
- Product Design and Strategy
- Graphic Art for TBC swag and for our website
Learn more here
Tech By Choice
Diverse By Design
Have any questions or looking for way to help support our mission? Feel free to contact us any time through any of our social media accounts below, or via email.
Hope to hear from you soon!


| Key Model | Explaination |
|---|---|
| Quality Attribute Template (QA Template) | Create templates for quality attributes, which can be then used while creating of a new project. The big advantage of templates is the option to add variables. |
| Catalog | Group multiple quality attribute templates into one catalog and re-use the catalog for creating new projects. Every project of the same catalog has automatically all quality attribute templates inside. |
| Project | This is the most important element in QUALI-T, because it is the original intention of the software. Specify quality attributes for the project you are working on. A project consists of a set of quality attributes instances. |
| Quality Attribute Instance (QA) | Quality Attribute Instances live inside projects. After creating the project based on a catalog, each of the quality attribute template gets an instance. The instance is now decoupled from the catalog and can be edited. The user wants typically to change values of the variables, change quality property states or export to Issue Tracking System (JIRA). |
| Quality Properties (QP) | Quality Properties are useful for measuring some specific properties of a quality attribute. QP's are boolean values and can either be satisfied or not. An example could be S - Specific. Each quality attribute has a checkbox next to it. If the quality property is specific, the checkbox could be checked. This means that the S - Specific quality property is satisfied. |
Question: We have a new project for our project initiator and I'm new to QUALI-T. How can I benefit from this software?
| Step | Instructions |
|---|---|
| 1. Create Project Initiator | First you need to create the projectInitiator in the QUALI-T system. |
| 2. Check existing catalogs & quality attributes | Maybe there is already a matching catalog with quality attributes. Check the list of catalogs. |
| 3. Create a new catalog, if no matching catalog was found. | Create desired catalog with quality attribute templates. Specify variables for quality attribute templates. |
| 4. Create Project by using a catalog | Specify your projects properties (name, JIRA connection, quality properties) and choose the matching catalog. |
| 5. Edit Project | After creating the project you will be in the edit mode. Specify each quality attribute instance by filling out the variables. |
| 6. Administrate the project in the future | Keep in mind that you have also an overview of all quality attribute instances in the project. You can set the state of quality properties, export to Issue Management System, check for validity (fuzziness detector API). |
Question: What is the role changer, where can I find it and what is the effect?
The role changer allows you to change your current view to a specific roles view. This means that you put on a
hat for a role and
you can work in this role without disturbance. You can find the roles changer in the navigation (left side) next
to the QUALI-T logo.
Available roles are:
| Role | Description |
|---|---|
| CC - Catalog Curator | This role usualy creates new catalogs. |
| AA - Architecture Analyst | This role usualy creates new projects, quality attribute templates, project initiators. |
| AS - Architecture Synthesizer | This role usualy creates creates and updates quality attribute templates. Edits project's instances and runs reports.. |
| AE - Architecture Evaluator | This role usualy creates updates status of quality properties in a project and run reports. |
| Admin | Superuser role |


A group of cells can move together from one area in the body to another through collective cell migration. Evidence from Cheung et al.[1] suggests that in certain breast cancer cells, some cells lead a collection of following cells to invade other tissues. This is an important behavior to understand as it can lead to metastasis forming. The behavior observed by Cheung et al. is that certain cells (expressing cytokeratin-14) pull on the extracellular matrix (ECM), causing other cells to follow them, resulting in collective migration.
The ECM model is built on PhysiCell[2], an open source cell-based, multicellular 3-D modeling framework written in C++. If you are not already familiar with PhysiCell please start by reviewing it (see references and Install section below) prior to diving deeply into the modeling code here.
The ECM model is made up of a set of small ECM units, each having 3 primary components: fiber orientation, anisotropy, and density. In our code, we have two cell types: leader cells (K14+) and follower cells (K14-). In the general model, cells and the ECM are capable of a bidirectional relationship. In our specific example, leaders can modify the ECM but are not affected by the ECM, while the follower cells are affected by the ECM but cannot change it.
Oxygen is introduced at the boundary of the domain with Dirichlet conditions. The decay-length scale for oxygen is 1,000 μm, with a length scale of 100 μm for cell-dense regions, and boundary value of 38 mmHg.
In PhysiCell, an agent/cell-based modeling program each cell is modeled as a software agent. This leads to each cell possibly containing unique values and behavior. Through simulations paired with the notion that each cell is unique, biologically-relevant behavior has the potential to occur at a macroscopic level, above that of an individual agent. In our example code, we alter the adhesion and base cell-speed parameters to explore how the resulting emergent cell patterns are impacted by the ECM models, looking for biologically relevant behavior.
Leader cells represent invasive cells. They follow chemotactically the oxygen gradient, which originates at the simulation boundary. Meaning, the leaders will be actively moving outwards the gradient. They are able to remodel the ECM. Additionally, they are able to respond to both leader and follower cells with physical adhesion and repulsion.
Follower cells represent less invasive but still cancerous cells (a different phenotype than leaders). Follower cells follow cues from the ECM with a 100% sensitivity. Additionally, they are capable of 2-D random motion when no fibers are present in the simulation. Similar to our leader cells, they are able to respond to both leader and follower cells with physical adhesion and repulsion.
The Extracellular Matrix (ECM) Model is made of a set of small ECM units, each having 3 components: fiber orientation, anisotropy, and density. In our code, we have two cell types: leader cells (K14+) and follower cells (K14-). Leaders can modify the ECM as they come in contact with it but are not affected by the ECM. Follower cells are affected by the ECM but cannot change it. Taken together (at the level of the cell ensemble), this creates a bidirectional relationship with the ECM and cells.
( f ) which represents the average orientation of all the local fibers within a given voxel.
Given as a unit vector. Although the fibers are represented as a vector, they are better thought of as a line segment as cells have no innate preferred direction of travel based on the “direction” of the unit vector representing fiber orientation.
( a ) (ranging from 0 to 1) which represents how aligned the ECM fibers are as well as elements of fiber resistance to deformation and realignment.
An anisotropy value closer to 0 means the fibers are less aligned, while an anisotropy value closer to 1 is more aligned and more resistant to realignment.
( ρ ) which represents how much overall fiber is present relative to how much space is available to be filled with fibers.
A scalar ranging from 0 to 1
Realign the fibers/ECM proportionally to the cell's speed, a base rate, and how aligned the fibers already are:
Using the above rate, fibers are updated with time using the following differential equation:
Alter anisotropy locally in proportion to cell speed:
Using the above function, anisotropy is updated and bound between 0 and 1:
Increase and decrease density (secrete/destroy) to a specific target:
Fiber orientation influences the cell's motility vector, while anisotropy controls the amount of influence it has. To account for the lack of preferred direction of travel along ECM fibers but our storage of the orientation as a vector, we decompose the cell’s motility vector in the direction along the fiber (within that voxel) and the direction perpendicular to the fiber by:
We then take two dot products: the dot product of the motility vector and fiber direction, and the dot product with the motility vector and vector perpendicular to the fiber.
The actual motility vector is calculated using the equation below:
Where γ is the produce of the cell's sensitivity to the ECM and the local anisotropy:
Overall, this lets the cell’s preferred direction of travel select which direction along the ECM fiber it moves. Cells that are on completely anisotropic ECM and that are highly sensitive to ECM orientation will conduct a 1-D random walk along the fibers.
Too little density and cells have nothing to "grab"
Too much density and cells cannot pass through the dense fibers
This is represented this piecewise function with a peak at ρideal:
Modify the parameters in the "config basics" and "user params" tabs. Click the "run" button once ready.
To view the ECM cell, substrate and fiber plots, click the "Out: Plots" tab, and slide the bar to advance through simulation frames. Note that as the simulation runs, the "max" field (maximum frame number) will increase, so you can view more simulation frames. You can select a substrate from the drop-down menu and toggle the fibers.
To run the model without ECM influence, set “default_ECM_sensitivity” and “default_hysteresis_bias” to 0.0. To run the model with ECM influence but no memory set “default_hysteresis_bias” to 0.0 and “default_ECM_sensitivity” to 1.0. To run the model with ECM and memory set “default_hysteresis_bias” and “default_ECM_sensitivity” to 1.0.
observe differences among those model selections, we recommend setting adhesion to 6.5 and speed to 0.3. A good alternative is adhesion and speed set to 10 and 0.5, respectively. Ensure that cell_motility_ECM_interaction_model_selector is set to “follower hysteresis/no follower chemotaxis” and that unit_test_setup is 0.0.
[1] Cheung JK, Gabrielson E, Werb Z, Ewald AJ (2013) Collective invasion in breast cancer requires a conserved basal epithelial program. Cell 155(7):1639-51. doi: 10.1016/j.cell.2013.11.029
[2] Ghaffarizadeh A, Heiland R, Friedman SH, Mumenthaler SM, Macklin P (2018) PhysiCell: An open source physics-based cell simulator for 3-D multicellular systems. PLoS Comput Biol 14(2): e1005991. https://doi.org/10.1371/journal.pcbi.1005991


This page shows tracking statistics for my website using Pirsch and Chart.Js. The data shows unique visitors. All times and dates are UTC.
Active visitors within the last five minutes: {{.ActiveVisitors}}
The next diagram shows active visitors for each hour of today.
This is the cumulated visitor count per hour on each day of the selected time frame.
Here are the top 10 languages used by my visitors.
| Language | Absolute | Relative |
|---|---|---|
| {{if $data.Language}}{{$data.Language}}{{else}}(not set){{end}} | {{$data.Visitors}} | {{round (multiply $data.RelativeVisitors 100)}} % |


Tyler Thomas is currently wandering the world in search of something. Right now, he is living Australia and wondering how it is that he ended up there. He pursues a host of different interests at different times, currently being interested in metamathematics and Ukulele.
Contact
Email<EMAIL_ADDRESS>PGP: Key


Filter async modes
Options for parallelism both in DART and in the model advances:
Simplest case, async=0:
This is a single MPI executable, with each call to the
model being simply a subroutine call from each MPI task.
To the DART mpi intro document
Parallel advance, async=2:
The filter executable is one MPI program, and the model is a
single, sequential executable. Each MPI task uses the
unix "system()" call to invoke a shell script (advance_model.csh)
which runs the models as independent programs.
To the DART mpi intro document
Other views of how the async=2 option is structured; these
may be more or less helpful.
Parallel advance, async=2:
Parallel advance, async=2, second version:
Parallel model advance, async=2, showing how data is communicated
between filter and the model thru intermediate files.
IC are 'initial condition' files, UD are 'updated' files.
Parallel model advance, async=4:
The filter executable is one MPI program, and the model is also
an MPI program. The filter executable communicates with the
runme_filter shell script, which sequentially invokes mpirun
to advance each of the model runs, one per ensemble member,
still using advance_model.csh.
To the DART mpi intro document
Parallel model advance, async=4, showing how data is communicated
between filter and the model thru intermediate files.
IC are 'initial condition' files, UD are 'updated' files.
Terms of Use
DART software - Copyright UCAR. This open source software is provided
by UCAR, "as is", without charge, subject to all terms of use at
http://www.image.ucar.edu/DAReS/DART/DART_download


#include <array_ops.h>
Removes dimensions of size 1 from the shape of a tensor.
Given a tensor input
, this operation returns a tensor of the same type with all dimensions of size 1 removed. If you don't want to remove all size 1 dimensions, you can remove specific size 1 dimensions by specifying axis
.
For example:
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1] shape(squeeze(t)) ==> [2, 3]
Or, to remove specific size 1 dimensions:
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1] shape(squeeze(t, [2, 4])) ==> [1, 2, 3, 1]
Arguments:
input
to squeeze.Optional attributes (see Attrs
):
[-rank(input), rank(input))
.Returns:
Output
: Contains the same data as input
, but has one or more dimensions of size 1 removed.
Constructors and Destructors |
|
|---|---|
Squeeze(const ::tensorflow::Scope & scope, ::tensorflow::Input input)
|
|
Squeeze(const ::tensorflow::Scope & scope, ::tensorflow::Input input, const Squeeze::Attrs & attrs)
|
Public attributes |
|
|---|---|
operation
|
|
output
|
Public functions |
|
|---|---|
node() const
|
::tensorflow::Node *
|
operator::tensorflow::Input() const
|
|
operator::tensorflow::Output() const
|
|
Public static functions |
|
|---|---|
Axis(const gtl::ArraySlice< int > & x)
|
Structs |
|
|---|---|
|
tensorflow:: |
Optional attribute setters for Squeeze. |
Operation operation
::tensorflow::Output output
Squeeze( const ::tensorflow::Scope & scope, ::tensorflow::Input input )
Squeeze( const ::tensorflow::Scope & scope, ::tensorflow::Input input, const Squeeze::Attrs & attrs )
::tensorflow::Node * node() const
operator::tensorflow::Input() const
operator::tensorflow::Output() const
Attrs Axis( const gtl::ArraySlice< int > & x )


Monu Gurjar (Founder And Owner)
Praesent faucibus malesuucibus. Donec laoreet metus id laoreet
Not Appointed Yet (CEO)
We want our CEO to be a motivator and a great leader. We also need you to be able to see the big picture and think out of the box.
Nitin Kumar (UX Designer)
an advocate for the end-users of a website. Key areas of focus include information architecture, user research, branding, visual design, and content.
Kapil Gurjar (Developer and Maintainer)
He is responsible for the coding, design and layout of a website according to our company's specifications. As the role takes into consideration user experience and function, a certain level of both graphic design and computer programming is necessary.
Amit Gurjar (Developer)
Gemus at magna non nunc tristique rhoncus. Aliquam nibh antegestas id dictum ammodo luctus liberober.
Mraesent faucibus malesuadam faucibusonec laoreet metus ideDilip Gurjar (Maintainer)
He is an IT specialist and He manage the website. He oversee the functionality of the website, evaluate and manage website performance, facilitate hosting and server management, and develop, maintain and update website content


import 'dart:io';
main() async {
var server = await HttpServer.bind(InternetAddress.ANY_IP_V4, 8777);
print('serving on port ${server.port}');
// HttpServer extends Stream, so using await-for
// will run the loop body when a request is added to the stream.
await for (HttpRequest req in server) {
// resposne
req.response
..write("welcome to my dart server")
..close();
}
}
$ dart http_server.dart
serving on port 8777
opening localhost:8777
:
welcome to my dart server


If you were to try and analyze statistics for your favorite hockey team, or try to predict an outcome of the next match you’d probably browse the internet in search of convenient tools to get the data you want from the NHL website.  Therefore, the rpuck library is designed to allow users the ability to conveniently get both relevant and historical statistics from the publicly available but as of yet undocumented NHL.com API. As of now the package has limited functionality, and is considered a work-in-progress. We will add in additional functionality in the coming weeks which can easily be extended based on feedback we recieve.
You can install the development version of rpuck from GitHub with:
# install.packages("devtools")
devtools::install_github("UBC-MDS/rpuck")
draft_pick(pick_number=NULL, round_number=NULL, year=NULL)
:
draft_pick( )
function makes an API call to the drafts summary on the NHL.com API. The function returns information about draft picks for the specified arguments and stores them in a data frame.attendance(regular=True, playoffs=True, start_season=NULL, end_season=NULL)
:
team_stats(start_season=NULL, end_season=NULL)
:
team_stats( )
function makes an API call to the team summary endpoint on the NHL.com API. The function returns team seasonal stats for given seasons sorted by total team points.player_stats(start_date=NULL, end_date=NULL)
:
player_stats( )
function makes an API call to the player summary endpoint on the NHL.com API. The function returns the top 100 player stats for a given date range as sorted by total points.You can also find function descriptions and their use cases in package vignettes
The package can extract and visualize data from NHL statistics in convenient format. Below is an example of how one can use rpuck.
To load the package:
Get a chart showing the attendance over a specified time period:
attendance(regular=TRUE, playoffs=FALSE, start_season= 1990, end_season=2018)
Result of draft by pick number, draft number and year in summary report:
draft_pick(pick_number = 1, round_number = 2, year = 2019)
#> Player triCode Pick_number Round_number Year
#> 11402 Shane Pinto OTT 1 2 2019
Query the top 100 player’s stats from the player’s summary report:
player_s <- player_stats(start_date = "2019-10-02", end_date = "2020-02-28")
head(player_s)
#> assists evGoals evPoints faceoffWinPct gameWinningGoals gamesPlayed goals
#> 1 62 24 59 0.51809 10 64 37
#> 2 44 27 54 0.37500 9 65 46
#> 3 56 25 67 0.50000 4 63 32
#> 4 56 20 48 0.47892 5 58 31
#> 5 52 23 56 0.42693 4 63 33
#> 6 55 20 52 0.40816 5 65 26
#> lastName otGoals penaltyMinutes playerId plusMinus points pointsPerGame
#> 1 Draisaitl 2 16 8477934 -11 99 1.54687
#> 2 Pastrnak 1 34 8477956 20 90 1.38461
#> 3 Panarin 0 18 8478550 35 88 1.39682
#> 4 McDavid 0 28 8478402 -7 87 1.50000
#> 5 MacKinnon 2 12 8477492 9 85 1.34920
#> 6 Marchand 0 72 8473419 24 81 1.24615
#> positionCode ppGoals ppPoints shGoals shPoints shootingPct shootsCatches
#> 1 C 13 40 0 0 0.18686 L
#> 2 R 19 36 0 0 0.18039 R
#> 3 L 7 21 0 0 0.16494 R
#> 4 C 11 39 0 0 0.16062 L
#> 5 C 10 29 0 0 0.11262 R
#> 6 L 5 26 1 3 0.14857 L
#> shots skaterFullName timeOnIcePerGame
#> 1 198 Leon Draisaitl 1361.516
#> 2 255 David Pastrnak 1146.369
#> 3 194 Artemi Panarin 1219.460
#> 4 193 Connor McDavid 1323.000
#> 5 293 Nathan MacKinnon 1278.889
#> 6 175 Brad Marchand 1178.246
Stats for teams specified by start year or start year and end year:
attendance <- team_stats(start_season = "19801981", end_season = "19891990")
head(attendance)
#> faceoffWinPct gamesPlayed goalsAgainst goalsAgainstPerGame goalsFor
#> 1 NA 80 338 4.2250 230
#> 2 NA 80 336 4.2000 343
#> 3 NA 80 317 3.9625 321
#> 4 NA 80 287 3.5875 306
#> 5 NA 80 394 4.9250 257
#> 6 NA 80 333 4.1625 311
#> goalsForPerGame losses otLosses penaltyKillNetPct penaltyKillPct pointPct
#> 1 2.8750 49 NA 0.757679 0.733789 0.30000
#> 2 4.2875 34 NA 0.751445 0.734105 0.50000
#> 3 4.0125 34 NA 0.798449 0.771318 0.48750
#> 4 3.8250 35 NA 0.792880 0.754046 0.50000
#> 5 3.2125 53 NA 0.732323 0.724748 0.28125
#> 6 3.8875 39 NA 0.767068 0.730924 0.46250
#> points powerPlayNetPct powerPlayPct regulationAndOtWins seasonId
#> 1 48 0.182119 0.215231 17 19821983
#> 2 80 0.185185 0.205387 34 19821983
#> 3 78 0.216117 0.238095 32 19821983
#> 4 80 0.203821 0.229299 35 19821983
#> 5 45 0.183333 0.225000 18 19821983
#> 6 74 0.208459 0.229607 33 19821983
#> shotsAgainstPerGame shotsForPerGame teamFullName teamId ties wins
#> 1 32.0875 25.6875 New Jersey Devils 1 14 17
#> 2 32.5375 28.9125 Quebec Nordiques 32 12 34
#> 3 33.0875 28.5875 Calgary Flames 20 14 32
#> 4 29.8000 28.5250 New York Rangers 3 10 35
#> 5 32.6875 30.2250 Pittsburgh Penguins 5 9 18
#> 6 31.6375 31.7625 Winnipeg Jets (1979) 33 8 33
#> winsInRegulation winsInShootout
#> 1 17 0
#> 2 34 0
#> 3 32 0
#> 4 35 0
#> 5 18 0
#> 6 33 0
We have included a variety of tests for each function in the tests\testthat
directory. The tests check that the functions error gracefully and that proper function calls return the correct data/objects.


"""
Optimise strategy to use
We'll write a function that will return the Sharpe ratio for a strategy using the NIFTY
The below code is what we normally use to test a strategy
We want the classifier to be XGBClassifier and the params to be what we pass in
"""
from xgboost.sklearn import XGBClassifier
from hyperopt import hp
from hyperopt import fmin, tpe, hp, STATUS_OK,Trials
# We want to tune the Gradient boosted classifier, so that we find the optimum set of parameters
# For that there are 2 things we need to define
# A function that will be minimized by hyperopt
from analysis.src.setup import *
from analysis.src.testAndTrain import *
[docs]def score(params):
""" The function returns -score as we want to maximize Sharpe but hyperopt will try to minimize. A Search space with all the combinations over which the function will be minimized.
:param name: The name to use
:type name: None
:param state: None
:type state: None
:returns: None
:raises: None
"""
params['n_estimators']=int(params['n_estimators'])
options = {'qtype': 'adjclose',
'tables': ["stockProd", "tradingDays"],
'freq': 0, # The frequency of trading, daily=0, monthly=1,weekly=2
'offset': 1,
# The offset if the period > 1day, ie which trading day in the month/week the strategy will be executed
'pure': 0, # from here we have the features , the returns as is
'cal': 0, # Calendar features
'history': 0, # last 3 periods returns
'momentum': 0, # momentum features
'jump': 0, # jump features
'value': 0, # long term reversal features
'prevWeeks': 1, # Now by turning this to 1 we can run a model which includes previous weeks
'algo': XGBClassifier,
'algo_params': params
}
supportTickers = None
ticker = "RIO"
trainStart = "2010-01-01"
testPeriod = ["2013-06-01", "2016-04-01"]
print ("Training with params")
result = backtestResults(ticker, trainStart, testPeriod, options, supportTickers)
score = result[0]
print (params)
print ("\t Score {0}".format(score))
return {'loss':-score,'status':STATUS_OK}
def main():
space ={
'n_estimators':hp.quniform('n_estimators',100,1000,1),
'learning_rate':hp.quniform('learning_rate',0.025,0.5,0.025),
'max_depth':hp.quniform('max_depth',1,13,1),
'min_child_weight': hp.quniform('min_child_weight',1,6,1),
'subsample': hp.quniform('subsample',0.5,1,0.05),
'gamma':hp.quniform('gamma',0.5,1,0.05),
'colsample_bytree':hp.quniform('colsample_bytree',0.5,1,0.05),
'nthread':6,
'silent':1
}
trials = Trials()
best = fmin(score,space,algo=tpe.suggest,trials=trials,max_evals=250)
print(best)
if __name__ == "__main__":
main()