, an existing instance can be used by setting hibernate.cache.hazelcast.instance_name
Hibernate property to HazelcastInstance
's name. For more information see Named HazelcastInstance.
Disabling shutdown during SessionFactory.close()
Shutting down HazelcastInstance
can be disabled during SessionFactory.close()
. To achieve this set the Hibernate property hibernate.cache.hazelcast.shutdown_on_session_factory_close
to false. (In this case Hazelcast property hazelcast.shutdownhook.enabled
should not be set to false.) Default value is true
.
<property name="hibernate.cache.hazelcast.use_native_client">true</property>
To setup Native Client properly, you should add Hazelcast group-name, group-password and cluster member address properties. Native Client will connect to defined member and will get addresses of all members in the cluster. If the connected member dies or leaves the cluster, client will automatically switch to another member in the cluster.
<property name="hibernate.cache.hazelcast.native_client_address"><IP_ADDRESS></property>
<property name="hibernate.cache.hazelcast.native_client_group">dev</property>
<property name="hibernate.cache.hazelcast.native_client_password">dev-pass</property>
NOTE: To use Native Client, you should add hazelcast-client-<version>.jar
into your classpath. Refer to Native Clients for more information.
Hibernate has four cache concurrency strategies: read-only, read-write, nonstrict-read-write and transactional. But, Hibernate does not force cache providers to support all strategies. Hazelcast supports first three (read-only, read-write, nonstrict-read-write) of these strategies. It has no support for transactional strategy yet.
<class name="eg.Immutable" mutable="false">
<cache usage="read-only"/>
....
</class>
<class name="eg.Cat" .... >
<cache usage="read-write"/>
....
<set name="kittens" ... >
<cache usage="read-write"/>
....
</set>
</class>
<class-cache usage="read-only" class="eg.Immutable"/>
<class-cache usage="read-write" class="eg.Cat"/>
<collection-cache collection="eg.Cat.kittens" usage="read-write"/>
OR
@Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
public class Cat implements Serializable {
...
}
Accessing underlying HazelcastInstance
Using com.hazelcast.hibernate.instance.HazelcastAccessor
you can access the underlying HazelcastInstance
used by Hibernate SessionFactory.
SessionFactory sessionFactory = ...;
HazelcastInstance hazelcastInstance = HazelcastAccessor
.getHazelcastInstance(sessionFactory);
Changing/setting lock timeout value of read-write strategy
Lock timeout value can be set using hibernate.cache.hazelcast.lock_timeout_in_seconds
Hibernate property. Value should be in seconds and default value is 300 seconds.
If you are using Tomcat as your web container, please see our Tomcat based Web Session Replication.
*Sample Code: Please see our sample application for Filter Based Web Session Replication.*
Assume that you have more than one web servers (A, B, C) with a load balancer in front of them. If server A goes down, your users on that server will be directed to one of the live servers (B or C), but their sessions will be lost.
So we have to have all these sessions backed up somewhere if we do not want to lose the sessions upon server crashes. Hazelcast Web Manager (WM) allows you to cluster user HTTP sessions automatically. The following are required for enabling Hazelcast Session Clustering:
Target application or web server should support Java 1.6 or higher
Target application or web server should support Servlet 3.0 or higher spec
Session objects that need to be clustered have to be Serializable
Here are the steps to setup Hazelcast Session Clustering:
Put the hazelcast
and hazelcast-wm
jars in your WEB-INF/lib
directory. Optionally, if you wish to connect to a cluster as a client, add hazelcast-client
as well.
Put the following XML into web.xml
file. Make sure Hazelcast filter is placed before all the other filters if any; put it at the top for example.
<filter>
<filter-name>hazelcast-filter</filter-name>
<filter-class>com.hazelcast.web.WebFilter</filter-class>
<!--
Name of the distributed map storing
your web session objects
-->
<init-param>
<param-name>map-name</param-name>
<param-value>my-sessions</param-value>
</init-param>
<!--
TTL value of the distributed map storing
your web session objects.
Any integer between 0 and Integer.MAX_VALUE.
Default is 0 which is infinite.
-->
<init-param>
<param-name>session-ttl-seconds</param-name>
<param-value>0</param-value>
</init-param>
<!--
How is your load-balancer configured?
sticky-session means all requests of a session
is routed to the node where the session is first created.
This is excellent for performance.
If sticky-session is set to false, when a session is updated
on a node, entry for this session on all other nodes is invalidated.
You have to know how your load-balancer is configured before
setting this parameter. Default is true.
-->
<init-param>
<param-name>sticky-session</param-name>
<param-value>true</param-value>
</init-param>
<!--
Name of session id cookie
-->
<init-param>
<param-name>cookie-name</param-name>
<param-value>hazelcast.sessionId</param-value>
</init-param>
<!--
Domain of session id cookie. Default is based on incoming request.
-->
<init-param>
<param-name>cookie-domain</param-name>
<param-value>.mywebsite.com</param-value>
</init-param>
<!--
Should cookie only be sent using a secure protocol? Default is false.
-->
<init-param>
<param-name>cookie-secure</param-name>
<param-value>false</param-value>
</init-param>
<!--
Should HttpOnly attribute be set on cookie ? Default is false.
-->
<init-param>
<param-name>cookie-http-only</param-name>
<param-value>false</param-value>
</init-param>
<!--
Are you debugging? Default is false.
-->
<init-param>
<param-name>debug</param-name>
<param-value>true</param-value>
</init-param>
<!--
Configuration xml location;
* as servlet resource OR
* as classpath resource OR
* as URL
Default is one of hazelcast-default.xml
or hazelcast.xml in classpath.
-->
<init-param>
<param-name>config-location</param-name>
<param-value>/WEB-INF/hazelcast.xml</param-value>
</init-param>
<!--
Do you want to use an existing HazelcastInstance?
Default is null.
-->
<init-param>
<param-name>instance-name</param-name>
<param-value>default</param-value>
</init-param>
<!--
Do you want to connect as a client to an existing cluster?
Default is false.
-->
<init-param>
<param-name>use-client</param-name>
<param-value>false</param-value>
</init-param>
<!--
Client configuration location;
* as servlet resource OR
* as classpath resource OR
* as URL
Default is null.
-->
<init-param>
<param-name>client-config-location</param-name>
<param-value>/WEB-INF/hazelcast-client.properties</param-value>
</init-param>
<!--
Do you want to shutdown HazelcastInstance during
web application undeploy process?
Default is true.
-->
<init-param>
<param-name>shutdown-on-destroy</param-name>
<param-value>true</param-value>
</init-param>
<!--
Do you want to cache sessions locally in each instance?
Default is false.
-->
<init-param>
<param-name>deferred-write</param-name>
<param-value>false</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>hazelcast-filter</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>FORWARD</dispatcher>
<dispatcher>INCLUDE</dispatcher>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>
<listener>
<listener-class>com.hazelcast.web.SessionListener</listener-class>
</listener>
war
file as you would normally do.It is that easy. All HTTP requests will go through Hazelcast WebFilter
and it will put the session objects into Hazelcast distributed map if needed.
*Sample Code: Please see our sample application for Spring Security Support.*
If Spring based security is used for application, you should use com.hazelcast.web.spring.SpringAwareWebFilter
instead of com.hazelcast.web.WebFilter
in your filter definition.
...
<filter>
<filter-name>hazelcast-filter</filter-name>
<filter-class>com.hazelcast.web.spring.SpringAwareWebFilter</filter-class>
...
</filter>
...
SpringAwareWebFilter
notifies Spring by publishing events to Spring context and these events are used by org.springframework.security.core.session.SessionRegistry
instance.
As like before, you must also define com.hazelcast.web.SessionListener
in your web.xml
. However, it is not needed to define org.springframework.security.web.session.HttpSessionEventPublisher
in your web.xml
as before, since SpringAwareWebFilter
already informs Spring about session based events like create or destroy.
Hazelcast Session Replication works as P2P by default. You need to set use-client
parameter to true to switch to Client/Server architecture. P2P mode is more flexible and requires no configuration in advance while in Client/Server architecture, clients need to connect to an existing Hazelcast Cluster. In case of connection problems, clients will try to reconnect to the cluster. Default retry count is 3.
deferred-write
If the value for deferred-write
is set as true, Hazelcast will cache the session locally and will update the local session on set or deletion of an attribute. Only at the end of request, it will update the distributed map with all the updates. So, it will not be updating the distributed map on each attribute update. It will only call it once at the end of request. It will be also caching it, i.e. whenever there is a read for the attribute, it will read it from the cache.
Important note about deferred-write=false
setting:
If deferred-write
is false, you will not have local attribute cache as mentioned above. In this case, any update (i.e. setAttribute
) on the session will directly be available in the cluster. One exception to this behavior is the changes to the session attribute objects. To update an attribute cluster wide, setAttribute
has to be called after making changes to the attribute object.
Following example explains how to update an attribute in the case of deferred-write=false
setting:
session.setAttribute("myKey", new ArrayList());
List list1 = session.getAttribute("myKey");
list1.add("myValue");
session.setAttribute("myKey", list1); // changes updated in the cluster
SessionId generation is done by Hazelcast Web Session Module if session replication is configured in the web application. Default cookie name for the sessionId is hazelcast.sessionId
and this is configurable with cookie-name
parameter in the web.xml
file of the application.
hazelcast.sessionId
is just a UUID prefixed with “HZ” character and without “-“ character, e.g. HZ6F2D036789E4404893E99C05D8CA70C7
.
When called by the target application, the value of HttpSession.getId()
is the same as the value of hazelcast.sessionId
.
Hazelcast automatically removes sessions from the cluster if session is expired on the Web Container. This removal is done by com.hazelcast.web.SessionListener
which is an implementation of javax.servlet.http.HttpSessionListener
.
Default session expiration configuration depends on Servlet Container that is being used. You can also define it in your web.xml.
<session-config>
<session-timeout>60</session-timeout>
</session-config>
If you want to override session expiry configuration with Hazelcast specific configuration, session-ttl-seconds
can be used to specify TTL on Hazelcast Session Replication Distributed Map.
Hazelcast holds whole session attributes in a distributed map and in local HTTP session. Local session is required for fast access to data and distributed map is needed for fail-safety.
If sticky-session
is not used, whenever a session attribute is updated in a node (in both node local session and clustered cache), that attribute should be invalidated in all other nodes' local sessions, because now they have dirty value. So, when a request arrives to one of those other nodes, that attribute value is fetched from clustered cache.
To overcome performance penalty of sending invalidation messages during updates, sticky sessions can be used. If Hazelcast knows sessions are sticky, invalidation will not be sent, because Hazelcast assumes there is no other local session at the moment. When a server is down, requests belonging to a session hold in that server will routed to other one and that server will fetch session data from clustered cache. That means, using sticky sessions, one will not suffer performance penalty of accessing clustered data and can benefit recover from a server failure.
NOTE: This feature is supported for Hazelcast Enterprise 3.3 or higher.
*Sample Code: Please see our sample application for Tomcat Based Web Session Replication.*
Session Replication with Hazelcast Enterprise is a container specific module where no application change is required to enable session replication for JEE Web Applications.
Features
Supported Containers
Tomcat Web Session Replication Module has been tested against following containers.
Latest tested versions are 6.0.39 and 7.0.40.
Requirements
Tomcat Session Replication in Hazelcast Enterprise is a Hazelcast Module where each created HttpSession
Object is kept in Hazelcast Distributed Map. Additionally, if configured with Sticky Sessions, each Tomcat Instance has its own local copy of Session for performance boost.
As the sessions are in Hazelcast Distributed Map, you can use all the available features offered by Hazelcast Distributed Map implementation such as MapStore and WAN Replication.
Tomcat Web Sessions run in two different modes:
This launches embedded Hazelcast Node in each server instance.
Features
This type of deployment is the simplest approach. You can just configure your Tomcat and launch. There is no need for an external Hazelcast cluster.
Sample P2P Configuration to use Hazelcast Session Replication
$HAZELCAST_ENTERPRISE_ROOT
.$HAZELCAST_ENTERPRISE_ROOT/bin/hazelcast.xml
with the provided Hazelcast Enterprise License Key. Put $HAZELCAST_ENTERPRISE_ROOT/lib/hazelcast-enterprise-all-
<version>.jar
, $HAZELCAST_ENTERPRISE_ROOT/lib/hazelcast-enterprise-
<tomcatversion>-
<version>.jar
and hazelcast.xml
to the folder $CATALINA_HOME/lib/
.
Put <Listener>
tag into the file $CATALINA_HOME$/conf/server.xml
as shown below.
<Server>
...
<Listener className="com.hazelcast.session.P2PLifecycleListener"/>
...
</Server>
<Manager>
tag into the file $CATALINA_HOME$/conf/context.xml
as shown below.<Context>
...
<Manager className="com.hazelcast.session.HazelcastSessionManager"/>
...
</Context>
Optional Listener Tag Parameters
configLocation
attribute into <Listener>
tag. It is optional. If not provided, hazelcast.xml
in the classpath is used by default. URL or full filesystem path as a configLocation
value is also supported.In this deployment type, Tomcat instances work as clients to an existing Hazelcast Cluster.
Features
Sample Client/Server Configuration to use Hazelcast Session Replication
$HAZELCAST_ENTERPRISE_ROOT
.Put $HAZELCAST_ENTERPRISE_ROOT/lib/hazelcast-client-
<version>.jar
, $HAZELCAST_ENTERPRISE_ROOT/lib/hazelcast-enterprise-
<version>.jar
and $HAZELCAST_ENTERPRISE_ROOT/lib/hazelcast-enterprise-
<tomcatversion>-
<version>.jar
to the folder $CATALINA_HOME/lib/
.
Put <Listener>
tag into the $CATALINA_HOME$/conf/server.xml
as shown below.
<Server>
...
<Listener className="com.hazelcast.session.ClientServerLifecycleListener"/>
...
</Server>
<Manager>
tag in the $CATALINA_HOME$/conf/context.xml
as shown below.<Context>
<Manager className="com.hazelcast.session.HazelcastSessionManager"
clientOnly="true"/>
</Context>
Launch a Hazelcast Instance using $HAZELCAST_ENTERPRISE_ROOT/bin/server.sh
or $HAZELCAST_ENTERPRISE_ROOT/bin/server.bat
.
Start Tomcat instances with a configured load balancer and deploy web application.
Optional Listener Tag Parameters
configLocation
attribute into <Listener>
tag. It is optional. If not provided, hazelcast-client-default.xml
in hazelcast-client-
<version>.jar
file is used by default. Any client XML file in the classpath, URL or full filesystem path as a configLocation
value is also supported.<Manager>
tag is used both in P2P and Client/Server mode. Following parameters are used to configure Tomcat Session Replication Module to better serve your needs.
mapName
attribute into <Manager>
tag. Its default value is default Hazelcast Distributed Map. Use this attribute if you have specially configured map for special cases like WAN Replication, Eviction, MapStore, etc.sticky
attribute into <Manager>
tag. Its default value is true.processExpiresFrequency
attribute into <Manager>
tag. It specifies the frequency of session validity check, in seconds. Its default value is 6 and minimum value that can be set is 1.deferredWrite
attribute into <Manager>
tag. Its default value is true.Tomcat Web Session Replication Module has its own nature of caching. Attribute changes during the HTTP Request/HTTP Response cycle is cached by default. Distributing those changes to the Hazelcast Cluster is costly. Because of that, Session Replication is only done at the end of each request for updated and deleted attributes. The risk in this approach is to lose data in case a Tomcat crash happens in the middle of HTTP Request operation.
You can change that behavior by setting deferredWrite=false
in your <Manager>
tag configuration. By disabling it, all updates that are done on session objects are directly distributed into Hazelcast Cluster.
Based on Tomcat configuration or sessionTimeout
setting in web.xml
, sessions are expired over time. This requires a cleanup on Hazelcast Cluster as there is no need to keep expired sessions in the cluster.
processExpiresFrequency
which is defined in <Manager>
is the only setting to control the behavior of session expiry policy in Tomcat Web Session Replication Module. By setting this, you can set the frequency of the session expiration checks in the Tomcat Instance.
Tomcat can be configured in two ways to enable Session Replication for deployed applications.
Server Context.xml Configuration
By configuring $CATALINA_HOME$/conf/context.xml
, you can enable session replication for all applications deployed in the Tomcat Instance.
Application Context.xml Configuration
By configuring $CATALINA_HOME/conf/[enginename]/[hostname]/[applicationName].xml
, you can enable Session Replication per deployed application.
Sticky Sessions (default)
Sticky Sessions are used to improve the performance since the sessions do not move around the cluster.
Request goes always to the same instance where the session was firstly created. By using a sticky session, you eliminate session replication problems mostly, except for the failover cases. In case of failovers, Hazelcast helps you not lose existing sessions.
Non-Sticky Sessions
Non-Sticky Sessions are not good for performance because you need to move session data all over the cluster every time a new request comes in.
However, load balancing might be super easy with Non-Sticky caches. In case of heavy load, you can distribute the request to the least used Tomcat instance. Hazelcast supports Non-Sticky Sessions as well.
Each HTTP Request is redirected to the same Tomcat instance if sticky sessions are enabled. The parameter jvmRoute
is added to the end of session ID as a suffix, to make Load Balancer aware of the target Tomcat instance.
When Tomcat Failure happens and Load Balancer cannot redirect the request to the owning instance, it sends request to one of the available Tomcat instances. As jvmRoute
parameter of session ID is different than that of target Tomcat instance, Hazelcast Session Replication Module updates the session ID of the session with the new jvmRoute
parameter. That actually means that, Session is moved to another Tomcat instance and Load Balancer will redirect all subsequent HTTP Requests to the new Tomcat Instance.
NOTE: If stickySession is enabled, jvmRoute
parameter must be set in $CATALINA_HOME$/conf/server.xml
and unique among Tomcat instances in the cluster.
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat-8080">
Please see our sample application for Spring Configuration.
Classpath Configuration
This configuration requires following jar file in the classpath:
hazelcast-
<version>.jar
Bean Declaration
You can declare Hazelcast Objects using the default Spring beans namespace. You can find an example usage of Hazelcast Instance declaration as follows:
<bean id="instance" class="com.hazelcast.core.Hazelcast" factory-method="newHazelcastInstance">
<constructor-arg>
<bean class="com.hazelcast.config.Config">
<property name="groupConfig">
<bean class="com.hazelcast.config.GroupConfig">
<property name="name" value="dev"/>
<property name="password" value="pwd"/>
</bean>
</property>
<!-- and so on ... -->
</bean>
</constructor-arg>
</bean>
<bean id="map" factory-bean="instance" factory-method="getMap">
<constructor-arg value="map"/>
</bean>
Classpath Configuration
Hazelcast-Spring integration requires following jar files in the classpath:
hazelcast-spring-
<version>.jar
hazelcast-
<version>.jar
or
hazelcast-all-
<version>.jar
Bean Declaration
Hazelcast has its own namespace hazelcast for bean definitions. You can easily add namespace declaration xmlns:hz="http://www.hazelcast.com/schema/spring" to beans
tag in context file so that hz namespace shortcut can be used as a bean declaration.
Here is an example schema definition for Hazelcast 3.3.x:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:hz="http://www.hazelcast.com/schema/spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.hazelcast.com/schema/spring
http://www.hazelcast.com/schema/spring/hazelcast-spring.xsd">
<hz:hazelcast id="instance">
<hz:config>
<hz:group name="dev" password="password"/>
<hz:network port="5701" port-auto-increment="false">
<hz:join>
<hz:multicast enabled="false"
multicast-group="<IP_ADDRESS>"
multicast-port="54327"/>
<hz:tcp-ip enabled="true">
<hz:members><IP_ADDRESS>, <IP_ADDRESS></hz:members>
</hz:tcp-ip>
</hz:join>
</hz:network>
<hz:map name="map"
backup-count="2"
max-size="0"
eviction-percentage="30"
read-backup-data="true"
eviction-policy="NONE"
merge-policy="com.hazelcast.map.merge.PassThroughMergePolicy"/>
</hz:config>
</hz:hazelcast>
<hz:client id="client">
<hz:group name="${cluster.group.name}" password="${cluster.group.password}" />
<hz:network connection-attempt-limit="3"
connection-attempt-period="3000"
connection-timeout="1000"
redo-operation="true"
smart-routing="true">
<hz:member><IP_ADDRESS>:5701</hz:member>
<hz:member><IP_ADDRESS>:5701</hz:member>
</hz:network>
</hz:client>
Hazelcast Supported Type Configurations and Examples
map
multiMap
replicatedmap
queue
topic
set
list
executorService
idGenerator
atomicLong
atomicReference
semaphore
countDownLatch
lock
<hz:map id="map" instance-ref="client" name="map" lazy-init="true" />
<hz:multiMap id="multiMap" instance-ref="instance" name="multiMap"
lazy-init="false" />
<hz:replicatedmap id="replicatedmap" instance-ref="instance"
name="replicatedmap" lazy-init="false" />
<hz:queue id="queue" instance-ref="client" name="queue"
lazy-init="true" depends-on="instance"/>
<hz:topic id="topic" instance-ref="instance" name="topic"
depends-on="instance, client"/>
<hz:set id="set" instance-ref="instance" name="set" />
<hz:list id="list" instance-ref="instance" name="list"/>
<hz:executorService id="executorService" instance-ref="client"
name="executorService"/>
<hz:idGenerator id="idGenerator" instance-ref="instance"
name="idGenerator"/>
<hz:atomicLong id="atomicLong" instance-ref="instance" name="atomicLong"/>
<hz:atomicReference id="atomicReference" instance-ref="instance"
name="atomicReference"/>
<hz:semaphore id="semaphore" instance-ref="instance" name="semaphore"/>
<hz:countDownLatch id="countDownLatch" instance-ref="instance"
name="countDownLatch"/>
<hz:lock id="lock" instance-ref="instance" name="lock"/>
Hazelcast also supports lazy-init
, scope
and depends-on
bean attributes.
<hz:hazelcast id="instance" lazy-init="true" scope="singleton">
...
</hz:hazelcast>
<hz:client id="client" scope="prototype" depends-on="instance">
...
</hz:client>
For map-store, you should set either class-name or implementation attribute
<hz:config>
<hz:map name="map1">
<hz:near-cache time-to-live-seconds="0" max-idle-seconds="60"
eviction-policy="LRU" max-size="5000" invalidate-on-change="true"/>
<hz:map-store enabled="true" class-name="com.foo.DummyStore"
write-delay-seconds="0"/>
</hz:map>
<hz:map name="map2">
<hz:map-store enabled="true" implementation="dummyMapStore"
write-delay-seconds="0"/>
</hz:map>
<bean id="dummyMapStore" class="com.foo.DummyStore" />
</hz:config>
Hazelcast Distributed Objects could be marked with @SpringAware if the object wants
ApplicationContextAware
, BeanNameAware
,InitializingBean
, @PostConstruct
.Hazelcast Distributed ExecutorService
or more generally any Hazelcast managed object can benefit from these features. To enable SpringAware objects, you have to first configure HazelcastInstance as explained in Spring Configuration section.
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:hz="http://www.hazelcast.com/schema/spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.hazelcast.com/schema/spring
http://www.hazelcast.com/schema/spring/hazelcast-spring.xsd">
<context:annotation-config />
<hz:hazelcast id="instance">
<hz:config>
<hz:group name="dev" password="password"/>
<hz:network port="5701" port-auto-increment="false">
<hz:join>
<hz:multicast enabled="false" />
<hz:tcp-ip enabled="true">
<hz:members><IP_ADDRESS>, <IP_ADDRESS></hz:members>
</hz:tcp-ip>
</hz:join>
</hz:network>
...
</hz:config>
</hz:hazelcast>
<bean id="someBean" class="com.hazelcast.examples.spring.SomeBean"
scope="singleton" />
...
</beans>
Distributed Map Example:
@SpringAware
@Component("someValue")
@Scope("prototype")
public class SomeValue implements Serializable, ApplicationContextAware {
private transient ApplicationContext context;
private transient SomeBean someBean;
private transient boolean init = false;
public void setApplicationContext( ApplicationContext applicationContext )
throws BeansException {
context = applicationContext;
}
@Autowired
public void setSomeBean( SomeBean someBean) {
this.someBean = someBean;
}
@PostConstruct
public void init() {
someBean.doSomethingUseful();
init = true;
}
...
}
HazelcastInstance hazelcastInstance =
(HazelcastInstance) context.getBean( "hazelcast" );
SomeValue value = (SomeValue) context.getBean( "someValue" )
IMap<String, SomeValue> map = hazelcastInstance.getMap( "values" );
map.put( "key", value );
@PostConstruct
.HazelcastInstance hazelcastInstance =
(HazelcastInstance) context.getBean( "hazelcast" );
IMap<String, SomeValue> map = hazelcastInstance.getMap( "values" );
SomeValue value = map.get( "key" );
Assert.assertTrue( value.init );
ExecutorService Example:
@SpringAware
public class SomeTask
implements Callable<Long>, ApplicationContextAware, Serializable {
private transient ApplicationContext context;
private transient SomeBean someBean;
public Long call() throws Exception {
return someBean.value;
}
public void setApplicationContext( ApplicationContext applicationContext )
throws BeansException {
context = applicationContext;
}
@Autowired
public void setSomeBean( SomeBean someBean ) {
this.someBean = someBean;
}
}
someBean
is autowired.HazelcastInstance hazelcastInstance =
(HazelcastInstance) context.getBean( "hazelcast" );
SomeBean bean = (SomeBean) context.getBean( "someBean" );
Future<Long> f = hazelcastInstance.getExecutorService().submit(new SomeTask());
Assert.assertEquals(bean.value, f.get().longValue());
// choose a member
Member member = hazelcastInstance.getCluster().getMembers().iterator().next();
Future<Long> f2 = (Future<Long>) hazelcast.getExecutorService()
.submitToMember(new SomeTask(), member);
Assert.assertEquals(bean.value, f2.get().longValue());
NOTE: Spring managed properties/fields are marked as transient
.
Please see our sample application for Spring Cache.
As of version 3.1, Spring Framework provides support for adding caching into an existing Spring application.
<cache:annotation-driven cache-manager="cacheManager" />
<hz:hazelcast id="hazelcast">
...
</hz:hazelcast>
<bean id="cacheManager" class="com.hazelcast.spring.cache.HazelcastCacheManager">
<constructor-arg ref="instance"/>
</bean>
Annotation Based Configuration does not require any XML definition.
CachingConfiguration
class with related Annotations.@Configuration
@EnableCaching
public class CachingConfiguration implements CachingConfigurer{
@Bean
public CacheManager cacheManager() {
ClientConfig config = new ClientConfig();
HazelcastInstance client = HazelcastClient.newHazelcastClient(config);
return new HazelcastCacheManager(client);
}
@Bean
public KeyGenerator keyGenerator() {
return null;
}
CachingConfiguration
.AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(CachingConfiguration.class);
context.refresh();
For more information about Spring Cache, please see Spring Cache Abstraction.
Please see our sample application for Hibernate 2nd Level Cache Config.
If you are using Hibernate with Hazelcast as 2nd level cache provider, you can easily create RegionFactory
instances within Spring configuration (by Spring version 3.1). That way, it is possible to use same HazelcastInstance
as Hibernate L2 cache instance.
<hz:hibernate-region-factory id="regionFactory" instance-ref="instance"
mode="LOCAL" />
...
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.LocalSessionFactoryBean"
scope="singleton">
<property name="dataSource" ref="dataSource"/>
<property name="cacheRegionFactory" ref="regionFactory" />
...
</bean>
Hibernate RegionFactory Modes
Please refer to Hibernate RegionFactory Options section for more information.
Spring tries to create a new Map
/Collection
instance and fill the new instance by iterating and converting values of the original Map
/Collection
(IMap
, IQueue
, etc.) to required types when generic type parameters of the original Map
/Collection
and the target property/attribute do not match.
Since Hazelcast Map
s/Collection
s are designed to hold very large data which a single machine cannot carry, iterating through whole values can cause out of memory errors.
To avoid this issue, either target property/attribute can be declared as un-typed Map
/Collection
as shown below.
public class SomeBean {
@Autowired
IMap map; // instead of IMap<K, V> map
@Autowired
IQueue queue; // instead of IQueue<E> queue
...
}
Or, parameters of injection methods (constructor, setter) can be un-typed as shown below.
public class SomeBean {
IMap<K, V> map;
IQueue<E> queue;
// Instead of IMap<K, V> map
public SomeBean(IMap map) {
this.map = map;
}
...
// Instead of IQueue<E> queue
public void setQueue(IQueue queue) {
this.queue = queue;
}
...
}
RELATED INFORMATION
For more information please see Spring issue-3407.
  Elastic Memory is Hazelcast with off-heap (direct) memory storage to avoid GC pauses. Even if you have terabytes of cache in-memory with lots of updates, GC will have almost no effect; resulting in more predictable latency and throughput.
Here are the steps to enable Elastic Memory:
Set the maximum direct memory JVM can allocate, e.g. java -XX:MaxDirectMemorySize=60G
Enable Elastic Memory by setting hazelcast.elastic.memory.enabled
Hazelcast configuration property to true.
Set the total direct memory size for HazelcastInstance by setting hazelcast.elastic.memory.total.size
Hazelcast configuration property. Size can be in MB or GB and abbreviation can be used, such as 60G and 500M.
Set the chunk size by setting hazelcast.elastic.memory.chunk.size
Hazelcast configuration property. Hazelcast will partition the entire off-heap memory into chunks. Default chunk size is 1K.
You can enable sun.misc.Unsafe
based off-heap storage implementation instead of java.nio.DirectByteBuffer
based one, by setting hazelcast.elastic.memory.unsafe.enabled property
to true. Default value is false.
Configure maps that will use Elastic Memory by setting InMemoryFormat
to OFFHEAP. Default value is BINARY.
Below is the declarative configuration.
<hazelcast>
...
<map name="default">
...
<in-memory-format>OFFHEAP</in-memory-format>
</map>
</hazelcast>
And, the programmatic configuration:
MapConfig mapConfig = new MapConfig();
mapConfig.setInMemoryFormat( InMemoryFormat.OFFHEAP );
There are currently three ways to connect to a running Hazelcast cluster:
Native Clients enable you to perform almost all Hazelcast operations without being a member of the cluster. It connects to one of the cluster members and delegates all cluster wide operations to it (dummy client) or connects to all of them and delegate operations smartly (smart client). When the relied cluster member dies, client will transparently switch to another live member.
There can be hundreds, even thousands of clients connected to the cluster. But, by default there are core count * 10 threads on the server side that will handle all the requests (e.g. if the server has 4 cores, it will be 40).
Imagine a trading application where all the trading data stored and managed in a Hazelcast cluster with tens of nodes. Swing/Web applications at traders' desktops can use Native Clients to access and modify the data in the Hazelcast cluster.
Currently, Hazelcast has Native Java, C++ and .NET Clients available.
Java client is the most full featured client. It is offered both with Hazelcast and Hazelcast Enterprise. Main idea behind the Java client is to provide the same Hazelcast functionality by proxying each operation through a Hazelcast node. It can be used to access and change distributed data or listen distributed events of an already established Hazelcast cluster from another Java application.
There are two dependencies that you should include in your classpath to start using Hazelcast client: hazelcast.jar
and hazelcast-client.jar
.
After adding these dependencies, you can start using Hazelcast client as if you are using Hazelcast API. The differences will be extracted throughout the below sections.
If you prefer to use maven, simply add below lines to your pom.xml
.
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-client</artifactId>
<version>$LATEST_VERSION$</version>
</dependency>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast</artifactId>
<version>$LATEST_VERSION$</version>
</dependency>
First step is configuration. Java client can be configured declaratively or programmatically. We will use the programmatic approach throughout this tutorial. Please refer to Java Client Declarative Configuration for details.
ClientConfig clientConfig = new ClientConfig();
clientConfig.getGroupConfig().setName("dev").setPassword("dev-pass");
clientConfig.getNetworkConfig().addAddress("<IP_ADDRESS>", "<IP_ADDRESS>:5702");
Second step is the initialization of HazelcastInstance to be connected to the cluster.
HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);
This client interface is your gateway to access all Hazelcast distributed objects.
Let's create a map and populate with some data;
IMap<String, Customer> mapCustomers = client.getMap("customers");//creates the map proxy
mapCustomers.put("1", new Customer("Joe", "Smith"));
mapCustomers.put("2", new Customer("Ali", "Selam"));
mapCustomers.put("3", new Customer("Avi", "Noyan"));
As a final step, if you are done with your client, you can just shut it down as shown below. This will release all used resources and close connections to cluster.
client.shutdown();
Client has two operation modes because of the distributed nature of the data and cluster.
In this mode, clients connect to each cluster node. As each data partition uses the well known consistent hashing algorithm, each client can send operation to the relevant cluster node which will increase the overall throughput and efficiency. This mode is the default.
For some cases, the clients can be required to connect to a single node instead of each node in the cluster. Firewalls, security or some custom networking issues can be the reason for that.
In this mode, client will only connect to one of the configured addresses. This single node will behave as a gateway to other nodes. For any operation requested from the client, it will redirect the request to the relevant node and return the response back to the client returned from this node.
There are two main failure cases to be aware of and configured to achieve a proper behavior.
While client is trying to connect initially to one of the members in the ClientNetworkConfig.addressList
, all members might be not available. Instead of giving up, throwing an exception and stopping the client, it will attempt to retry as much as connectionAttemptLimit
times. Please see Connection Attempt Limit.
Client executes each operation through the already established connection to cluster. If this connection(s) disconnects or drops, client will try to reconnect as configured.
While sending the requests to related nodes, it is possible that operation fails due to various reasons. For any read only operations, you can have your client retrying to send the operation by enabling redoOperation
. Please see Redo Operation.
And, the number of retries is given with the property hazelcast.client.request.retry.count
in ClientProperties
. It will resend the request as many as RETRY-COUNT then it will throw an exception. Please see Client Properties.
Most of the Distributed Data Structures are supported by the client. Please check for the exceptions for the clients in other languages.
As a general rule, these data structures are configured on the server side and simply accessed through a proxy on the client side.
You can use any Distributed Map object with client as shown below.
Imap<Integer, String> map = client.getMap(“myMap”);
map.put(1, “Ali”);
String value= map.get(1);
map.remove(1);
As Locality is ambiguous for the client, addEntryListener
and localKeySet
are not supported. Please see Distributed Map for more information.
A sample usage is shown below.
MultiMap<Integer, String> multiMap = client.getMultiMap("myMultiMap");
multiMap.put(1,”ali”);
multiMap.put(1,”veli”);
Collection<String> values = multiMap.get(1);
As Locality is ambiguous for the client, addEntryListener
, localKeySet
and getLocalMultiMapStats
are not supported. Please see Distributed MultiMap for more information.
A sample usage is shown below.
IQueue<String> myQueue = client.getQueue(“theQueue”);
myQueue.offer(“ali”)
getLocalQueueStats
is not supported as locality is ambiguous for the client. Please see Distributed Queue for more information.
getLocalTopicStats
is not supported as locality is ambiguous for the client.
Below distributed data structures are also supported by the client. Since their logic is the same in both the node side and client side, you can refer to their sections as listed below.
Below services are provided for some common functionalities on the client side.
This service is for distributed computing. It can be used to execute tasks on the cluster on designated partition or even on all partitions. It can also be used to process entries. Please see Distributed Executor Service for more information.
IExecutorService executorService = client.getExecutorService("default");
After getting an instance of IExecutorService
, it can be used as the interface with the one provided on the server side. Please see Distributed Computing chapter for detailed usage.
NOTE: This service is only supported by the Java client.
If you need to track clients and want to listen their connection events, see the below code.
final ClientService clientService = client.getClientService();
final Collection<Client> connectedClients = clientService.getConnectedClients();
clientService.addClientListener(new ClientListener() {
@Override
public void clientConnected(Client client) {
//Handle client connected event
}
@Override
public void clientDisconnected(Client client) {
//Handle client disconnected event
}
});
Partition service is used to find the partition of a key. It will return all partitions. See the below code.
PartitionService partitionService = client.getPartitionService();
//partition of a key
Partition partition = partitionService.getPartition(key);
//all partitions
Set<Partition> partitions = partitionService.getPartitions();
Lifecycle handling is to;
LifecycleService lifecycleService = client.getLifecycleService();
if(lifecycleService.isRunning()){
//it is running
}
//shutdown client gracefully
lifecycleService.shutdown();
Listeners can be configured to listen to various event types on the client side. Global events not relating to any distributed object can be configured through ListenerConfig. Whereas, distributed object listeners like map entry listeners or list item listeners should be configured through their proxies. You can refer to the related sections under each distributed data structure in this reference manual.
Transactional distributed objects are supported on the client side. Please see Transactions chapter on how to use them.
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = clientConfig.getNetworkConfig();
Address List is the initial list of cluster addresses to which the client will connect. Client uses this list to find an alive node. Although it may be enough to give only one address of a node in the cluster (since all nodes communicate with each other), it is recommended to give all nodes’ addresses.
If the port part is omitted then 5701, 5702, and 5703 will be tried in a random order.
clientConfig.getNetworkConfig().addAddress("<IP_ADDRESS>", "<IP_ADDRESS>:5703");
You can provide multiple addresses with ports provided or not as seen above. The provided list is shuffled to try them in a random order.
Default value is localhost.
This parameter defines whether the client is smart or a dummy one.
//sets client to dummy client mode
clientConfig.getNetworkConfig().setSmartRouting(false);
Default is smart client mode.
Enables/disables redo-able operations as described in Retry-able Operation Failure.
//enables redo
clientConfig.getNetworkConfig().setRedoOperation(true);
Default is disabled.
Timeout value in milliseconds for nodes to accept client connection requests.
//enables redo
clientConfig.getNetworkConfig().setConnectionTimeout(1000);
Default value is 5000 milliseconds.
While client is trying to connect initially to one of the members in the ClientNetworkConfig.addressList
, all might be not available. Instead of giving up, throwing exception and stopping the client, it will attempt to retry as much as ClientNetworkConfig.connectionAttemptLimit
times.
//enables redo
clientConfig.getNetworkConfig().setConnectionAttemptLimit(5);
Default value is 2.
The duration in milliseconds between connection attempts defined by ClientNetworkConfig.connectionAttemptLimit
.
//enables redo
clientConfig.getNetworkConfig().setConnectionAttemptPeriod(5000);
Default value is 3000.
Client configuration to set a socket intercepter. Any class implementing com.hazelcast.nio.SocketInterceptor
is a socket Interceptor.
public interface SocketInterceptor {
void init(Properties properties);
void onConnect(Socket connectedSocket) throws IOException;
}
Socket interceptor has two steps. First, it will be initialized by the configured properties. Second, it will be informed just after socket is connected using onConnect
.
SocketInterceptorConfig socketInterceptorConfig = clientConfig
.getNetworkConfig().getSocketInterceptorConfig();
MyClientSocketInterceptor myClientSocketInterceptor = new MyClientSocketInterceptor();
socketInterceptorConfig.setEnabled(true);
socketInterceptorConfig.setImplementation(myClientSocketInterceptor);
If you want to configure it with a class name instead of an instance;
SocketInterceptorConfig socketInterceptorConfig = clientConfig
.getNetworkConfig().getSocketInterceptorConfig();
MyClientSocketInterceptor myClientSocketInterceptor = new MyClientSocketInterceptor();
socketInterceptorConfig.setEnabled(true);
//These properties are provided to interceptor during init
socketInterceptorConfig.setProperty("kerberos-host","kerb-host-name");
socketInterceptorConfig.setProperty("kerberos-config-file","kerb.conf");
socketInterceptorConfig.setClassName(myClientSocketInterceptor);
Please see Socket Interceptor section for more information.
Network socket options can be configured using SocketOptions
. It has below methods.
socketOptions.setKeepAlive(x)
: Enables/disables SO_KEEPALIVE socket option. Default is true
.
socketOptions.setTcpNoDelay(x)
: Enables/disables TCP_NODELAY socket option. Default is true
.
socketOptions.setReuseAddress(x)
: Enables/disables SO_REUSEADDR socket option. Default is true
.
socketOptions.setLingerSeconds(x)
: Enables/disables SO_LINGER with the specified linger time in seconds. Default is 3
.
socketOptions.setBufferSize(x)
: Sets the SO_SNDBUF and SO_RCVBUF options to the specified value in KB for this Socket. Default is 32
.
SocketOptions socketOptions = clientConfig.getNetworkConfig().getSocketOptions();
socketOptions.setBufferSize(32);
socketOptions.setKeepAlive(true);
socketOptions.setTcpNoDelay(true);
socketOptions.setReuseAddress(true);
socketOptions.setLingerSeconds(3);
SSL can be used to secure the connection between client and the nodes. Please see SSLConfig section on how to configure it.
Below sample declarative and programmatic configurations show how to configure a Java client for connecting to a Hazelcast cluster in AWS.
Declarative Configuration:
<aws enabled="true">
<!-- optional default value is false -->
<inside-aws>false</inside-aws>
<access-key>my-access-key</access-key>
<secret-key>my-secret-key</secret-key>
<!-- optional, default is us-east-1 -->
<region>us-west-1</region>
<!-- optional, default is ec2.amazonaws.com. If set, region shouldn't be set
as it will override this property
-->
<host-header>ec2.amazonaws.com</host-header>
<!-- optional -->
<security-group-name>hazelcast-sg</security-group-name>
<!-- optional -->
<tag-key>type</tag-key>
<!-- optional -->
<tag-value>hz-nodes</tag-value>
</aws>
Programmatic Configuration:
ClientConfig clientConfig = new ClientConfig();
ClientAwsConfig clientAwsConfig = new ClientAwsConfig();
clientAwsConfig.setInsideAws( false )
.setAccessKey( "my-access-key" )
.setSecretKey( "my-secret-key" )
.setRegion( "us-west-1" )
.setHostHeader( "ec2.amazonaws.com" )
.setSecurityGroupName( ">hazelcast-sg" )
.setTagKey( "type" )
.setTagValue( "hz-nodes" );
clientConfig.getNetworkConfig().setAwsConfig( clientAwsConfig );
HazelcastInstance client = HazelcastClient.newHazelcastClient( clientConfig );
NOTE: If inside-aws
parameter is not set, private addresses of nodes will always be converted to public addresses. And, client will use public addresses to connect to nodes. In order to use private addresses, you should set it to true
. Also note that, when connecting outside from AWS, setting inside-aws
parameter to true
will cause the client not to be able to reach to the nodes.
LoadBalancer
allows you to send operations to one of a number of endpoints(Members). Its main purpose is to determine the next Member
if queried. It is up to the implementation to use different load balancing policies. The interface com.hazelcast.client.LoadBalancer
should be implemented for that purpose.
If the client is configured as a smart one, only the operations that are not key based will be routed to the endpoint returned by the LoadBalancer. If it is not a smart client, LoadBalancer
will be ignored.
For the configuration of a client load balancer, please see Load Balancer Config and Java Client Declarative Configuration.
```
Hazelcast distributed map has a Near Cache feature to reduce network latencies. As the client always requests data from the cluster nodes, it can be helpful for some use cases to configure a near cache on the client side. The client supports the exact same near cache used in Hazelcast distributed map.
If SSL is desired to be enabled for the client-cluster connection, this parameter should be set. Once set, the connection (socket) is established out of an SSL factory defined either by a factory class name or factory implementation. Please see SSLConfig class in com.hazelcast.config
package at the JavaDocs page of Hazelcast Documentation web page.
Hazelcast Java client can be configured in two ways, declaratively or programmatically.
Java client can be configured using an XML configuration file. Below is a generic template of a declarative configuration.
<hazelcast-client xsi:schemaLocation=
"http://www.hazelcast.com/schema/client-config hazelcast-client-config-3.3.xsd"
xmlns="http://www.hazelcast.com/schema/client-config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<!--Cluster name to connect-->
<group>
<name>GROUP_NAME</name>
<password>GROUP_PASS</password>
</group>
<!--client properties-->
<properties>
<property name="hazelcast.client.connection.timeout">10000</property>
<property name="hazelcast.client.retry.count">6</property>
</properties>
<!--Network configuration details-->
<network>
<cluster-members>
<!--initial cluster members to connect-->
<address><IP_ADDRESS></address>
<address><IP_ADDRESS></address>
</cluster-members>
<smart-routing>true</smart-routing>
<redo-operation>true</redo-operation>
<socket-interceptor enabled="true">
<!--socket-interceptor configuration details-->
</socket-interceptor>
<aws enabled="true" connection-timeout-seconds="11">
<!--AWS configuration details-->
</aws>
</network>
<!--local executor pool size-->
<executor-pool-size>40</executor-pool-size>
<!--security credentials configuration-->
<security>
<credentials>com.hazelcast.security.UsernamePasswordCredentials</credentials>
</security>
<listeners>
<!--listeners-->
</listeners>
<serialization>
<!--serialization configuration details-->
</serialization>
<proxy-factories>
<!--ProxyFactory configuration details-->
</proxy-factories>
<!--load balancer configuration-->
<!-- type can be "round-robin" or "random" -->
<load-balancer type="random"/>
<near-cache name="mapName">
<!--near cache configuration details of a map-->
</near-cache>
</hazelcast-client>
Using the configuration API, a ClientConfig
configured as required. Please refer to the related sections and JavaDocs for more information.
ClientNetworkConfig includes the below listed configuration options, which are explained under Network Configuration Options section.
Clients should provide group name and password in order to connect to the cluster.
It can be configured using GroupConfig
, as shown below.
clientConfig.setGroupConfig(new GroupConfig("dev","dev-pass"));
The following code snippet shows the programmatic configuration of the load balancer.
clientConfig.setLoadBalancer(yourLoadBalancer);
In the cases where the security established with GroupConfig
is not enough and you want your clients connecting securely to the cluster, ClientSecurityConfig
can be used. This configuration has a credentials
parameter with which IP address and UID are set. Please see ClientSecurityConfig.java
in our code.
For the client side serialization, Hazelcast configuration is used. Please refer to Serialization chapter.
Global event listeners can be configured using ListenerConfig
as shown below.
ClientConfig clientConfig = new ClientConfig();
ListenerConfig listenerConfig = new ListenerConfig(LifecycleListenerImpl);
clientConfig.addListenerConfig(listenerConfig);
ClientConfig clientConfig = new ClientConfig();
ListenerConfig listenerConfig = new ListenerConfig("com.hazelcast.example.MembershipListenerImpl");
clientConfig.addListenerConfig(listenerConfig);
There are three types of event listeners that can be added.
RELATED INFORMATION
Please refer to Hazelcast JavaDocs and see LifecycleListener, MembershipListener and DistributedObjectListener in com.hazelcast.core
package.
A near cache on the client side can be configured by providing a configuration per map name, as shown below.
ClientConfig clientConfig = new ClientConfig();
CacheConfig nearCacheConfig = new NearCacheConfig();
nearCacheConfig.setName("mapName");
clientConfig.addNearCacheConfig(nearCacheConfig);
Wildcards can be used for the map name. See below samples.
nearCacheConfig.setName("map*");
nearCacheConfig.setName("*map");
A custom classLoader
can be configured. It will be used by serialization service and to load any class configured in configuration like event listeners or ProxyFactories.
Hazelcast has an internal executor service (different from the data structure Executor Service) that has threads and queues to perform internal operations such as handling responses. This parameter specifies the size of the pool of threads which perform these operations laying in the executor's queue. If not configured, this parameter has the value as 5 * core size of the client (i.e. it is 20 for a machine that has 4 cores).
There are some advanced client configuration properties to tune some aspects of Hazelcast Client. These can be set as property name and value pairs through declarative configuration, programmatic configuration or JVM system property. Please see Advanced Configuration Properties section to learn how to set these properties.
Below table lists the client configuration properties with their descriptions.
| Property Name | Default Value | Type | Description |
|---|---|---|---|
hazelcast.client.heartbeat.timeout |
300000 | string | Timeout for the heartbeat messages sent by the client to members. If there is no any message passing between client and member within the given time via this property in milliseconds the connection will be closed. |
hazelcast.client.heartbeat.interval |
10000 | string | The frequency of heartbeat messages sent by the clients to members. |
hazelcast.client.max.failed.heartbeat.count |
3 | string | When the count of failed heartbeats sent to members reaches this value, the cluster is deemed as dead by the client. |
hazelcast.client.request.retry.count |
20 | string | The retry count of the connection requests by the client to the members. |
hazelcast.client.request.retry.wait.time |
250 | string | The frequency of the connection retries. |
hazelcast.client.event.thread.count |
5 | string | Thread count for handling incoming event packets. |
hazelcast.client.event.queue.capacity |
1000000 | string | Default value of the capacity of executor that handles incoming event packets. |
Please refer to Client Code Samples.
You can use Native C++ Client to connect to Hazelcast nodes and perform almost all operations that a node can perform. Different from nodes, clients do not hold data. It is by default a smart client, i.e. it knows where the data is and asks directly to the correct node. This feature can be disabled (using ClientConfig::setSmart
method) if you do not want the clients to connect every node.
Features of C++ Clients are:
Hazelcast C++ Client is shipped with 32/64 bit, shared and static libraries. Compiled static libraries of dependencies are also available in the release. Dependencies are zlib and shared_ptr from the boost libraries.
Downloaded release folder consists of:
And each of the folders above contains the following:
examples/
hazelcast/
external/
C++ Client is tested on Linux 32/64, Mac 64 and Windows 32/64 bit machines. For each of the headers above, it is assumed that you are in the correct folder for your platform. Folders are Mac_64, Windows_32, Windows_64, Linux_32 or Linux_64.
For Linux, there are two distributions; 32 bit and 64 bit.
Sample script to build with static library:
g++ main.cpp -pthread -I./external/include -I./hazelcast/include
./hazelcast/lib/libHazelcastClientStatic_64.a
./external/lib/libz.a
Sample script to build with shared library:
g++ main.cpp -lpthread -Wl,–no-as-needed -lrt -I./external/include -I./hazelcast/include -L./hazelcast/lib -lHazelcastClientShared_64 ./external/lib/libz.a
For Mac, there is only one distribution which is 64 bit.
Sample script to build with static library:
g++ main.cpp -I./external/include -I./hazelcast/include ./hazelcast/lib/libHazelcastClientStatic_64.a ./external/lib/darwin/libz.a
Sample script to build with shared library:
g++ main.cpp -I./external/include -I./hazelcast/include -L./hazelcast/lib -lHazelcastClientShared_64 ./external/lib/darwin/libz.a
For Windows, there are two distributions; 32 bit and 64 bit. Current release have only Visual Studio 2010 compatible libraries. For others, please contact with<EMAIL_ADDRESS>A Hazelcast node should be running to make below sample codes work.
#include <hazelcast/client/HazelcastAll.h>
#include <iostream>
using namespace hazelcast::client;
int main() {
ClientConfig clientConfig;
Address address( "localhost", 5701 );
clientConfig.addAddress( address );
HazelcastClient hazelcastClient( clientConfig );
IMap<int,int> myMap = hazelcastClient.getMap<int ,int>( "myIntMap" );
myMap.put( 1,3 );
boost::shared_ptr<int> value = myMap.get( 1 );
if( value.get() != NULL ) {
//process the item
}
return 0;
}
#include <hazelcast/client/HazelcastAll.h>
#include <iostream>
#include <string>
using namespace hazelcast::client;
int main() {
ClientConfig clientConfig;
Address address( "localhost", 5701 );
clientConfig.addAddress( address );
HazelcastClient hazelcastClient( clientConfig );
IQueue<std::string> queue = hazelcastClient.getQueue<std::string>( "q" );
queue.offer( "sample" );
boost::shared_ptr<std::string> value = queue.poll();
if( value.get() != NULL ) {
//process the item
}
return 0;
}
#include "hazelcast/client/ClientConfig.h"
#include "hazelcast/client/EntryEvent.h"
#include "hazelcast/client/IMap.h"
#include "hazelcast/client/Address.h"
#include "hazelcast/client/HazelcastClient.h"
#include <iostream>
#include <string>
using namespace hazelcast::client;
class SampleEntryListener {
public:
void entryAdded( EntryEvent<std::string, std::string> &event ) {
std::cout << "entry added " << event.getKey() << " "
<< event.getValue() << std::endl;
};
void entryRemoved( EntryEvent<std::string, std::string> &event ) {
std::cout << "entry added " << event.getKey() << " "
<< event.getValue() << std::endl;
}
void entryUpdated( EntryEvent<std::string, std::string> &event ) {
std::cout << "entry added " << event.getKey() << " "
<< event.getValue() << std::endl;
}
void entryEvicted( EntryEvent<std::string, std::string> &event ) {
std::cout << "entry added " << event.getKey() << " "
<< event.getValue() << std::endl;
}
};
int main( int argc, char **argv ) {
ClientConfig clientConfig;
Address address( "localhost", 5701 );
clientConfig.addAddress( address );
HazelcastClient hazelcastClient( clientConfig );
IMap<std::string,std::string> myMap = hazelcastClient
.getMap<std::string ,std::string>( "myIntMap" );
SampleEntryListener * listener = new SampleEntryListener();
std::string id = myMap.addEntryListener( *listener, true );
// Prints entryAdded
myMap.put( "key1", "value1" );
// Prints updated
myMap.put( "key1", "value2" );
// Prints entryRemoved
myMap.remove( "key1" );
// Prints entryEvicted after 1 second
myMap.put( "key2", "value2", 1000 );
// WARNING: deleting listener before removing it from hazelcast leads to crashes.
myMap.removeEntryListener( id );
// Delete listener after remove it from hazelcast.
delete listener;
return 0;
};
Assume that you have the following two classes in Java and you want to use it with C++ client.
class Foo implements Serializable {
private int age;
private String name;
}
class Bar implements Serializable {
private float x;
private float y;
}
First, let them implement Portable
or IdentifiedDataSerializable
as shown below.
class Foo implements Portable {
private int age;
private String name;
public int getFactoryId() {
// a positive id that you choose
return 123;
}
public int getClassId() {
// a positive id that you choose
return 2;
}
public void writePortable( PortableWriter writer ) throws IOException {
writer.writeUTF( "n", name );
writer.writeInt( "a", age );
}
public void readPortable( PortableReader reader ) throws IOException {
name = reader.readUTF( "n" );
age = reader.readInt( "a" );
}
}
class Bar implements IdentifiedDataSerializable {
private float x;
private float y;
public int getFactoryId() {
// a positive id that you choose
return 4;
}
public int getId() {
// a positive id that you choose
return 5;
}
public void writeData( ObjectDataOutput out ) throws IOException {
out.writeFloat( x );
out.writeFloat( y );
}
public void readData( ObjectDataInput in ) throws IOException {
x = in.readFloat();
y = in.readFloat();
}
}
Then, implement the corresponding classes in C++ with same factory and class ID as shown below:
class Foo : public Portable {
public:
int getFactoryId() const {
return 123;
};
int getClassId() const {
return 2;
};
void writePortable( serialization::PortableWriter &writer ) const {
writer.writeUTF( "n", name );
writer.writeInt( "a", age );
};
void readPortable( serialization::PortableReader &reader ) {
name = reader.readUTF( "n" );
age = reader.readInt( "a" );
};
private:
int age;
std::string name;
};
class Bar : public IdentifiedDataSerializable {
public:
int getFactoryId() const {
return 4;
};
int getClassId() const {
return 2;
};
void writeData( serialization::ObjectDataOutput& out ) const {
out.writeFloat(x);
out.writeFloat(y);
};
void readData( serialization::ObjectDataInput& in ) {
x = in.readFloat();
y = in.readFloat();
};
private:
float x;
float y;
};
Now, you can use class Foo
and Bar
in distributed structures. For example as Key or Value of IMap
or as an Item in IQueue
.
You can use native .NET client to connect to Hazelcast nodes. All you need is to add HazelcastClient3x.dll
into your .NET project references. The API is very similar to Java native client.
.NET Client has following distributed objects:
IMap<K,V>
IMultiMap<K,V>
IQueue<E>
ITopic<E>
IHList<E>
IHSet<E>
IIdGenerator
ILock
ISemaphore
ICountDownLatch
IAtomicLong
ITransactionContext
ITransactionContext can be used to obtain;
ITransactionalMap<K,V>
ITransactionalMultiMap<K,V>
ITransactionalList<E>
ITransactionalSet<E>
Sample code is shown below.
using Hazelcast.Config;
using Hazelcast.Client;
using Hazelcast.Core;
using Hazelcast.IO.Serialization;
using System.Collections.Generic;
namespace Hazelcast.Client.Example
{
public class SimpleExample
{
public static void Test()
{
var clientConfig = new ClientConfig();
clientConfig.GetNetworkConfig().AddAddress( "<IP_ADDRESS>" );
clientConfig.GetNetworkConfig().AddAddress( "<IP_ADDRESS>:5702" );
// Portable Serialization setup up for Customer Class
clientConfig.GetSerializationConfig()
.AddPortableFactory( MyPortableFactory.FactoryId, new MyPortableFactory() );
IHazelcastInstance client = HazelcastClient.NewHazelcastClient( clientConfig );
// All cluster operations that you can do with ordinary HazelcastInstance
IMap<string, Customer> mapCustomers = client.GetMap<string, Customer>( "customers" );
mapCustomers.Put( "1", new Customer( "Joe", "Smith" ) );
mapCustomers.Put( "2", new Customer( "Ali", "Selam" ) );
mapCustomers.Put( "3", new Customer( "Avi", "Noyan" ) );
ICollection<Customer> customers = mapCustomers.Values();
foreach (var customer in customers)
{
//process customer
}
}
}
public class MyPortableFactory : IPortableFactory
{
public const int FactoryId = 1;
public IPortable Create( int classId ) {
if ( Customer.Id == classId )
return new Customer();
else
return null;
}
}
public class Customer : IPortable
{
private string name;
private string surname;
public const int Id = 5;
public Customer( string name, string surname )
{
this.name = name;
this.surname = surname;
}
public Customer() {}
public int GetFactoryId()
{
return MyPortableFactory.FactoryId;
}
public int GetClassId()
{
return Id;
}
public void WritePortable( IPortableWriter writer )
{
writer.WriteUTF( "n", name );
writer.WriteUTF( "s", surname );
}
public void ReadPortable( IPortableReader reader )
{
name = reader.ReadUTF( "n" );
surname = reader.ReadUTF( "s" );
}
}
}
Hazelcast .NET client can be configured via API or XML. To start the client, a configuration can be passed or can be left empty to use default values.
NOTE: .NET and Java clients are similar in terms of configuration. Therefore, you can refer to Java Client section for configuration aspects. For information on .NET API documentation, please refer to the API document provided along with the Hazelcast Enterprise license.
After configuration, one can obtain a client using one of the static methods of Hazelcast like as shown below.
IHazelcastInstance client = HazelcastClient.NewHazelcastClient(clientConfig);
...
IHazelcastInstance defaultClient = HazelcastClient.NewHazelcastClient();
...
IHazelcastInstance xmlConfClient = Hazelcast
.NewHazelcastClient(@"..\Hazelcast.Net\Resources\hazelcast-client.xml");
IHazelcastInstance interface is the starting point where all distributed objects can be obtained using it.
var map = client.GetMap<int,string>("mapName");
...
var lock= client.GetLock("thelock");
Hazelcast provides REST interface, i.e. it provides an HTTP service in each node so that your map
and queue
can be accessed using HTTP protocol. Assuming mapName
and queueName
are already configured in your Hazelcast, its structure is shown below:
http://node IP address:port/hazelcast/rest/maps/mapName/key
http://node IP address:port/hazelcast/rest/queues/queueName
For the operations to be performed, standard REST conventions for HTTP calls are used.
Assume that your cluster members are as shown below.
Members [5] {
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
}
NOTE: All of the requests below can return one of the following two in case of a failure
HTTP/1.1 400 Bad Request
Content-Length: 0
< HTTP/1.1 500 Internal Server Error
< Content-Length: 0
Creating/Updating Entries in a Map
You can put a new key1/value1
entry into a map by using POST call to
http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName/key1
URL. This call's content body should contain the value of the key. Also, if the call contains the MIME type, Hazelcast stores this information, too.
A sample POST call is shown below.
$ curl -v -X POST -H "Content-Type: text/plain" -d "bar"
http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName/foo
It will return the following if successful:
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
Retrieving Entries from a Map
If you want to retrieve an entry, you can use GET call to http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName/key1
. You can also retrieve this entry from another member of your cluster such as
http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName/key1
.
A sample GET call is shown below.
$ curl -X GET http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName/foo
It will return the following if there is a corresponding value:
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 3
bar
As you can see, GET call returned value, its length and also the MIME type (text/plain
) since POST call sample shown above included the MIME type.
It will return the following if there is no mapping for the given key:
< HTTP/1.1 204 No Content
< Content-Length: 0
Removing Entries from a Map
You can use DELETE call to remove an entry. A sample DELETE call is shown below with its returns.
$ curl -v -X DELETE http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName/foo
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
If you leave key empty as follows, it will delete all entries from map.
$ curl -v -X DELETE http://<IP_ADDRESS>:5701/hazelcast/rest/maps/mapName
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
Offering Items on a Queue
You can use POST call to create an item on the queue. A sample is shown below.
$ curl -v -X POST -H "Content-Type: text/plain" -d "foo"
http://<IP_ADDRESS>:5701/hazelcast/rest/queues/myEvents
Above call is equivalent to HazelcastInstance#getQueue("myEvents").offer("foo");
.
It will return the following if successful:
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
It will return the following if queue is full and item is not be able to offered to queue:
< HTTP/1.1 503 Service Unavailable
< Content-Length: 0
Retrieving Items from a Queue
DELETE call can be used for retrieving. Note that, poll timeout should be stated while polling for queue events by an extra path parameter.
A sample is shown below (10 being the timeout value).
$ curl -v -X DELETE \http://<IP_ADDRESS>:5701/hazelcast/rest/queues/myEvents/10
Above call is equivalent to HazelcastInstance#getQueue("myEvents").poll(10, SECONDS);
. Below is the returns of above call.
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 3
foo
When the timeout is reached, the return will be No Content
success, i.e. there is no item on the queue to be returned.
< HTTP/1.1 204 No Content
< Content-Length: 0
Getting the size of the queue
$ curl -v -X GET \http://<IP_ADDRESS>:5701/hazelcast/rest/queues/myEvents/size
Above call is equivalent to HazelcastInstance#getQueue("myEvents").size();
. Below is a sample return of above call.
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 1
5
Besides the above operations, you can check the status of your cluster, a sample of which is shown below.
$ curl -v http://<IP_ADDRESS>:5701/hazelcast/rest/cluster
The return will be similar to the following:
< HTTP/1.1 200 OK
< Content-Length: 119
Members [5] {
Member [<IP_ADDRESS>:5701] this
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
}
ConnectionCount: 5
AllConnectionCount: 20
RESTful access is provided through any member of your cluster.
NOTE: You need to handle the failures on REST polls as there is no transactional guarantee.
NOTE: Hazelcast Memcache Client only supports ASCII protocol. Binary Protocol is not supported.
A Memcache client written in any language can talk directly to Hazelcast cluster. No additional configuration is required. Assume that your cluster members are as below.
Members [5] {
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
Member [<IP_ADDRESS>:5701]
}
And you have a PHP application that uses PHP Memcache client to cache things in Hazelcast. All you need to do is have your PHP Memcache client connect to one of these members. It does not matter which member the client connects to because Hazelcast cluster looks as one giant machine (Single System Image). PHP client code sample:
<?php
$memcache = new Memcache;
$memcache->connect( '<IP_ADDRESS>', 5701 ) or die ( "Could not connect" );
$memcache->set( 'key1', 'value1', 0, 3600 );
$get_result = $memcache->get( 'key1' ); // retrieve your data
var_dump( $get_result ); // show it
?>
Notice that Memcache client is connecting to <IP_ADDRESS>
and using port5701
. Java client code sample with SpyMemcached client:
MemcachedClient client = new MemcachedClient(
AddrUtil.getAddresses( "<IP_ADDRESS>:5701 <IP_ADDRESS>:5701" ) );
client.set( "key1", 3600, "value1" );
System.out.println( client.get( "key1" ) );
If you want your data to be stored in different maps (e.g. to utilize per map configuration), you can do that with a map name prefix as following:
MemcachedClient client = new MemcachedClient(
AddrUtil.getAddresses( "<IP_ADDRESS>:5701 <IP_ADDRESS>:5701" ) );
client.set( "map1:key1", 3600, "value1" ); // store to *hz_memcache_map1
client.set( "map2:key1", 3600, "value1" ); // store to hz_memcache_map2
System.out.println( client.get( "key1" ) ); // get from hz_memcache_map1
System.out.println( client.get( "key2" ) ); // get from hz_memcache_map2
hz_memcache prefix_ is to separate Memcache maps from Hazelcast maps. If no map name is given, it will be stored in default map named as hz_memcache_default.
An entry written with a Memcache client can be read by another Memcache client written in another language.
CAS operations are not supported. In operations getting CAS parameters like append, CAS values are ignored.
Only a subset of statistics are supported. Below is the list of supported statistic values.
Java objects that you put into Hazelcast need to be serialized since Hazelcast is a distributed system. The data and its replicas are stored in different partitions on multiple nodes. The data you need may not be present on the local machine and Hazelcast retrieves that data from another machine. And this requires serialization.
Hazelcast serializes all your objects into an instance of com.hazelcast.nio.serialization.Data
. Data
is the binary representation of an object.
Serialization is used when;
Hazelcast optimizes the serialization for the below types, and the user cannot override this behavior:
Hazelcast also optimizes the following types. However, you can override them by creating a custom serializer and registering it. See Custom Serialization for more information.
As we have said, Hazelcast optimizes all of the above object types and you do not need to worry about their (de)serializations.
When it comes to complex objects, below interfaces are used for serialization and deserialization.
java.io.Serializable
java.io.Externalizable
com.hazelcast.nio.serialization.DataSerializable
com.hazelcast.nio.serialization.IdentifiedDataSerializable
com.hazelcast.nio.serialization.Portable
, and
StreamSerializer
, ByteArraySerializer
)When Hazelcast serializes an object into Data
:
(1) It first checks whether the object is an instance of com.hazelcast.nio.serialization.DataSerializable
,
(2) If above fails, then it checks if it is an instance of com.hazelcast.nio.serialization.Portable
,
(3) If above fails, then it checks whether the object is of a well-known type like String, Long, Integer, etc. and user specified types like ByteArraySerializer or StreamSerializer,
(4) If above checks fail, Hazelcast will use Java serialization.
If all of the above checks do not work, then serialization will fail. When a class implements multiple interfaces, above steps are important to determine the serialization mechanism that Hazelcast will use. And when a class definition is required for any of these serializations, all the classes needed by the application should be on the classpath, Hazelcast does not download them automatically.
Below table provides a comparison between the interfaces listed in the previous section to help you in deciding which interface to use in your applications.
| Serialization Interface | Advantages | Drawbacks |
|---|---|---|
| Serializable | - A standard and basic Java interface - No implementation is required |
- More time and CPU usage - More space occupancy |
| Externalizable | - A standard Java interface - More efficient than Serializable when it comes to CPU and memory usage |
- Serialization interface must be implemented |
| DataSerializable | - More efficient than Serializable when it comes to CPU and memory usage | - Specific to Hazelcast |
| IdentifiedDataSerializable | - More efficient than Serializable when it comes to CPU and memory usage - Reflection is not used during deserialization |
- Specific to Hazelcast - Serialization interface must be implemented - A Factory and configuration must be implemented |
| Portable | - More efficient than Serializable when it comes to CPU and memory usage - Reflection is not used during deserialization - Versioning is supported Partial deserialization is supported during Queries |
- Specific to Hazelcast - Serialization interface must be implemented - A Factory and configuration must be implemented - Class definition is also sent with data but stored only once per class |
| Custom Serialization | - Does not require class to implement an interface - Convenient and flexible - Can be based on StreamSerializer ByteArraySerializer |
- Serialization interface must be implemented - Plug in and configuration is required |
Let's dig into the details of the above serialization mechanisms in the following sections.
Often, a class needs to implement java.io.Serializable
interface and therefore native Java serialization is the easiest way for serialization. Let's take a look at the sample code below.
public class Employee implements Serializable {
private static final long serialVersionUID = 1L;
private String surname;
public Employee( String surname ) {
this.surname = surname;
}
}
Here, the fields that are non-static and non-transient are automatically serialized. To eliminate class compatibility issues, it is recommended to add a serialVersionUID
, as shown above. Also, when you are using methods that perform byte-content comparisons (e.g. IMap.replace()
) and if byte-content of equal objects is different, you may face with unexpected behaviors. So, if the class relies on, for example a hash map, replace
method may fail. The reason for this is the hash map, being a serialized data structure with unreliable byte-content.
Hazelcast also supports java.io.Externalizable
. This interface offers more control on the way how the fields are serialized or deserialized. Compared to native Java serialization, it also can have a positive effect on the performance. There is no need for a serialVersionUID
.
Let's take a look the sample code below.
public class Employee implements Externalizable {
private String surname;
public Employee(String surname) {
this.surname = surname;
}
@Override
public void readExternal( ObjectInput in )
throws IOException, ClassNotFoundException {
this.surname = in.readUTF();
}
@Override
public void writeExternal( ObjectOutput out )
throws IOException {
out.writeUTF(surname);
}
}
As can be seen, writing and reading of fields are written explicitly. Note that, reading should be performed in the same order as writing.
As mentioned in the previous section, Java serialization is an easy mechanism. However, we do not have a control on the way of serialization or deserialization of the fields. Moreover, this mechanism can lead to excessive CPU loads since it keep tracks of objects to handle the cycles and streams class descriptors. These are performance decreasing factors and thus, serialized data may not have an optimal size.
DataSerializable interface of Hazelcast overcomes these issues. Here is an example of a class implementing com.hazelcast.nio.serialization.DataSerializable
interface.
public class Address implements DataSerializable {
private String street;
private int zipCode;
private String city;
private String state;
public Address() {}
//getters setters..
public void writeData( ObjectDataOutput out ) throws IOException {
out.writeUTF(street);
out.writeInt(zipCode);
out.writeUTF(city);
out.writeUTF(state);
}
public void readData( ObjectDataInput in ) throws IOException {
street = in.readUTF();
zipCode = in.readInt();
city = in.readUTF();
state = in.readUTF();
}
}
Let's take a look at another example which is encapsulating a DataSerializable
field.
public class Employee implements DataSerializable {
private String firstName;
private String lastName;
private int age;
private double salary;
private Address address; //address itself is DataSerializable
public Employee() {}
//getters setters..
public void writeData( ObjectDataOutput out ) throws IOException {
out.writeUTF(firstName);
out.writeUTF(lastName);
out.writeInt(age);
out.writeDouble (salary);
address.writeData (out);
}
public void readData( ObjectDataInput in ) throws IOException {
firstName = in.readUTF();
lastName = in.readUTF();
age = in.readInt();
salary = in.readDouble();
address = new Address();
// since Address is DataSerializable let it read its own internal state
address.readData(in);
}
}
As you can see, since address
field itself is DataSerializable
, it is calling address.writeData(out)
when writing and address.readData(in)
when reading. Also note that, the order of writing and reading fields should be the same. While Hazelcast serializes a DataSerializable, it writes the className
first and when de-serializes it, className
is used to instantiate the object using reflection.
NOTE: Since Hazelcast needs to create an instance during deserialization,DataSerializable
class has a no-arg constructor.
NOTE: DataSerializable
is a good option if serialization is only needed for in-cluster communication.
For a faster serialization of objects, avoiding reflection and long class names, Hazelcast recommends to implement com.hazelcast.nio.serialization.IdentifiedDataSerializable
which is slightly better version of DataSerializable
.
DataSerializable
uses reflection to create a class instance, as mentioned in above section. But, IdentifiedDataSerializable
uses a factory for this purpose and it is faster during deserialization which requires new instance creations.
IdentifiedDataSerializable
extends DataSerializable
and introduces two new methods.
int getId();
int getFactoryId();
IdentifiedDataSerializable
uses getId()
instead of class name and uses getFactoryId()
to load the class given the Id. To complete the implementation, a com.hazelcast.nio.serialization.DataSerializableFactory
should also be implemented and registered into SerializationConfig
which can be accessed from Config.getSerializationConfig()
. Factory's responsibility is to return an instance of the right IdentifiedDataSerializable
object, given the Id. So far this is the most efficient way of Serialization that Hazelcast supports off the shelf.
Let's take a look at the below example codes and configuration to see IdentifiedDataSerializable
in action.
public class Employee
implements IdentifiedDataSerializable {
private String surname;
public Employee() {}
public Employee( String surname ) {
this.surname = surname;
}
@Override
public void readData( ObjectDataInput in )
throws IOException {
this.surname = in.readUTF();
}
@Override
public void writeData( ObjectDataOutput out )
throws IOException {
out.writeUTF( surname );
}
@Override
public int getFactoryId() {
return EmployeeDataSerializableFactory.FACTORY_ID;
}
@Override
public int getId() {
return EmployeeDataSerializableFactory.EMPLOYEE_TYPE;
}
@Override
public String toString() {
return String.format( "Employee(surname=%s)", surname );
}
}
Each of the methods getId
and getFactoryId
should return a unique positive number within the EmployeeDataSerializableFactory
. Now, let's create an instance of this EmployeeDataSerializableFactory
.
public class EmployeeDataSerializableFactory
implements DataSerializableFactory{
public static final int FACTORY_ID = 1;
public static final int EMPLOYEE_TYPE = 1;
@Override
public IdentifiedDataSerializable create(int typeId) {
if ( typeId == EMPLOYEE_TYPE ) {
return new Employee();
} else {
return null;
}
}
}
Only method that should be implemented is create
, as seen in the above sample. It is recommended to use switch
-`case
statement instead of multiple if
-else
lines if you have a lot of subclasses. Hazelcast throws an exception if null is returned for typeId
.
Now, as the last step, you need to register EmployeeDataSerializableFactory
declaratively (in the configuration file hazelcast.xml
) as shown below. Note that, factory-id
has the same value of FACTORY_ID
in the above code. This is crucial to enable Hazelcast to find the correct factory.
<hazelcast>
...
<serialization>
<data-serializable-factories>
<data-serializable-factory
factory-id="1">EmployeeDataSerializableFactory
</data-serializable-factory>
</data-serializable-factories>
</serialization>
...
</hazelcast>
As an alternative to the existing serialization methods, Hazelcast offers a language/platform independent Portable serialization that have the following advantages:
In order to support these features, a serialized Portable object is offered containing meta information like the version and the concrete location of the each field in the binary data. This way Hazelcast is able to navigate in the byte[]
and de-serialize only the required field without actually de-serializing the whole object which improves the Query performance.
With multi-version support, you can have two nodes where each of them having different versions of the same object and Hazelcast will store both meta information and use the correct one to serialize and de-serialize Portable objects depending on the node. This is very helpful when you are doing a rolling upgrade without shutting down the cluster.
Also note that, Portable serialization is totally language independent and is used as the binary protocol between Hazelcast server and clients.
A sample Portable implementation of a Foo class would look like the following.
public class Foo implements Portable{
final static int ID = 5;
private String foo;
public String getFoo() {
return foo;
}
public void setFoo( String foo ) {
this.foo = foo;
}
@Override
public int getFactoryId() {
return 1;
}
@Override
public int getClassId() {
return ID;
}
@Override
public void writePortable( PortableWriter writer ) throws IOException {
writer.writeUTF( "foo", foo );
}
@Override
public void readPortable( PortableReader reader ) throws IOException {
foo = reader.readUTF( "foo" );
}
}
Similar to IdentifiedDataSerializable
, a Portable Class must provide classId
andfactoryId
. The Factory object will be used to create the Portable object given the classId
.
A sample Factory
could be implemented as following:
public class MyPortableFactory implements PortableFactory {
@Override
public Portable create( int classId ) {
if ( Foo.ID == classId )
return new Foo();
else
return null;
}
}
The last step is to register the Factory
to the SerializationConfig
. Below are the programmatic and declarative configurations for this step in order.
Config config = new Config();
config.getSerializationConfig().addPortableFactory( 1, new MyPortableFactory() );
<hazelcast>
<serialization>
<portable-version>0</portable-version>
<portable-factories>
<portable-factory factory-id="1">
com.hazelcast.nio.serialization.MyPortableFactory
</portable-factory>
</portable-factories>
</serialization>
</hazelcast>
Note that the id
that is passed to the SerializationConfig
is same as the factoryId
that Foo
class returns.
More than one versions of the same class may need to be serialized and deserialized. For example, a client may have an older version of a class, and the node to which it is connected can have a newer version of the same class.
Portable serialization supports versioning. Version can be provided declaratively in the configuration file hazelcast.xml
using the portable-version
tag, as shown below.
<serialization>
<portable-version>1</portable-version>
<portable-factories>
<portable-factory factory-id="1">
PortableFactoryImpl
</portable-factory>
</portable-factories>
</serialization>
Things to consider related to versioning is given below:
int
to float
).One needs to be careful when serializing null portables. Hazelcast lazily creates a class definition of portable internally
when the user first serializes. This class definition is stored and used later for deserializing that portable class. When
the user tries to serialize a null portable and if there is no class definition at the moment, Hazelcast throws an
exception saying that com.hazelcast.nio.serialization.HazelcastSerializationException: Cannot write null portable
without explicitly registering class definition!
.
There are two solutions to get rid of this exception. First way, the user can put a non-null portable class of the same type before any other operation. Second way, a class definition can be registered manually in serialization config as shown below.
Config config = new Config();
final ClassDefinition classDefinition = new ClassDefinitionBuilder(Foo.factoryId, Foo.getClassId)
.addUTFField("foo").build();
config.getSerializationConfig().addClassDefinition(classDefinition);
Hazelcast.newHazelcastInstance(config);
Putting a DistributedObject (e.g. Hazelcast Semaphore, Queue, etc.) in a machine and getting it from another one is not a straightforward operation. For this case, passing the ID and type of the DistributedObject can be a solution. For deserialization, try to get the object from HazelcastInstance. For instance, if your distributed object is an instance of IQueue, you can either use HazelcastInstance.getQueue(id)
or Hazelcast.getDistributedObject
.
Also, HazelcastInstanceAware
interface can be used in the case of a deserialization of a Portable DistributedObject and if it gets an ID to be looked up. HazelcastInstance is set after deserialization and hence, you first need to store the ID and then retrieve the DistributedObject using setHazelcastInstance
method.
Hazelcast lets you plug a custom serializer to be used for serialization of objects. StreamSerializer
and ByteArraySerializer
interfaces can be used for this purpose. Let's see the details in below subsections.
You can use a stream in order to serialize and deserialize data by means of StreamSerializer
. It is a good option for your own implementations and it can also be adapted to external serialization libraries like Kryo, JSON and protocol buffers.
First, let's create a simple object.
public class Employee {
private String surname;
public Employee( String surname ) {
this.surname = surname;
}
}
Now, let's implement StreamSerializer for Employee
class.
public class EmployeeStreamSerializer
implements StreamSerializer<Employee> {
@Override
public int getTypeId () {
return 1;
}
@Override
public void write( ObjectDataOutput out, Employee employee )
throws IOException {
out.writeUTF(employee.getSurname());
}
@Override
public Employee read( ObjectDataInput in )
throws IOException {
String surname = in.readUTF();
return new Employee(surname);
}
@Override
public void destroy () {
}
}
Of course, in practice, classes may have many fields. Just make sure the fields are read in the same order as they are written. Another consideration should be the type ID. It must be unique and greater than or equal to 1. Uniqueness of it enables Hazelcast to determine which serializer will be used during deserialization.
And now, as the last step, let's register the EmployeeStreamSerializer
in the configuration file hazelcast.xml
, as shown below.
<serialization>
<serializers>
<serializer type-class="Employee" class-name="EmployeeStreamSerializer" />
</serializers>
</serialization>
NOTE: StreamSerializer
cannot be created for well-known types (e.g. Long, String) and primitive arrays. Hazelcast already registers these types.
Let's take a look at another example implementing StreamSerializer.
public class Foo {
private String foo;
public String getFoo() {
return foo;
}
public void setFoo( String foo ) {
this.foo = foo;
}
}
Assume that our custom serialization will serialize
Foo into XML. First we need to implement a
com.hazelcast.nio.serialization.StreamSerializer
. A very simple one that uses XMLEncoder and XMLDecoder, would look like the following:
public static class FooXmlSerializer implements StreamSerializer<Foo> {
@Override
public int getTypeId() {
return 10;
}
@Override
public void write( ObjectDataOutput out, Foo object ) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
XMLEncoder encoder = new XMLEncoder( bos );
encoder.writeObject( object );
encoder.close();
out.write( bos.toByteArray() );
}
@Override
public Foo read( ObjectDataInput in ) throws IOException {
InputStream inputStream = (InputStream) in;
XMLDecoder decoder = new XMLDecoder( inputStream );
return (Foo) decoder.readObject();
}
@Override
public void destroy() {
}
}
Note that typeId
must be unique as Hazelcast will use it to lookup the StreamSerializer while it de-serializes the object. Now, the last required step is to register the StreamSerializer to the Configuration. Below are the programmatic and declarative configurations for this step in order.
SerializerConfig sc = new SerializerConfig()
.setImplementation(new FooXmlSerializer())
.setTypeClass(Foo.class);
Config config = new Config();
config.getSerializationConfig().addSerializerConfig(sc);
<hazelcast>
<serialization>
<serializers>
<serializer type-class="com.www.Foo">com.www.FooXmlSerializer</serializer>
</serializers>
</serialization>
</hazelcast>
From now on, Hazelcast will use FooXmlSerializer
to serialize Foo objects. This way one can write an adapter (StreamSerializer) for any Serialization framework and plug it into Hazelcast.
ByteArraySerializer
exposes the raw ByteArray used internally by Hazelcast. It is a good option if the serialization library you are using deals with ByteArrays instead of streams.
Let's implement ByteArraySerializer for the Employee
class mentioned in previous sections.
public class EmployeeByteArraySerializer
implements ByteArraySerializer<Employee> {
@Override
public void destroy () {
}
@Override
public int getTypeId () {
return 1;
}
@Override
public byte[] write( Employee object )
throws IOException {
return object.getName().getBytes();
}
@Override
public Employee read( byte[] buffer )
throws IOException {
String surname = new String( buffer );
return new Employee( surname );
}
}
And, as usual, let's register the EmployeeByteArraySerializer
in the configuration file hazelcast.xml
, as shown below.
<serialization>
<serializers>
<serializer type-class="Employee">EmployeeByteArraySerializer</serializer>
</serializers>
</serialization>
You can implement HazelcastInstanceAware
interface to access distributed objects for some cases where an object is deserialized and needs access to HazelcastInstance.
Let's implement it for the Employee
class mentioned in previous sections.
public class Employee
implements Serializable, HazelcastInstanceAware {
private static final long serialVersionUID = 1L;
private String surname;
private transient HazelcastInstance hazelcastInstance;
public Person( String surname ) {
this.surname = surname;
}
@Override
public void setHazelcastInstance( HazelcastInstance hazelcastInstance ) {
this.hazelcastInstance = hazelcastInstance;
System.out.println( "HazelcastInstance set" );
}
@Override
public String toString() {
return String.format( "Person(surname=%s)", surname );
}
}
After deserialization, object is checked whether it implements HazelcastInstanceAware
and the method setHazelcastInstance
is called. Notice the hazelcastInstance
being transient
. It is because this field should not be serialized.
It may be a good practice to inject a HazelcastInstance into a domain object (e.g. Employee
in the above sample) when used together with Runnable
/Callable
implementations. These runnables/callables are executed by IExecutorService which sends them to another machine. And after a task is deserialized, run/call method implementations need to access HazelcastInstance.
We recommend you only to set the HazelcastInstance field while using setHazelcastInstance
method and not to execute operations on the HazelcastInstance. Because, when HazelcastInstance is injected for a HazelcastInstanceAware
implementation, it may not be up and running at the injection time.
You can gather various statistics from your distributed data structures via Statistics API. Since the data structures are distributed in the cluster, the Statistics API provides statistics for the local portion (1/Number of Nodes) of data on each node.
The IMap
interface has a getLocalMapStats()
method which returns a
LocalMapStats
object that holds local map statistics.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IMap<String, Customer> customers = hazelcastInstance.getMap( "customers" );
LocalMapStats mapStatistics = customers.getLocalMapStats();
System.out.println( "number of entries owned on this node = "
+ mapStatistics.getOwnedEntryCount() );
Below is the list of metrics that can be accessed via LocalMapStats
object.
/**
* Returns the number of entries owned by this member.
*/
long getOwnedEntryCount();
/**
* Returns the number of backup entries hold by this member.
*/
long getBackupEntryCount();
/**
* Returns the number of backups per entry.
*/
int getBackupCount();
/**
* Returns memory cost (number of bytes) of owned entries in this member.
*/
long getOwnedEntryMemoryCost();
/**
* Returns memory cost (number of bytes) of backup entries in this member.
*/
long getBackupEntryMemoryCost();
/**
* Returns the creation time of this map on this member.
*/
long getCreationTime();
/**
* Returns the last access (read) time of the locally owned entries.
*/
long getLastAccessTime();
/**
* Returns the last update time of the locally owned entries.
*/
long getLastUpdateTime();
/**
* Returns the number of hits (reads) of the locally owned entries.
*/
long getHits();
/**
* Returns the number of currently locked locally owned keys.
*/
long getLockedEntryCount();
/**
* Returns the number of entries that the member owns and are dirty (updated
* but not persisted yet).
* dirty entry count is meaningful when there is a persistence defined.
*/
long getDirtyEntryCount();
/**
* Returns the number of put operations
*/
long getPutOperationCount();
/**
* Returns the number of get operations
*/
long getGetOperationCount();
/**
* Returns the number of Remove operations
*/
long getRemoveOperationCount();
/**
* Returns the total latency of put operations. To get the average latency,
* divide to number of puts
*/
long getTotalPutLatency();
/**
* Returns the total latency of get operations. To get the average latency,
* divide to number of gets
*/
long getTotalGetLatency();
/**
* Returns the total latency of remove operations. To get the average latency,
* divide to number of gets
*/
long getTotalRemoveLatency();
/**
* Returns the maximum latency of put operations. To get the average latency,
* divide to number of puts
*/
long getMaxPutLatency();
/**
* Returns the maximum latency of get operations. To get the average latency,
* divide to number of gets
*/
long getMaxGetLatency();
/**
* Returns the maximum latency of remove operations. To get the average latency,
* divide to number of gets
*/
long getMaxRemoveLatency();
/**
* Returns the number of Events Received
*/
long getEventOperationCount();
/**
* Returns the total number of Other Operations
*/
long getOtherOperationCount();
/**
* Returns the total number of total operations
*/
long total();
/**
* Cost of map & near cache & backup in bytes
* todo in object mode object size is zero.
*/
long getHeapCost();
/**
* Returns statistics related to the Near Cache.
*/
NearCacheStats getNearCacheStats();
Near Cache statistics can be accessed from LocalMapStats
via
getNearCacheStats()
method which will return a NearCacheStats
object.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
IMap<String, Customer> customers = node.getMap( "customers" );
LocalMapStats mapStatistics = customers.getLocalMapStats();
NearCacheStats nearCacheStatistics = mapStatistics.getNearCacheStats();
System.out.println( "near cache hit/miss ratio= "
+ nearCacheStatistics.getRatio() );
Below is the list of metrics that can be accessed via NearCacheStats
object.
This behavior applies to both client and node near caches.
/**
* Returns the creation time of this NearCache on this member
*/
long getCreationTime();
/**
* Returns the number of entries owned by this member.
*/
long getOwnedEntryCount();
/**
* Returns memory cost (number of bytes) of entries in this cache.
*/
long getOwnedEntryMemoryCost();
/**
* Returns the number of hits (reads) of the locally owned entries.
*/
long getHits();
/**
* Returns the number of misses of the locally owned entries.
*/
long getMisses();
/**
* Returns the hit/miss ratio of the locally owned entries.
*/
double getRatio();
The MultiMap
interface has a getLocalMultiMapStats()
method which returns a
LocalMultiMapStats
object that holds local MultiMap statistics.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
MultiMap<String, Customer> customers = node.getMultiMap( "customers" );
LocalMultiMapStats multiMapStatistics = customers.getLocalMultiMapStats();
System.out.println( "last update time = "
+ multiMapStatistics.getLastUpdateTime() );
Below is the list of metrics that can be accessed via LocalMultiMapStats
object.
/**
* Returns the number of entries owned by this member.
*/
long getOwnedEntryCount();
/**
* Returns the number of backup entries hold by this member.
*/
long getBackupEntryCount();
/**
* Returns the number of backups per entry.
*/
int getBackupCount();
/**
* Returns memory cost (number of bytes) of owned entries in this member.
*/
long getOwnedEntryMemoryCost();
/**
* Returns memory cost (number of bytes) of backup entries in this member.
*/
long getBackupEntryMemoryCost();
/**
* Returns the creation time of this map on this member.
*/
long getCreationTime();
/**
* Returns the last access (read) time of the locally owned entries.
*/
long getLastAccessTime();
/**
* Returns the last update time of the locally owned entries.
*/
long getLastUpdateTime();
/**
* Returns the number of hits (reads) of the locally owned entries.
*/
long getHits();
/**
* Returns the number of currently locked locally owned keys.
*/
long getLockedEntryCount();
/**
* Returns the number of entries that the member owns and are dirty (updated
* but not persisted yet).
* dirty entry count is meaningful when there is a persistence defined.
*/
long getDirtyEntryCount();
/**
* Returns the number of put operations
*/
long getPutOperationCount();
/**
* Returns the number of get operations
*/
long getGetOperationCount();
/**
* Returns the number of Remove operations
*/
long getRemoveOperationCount();
/**
* Returns the total latency of put operations. To get the average latency,
* divide to number of puts
*/
long getTotalPutLatency();
/**
* Returns the total latency of get operations. To get the average latency,
* divide to number of gets
*/
long getTotalGetLatency();
/**
* Returns the total latency of remove operations. To get the average latency,
* divide to number of gets
*/
long getTotalRemoveLatency();
/**
* Returns the maximum latency of put operations. To get the average latency,
* divide to number of puts
*/
long getMaxPutLatency();
/**
* Returns the maximum latency of get operations. To get the average latency,
* divide to number of gets
*/
long getMaxGetLatency();
/**
* Returns the maximum latency of remove operations. To get the average latency,
* divide to number of gets
*/
long getMaxRemoveLatency();
/**
* Returns the number of Events Received
*/
long getEventOperationCount();
/**
* Returns the total number of Other Operations
*/
long getOtherOperationCount();
/**
* Returns the total number of total operations
*/
long total();
/**
* Cost of map & near cache & backup in bytes
* todo in object mode object size is zero.
*/
long getHeapCost();
The IQueue
interface has a getLocalQueueStats()
method which returns a
LocalQueueStats
object that holds local queue statistics.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
IQueue<Order> orders = node.getQueue( "orders" );
LocalQueueStats queueStatistics = orders.getLocalQueueStats();
System.out.println( "average age of items = "
+ queueStatistics.getAvgAge() );
Below is the list of metrics that can be accessed via LocalQueueStats
object.
/**
* Returns the number of owned items in this member.
*/
long getOwnedItemCount();
/**
* Returns the number of backup items in this member.
*/
long getBackupItemCount();
/**
* Returns the min age of the items in this member.
*/
long getMinAge();
/**
* Returns the max age of the items in this member.
*/
long getMaxAge();
/**
* Returns the average age of the items in this member.
*/
long getAvgAge();
/**
* Returns the number of offer/put/add operations.
* Offers returning false will be included.
* #getRejectedOfferOperationCount can be used
* to get the rejected offers.
*/
long getOfferOperationCount();
/**
* Returns the number of rejected offers. Offer
* can be rejected because of max-size limit
* on the queue.
*/
long getRejectedOfferOperationCount();
/**
* Returns the number of poll/take/remove operations.
* Polls returning null (empty) will be included.
* #getEmptyPollOperationCount can be used to get the
* number of polls returned null.
*/
long getPollOperationCount();
/**
* Returns number of null returning poll operations.
* Poll operation might return null, if the queue is empty.
*/
long getEmptyPollOperationCount();
/**
* Returns number of other operations
*/
long getOtherOperationsCount();
/**
* Returns number of event operations
*/
long getEventOperationCount();
The ITopic
interface has a getLocalTopicStats()
method which returns a
LocalTopicStats
object that holds local topic statistics.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
ITopic<Object> news = node.getTopic( "news" );
LocalTopicStats topicStatistics = news.getLocalTopicStats();
System.out.println( "number of publish operations = "
+ topicStatistics.getPublishOperationCount() );
Below is the list of metrics that can be accessed via LocalTopicStats
object.
/**
* Returns the creation time of this topic on this member
*/
long getCreationTime();
/**
* Returns the total number of published messages of this topic on this member
*/
long getPublishOperationCount();
/**
* Returns the total number of received messages of this topic on this member
*/
long getReceiveOperationCount();
The IExecutorService
interface has a getLocalExecutorStats()
method which returns a
LocalExecutorStats
object that holds local executor statistics.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
IExecutorService orderProcessor = node.getExecutorService( "orderProcessor" );
LocalExecutorStats executorStatistics = orderProcessor.getLocalExecutorStats();
System.out.println( "completed task count = "
+ executorStatistics.getCompletedTaskCount() );
Below is the list of metrics that can be accessed via LocalExecutorStats
object.
/**
* Returns the number of pending operations of the executor service
*/
long getPendingTaskCount();
/**
* Returns the number of started operations of the executor service
*/
long getStartedTaskCount();
/**
* Returns the number of completed operations of the executor service
*/
long getCompletedTaskCount();
/**
* Returns the number of cancelled operations of the executor service
*/
long getCancelledTaskCount();
/**
* Returns the total start latency of operations started
*/
long getTotalStartLatency();
/**
* Returns the total execution time of operations finished
*/
long getTotalExecutionLatency();
Hazelcast members expose various management beans which includes statistics about distributed data structures and states of Hazelcast node internals.
The metrics are local to the nodes, i.e. they do not reflect cluster wide values.
You can find the JMX API definition below with descriptions and the API methods in parenthesis.
Atomic Long (IAtomicLong
)
name
)currentValue
)set(v)
)addAndGet(v)
)compareAndSet(e,v)
)decrementAndGet()
)getAndAdd(v)
)getAndIncrement()
)getAndSet(v)
)incrementAndGet()
)partitionKey
)Atomic Reference ( IAtomicReference
)
name
)partitionKey
)Countdown Latch ( ICountDownLatch
)
name
)count
)countDown()
)partitionKey
)Executor Service ( IExecutorService
)
localPendingTaskCount
)localStartedTaskCount
)localCompletedTaskCount
)localCancelledTaskCount
)localTotalStartLatency
)localTotalExecutionLatency
)List ( IList
)
name
)clear
)totalAddedItemCount
)totalRemovedItemCount
)Lock ( ILock
)
name
)lockObject
)partitionKey
)Map ( IMap
)
name
)size
)config
)localOwnedEntryCount
)localOwnedEntryMemoryCost
)localBackupEntryCount
)localBackupEntryMemoryCost
)localBackupCount
)localCreationTime
)localLastAccessTime
)localLastUpdateTime
)localHits
)localLockedEntryCount
)localDirtyEntryCount
)localPutOperationCount
)localGetOperationCount
)localRemoveOperationCount
)localTotalPutLatency
)localTotalGetLatency
)localTotalRemoveLatency
)localMaxPutLatency
)localMaxGetLatency
)localMaxRemoveLatency
)localEventOperationCount
)localOtherOperationCount
)localTotal
)localHeapCost
)totalAddedEntryCount
)totalRemovedEntryCount
)totalUpdatedEntryCount
)totalEvictedEntryCount
)clear()
)values(p)
)entrySet(p)
)MultiMap ( MultiMap
)
name
)size
)localOwnedEntryCount
)localOwnedEntryMemoryCost
)localBackupEntryCount
)localBackupEntryMemoryCost
)localBackupCount
)localCreationTime
)localLastAccessTime
)localLastUpdateTime
)localHits
)localLockedEntryCount
)localPutOperationCount
)localGetOperationCount
)localRemoveOperationCount
)localTotalPutLatency
)localTotalGetLatency
)localTotalRemoveLatency
)localMaxPutLatency
)localMaxGetLatency
)localMaxRemoveLatency
)localEventOperationCount
)localOtherOperationCount
)localTotal
)clear()
)Replicated Map ( ReplicatedMap
)
name
)size
)config
)localOwnedEntryCount
)localCreationTime
)localLastAccessTime
)localLastUpdateTime
)localHits
)localPutOperationCount
)localGetOperationCount
)localRemoveOperationCount
)localTotalPutLatency
)localTotalGetLatency
)localTotalRemoveLatency
)localMaxPutLatency
)localMaxGetLatency
)localMaxRemoveLatency
)localEventOperationCount
)localReplicationEventCount
)localOtherOperationCount
)localTotal
)totalAddedEntryCount
)totalRemovedEntryCount
)totalUpdatedEntryCount
)clear()
)values()
)entrySet()
)Queue ( IQueue
)
name
)QueueConfig
)partitionKey
)localOwnedItemCount
)localBackupItemCount
)localMinAge
)localMaxAge
)localAveAge
)localOfferOperationCount
)localRejectedOfferOperationCount
)localPollOperationCount
)localEmptyPollOperationCount
)localOtherOperationsCount
)localEventOperationCount
)totalAddedItemCount
)totalRemovedItemCount
)clear()
)Semaphore ( ISemaphore
)
name
)available
)partitionKey
)drain()
)reduce(v)
)release(v)
)Set ( ISet
)
name
)partitionKey
)totalAddedItemCount
)totalRemovedItemCount
)clear()
)Topic ( ITopic
)
name
)config
)localCreationTime
)localPublishOperationCount
)localReceiveOperationCount
)totalMessageCount
)Hazelcast Instance ( HazelcastInstance
)
name
)version
)build
)config
)configSource
)groupName
)port
)clusterTime
)memberCount
)Members
)running
)shutdown()
)HazelcastInstance.Node
)address
)masterAddress
)HazelcastInstance.EventService
)eventThreadCount
)eventQueueSize
)eventQueueCapacity
)HazelcastInstance.OperationService
)responseQueueSize
)operationExecutorQueueSize
)runningOperationsCount
)remoteOperationCount
)executedOperationCount
)operationThreadCount
)HazelcastInstance.ProxyService
)proxyCount
)HazelcastInstance.PartitionService
)partitionCount
)activePartitionCount
)HazelcastInstance.ConnectionManager
)clientConnectionCount
)activeConnectionCount
)connectionCount
)HazelcastInstance.ClientEngine
)clientEndpointCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)HazelcastInstance.ManagedExecutorService
)name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)You can monitor your Hazelcast members via the JMX protocol.
Add the following system properties to enable JMX agent:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=\_portNo\_
(to specify JMX port) (optional)-Dcom.sun.management.jmxremote.authenticate=false
(to disable JMX auth) (optional)Enable Hazelcast property hazelcast.jmx
(please refer to Advanced Configuration Properties);
-Dhazelcast.jmx=true
Use jconsole, jvisualvm (with mbean plugin) or another JMX compliant monitoring tool.
Hazelcast allows you to register for membership events to get notified when members added or removed. You can also get the set of cluster members.
import com.hazelcast.core.*;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Cluster cluster = hazelcastInstance.getCluster();
cluster.addMembershipListener( new MembershipListener() {
public void memberAdded( MembershipEvent membershipEvent ) {
System.out.println( "MemberAdded " + membershipEvent );
}
public void memberRemoved( MembershipEvent membershipEvent ) {
System.out.println( "MemberRemoved " + membershipEvent );
}
} );
Member localMember = cluster.getLocalMember();
System.out.println ( "my inetAddress= " + localMember.getInetAddress() );
Set setMembers = cluster.getMembers();
for ( Member member : setMembers ) {
System.out.println( "isLocalMember " + member.localMember() );
System.out.println( "member.inetaddress " + member.getInetAddress() );
System.out.println( "member.port " + member.getPort() );
}
You can define various member attributes on your Hazelcast members. You can use member attributes to tag your members as your business logic requirements.
In order to define member attribute on a member you can either
Provide MemberAttributeConfig
to your Config
object.
Provide member attributes at runtime via attribute setter methods on Member
interface.
For example, you can tag your members with their CPU characteristics and you can route CPU intensive tasks to those CPU-rich members.
MemberAttributeConfig fourCore = new MemberAttributeConfig();
memberAttributeConfig.setIntAttribute( "CPU_CORE_COUNT", 4 );
MemberAttributeConfig twelveCore = new MemberAttributeConfig();
memberAttributeConfig.setIntAttribute( "CPU_CORE_COUNT", 12 );
MemberAttributeConfig twentyFourCore = new MemberAttributeConfig();
memberAttributeConfig.setIntAttribute( "CPU_CORE_COUNT", 24 );
Config member1Config = new Config();
config.setMemberAttributeConfig( fourCore );
Config member2Config = new Config();
config.setMemberAttributeConfig( twelveCore );
Config member3Config = new Config();
config.setMemberAttributeConfig( twentyFourCore );
HazelcastInstance member1 = Hazelcast.newHazelcastInstance( member1Config );
HazelcastInstance member2 = Hazelcast.newHazelcastInstance( member2Config );
HazelcastInstance member3 = Hazelcast.newHazelcastInstance( member3Config );
IExecutorService executorService = member1.getExecutorService( "processor" );
executorService.execute( new CPUIntensiveTask(), new MemberSelector() {
@Override
public boolean select(Member member) {
int coreCount = (int) member.getIntAttribute( "CPU_CORE_COUNT" );
// Task will be executed at either member2 or member3
if ( coreCount > 8 ) {
return true;
}
return false;
}
} );
HazelcastInstance member4 = Hazelcast.newHazelcastInstance();
// We can also set member attributes at runtime.
member4.setIntAttribute( "CPU_CORE_COUNT", 2 );
To prevent data loss when shutting down a node, Hazelcast provides graceful shutdown feature. This is performed by calling the method Hazelcast.shutdown
. Once it is called, it checks the below conditions to ensure the node is safe to shutdown:
Even if the above conditions are not met, then they are forced to be completed by Hazelcast.shutdown
. Eventually, when this method returns, it means node has been brought to a safe state and it can be shutdown without any data loss.
What if you want to be sure that your cluster is in a safe state? What does it mean that cluster is safe to shutdown without any data loss?
There may be some use cases like rolling upgrades, development/testing or any logic that requires a cluster/member to be safe. To provide this, Hazelcast offers PartitionService
interface with the methods isClusterSafe
, isMemberSafe
, isLocalMemberSafe
and forceLocalMemberToBeSafe
. These methods can also be deemed as decoupled pieces from the method Hazelcast.shutdown
.
public interface PartitionService {
...
...
boolean isClusterSafe();
boolean isMemberSafe(Member member);
boolean isLocalMemberSafe();
boolean forceLocalMemberToBeSafe(long timeout, TimeUnit unit);
}
The method isClusterSafe
checks whether the cluster is in a safe state. It returns true
if there are no active partition migration and there are sufficient backups for each partition. Once it returns true
it means the cluster is safe and a node can be shutdown without data loss.
The method isMemberSafe
checks whether a specific node is in a safe state. This check controls if first backups of partitions of the given node are synched with the primary ones. Once it returns true
it means the given node is safe and it can be shutdown without data loss. Similarly, the method isLocalMemberSafe
does the same check for the local member. And, the method forceLocalMemberToBeSafe
forces the owned and backup partitions to be synchronized to make the local member safe.
NOTE: These methods are available from Hazelcast 3.3.
PartitionService partitionService = hazelcastInstance.getPartitionService().isClusterSafe()
if (partitionService().isClusterSafe()) {
hazelcastInstance.shutdown(); // or terminate
}
OR
PartitionService partitionService = hazelcastInstance.getPartitionService().isClusterSafe()
if (partitionService().isLocalMemberSafe()) {
hazelcastInstance.shutdown(); // or terminate
}
RELATED INFORMATION
For more code samples please refer to PartitionService Code Samples.
Hazelcast Management Center enables you to monitor and manage your nodes running Hazelcast. In addition to monitoring overall state of your clusters, you can also analyze and browse your data structures in detail, update map configurations and take thread dump from nodes. With its scripting and console module, you can run scripts (JavaScript, Groovy, etc.) and commands on your nodes.
There are two options for installing Hazelcast Management Center. You can either deploy mancenter
-version.war
application into your Java application server/container or start Hazelcast Management Center directly from the command line and then have the Hazelcast nodes communicate with that web application. That means, your Hazelcast nodes should know the URL of mancenter
application before they start.
Here are the steps:
mancenter
-version.war
file. mancenter
-version.war
file from the command line:java -jar mancenter-*version*.war 8080 mancenter
http://localhost:8080/mancenter
).http://localhost:8080/mancenter
.http://localhost:8080/mancenter
is up.hazelcast.xml
.
Browse to http://localhost:8080/mancenter
and login. Initial login username/password is admin/admin
Management Center creates a directory with name "mancenter" under your "user/home" directory to save data files. You can change the data directory by setting hazelcast.mancenter.home
system property.
Once the page is loaded after selecting a cluster, tool's home page appears as shown below.
This page provides the fundamental properties of the selected cluster which are explained in Home Page section.
It also has a toolbar on the top and a menu on the left.
Toolbar has the following buttons:
Cluster Selector: It is used to switch between clusters. When the mouse is moved onto this item, a drop down list of clusters appears.
The user can select any cluster and once selected, the page immediately loads with the selected cluster's information.
NOTE: Some of the above listed toolbar items are not visible to the users who are not admin or have read-only permission. Also, some of the operations explained in the later sections cannot be performed by users with read-only permission. Please see Administration for details.
Home page includes a menu on the left which lists the distributed data structures in the cluster and also all cluster members (nodes), as shown below.
NOTE: Distributed data structures will be shown there when the proxies are created for them.
Menu items can be expanded/collapsed by clicking on them. Below is the list of menu items with the links to their explanations.
Each time an item from the toolbar or menu is selected, it is added to main view as a tab, as shown below.
In the above example, Home, Scripting, Console, queue1 and map1 windows can be seen as tabs. Windows can be closed using the icon on each tab (except the Home Page; it cannot be closed).
This is the first page appearing after logging in. It gives an overview of the cluster connected. Below subsections describe each portion of the page.
This part of the page provides information related to load and utilization of CPUs for each node, as shown below.
First column lists the nodes with their IPs and ports. Then, the loads on each CPU for the last 1, 5 and 15 minutes are listed. The last column (Chart) shows the utilization of CPUs graphically. When you move the mouse cursor on a desired graph, you can see the CPU utilization at the time to which cursor corresponds. Graphs under this column shows the CPU utilizations approximately for the last 2 minutes.
This part of the page provides information related to memory usages for each node, as shown below.
First column lists the nodes with their IPs and ports. Then, used and free memories out of the total memory reserved for Hazelcast usage are shown, in real-time. Max column lists the maximum memory capacity of each node and Percent column lists the percentage value of used memory out of the maximum memory. The last column (Chart) shows the memory usage of nodes graphically. When you move the mouse cursor on a desired graph, you can see the memory usage at the time to which cursor corresponds. Graphs under this column shows the memory usages approximately for the last 2 minutes.
This part of the page graphically provides the cluster wise breakdown of memory, as shown below. Blue area is the memory used by maps, dark yellow area is the memory used by non-Hazelcast entities and green area is the free memory (out of whole cluster`s memory capacity).
In the above example, you can see 0.32% of the total memory is used by Hazelcast maps (it can be seen by moving the mouse cursor on it), 58.75% is used by non-Hazelcast entities and 40.85% of the total memory is free.
This part is actually the breakdown of the blue area shown in Memory Distribution graph explained above. It provides the percentage values of the memories used by each map, out of the total cluster memory reserved for all Hazelcast maps.
In the above example, you can see 49.55% of the total map memory is used by map1 and 49.55% is used by map2.
This pie chart shows what percentage of partitions each node has, as shown below.
You can see each node's partition percentages by moving the mouse cursor on the chart. In the above example, you can see the node "<IP_ADDRESS>:5708" has 5.64% of the total partition count (which is 271 by default and configurable, please see Advanced Configuration Properties).
You can monitor your caches' metrics by clicking the cache name listed on the left panel under Caches menu item. A new tab for monitoring that cache instance is opened on the right, as shown below.
On top of the page, there are four charts to monitor the Gets, Puts, Removals and Evictions in real-time. For all charts' X-axis shows the current system time and a chart can be opened as a separate dialog by clicking on the button placed at top right of each chart.
Under these charts, there is Cache Statistics Data Table. From left to right, this table lists the IP addresses and ports of each node, get, put, removal, eviction, hit and miss count per second in real-time.
You can navigate through the pages using the buttons placed at the bottom right of the table (First, Previous, Next, Last). The order of the listings in each column can be ascended or descended by clicking on column headings.
Map instances are listed under the Maps menu item on the left. When you click on a map, a new tab for monitoring that map instance is opened on the right, as shown below. In this tab, you can monitor metrics and also re-configure the selected map.
Below subsections explain the portions of this window.
Map Browser is a tool used to retrieve properties of the entries stored in the selected map. It can be opened by clicking on the Map Browser button, located at top right of the window. Once opened, the tool appears as a dialog, as shown below.
Once the key and key's type is specified and Browse button is clicked, key's properties along with its value is listed.
By using Map Config tool, you can set selected map's attributes like the backup count, TTL, and eviction policy. It can be opened by clicking on the Map Config button, located at top right of the window. Once opened, the tool appears as a dialog, as shown below.
Change any attribute as required and click Update button to save changes.
Besides Map Browser and Map Config tools, this page has many monitoring options explained below. All of these perform real-time monitoring.
On top of the page, there are small charts to monitor the size, throughput, memory usage, backup size, etc. of the selected map in real-time. All charts' X-axis shows the current system time. Other small monitoring charts can be selected using button placed at top right of each chart. When it is clicked, the whole list of monitoring options are listed, as shown below.
When you click on a desired monitoring, the chart is loaded with the selected option. Also, a chart can be opened as a separate dialog by clicking on the button placed at top right of each chart. Below monitoring charts are available:
Under these charts, there are Map Memory and Map Throughput data tables. Map Memory data table provides memory metrics distributed over nodes, as shown below.
From left to right, this table lists the IP address and port, entry counts, memory used by entries, backup entry counts, memory used by backup entries, events, hits, locks and dirty entries (in the cases where MapStore is enabled, these are the entries that are put to/removed from the map but not written to/removed from a database yet) of each node in the map. You can navigate through the pages using the buttons placed at the bottom right of the table (First, Previous, Next, Last). The order of the listings in each column can be ascended or descended by clicking on column headings.
Map Throughput data table provides information about the operations (get, put, remove) performed on each node in the map, as shown below.
From left to right, this table lists the IP address and port of each node, put, get and remove operations on each node, average put, get, remove latencies and maximum put, get, remove latencies on each node.
You can select the period in the combo box placed at top right corner of the window, for which the table data will be shown. Available values are Since Beginning, Last Minute, Last 10 Minutes and Last 1 Hour.
You can navigate through the pages using the buttons placed at the bottom right of the table (First, Previous, Next, Last). The order of the listings in each column can be ascended or descended by clicking on column headings.
Using the menu item Queues, you can monitor your queues data structure. When you expand this menu item and click on a queue, a new tab for monitoring that queue instance is opened on the right, as shown below.
On top of the page, there are small charts to monitor the size, offers and polls of the selected queue in real-time. All charts' X-axis shows the current system time. And a chart can be opened as a separate dialog by clicking on the button placed at top right of each chart. Below monitoring charts are available:
Under these charts, there are Queue Statistics and Queue Operation Statistics tables. Queue Statistics table provides item and backup item counts in the queue and age statistics of items and backup items at each node, as shown below.