Ehcache Configuration

Ehcache configurations for session replication

The following sections give an overview of ready to use Ehcache configurations for session replication packaged in the MPI distribution.

Without replication (default): ehcache-no-replication.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<?xml version="1.0"?>
 
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" monitoring="autodetect" dynamicConfig="false">
 
  <cache name="mpiSessionCache"
         maxElementsInMemory="0"
         timeToLiveSeconds="1800"
         timeToIdleSeconds="1800"
         memoryStoreEvictionPolicy="LRU"
         diskPersistent="false"
         overflowToDisk="false"
         maxElementsOnDisk="0"
         eternal="false" />
</ehcache>

Replication through RMI

Asynchronous: ehcache-rmi-async.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
<?xml version="1.0"?>
 
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" monitoring="autodetect" dynamicConfig="false">
 
  <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
    properties="peerDiscovery=automatic,
              multicastGroupAddress=230.0.0.1,
              multicastGroupPort=4446,
              timeToLive=32"/>
 
  <cacheManagerPeerListenerFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" />
 
  <cache name="mpiSessionCache"
         maxElementsInMemory="0"
         timeToLiveSeconds="1800"
         timeToIdleSeconds="1800"
         memoryStoreEvictionPolicy="LRU"
         diskPersistent="false"
         overflowToDisk="false"
         maxElementsOnDisk="0"
         eternal="false">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
      properties="replicateAsynchronously=true,
                  asynchronousReplicationIntervalMillis=100,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=false,
                  replicateRemovals=true" />
    <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory"
      properties="bootstrapAsynchronously=false"  />
  </cache>
</ehcache>
Synchronous: ehcache-rmi-sync.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
<?xml version="1.0"?>
 
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" monitoring="autodetect" dynamicConfig="false">
 
  <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
    properties="peerDiscovery=automatic,
                multicastGroupAddress=230.0.0.1,
                multicastGroupPort=4446,
                timeToLive=32"/>
 
  <cacheManagerPeerListenerFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" />
 
  <cache name="mpiSessionCache"
         maxElementsInMemory="0"
         timeToLiveSeconds="1800"
         timeToIdleSeconds="1800"
         memoryStoreEvictionPolicy="LRU"
         diskPersistent="false"
         overflowToDisk="false"
         maxElementsOnDisk="0"
         eternal="false">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
      properties="replicateAsynchronously=false,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=false,
                  replicateRemovals" />
    <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory"
      properties="bootstrapAsynchronously=false"  />
  </cache>
</ehcache>

For more information about RMI and Ehcache consult Ehcache RMI Replicated Caching page.

Replication through JGroups TCP

Asynchronous: ehcache-tcp-async.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" name="JGroupsCache">
 
  <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
    properties="connect=
                TCP(bind_port=7800;bind_addr=localhost):
                TCPPING(initial_hosts=localhost[7800],localhost[7801];port_range=1;timeout=3000;num_initial_members=3):
                VERIFY_SUSPECT(timeout=1500):
                STATS:
                pbcast.NAKACK(retransmit_timeout=3000;use_mcast_xmit=false):
                UNICAST:
                pbcast.GMS(join_timeout=5000;print_local_addr=true)"
    propertySeparator="::" />
 
 
  <cache name="mpiSessionCache"
         eternal="false"
         timeToIdleSeconds="1800"
         timeToLiveSeconds="1800"
         maxElementsInMemory="0"
         overflowToDisk="false"
         diskPersistent="false"
         memoryStoreEvictionPolicy="LRU">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
      properties="replicateAsynchronously=true,
                  asynchronousReplicationIntervalMillis=100,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=false,
                  replicateRemovals=true" />
 
    <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"
      properties="bootstrapAsynchronously=false" />
  </cache>
</ehcache>
Synchronous: ehcache-tcp-sync.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" name="JGroupsCache">
 
  <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
    properties="connect=
                TCP(bind_port=7800;bind_addr=localhost):
                TCPPING(initial_hosts=localhost[7800],localhost[7801];port_range=1;timeout=3000;num_initial_members=3):
                VERIFY_SUSPECT(timeout=1500):
                STATS:
                pbcast.NAKACK(retransmit_timeout=3000;use_mcast_xmit=false):
                UNICAST:
                RSVP:
                pbcast.GMS(join_timeout=5000;print_local_addr=true)"
    propertySeparator="::" />
 
 
  <cache name="mpiSessionCache"
         eternal="false"
         timeToIdleSeconds="1800"
         timeToLiveSeconds="1800"
         maxElementsInMemory="0"
         overflowToDisk="false"
         diskPersistent="false"
         memoryStoreEvictionPolicy="LRU">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
      properties="replicateAsynchronously=false,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=false,
                  replicateRemovals=true" />
 
    <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"
      properties="bootstrapAsynchronously=false" />
  </cache>
</ehcache>

Replication through JGroups UDP

Asynchronous: ehcache-udp-async.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
<?xml version="1.0"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" monitoring="autodetect" dynamicConfig="false">
 
  <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
    properties="connect=
                UDP(mcast_addr=228.8.8.8;mcast_port=7600;ip_ttl=32;mcast_send_buf_size=150000;mcast_recv_buf_size=80000):
                PING(timeout=2000;num_initial_members=3):
                MERGE2(min_interval=5000;max_interval=10000):
                FD_SOCK:
                VERIFY_SUSPECT(timeout=1500):
                STATS:
                pbcast.NAKACK:
                UNICAST:
                pbcast.STABLE(desired_avg_gossip=20000):
                FRAG:
                pbcast.GMS(join_timeout=5000;print_local_addr=true)"
    propertySeparator="::"
  />
 
  <cache name="mpiSessionCache"
         maxElementsInMemory="0"
         timeToLiveSeconds="1800"
         timeToIdleSeconds="1800"
         memoryStoreEvictionPolicy="LRU"
         diskPersistent="false"
         overflowToDisk="false"
         maxElementsOnDisk="0"
         eternal="false">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
      properties="replicateAsynchronously=true,
                  asynchronousReplicationIntervalMillis=100,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=false,
                  replicateRemovals=true" />
    <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"
      properties="bootstrapAsynchronously=false"  />
  </cache>
</ehcache>
Synchronous: ehcache-udp-sync.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
<?xml version="1.0"?>
 
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
  updateCheck="false" monitoring="autodetect" dynamicConfig="false">
 
  <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
    properties="connect=
               UDP(mcast_addr=228.8.8.8;mcast_port=7600;ip_ttl=32;mcast_send_buf_size=150000;mcast_recv_buf_size=80000):
               PING(timeout=2000;num_initial_members=3):
               MERGE2(min_interval=5000;max_interval=10000):
               FD_SOCK:
               VERIFY_SUSPECT(timeout=1500):
               STATS:
               pbcast.NAKACK:
               UNICAST:
               RSVP:
               pbcast.STABLE(desired_avg_gossip=20000):
               FRAG:
               pbcast.GMS(join_timeout=5000;print_local_addr=true)"
    propertySeparator="::"
  />
 
  <cache name="mpiSessionCache"
         maxElementsInMemory="0"
         timeToLiveSeconds="1800"
         timeToIdleSeconds="1800"
         memoryStoreEvictionPolicy="LRU"
         diskPersistent="false"
         overflowToDisk="false"
         maxElementsOnDisk="0"
         eternal="false">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
      properties="replicateAsynchronously=false,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=false,
                  replicateRemovals=true" />
    <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"
      properties="bootstrapAsynchronously=false"  />
  </cache>
</ehcache>

Use an Ehcache configuration

The use of an Ehcache configuration for session replication can be configured via MPI Configuration Properties:

sessionEhcacheConfigXml.resourceLocation=[configuration_to_be_used]

To use one of the packaged configurations (e.g. ehcache-rmi-sync.xml), set the property as follows:

sessionEhcacheConfigXml.resourceLocation=classpath:ehcache-rmi-sync.xml

To use a custom configuration (e.g. /etc/mpi/ehcache.xml), set the property as follows:

sessionEhcacheConfigXml.resourceLocation=file:/etc/mpi/ehcache.xml

If you don't specify a configuration for session replication, the configuration found under $MPI_CONFIG_HOME/ehcache.xml is used.

Considerations when choosing an Ehcache session replication strategy

When deciding which replication strategy to choose (RMI, JGroups TCP or JGroups UDP), consider the setup of your network infrastructure and the required features from the replication:

  • Synchronous vs. asynchronous replication
    • Synchronous means high data refresh with high I/O network traffic over the network.
    • Asynchronous means working with old data but you save the network traffic.
    • Session replication in MPI happens between the two MPI requests, createPaReqIfEnrolled and validatePaRes, track the span between them to verify that session replication between the nodes happened before the validatePaRes request was received in the system.
    • Note that if using JGroups setup synchronous session replication for Ehcache, there is a limitation of not fully synchronizing the nodes before response is returned by MPI.
      If immediate session synchronization between the nodes is a must, use of ehcache-rmi-sync.xml configuration is recommended.
  • TCP vs. UDP replication
    • TCP is fully reliable protocol, but has restriction that all the nodes in the cluster must be known and configured before starting the system.
    • On the other hand, UDP protocol offers flexibility to subscribe and unsubscribe nodes at the replication service at runtime. It is not reliable protocol but JGroups has internal mechanisms that fix this feature.
    • UDP uses multicasting, but Virtualization technologies like Xen and VMWare may be blocking multicast.
      Verify that IP Multicasting is enabled and nodes can find them self on designated addresses and ports. (check It doesn't work section fro more details)
    • Use of IPv6 might be source of problems, if the nodes are not able to find each other, try specifying -Djava.net.preferIP4Stack=true property when starting the nodes.
      (check Problems with IPv6 for more details)