Benchmark-Suite

Benchmarks of different IMDG Products. Focused on Off-Heap capabilities.

Please refer to original project: Coherence-hazelcast-benchmark-suite

Product List

Vendor	Product	Version
Oracle	Coherence	12.2.1-3-0
Pivotal	Apache Geode	1.8.0
Hazelcast	Hazelcast IMDG	3.11.1
GridGain	Apache Ignite	2.7.0

Latest Benchmark Results

Raw JMH Benchmark Results

Test Strategy

Since Ignite does not allow On-Heap Only, all the cached maps are configured for Off-Heap only. Benchmark is configured for Sampling for 100K iterations measured in microseconds.

The following operations are benchmarked:

Description	Method Name
Insert single Trade	#b01_insertTradeSingle
Insert bulk Trades	#b02_insertTradeBulk
Get single Trade	#b03_getTradeSingle
One attribute trade query	#b04_getTradeOneFilter
Three attribute trade query	#b05_getTradesThreeFilter
One attribute (indexed) trade query	#b06_getTradeIndexedFilter
Get trades by id using range query	#b07_getTradeIdRangeFilter

Building the Benchmark Applications

Build all

gradle -Dorg.gradle.daemon=false clean buildAll

Build and package each individual application

Coherence

Due to Oracle licensing constraints, coherence is not available in any maven repository. Therefore, you must download it first from Oracle Downloads.

1. Download the latest version
2. Install the archive locally to your computer and install into maven repository. Please note that you must update specified version in ./coherence/build.properties if it has changed. And, example below needs to be updated as well to reflect this.
3. Example:

$java -jar fmw_12.2.1.3.0_coherence.jar
$cd ~/Oracle_home/Middleware/Oracle_home
$mvn install:install-file -Dfile=./coherence/lib/coherence.jar -DgroupId=com.oracle.coherence -DartifactId=coherence -Dversion=12.2.1.3.0 -Dpackaging=jar

Now, you can build it ...

gradle -Dorg.gradle.daemon=false coherence:clean coherence:buildAll

Geode

gradle -Dorg.gradle.daemon=false geode:clean geode:buildAll

Hazelcast

gradle -Dorg.gradle.daemon=false hazelcast:clean hazelcast:buildAll

Ignite

gradle -Dorg.gradle.daemon=false ignite:clean ignite:buildAll

Deploy the Applications

Hazelcast

• Pre-Install Requirements
  • Enterprise HD License required. If you do not have one, you will need to ask Sales for trial license.
  • edit your .bash_profile to add the following ...

export HAZELCAST_LICENSE_KEY=XXXXXXXXXXXXXXXXXXXXXXXXX

• Extract the libraries

cd ./hazelcast/build/distributions
tar xvf benchmark.hazelcast-1.0-SNAPSHOT.tar
cd ./bin

Start the cluster

• Locally...

./startHazelcastMemberLocal.sh &
./startHazelcastMemberLocal.sh &

• On Server(s)...

./startHazelcastMember.sh &
./startHazelcastMember.sh &

Coherence

• Extract the libraries

cd ./coherence/build/distributions
tar xvf benchmark.coherence-1.0-SNAPSHOT.tar
cd ./bin

Start the cluster

• Locally...

./startCoherenceMemberLocal.sh &
./startCoherenceMemberLocal.sh &

• On Server(s)...

./startCoherenceMember1.sh &
./startCoherenceMember2.sh &

Geode

• Extract the libraries

cd ./geode/build/distributions
tar xvf benchmark.geode-1.0-SNAPSHOT.tar
cd ./bin

Start the cluster

• Locally...

./startMember1Local.sh &
./startMember2Local.sh &

• On Server(s)...

./startMember1.sh &
./startMember2.sh &

Ignite

• Extract the libraries

cd ./ignite/build/distributions
tar xvf benchmark.ignite-1.0-SNAPSHOT.tar
cd ./bin

Start the cluster

• Locally...

./startIgniteMemberLocal.sh &
./startIgniteMemberLocal.sh &

• On Server(s)...

./startIgniteMember.sh &
./startIgniteMember.sh &

Running the Benchmarks

Hazelcast

• Locally

./ runHazelcastBenchmarkLocal.sh

• On Server

./ runHazelcastBenchmark.sh

Coherence

• Locally

./ runCoherenceBenchmarkLocal.sh

• On Server

./ runCoherenceBenchmark.sh

Geode

• Locally

./ runGeodeBenchmarkLocal.sh

• On Server

./ runGeodeBenchmark.sh

Ignite

• Locally

./ runIgniteBenchmarkLocal.sh

• On Server

./ runIgniteBenchmark.sh

Back to Top