当前位置:网站首页>Technical foundation Apache Cassandra 4.0 benchmark

Technical foundation Apache Cassandra 4.0 benchmark

2021-01-24 02:12:13 itread01

 

 

Apache Cassandra 4.0 It has been released Beta edition , This is the first support JDK 11 And higher JDK Version of Cassandra edition .
 
Time delay vs Apache Cassandra It's an obvious concern for users , So you're right JDK 11 New low latency garbage collector introduced in ZGC(Z Garbage Collector) High hopes .
 
We will see Cassandra 4.0 Brings powerful performance improvements , And some new garbage collectors (ZGC、 In especial Shenandoah) To a large extent, the launch of the company has enhanced the effect of these improvements .
 
Tested by the author ,Cassandra 4.0 In terms of throughput and delay 25% To 30% The promotion of . So when using the same garbage collector ,Cassandra 4.0 It's easy to beat Cassandra 3.11.6.

 
01 Test method
 
The next benchmark is to use tlp-cluster stay AWS Build and configure Apache Cassandra A mass of , And use tlp-stress For load generation and indicator collection .
 
All the tools used in this test are open source , And just have one AWS Account number , Anyone can easily reproduce the process and results of this test .
 
There are three clusters in this test r3.2xlarge Example as node , Another one c3.2xlarge Examples are used as pressure measurement nodes .
 
In addition to garbage collection and heap memory settings , We used Apache Cassandra Default settings for .
 
Clusters are generated and configured by the latest version of tlp-cluster Accomplished . besides , We've also added some auxiliary scripts (helper script), In this way, we can put Reaper and Medusa Automation of cluster generation and installation process .
 
Install and configure according to the file tlp-cluster After the tool , You can always build the same For benchmarking Cassandra It's all over .
 
# 3.11.6 CMS JDK8build_cluster.sh -n CMS_3-11-6_jdk8 -v 3.11.6 --heap=16 --gc=CMS -s 1 -i r3.2xlarge --jdk=8 --cores=8# 3.11.6 G1 JDK8build_cluster.sh -n G1_3-11-6_jdk8 -v 3.11.6 --heap=31 --gc=G1 -s 1 -i r3.2xlarge --jdk=8 --cores=8# 4.0 CMS JDK11build_cluster.sh -n CMS_4-0_jdk11 -v 4.0~alpha4 --heap=16 --gc=CMS -s 1 -i r3.2xlarge --jdk=11 --cores=8# 4.0 G1 JDK14build_cluster.sh -n G1_4-0_jdk14 -v 4.0~alpha4 --heap=31 --gc=G1 -s 1 -i r3.2xlarge --jdk=14 --cores=8# 4.0 ZGC JDK11build_cluster.sh -n ZGC_4-0_jdk11 -v 4.0~alpha4 --heap=31 --gc=ZGC -s 1 -i r3.2xlarge --jdk=11 --cores=8# 4.0 ZGC JDK14build_cluster.sh -n ZGC_4-0_jdk14 -v 4.0~alpha4 --heap=31 --gc=ZGC -s 1 -i r3.2xlarge --jdk=14 --cores=8# 4.0 Shenandoah JDK11build_cluster.sh -n Shenandoah_4-0_jdk11 -v 4.0~alpha4 --heap=31 --gc=Shenandoah -s 1 -i r3.2xlarge --jdk=11 --cores=8  
Be careful : To control variables in benchmarking , We'll use the same group throughout the test EC2 Examples to test .
 
Use the following script appropriately , You can do it Cassandra 3.11.6 To Cassandra 4.0~alpha4 And different versions of JDK The replacement of :
#!/usr/bin/env bashOLD=$1NEW=$2curl -sL https://github.com/shyiko/jabba/raw/master/install.sh | bash. ~/.jabba/jabba.shjabba uninstall $OLDjabba install $NEWjabba alias default $NEWsudo update-alternatives --install /usr/bin/java java ${JAVA_HOME%*/}/bin/java 20000sudo update-alternatives --install /usr/bin/javac javac ${JAVA_HOME%*/}/bin/java

   

Calling JDK Version management tools jabba When , You can use the following JDK value :
OpenJDK 8 Used Ubuntu Of apt Tool installation complete .
 
Here's in the benchmark , In different JDK Under the version ,java -version The output of :
  • jdk8
openjdk version "1.8.0_252"OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1~18.04-b09)OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)
  • jdk8 with Shenandoah
openjdk version "11.0.2" 2019-01-15OpenJDK Runtime Environment 18.9 (build 11.0.2+9)OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
  • jdk11 with Shenandoah
openjdk version "11.0.8-testing" 2020-07-14OpenJDK Runtime Environment (build 11.0.8-testing+0-builds.shipilev.net-openjdk-shenandoah-jdk11-b277-20200624)OpenJDK 64-Bit Server VM (build 11.0.8-testing+0-builds.shipilev.net-openjdk-shenandoah-jdk11-b277-20200624, mixed mode)
  • jdk14
openjdk version "14.0.1" 2020-04-14OpenJDK Runtime Environment (build 14.0.1+7)OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)

 
02 CMS
 
CMS (Concurrent Mark Sweep) The collector is currently Apache Cassandra Default garbage collector . Because it's in JDK 14 Has been removed from , So all the tests are based on JDK 8 or JDK 11 In progress .
 
The following settings are used for CMS The benchmark of :
 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSWaitDuration=10000 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:+CMSClassUnloadingEnabled -XX:ParallelGCThreads=8 -XX:ConcGCThreads=8 -Xms16G -Xmx16G -Xmn8G

  

Please note ,-XX:+UseParNewGC Arguments have been removed from JDK 11 Remove , After that, it becomes an implicit argument (implicit parameter). Using this argument will prevent JVM The start of .
 
We will CMS Maximum heap memory of (max heap) Restriction on 16GB, Otherwise it could trigger major collection A long pause .

 
03 G1
 
comparison CMS The collector ,G1GC( namely Garbage-First Garbage Collector, Garbage priority garbage collector ) It's easier to configure , Because it can dynamically adjust the size of the younger generation .
 
But G1GC More suitable for large heap memory (>=24GB)—— That's why it didn't become Cassandra The default garbage collector . in addition , Although its throughput is better than CMS Better , But it's more delayed than what's missed CMS It should be long .
 
The following settings are used for G1 The benchmark of :
 -XX:+UseG1GC -XX:G1RSetUpdatingPauseTimePercent=5 -XX:MaxGCPauseMillis=300 -XX:InitiatingHeapOccupancyPercent=70 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=8 -Xms31G -Xmx31G

  

For the sake of Cassandra 4.0 Benchmarking , We're executing G1 The test used JDK14.
 
We use 31GB The heap memory size of , So we can benefit from the compression index (compressed oops), And can have the most addressable objects with the smallest heap memory size .

 
04 ZGC
 
ZGC(Z Garbage Collector) yes JDK The latest garbage collector in , Its main focus is to put the world on hold (stop-the-world) The time delay is reduced to 10ms Within .ZGC It should also ensure that the heap memory size has no effect on the pause time , This allows its heap memory size to be expanded to 16TB.
 
If all the desired effects can be satisfied ,ZGC This makes it unnecessary to use off heap storage , And it can simplify Apache Cassandra Some development tasks of .
 
The following settings are used for ZGC The benchmark of :
 -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:ConcGCThreads=8 -XX:ParallelGCThreads=8 -XX:+UseTransparentHugePages -verbose:gc -Xms31G -Xmx31G

  

We need to use -XX:+UseTransparentHugePages As a flexible way to avoid Linux Enable large memory pages in (page).
 
Despite the official ZGC This could cause a surge in latency, according to the document , From the results of the test, it doesn't seem to happen . We might be able to use large memory pages for multiple throughput tests , So as to determine what impact this method will have on the benchmark results .
 
Please note ,ZGC Compression metrics cannot be used , But also not affected by “32GB Threshold value ” The limitation of . We are right ZGC In the test of , Use and in G1 The same thing in the test 31GB Heap memory of . So , In both cases, the free memory size of the system will be the same .

 
05 Shenandoah
 
Shenandoah It's a red hat (RedHat) Development of low latency garbage collector . stay JDK 8 and 11 in , It's ported backwards as a (backport) There is a version of . From Java 13 Start , It's contained in OpenJDK In the main line version of .
 
And ZGC The same thing , In most cases Shenandoah It's a concurrent garbage collector . The goal is that the pause time does not increase linearly with the increase of heap memory .
 
The following settings are used for Shenandoah The benchmark of :
 
 -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ConcGCThreads=8 -XX:ParallelGCThreads=8 -XX:+UseTransparentHugePages -Xms31G -Xmx31G

  

Shenandoah Compression metrics should be available , Therefore, we can benefit from the use ratio 32GB Smaller piles .

 
06 Cassandra 4.0 Of JVM To configure
 
Cassandra 4.0 The versions are Java 8 and Java 11 There are different jvm.options Archives . They are :
  • conf/jvm-server.options
  • conf/jvm8-server.options
  • conf/jvm11-server.options
If Cassandra From 3.11 Version upgrade to 4.0 edition , What was already there jvm.options Archives can still be used , As long as it's renamed jvm-server.options, And will jvm8-server.options and jvm11-server.options Just remove the file . But , It's not a recommended way .
 
The recommended way is to change the original jvm.options The settings in the file , Re apply to new jvm-server.options and jvm8-server.options On the file . These specific Java Options file (option file) Most of them are related to garbage collection arguments .
 
once jvm-server.options and jvm8-server.options The files are updated and in place , To configure jvm11-server.options Archives and from JDK 8 Switch to JDK 11 It's much easier .

 
07 Workload
 
In this benchmark, there are 8 A thread , And limit the reading and writing ratio to 80% Write /20% Read .tlp-stress Use a lot of asynchronous queries , This makes it as long as a limited number of thread pressure test, a careless may let Cassandra Node overload . In this load test , Each thread sends... Concurrently at one time 50 A query statement .
 
The key space in this test (keyspace) The replication factor of (replication factor) For 3, And all statements are at the consistency level LOCAL_ONE To execute .
 
For all garbage collectors and Cassandra Test version , The number of operations per second 25k、40k、45k、50k The growth rate of , So we can evaluate their performance under different stress levels .
 
Below tlp-stress Statement is used in this test :
tlp-stress run BasicTimeSeries -d 30m -p 100M -c 50 --pg sequence -t 8 -r 0.2 --rate <desired rate> --populate 200000

  

All the workloads execute 30 Minutes , Will 5 to 16GB The data are loaded into each node and the reasonable compaction load is considered .
 
Please note , The purpose of this test is not to evaluate Cassandra The optimal performance of , Because for different workloads , There are many ways to debug to get the best performance .
 
The purpose of this test is not to debug these garbage collectors , Although they have exposed many arguments and options that can improve performance for a specific workload .
 
What these benchmarks want to achieve is : Using most of the default settings and Cassandra With the same load , Provides a fair comparison of different garbage collectors .

 
 
08  Benchmark results
 
3.11.6 25k-40k ops/s
 
 
4.0 25k-40k ops/s
 
 
4.0 45k-50k ops/s
 
In terms of throughput ,Cassandra 3.11.6 Up to 41k ops/s, and Cassandra 4.0 It's as high as 51k ops/s. Both versions use CMS As a garbage collector , And the upgraded Cassandra 4.0 than Cassandra 3.11.6 We have achieved 25%.
 
4.0 A lot of performance improvements in this version can be used to explain this result , Especially about compaction (compaction) Heap pressure problem caused by operation ( Examine CASSANDRA-14654 As an example ).
 
stay Cassandra 3.11.6 In a cluster JDK 8 Of Shenandoah stay 40k ops/s In the load test of , Not only failed to achieve the highest throughput , There was also a query failure .
 
And by Cassandra 4.0 Clustering and JDK 11,Shenandoah It's a lot better —— Its maximum throughput in this case 49.6k ops/s, Almost able to catch up with 4.0 stay CMS The throughput of the next year has increased .
 
Use JDK 8 and Cassandra 3.11.6,G1 and Shenandoah On the whole, the throughput can only be up to 36k ops/s.
 
Use JDK 14 and JDK 11, The former seems to make G1 There's also a slight improvement in my performance —— From 47k/s Up to 50k/s.
 
Whether it's using JDK 14 Or JDK 11,ZGC We can't compete with other garbage collectors in terms of throughput , The highest can only reach 41k ops/s.
 
stay Cassandra 3.11.6 Conglomeration ,JDK 8 Of Shenandoah Under moderate load , It shows a very impressive low latency . But , Its performance will decrease with the increase of load pressure .
 
In the use of CMS In the case of ,Cassandra 4.0 The average of p99(99 Percentile ) Value lies in 11ms To 31ms Between , And the throughput is as high as 50k ops/s. Under moderate load , Read operation P99 Average value in Cassandra 3.11.6 Middle is 17ms, And in the Cassandra 4.0 And it's down to 11.5ms. In contrast , In terms of delay Cassandra 4.0 Yes 30% The promotion of .
 
Cassandra 4.0 In terms of throughput and delay 25% To 30% The promotion of , So when using the same garbage collector ,Cassandra 4.0 It's easy to beat Cassandra 3.11.6.
 
It is worth mentioning that , stay Cassandra 3.11.6 Conglomeration ,Shenandoah The latency is very low under moderate load , However, its performance under pressure makes us worry about its ability to handle sudden increase of load .
 
Although ZGC It shows a very impressive low latency under moderate load , Especially in use JDK 14 When , But its highest throughput is not equal to Shenandoah Match each other .
 
In almost all load tests ,Shenandoah The average latency of read and write operations p99 The values are all the lowest .Shenandoah Such a low delay plus it's in Cassandra 4.0 The throughput that can be achieved in , Make it a way to Cassandra 4.0 When upgrading , A garbage collector worth considering .
 
Under moderate load , The average latency of read operations p99 Only value 2.64ms It's quite impressive in itself . On this basis , If you know that this information is recorded by the client , You have to be right Shenandoah I look at it with new eyes .
 
In most cases ,G1 Maximum p99 Value corresponds to the maximum pause time it is configured for , namely 300ms. If you want to reduce the pause time of the target , You may not want to see the effect in the case of high load , It could even trigger a longer pause .
 
Under moderate load ,Shenandoah The average of p99 Delay can be reduced 77%, That is, the maximum delay is only 2.64ms. This can be a significant improvement for delay sensitive use cases —— Compared with Cassandra 3.11.6 Medium CMS, Read operation p99 The delay is greatly reduced 85%!
 
It is worth mentioning that ,JDK 14 Medium ZGC It has good performance under medium load , Unfortunately, it can't keep the same performance at a higher throughput rate . We are optimistic that ZGC It will be improved in the next few months , In the end, it may be enough to Shenandoah One up and one down .

 
09  reflection
 
G1 By removing the need to debug the size of different generations Cassandra Ease of use , however It's at the expense of certain efficiency .
 
New issue Apache Cassandra 4.0 It's a very impressive performance enhancement , It will allow the use of things like Shenandoah and ZGC This new generation of garbage collectors . These collectors are easy to use , It doesn't take much subtle debugging , And it's more efficient on delay .
 
If you use Cassandra 3.11.6, It's hard for us to recommend Shenandoah, Because Cassandra The performance of nodes under high load is not satisfactory . But from JDK 11 and Cassandra 4.0 Start ,Shenandoah There's an amazing improvement in latency , At the same time, it can support almost Cassandra The maximum throughput that a database can provide .
 
Since this benchmark focuses on specific workloads , Your test results may vary depending on the situation . But for delay sensitive use cases , The results of this test let us know Apache Cassandra I feel quite optimistic about my future , Because it will bring us more than Cassandra 3.11.6 A big improvement in .
 
Download the latest Apache 4 And try it yourself . If you have any feedback , Remember to use social email or ASF Slack Contact us . Click here to view our contact information .

版权声明
本文为[itread01]所创,转载请带上原文链接,感谢
https://chowdera.com/2021/01/20210124021047757y.html

随机推荐