Reasoning
In preparation for the Fall 2006 semester UW Madison was looking at ways to improve portal performance. The initial effort was focused at tuning the JVM heap for the way uPortal uses memory.
Preparation
Reading
Before starting to make changes to the heap config following white papers were read to understand the options available for the heap and garbage collectors.
- Java Tuning White Paper
- A general overview of tuning strategies for the JVM.
- Tuning Garbage Collection with Java 5
- Details about the GC options for Java 5.
- Understanding Concurrent Mark Sweep Garbage Collector Logs
- Provides good explanations of all that info the GC logs.
Testing
We used a jMeter load test script to test each change made to the JVM configuration. The test consisted of the script logging into the portal, visiting one other tab (chosen at random) and logging out. Each test latest 2 hours and was capped at a throughput of 2 requests per second. The throughput cap only applied to the initial view of the home tab after login and the random tab visit.
Break down of Configuration
# Use the Hot Spot server compiler
-server
# Set the initial & max heap size to the same value. This makes monitoring heap usage a bit easier
-Xms1280m
-Xmx1280m
# Set the NewSize and MaxNewSize (space for eden and survior) to half of the max heap. uPortal creates a lot of temporary objects, the large NewSize provides enough space for the objects to be created, used and die between GCs on the eden heap. This results in less of a need for objects in eden to be copied to survior. DO NOT set this to more than half of the max heap or the GC can't fulfill the Young Generation Guarantee which will cause constant full GCs
-XX:NewSize=640m
-XX:MaxNewSize=640m
# Set the survivor heap ratio. To determine the size of your eden and survior spaces use the following forumula "SurviorSize = MaxNewSize / (SurvivorRatio + 2)" and "EdenSize = SurviorSize * SurvivorRatio". Remember that there are two Survivor spaces and one Eden space.
-XX:SurvivorRatio=5
-XX:TargetSurvivorRatio=90
-XX:MaxTenuringThreshold=12
-XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing
-XX:+CMSParallelRemarkEnabled
-XX:+UseParNewGC
-XX:PermSize=64m
-XX:MaxPermSize=64m
-XX:+UseTLAB
- Enable class unloading (needed with ConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled
- debug and memory tweaks to avoid Hotspot Compiler Failure
-XX:+PrintCompilation
-XX:CodeCacheMinimumFreeSpace=2M
-XX:ReservedCodeCacheSize=64M
-XX:CompileCommandFile=/my/portal/bin/hotspot_compiler
- Enable JMX Remote Monitoring
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9000
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/my/portal/bin/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=/my/portal/bin/jmxremote.access
- turn on some debug for GC
-verbose:gc
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-XX:+PrintTenuringDistribution
-Xloggc:/my/portal/logs/portal/gc.log
- Enable remote debugging port
-Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n