Java memory improvment since Java 8

Dumping des informations mémoires.
Une JVM configuré pour 7Go en prend 13G en mémoire.

Ajout du flag :

-XX:NativeMemoryTracking=summary 

Attention impact un peu les performances.

Résultat, la différence était dans "Symbol" : Here is the NMT report about the symbol allocations, such as the string table and constant pool.

Et en effet cette JVM manipule beaucoup de chaines qu'elle garde en mémoire qui pourrait être interned

https://stackoverflow.com/questions/10578984/what-is-java-string-interning

 http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#intern()

 Basically doing String.intern() on a series of strings will ensure that all strings having same contents share same memory. So if you have list of names where 'john' appears 1000 times, by interning you ensure only one 'john' is actually allocated memory. 

Heap dump pour la suite.

-XX:NativeMemoryTracking=summary

which produces an overview of the memory usage by the components of the JVM. It actually gives a pretty good picture of the "cost" of having a JVM.

Enabling detailed native memory tracking (NMT) causes a 5% to 10% performance overhead. The summary mode merely has an impact in memory usage as shown below and is usually enough.
It is necessary to note that while the above command indicate a scale in KB for the JVM it really means KiB.

JVM native memory tracking report

$ jcmd $(pidof java) VM.native_memory

Example of output :

Native Memory Tracking:
Total: reserved=7168324KB, committed=5380868KB                               
-                 Java Heap (reserved=4456448KB, committed=4456448KB)        
                        (mmap: reserved=4456448KB, committed=4456448KB)

-                     Class (reserved=1195628KB, committed=165788KB)         
                        (classes #28431)                                 
                        (  instance classes #26792, array classes #1639)
                        (malloc=5740KB #87822)
                        (mmap: reserved=1189888KB, committed=160048KB)
                        (  Metadata:   )
                        (    reserved=141312KB, committed=139876KB)
                        (    used=135945KB)
                        (    free=3931KB)
                        (    waste=0KB =0.00%)
                        (  Class space:)
                        (    reserved=1048576KB, committed=20172KB)
                        (    used=17864KB)
               ....

Java memory dump on outofmemory

-XX:+PrintGCDateStamps
-verbose:gc
-XX:+PrintGCDetails
-Xloggc:"<path to log>"
-XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=<path to dump>`date`.hprof

-XX:+PrintGCApplicationStoppedTime
-XX:+PrintGCDetails
-XX:+PrintSafepointStatistics
-XX:PrintSafepointStatisticsCount=1
-XX:+SafepointTimeout
-XX:SafepointTimeoutDelay=<ms before timeout log is printed>

Réduire l'usage du swap

cat /proc/sys/vm/swappiness

#/etc/sysctl.conf 
vm.swappiness = 10

swapoff -a
swapon -a

sysctl vm.swappiness
10
sysctl vm.swappiness=10
sysctl -w vm.swappiness=10

Consommation mémoire par user (RSS resident). Remplacer par $5 pour avoir la taille virtuelle.

for USER in $(ps haux | awk '{print $1}' | sort -u); do     ps haux | awk -v user=$USER '$1 ~ user { sum += $6} END { print user "    \t" sum" ko" "\t" sum/1024" Mo" "\t"sum/1024/1024" Go"; }'            ; done

VSZ is the Virtual Memory Size
RSS is the Resident Set Size (swap non compté)

So if process A has a 500K binary and is linked to 2500K of shared libraries, has 200K of stack/heap allocations of which 100K is actually in memory (rest is swapped), and has only actually loaded 1000K of the shared libraries and 400K of its own binary then:

RSS: 400K + 1000K + 100K = 1500K
VSZ: 500K + 2500K + 200K = 3200K

Java heap dump

Java :

jmap -dump:format=b,file=<logdump>/dump1.hprof <pid>

Jrockit :

jrcmd <pid> hprofdump filename=<logdump>/dump1.hprof

Theadump stack :

PID=12345
for ((a=1; a <= 60; a++)); do jstack $PID > thread-dump.$PID.$a; sleep 5; echo $PID.$a; done