memory - Coldfusion JVM heap vs RAM usage

I need help clarifying memory usage on a CentOS server running Coldfusion/Apache.

I've got a script that reads a file into memory and process it line by line, inserts into DB and deletes the file. Not the best process, working on it, but it'll have to do for now.

When the file is read it takes up loads of memory, it was causing the server to use the swap file and it was really unresponsive - total RAM usage in TOP was 3.9/3.9 GB and swap was 1.5/1.9

I had to reboot it, total used RAM dropped to about 2GB, I ran the script and it went back up to 3GB. It's slowly increased over the last few hours to 3.3GB - I've still got some space and the sites are all responsive but memory isn't being freed.

Also since rebooting, FusionReactor shows my Max and Allocated Heap as 3.93GB, and Used ranging from 0.3GB to 1GB - which is inconsistent with what TOP says, which is CF9 on 58% of 3.9, which is around 2.25GB

The script imports data overnight, so I'm guessing when it runs later, it'll spill into the swap and slow the site down overnight (low traffic overnight but still needs fixing - I'll have to disable for now)

So the 2 things I don't understand:

• why isn't the RAM dropping back from 3 -> 2 when the file has been read, and the script has finished (missing a fileClose - too many CFCs?)


• why does my FusionReactor Memory Used Heap show a max of 1GB, when TOP shows 58% of 3.9GB (2.25GB) for CF9

UPDATE - jvm.config, less most of it's comments

# Where to find JVM
java.home=/opt/coldfusion9/runtime/jre/


# Arguments to VM
java.args=-server  -Djava.awt.headless=true -Xms4096m -Xmx4096m 
    -Dsun.io.useCanonCaches=false -XX:ParallelGCThreads=2 -XX:PermSize=64m 
    -XX:+UseConcMarkSweepGC -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:SurvivorRatio=4
    -XX:+UseParNewGC -XX:MaxPermSize=192m -Dcoldfusion.rootDir={application.home}/../ 
    -Dcoldfusion.libPath={application.home}/../lib -Dcoldfusion.classPath=
    {application.home}/../lib/updates,{application.home}/../lib,
    {application.home}/../gateway/lib/,{application.home}/../wwwroot/WEB-INF/flex/jars,
    {application.home}/../wwwroot/WEB-INF/cfform/jars -
    javaagent:/opt/fusionreactor/instance/coldfusion.CF9Standalone.cc02/                 

    fusionreactor.jar=name=coldfusion.CF9Standalone.cc02,address=8088

# commas will be converted to platform specific separator and the result will be passed
# as -Djava.ext.dirs= to the VM
java.ext.dirs={jre.home}/lib/ext

# where to find shared libraries
java.library.path={application.home}/../lib
system.path.first=false


# set the current working directory
java.user.dir={application.home}/../../lib

# JVM classpath
java.class.path={application.home}/servers/lib,{application.home}/../lib/
    macromedia_drivers.jar,        
    {application.home}/lib/cfmx_mbean.jar,{application.home}/../lib/oosdk/classes,
    {application.home}/../lib/oosdk/lib,{application.home}/lib

Answer

Your heap settings configure 4gigs not 2 - that's what the Xms4096m -Xmx4096m are telling you. Perm size adds another 200 megs. your New size (the amount that young gen grabs onto in a single operation) is very high - usually this number is between 64m and 256m - giving generation space to roam up and down the young heap.

So the fact that you see 3.9g active is correct - although I would expect to see something like 4.2. Still Top sometimes does odd math with JVM allocations.

You are correct that ColdFusion/Jrun read the file into memory (into the heap) as a whole - so your heap needs to be large enough to accomodate that. Still, even a small file if it triggers a young generation will result in a full gigabyte of allocation because of your newsize param.

If you only have 4 gigs total on the server my suggestion would be that you set newsize and maxnewsize to 256m, and set your heap to 3072m like so Xms3072m -Xmx3072m

These settings apply to a 64 bit CF 9 server - if you are running 32bit then you have the hard limitations of around 2 gigs for the heap due to the contiguous mem problem native to all 32 bit platforms. Good luck.