[Mondrian] Using aggregate tables uses too much memory

Julian Hyde jhyde at pentaho.com
Tue Jan 10 20:57:25 EST 2012


It's not what you think -- running out of memory because of extra data in memory. Mondrian stores data from agg tables in its cache in the same way that it stores data from other tables.

So, there is some other memory problem. If the only thing you have changed is to enable agg tables, maybe it is connected to the metadata for those agg tables. At this point I can't say for sure that it is even a Mondrian issue.

A couple of things you could do to self-diagnose:

You could try 'jmap -histo' to dump the heap with and without agg tables and see whether there is any significant difference in the number of objects.

You could also asking for several JVM thread dumps at random times as the system gets slower and slower. Maybe there is an inefficient algorithm that is burning both memory and CPU (and therefore will show up on some of the thread dumps).

Julian


On Jan 10, 2012, at 2:07 PM, Monir Hossain wrote:

Dear Mondrian Developer,

I have a slight problem. I am trying optimize Mondrian's performance via the use of aggregate tables. The volume of data we have is pretty large. After enabling the use of aggregate table with about 25 or so aggregate tables, the system comes to halt and crashes due to "out of memory". I gave it 2 GB of ram (that's all i have to work with for now). When not using the aggregate table it works fine, slow but it works and uses at most half of it's maximum memory.  I was wondering if you guys could shed some light on how it is Mondrian caching aggregate tables. I would also appreciate any other helpful tips on aggregate/performance tuning.

P.S. Currently we have about 3GB database but it's not even .1% of what we'll have.



Thanks,

Monir Hossain
Software Engineering, Rochester Institute of Technology _______________________________________________
Mondrian mailing list
Mondrian at pentaho.org<mailto:Mondrian at pentaho.org>
http://lists.pentaho.org/mailman/listinfo/mondrian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20120110/58dae08f/attachment.html 


More information about the Mondrian mailing list