[Mondrian] Re: Eigenbase perforce change 8645 for review

Richard Emberson remberson at edgedynamics.com
Mon Feb 5 09:51:45 EST 2007


I suspect that because Derby uses more memory, the queries
actually come near to out-of-memory.
The memory monitor is set to trigger when memory used
is 90% of the maximum memory
(mondrian.util.memoryMonitor.percentage.threshold=90 by
default).
It triggers after garbage collection, from the
java.lang.management.MemoryPoolMXBean documentation for the
setUsageThreshold method:

* Sets the threshold of this memory pool to the given <tt>threshold</tt>
* value if this memory pool supports the usage threshold.
* The usage threshold crossing checking is enabled in this memory pool
* if the threshold is set to a positive value.
* The usage threshold crossing checking is disabled
* if it is set to zero.

Note, I use percentage rather than absolute memory size while the
above Java5 API uses absolute memory size.
The memory "usage" is how much is actually being used.

Setting the property value to 0% is the same as setting the
usage threshold to zero.

I could be that 64M is too small when using Derby if one wishes
to use 90% as the notification limit.

As a general note, whether to set it at 90% or 95%, etc,
depends upon how much memory can possibly be consumed between
calls to Query.checkCancelOrTimeout. Writing code that is
a good Java5-memory-monitoring citizen will take some time
to figure out the do's and don'ts.
For example, both StringBuffer and StringBuilder are NOT
good citizens, they both use the AbstractStringBuilder.expandCapacity
method which takes the current length of the internal
array of chars and doubles it - this is not an incremental
grow strategy, rather its a undisciplined memory
allocation. For large StringBu*ers there is no chance of the
memory monitor to notify the client if they have to grow.
A good citizen String-maker would use an array of arrays of
chars and slowly grow the number of small (say, 50k) arrays
it uses when pushed to hold very large strings.

Richard


John V. Sichi wrote:
> Another problem with this checkin.  When I run "ant test" with Derby on 
> Linux and Sun JDK 1.5 (which defaults to 64M max heap), lots of tests 
> error out with MemoryLimitExceededException.  Are you sure it isn't 
> somehow checking *before* garbage collection?
> 
> When I set mondrian.util.memoryMonitor.enabled=false, all is well again.
> 
> JVS
> 
> John V. Sichi wrote:
>> Richard, looks like you accidentally clobbered Zelaine's addition of 
>> the IterationLimit parameter description when you merged?
>>
>> JVS
>>
>> Richard Emberson wrote:
>>> http://p4web.eigenbase.org/@md=d&c=6PU@//8645?ac=10
>>>
>>> Change 8645 by emberson at bortei.head on 2007/02/03 15:59:41
>>>
>>>     MONDRIAN
>>>        Added ObjectFactory and Java5 memory monitoring support.
>>>        Now when using Java5, if a query is using "too much" memory
>>>        a MemoryLimitExceededException is thrown rather than
>>>        an OutOfMemoryError.
>>> 281,288d284
>>> <       <td><a 
>>> href="api/mondrian/olap/MondrianProperties.html#IterationLimit">
>>> <       <code>mondrian.rolap.IterationLimit</code></a></td>
>>> <       <td>int</td>
>>> <       <td>0</td>
>>> <       <td>If > 0, the maximum number of iterations allowed when 
>>> evaluating an
>>> <         aggregate.  If 0, there is no limit.</td>
>>> <     </tr>
>>> <     <tr>
>>
> 
> 


-- 
Quis custodiet ipsos custodes:
This email message is for the sole use of the intended recipient(s) and
may contain confidential information.  Any unauthorized review, use,
disclosure or distribution is prohibited.  If you are not the intended
recipient, please contact the sender by reply email and destroy all
copies of the original message.



More information about the Mondrian mailing list