[Mondrian] RE: Eigenbase perforce change 12887 for review

Eric McDermid mcdermid at stonecreek.com
Fri Jun 19 16:20:33 EDT 2009


On a similar note, there are a couple of changes in the code I'm  
porting up that significantly reduce memory usage (again, this is code  
I didn't write, so I can't give precise characterizations at the  
moment).

1) Substituting of Flat5Map (a simple variation of Apache commons'  
Flat3Map) for HashMap in to map property names to values in  
RolapMember (mapPropertyNameToValue).  I don't have exact numbers, but  
my understanding is that this dramatically improves the memory  
footprint at the cost of a slight performance slowdown where there are  
< 6 properties.  (If the performance impact is a concern, it's easy to  
make configurable.)

I'm not sure about packaging/licenses on this one -- since Flat5Map is  
a derivative of Apache's Flat3Map, I assume it must be delivered under  
the Apache license.  I don't think you can simply repackage and  
distribute Apache code under the EPL, so my guess is I'd need to find  
a place to put it under the Apache license, package it in a separate  
jar and then reference it in?

If that's too much hassle, it might be easier to just change Mondrian  
to use Flat3Map, and continue to use Flat5Map just in my client's  
private version of the code.

2) Use an Apache commons LRUMap in SqlMemberSource to allow  
makeMember(...) to reuse property value objects rather than store  
multiple references to identical ones.  In other words, if there are  
10000 members all of which have the same 3 property values strings, we  
want memory allocated just for the 3 value strings, rather than 3 *  
10000 of them.

I'm a little wary of the latter optimization, since I'm not sure if  
there are instances where a property value can be something other than  
a String/immutable object.  Anyone know definitively one way or the  
other?

  -- Eric


On Jun 19, 2009, at 11:02 AM, Peter Tran wrote:

> I haven't looked into Apache commons-primitives. Before these  
> available, we made use of Colt libraries map features:
>
> http://acs.lbl.gov/~hoschek/colt/api/cern/colt/map/package- 
> summary.html
>
> They made a huge difference in the GC time.  I would say yes, it was  
> definitely worth the effort.
>
> -Peter
>
> -----Original Message-----
> From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org 
> ] On Behalf Of Julian Hyde
> Sent: Friday, June 19, 2009 11:50 AM
> To: 'Mondrian developer mailing list'
> Subject: [Mondrian] RE: Eigenbase perforce change 12887 for review
>
> By the way, we use lists of integers, maps to and from integers,  
> maps to
> doubles, all over the code, and in some performance-critical areas.  
> Java
> autoboxing makes it easy to forget that every access to these  
> structures
> involves converting a primitive type to an object, often allocating an
> object for the purpose.
>
> I've been thinking of using apache commons-primitives for these  
> structures.
> (See
> http://commons.apache.org/primitives/apidocs/org/apache/commons/collections/
> primitives/IntList.html for instance.) Anyone have any experience  
> with these
> classes? Is it worth the effort?
>
> Julian
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>





More information about the Mondrian mailing list