[Mondrian] Dimension member caching

michael.pflug at thomsonreuters.com michael.pflug at thomsonreuters.com
Thu Aug 14 13:03:53 EDT 2008


Thanks Mark.  After some more discussion we agree that the best way to
achieve our goal of never caching a dimension would not fall in line
with 10788 because, as you pointed out, 10788 is intended to minimize
dimension table reads, not force them.  Our current thought is that we
can implement a new method in the CacheControl API that takes a
hierarchy, gets a MemberCacheHelper and calls MemberCacheHelper.flush()
to flush the appropriate hierarchy.  Does this sound like a valid
solution?  

 

Thanks,

Mike

 

 

________________________________

From: Marc Berkowitz [mailto:marcb52 at speakeasy.net] 
Sent: Saturday, August 09, 2008 12:25 AM
To: Mondrian developer mailing list
Cc: Pflug, Michael (TH USA)
Subject: Re: [Mondrian] Dimension member caching

 

(I wrote this code -- and am still fixing it -- so I'll try to answer
these questions.)

michael.pflug at thomsonreuters.com wrote: 

I plan to utilize the dimension member caching feature implemented
with check in 10788 but I have a few questions I'm hoping the community
can help me out with before I get too far:   
 
1)  The feature is dependant on having the property
mondrian.rolap.EnableRolapCubeMemberCache=false, but I don't fully
understand the consequences of setting this property to false.  Can
anyone elaborate? 
  

This is a tricky point;  my own understanding may be warped, but I see
it like this:
Usually a schema defines dimension members by referring to a dimension
table. Mondrian creates a RolapMember object when it is needed, reading
its attributes from the dimension table. The resulting object is saved
in a cache so it can be found again, avoiding reading from the table
again. The cache also guarantees there is a unique RolapMember for each
member, and the cache has "indexes" to find related members (parent,
children, level-peers) in constant time. It is a soft cache - unused
members can be dropped from the cache for garbage collection -- 
and this is valid because should the member be needed again, it will
just be read from the table and put again into the cache.

This design assumes that the dimension table doesn't change. My new
feature helps deal with a dimension that changes in known small ways,
letting an application edit selected items in the cache, or flush them
from the cache to be reloaded with new values.

However, checkin 10203 of last november changed the cache design,
introducing a two level cache. The goal was to deal with shared members
in a shared dimension,  used by several cubes. Now a member is
represented by both a global RolapMember (for all cubes) and a local
RolapCubeMember (one for each relevant cube). 
The cache structure is much more complicated -- so it seemed risky to
alter it to add the new features. Therefore I implemented them to act on
the simpler, single level member cache.
The property EnableRolapCubeMemberCache was added so an application can
choose the old one-level cache (which supports my cache edits), or the
newer two-level cache
(which does not, but offers the advantages of checkin 10203). I assume
that 10203 pays off for shared dimensions, and has no advantage
otherwise.




 
2)  In the unit test MemberCacheControlTest.testDeleteCommand(), the
test allows two possible outcomes - one that flushes only the desired
member and one that flushes the entire level.  Walking through the test
it always seems to flush the entire level which is arguably less
desirable. Does anyone understand this behavior?  
 
  

Flushing more than requested is certainly sub-optimal, but strictly
speaking it's not incorrect, since mondrian might flush the member cache
spontaneously to reclaim space.



3)  The code specifically does not allow flushing of parent-child
dimensions and, as luck would have it, the dimension I am most
interested in flushing is parent-child.  There is no comment as to why
parent-child isn't allowed - can anyone fill me in on what the
difficultly might be in implementing this?
  

I avoided parent/child hierarchies because they have a more complex
structure in the member cache. The existing design of member-cache edits
might accommodate them, or it might require a complete rewrite. I'd have
to try it and see.  However I think a refactoring of the member cache is
likely in the near term, so it seems easier and maybe faster to wait for
that, rtsher than do a rewrite.



 
4)  Finally, my specific use case is that I have a volatile dimension
that I wish never to be cached.  I could clear the cache manually before
each query, but I'd rather specify an optional property in my schema
telling Mondrian to never cache the dimension.  Does anyone have any
objections to that implementation

No, but that's a different goal to  10788, which is intended to minimize
dimension table reads, not force them.
And it seems useful and (probably) simpler to implement than
member-cache-edits!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20080814/bf2dcfcb/attachment.html 


More information about the Mondrian mailing list