[Mondrian] multiple schemas? cache management?

Pappyn Bart Bart.Pappyn at vandewiele.com
Wed Dec 5 04:21:08 EST 2007


Hi Ati,
 
I have a real world application that is running good at more than 15
sites for now.  My database has two fact tables, both identical in
layout.  One contains the hot data (today), the other one the archived
data.  My system is a sort of real-time olap (data is updated all the
time, but only reflected every 5 minutes in the hot fact table).  My
database is updated using transactions.
 
The first is implemented in a cube with all cache turned off and does
not have aggregate tables.  The second one has cache turned on and has
aggregate tables.  Both are combined in a virtual cube, both share the
same dimensions.
 
The aggregate tables are calculated at night.  I use the
DataSourceChangeListener to check whether an aggregation table is
changed.   I use RolapSchema.clearCache(); to flush the cache after all
aggregation tables are done.  I flush dimension cache using the
DataSourceChangeListener, in that case, there is no need to clear the
full cache.  I have a special table in my database containing the name
of the table and the time the aggregation is done.  This way I can
communicate using the DataSourceChangeListener and the database without
having to know about the ETL process that is running in C++.
 
I use the previous version of mondrian (2.3.2), I know the function
RolapSchema.clearCache() was deprecated and I assume it no longer exists
in the latest version.  I don't see the benefit of only flushing certain
parts of the cube, since a) I don't know what to flush b) Almost
everything is changed.  Also I assume that dimension cache is still not
flushable (I might be wrong about that), besides using
DataSourceChangeListener.
 
There is a period at night where results might be wrong using this
approach, but since at that time, my users only check data from today
(monitoring purpose), there are no huge problems.
 
The performance is ok (2-10 sec for a report), the database is 5 - 10 GB
for 1 year data.  
 
I am using this approach for over a year now, and is actively running on
multiple sites, without any problems.
 
While I developed DataSourceChangeListener for the 2.3.2 release, I do
not have the time for the moment to check if it is still working in the
latest release.  I had to stop taking the latest version of mondrian,
because my application is using almost all features that mondrian
contains.  And every build of mondrian broke something in my
application.  So maintaining DataSourceChangeListener was a never ending
story and I had no time left for developing my own application.
 
There are multiple users that are using the system at the same time, I
also had to develop special features for mondrian for this (for cache
flushing and real time results). Not sure if they are still in place in
the latest release.
 
Kind regards,
Bart

________________________________

From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
On Behalf Of Ati Rosselet
Sent: dinsdag 4 december 2007 18:44
To: Mondrian developer mailing list
Subject: [Mondrian] multiple schemas? cache management?


In previous versions of mondrian, I could just provide a different
schema with a query (notably through jpivot) and the data for that
schema would be loaded and cached... seperately.  This was important
because we had multiple schemas in different files, each with their own
set of roles etc .. and it worked fine keeping them seperate.  To my
suprise, when we ported to the new mondrain, it seems to just cache the
FIRST schema file we provide, and ignore any others... thus of course
leading to the hierarchy/dimension/member not found errors.  My question
is simply... what changed.. and is it still possible to use multiple
schemas via jpivot especially.... 

a seperate question - we're having a tough time getting our brains
around the new cache invalidation - is there any way to just say.. ok...
reload everything (or pertaining to a certain cube??)?   We have to
reload all the aggregate tables using a procedure anyways, so pretty
much everything is affected.. or can be... is there no way to just say
cache.clear().. or something?  Does anyone have any real life examples
of cache management they could share?
Cheers
Ati



______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20071205/34152e20/attachment.html 


More information about the Mondrian mailing list