[Mondrian] Flushing SegmentCache

[Pedro Alves]

Don't get me wrong - we *will* migrate this to the new code. I actually
thought this was going to be 4.0 timeframe but I guess we need to speed
things up.


However, this code is for CDC, the next member of the ctools we'll be
launching hopefully this week. And as you know, the ctools are available to
any existing pentaho installation. Our goal would be to maintain, at least
as much as possible, compatibility with some existing pentaho versions, on
this case with some mondrian versions.


On this specific case, the distributed cache part is only available on
mondrian 3.e (pentaho 3.10, whatever marketing version that is), but at
least on that I wouln't want it to be feaure crippled :(


-pedro



On Monday, February 13, 2012, Luc Boudreau <lucboudreau at gmail.com> wrote:
> I'm saying we have found some issues with the cache management code in
> 3.3.X, but it should still work (except a few corner cases). Please
> log a Jira case with as much information and reproducible tests as you
> can so we can look into your issue. Regardless of the fact that it
> works on 3.3 or not, I'd like to make sure we fixed the problem for
> the next release.
>
> Also, just a heads up. We have modified the SegmentCache SPI and
> simplified it greatly for the next release. For one thing, you won't
> need to implement the SPI using Java Futures, nor will you need to
> implement SegmentCache.flush() on the SPI side (Mondrian will take
> care of that). I strongly suggest that you take a look at the next
> version of the SPI; it might make your life that much easier.
>
>
http://perforce.eigenbase.org:8080/@rev1=head@//open/mondrian/src/main/mondrian/spi/SegmentCache.java
>
> Please also note that the next version of the SPI includes a
> SegmentCacheListener system. Your cache implementation should make use
> of it for optimal performance. Mondrian will call
> SegmentCache.addListener() at runtime and will register a listener
> object with your implementation. This makes Mondrian much more
> efficient in maintaining an index of the contents of the external
> caches by using deltas to figure out how to sync up the segment
> indexes. Listeners and near-real-time updates of the indexes helps
> Mondrian perform in-memory rollups of your cell data (this is an order
> of magnitude more efficient than SQL access). I know, it sounds pretty
> hermetic and complicated. We still haven't had the chance to write a
> neat blog post to explain it all yet.
>
> Bottom line: I suggest you invest the little time necessary to develop
> your code against 3.4.0. The new & improved SegmentCache SPI is most
> likely the one we will support long-term wise. Version 3.4.0 is almost
> out of the door, so the transition should be as smooth as can be.
>
> Cheers!
>
> Luc
>
>
> On Mon, Feb 13, 2012 at 3:07 PM, Pedro Alves <pmgalves at gmail.com> wrote:
>>
>> This is for a plugin we want to release for 3.3 also, earlier if
possible.
>>
>> Are you saying that it's not possible to do selective cache clearing at
all
>> :-(
>>
>>
>>
>> -pedro
>>
>>
>> Luc Boudreau <lucboudreau at gmail.com> wrote:
>>>
>>> For our next release (due some time in March) we have modified
>>> substantially the code which manages segments and cell data. I remember
>>> seeing that issue, so I suggest you grab the latest binary build off
our CI
>>> server and see if it is gone.
>>>
>>> If not,  please log a jira case (with a test case on Foodmart
preferably).
>>>
>>> Luc
>>>
>>> On Feb 13, 2012 2:26 PM, "Tiago Ferreira" <tiago.ferreira at webdetails.pt>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm using a custom SegmentCache implementation in mondrian 3.3.
>>>> Using cacheControl.flush(cacheControl.createMeasuresRegion(cube)) as in
>>>> the examples in
http://mondrian.pentaho.com/documentation/cache_control.php
>>>> for flushing a cube, my SegmentCache's flush method always gets
invoked with
>>>> an empty ConstrainedColumn[] and segments aren't removed.
>>>> Any clues on what I might be doing wrong (or not getting)?
>>>>
>>>> Thanks
>>>>
>>>> --
>>>> Tiago G.F.
>>>>
>>>>
>>>> _______________________________________________
>>>> Mondrian mailing list
>>>> Mondrian at pentaho.org
>>>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20120213/96f189f4/attachment.html