<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<STYLE type=text/css>DIV {
MARGIN: 0px
}
</STYLE>
<META content="MSHTML 6.00.5730.11" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>I was also thinking of moving to ThreadLocal, I will take
over your modifications as soon as they</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>are checked in - thanks.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>I did develop some new features in the mean while and added
the possibility to attach a data source</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>listener to a star. </FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>Because there are other multi-user related problems with
loading of aggregates, I designed a new principle.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>Now agg cache is checked before a query is run. If
any aggregation has changed due to changes in the database,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>a new - thread local - aggregation is made. The
thread will fill this aggregation. Any other threads that where
using</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>global (non thread local) cache, will not be interfered
(what was the case in the past). After the thread has
finished,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>it will try to move the local cache to the global cache in
case no other threads are using the aggregation.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>A time of creation is maintained, so only the latest
changes are put into global cache.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>If the thread has cache turned off, then the principle that
is currently in place, stays the same, thread local cache</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>is flushed after the query has
finished.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>Later on, if transactions are put in to place (I will not
do this in the upcoming patch), I think multi-user access</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>and data integrity will be much better.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>The only problem - up to now - is the hierarchy cache,
since it does not follow the execution of a query, but</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>I am thinking of applying the same principle of that of the
agg cache.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=828310811-23012007><FONT face=Arial
color=#0000ff size=2>Bart</FONT></SPAN></DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> mondrian-bounces@pentaho.org
[mailto:mondrian-bounces@pentaho.org] <B>On Behalf Of </B>Julian
Hyde<BR><B>Sent:</B> dinsdag 23 januari 2007 11:57<BR><B>To:</B> 'Mondrian
developer mailing list'<BR><B>Subject:</B> RE: [Mondrian]
Re:VirtualCubeTest.testCalculatedMemberAcrossCubesfailing on
SMP<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>I think the problem is with how mondrian evaluates members
using multiple passes. When the measures are coming from a virtual cube, of
course there are multiple real cubes, and each of those has a cell reader. But
the code in RolapResult assumes there is only one cell
reader.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>Mondrian should check the cell readers for all applicable
cubes, and only emit a result when all cell readers have been
populated.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>I haven't implemented the fix yet, but this cause seems
very plausible to me.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>I'm not exactly sure why this problem surfaced after Bart's
change - maybe thread-local caches increased the chances of one cache being
populated and another not - or why it appears on SMP
machines.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>By the way, in an effort to get this working, I removed
Bart's RolapStarAggregationKey (a compound key of BitKey and thread id) and
moved to a two-tier hashing scheme. The first tier is a ThreadLocal of maps, and
the second tier is a map. Threads which want access to the global map just skip
the first tier. Given the difficulties obtaining a unique id for a thread, using
a ThreadLocal seemed cleaner. So, even though this didn't fix the bug, I'm going
to check in.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>Julian</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000080 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> mondrian-bounces@pentaho.org
[mailto:mondrian-bounces@pentaho.org] <B>On Behalf Of </B>michael
bienstein<BR><B>Sent:</B> Monday, January 22, 2007 12:06 PM<BR><B>To:</B>
Mondrian developer mailing list<BR><B>Subject:</B> Re : [Mondrian] Re:
VirtualCubeTest.testCalculatedMemberAcrossCubesfailing on
SMP<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman,new york,times,serif">I've
seen issues with server mode JIT before related to memory boundaries and
multiple threads. But that's mutiple threads and it was in JDK 1.4 (the
memory model changed in 1.5 I think). The issue is that the instructions
in the Java code can be run out of order to the way you've coded them.
E.g. a=1; b=2; a=b; can be run just a=2; b=2; because that's what it is
equivalent to. The only way to force it to do what you really expected
is to synchronize your accesses because that prevents the instruction
re-ordering across the memory boundary. This was an issue in Apache
Struts at one point because they used a custom Map implementation called
"FastHashMap" which gets filled with values and then flipped to be in
immutable mode. The problem was that the get() method tested if it was
flipped already without synchronizing which looked safe because the flip flag
was set only after the insertion code. But the JIT reversed the order
and the flip was done before the last insertions leading to certain problems
on high-end servers.<BR><BR>All that's a moot point if we can't see how
multiple threads are being used.<BR><BR>Michael<BR><BR>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman,new york,times,serif">-----
Message d'origine ----<BR>De : John V. Sichi <jsichi@gmail.com><BR>À :
Pappyn Bart <Bart.Pappyn@vandewiele.com><BR>Cc : Mondrian developer
mailing list <mondrian@pentaho.org><BR>Envoyé le : Lundi, 22 Janvier
2007, 20h10mn 24s<BR>Objet : [Mondrian] Re:
VirtualCubeTest.testCalculatedMemberAcrossCubes failing on SMP<BR><BR>
<DIV>Something interesting: I noticed that HotSpot was
automatically <BR>selecting Server mode on the SMP machine (whereas on my
laptop it <BR>autoselects Client mode). I changed build.xml to
force usage of Client <BR>mode, and then "ant test" ran through with no
failures.<BR><BR>Haven't tried forcing Server mode on laptop yet; if it's
timing-related, <BR>it may just be that it takes the combination of a very
fast machine plus <BR>Server mode to hit it.<BR><BR>Also, Bart, if you want to
send me code modifications to enable tracing <BR>targeted at debugging the
problem, please do, and I'll run it through on <BR>the SMP machine and send
you the output.<BR><BR>JVS<BR><BR>John V. Sichi wrote:<BR>> Pappyn Bart
wrote:<BR>>> John,<BR>>><BR>>> Could you tell me what kind
of OS is running on your<BR>>> 4-way SMP machine ? And on you
laptop ?<BR>> <BR>> SMP: Red Hat Enterprise Linux (not sure
of version); JVM is HotSpot <BR>> 1.5.0_10-b03<BR>> <BR>>
Laptop: Edgy Eft version of Ubuntu Linux; JVM is HotSpot
1.5.0_04-b05<BR>> <BR>> My guess is that it's likely to be a
timing-sensitive thing. The <BR>> property trigger stuff looked
suspicious, but I tried disabling that out <BR>> and it still
happens. I also tried disabling <BR>>
testFormatStringExpressionCubeNoCache (since it runs just before and has
<BR>> cache disabled on a base cube underlying a virtual cube) but the
failure <BR>> still occurred.<BR>> <BR>> JVS<BR>>
<BR><BR>_______________________________________________<BR>Mondrian mailing
list<BR>Mondrian@pentaho.org<BR><A
href="http://lists.pentaho.org/mailman/listinfo/mondrian"
target=_blank>http://lists.pentaho.org/mailman/listinfo/mondrian</A><BR></DIV></DIV><BR></DIV></DIV><BR>
<HR SIZE=1>
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions !
Profitez des connaissances, des opinions et des expériences des internautes
sur <A
href="http://fr.rd.yahoo.com/evt=42054/*http://fr.answers.yahoo.com">Yahoo!
Questions/Réponses</A>.</BLOCKQUOTE><BR>______________________________________________________________________<BR>This
email has been scanned by the Email Security
System.<BR>______________________________________________________________________<BR></BODY></HTML>