<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<STYLE type=text/css>DIV {
MARGIN: 0px
}
</STYLE>
<META content="MSHTML 6.00.5730.11" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>Connections and transactions should live as long as the
cache, not a single mdx query.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>I am about to check in other changes to the rolapstar like
I described before in this thread,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>please note that transactions and connection change will
not be there yet. The changes</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>I will check in have most of to do with multi-user access
and the plugin.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>When a query is executed, first changes are checked by
using the data source change listener plugin.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>When changes are detected, than the transaction for - this
thread only - should be stopped and a new</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>one should be taken. Other concurrently running
threads should use the old connection/transactions.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>When new queries are executed after changes have check in,
a new connection/transaction should be started.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>That way, a single mdx query should always look at the
same data - what should be the whole point.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>The changes I have made now apply to the data source change
listener plugin, but a similar approach could</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>be taken for explicit cache flushing.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=781370113-23012007><FONT face=Arial
color=#0000ff size=2>Bart</FONT></SPAN></DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> mondrian-bounces@pentaho.org
[mailto:mondrian-bounces@pentaho.org] <B>On Behalf Of </B>michael
bienstein<BR><B>Sent:</B> dinsdag 23 januari 2007 13:52<BR><B>To:</B> Mondrian
developer mailing list<BR><B>Subject:</B> Re : [Mondrian]
Re:VirtualCubeTest.testCalculatedMemberAcrossCubesfailing on
SMP<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">Just
two thoughts on this:</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">1)
Currently I think that a HashMap is used for the global cache. HashMap is
not safely synchronized. There is a synchronize block that is too large
probably - the whole aggregations data. </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">2)
Two-tier using threadlocal sounds good. Can we do this idea:</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">interface
QueryContext {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
Connection getConnection(DataSource ds);</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
some sort of common filter for aggregation cache and hierarchy caches</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
void dispose();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">class
QueryContextImpl {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
private HashMap<DataSource, Connection> openConnections = new
HashMap<DataSource,Connection>();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
public Connection getConnection(DataSource ds) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
Connection c = openConnections.get(ds);</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
if (c != null) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
return c;</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
try {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
c = ds.getConnection();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
openConnections.put(ds, c);</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
return c;</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
} catch (SQLException ex) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
throw new
MondrianExceptionOrSomething(ex);</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
//TODO some filtering to the global aggregation and hierarchy caches</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
public void dispose() {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
for (Connection c : openConnections.valueSet()) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
try {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
c.close();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
} catch (SQLException ex) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
log it ...</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">RolapResult.java:</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">{</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
private static ThreadLocal<QueryContext> qContext = new
ThreadLocal<QueryContext>();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
public static QueryContext getQueryContext() {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
return qContext.get();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
public RolapResult(...) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
...</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
if (!execute) {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
return;</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
//Going to execute</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
QueryContext qc = createQueryContext();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
qContext.set(qc);</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
try {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
//Do execute stuff here</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
} finally {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
qContext.clear();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
qc.dispose();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> ...</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
} </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
//Use a property to override the class used? That way we can configure
each Connection specifically?</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
public QueryContext createQueryContext() {</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
return new QueryContextImpl();</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
}<BR>}<BR>//All places in the code base that use DataSource to obtain a
Connection in the context of a query should use:</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">Connection
c = RolapResult.getQueryContext().getConnection(ds);</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">That
way we only open one connection per query and we use the database's transaction
system.</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">I
found this hard to do because of the RolapConnection/RolapCube constructors
calling each other somehow (can't remember the details).</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">Michael</DIV>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">-----
Message d'origine ----<BR>De : Julian Hyde <julianhyde@speakeasy.net><BR>À
: Mondrian developer mailing list <mondrian@pentaho.org><BR>Envoyé le :
Mardi, 23 Janvier 2007, 11h57mn 24s<BR>Objet : RE: [Mondrian] Re:
VirtualCubeTest.testCalculatedMemberAcrossCubesfailing on SMP<BR><BR>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>I think the problem is with how mondrian evaluates members
using multiple passes. When the measures are coming from a virtual cube, of
course there are multiple real cubes, and each of those has a cell reader. But
the code in RolapResult assumes there is only one cell
reader.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>Mondrian should check the cell readers for all applicable
cubes, and only emit a result when all cell readers have been
populated.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>I haven't implemented the fix yet, but this cause seems
very plausible to me.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>I'm not exactly sure why this problem surfaced after Bart's
change - maybe thread-local caches increased the chances of one cache being
populated and another not - or why it appears on SMP
machines.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>By the way, in an effort to get this working, I removed
Bart's RolapStarAggregationKey (a compound key of BitKey and thread id) and
moved to a two-tier hashing scheme. The first tier is a ThreadLocal of maps, and
the second tier is a map. Threads which want access to the global map just skip
the first tier. Given the difficulties obtaining a unique id for a thread, using
a ThreadLocal seemed cleaner. So, even though this didn't fix the bug, I'm going
to check in.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=200204710-23012007><FONT face=Verdana
color=#000080 size=2>Julian</FONT></SPAN></DIV></DIV></DIV><BR>
<HR SIZE=1>
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions !
Profitez des connaissances, des opinions et des expériences des internautes sur
<A href="http://fr.rd.yahoo.com/evt=42054/*http://fr.answers.yahoo.com">Yahoo!
Questions/Réponses</A>.
<BR>______________________________________________________________________<BR>This
email has been scanned by the Email Security
System.<BR>______________________________________________________________________<BR></BODY></HTML>