[Mondrian] Mondrian Query Timeout

Julian Hyde jhyde at pentaho.com
Mon Jul 11 18:21:28 EDT 2011


However we solve this, we need a data structure that records all SQL
statements that are running in the service of a particular MDX statement.
The Execution class that I added on Friday (see
<http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10>
http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10 ) is perfect for this.
 
I couldn't tell whether you were suggesting this, but just in case, let me
state for the record: I think that killing Java threads is a bad idea. It is
very likely to make a mess. I think that Java code under mondrian's control
should continue to poll the 'canceled' flag. (Which on Friday moved from
Query to Execution.)
 
For SQL statements, mondrian should definitely call cancel. If the JDBC
driver of the database concerned is well-implemented, it will do something
sensible with the cancel request. (Probably three things: cancel the
synchronous call fairly quicky, throw a SQLException indicating that the
statement is canceled, and send a message to the deeper parts of the
infrastructure to cancel asynchronous processing, slower but more
exhaustively.)
 
Since Mondrian is itself an JDBC server (by virtue of olap4j being an
extension to JDBC), we should do something similar. It's worth asking: what
MondrianOlap4jStatement.executeOlapQuery() should do when canceled? I expect
that Execute.cancel will be polled every ~millisecond, so cancel the
statement should come back fairly quickly. We could make it come back
quicker still if executeOlapQuery were making a call to another thread. Then
we could interrupt that call without damaging either of the threads. 
 
Let me be clear that this is a "nice to have". And there are some
implementation details I haven't solved:
 
(a) I am speculating that there is an 'cancelable future' in JDK 1.6
off-the-shelf. I mean: create a Future, have another thread (from an
ExecutionService presumably) do the work, and allow any 3rd thread to cancel
that call. Is this available in the JDK?
 
(b) If the calls you need are not available in JDK 1.4 or JDK 1.5, it's OK
if they have inferior behavior. But it still needs to build & run in those
JDKs.
 
(c) MondrianOlap4jStatement.executeOlapQuery is not necessarily the right
place in the architecture to make the asynchronous 'slice'. Find the ideal
place, maybe a little ways below it. But note that Locus.peek() etc. uses a
thread-local.
 
Julian


  _____  

From: Luc Boudreau [mailto:lucboudreau at gmail.com] 
Sent: Monday, July 11, 2011 8:57 AM
To: Julian Hyde
Cc: Mondrian developer mailing list
Subject: Mondrian Query Timeout



Julian,

 

Following your commit on the statements refactoring, I have started to think
about how we would now enforce a proper query timeout. (MONDRIAN-415). 

 

What I had in mind is a simple system where, when creating RolapResult
objects (in RolapConnection.execute()) we fork the execution to a second
thread (let's call it the execution thread). The first thread (let's call it
the user thread) will wait a predetermined amount of time (the actual
timeout) and if the execution thread has not completed, the user thread will
perform the following.

*	Stop all SQL statements currently associated to this Execution. 

*	Do house cleaning. 

*	Bake a cake. (optional)


*	Throw back a timeout exception.

This has several advantages. First off, we could remove all checks for query
timeout in the evaluator and centralize everything in a single place.
Second, as it stands now, there is no way to do a "hard stop" of any SQL
statements running for a given Execution as we sit in the same thread. As
long as the SQL Statement has not returned, we have no way to check for a
timeout. As of now, the best we can do is to set an arbitrary JDBC timeout,
but since there is no telling how many SQL statements will be issued, nor
their priority, we cannot determine the proper JDBC timeout value to apply.
By having the user thread overview the execution process, it becomes much
simpler to manage and cleanup all the resources used by a query execution.




Should I try this approach and submit a first POC? 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20110711/f2688ad5/attachment.html 


More information about the Mondrian mailing list