[Mondrian] Mondrian Query Timeout

Joe Barnett thejoe at gmail.com
Wed Jul 13 18:25:38 EDT 2011


One potential concern I have with this is that we have a UDF that
essentially relies upon ThreadLocals that are set up by our
application to control routing to the appropriate database.  Actually,
the DataSources we pass to mondrian rely on those ThreadLocals as
well, so not actually UDF specific.  It sounds like this approach
would have mondrian run in a separate thread context, and hence not
have the appropriate ThreadLocal values set.

Are you planning on adding any hooks to the thread fork where we might
be able to set up the thread appropriately?  We'd also probably have
to adjust some things (connection pools, etc) to expect an additional
thread per thread-that-calls-into-mondrian, but would need to study
the exact execution patterns a little more closely to see if that's
really necessary.

Do you have a sense of the differences in response time between
cancel()ing the request without this extra thread vs/ with it?  Seems
like if the code is able to poll often enough (and
interrupt()/cancel() any underlying IO), the difference should be
negligible enough?  But not sure exactly where all the
cancel-checking-hooks are.

Other thoughts?

-Joe

On Tue, Jul 12, 2011 at 7:35 AM, Luc Boudreau <lucboudreau at gmail.com> wrote:
>
> Good. That's exactly what I had in mind.
>
> As for b), we are using backport-util-concurrent so I think we're good.I'll
> make sure to test thoroughly.
>
> Luc
>
>
>
> On Mon, Jul 11, 2011 at 6:21 PM, Julian Hyde <jhyde at pentaho.com> wrote:
>>
>> However we solve this, we need a data structure that records all SQL
>> statements that are running in the service of a particular MDX statement.
>> The Execution class that I added on Friday (see
>> http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10 ) is perfect for this.
>>
>> I couldn't tell whether you were suggesting this, but just in case, let me
>> state for the record: I think that killing Java threads is a bad idea. It is
>> very likely to make a mess. I think that Java code under mondrian's control
>> should continue to poll the 'canceled' flag. (Which on Friday moved from
>> Query to Execution.)
>>
>> For SQL statements, mondrian should definitely call cancel. If the JDBC
>> driver of the database concerned is well-implemented, it will do something
>> sensible with the cancel request. (Probably three things: cancel the
>> synchronous call fairly quicky, throw a SQLException indicating that the
>> statement is canceled, and send a message to the deeper parts of the
>> infrastructure to cancel asynchronous processing, slower but more
>> exhaustively.)
>>
>> Since Mondrian is itself an JDBC server (by virtue of olap4j being an
>> extension to JDBC), we should do something similar. It's worth asking: what
>> MondrianOlap4jStatement.executeOlapQuery() should do when canceled? I expect
>> that Execute.cancel will be polled every ~millisecond, so cancel the
>> statement should come back fairly quickly. We could make it come back
>> quicker still if executeOlapQuery were making a call to another thread. Then
>> we could interrupt that call without damaging either of the threads.
>>
>> Let me be clear that this is a "nice to have". And there are some
>> implementation details I haven't solved:
>>
>> (a) I am speculating that there is an 'cancelable future' in JDK 1.6
>> off-the-shelf. I mean: create a Future, have another thread (from an
>> ExecutionService presumably) do the work, and allow any 3rd thread to cancel
>> that call. Is this available in the JDK?
>>
>> (b) If the calls you need are not available in JDK 1.4 or JDK 1.5, it's OK
>> if they have inferior behavior. But it still needs to build & run in those
>> JDKs.
>>
>> (c) MondrianOlap4jStatement.executeOlapQuery is not necessarily the right
>> place in the architecture to make the asynchronous 'slice'. Find the ideal
>> place, maybe a little ways below it. But note that Locus.peek() etc. uses a
>> thread-local.
>>
>> Julian
>>
>> ________________________________
>> From: Luc Boudreau [mailto:lucboudreau at gmail.com]
>> Sent: Monday, July 11, 2011 8:57 AM
>> To: Julian Hyde
>> Cc: Mondrian developer mailing list
>> Subject: Mondrian Query Timeout
>>
>> Julian,
>>
>>
>>
>> Following your commit on the statements refactoring, I have started to
>> think about how we would now enforce a proper query timeout. (MONDRIAN-415).
>>
>>
>>
>> What I had in mind is a simple system where, when creating RolapResult
>> objects (in RolapConnection.execute()) we fork the execution to a second
>> thread (let’s call it the execution thread). The first thread (let’s call it
>> the user thread) will wait a predetermined amount of time (the actual
>> timeout) and if the execution thread has not completed, the user thread will
>> perform the following.
>>
>> Stop all SQL statements currently associated to this Execution.
>> Do house cleaning.
>> Bake a cake. (optional)
>> Throw back a timeout exception.
>>
>> This has several advantages. First off, we could remove all checks for
>> query timeout in the evaluator and centralize everything in a single place.
>> Second, as it stands now, there is no way to do a “hard stop” of any SQL
>> statements running for a given Execution as we sit in the same thread. As
>> long as the SQL Statement has not returned, we have no way to check for a
>> timeout. As of now, the best we can do is to set an arbitrary JDBC timeout,
>> but since there is no telling how many SQL statements will be issued, nor
>> their priority, we cannot determine the proper JDBC timeout value to apply.
>> By having the user thread overview the execution process, it becomes much
>> simpler to manage and cleanup all the resources used by a query execution.
>>
>> Should I try this approach and submit a first POC?
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>


More information about the Mondrian mailing list