[Mondrian] Multithreading, Parallel Batch / Query Execution

[michael bienstein]

Hi Tushar,

Have you seen my suggestion for this sort of thing at:
http://forums.pentaho.org/showthread.php?t=52318
?

No one has yet commented on it.

I see one major hurdle to integrating it: it requires intercepting all incoming requests to the system.  That can be easily worked around though through either removing this as a requirement and using the Query to store objects (instead of for example your ThreadLocals) or using a Filter in the app server.

Basically there are a number of design issues to address when you start doing multiple threads:
1) How to have query-local objects or values stored?  This is your issue with ThreadLocals.
2) How to coordinate the threads so that a failure in one will end the others ASAP?
How to ensure no thread continues to run when the initial request exits?
3) How to handle errors in one thread?
4) Where do you get the thread from?  In an app server you shouldn't make your own threads.  In fact Julian wants a mode that allows only one thread as now.

I also put in the code for handling thread-bound local transactions.  That is easy to take out if desired.  The API could then be simply:

interface TaskSystem {
    void waitForAll(Set<Task> tasks) throws WaitForAllException;
}

interface Task {
    void run();
    void abort();
}

class WaitForAllException extends RuntimeException {
    Map<Task, Throwable> getCauses();
}

Michael



      
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20070428/d9eea58e/attachment.html