[Mondrian] Multithreading, Parallel Batch / Query Execution

Sat Apr 28 02:32:06 EDT 2007

Hi Tushar,

Have you seen the code I suggested for this sort of thing at:
http://forums.pentaho.org/showthread.php?t=52318
?

The major issu

----- Message d'origine ----
De : Tushar Apshankar <tapshank at thoughtworks.com>
À : Mondrian developer mailing list <mondrian at pentaho.org>
Cc : Mondrian developer mailing list <mondrian at pentaho.org>; mondrian-bounces at pentaho.org
Envoyé le : Vendredi, 27 Avril 2007, 16h04mn 20s
Objet : Re : [Mondrian] Multithreading, Parallel Batch / Query Execution

Hi,

To achieve Parallel Batch / Query Execution,
we need to give execution tasks to separate threads.

We could think of 2 options to achieve
it. Both of them have their pros and cons in terms of performance and ThreadLocal
variable management.

1) New Thread Creation:

        Pros:
We change the thread local variable to inheritable thread local and the
child thread automatically get the thread local values of the parent thread.

        Cons:
Creating new threads is not a good idea. Too many threads running in a
system should not be a problem though, as we can limit the number of Threads
created using java.util.concurrent.Semaphore/BigInteger etc.

2) Thread Pool:

        Pros:
It would be faster (It looks like a valid assumption.)

        Cons:
 We will have to do some manipulation to pass the thread local variables
to worker threads of Thread Pool. All ThreadLocal variables of a Thread
are stored in a ThreadLocalMap. Thread class does not provide getter/setter
for ThreadLocalMap. We could still get/set the ThreadLocalMap using reflection.
This solution is working fine.

Is there any other option? If not, which
one of the above looks better approach to you?

Please let us know your thoughts.

Thanks and regards,

Tushar

Tushar Apshankar <tapshank at thoughtworks.com>

Sent by: mondrian-bounces at pentaho.org
04/25/2007 06:27 PM

Please respond to

Mondrian developer mailing list <mondrian at pentaho.org>

To

Mondrian developer mailing list <mondrian at pentaho.org>

cc

Subject

Re : [Mondrian] Multithreading, Parallel
Batch / Query Execution

Hi, 

To improve performance, we started looking at ReentrantReadWriteLock and
ConcurrentHashMap (from java.util.concurrent) package.

We are also looking at ThreadPoolExecutor and related classes to implment
parallel query / batch execution (most of these classes are also available
in java.util.concurrent as well as backport jar). 

Since compatibility with Java 1.4 is essential, we started looking at the
build script retroweave target that uses Retroweaver 1.2.4.

Retroweaver without our changes works fine. We ran into problems while
using the current retroweaver version and build target after our changes.

We have a simple class LockTest to reproduce the problem:

=============================================================================================

package mondrian; 

import java.util.concurrent.ConcurrentHashMap; 

import java.util.concurrent.locks.ReentrantReadWriteLock;

import java.util.concurrent.locks.Lock; 

public class LockTest { 

    ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<String,
String>(); 

    ReentrantReadWriteLock lock; 

    Lock r ;              

    Lock w ; 

    public LockTest() 

    { 

        cache = new ConcurrentHashMap();

        lock = new ReentrantReadWriteLock();

        System.out.println("Before lock = "
+ lock); 

        w = lock.writeLock(); 

        r = lock.readLock(); 

        System.out.println("After lock = "
+ lock); 

    } 

    public static void main(String[] args) {

        LockTest lt = new LockTest();

        System.out.println("i am inside main method");

    } 

} 

=============================================================================================

When we retroweave it with retroweaver 1.2.4 (using ANT as well as retroweaver
UI) and run the weaved class using Java 1.4.2, we get java.lang.NoSuchMethodError.
Stack is: 

java.lang.NoSuchMethodError: edu.emory.mathcs.backport.java.util.concurrent.locks.ReentrantReadWriteLock.writeLock()

Ledu/emory/mathcs/backport/java/util/concurrent/locks/ReentrantReadWriteLock$WriteLock;

[Loaded java.lang.StackTraceElement from C:\j2sdk1.4.2_14\jre\lib\rt.jar]

        at mondrian.LockTest.<init>(LockTest.java:26)

        at mondrian.LockTest.main(LockTest.java:34):

We have backport jar in our class path (please note that we are NOT getting
ClassNotFoundExcecption). 

Did someone face this issue earlier? Please let us know if we are missing
something here. 

We also tried out retroweaver 2.0 without success.

As an alternative, we tried Retrotranslator(http://retrotranslator.sourceforge.net)
and everything worked in first attempt. We propose to create a new target
for Retrotranslator and add the related libraries. Please let us know your
thoughts on this 

Thanks and regards, 

Tushar, Ajit 

_______________________________________________

Mondrian mailing list

Mondrian at pentaho.org

http://lists.pentaho.org/mailman/listinfo/mondrian

_______________________________________________
Mondrian mailing list
Mondrian at pentaho.org
http://lists.pentaho.org/mailman/listinfo/mondrian

___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20070428/d0acfd5e/attachment.html 