[Mondrian] RolapSchema #122

Sherman Wood swood at jaspersoft.com
Thu Feb 19 19:04:12 EST 2009


I was not able to recreate the cache problem with the usual file system
access or my JasperServer repository VFS implementation.

 

file.getName().getURI().equals(url) could be false for no good reason. I
played with the test suite and got:

 

     [java] Re-resolving file: returned VFS url was:
file:///C:/Users/swood/Documents/pf/sgwood_perforce.eigenbase.org_1666_1/o
pen/mondrian/demo/FoodMart.xml, not given url:
C:/Users/swood/Documents/pf/sgwood_perforce.eigenbase.org_1666_1/open/mond
rian/demo/FoodMart.xml

 

Doing this all the time will be overhead, but doesn't break things. 

 

Sherman

 

From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
On Behalf Of Matt Campbell
Sent: Thursday, February 19, 2009 6:33 AM
To: Mondrian developer mailing list
Subject: Re: [Mondrian] RolapSchema #122

 

I took silence for assent and have checked in 12382, which clears the VFS
cache for the requested file when this inconsistency is encountered.  I
also did some minor cleanup to eliminate some of the duplicate code.  Let
me know if that change looks reasonable to you.

On Wed, Feb 18, 2009 at 10:14 AM, Matt Campbell <mkambol at gmail.com> wrote:

It does certainly seem to be a vfs bug.  As a workaround, I added the
following block right after the .resolveFile().  This validates that the
url passed in matches the url of the file retrieved, and if not clears the
file from the cache and reloads.

 

 if ( !file.getName().getURI().equals(url)) {

            fsManager.getFilesCache().removeFile(file.getFileSystem(),
file.getName());

            file = fsManager.resolveFile(userDir, url);

        }

 

Any objection to me checking in this change?  If we find a better
workaround to the vfs bug we can always drop this check.

 

 

On Tue, Feb 17, 2009 at 6:20 PM, Sherman Wood <swood at jaspersoft.com>
wrote:

I was just going to refactor RolapSchema.load() to use
Util.readVirtualFile() because of that identical code.

 

The default configuration of Apache VFS we are using should be getting new
content each time a URI is "resolved"
(http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/CacheStrateg
y.html#ON_RESOLVE ), which happens each time .readVirtualFile is called.

 

I'll do some further tests, taking into account Will's comments about
possible VFS HTTP bugs.

 

 

Sherman

 

From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
On Behalf Of Matt Campbell
Sent: Wednesday, February 18, 2009 6:14 AM


To: Mondrian developer mailing list

Subject: Re: [Mondrian] RolapSchema #122

 

I just noticed that RolapSchema.load() uses identical code to
Util.readVirtualFile().  It seems likely that it would have the same
problem with cached files.

On Tue, Feb 17, 2009 at 12:18 PM, Matt Campbell <mkambol at gmail.com> wrote:

It looks like the parameters in the URL are ignored on the second call to
readVirtualFile().  The unit test below shows the issue.  The URI of the
second file returned by .readVirtualFile ends up being identical to uri1.

 

Could you revert your change until this is fixed?

 

 public void testReadVirtualFile() {

        try {

            String uri1 = "http://localhost:8088/?URI1";

            String uri2 = "http://localhost:8088/?URI2";

            FileObject file1 = Util.readVirtualFile(uri1);

            FileObject file2 = Util.readVirtualFile(uri2);

 

            assertEquals(file1.getName().getURI(), uri1);

            assertEquals(file2.getName().getURI(), uri2);

        } catch(Exception e) {

             fail("Threw an exception while reading virtual file");

        }

    }

 

On Mon, Feb 16, 2009 at 4:36 PM, Sherman Wood <swood at jaspersoft.com>
wrote:

 

Yep - that looks weird and bad. I'll look into it too.

 

Sherman

 

From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
On Behalf Of Matt Campbell
Sent: Tuesday, February 17, 2009 4:01 AM
To: Mondrian developer mailing list
Subject: [Mondrian] RolapSchema #122

 

Sherman,

It looks like a bug was introduced in the code which retrieves schemas
from the Pool.  In Pool.get() there is code which first loads the schema
from a given URL, then computes its checksum.  I've noticed that the new
Util.readVirtualFile() method will sometimes return the content of the
*previous* schema loaded.  

 

In the current code there is actually an extra read of the catalog--see
the bolded section below.  It's leftover from the previous approach for
loading the schema.  This should probably be cleaned up, but it helped me
find the issue.  When I've traced through this code I've seen that the
catalogStr returned by the old code (Util.readURL) is sometimes different
from the schema returned by Util.readVirtualFile.

 

I'm going to try to come up with a clean reproducible test case tomorrow,
but I thought I'd pass on what I've seen in case you see something
obviously wrong.

 

                try {

                    if (catalogStr == null) {

                        catalogStr = Util.readURL(catalogUrl);

                        // Use VFS to get the content

 

                        FileObject file =
Util.readVirtualFile(catalogUrl);

 

                        StringBuilder buf = new StringBuilder(1000);

                        FileContent fileContent = file.getContent();

                        InputStream in = fileContent.getInputStream();

                        int n;

                        while ((n = in.read()) != -1) {

                            buf.append((char) n);

                        }

                        catalogStr = buf.toString();

                    }

                    md5Bytes = encodeMD5(catalogStr);

                } catch (Exception ex) {

                    // Note, can not throw an Exception from this method

                    // but just to show that all is not well in Mudville

                    // we print stack trace (for now - better to change

                    // method signature and throw).

                    ex.printStackTrace();

                }

 

-Matt

 

_______________________________________________
Mondrian mailing list
Mondrian at pentaho.org
http://lists.pentaho.org/mailman/listinfo/mondrian

 

 


_______________________________________________
Mondrian mailing list
Mondrian at pentaho.org
http://lists.pentaho.org/mailman/listinfo/mondrian

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090219/e3162630/attachment.html 


More information about the Mondrian mailing list