[Mondrian] RolapSchema #122

Matt Campbell mkambol at gmail.com
Fri Feb 20 09:39:50 EST 2009


Maybe we should add a check to see if FileObject is of type HttpFileObject,
since I don't think that should ever have a different URL than what's passed
in.

On Thu, Feb 19, 2009 at 7:04 PM, Sherman Wood <swood at jaspersoft.com> wrote:

>  I was not able to recreate the cache problem with the usual file system
> access or my JasperServer repository VFS implementation.
>
>
>
> file.getName().getURI().equals(url) could be false for no good reason. I
> played with the test suite and got:
>
>
>
>      [java] Re-resolving file: returned VFS url was:
> file:///C:/Users/swood/Documents/pf/sgwood_perforce.eigenbase.org_1666_1/open/mondrian/demo/FoodMart.xml,
> not given url:
> C:/Users/swood/Documents/pf/sgwood_perforce.eigenbase.org_1666_1/open/mondrian/demo/FoodMart.xml
>
>
>
> Doing this all the time will be overhead, but doesn't break things.
>
>
>
> Sherman
>
>
>
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Thursday, February 19, 2009 6:33 AM
>
> *To:* Mondrian developer mailing list
> *Subject:* Re: [Mondrian] RolapSchema #122
>
>
>
> I took silence for assent and have checked in 12382, which clears the VFS
> cache for the requested file when this inconsistency is encountered.  I also
> did some minor cleanup to eliminate some of the duplicate code.  Let me know
> if that change looks reasonable to you.
>
> On Wed, Feb 18, 2009 at 10:14 AM, Matt Campbell <mkambol at gmail.com> wrote:
>
> It does certainly seem to be a vfs bug.  As a workaround, I added the
> following block right after the .resolveFile().  This validates that the url
> passed in matches the url of the file retrieved, and if not clears the file
> from the cache and reloads.
>
>
>
>  if ( !file.getName().getURI().equals(url)) {
>
>             fsManager.getFilesCache().removeFile(file.getFileSystem(),
>  file.getName());
>
>             file = fsManager.resolveFile(userDir, url);
>
>         }
>
>
>
> Any objection to me checking in this change?  If we find a better
> workaround to the vfs bug we can always drop this check.
>
>
>
>
>
> On Tue, Feb 17, 2009 at 6:20 PM, Sherman Wood <swood at jaspersoft.com>
> wrote:
>
> I was just going to refactor RolapSchema.load() to use
> Util.readVirtualFile() because of that identical code.
>
>
>
> The default configuration of Apache VFS we are using should be getting new
> content each time a URI is "resolved" (
> http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/CacheStrategy.html#ON_RESOLVE), which happens each time .readVirtualFile is called.
>
>
>
> I'll do some further tests, taking into account Will's comments about
> possible VFS HTTP bugs.
>
>
>
>
>
> Sherman
>
>
>
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Wednesday, February 18, 2009 6:14 AM
>
>
> *To:* Mondrian developer mailing list
>
> *Subject:* Re: [Mondrian] RolapSchema #122
>
>
>
> I just noticed that RolapSchema.load() uses identical code to
> Util.readVirtualFile().  It seems likely that it would have the same problem
> with cached files.
>
> On Tue, Feb 17, 2009 at 12:18 PM, Matt Campbell <mkambol at gmail.com> wrote:
>
> It looks like the parameters in the URL are ignored on the second call to
> readVirtualFile().  The unit test below shows the issue.  The URI of the
> second file returned by .readVirtualFile ends up being identical to uri1.
>
>
>
> Could you revert your change until this is fixed?
>
>
>
>  public void testReadVirtualFile() {
>
>         try {
>
>             String uri1 = "http://localhost:8088/?URI1";
>
>             String uri2 = "http://localhost:8088/?URI2";
>
>             FileObject file1 = Util.readVirtualFile(uri1);
>
>             FileObject file2 = Util.readVirtualFile(uri2);
>
>
>
>             assertEquals(file1.getName().getURI(), uri1);
>
>             assertEquals(file2.getName().getURI(), uri2);
>
>         } catch(Exception e) {
>
>              fail("Threw an exception while reading virtual file");
>
>         }
>
>     }
>
>
>
> On Mon, Feb 16, 2009 at 4:36 PM, Sherman Wood <swood at jaspersoft.com>
> wrote:
>
>
>
> Yep – that looks weird and bad. I'll look into it too.
>
>
>
> Sherman
>
>
>
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Tuesday, February 17, 2009 4:01 AM
> *To:* Mondrian developer mailing list
> *Subject:* [Mondrian] RolapSchema #122
>
>
>
> Sherman,
>
> It looks like a bug was introduced in the code which retrieves schemas from
> the Pool.  In Pool.get() there is code which first loads the schema from a
> given URL, then computes its checksum.  I've noticed that the new
> Util.readVirtualFile() method will sometimes return the content of the
> *previous* schema loaded.
>
>
>
> In the current code there is actually an extra read of the catalog--see the
> bolded section below.  It's leftover from the previous approach for loading
> the schema.  This should probably be cleaned up, but it helped me find the
> issue.  When I've traced through this code I've seen that the catalogStr
> returned by the old code (Util.readURL) is sometimes different from the
> schema returned by Util.readVirtualFile.
>
>
>
> I'm going to try to come up with a clean reproducible test case tomorrow,
> but I thought I'd pass on what I've seen in case you see something obviously
> wrong.
>
>
>
>                 try {
>
>                     if (catalogStr == null) {
>
>                        * catalogStr = Util.readURL(catalogUrl);*
>
>                         // Use VFS to get the content
>
>
>
>                         FileObject file = Util.readVirtualFile(catalogUrl);
>
>
>
>                         StringBuilder buf = new StringBuilder(1000);
>
>                         FileContent fileContent = file.getContent();
>
>                         InputStream in = fileContent.getInputStream();
>
>                         int n;
>
>                         while ((n = in.read()) != -1) {
>
>                             buf.append((char) n);
>
>                         }
>
>                      *   catalogStr = buf.toString();*
>
>                     }
>
>                     md5Bytes = encodeMD5(catalogStr);
>
>                 } catch (Exception ex) {
>
>                     // Note, can not throw an Exception from this method
>
>                     // but just to show that all is not well in Mudville
>
>                     // we print stack trace (for now - better to change
>
>                     // method signature and throw).
>
>                     ex.printStackTrace();
>
>                 }
>
>
>
> -Matt
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
>
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090220/f7d4757a/attachment.html 


More information about the Mondrian mailing list