[Mondrian] RolapSchema #122
Matt Campbell
mkambol at gmail.com
Fri Feb 20 09:39:50 EST 2009
Maybe we should add a check to see if FileObject is of type HttpFileObject,
since I don't think that should ever have a different URL than what's passed
in.
On Thu, Feb 19, 2009 at 7:04 PM, Sherman Wood <swood at jaspersoft.com> wrote:
> I was not able to recreate the cache problem with the usual file system
> access or my JasperServer repository VFS implementation.
>
>
>
> file.getName().getURI().equals(url) could be false for no good reason. I
> played with the test suite and got:
>
>
>
> [java] Re-resolving file: returned VFS url was:
> file:///C:/Users/swood/Documents/pf/sgwood_perforce.eigenbase.org_1666_1/open/mondrian/demo/FoodMart.xml,
> not given url:
> C:/Users/swood/Documents/pf/sgwood_perforce.eigenbase.org_1666_1/open/mondrian/demo/FoodMart.xml
>
>
>
> Doing this all the time will be overhead, but doesn't break things.
>
>
>
> Sherman
>
>
>
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Thursday, February 19, 2009 6:33 AM
>
> *To:* Mondrian developer mailing list
> *Subject:* Re: [Mondrian] RolapSchema #122
>
>
>
> I took silence for assent and have checked in 12382, which clears the VFS
> cache for the requested file when this inconsistency is encountered. I also
> did some minor cleanup to eliminate some of the duplicate code. Let me know
> if that change looks reasonable to you.
>
> On Wed, Feb 18, 2009 at 10:14 AM, Matt Campbell <mkambol at gmail.com> wrote:
>
> It does certainly seem to be a vfs bug. As a workaround, I added the
> following block right after the .resolveFile(). This validates that the url
> passed in matches the url of the file retrieved, and if not clears the file
> from the cache and reloads.
>
>
>
> if ( !file.getName().getURI().equals(url)) {
>
> fsManager.getFilesCache().removeFile(file.getFileSystem(),
> file.getName());
>
> file = fsManager.resolveFile(userDir, url);
>
> }
>
>
>
> Any objection to me checking in this change? If we find a better
> workaround to the vfs bug we can always drop this check.
>
>
>
>
>
> On Tue, Feb 17, 2009 at 6:20 PM, Sherman Wood <swood at jaspersoft.com>
> wrote:
>
> I was just going to refactor RolapSchema.load() to use
> Util.readVirtualFile() because of that identical code.
>
>
>
> The default configuration of Apache VFS we are using should be getting new
> content each time a URI is "resolved" (
> http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/CacheStrategy.html#ON_RESOLVE), which happens each time .readVirtualFile is called.
>
>
>
> I'll do some further tests, taking into account Will's comments about
> possible VFS HTTP bugs.
>
>
>
>
>
> Sherman
>
>
>
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Wednesday, February 18, 2009 6:14 AM
>
>
> *To:* Mondrian developer mailing list
>
> *Subject:* Re: [Mondrian] RolapSchema #122
>
>
>
> I just noticed that RolapSchema.load() uses identical code to
> Util.readVirtualFile(). It seems likely that it would have the same problem
> with cached files.
>
> On Tue, Feb 17, 2009 at 12:18 PM, Matt Campbell <mkambol at gmail.com> wrote:
>
> It looks like the parameters in the URL are ignored on the second call to
> readVirtualFile(). The unit test below shows the issue. The URI of the
> second file returned by .readVirtualFile ends up being identical to uri1.
>
>
>
> Could you revert your change until this is fixed?
>
>
>
> public void testReadVirtualFile() {
>
> try {
>
> String uri1 = "http://localhost:8088/?URI1";
>
> String uri2 = "http://localhost:8088/?URI2";
>
> FileObject file1 = Util.readVirtualFile(uri1);
>
> FileObject file2 = Util.readVirtualFile(uri2);
>
>
>
> assertEquals(file1.getName().getURI(), uri1);
>
> assertEquals(file2.getName().getURI(), uri2);
>
> } catch(Exception e) {
>
> fail("Threw an exception while reading virtual file");
>
> }
>
> }
>
>
>
> On Mon, Feb 16, 2009 at 4:36 PM, Sherman Wood <swood at jaspersoft.com>
> wrote:
>
>
>
> Yep – that looks weird and bad. I'll look into it too.
>
>
>
> Sherman
>
>
>
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Tuesday, February 17, 2009 4:01 AM
> *To:* Mondrian developer mailing list
> *Subject:* [Mondrian] RolapSchema #122
>
>
>
> Sherman,
>
> It looks like a bug was introduced in the code which retrieves schemas from
> the Pool. In Pool.get() there is code which first loads the schema from a
> given URL, then computes its checksum. I've noticed that the new
> Util.readVirtualFile() method will sometimes return the content of the
> *previous* schema loaded.
>
>
>
> In the current code there is actually an extra read of the catalog--see the
> bolded section below. It's leftover from the previous approach for loading
> the schema. This should probably be cleaned up, but it helped me find the
> issue. When I've traced through this code I've seen that the catalogStr
> returned by the old code (Util.readURL) is sometimes different from the
> schema returned by Util.readVirtualFile.
>
>
>
> I'm going to try to come up with a clean reproducible test case tomorrow,
> but I thought I'd pass on what I've seen in case you see something obviously
> wrong.
>
>
>
> try {
>
> if (catalogStr == null) {
>
> * catalogStr = Util.readURL(catalogUrl);*
>
> // Use VFS to get the content
>
>
>
> FileObject file = Util.readVirtualFile(catalogUrl);
>
>
>
> StringBuilder buf = new StringBuilder(1000);
>
> FileContent fileContent = file.getContent();
>
> InputStream in = fileContent.getInputStream();
>
> int n;
>
> while ((n = in.read()) != -1) {
>
> buf.append((char) n);
>
> }
>
> * catalogStr = buf.toString();*
>
> }
>
> md5Bytes = encodeMD5(catalogStr);
>
> } catch (Exception ex) {
>
> // Note, can not throw an Exception from this method
>
> // but just to show that all is not well in Mudville
>
> // we print stack trace (for now - better to change
>
> // method signature and throw).
>
> ex.printStackTrace();
>
> }
>
>
>
> -Matt
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
>
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090220/f7d4757a/attachment.html
More information about the Mondrian
mailing list