This is the mail archive of the
xsl-list@mulberrytech.com
mailing list .
RE: Performance optimization and avoiding Out of Memory errors
- From: "James Fuller" <james dot fuller at o-idev dot com>
- To: <xsl-list at lists dot mulberrytech dot com>
- Date: Fri, 6 Sep 2002 11:09:43 +0100
- Subject: RE: [xsl] Performance optimization and avoiding Out of Memory errors
- Reply-to: xsl-list at lists dot mulberrytech dot com
> -----Original Message-----
> From: owner-xsl-list@lists.mulberrytech.com
> [mailto:owner-xsl-list@lists.mulberrytech.com]On Behalf Of Lewden Romain
Hello Lewden,
> I'm working with quite important XML Data Base ( 10 Mo) and trying to
> extract information and transform it into HTML files and rtf format.
> I work with Xt processor (release 19990511)
> I have two big problems :
> 1)time of genration is very important
> I create a predocument xml of 10 Mo (with 10 chained requests about 6,000
> objects)
> This step takes 3 hours, with a station of 512 RAM Memory
first off, with large processing scenarios, as you have....have you
considered using a SAX approach ?
what is the format of this pre-document.....I assume it is XML. what are you
doing in this pre-step ?
have you thought of using an XML native repository like xindice or
Tamino....seems like you also benefit from db style optimisations dealing
with so much data.
> Then I create in HTML a file for each object; it takes me 14 hours.
> it is a recursive call :
> <dog level="0" name="Toto1">
> <dog level="1" name="Toto1">
> <dog level="2" name="Toto1"
> <dog level="3" name="Toto1">
> <cat>ieodfli</cat>
> <cat>ieodfli</cat>
> </dog>
> </dog>
> <dog level="2" name="Mydog">
> <dog level="3" name="Toto1">
> <cat>ieodfli</cat>
> <cat>ieodfli</cat>
> </dog>
> </dog>
> </dog>
> </dog>
> I've a template for "dog", with recursive calls, and if I'm in level "2" I
> create a separate HTML page. I've 6000 elements at level 2 and it takes me
> 10 hours. How can I optimize this time ?
hmmmm, why do you store level ? I think that if u described your entire
process that we could come up with something optimised.....otherwise the
structure of your XML ( is this lorem ipsum, or are u really doing something
with dogs and cats ?) is slightly confusing.
without seeing your template, or use of keys, etc.....it is hard to comment.
> 2) When I work in rtf, I can have out of memory errors, because I
> write less
> files but they become huge (9 Mo octets).
> All the output file is kept in memory in memory until the stylesheet is
> performed.
> How can I introduce an option to force my program to write "in
> real time" in
> the output file ?
my gut feeling, is that you should investigate Saxon (
saxon.sourceforge.net ) as it has some switches to assist in large file
processing.....and it is also a java XSLT processor, which will integrate
with your environment easily.
gl, jim fuller
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list