Thread: Java XML parsing for HUGE size file and no root tags

Hi All,

I am working on situation where i have to write code to parse XML files with following :

1) Size of file will be Huge in future (So the code should be smart enough to deal with it). It may maximum reach upto 8GB.
2) The XML is not proper as it has no fix root tag rather have multiple XML in a single XML. (So we may need to add root tag and delete repated <?xml version and <!DOCTYPE ? tags) or split into multiple xml before parsing.
3) There is no XSD provide by the client rather DTD is given.

Do anyone has anything to share for such problem statement. I am thinking of multiple approach to deal with this situation like using shell script, SAX or sTax etc.

Just came to this forum, to see if we have similar problem faced by someone on future and i can be lucky

Originally Posted by aussiemcgr

Out of curiosity: why use xml if you are ignoring some of the basic standards (like root nodes) and enormous file sizes?
I'm asking because I suspect you may be attempting to do something in xml which may be far more suited to do in some other fashion.

Hi aussiemcgr,

We are getting this xml from the some other application with which we are going to integrate so this is what the requirement is and we have to use the same XML.

We have to parse this XML and finally inster the date from it into out database after validation.

Please let me know if you have some views.

thanks...

Hi,

It's a XML file only. we r thinking of 2 approaches:

1) One thing to manipulate XML by adding root tags and deleted some repeated stuff and parse it using SAX or Stax as they don't load everything in memory ...
2) As you said break it up, put in temp folder and than process it one by one. (looks to be right choice if option 1 don't give good response in terms of performance)

thanks