MSXML Fails to Load file larger than 1 GB

[Trey Shaffer]

Forgive me, in advance, for this being vague...

First question, is this the right forum?

I am using MSXML within a vbScript, not associated with a web page.

My problem seems to be with size of an XML document.

I am using the .Load method as a simple validation tool to see if a file is well formed.

I download the file, via FTP, open it with MSXML, and check the return value for success/failure.

This works for files of size less than 1 GB, but fails for files greater than 1 GB. The threshold is close to 1 GB, though I haven't narrowed it with precision. 900 MB definitely works. 1.1 GB definitely fails.

Does this sound familiar?
Am I omitting anything?

[Martin Honnen]

Post by Trey Shaffer
Forgive me, in advance, for this being vague...
First question, is this the right forum?
I am using MSXML within a vbScript, not associated with a web page.
My problem seems to be with size of an XML document.
I am using the .Load method as a simple validation tool to see if a file is well formed.
I download the file, via FTP, open it with MSXML, and check the return
value for success/failure.
This works for files of size less than 1 GB, but fails for files greater
than 1 GB. The threshold is close to 1 GB, though I haven't narrowed it
with precision. 900 MB definitely works. 1.1 GB definitely fails.
Does this sound familiar?
Am I omitting anything?

A DOM parser loads the complete document into a tree structure in
memory. You could simply run out of memory with files that large.
With VB you could try the MSXML SAX parser instead, if all you want is
check for well-formedness.
Or switch to .NET and use XmlReader, that also does have a low memory
footprint.

With VBScript I don't see any approach, if memory is the problem.

What error exactly do you get when you say MSXML fails?

[Trey Shaffer]

Thanks for input... As for the error, the statement:

booleanVariable = DocObject.Load( FileName )

returns "False"

The process runs on a server with quite a bit of memory. If there is a
memory constraint, it is probably not physical memory...

Post by Trey Shaffer
Forgive me, in advance, for this being vague...
First question, is this the right forum?
I am using MSXML within a vbScript, not associated with a web page.
My problem seems to be with size of an XML document.
I am using the .Load method as a simple validation tool to see if a file is well formed.
I download the file, via FTP, open it with MSXML, and check the return
value for success/failure.
This works for files of size less than 1 GB, but fails for files greater
than 1 GB. The threshold is close to 1 GB, though I haven't narrowed it
with precision. 900 MB definitely works. 1.1 GB definitely fails.
Does this sound familiar?
Am I omitting anything?

A DOM parser loads the complete document into a tree structure in memory.
You could simply run out of memory with files that large.
With VB you could try the MSXML SAX parser instead, if all you want is
check for well-formedness.
Or switch to .NET and use XmlReader, that also does have a low memory
footprint.
With VBScript I don't see any approach, if memory is the problem.
What error exactly do you get when you say MSXML fails?
--
Martin Honnen --- MVP XML
http://msmvps.com/blogs/martin_honnen/

[Martin Honnen]

Post by Trey Shaffer
booleanVariable = DocObject.Load( FileName )
returns "False"

Then check what the parseError properties tell you i.e.
DocObject.parseError.reason

[Trey Shaffer]

Martin,
Thanks for your help... The reason was "Not enough storage to complete the
operation".

That doesn't seem like something to be fixed. I'm assuming this is some
software imposed memory constraint. The server running the script has
plenty of physical RAM. It is a Windows 2000 box, but am using MSXML 6.0.
We will soon replace it with a 2008 box. Maybe that will fare better.

Post by Martin Honnen

Post by Trey Shaffer
booleanVariable = DocObject.Load( FileName )
returns "False"

Then check what the parseError properties tell you i.e.
DocObject.parseError.reason
--
Martin Honnen --- MVP XML
http://msmvps.com/blogs/martin_honnen/

[Julian F. Reschke]

Post by Trey Shaffer
Martin,
Thanks for your help... The reason was "Not enough storage to complete the
operation".
That doesn't seem like something to be fixed. I'm assuming this is some
software imposed memory constraint. The server running the script has
plenty of physical RAM. It is a Windows 2000 box, but am using MSXML 6.0.
We will soon replace it with a 2008 box. Maybe that will fare better.
...

Likely not.

Keep in mind that the in-memory representation of a 1GB XML file will
take much more space than 1 GB.

BR, Julian