Reading XML Files

[pkenderdine]

Can someone please advise the best approach for reading in many 5000+
xml files from disk into a data set. Here is the code so far. On my
computer (may not be best spec) is takes about 3 minutes to read and
load the dataset. It doesn't seem to be the reading of the files as so
much as the loading of the data into the dataset. Any help would be
appreciated.

cenOcr = new DataSet();
// Create the XmlSchemaSet class.
XmlSchemaSet sc = new XmlSchemaSet();

// Add the schema to the collection.
sc.Add(null, Application.StartupPath + "\
\APCenSchema.xsd");
sc.Compile();

XmlReaderSettings cenSettings = new XmlReaderSettings();
cenSettings.Schemas.Add(sc);
cenSettings.ValidationType = ValidationType.Schema;
cenSettings.ValidationEventHandler += new
ValidationEventHandler(cenSettingsValidationEventHandler);

DirectoryInfo dir = new DirectoryInfo(strPathname);
foreach (FileInfo f in dir.GetFiles("*.xml"))
{
/*
* Load XML files into dataset
*
*/
XmlReader ocrResults = XmlReader.Create(f.FullName,
cenSettings);
cenOcr.ReadXml(ocrResults);

}

I have try many differen methods. I tried using the
XmlReadMode.ReadSchema but this did not load any data into the
dataset.

Regards
Phil

 

[Chris Nahr]

You're opening 5,000+ XML files, parsing them, validating them against
a schema, and storing the results in a dataset... sounds like a lot of
work to me. Depending on your system that might well take several
minutes. I don't think there's a way to greatly speed up this job.

 

[pkenderdine]

You're opening 5,000+ XML files, parsing them, validating them against
a schema, and storing the results in a dataset... sounds like a lot of
work to me.  Depending on your system that might well take several
minutes.  I don't think there's a way to greatly speed up this job.
--http://www.kynosarges.de

I have managed to reduce my 3 minutes to 17 seconds. I read in all the
xml files one at a time, output a temporary xml file containing all
the xml. I then read that file into my dataset.

Regards
Phil