Replacement for DataTable

[Brian Richards]

Can anyone suggest a replacement, 3rd party or otherwise, object for
DataTable. We're running into performance issues with large dataset (>30k)
rows. Looking for something that can do binary serialization, typed column
row data, primary keys, filtering etc. SQL desktop engine or some other
database is not an option at this point.

Thanks

Briann

 

[=?ISO-8859-1?Q?G=F6ran_Andersson?=]

Can anyone suggest a replacement, 3rd party or otherwise, object for
DataTable. We're running into performance issues with large dataset (>30k)
rows. Looking for something that can do binary serialization, typed column
row data, primary keys, filtering etc. SQL desktop engine or some other
database is not an option at this point.

Why isn't a database an option? My very first suggestion would be just that.

 

[moo]

Why isn't a database an option? My very first suggestion would be just that.

My database teacher would laugh at me for saying this, but have you
considered XML and XPath? I'm not sure of the performance
implications, but it's the alternative that springs most readily to
mind...

 

[AlexS]

Datasets are keeping complete result sets and related data in memory, so you
have to deal with unavoidable performance issues.

If you create data sets and tables programmatically, you must implement some
kind of db engine to persist data and page through big tables / result sets.
Which doesn't make much sense considering availability of various db
engines, both commercial and free ones. All replacements will implement some
kind of db engine. XML won;t solve your performance issues and might make
situation worse, as parsing complete files will clog your memory with lots
of residuals. Also, it'll double memory consumption.

Now you have troubles with 30K rows. Consider what kind of issues you'll
have with 300K rows. Don't bump into brick wall.

Real question is: what kind of issues you have with 30K rows. This number is
not that big. Any specific details?

 

[Brian Richards]

My database teacher would laugh at me for saying this, but have you

considered XML and XPath? I'm not sure of the performance
implications, but it's the alternative that springs most readily to
mind...

I'm pretty sure XML is the exact reason that the DataTable performance is so
awful. Because it's XML internally.

 

[Jesse Houwing]

* Brian Richards wrote, On 22-7-2007 2:38:

Can anyone suggest a replacement, 3rd party or otherwise, object for
DataTable. We're running into performance issues with large dataset (>30k)
rows. Looking for something that can do binary serialization, typed column
row data, primary keys, filtering etc. SQL desktop engine or some other
database is not an option at this point.

Thanks

Briann

Have you tried SQL Server CE (previously only available on Windows
Mobile, but work sin any .NET app now).

Jesse

 

[=?ISO-8859-1?Q?G=F6ran_Andersson?=]

I'm pretty sure XML is the exact reason that the DataTable performance is so
awful. Because it's XML internally.

I don't think that's the case, as DataTables work with data in native
format, and XML is only text.

 

[Jon Skeet [C# MVP]]

I'm pretty sure XML is the exact reason that the DataTable performance is so
awful. Because it's XML internally.

No it's not. Just because it can load from XML and save to XML doesn't
mean its internal storage is XML.