Replacement for DataTable

  • Thread starter Thread starter Brian Richards
  • Start date Start date
B

Brian Richards

Can anyone suggest a replacement, 3rd party or otherwise, object for
DataTable. We're running into performance issues with large dataset (>30k)
rows. Looking for something that can do binary serialization, typed column
row data, primary keys, filtering etc. SQL desktop engine or some other
database is not an option at this point.

Thanks

Briann
 
Brian said:
Can anyone suggest a replacement, 3rd party or otherwise, object for
DataTable. We're running into performance issues with large dataset (>30k)
rows. Looking for something that can do binary serialization, typed column
row data, primary keys, filtering etc. SQL desktop engine or some other
database is not an option at this point.
Click to expand...

Why isn't a database an option? My very first suggestion would be just that.
 
Why isn't a database an option? My very first suggestion would be just that.
Click to expand...


My database teacher would laugh at me for saying this, but have you
considered XML and XPath? I'm not sure of the performance
implications, but it's the alternative that springs most readily to
mind...
 
Datasets are keeping complete result sets and related data in memory, so you
have to deal with unavoidable performance issues.

If you create data sets and tables programmatically, you must implement some
kind of db engine to persist data and page through big tables / result sets.
Which doesn't make much sense considering availability of various db
engines, both commercial and free ones. All replacements will implement some
kind of db engine. XML won;t solve your performance issues and might make
situation worse, as parsing complete files will clog your memory with lots
of residuals. Also, it'll double memory consumption.

Now you have troubles with 30K rows. Consider what kind of issues you'll
have with 300K rows. Don't bump into brick wall.

Real question is: what kind of issues you have with 30K rows. This number is
not that big. Any specific details?
 
My database teacher would laugh at me for saying this, but have you
considered XML and XPath? I'm not sure of the performance
implications, but it's the alternative that springs most readily to
mind...
Click to expand...

I'm pretty sure XML is the exact reason that the DataTable performance is so
awful. Because it's XML internally.
 
* Brian Richards wrote, On 22-7-2007 2:38:
Can anyone suggest a replacement, 3rd party or otherwise, object for
DataTable. We're running into performance issues with large dataset (>30k)
rows. Looking for something that can do binary serialization, typed column
row data, primary keys, filtering etc. SQL desktop engine or some other
database is not an option at this point.

Thanks

Briann
Click to expand...

Have you tried SQL Server CE (previously only available on Windows
Mobile, but work sin any .NET app now).

Jesse
 
Brian said:
I'm pretty sure XML is the exact reason that the DataTable performance is so
awful. Because it's XML internally.
Click to expand...

I don't think that's the case, as DataTables work with data in native
format, and XML is only text.
 
Brian Richards said:
I'm pretty sure XML is the exact reason that the DataTable performance is so
awful. Because it's XML internally.
Click to expand...

No it's not. Just because it can load from XML and save to XML doesn't
mean its internal storage is XML.
 
Back
Top