Large DataSet problem : Please help

[palgre]

Hi ALL,

I am working in an windows based application using SQL Server 2000 as
database. There are few tables (refer parent tables) in the
application which are uploaded by a seprate application.

My application fetches data from parent tables and put data in
separate tables (chils tables) of application.
I am using dataset to fetch data from parent tables and insert/update
data in child table.
the problem is the records are so high (3 4 millions) then data is too
large and it takes hell lot of time to complete the process. Also
application server CPU utilization shoots out to max.

What will be the best way to achive this.
1. Should i use DataRepeater instead of dataset. or
2. Should i do processing in chunks. how can I do processing in
chunks???

or is there any other way i can process data.

Thanks
Sumit

 

[Guest]

(e-mail address removed) wrote in @w3g2000hsg.googlegroups.com:

the problem is the records are so high (3 4 millions) then data is too
large and it takes hell lot of time to complete the process. Also
application server CPU utilization shoots out to max.

What will be the best way to achive this.
1. Should i use DataRepeater instead of dataset. or
2. Should i do processing in chunks. how can I do processing in
chunks???

More 10,000 rows in general is too much data for a dataset.

You should be processing in chunks - take a look at "Paging".

Also do you need ALL the data? Why are you sending so much data to the
client?