Data Access Tier with Large Result Set

  • Thread starter Thread starter dm1608
  • Start date Start date
D

dm1608

I'm wondering what the best practice should be when processing a large
result set within a Data tier. Basically, I have a C# app that runs a query
and loops thru over 500K rows and does a few things per row. I would like
to implement a DAL, but I think this would be stupid considering I'd have to
populate a generic collection with all the data, then pass from DAL to BAL.

How to folks handle this in the real world? Running multiple smaller
queries really isn't an option.
 
Hello dm1608,

You are relly need to do it in one DataSet/collection? You can apply async
scheme and send data by blocks.
BTW, if u need to send it bulky - zip it and send


d> I'm wondering what the best practice should be when processing a
d> large result set within a Data tier. Basically, I have a C# app that
d> runs a query and loops thru over 500K rows and does a few things per
d> row. I would like to implement a DAL, but I think this would be
d> stupid considering I'd have to populate a generic collection with all
d> the data, then pass from DAL to BAL.
d>
d> How to folks handle this in the real world? Running multiple
d> smaller queries really isn't an option.
d>
---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsch
 
Need to pass the whole (i.e. 500K) generic collection back to client? Or is
this client direct to sql?

--
William Stacey [MVP]

| I'm wondering what the best practice should be when processing a large
| result set within a Data tier. Basically, I have a C# app that runs a
query
| and loops thru over 500K rows and does a few things per row. I would like
| to implement a DAL, but I think this would be stupid considering I'd have
to
| populate a generic collection with all the data, then pass from DAL to
BAL.
|
| How to folks handle this in the real world? Running multiple smaller
| queries really isn't an option.
|
|
|
 
Well, I actually plan on running my utility app on the server directory
where SQL Server is installed. However, there would be nothing to stop me
from running it from another workstation -- other than poor performance.
 
Back
Top