Do I need to use GC.Collect? If yes, where?

[Antonio]

Hello, here is what I am doing.

1) user accesses web page and clicks button to build report
2) web page accesses web service whose data type is DataSet
(public DataSet methodname()...)
3) web service connects to SqlServer, runs BIG report
4) data is returned to web service, web service returns DataSet to web
front-end
5) web front-end prompts user to save Excel file which contains the
DataSet data

While #3 is running, the aspnet_ws.exe process on the web server grows
big, like 200-500 MB.

The problem is the server doesn't seem to release that memory after the
report is done. A few more report runs and the server starts giving
memory errors.

I have seen people saying use GC.Collect() but where would I put that?

 

[Nicholas Paldino [.NET/C# MVP]]

Antonio,

No, you shouldn't be calling GC.Collect, ^especially^ in an ASP.NET
environment. ASP.NET gets into a nice little groove (in terms of GC's, and
when the pattern of use is consistent) which you don't want to mess with by
running your own GC's.

The errors that you are probably getting are due to the % of memory
ASP.NET is allowed to use before the process is recycled. You would have to
up this value in the web.config file for your application, to let your
ASP.NET process take up more memory.

If your report is returning data that is between 200-500 MB, then you
really need to reconsider the design of the report.

Also, Task Manager is not really suitable for monitoring the memory
consumption of an application. You should be using the performance counters
to determine what is being consumed/released, etc, etc.

In the end, ^don't^ call GC.Collect, as in this case, it isn't going to
do anything for you. Just make sure you dispose of all types that implement
IDisposable, and if your report is really that big, then raise the value of
the allowed memory consumption in the web.config file.

Hope this helps.

 

[Vagabond Software]

Hello, here is what I am doing.

1) user accesses web page and clicks button to build report
2) web page accesses web service whose data type is DataSet
(public DataSet methodname()...)
3) web service connects to SqlServer, runs BIG report
4) data is returned to web service, web service returns DataSet to web
front-end
5) web front-end prompts user to save Excel file which contains the
DataSet data

While #3 is running, the aspnet_ws.exe process on the web server grows
big, like 200-500 MB.

The problem is the server doesn't seem to release that memory after the
report is done. A few more report runs and the server starts giving
memory errors.

I have seen people saying use GC.Collect() but where would I put that?

You would probably add a GC.Collect after you call Dispose on the object.
However, I wonder if that is the problem? Is your 200-500 MB data a group
of unmanaged Excel objects? If so, are you sure you're releasing ALL of the
objects?

Carl

 

[Antonio]

ok so don't call GC.Collect(). I have added .Dispose to the datagrids
and datasets. The actual excel file downloaded is usually between 5
and 25 MB. I am not really sure why the process grows by 200 or more
MB. What performance counters would you suggest I watch in perfmon?

thank you Nicholas

 

[Willy Denoyette [MVP]]

The question is not how big is the excel file, the question is how big is
the dataset.
And what datagrids are you talking about? This is a webservice, what are the
datagrids used for?

Willy.

 

[Nicholas Paldino [.NET/C# MVP]]

Antonio,

It depends on what you want to see. If you run perfmon (from the run
dialog box), you will see a gazillion (ok, almost) entries for performance
counters. You can see things such as number of collections, memory
allocated, etc, etc. They are pretty self-describing, and can give you a
plethora of information.

Also, when working with the Excel sheet, you should make sure that you
call the static ReleaseComObject on the Marshal class to correctly release
the COM objects produced by it (it can leave references lying around).

 

[Antonio]

The datagrid is to get the excel output. See below. I found this in
the asp.net ng. I call the web service which returns the dataset, then
I pass the dataset into this static method (I made it a static method
so all of my web pages can call it.

public static void ExportExcel(System.Data.DataSet ds, int TableIndex,
string FileNameNoExtension, System.Web.HttpResponse response) {
response.Clear();
response.Charset = "";
response.ContentType = "application/vnd.ms-excel";
response.AddHeader("content-disposition", "attachment;filename=" +
FileNameNoExtension + ".xls");
System.IO.StringWriter stringWrite = new System.IO.StringWriter();
System.Web.UI.HtmlTextWriter htmlWrite = new
System.Web.UI.HtmlTextWriter(stringWrite);
System.Web.UI.WebControls.DataGrid dg = new
System.Web.UI.WebControls.DataGrid();
dg.DataSource = ds.Tables[TableIndex];
dg.DataBind();
dg.RenderControl(htmlWrite);
response.Write(stringWrite.ToString());
response.End();
ds.Dispose();
}

 

[John Murray]

Are you releasing your Excel references with Marshal.ReleaseCOMObject,
or just setting them to null?

http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q317109

Please note that this is one of the cases where MS actually does
reference manually calling GC.Collect ... although it's sort of a hack
to deal with reference counting problems in the RCW.

 

[Willy Denoyette [MVP]]

The datagrid is to get the excel output. See below. I found this in
the asp.net ng. I call the web service which returns the dataset, then
I pass the dataset into this static method (I made it a static method
so all of my web pages can call it.

public static void ExportExcel(System.Data.DataSet ds, int TableIndex,
string FileNameNoExtension, System.Web.HttpResponse response) {
response.Clear();
response.Charset = "";
response.ContentType = "application/vnd.ms-excel";
response.AddHeader("content-disposition", "attachment;filename=" +
FileNameNoExtension + ".xls");
System.IO.StringWriter stringWrite = new System.IO.StringWriter();
System.Web.UI.HtmlTextWriter htmlWrite = new
System.Web.UI.HtmlTextWriter(stringWrite);
System.Web.UI.WebControls.DataGrid dg = new
System.Web.UI.WebControls.DataGrid();
dg.DataSource = ds.Tables[TableIndex];
dg.DataBind();
dg.RenderControl(htmlWrite);
response.Write(stringWrite.ToString());
response.End();
ds.Dispose();
}

Ok, if I understand you correctly, you have following logical tiers:

1. browser client
2. a web application (asp.net)
3. a web service
4. a SQL DB

How about the physical tiers, do 2 and 3 run on the same server (not really
a server, if I'm not mistaken you run this on XP)?
Who's producing the Excel output? And what's the relation between this and
the DS generated by the report?

How large is the dataset passed from the WS to the WA, this is important as
if both run in the same process (aspnet_wp.exe) the working set will contain
two instances of the same DS at some point in time.


Willy.

 

[Antonio]

Yes!

The server are Windows Server 2000 SP4. 2 and 3 DO run on the same
server! #2 is producing the excel file. #3 only connects to the
database and returns to #2 the dataset. then the dataset is handed to
the static method that productces the excel file.

I do not know how big the dataset is but I will try to look.

 

[Willy Denoyette [MVP]]

Yes!

The server are Windows Server 2000 SP4. 2 and 3 DO run on the same
server! #2 is producing the excel file. #3 only connects to the
database and returns to #2 the dataset. then the dataset is handed to
the static method that productces the excel file.

I do not know how big the dataset is but I will try to look.

Does 2 produce the Excel file through Office automation (COM)? If the answer
is yes, then all I can say is change you design and don't run Office
automation on the server from web applications. Office is not designed to be
used in such scenarios, "it doesn't scale" is an often heard issue but it's
not the worse.
Check following KB article as to why it is like this:
http://support.microsoft.com/default.aspx?scid=kb;en-us;257757

Willy.

 

[Antonio]

Yes thank you that is a great answer. This is not using COM or any
office interop though. Please see the code in my earlier post.

Maybe I can try to pass to the static method the dataset by ref as to
not copy it in memory. Does that make a good suggestion?