Dowload File

[rony_16]

Hi,
I have problem with downloading a file using WebRequest and
HttpWebResponse.
Inorder to download this file i need to call to url that tells to
prepare the file for download,
and then i call to another url that downloads the file.
I have a problem with that because:

1. i do not know when the server finished the preparetion.
2.i checked the HttpStatusCode but he doen't changes
(myresponse.StatusCode).

the problem causes the application to download only a part of the file
and that because the server hasn't prepared the file yet.
if i give him Thread.Sleep(100000) i do get the full file, but the file
can be 100Mb and 10sec won't help.

the code is:
//Preparing the file todownload...
//---------------------------------------------------------------------------------------------------------
HttpWebRequest webRequest3 = WebRequest.Create(d1) as
HttpWebRequest;
webRequest3.KeepAlive = true;
webRequest3.Headers.Add("Keep-Alive", "300");
webRequest3.Headers.Add("Accept","text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5");
webRequest3.Headers.Add("Accept-Language",
"en-us,en;q=0.5");
webRequest3.Headers.Add("Accept-Encoding", "gzip,deflate");
webRequest3.Headers.Add("Accept-Charset",
"ISO-8859-1,utf-8;q=0.7,*;q=0.7");

webRequest3.CookieContainer = cookies;
HttpWebResponse myResponse3 =
(HttpWebResponse)webRequest3.GetResponse();

Thread.Sleep(10000);
Stream ReceiveStream3 = myResponse3.GetResponseStream();

//-----------------------------------------------------------------------------------

//Downloading ....
//--------------------------------------------------------------------------------------
Console.WriteLine("Connecting to download link...");
HttpWebRequest webRequest2 = WebRequest.Create(re) as
HttpWebRequest;
webRequest2.ContentType =
"application/x-www-form-urlencoded";
webRequest2.KeepAlive = true;
webRequest2.CookieContainer = cookies;
HttpWebResponse myResponse2 =
(HttpWebResponse)webRequest2.GetResponse();
Stream ReceiveStream2 = myResponse2.GetResponseStream();
Console.WriteLine("Downloading...");

Common.SaveStreamToFile(@"C:\scrapingtest\file.csv",
ReceiveStream2);
Console.WriteLine("Done Downloading");

Thanks, Rony

 

[Kevin Spencer]

You're using the blocking "GetResponse" overload of the HttpWebRequest
method. This means that when the method call returns, the Response is there.
No need to Thread.Sleep.

--
HTH,

Kevin Spencer
Microsoft MVP
Digital Carpenter

A man, a plan, a canal,
a palindrome that has gone to s**t.

 

[rony_16]

I put the Thread.Sleep to check if i give a delay the file will be
full.
without it i get a file that is not fully downloaded.

How can i check if the link i was calling , fineshed prepering the the
file?

 

[Damien]

Hi,
I have problem with downloading a file using WebRequest and
HttpWebResponse.
Inorder to download this file i need to call to url that tells to
prepare the file for download,
and then i call to another url that downloads the file.
I have a problem with that because:

1. i do not know when the server finished the preparetion.
2.i checked the HttpStatusCode but he doen't changes
(myresponse.StatusCode).

the problem causes the application to download only a part of the file
and that because the server hasn't prepared the file yet.
if i give him Thread.Sleep(100000) i do get the full file, but the file
can be 100Mb and 10sec won't help.

the code is:

Thanks, Rony

Let me see if I can understand (and illustrate your problem to others).
You have a system where you perform a web request, say for instance to
http://site/prepareTheFile.aspx, and that produces a file (lets call it
TheFile.txt) which is accessible using a second url
http://site/PreparedFiles/TheFile.txt, and your problem is that you do
not know when the file is completely written (since prepareTheFile.aspx
continues to perform processing after it has returned to you).

If the above describes your situation approximately, then you're not
going to be able to fix this purely from the client end. Is the web
service one over which you have control? If not, then the best you can
do (if the file format is one in which you can tell whether or not it
is complete), is to repeatedly download the file and check whether it
is complete (I'd recommend starting at a wait of 1 second and then
doubling the amount of time you wait each time).

If you do have control over the server end, I can see two options.
Either make the prepareTheFile.aspx page wait until it's completed
production of the file before completing, or prepare the file in a
temporary location (on the same volume as it's eventual location), and
then Move it to the downloadable location once it's complete (in this
scenario, the client again has to poll for completion).

Damien

 

[Leon_Amirreza]

Hi,
just want to share some a like problem (the file gets downloaded partially
and the downloader app fnishes downloading normally without any error, hint
.... ) downloading files from linux servers using windows tools (like IE,
DAP, ...) !?
I tried netscape on linux servers and never had the this problem?
It seems that there maybe some incompatibily or somthing with windows and
some linux file servers? (it maybe irrelevant, or i maybe wrong, ...)

just wanted to share the experience