How can I avoid ftp timout issues?

[Jason]

Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?

I am using SSIS and I have tried multiple ways of ftp'ing the files in my
directory to my local hard drive, but the tasks keep failing after a large
file is downloaded. The error I'm getting is:

An error occurred in the requested FTP operation. Detailed error
description: Receiving file "usfile.zip".
The operation timed out

The file is about 120MB.

I tried using the ftp task, as well as writing (reverse engineering) an ftp
script task using FtpWebRequest.Create and both methods work as long as the
files are small.

It looks like the ftp timeout property has to be set between 1 and 300, so
even though I am explicitly assigning a timeout property of 0 in my
connection via script, it is still timing out.

Any suggestions are much appreciated. I've been struggling with this issue
for about a week now.

Thanks!

Jason
http://www.lonestarfinancing.com

 

[Jason]

Hello Steve,

I checked microsoft's site, and you can see here that the timeout setting
must be between 1 and 300:
http://technet.microsoft.com/en-us/...time.ftpclientconnection.timeout(SQL.90).aspx.

I tried changing it to 0 or a higher value, but it doesn't look it actually
accepts any value other than between 1 and 300.


I've been googling this issue for a week now, and I'm not sure what to do.
Do you know how I can tell if the problem is on my side or on the server?

Thanks,

Jason
http://www.lonestarfinancing.com

 

[J.B. Moreno]

Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?

FTP downloads take place on two connections, most likely your command
connection is timing out for lack of use...you might give it a kick and
send it a command every once in a while.

 

[Jason]

Hello JB,

I am new to programming, but below is the code I'm using. It works until a
large file (120MB) is downloaded. Once it downloads, no other files in the
directory are downloaded, and I get an error msg saying my connection timed
out.


Can you send me an example of what I should do based on my code below?

------------
'And we obtain our file(s):

ftp.ReceiveFiles(sbDownloadFiles.ToString().Substring(0, _

sbDownloadFiles.ToString.Length - 1).Split("|".ToCharArray()), _

Dts.Variables("FtpDestination").Value.ToString, True, False)
-------------

Also, do I need to set KeepAlive to True? I thought is defaulted to True.

Here is my code:

------------------------------------


Public Sub Main()

Dim ftp As FtpClientConnection

Dim retrieveFiles As Boolean = False

Dim ftpWeb As FtpWebRequest

Try

ftpWeb = CType(FtpWebRequest.Create("ftp://" +
Dts.Variables("FTPURL").Value.ToString + _

"/" + Dts.Variables("FTPDirectory").Value.ToString), FtpWebRequest)

ftpWeb.Credentials = New
NetworkCredential(Dts.Variables("FTPUser").Value.ToString,
Dts.Variables("FTPPassword").Value.ToString())

ftpWeb.Method = WebRequestMethods.Ftp.ListDirectoryDetails

'Using fileReader As StreamReader = File.OpenText(baseFileName)

Dim srResponse As New StreamReader(ftpWeb.GetResponse().GetResponseStream())

Dim readLine As String

Dim stringComponents() As String

Dim sbDownloadFiles As New StringBuilder()

Dim fileEnding As String = Dts.Variables("FtpFileEnding").Value.ToString

Dim ignoreDate As Boolean =
Convert.ToBoolean(Dts.Variables("IgnoreDate").Value.ToString)

While Not srResponse.EndOfStream()

readLine = srResponse.ReadLine()

If readLine.Trim().EndsWith(fileEnding) Then

stringComponents = readLine.Split(" ".ToCharArray())

'Since the file is updated at least once a month, we just check the day:

If ignoreDate Or Math.Abs(DateTime.Now.Day -
Int32.Parse(stringComponents(17))) < 2 Then

'This file has been modified within the last day or so...

retrieveFiles = True

sbDownloadFiles.Append(stringComponents(19) + "|")

End If

End If

End While

srResponse.Close()

srResponse = Nothing

ftpWeb = Nothing

If retrieveFiles Then

'Now we need to build our FTP connection:

Dim fiRemoteFile As FileInfo

Dim ftpConnectionManager As ConnectionManager

ftpConnectionManager = Dts.Connections.Add("FTP")

ftpConnectionManager.Properties("ServerName").SetValue(ftpConnectionManager,
Dts.Variables("FTPURL").Value)

ftpConnectionManager.Properties("ServerPort").SetValue(ftpConnectionManager,
Dts.Variables("FTPPort").Value)

ftpConnectionManager.Properties("ServerUserName").SetValue(ftpConnectionManager, Dts.Variables("FTPUser").Value)

ftpConnectionManager.Properties("ServerPassword").SetValue(ftpConnectionManager, Dts.Variables("FTPPassword").Value)

ftpConnectionManager.Properties("Timeout").SetValue(ftpConnectionManager,
Dts.Variables("FTPTimeout").Value)

ftpConnectionManager.Properties("ChunkSize").SetValue(ftpConnectionManager,
Dts.Variables("FtpChunkSize").Value)

ftpConnectionManager.Properties("Retries").SetValue(ftpConnectionManager,
Dts.Variables("FTPRetries").Value)

ftp = New FtpClientConnection(ftpConnectionManager.AcquireConnection(Nothing))

'Next we connect to the ftp server

ftp.Connect()

ftp.SetWorkingDirectory(Dts.Variables("FTPDirectory").Value.ToString)

'And we obtain our file(s):

ftp.ReceiveFiles(sbDownloadFiles.ToString().Substring(0, _

sbDownloadFiles.ToString.Length - 1).Split("|".ToCharArray()), _

Dts.Variables("FtpDestination").Value.ToString, True, False)

End If

'If we did not download anything new there is nothing to process...

Dts.Variables("ContinueProcessing").Value = retrieveFiles

Dts.TaskResult = Dts.Results.Success

Catch exError As Exception

Throw exError

Finally

If Not ftpWeb Is Nothing Then

ftpWeb = Nothing

End If

If Not ftp Is Nothing Then

ftp.Close()

End If

End Try

End Sub

-------------------------------------


Thanks,
Jason

 

[J. Moreno]

Hello JB,

I am new to programming, but below is the code I'm using. It works until
a large file (120MB) is downloaded. Once it downloads, no other files in
the directory are downloaded, and I get an error msg saying my connection
timed out.

Can you send me an example of what I should do based on my code below?

You probably can't do it with FtpWebRequest, it's not meant to be a full
blown FTP client. If it doesn't work under your circumstances (say your
connecting to an IBM Mainframe, or apparently downloading more than xMB)
then it doesn't work and you'll have to write your own FTP code, or get a
library that does it (not hard to do), and use that instead.

The program that I maintain that does FTP uses a clsFTP converted to VB in
2002 by Vick S.:
<http://www.tek-tips.com/viewthread.cfm?qid=1074156&page=8>.

Changing it back to C# isn't too hard (I did it while trying to add SSL
support).

 

[Jialiang Ge [MSFT]]

Hello Jason,

Apart from J. Moreno and Steve's solutions, I'd suggest your trying posting
this SQL Server DTS question to the microsoft.public.sqlserver.dts
newsgroup.

Regards,
Jialiang Ge ([email protected], remove 'online.')
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================

 

[kimiraikkonen]

Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?

I am using SSIS and I have tried multiple ways of ftp'ing the files in my
directory to my local hard drive, but the tasks keep failing after a large
file is downloaded. The error I'm getting is:

An error occurred in the requested FTP operation. Detailed error
description: Receiving file "usfile.zip".
The operation timed out

The file is about 120MB.

I tried using the ftp task, as well as writing (reverse engineering) an ftp
script task using FtpWebRequest.Create and both methods work as long as the
files are small.

It looks like the ftp timeout property has to be set between 1 and 300, so
even though I am explicitly assigning a timeout property of 0 in my
connection via script, it is still timing out.

Any suggestions are much appreciated. I've been struggling with this issue
for about a week now.

Thanks!

Jasonhttp://www.lonestarfinancing.com

That may not do the trick but you may try, have you tried:

My.Computer.Network.DownloadFile(....) method and increase
"connectionTimeout" parameter to what you want.(It represents 32-bit
integer to specify).

See this:
http://msdn.microsoft.com/en-us/library/ack30t8y.aspx

Onur Güzel