FTP NLST VB.Net

  • Thread starter Thread starter gabedog
  • Start date Start date
G

gabedog

I just started maintaining an existing VB.Net FTP application that
keeps directories in sync accross a firewall. The number of files and
the contents of the files has grown quite large (very recently due to a
new client)

There are anywhere from 4,000 to 40,000 files in a directory. The file
names are 20+ chars. An exception is being kicked out in the
application. The exception occurs during an NLST command when there are
more than the 4,000 files in a directory. The exception is:

"An existing connection was forcibly closed by the remote host. "

Is there a limit on the NLST command that would be causing this?

Thanks in advance --- Of course this happening very close to another
production release :- )

Gabe--
 
Check to see if you have a timeout property on the remote server and in
the existing FTP application. Up them up to some obscene number and see
if there is any change in the situation.

If it appears to be fixed, set down and begin to write code that will
not bump up against a standard timeout setting. If this didn't work I am
sorry.
 
I just started maintaining an existing VB.Net FTP application that
keeps directories in sync accross a firewall. The number of files and
the contents of the files has grown quite large (very recently due to a
new client)

There are anywhere from 4,000 to 40,000 files in a directory. The file
names are 20+ chars. An exception is being kicked out in the
application. The exception occurs during an NLST command when there are
more than the 4,000 files in a directory. The exception is:

"An existing connection was forcibly closed by the remote host. "

Is there a limit on the NLST command that would be causing this?

Thanks in advance --- Of course this happening very close to another
production release :- )

I've looked at the RFC, and I don't see any kind of limit defined for
the NLST command.
 
Some servers have a setting (such as UNIX, etc) that requests that take
over a certain amount of time the server will automatically bomb it out.

Some controls also have this setting as a property of the control. (Dart
Telnet, FTP controls come to mind)

It assumes any request made that takes longer than x seconds is an
erroneous request and returns as a failure.
 
Thank you so much for your help. ---

I've upped the connection timeout limit on the FTP sites, and I'm
getting the same error. I'm going to talk to some of our network folks
and see about the server issue.

Thanks, Gabe
 
[This followup was posted to microsoft.public.dotnet.languages.vb and a
copy was sent to the cited author.]

I just started maintaining an existing VB.Net FTP application that
keeps directories in sync accross a firewall. The number of files and
the contents of the files has grown quite large (very recently due to a
new client)

There are anywhere from 4,000 to 40,000 files in a directory. The file
names are 20+ chars. An exception is being kicked out in the
application. The exception occurs during an NLST command when there are
more than the 4,000 files in a directory. The exception is:

"An existing connection was forcibly closed by the remote host. "
Did this happen before your new client uploaded his files?
Is there a limit on the NLST command that would be causing this?
There is not a limit as far as I know.

About the only thing I could see is a firewall configuration problem.
The author of NcFTP and NcFTPD wrote a nice little article about this:

http://www.ncftp.com/ncftpd/doc/misc/ftp_and_firewalls.html

Our Indy Knowlege base has a complete section devoted to firewall/proxy
configurations that is worth reading even if you do not use Indy. The
knowlege base is here:

http://www.indyproject.org/KB/

A dirty little secret about FTP is that it's NOT firewall friendly at
all. In fact, sometimes, I think that FTP is a good leason in how NOT
to write a protocol.

--
J. Peter Mugaas - Indy Pit Crew
Internet Direct (Indy) Website - http://www.indyproject.org
Check our my blog at
http://www.indyproject.org/Sockets/Blogs/JPeterMugaas/index.iwp
If I want to do business with you, I will contact you. Otherwise, do
not contact me.
 
FTP isnt bad. If you want an example of a horrible protocol, look at IMAP4.

Note that I'm saying this not for your benefit but for the benefit of
our listening audience.

FTP does have it's own pitfalls that really bite.

1) There is no original directory listing format so parsing the output
is always going to be a problem that can never be solved. Even dates
are not as reliable as you think because you don't know what time zone
are dealing with.
2) There is a separate data channel. During transmissions, the
control channel is idle and unfortunately, a firewall might cut an
idle connection. I suspect that's what our original poster complained
about.
3) To establish a data channel, an IP address is specified along with
a port. That means that someone could use a FTP server to access an
internal network or transmit a specially crafted file as part of an
attack against someone else. Another problem is that some
workstations are in internal networks and those have special
designated addresses that don't make any sense on the Internet (in
fact, a lot of routers will drop packets with those internal network
IP addresses. Usually, NAT's can change those IP addresses but in FTP
with SSL, that's impossible. In IPv6, a special command had to be
introduced because the standard PORT and PASV commands can't
communicate an IPv6 address.
4) There is no standardized way to modify some attributes of a file
such as permissions, ownership, and time stamps. SITE commands have
been used but there's no standardized SITE commands at all.
Unfortunately, the MFF draft doesn't seem to be adopted widely and
attempts to solve that problem.
5) In PASV transfers the server listens on a specific port and the
client connects to that port. That makes it easy for someone else to
hijack that data connection.
6) During a data transfer, in order to abort, you have to a special
command and that's not always implemented correctly by FTP servers. To
implement the abort command correctly involves quite some complexity.
SFTP (SSH file transfer) is better because you send data in block
sizes and to abort, you simply stop requesting those blocks. 7) In FTP
PASV transfers, the server has to connect to the client from a port in
the reserved port range (1-1024). That makes it impossible for the
server to completely give up administrative privileges after the
initial bind.

I'm sure that I can go on quite bit about this stuff and I admit that
some of the problems are worked around to some extent. If anything,
FTP has probably provided a good lesson that the writers of FSP
(http://fsp.sf.net) and the SSH file transfer subsystem have learned
from.
 
Back
Top