Socket.Select() > 64 connections

[Stuart]

I am trying to execute a Socket.Select() statement on an arraylist of
sockets. The problem is that I can only go up to 64 sockets at a time. I
know that I have to manipulate the FD_SetSize to increase the limit. How
do you do that in C# in the .NET Visual Studio environment?

I could potentially have 1000's of simultaneous connections, so running
each socket in it's own thread doesn't seem very practical.

Stuart.

 

[Guest]

I did this in C++ some time ago using the concepts of "Overlapped IO" and "IO
Completion Ports". This is very scalable (my service talked to hundreds of
clients simultaneously and never lost a connection or data) and only uses few
threads based on the number of processors in your server. There were examples
on MSDN. I'm sure there's one for the .NET environment. Search on these key
words and perhaps you can find something. Hope this helps.

 

[Lenard Gunda]

Hey,

Have you tried out that limit in reality?

I fired up Reflector and took a peek at how Socket.Select() works in the
2.0 Framework, and it checks that not all sets are empty, and that no
one of them contains more than 0x10000 sockets (which would be a lot).

I guess this would mean that it should work for sets that are between 1
and 0x10000 in size. (anyone can confirm?)

-Lenard

 

[Lenard Gunda]

Hi,

I checked further. The managed Select() converts the sets to IntPtr[]
arrays. The first element contains the number of items (socket handles)
in the array, while subsequent elements are the elements from the set.

I checked winsock2.h from the platform SDK, and an fd_set is just
implemented like that. A single unsigned number containing the number of
sockets in the set, and then an array of SOCKET handles. When you set
FD_SETSIZE in C++, you have to set it before you include windows.h (or
winsock2.h). This way, the fd_set type is declared with a constant size
of FD_SETSIZE. Although this is a C# newsgroup, I hope some C++ is not a
problem. So here is the part from winsock2.h:

#ifndef FD_SETSIZE
#define FD_SETSIZE 64
#endif /* FD_SETSIZE */

typedef struct fd_set {
u_int fd_count; /* how many are SET? */
SOCKET fd_array[FD_SETSIZE]; /* an array of SOCKETs */
} fd_set;

This means that you should not worry about this in .NET, because it is
implemented for you transparently. Go ahead and use Socket.Select() for
socket arrays that contain less than 0x10000 elements. (thats 65536 in
decimal). It will work.

-Lenard

 

[Stuart]

There is a limit in 1.1 for sure. I haven't tried it on 2.0 yet. I
tried running my .exe on Mono, and it bombed out at over 1000
connections, which is a lot better than 64!

Stuart.

 

[Lenard Gunda]

Hi,

I checked the .NET 1.1 version of Select with Reflector, and I think it
should work there too with many sockets.

What kind of IList do you use for your sets? Is it an array or ArrayList
or something of your own?

-Lenard

 

[Guest]

Do these connections have to be "up" all the time, or is there just some data
exchanged and then you could recycle the socket for another connection?
In that case you could use a thread pool with asynchronous callbacks.

Ami Bar has an excellent piece of work he calls "Smart Threadpool" that you
can find on codeproject.com

Peter

 

[Stuart]

Hi,

I checked the .NET 1.1 version of Select with Reflector, and I think it
should work there too with many sockets.

What kind of IList do you use for your sets? Is it an array or ArrayList
or something of your own?

-Lenard

Here is a simplified version of the code:

ArrayList sockList = new ArrayList();
ArrayList copyList = new ArrayList();

Socket SocketRX = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);

IPEndPoint iep = new IPEndPoint(IPAddress.Any, clsMain.PortTCP);
byte[] data;
int recv;

SocketRX.SetSocketOption(SocketOptionLevel.Socket,SocketOptionName.ReuseAddress,
1);
SocketRX.Bind(iep);
SocketRX.Listen(1000);

while (!quit)
{
if (SocketRX.Poll(0,SelectMode.SelectRead)) //Test for new connections
{ //New Connection
Socket NewSocket = SocketRX.Accept(); //Accept the connection
SockInfo.IPEndP=(IPEndPoint)NewSocket.RemoteEndPoint;
sockList.Add(NewSocket); //Add Socket to the SocketList

Console.WriteLine("Soc: Connected to {0}", SockInfo.IPEndP.ToString());
}
else
{ //Check for any pending data
copyList = new ArrayList(sockList);
Socket.Select(copyList, null, null, 5000000); //block for .5 second
foreach(Socket client in copyList) //Read data from sockets that have
data
{
data = new byte[RXBufferSize];
try
{
recv = client.Receive(data); //Read the incomming data
}
catch (Exception ex)
{
recv=0;
}
if (recv == 0)
{
CloseSocket(client);
}
else
{
//Proccess my data here....
}
}
}
}

 

[Stuart]

Do these connections have to be "up" all the time, or is there just some data
exchanged and then you could recycle the socket for another connection?
In that case you could use a thread pool with asynchronous callbacks.

Ami Bar has an excellent piece of work he calls "Smart Threadpool" that you
can find on codeproject.com

The sockets have to be up all the time. That data thoughput is small,
and random.

Stuart.

 

[Vadym Stetsyak]

You can utilize sockets in async mode.
I mean Socket.BeginReceive/EndReceive and
Socket.BeginSend/EndSend

This approach is more efficient because it uses system thread pool with
overlapped I/O ( as in good old win32 times was made with the help of
CompletionPort)

Have a look at for more details

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/progthrepool.asp