Threading a server

[David]

Hi all,

A while ago, I asked about threading, but I am new to it and really
struggling.

I thought I had it cracked and the project has gone live, but quickly
presented problems due to the threading.

I have the basic smtp mailserver from here, that I have modified to suit me.
http://forums.whirlpool.net.au/forum-replies-archive.cfm/654973.html

However, it has a blocking AcceptTcpClient which is not good for me. There
are potentially many connections required at a time.

Here is my code so far...

*******************************************************************************

private void ReceiveSMTP()
{

System.Net.IPAddress ipaddress = System.Net.IPAddress.Any;
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);
BackgroundWorker bw = new BackgroundWorker();

bw.DoWork += delegate(object sender, DoWorkEventArgs e)
{

Thread runThread = new Thread(new ThreadStart(RunSMTP));
runThread.Start();

};



bw.RunWorkerCompleted += delegate(object sender,
RunWorkerCompletedEventArgs e)
{

string MailContent = (string)e.Result;

};


bw.RunWorkerAsync();


}



private void RunSMTP()
{
System.Net.IPAddress ipaddress = System.Net.IPAddress.Any;
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);

TcpListener listener = new TcpListener(ipaddress, PortNo);
listener.Start();
while (true)
{
SMTP.SMTPServer handler = new
SMTP.SMTPServer(listener.AcceptTcpClient());
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

Thread thread = new System.Threading.Thread(new
ThreadStart(handler.Run));
thread.Start();

// Watchdog.
// If the thread is still going after 3 minutes, abort the
thread.
thread.Join(180000);
if (thread.IsAlive)
{
thread.Abort();
}

}
}


*******************************************************************************

The bw.RunWorkerCompleted is no longer being used. My original question was
about getting information out of the thread. I have gone the other way and
am now passing information into the thread and letting the thread do the
remaining part of hte work.

The handler.Run calls the Run in the mailserver from the link above. (There
are a few modifications that the mailserver.)



However, the problem is that even though I have attempted to put RunSMTP
into its own thread, I now realise that this wouldn't work, but don't have
enough knowledge to fix it.

If I send one email at a time to the server, it works fine, and as long as
that email finishes before the next email, everything is hunky dorey
(thereby leading me into a false sense of security).

However, while one mail is being handled, then if another comes in, then it
is sat waiting until the first one finishes, and if the first one fails to
send a quit command, it is waiting until my thread.Abort is done. (3
minutes).


I have also tried
http://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener.beginaccepttcpclient.aspx
to work around the blocking issue, but I REALLY don't understand it. I have
also tried
http://thesoftwareeconomist.com/ind...eaded-tcp-server&tmpl=component&print=1&page=
and putting my handler into the ListenerThreadDelegate

protected static void ListenerThreadDelegate(object obj)
{
IPAddress ipAddress = (IPAddress)obj;
TcpListener listener = null;
try
{
if (ipAddress != null)
{
listener = new TcpListener(((IPAddress)ipAddress),
portToListenOn); //_port);
TCPListenerCollection.Add(listener);
listener.Start();
while (true)
{
try
{
if (listener.Pending())
{
Socket clientSocket =
listener.AcceptSocket();

// Old SMTP
SMTP.SMTPServer handler = new
SMTP.SMTPServer(listener.AcceptTcpClient());
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

// Old
//Thread thread = new
System.Threading.Thread(new ThreadStart(handler.Run));
//thread.Start();
// New
//Thread newThread = new Thread(new
ParameterizedThreadStart(SimpleTCPServer.ProcessMessages));
Thread newThread = new Thread(new
ParameterizedThreadStart(handler.Run));
newThread.Start(clientSocket);

// Watchdog.
// If the thread is still going after 3
minutes, abort the thread.
newThread.Join(180000);
if (newThread.IsAlive)
{
newThread.Abort();
}
// End Old SMTP

}
}
catch (SocketException ex)
{
//Log error
}
}
}
else
{
return;
}
}
catch (System.Net.Sockets.SocketException exception)
{
//Log error
}
catch (Exception exception)
{
//Log error
}
}

but I am just getting high processor usage AND for some reason, it is not
even picking up my connection.


Any help on how I should make this work would be VERY MUCH appreciated. I
have a live server offline at the moment. (Oh, I am using .NET 3.5, C#)

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[David]

Sorted it... I found this and re-wrote it to handle my smtp. I can now
connect to the server from many clients.

http://www.switchonthecode.com/tutorials/csharp-tutorial-simple-threaded-tcp-server

Stressful time over (hopefully).
--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

Hi all,

A while ago, I asked about threading, but I am new to it and really
struggling.

I thought I had it cracked and the project has gone live, but quickly
presented problems due to the threading.

I have the basic smtp mailserver from here, that I have modified to suit
me.
http://forums.whirlpool.net.au/forum-replies-archive.cfm/654973.html

However, it has a blocking AcceptTcpClient which is not good for me. There
are potentially many connections required at a time.

Here is my code so far...

*******************************************************************************

private void ReceiveSMTP()
{

System.Net.IPAddress ipaddress = System.Net.IPAddress.Any;
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);
BackgroundWorker bw = new BackgroundWorker();

bw.DoWork += delegate(object sender, DoWorkEventArgs e)
{

Thread runThread = new Thread(new
ThreadStart(RunSMTP));
runThread.Start();

};



bw.RunWorkerCompleted += delegate(object sender,
RunWorkerCompletedEventArgs e)
{

string MailContent = (string)e.Result;

};


bw.RunWorkerAsync();


}



private void RunSMTP()
{
System.Net.IPAddress ipaddress = System.Net.IPAddress.Any;
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);

TcpListener listener = new TcpListener(ipaddress, PortNo);
listener.Start();
while (true)
{
SMTP.SMTPServer handler = new
SMTP.SMTPServer(listener.AcceptTcpClient());
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

Thread thread = new System.Threading.Thread(new
ThreadStart(handler.Run));
thread.Start();

// Watchdog.
// If the thread is still going after 3 minutes, abort the
thread.
thread.Join(180000);
if (thread.IsAlive)
{
thread.Abort();
}

}
}


*******************************************************************************

The bw.RunWorkerCompleted is no longer being used. My original question
was about getting information out of the thread. I have gone the other way
and am now passing information into the thread and letting the thread do
the remaining part of hte work.

The handler.Run calls the Run in the mailserver from the link above.
(There are a few modifications that the mailserver.)



However, the problem is that even though I have attempted to put RunSMTP
into its own thread, I now realise that this wouldn't work, but don't have
enough knowledge to fix it.

If I send one email at a time to the server, it works fine, and as long as
that email finishes before the next email, everything is hunky dorey
(thereby leading me into a false sense of security).

However, while one mail is being handled, then if another comes in, then
it is sat waiting until the first one finishes, and if the first one fails
to send a quit command, it is waiting until my thread.Abort is done. (3
minutes).


I have also tried
http://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener.beginaccepttcpclient.aspx
to work around the blocking issue, but I REALLY don't understand it. I
have also tried
http://thesoftwareeconomist.com/ind...eaded-tcp-server&tmpl=component&print=1&page=
and putting my handler into the ListenerThreadDelegate

protected static void ListenerThreadDelegate(object obj)
{
IPAddress ipAddress = (IPAddress)obj;
TcpListener listener = null;
try
{
if (ipAddress != null)
{
listener = new TcpListener(((IPAddress)ipAddress),
portToListenOn); //_port);
TCPListenerCollection.Add(listener);
listener.Start();
while (true)
{
try
{
if (listener.Pending())
{
Socket clientSocket =
listener.AcceptSocket();

// Old SMTP
SMTP.SMTPServer handler = new
SMTP.SMTPServer(listener.AcceptTcpClient());
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

// Old
//Thread thread = new
System.Threading.Thread(new ThreadStart(handler.Run));
//thread.Start();
// New
//Thread newThread = new Thread(new
ParameterizedThreadStart(SimpleTCPServer.ProcessMessages));
Thread newThread = new Thread(new
ParameterizedThreadStart(handler.Run));
newThread.Start(clientSocket);

// Watchdog.
// If the thread is still going after 3
minutes, abort the thread.
newThread.Join(180000);
if (newThread.IsAlive)
{
newThread.Abort();
}
// End Old SMTP

}
}
catch (SocketException ex)
{
//Log error
}
}
}
else
{
return;
}
}
catch (System.Net.Sockets.SocketException exception)
{
//Log error
}
catch (Exception exception)
{
//Log error
}
}

but I am just getting high processor usage AND for some reason, it is not
even picking up my connection.


Any help on how I should make this work would be VERY MUCH appreciated. I
have a live server offline at the moment. (Oh, I am using .NET 3.5, C#)

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[David]

Hhhmmmm

I had it fixed and working, then I put in a watchdog...

this.tcpListener.Start();

while (true)
{
//blocks until a client has connected to the server
TcpClient client = this.tcpListener.AcceptTcpClient();

//create a thread to handle communication
//with connected client
Thread clientThread = new Thread(new
ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);

// Watchdog.
// If the thread is still going after 3 minutes, abort the
thread.
//clientThread.Join(180000);
clientThread.Join(30000);
if (clientThread.IsAlive)
{
clientThread.Abort();
}
}

However, this then appears to block again. Perhaps this is what was causing
my problem before.

How do I get around this problem? (I have verified it is the watchdog by
removing it)

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

Hi all,

A while ago, I asked about threading, but I am new to it and really
struggling.

I thought I had it cracked and the project has gone live, but quickly
presented problems due to the threading.

I have the basic smtp mailserver from here, that I have modified to suit
me.
http://forums.whirlpool.net.au/forum-replies-archive.cfm/654973.html

However, it has a blocking AcceptTcpClient which is not good for me. There
are potentially many connections required at a time.

Here is my code so far...

*******************************************************************************

private void ReceiveSMTP()
{

System.Net.IPAddress ipaddress = System.Net.IPAddress.Any;
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);
BackgroundWorker bw = new BackgroundWorker();

bw.DoWork += delegate(object sender, DoWorkEventArgs e)
{

Thread runThread = new Thread(new
ThreadStart(RunSMTP));
runThread.Start();

};



bw.RunWorkerCompleted += delegate(object sender,
RunWorkerCompletedEventArgs e)
{

string MailContent = (string)e.Result;

};


bw.RunWorkerAsync();


}



private void RunSMTP()
{
System.Net.IPAddress ipaddress = System.Net.IPAddress.Any;
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);

TcpListener listener = new TcpListener(ipaddress, PortNo);
listener.Start();
while (true)
{
SMTP.SMTPServer handler = new
SMTP.SMTPServer(listener.AcceptTcpClient());
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

Thread thread = new System.Threading.Thread(new
ThreadStart(handler.Run));
thread.Start();

// Watchdog.
// If the thread is still going after 3 minutes, abort the
thread.
thread.Join(180000);
if (thread.IsAlive)
{
thread.Abort();
}

}
}


*******************************************************************************

The bw.RunWorkerCompleted is no longer being used. My original question
was about getting information out of the thread. I have gone the other way
and am now passing information into the thread and letting the thread do
the remaining part of hte work.

The handler.Run calls the Run in the mailserver from the link above.
(There are a few modifications that the mailserver.)



However, the problem is that even though I have attempted to put RunSMTP
into its own thread, I now realise that this wouldn't work, but don't have
enough knowledge to fix it.

If I send one email at a time to the server, it works fine, and as long as
that email finishes before the next email, everything is hunky dorey
(thereby leading me into a false sense of security).

However, while one mail is being handled, then if another comes in, then
it is sat waiting until the first one finishes, and if the first one fails
to send a quit command, it is waiting until my thread.Abort is done. (3
minutes).


I have also tried
http://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener.beginaccepttcpclient.aspx
to work around the blocking issue, but I REALLY don't understand it. I
have also tried
http://thesoftwareeconomist.com/ind...eaded-tcp-server&tmpl=component&print=1&page=
and putting my handler into the ListenerThreadDelegate

protected static void ListenerThreadDelegate(object obj)
{
IPAddress ipAddress = (IPAddress)obj;
TcpListener listener = null;
try
{
if (ipAddress != null)
{
listener = new TcpListener(((IPAddress)ipAddress),
portToListenOn); //_port);
TCPListenerCollection.Add(listener);
listener.Start();
while (true)
{
try
{
if (listener.Pending())
{
Socket clientSocket =
listener.AcceptSocket();

// Old SMTP
SMTP.SMTPServer handler = new
SMTP.SMTPServer(listener.AcceptTcpClient());
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

// Old
//Thread thread = new
System.Threading.Thread(new ThreadStart(handler.Run));
//thread.Start();
// New
//Thread newThread = new Thread(new
ParameterizedThreadStart(SimpleTCPServer.ProcessMessages));
Thread newThread = new Thread(new
ParameterizedThreadStart(handler.Run));
newThread.Start(clientSocket);

// Watchdog.
// If the thread is still going after 3
minutes, abort the thread.
newThread.Join(180000);
if (newThread.IsAlive)
{
newThread.Abort();
}
// End Old SMTP

}
}
catch (SocketException ex)
{
//Log error
}
}
}
else
{
return;
}
}
catch (System.Net.Sockets.SocketException exception)
{
//Log error
}
catch (Exception exception)
{
//Log error
}
}

but I am just getting high processor usage AND for some reason, it is not
even picking up my connection.


Any help on how I should make this work would be VERY MUCH appreciated. I
have a live server offline at the moment. (Oh, I am using .NET 3.5, C#)

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[David]

Hi all,

I still got problems with it...

I have sorted out the watchdog process by having a parent thread. May not be
the most elegant. I then experienced 100% CPU so have put in thread.sleep to
sort that.

However, when I test locally, it all appears to work, though when I put it
live and sendmail is sending to it, it receives first time and if the next
job comes within my timeout period of 3 minutes, it hangs.

Here is my source... Any help at all would be appreciated. This is in a
windows service.

I am getting myself into a really big hole with this and the client is (to
put it politely) a little upset. :-(



********************************************************************************************************
//windows service


private TcpListener tcpListener;
private Thread listenThread;

protected override void OnStart(string[] args)
{
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);

this.tcpListener = new TcpListener(IPAddress.Any, PortNo);
this.listenThread = new Thread(new
ThreadStart(ListenForClients));
this.listenThread.Start();
}

private void ListenForClients()
{
this.tcpListener.Start();

while (true)
{
//blocks until a client has connected to the server
TcpClient client = this.tcpListener.AcceptTcpClient();

//create a thread to handle communication
//with connected client
Thread clientThread = new Thread(new
ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);

Thread.Sleep(250);

}
}


private void HandleClientComm(object tclient)
{
while (true)
{
//blocks until a client has connected to the server
TcpClient client = (TcpClient)tclient;
//create a thread to handle communication
//with connected client
Thread clientThread = new Thread(new
ParameterizedThreadStart(HandleClient));
clientThread.Start(client);

Thread.Sleep(250);

// Watchdog.
// If the thread is still going after 3 minutes, abort the
thread.
clientThread.Join(180000);
if (clientThread.IsAlive)
{
clientThread.Abort();
}
}
}

private void HandleClient(object client)
{
try
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();

SMTP.SMTPServer handler = new SMTP.SMTPServer(tcpClient);
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

handler.Run(client);

Thread.Sleep(250);

tcpClient.Close();
}
catch
{
}
}


********************************************************************************************************
SMTP class This is a seperate file.

TcpClient client;
NetworkStream stream;
System.IO.StreamReader reader;
System.IO.StreamWriter writer;

static Encoding s_enc = Encoding.ASCII;

private string _CurrentCommand = string.Empty;
private string CurrentCommand
{
set { _CurrentCommand = value; }
get { return _CurrentCommand; }
}

private string _EmailContent = string.Empty;
public string EmailContent
{
get { return _EmailContent; }
}

private string _ExpectedSubject = string.Empty;
public string ExpectedSubject
{
get { return _ExpectedSubject; }
set { _ExpectedSubject = value; }
}

private string _PenHandlerURL = string.Empty;
public string PenHandlerURL
{
get { return _PenHandlerURL; }
set { _PenHandlerURL = value; }
}

private string _LimitIPAddress = string.Empty;
public string LimitIPAddress
{
set { _LimitIPAddress = value; }
}

private bool CloseConnection = false;

private bool HeloOK = false;

public SMTPServer(TcpClient client)
{
this.client = client;
stream = client.GetStream();
reader = new System.IO.StreamReader(stream);
writer = new System.IO.StreamWriter(stream);
writer.NewLine = "\r\n";
writer.AutoFlush = true;

}

private bool _LogToFile = false;
public bool LogToFile
{
get { return _LogToFile; }
set { _LogToFile = value; }
}

private bool _LogVerbose = false;
public bool LogVerbose
{
get { return _LogVerbose; }
set { _LogVerbose = value; }
}


public void Run(object obj)
{
IPEndPoint ipend = (IPEndPoint)client.Client.RemoteEndPoint;

if (_LimitIPAddress != string.Empty)
{
if (ipend.Address.ToString() != _LimitIPAddress)
{
writetolog("Attempted connection from : " +
ipend.Address.ToString() + " The connection will be closed.");
client.Close();
return;
}
}
else
{

writer.WriteLine("220 SMTP server");

string MailContent = string.Empty;

for (string line = reader.ReadLine(); line != null; line =
reader.ReadLine())
{
string CommandVerb = string.Empty;
string CommandTail = string.Empty;

// Parse the lines.
ReceiveCommand(line, ref CommandVerb, ref CommandTail);
// This function just splits the first part and second part of the command.

if (LogVerbose)
{
writetolog(line);
}
//Console.Error.WriteLine("Read line {0}", line);
switch (CommandVerb)
{
case "HELO":
//case "EHLO":
if (CurrentCommand == string.Empty)
{
writetolog(line);
CurrentCommand = "HELO";
writer.WriteLine("250 OK " + CommandTail);
HeloOK = true;
}
break;

case "EHLO":
writetolog(line);
writer.WriteLine("502"); // Report error, sender
should revert to HELO
break;

case "RSET":
writetolog(line);
writer.WriteLine("250 OK");
break;

case "NOOP":
writetolog(line);
writer.WriteLine("250 OK");
break;

case "MAIL":
if (HeloOK)
{
if (CommandTail.Length <= 100)
{
writetolog(line);
CurrentCommand = "MAIL";
writer.WriteLine("250 " + CommandTail +
" Sender OK");
}
else
{
writer.WriteLine("503 Email address too
long");
}
}
else
{
writer.WriteLine("503 Send HELO first");
}
break;

case "RCPT":
if (HeloOK)
{
if (CurrentCommand == "MAIL")
{
if (CommandTail.Length <= 2000)
{
writetolog(line);
CurrentCommand = "RCPT";
writer.WriteLine("250 " +
CommandTail);
}
}
else
{
writer.WriteLine("503 Send MAIL FROM:
first");
}

}
else
{
writer.WriteLine("503 Send HELO first");
}
break;

case "DATA":

if (CurrentCommand == "RCPT")
{
CurrentCommand = "DATA";

writetolog("DATA");

writer.WriteLine("354 End data with
<CR><LF>.<CR><LF>");

StringBuilder data = new StringBuilder();
String subject = "";
line = reader.ReadLine();
if (line != null && line != ".")
{
const string SUBJECT = "Subject: ";
if (line.StartsWith(SUBJECT))
subject =
line.Substring(SUBJECT.Length);
else data.AppendLine(line);
for (line = reader.ReadLine();
line != null && line != ".";
line = reader.ReadLine())
{
data.AppendLine(line);
}
}
String message = data.ToString();

//_EmailContent = message;
MailContent = message;

if (LogVerbose)
{
writetolog(MailContent);
}

Thread t = new Thread(delegate() {
HandleMessage(MailContent); });
t.Start();

Console.Error.WriteLine("Received ­ email
with subject: {0} and message: {1}",
subject, message);
writer.WriteLine("250 OK");
//client.Close();
//return;
}
else
{
writer.WriteLine("503 Send RCPT TO: first");
}

break;

case "QUIT":
writetolog(line);
CurrentCommand = string.Empty;
writer.WriteLine("221 Goodbye");
HeloOK = false;
client.Close();
return; // MailContent;
default:
writer.WriteLine("500");
break;
}
}
}
}

********************************************************************************************************

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[Ben Voigt [C++ MVP]]

Don't use Thread.Abort. Ever. We already told you that.

Get rid of all your Thread.Sleep calls as well. They are a horribly broken
way to implement cross-thread synchronization.

If you need a watchdog, have another service periodically connect to the
SMTP port and check for the banner. If it doesn't get it, kill the entire
process and restart it. For this periodic connection test, it is ok to use
Thread.Sleep, but better to use a Timer so that you can also process service
stop requests.

Hi all,

I still got problems with it...

I have sorted out the watchdog process by having a parent thread. May not
be the most elegant. I then experienced 100% CPU so have put in
thread.sleep to sort that.

However, when I test locally, it all appears to work, though when I put it
live and sendmail is sending to it, it receives first time and if the next
job comes within my timeout period of 3 minutes, it hangs.

Here is my source... Any help at all would be appreciated. This is in a
windows service.

I am getting myself into a really big hole with this and the client is (to
put it politely) a little upset. :-(



********************************************************************************************************
//windows service


private TcpListener tcpListener;
private Thread listenThread;

protected override void OnStart(string[] args)
{
int PortNo =
Convert.ToInt32(ConfigurationManager.AppSettings["SMTPPort"]);

this.tcpListener = new TcpListener(IPAddress.Any, PortNo);
this.listenThread = new Thread(new
ThreadStart(ListenForClients));
this.listenThread.Start();
}

private void ListenForClients()
{
this.tcpListener.Start();

while (true)
{
//blocks until a client has connected to the server
TcpClient client = this.tcpListener.AcceptTcpClient();

//create a thread to handle communication
//with connected client
Thread clientThread = new Thread(new
ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);

Thread.Sleep(250);

}
}


private void HandleClientComm(object tclient)
{
while (true)
{
//blocks until a client has connected to the server
TcpClient client = (TcpClient)tclient;
//create a thread to handle communication
//with connected client
Thread clientThread = new Thread(new
ParameterizedThreadStart(HandleClient));
clientThread.Start(client);

Thread.Sleep(250);

// Watchdog.
// If the thread is still going after 3 minutes, abort the
thread.
clientThread.Join(180000);
if (clientThread.IsAlive)
{
clientThread.Abort();
}
}
}

private void HandleClient(object client)
{
try
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();

SMTP.SMTPServer handler = new SMTP.SMTPServer(tcpClient);
handler.FilePath =
ConfigurationManager.AppSettings["TempDir"];
handler.LogToFile = LogToFile;
handler.LogVerbose = LogVerbose;
handler.ExpectedSubject =
ConfigurationManager.AppSettings["ExpectedSubjectStart"];
handler.LimitIPAddress =
ConfigurationManager.AppSettings["AllowedIPAddress"];
handler.PenHandlerURL =
ConfigurationManager.AppSettings["PenHandlerURL"];

handler.Run(client);

Thread.Sleep(250);

tcpClient.Close();
}
catch
{
}
}


********************************************************************************************************
SMTP class This is a seperate file.

TcpClient client;
NetworkStream stream;
System.IO.StreamReader reader;
System.IO.StreamWriter writer;

static Encoding s_enc = Encoding.ASCII;

private string _CurrentCommand = string.Empty;
private string CurrentCommand
{
set { _CurrentCommand = value; }
get { return _CurrentCommand; }
}

private string _EmailContent = string.Empty;
public string EmailContent
{
get { return _EmailContent; }
}

private string _ExpectedSubject = string.Empty;
public string ExpectedSubject
{
get { return _ExpectedSubject; }
set { _ExpectedSubject = value; }
}

private string _PenHandlerURL = string.Empty;
public string PenHandlerURL
{
get { return _PenHandlerURL; }
set { _PenHandlerURL = value; }
}

private string _LimitIPAddress = string.Empty;
public string LimitIPAddress
{
set { _LimitIPAddress = value; }
}

private bool CloseConnection = false;

private bool HeloOK = false;

public SMTPServer(TcpClient client)
{
this.client = client;
stream = client.GetStream();
reader = new System.IO.StreamReader(stream);
writer = new System.IO.StreamWriter(stream);
writer.NewLine = "\r\n";
writer.AutoFlush = true;

}

private bool _LogToFile = false;
public bool LogToFile
{
get { return _LogToFile; }
set { _LogToFile = value; }
}

private bool _LogVerbose = false;
public bool LogVerbose
{
get { return _LogVerbose; }
set { _LogVerbose = value; }
}


public void Run(object obj)
{
IPEndPoint ipend = (IPEndPoint)client.Client.RemoteEndPoint;

if (_LimitIPAddress != string.Empty)
{
if (ipend.Address.ToString() != _LimitIPAddress)
{
writetolog("Attempted connection from : " +
ipend.Address.ToString() + " The connection will be closed.");
client.Close();
return;
}
}
else
{

writer.WriteLine("220 SMTP server");

string MailContent = string.Empty;

for (string line = reader.ReadLine(); line != null; line =
reader.ReadLine())
{
string CommandVerb = string.Empty;
string CommandTail = string.Empty;

// Parse the lines.
ReceiveCommand(line, ref CommandVerb, ref CommandTail);
// This function just splits the first part and second part of the
command.

if (LogVerbose)
{
writetolog(line);
}
//Console.Error.WriteLine("Read line {0}", line);
switch (CommandVerb)
{
case "HELO":
//case "EHLO":
if (CurrentCommand == string.Empty)
{
writetolog(line);
CurrentCommand = "HELO";
writer.WriteLine("250 OK " + CommandTail);
HeloOK = true;
}
break;

case "EHLO":
writetolog(line);
writer.WriteLine("502"); // Report error,
sender should revert to HELO
break;

case "RSET":
writetolog(line);
writer.WriteLine("250 OK");
break;

case "NOOP":
writetolog(line);
writer.WriteLine("250 OK");
break;

case "MAIL":
if (HeloOK)
{
if (CommandTail.Length <= 100)
{
writetolog(line);
CurrentCommand = "MAIL";
writer.WriteLine("250 " + CommandTail +
" Sender OK");
}
else
{
writer.WriteLine("503 Email address too
long");
}
}
else
{
writer.WriteLine("503 Send HELO first");
}
break;

case "RCPT":
if (HeloOK)
{
if (CurrentCommand == "MAIL")
{
if (CommandTail.Length <= 2000)
{
writetolog(line);
CurrentCommand = "RCPT";
writer.WriteLine("250 " +
CommandTail);
}
}
else
{
writer.WriteLine("503 Send MAIL FROM:
first");
}

}
else
{
writer.WriteLine("503 Send HELO first");
}
break;

case "DATA":

if (CurrentCommand == "RCPT")
{
CurrentCommand = "DATA";

writetolog("DATA");

writer.WriteLine("354 End data with
<CR><LF>.<CR><LF>");

StringBuilder data = new StringBuilder();
String subject = "";
line = reader.ReadLine();
if (line != null && line != ".")
{
const string SUBJECT = "Subject: ";
if (line.StartsWith(SUBJECT))
subject =
line.Substring(SUBJECT.Length);
else data.AppendLine(line);
for (line = reader.ReadLine();
line != null && line != ".";
line = reader.ReadLine())
{
data.AppendLine(line);
}
}
String message = data.ToString();

//_EmailContent = message;
MailContent = message;

if (LogVerbose)
{
writetolog(MailContent);
}

Thread t = new Thread(delegate() {
HandleMessage(MailContent); });
t.Start();

Console.Error.WriteLine("Received ­ email
with subject: {0} and message: {1}",
subject, message);
writer.WriteLine("250 OK");
//client.Close();
//return;
}
else
{
writer.WriteLine("503 Send RCPT TO:
first");
}

break;

case "QUIT":
writetolog(line);
CurrentCommand = string.Empty;
writer.WriteLine("221 Goodbye");
HeloOK = false;
client.Close();
return; // MailContent;
default:
writer.WriteLine("500");
break;
}
}
}
}

********************************************************************************************************

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[dominicpouzin]

You are going through the right steps. Going through these
difficulties, and incremental solutions, is the right way to learn how
to do it.

A few comments:
1/ Generally speaking, you may want to look into using events in place
of loops. For example, look into the AutoResetEvent class. Instead of
wait-spinning (using a regular loop), you could wait on multiple
events, each associated with a spawned thread. This is way more
efficient, because it uses kernel objects. As soon as the event is
signaled, the thread waiting on the event will be immediately
unblocked (consider the CPU cycles you wasted by a wait statement, no
matter how small the way is). However, here it is not too practical if
you have tons of clients connecting , because you may in fact run out
of kernel objects.

2/ So, as an alternative, consider creating a background cleanup
thread, whose job is to clean up nonresponding clients (i.e. cleanup
open streams / TCP clients / spawned threads). It's indeed very easy
for a client to make a TCP server allocate a thread left hanging. For
example, the client could only write partial data to the stream (=
server is now blocked-reading from the stream), or refuse to read the
response (= server is now blocked-writing to the stream), etc.

3/ Yes, a thread abort is usually bad, but here if the thread is
blocked on a call (ex: server is now blocked-writing to the stream),
and the call won't stop thanks to specified timeouts (note: there are
a number of timeout properties on the TcpClient class), then it
becomes quite reasonable to abort the thread. A thread abort is not a
hard kill (internally a thread abort exception is gracefully generated
by .NET framework), and resources should be cleaned up properly. On
the other hand, killng a C++ thread is horrible...

4/ There are async versions of most methods to read / write from / to
streams, etc. Might want to look into those.

5/ You may also want to throttle / limit inbound connections, for
example using a Semaphore object. Spawning too many threads to server
many connecting clients could kill your server.

6/ On Windows, there is a large number of other subtle other issues
which you will only experience under heavy stress. For example, there
are limits to the number of half open connections, to how quickly a
port can be reused after an application stopped using it, etc.

Dominic Pouzin
http://www.data-applied.com

 

[David]

Hi Peter,

Thank you very much for your response and your comments. I do take them on
board. This is getting quite long and not sure if you will read it all, but
I have comments / questions towards the end about some of the technical
issues that you raise. I have hightlighted them starting a line with
*****************

The code I have given is almost complete. All I have missed is some
supporting functions (like parsing the incoming lines from the socket,
logging, and handling the resulting message (in the SMTP handler)) but
everything else is there that is required. There is a POP3 handler inside
the main part of the service, but this code is not referencing that at all.
There may be one or two variables that I have missed out, which mostly get
information from the app.config. The variables that are being sent to
handler are just variables being put into properties. These are for the
ancilliary functions after DATA has been handled (and those functions work
fine, as they are the same as the ones for the POP3 handler and the POP3 is
fine.) LogToFile and LogVerbose are just boolean values coming from
app.config as to wether the app should be logging.

With respect to my clients. I told them from the outset. They know I am a
web dev and have used me for that purpose in the past and like my work. I
have also done simple windows services for them. For this project, it
required a mailserver, purely to receive incoming mail. I had a working POP3
mailserver but they wanted one that was more pro-active, which only left
SMTP. Now, I have written a mailserver in the past in Delphi (in my early
days of programming) and I used TWSocket, which is event driven rather than
thread driven.

However, I could not find anything like TWSocket for C# and the only SMTP
sample I could find was the thread driven one.

I was not aware that thread.abort would cause a problem. I was reading a
book on threads and threading (to try and get my head around threads) and
this watchdog was mentioned. It seemed like a good idea, but only recently,
I realised it might cause other problems (i.e. it may not close the socket
or do any of the cleanup). For this, I searched google for "c# abort a
thread from the inside" but nothing really jumped out at me.

When I ran the server I was getting 100% CPU. I did a search for "c# thread
taking 100% cpu" to see if it might lead me in the right direction.
Obviously not. The one or two messages I found suggested using thread.sleep.
I didn't like that. I felt it was a hack, but as I didn't know any better...

Basically, I did as much as I could to resolve this issue myself and to
learn about threading. It was over 3 weeks ago since I asked the initial
question and I have been working on this for quite a bit of that time,
including long evenings and weekends (at no cost to the client, and the
client is aware of my dedication to this) to make this work. However, the
fact is that in this particular case, there was too much for me to learn and
I was under time pressure. I didn't really want to do the SMTP side, but I
did and tried to use the experience as a learning exercise. (What didn't
help is that a week into this project, my laptop hard disk died and so did
my backup hard disk, both at the same time, so was without my computer for a
week whilst I tried to resolve the issue.)

I have tried many threading examples to make this work. This final one
appeared to work for me, and I understood what I was doing with it more than
the other ones. As it turns out and with your comments to me, I have grossly
misunderstood.

When I tested this version of the code against problems I had in the past
with it, this appeared to work. It worked with my multiple telnet sessions
and it worked when I sent multiple emails from outlook express direct to it.
However, it failed when I put it on the live server.

I do have to learn and obviously I am making mistakes that I don't know how
to get out of. My research into it is leading me nowhere, hence my
questions. I don't ask simple questions, as I feel if it is that simple,
there will already be an answer I have overlooked, and quite often, I am
right. Hence the reason I am here. Threading at the moment, whilst may be
simple for you is being a nightmare for me.


*****************

With respect to your further answers on the topic...

The 100% CPU. I couldn't understand that either, but it happened on both my
machine and the server that I uploaded it to. OK, It might have only been
97 - 98% CPU, but still heavy load. There was no load intially. If I
connected one telnet and closed it, it was fine. If I connected two, then
closed one, it would 100%. To me, this meant that something was looping and
I couldn't figure out what. (I stuck breakpoints in my loops but they
weren't hit.)


On my slightly earlier version of this code, I only had one thread set up.
This was being blocked by the Thread.Join and was stopping any further
connections. I commented out the clientThread.Join, clientThread.IsAlive and
clientThread.Abort and everything appeared to work. However, I felt that I
needed the watchdog and with a reasonable amount of time (3 minutes). The
only way I could think of to get it to work is to put that thread in another
thread, so that the Thread.Join was not blocking any other threads. Perhaps
this is what was causing my 100% CPU.


The SMTPServer.Run taking an object that is not used. This was a bit of junk
left over from another attempt to run it without blocking (AcceptTcpClient
blocks it). The thread stuff for that was ParameterizedThreadStart, which of
course, passes a value. I couldn't start the thread without it, so put it
in, even though I never used it.


The timer that you mention. Would that be System.Threading.TimerCallback,
and would that be on the outside (i.e. in the main service rather than
inside the SMTP class). When would I initiate the timer? Would it be just
after I start the thread in the while(true) loop inside ListenForClients, or
would it be inside the thread that has just been started? Also, how would I
tell the timercallback which thread (or TcpClient) I need to finish (or will
it just know?). I am guessing that to deal with the timeout behaviour, the
timer has to somehow call into the thread to stop things and run a cleanup.
From what I see of threads, when I start one, as I am not naming a thread
(that I can see, as they all seem to have the same name) I don't know how I
can connect to that particular thread.


Mostly when I use exception catching, I will log it to a file and get the
Message and Stack Trace (and sometimes the values in certain variables). In
general, I don't use exceptions unless I specifically need them. Now, in
this case, I was getting an exception and due to my lack of understanding of
threading, I put it in to attempt to work out what was happening whilst
still keeping the rest of the process going.


With respect to the SMTPServer class, my thoughts exactly, but I posted it
to give as much information as possible (though you still say I am short of
code) to help me to resolve the issue.


The next message you posted commented on the loop. Putting that loop in I
guess was a mistake (and could be the cause of most of the problems). I only
realise that the loop could be an issue as you have just pointed it out. As
I am now using other threads to maintain an unblocked system, I guess this
loop is no longer required (as I have the loop in ListenForClients), as all
it will do is to start the child thread, wait for the timeout of
thread.join, timeout the connection then restart it again (which would
explain why after timeout, I get the SMTP banner message again in my telnet
window). Could it be this that is causing sendmail (from linux) to be having
the problem of sending the next message?

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[David]

Hi Dominic,

Thanks for the response. Some of these are still over my head, though some
of them I do understand but will take me a fair amount of time to write.

What I would like to do some time in the future is to probably write an
event driven socket system. In my research, I found very little to help with
writing an SMTP server like this (there are loads of examples of sending
SMTP with C#, practically nothing in receiving SMTP with C#).

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[David]

Thank you Pete for your response and staying with me on this. It is
appreciated. I do know how frustrating it can be to tell someone but because
of their limited understanding of a subject, the message is not immediately
getting through. One day, it will just click and I will wonder what all the
fuss was about.

Some of the info you have posted is still above me, but I will work through
what you have said and research further into the bits I don't understand
yet.

With your obvious limited overview of what I am trying to achieve, have I
gone about it in almost the right way? (What I mean by that is (with my
limited knowledge of threads thus far) does my code stand up to a reasonable
way of handling the situation I am trying to achieve, even though it is not
quite working properly?)

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

[...]
The 100% CPU. I couldn't understand that either, but it happened on both
my
machine and the server that I uploaded it to. OK, It might have only been
97 - 98% CPU, but still heavy load. There was no load intially. If I
connected one telnet and closed it, it was fine. If I connected two, then
closed one, it would 100%. To me, this meant that something was looping
and
I couldn't figure out what. (I stuck breakpoints in my loops but they
weren't hit.)

If you don't hit breakpoints in your loops, then the loops aren't looping
(either because the code never enters the loop, or because the loop is
blocked on some blocking call).

In any case, this is why it's so important you post a concise-but-complete
code example. Something someone can compile without any
additions/modifications, and immediately see whatever behavior it is
you're asking about. If you aren't able to debug it when you have the
complete system in hand, you can imagine how difficult it would be for
someone else to debug it when you've left parts out.

[...]
The SMTPServer.Run taking an object that is not used. This was a bit of
junk
left over from another attempt to run it without blocking
(AcceptTcpClient
blocks it). The thread stuff for that was ParameterizedThreadStart,
which of
course, passes a value. I couldn't start the thread without it, so put it
in, even though I never used it.

Not sure what you mean. In this case, you do use the parameter passed to
the thread when you call Start().

Obviously you need to get the TcpClient reference to your thread somehow.
There are lots of options, and you chose one. That part should work fine;
I just don't see why, given that the SMTPServer class already has a
reference to the TcpClient upon construction, you pass that reference
again (especially since I didn't see any code actually use the argument).

The timer that you mention. Would that be System.Threading.TimerCallback,

I would use System.Threading.Timer, yes. But either of the two
non-Forms-based Timers should work for you.

and would that be on the outside (i.e. in the main service rather than
inside the SMTP class). When would I initiate the timer? Would it be just
after I start the thread in the while(true) loop inside
ListenForClients, or
would it be inside the thread that has just been started?

You should initiate the timer at the point that makes the most sense for
you. It really depends on how much of the operation you want to include
in your timeout. You can start timing as soon as you've accepted the
connection, or you can wait until you've started executing the
SMTPServer.Run() method, or somewhere in between.

To some extent, where you put the timer also depends on what layer of your
implementation you want to have control over it. To you, is the timeout
something that is inherent in the SMTPServer class? If so, then that
class should be doing it. Or is it something that the service code is
responsible for? In that case, it would be started earlier, at a higher
level.

Also, how would I
tell the timercallback which thread (or TcpClient) I need to finish (or
will
it just know?).

There are a variety of approaches, but the most straightforward options
IMHO are to use either the "state" argument provided for the TimerCallback
delegate (it's very similar to the mechanism used for
ParameterizedThreadStart), or to use an anonymous method as the callback
delegate, capturing a variable in the anonymous method referencing the
client you need to handle.

I am guessing that to deal with the timeout behaviour, the
timer has to somehow call into the thread to stop things and run a
cleanup.

No...as I wrote before, all it has to do is close the TcpClient. That
will close the Socket associated with the TcpClient, interrupting any i/o
operation that might be in progress.

If you want the connection thread to exit even if it's already completed
the TcpClient operation, but not yet finished whatever other processing it
has to do, you'll have to incorporate other signaling, such as a boolean
flag, and/or closing other i/o objects that might be in use. In that
case, you will probably find it useful to encapsulate all that into a
single "interrupt" method for SMTPServer, that does all the necessary
signaling.

From what I see of threads, when I start one, as I am not naming a thread
(that I can see, as they all seem to have the same name) I don't know
how I
can connect to that particular thread.

You don't really need to interact with the thread itself. It's the
objects that the thread is using that are interesting (TcpClient,
SMTPServer, etc.)

Mostly when I use exception catching, I will log it to a file and get the
Message and Stack Trace (and sometimes the values in certain variables).
In
general, I don't use exceptions unless I specifically need them. Now, in
this case, I was getting an exception and due to my lack of
understanding of
threading, I put it in to attempt to work out what was happening whilst
still keeping the rest of the process going.

But what if the exception that was happening was in fact related to the
problem you're trying to solve? Simply ignoring it prevents you from
understanding the problem.

With respect to the SMTPServer class, my thoughts exactly, but I posted
it
to give as much information as possible (though you still say I am short
of
code) to help me to resolve the issue.

Note that there are parts of SMTPServer that are important for your
question, and other parts that are not. The problem isn't that you didn't
post ALL of it. The problem is that what you did post isn't sufficient
for reproducing the problem. This is separate from the other problem,
which is that what you did post includes code that isn't related to the
problem.

The next message you posted commented on the loop. Putting that loop in I
guess was a mistake (and could be the cause of most of the problems). I
only
realise that the loop could be an issue as you have just pointed it out.
As
I am now using other threads to maintain an unblocked system, I guess
this
loop is no longer required (as I have the loop in ListenForClients), as
all
it will do is to start the child thread, wait for the timeout of
thread.join, timeout the connection then restart it again (which would
explain why after timeout, I get the SMTP banner message again in my
telnet
window). Could it be this that is causing sendmail (from linux) to be
having
the problem of sending the next message?

I don't see how it could be. But without a complete code example, it's
not possible to say for sure.

That said, looking at the loop again, it's _possible_ it's where your CPU
utilization problem is coming from. In particular, with the code written
as it is, no matter what happens in your HandleClient() thread, the
HandleClientComm() thread is going to loop back and create a new thread to
try to handle the client all over again. In the scenario where the i/o
gets stuck (assuming you even have such a scenario), this isn't much of an
issue, because you'll always wait 3 seconds between each iteration of the
loop.

But consider what happens in the non-error case, where the HandleClient()
thread actually completes and closes the socket. Then, you'll still loop
back and create another HandleClient() thread, for a TcpClient that's
already been closed. The first time you try to do something with that
TcpClient, it'll throw an exception, quickly finishing up that thread. At
which point, the Thread.Join() method will return, causing the loop to go
through another iteration.

Basically, the one thread will just keep rapidly starting a new thread
over and over again, without any delays in either that thread or the one
it starts. That could definitely cause a CPU usage problem.

Note that had you actually checked the exception being thrown in
HandleClient(), rather than silently discarding all exceptions, you would
immediately have seen that an exception that's not expected was being
thrown, leading you directly to the problem.

I rest my case.

Pete