Threading and subroutines

[OpticTygre]

I have a class, ProcessFiles, with several subroutines it runs for each type
of file I want to "process."

First, I check directories for files. Then, based on the filenames of those
I find in the directory, I do IP address lookups from a database to see
where these files need to be sent. After I lookup the IP addresses, I
dimension a new class (ProcessThreads), and call ProcessThreads.Process for
each IP I pull out of the database. In psuedo code it looks like:

Private Sub ProcessUsers()

For each filename in FileList
Open database connection, and retrieve a list of IP's

Do while rs.read()
Add IP address to an array
Loop

If IPArray(0) <> "" Then
Dim ProcessThreads as New ProcessThreads
ProcessThreads.Process(FileName, IPArray)
End If

Next

End Sub

In ProcessThreads, it adds an instance of a new class to a threadpool, which
actually does the transfer of files, etc... ProcessThreads also keeps track
of the threads it calls, via some sample code I found elsewhere.

However, I'm curious about the above code. Each time I call ProcessThreads,
will this part of the program stall until that instance of the
ProcessThreads.Process routine finishes, or will this actually do what I
want it to, and call several instances of ProcessThreads at once (or right
in a row, really)?

-Jason

 

[Larry Serflaten]

I have a class, ProcessFiles, with several subroutines it runs for each type
of file I want to "process."

However, I'm curious about the above code. Each time I call ProcessThreads,
will this part of the program stall until that instance of the
ProcessThreads.Process routine finishes, or will this actually do what I
want it to, and call several instances of ProcessThreads at once (or right
in a row, really)?

You really haven't supplied enough information to answer your question.
You didn't post the code for ProcessThreads.Process so it is impossible
to tell what it will do...

If a kid brings home a bag of candy, will the mom find a Milky Way bar?
You really can't know until you dump out the bag, and examine the contents!
<g>

Also be aware that you don't gain CPU cycles (or network capacity) by adding
threads. In fact you take a few cycles away because of the management overhead.
You might want to think about using some sort of Queue such that only a few
threads are needed at a time, and the few background threads work on processing
whatever is in the queue, until it is emptied.


LFS

 

[OpticTygre]

What you stated is almost what I'm doing. The ProcessThreads.Process looks
something like this:

Public Sub Process(ByVal FileList() As String, ByVal IPArray() As String)
Dim n As Integer = 0

blnUploadingFiles = True

For n = 0 To IPArray.Length - 1
Dim Worker As New SFTPUpload
Worker.FileName = FileList
Worker.IPAddress = IPArray(n)
Worker.Password = SFTPParams.Password
Worker.Port = SFTPParams.Port
Worker.ThreadID = _workerID
Worker.Username = SFTPParams.UserName
_workerID += 1

Worker.OnWorkerStart = AddressOf Me.WorkerStart
Worker.OnWorkerEnd = AddressOf Me.WorkerEnd

Worker.Upload()
Next

'Wait until at least one thread has fully started
Do Until ((_completeWorkerList.Count > 0) Or
(_activeWorkerList.Count > 0))
Application.DoEvents()
Loop

'While the threads are working, check to see if someone turned off
the uploads,
'or see if there are new files. If there are new files, exit all
threads, and
'let the main timer tick again to load the new files and start the
threads over.

Do While ((_completeWorkerList.Count > 0) Or
(_activeWorkerList.Count > 0))
If m_ProcessUploads = Definitions.UploadProcessState.Stopped
Then
Call CloseThreads()
blnUploadingFiles = False
Exit Sub
End If

If CheckForNewFiles(FileList) = True Then
'Cancel all threads
Call CloseThreads()

Do Until _activeWorkerList.Count = 0
'Waiting for active threads to finish
Application.DoEvents()
Loop

blnUploadingFiles = False
Exit Sub 'Wait for the next timer tick and start the process
all over
End If
Application.DoEvents()
Loop

'Delete all files that were uploaded previously to the machines
Call DeleteUploadedFiles(FileList)

'Finished uploading files. Allow the timer to call this procedure
again
'if it needs to.
blnUploadingFiles = False

End Sub

Keep in mind, this code is still kindof ugly due to lack of time to clean it
up. There are some other subroutines in the class that handle adding and
removing the worker threads of the SFTPUpload class to the list, checking
for new files, deleting all successfully uploaded files, etc... But
basically, the ProcessThreads class initiates the SFTPUpload class into
different threads for each IP, and waits for them to either finish
successfully or be cancelled by the user. What I'm wondering, is that in
the ProcessFiles class, will it stall if I try and call multiple instances
of this class in a For..Next loop like I was doing? Btw, I'm using a
threadpool in the SFTPUpload class as a queue for my worker objects.

-Jason

 

[Larry Serflaten]

Do While ((_completeWorkerList.Count > 0) Or (_activeWorkerList.Count > 0))

What I'm wondering, is that in
the ProcessFiles class, will it stall if I try and call multiple instances
of this class in a For..Next loop like I was doing? Btw, I'm using a
threadpool in the SFTPUpload class as a queue for my worker objects.

From just a quick scan, it looks to me like the first time you call Process,
your calling code will be blocked until the above loop exits....

LFS