[Net 2.0] question about background worker and thread synchronization

[Steve B.]

Hi,

I'm building an application that performs some long operations against
files.
In order to view the progress, I'm using two background workers.

The first backgroundworker is listing recursively all files in a folder and
each files are queued into a Queue<FileInfo> object.
I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
because lisiting all files is quite long and I want to start my process as
soon as on file is found.

The second backgroundworker will execute the process agains each found
files.

I'm wondering how can I properly code my application in order to run the
second background worker only when files are filled. Note that I must run
the background worker from the window thread because the progressedchanged
must change controls properties.

Since the two background worker are started simultaneously, I need a way to
block the second BGWorker until the first file has been found. I using a
while clause but I'm not sure this is a good practice regarding the
performances.

I'm using by now this code snippet :

public partial class Form1 : Form
{
/// <summary>
/// Initializes a new instance of the <see cref="T:Form1"/> class.
/// </summary>
public Form1()
{
InitializeComponent();
m_filesToAnalyse = new Queue<FileInfo>();
m_filesToCompare = new Queue<FileScanResult>();
}
private Queue<FileInfo> m_filesToAnalyse;
private bool m_remainingFiles;
private void button1_Click(object sender, EventArgs e)
{
m_remainingFiles = true;
folderBrowsing.RunWorkerAsync(@"I:\Formations");
filesAnalysing.RunWorkerAsync();
}
private void AnalyseFolder(DirectoryInfo dir) // Used for recursive browsing
{
//Thread.Sleep(50);
folderBrowsing.ReportProgress(0, dir);
Debug.WriteLine("* " + dir.FullName);
foreach (FileInfo file in dir.GetFiles())
{
m_filesToAnalyse.Enqueue(file);
}
foreach (DirectoryInfo subDir in dir.GetDirectories())
{
AnalyseFolder(subDir);
}
}
#region folderBrowsing
private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
{
string folder = e.Argument.ToString();
DirectoryInfo dir = new DirectoryInfo(folder);
AnalyseFolder(dir);
}
private void folderBrowsing_ProgressChanged(object sender,
ProgressChangedEventArgs e)
{
listBox1.Items.Insert(0, string.Format(
"Folder : {0}",
((DirectoryInfo)e.UserState).FullName
));
}
private void folderBrowsing_RunWorkerCompleted(object sender,
RunWorkerCompletedEventArgs e)
{
m_remainingFiles = false;
listBox1.Items.Insert(0, "parcours fichiers fini");
}
#endregion
#region filesAnalysing
private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
{
lock (m_filesToAnalyse)
{
while (m_remainingFiles || m_filesToAnalyse.Count > 0)
{
if(m_filesToAnalyse.Count > 0)
{
FileInfo fi = m_filesToAnalyse.Dequeue();
filesAnalysing.ReportProgress(0, fi);
Debug.WriteLine("- " + fi.FullName);
/*
* Perform here the custom process
*/

);
}
}
}
}
private void filesAnalysing_ProgressChanged(object sender,
ProgressChangedEventArgs e)
{
listBox1.Items.Insert(0, string.Format(
"Fichier : {0}",
((FileInfo)e.UserState).FullName
));
}
private void filesAnalysing_RunWorkerCompleted(object sender,
RunWorkerCompletedEventArgs e)
{
listBox1.Items.Insert(0, "analyse fichiers fini");
}
#endregion
}

Am I on the right way ?

Thanks in advance,
Steve

 

[Dmytro Lapshyn [MVP]]

Hi Steve,

1) Cross-thread synchronization is usually done by using so-called
synchronization ojects. Consider AutoResetEvent or ManualResetEvent. Using
while() loops will make the waiting thread consume CPU cycles which is
obviously bad.

2) You don't need to run a worker on a UI thread in order to update the UI.
Consider Control.Invoke to marshal UI update requests to the main UI thread.

 

[Steve B.]

1. What is the difference between the two objects ?

2. I use the background worker in order to avoid invoking on controls. I
found the code is easier to read with progressed changed event that obscur
Control.Invoke, that's the goal of the background worker...

Steve

 

[Michael D. Ober]

Instead of using a straight queue, create a new class that inherits from the
queue class. Then override the add method to start your background method
when a file is added to the queue. You'll need to make the underlying queue
thread safe by using synclock ... end synclock statements (simplest and
sufficient) or reader/writer locks. If you only want a single thread doing
the actual file processing, you'll also need to include a thread object in
the derived class to track the lifetime of your worker thread.

Here's a sample shell to get you started.

imports system.io
imports system.threading

Public Class FileProcessor
inherits system.collections.queue

private th as new thread(addressof FileProcessor)
private so as new syncronizationcontext

private Shadows sub Add(byval File as string) ' I'm using Shadows
to block all queue.add methods
synclock so
mybase.add file
end synclock

if th.isAlive then exit sub
th.start
end sub

private sub FileProcessor()
do while not queue.isempty
synclock so
dim File as string = queue.dequeue()
end synclock

' Process your file
loop
end sub

end Class

Mike Ober

 

[Steve B.]

Your solution looks nice. I'm thinknig this could allow me to run multiples
instances of this class simultaneously (to enable object pooling why not).
However, if I separate this process into a clean class w/o any UI, how can I
"reconnect" to the thread that host controls life time ?
I won't be able to call control.invoke unless I pass the reference of the
control... but this would mean that the class can only be used in windows
app...

Steve

 

[Michael D. Ober]

I actually use it in a console application. Any thread can write to the
console safely. If you need to access the worker thread, make the thread
available via the class interface. I would abstract the thread operations
so that the class has the control over how the thread operates.

Mike Ober.

 

[Brian Gideon]

Steve,

It sounds like a blocking queue would be a good fit for the problem. A
blocking queue is similar to a normal queue except that the Dequeue
method is designed to block when the queue is empty. So what you'd do
is have your file searcher thread enqueue files to the queue and the
processing thread will dequeue them when they arrive. Once that thread
finishes processing the file it will call the Dequeue method again.
It's a shame this type of queue isn't included in the framework since
it is so useful. You'd have code own, but be careful. It's
surprisingly difficult to implement correctly.

Brian