System.IO.FileSystemWatcher is missing alot of files.

G

Guest

I have a Windows Form application that monitors a directory using the
System.IO.FileSystemWatcher.

On the event created I run a sub that sends an xml file over http to a
service.

When I copy 7000 files in the directory to be monitored only 123 are sent to
the service.

What might be going wrong?

More info:

The sending of the XML is more then just the http post. I also have to do
some stuff with xml first before sending it.
 
A

Al Reid

Philip Wagenaar said:
I have a Windows Form application that monitors a directory using the
System.IO.FileSystemWatcher.

On the event created I run a sub that sends an xml file over http to a
service.

When I copy 7000 files in the directory to be monitored only 123 are sent to
the service.

What might be going wrong?

More info:

The sending of the XML is more then just the http post. I also have to do
some stuff with xml first before sending it.

It's possible that you are missing events because it takes longer to process the files than it takes to generate the events and thus
some get lost. I had a similar situation using the FileSystemWatcher and I solved it by placing the event info into a FIFO and had
a separate worker thread unloading the FIFO and processing the events. This completely eliminated the problem. Perhaps this
approach would be appropriate here as well.
 
A

Al Reid

Philip Wagenaar said:
You mean working with threads?

Is it possible to post a sample?

:

Yes, I mean working with threads. If I get a chance, I'll see if I can't skinny down my working code and post the basics.
[/QUOTE]
 
A

Al Reid

Philip Wagenaar said:
You mean working with threads?

Is it possible to post a sample?

I found my original proof of concept code. It was implemented as a Windows Service. Most of the concepts were obtained from Google
searches, etc.

=================================================

Imports System

Imports System.Threading

Module General

Public goFIFO As New Collections.Queue

Public gblnStopThreads As Boolean

Public gstrSourceDir As String

Public gstrDestinationDir As String

Public goLogger As New CLog

Public goEventlog As EventLog

Public gintIdle As Integer

Public gintInterval As Integer

End Module

-----------------------------

Imports System.Threading

Imports System.io

Public Class WorkerThread

Public Shared Sub ProcessFiles()

Dim e As Object

Dim strFilePath As String

Dim strDestDir As String

Do Until gblnStopThreads And (goFIFO.Count = 0)

Try

If goFIFO.Count > 0 Then

e = goFIFO.Dequeue

If TypeOf e Is RenamedEventArgs Then

Dim strOldName As String = CType(e, RenamedEventArgs).OldFullPath.Replace(gstrSourceDir, gstrDestinationDir)

strDestDir = CType(e, RenamedEventArgs).FullPath.Replace(gstrSourceDir, gstrDestinationDir)

Rename(strOldName, strDestDir)

ElseIf TypeOf e Is FileSystemEventArgs Then

strFilePath = CType(e, FileSystemEventArgs).FullPath

strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir)

If (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then

If Not Directory.Exists(strDestDir) Then

strFilePath = strFilePath

strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir)

Directory.CreateDirectory(strDestDir)

End If

ElseIf New FileInfo(strDestDir).Directory.Exists = False Then

strFilePath = strFilePath

strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir)

Directory.CreateDirectory(Path.GetDirectoryName(strDestDir))

End If

If Not (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then

FileCopy(strFilePath, strDestDir)

End If

End If

Thread.Sleep(gintInterval)

Else

Thread.Sleep(gintIdle)

End If

Catch ex As Exception

goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error)

End Try

Loop

End Sub

End Class



-----------------------------



Imports System.ServiceProcess

Imports System

Imports System.Threading

Imports System.io

Imports Microsoft.Win32

Public Class MirrorService

Inherits System.ServiceProcess.ServiceBase

Private WithEvents moWatcher As FileSystemWatcher

Protected Overrides Sub OnStart(ByVal args() As String)

Me.AutoLog = True

goEventlog = Me.EventLog()

Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High

gstrSourceDir = "C:\WatcherTest\Source"

gstrDestinationDir = "F:\WatcherTest\Destination"

gintIdle = 5000

gintInterval = 100

Dim oWorker As New WorkerThread

Dim oWorkerThread As New Thread(New ThreadStart(AddressOf oWorker.ProcessFiles))

' Start the thread.

oWorkerThread.Start()

moWatcher = New FileSystemWatcher

moWatcher.InternalBufferSize = 16384

moWatcher.Path = gstrSourceDir

moWatcher.IncludeSubdirectories = True

moWatcher.EnableRaisingEvents = True

End Sub

Protected Overrides Sub OnStop()

gblnStopThreads = True

End Sub

Private Sub moWatcher_CreatedOrChanged(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles moWatcher.Changed,
moWatcher.Created

Try

If e.ChangeType <> WatcherChangeTypes.Deleted Then

goFIFO.Enqueue(e)

End If

Catch ex As Exception

goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error)

End Try

End Sub

Private Sub moWatcher_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles moWatcher.Renamed

goFIFO.Enqueue(CType(e, RenamedEventArgs))

End Sub

End Class

===================================================



Sorry that the cut and paste lost the formatting. I hope this helps.

==

Al Reid
 
G

Guest

Thanks! This really helped me alot!

Al Reid said:
I found my original proof of concept code. It was implemented as a Windows Service. Most of the concepts were obtained from Google
searches, etc.

=================================================

Imports System

Imports System.Threading

Module General

Public goFIFO As New Collections.Queue

Public gblnStopThreads As Boolean

Public gstrSourceDir As String

Public gstrDestinationDir As String

Public goLogger As New CLog

Public goEventlog As EventLog

Public gintIdle As Integer

Public gintInterval As Integer

End Module

-----------------------------

Imports System.Threading

Imports System.io

Public Class WorkerThread

Public Shared Sub ProcessFiles()

Dim e As Object

Dim strFilePath As String

Dim strDestDir As String

Do Until gblnStopThreads And (goFIFO.Count = 0)

Try

If goFIFO.Count > 0 Then

e = goFIFO.Dequeue

If TypeOf e Is RenamedEventArgs Then

Dim strOldName As String = CType(e, RenamedEventArgs).OldFullPath.Replace(gstrSourceDir, gstrDestinationDir)

strDestDir = CType(e, RenamedEventArgs).FullPath.Replace(gstrSourceDir, gstrDestinationDir)

Rename(strOldName, strDestDir)

ElseIf TypeOf e Is FileSystemEventArgs Then

strFilePath = CType(e, FileSystemEventArgs).FullPath

strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir)

If (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then

If Not Directory.Exists(strDestDir) Then

strFilePath = strFilePath

strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir)

Directory.CreateDirectory(strDestDir)

End If

ElseIf New FileInfo(strDestDir).Directory.Exists = False Then

strFilePath = strFilePath

strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir)

Directory.CreateDirectory(Path.GetDirectoryName(strDestDir))

End If

If Not (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then

FileCopy(strFilePath, strDestDir)

End If

End If

Thread.Sleep(gintInterval)

Else

Thread.Sleep(gintIdle)

End If

Catch ex As Exception

goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error)

End Try

Loop

End Sub

End Class



-----------------------------



Imports System.ServiceProcess

Imports System

Imports System.Threading

Imports System.io

Imports Microsoft.Win32

Public Class MirrorService

Inherits System.ServiceProcess.ServiceBase

Private WithEvents moWatcher As FileSystemWatcher

Protected Overrides Sub OnStart(ByVal args() As String)

Me.AutoLog = True

goEventlog = Me.EventLog()

Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High

gstrSourceDir = "C:\WatcherTest\Source"

gstrDestinationDir = "F:\WatcherTest\Destination"

gintIdle = 5000

gintInterval = 100

Dim oWorker As New WorkerThread

Dim oWorkerThread As New Thread(New ThreadStart(AddressOf oWorker.ProcessFiles))

' Start the thread.

oWorkerThread.Start()

moWatcher = New FileSystemWatcher

moWatcher.InternalBufferSize = 16384

moWatcher.Path = gstrSourceDir

moWatcher.IncludeSubdirectories = True

moWatcher.EnableRaisingEvents = True

End Sub

Protected Overrides Sub OnStop()

gblnStopThreads = True

End Sub

Private Sub moWatcher_CreatedOrChanged(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles moWatcher.Changed,
moWatcher.Created

Try

If e.ChangeType <> WatcherChangeTypes.Deleted Then

goFIFO.Enqueue(e)

End If

Catch ex As Exception

goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error)

End Try

End Sub

Private Sub moWatcher_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles moWatcher.Renamed

goFIFO.Enqueue(CType(e, RenamedEventArgs))

End Sub

End Class

===================================================



Sorry that the cut and paste lost the formatting. I hope this helps.

==

Al Reid
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top