At face value there does not appear to be anything that is an obvious
bottleneck, however you do call a number of methods that you have not
described, (OpenOutputFiles, InitializeInputVariables, Validate,
BuildOutputRecord, WriteOutputRecord, WriteOutputErrorRecord,
WriteOutputErrorRecord), and it is possible that there is bottleneck in any
of those. In addition you raise event ProcessStatus regularly and it would
be prudent to ensure that that whatever is handling that event is not
blocking the process for an inordinate length of time.
From my point of view the number of calls to Trim could be a factor and
perhaps some of of them are redundant. For example, take the line:
_sInActionCode = Trim(Mid(sPreferredInputFile, 1, 1))
If the first character of a 'record' always contains a non-space character
then Trim is redundant. In this case there are 3 new strings being created,
(remember that strings are immutable), and there is an overhead, abeit
small, involved in the creation of each string. Removing the Trim from this
line would mean that there are only 2 new strings being created thus
reducing the overhead accordingly. With the number of string operations in
your PreferredInputProcessing method this could be significant.
You might also try modifying the string parsing to the '.NET way'., for
example:
_sInActionCode = sPreferredInputFile.SubString(0, 1).Trim
or
_sInActionCode = sPreferredInputFile.SubString(0, 1)
I do not have any benchmarking data but it is possible that you might find a
performance increase.
Another place where, in my view there extraneous overhead is:
If sPreferredInputFile.Trim <> "" AndAlso sPreferredInputFile.Trim(""c)
<> "" AndAlso
sPreferredInputFile.Trim.Length >= 439 Then
Note here that you are using the System.String.Trim method rather than the
Microsoft.VisualBasic.Trim function. The Microsoft.VisualBasic.Trim function
returns the source string with leading and trailing space (&H20) characters
removed while the System.String.Trim method returns the source string after
white space characters are removed from the beginning and end. Note that
there is a difference between 'space' characters and 'white space'
characters. It is unclear what actual character is being specified in the
sPreferredInputFile.Trim(""c) clause but it is highly likely that it
qualifies as 'white space' and is therfore being removed by the first
clause. I would be inclined to code the test this:
sPreferredInputFile = sPreferredInputFile.Trim()
If sPreferredInputFile.Length >= 439 Then
The 3 string operations are now reduced to 1 and the number of comparison
operations is also reduced from three to one. Given the above you might be
able to refine your parsing code and identify further redundancies.
Thanks for your response
Here is the code:
Public Overrides Sub PreferredInputProcessing()
Dim InputFileReader As StreamReader
Dim UndefinedBenefitsFileWriter As StreamWriter
Dim DataBlock As MHNet.ApplicationBlocks.Data.SqlHelper
Dim ds As New DataSet()
Dim dr As SqlClient.SqlDataReader
Dim sSql As New StringBuilder()
Dim iNewOptionCount As Integer
Dim sPreferredInputFile As System.String
Dim iRecordsProcessed As System.Int32
'Open input file
InputFileReader = New StreamReader(_sInputFileLocation)
'Open output file
OpenOutputFiles()
'Create UndefinedBenefits file
UndefinedBenefitsFileWriter = New StreamWriter("C:\EDIFiles\" &
_sRateCode & "UndefinedBenefits.csv")
'Zero out the number of records processed.
ProcessReport.RecordsProcessed = 0
'Set validation properties
_bValidateEnrollType = True
_bValidateHeadofHouse = True
'The Celanese benefits don't have a value for PrimaryStatus in
the elig file,
'but are actually = Primary. So, clsMain defaults it to primary
status.
Select Case _sRateCode
Case "CELANESEMBH", "MMSI", "GLOBALHEALTH"
_bValidatePrimaryStatus = False
Case Else
_bValidatePrimaryStatus = True
End Select
_bValidateMaritalStatus = False
Do While InputFileReader.Peek > -1
InitializeInputVariables()
sPreferredInputFile = Nothing
sPreferredInputFile = InputFileReader.ReadLine
'/ skip blank lines
If sPreferredInputFile.Trim <> "" AndAlso
sPreferredInputFile.Trim(""c) <> "" AndAlso
sPreferredInputFile.Trim.Length >= 439 Then
iRecordsProcessed = iRecordsProcessed + 1
'/ update display every 100 records
If iRecordsProcessed Mod 100 = 0 Then
Status = "Records processed: " & iRecordsProcessed
RaiseEvent ProcessStatus(Me, New
System.EventArgs())
End If
'Set the input properties by extracting specific
'values from the input record.
_sInActionCode = Trim(Mid(sPreferredInputFile, 1, 1))
_sInCarrierMemId = Trim(Mid(sPreferredInputFile, 2,
25))
_sInLastName = Trim(Mid(sPreferredInputFile, 27, 60))
_sInFirstName = Trim(Mid(sPreferredInputFile, 87, 30))
_sInMiddleName = Trim(Mid(sPreferredInputFile, 117,
15))
_sInAddr1 = Trim(Mid(sPreferredInputFile, 132, 60))
_sInAddr2 = Trim(Mid(sPreferredInputFile, 192, 60))
_sInCity = Trim(Mid(sPreferredInputFile, 252, 30))
_sInState = Trim(Mid(sPreferredInputFile, 282, 2))
_sInZip = Trim(Mid(sPreferredInputFile, 284, 10))
_sInBenefitOption = Trim(Mid(sPreferredInputFile, 294,
60))
_sInEmployerGroup = Trim(Mid(sPreferredInputFile, 354,
15))
_sInOptionEffDate = Trim(Mid(sPreferredInputFile, 369,
8))
_sInHPEffDate = Trim(Mid(sPreferredInputFile, 377, 8))
_sInTermDate = Trim(Mid(sPreferredInputFile, 385, 8))
If _sInTermDate = "" Or Not
IsDateValid(AddDateDashes(_sInTermDate)) Then
_sInTermDate = _sMagicTermDate
End If
'TERMING PLANS to set date to manual date
Select Case _sRateCode
Case "MMSI"
If _sInTermDate > "20041231" Then
_sInTermDate = "20041231"
End If
End Select
_sInSex = sPreferredInputFile.Substring(392, 1).Trim
Dim sTmp As System.String
sTmp = Trim(Mid(sPreferredInputFile, 394, 8))
If sTmp <> "" Then
_sInDOB = Trim(Mid(sTmp, 1, 4)) & "-" &
Trim(Mid(sTmp, 5, 2)) & _
"-" & Trim(Mid(sTmp, 7, 2))
End If
_sInSSN = Trim(Mid(sPreferredInputFile, 402, 9))
_sInPhone = Trim(Mid(sPreferredInputFile, 411, 12))
If _sInPhone.Length = 12 Then
_sInPhone = Trim(Mid(_sInPhone, 1, 3)) &
Trim(Mid(_sInPhone, 5, 3)) & Trim(Mid(_sInPhone, 9, 4))
End If
sTmp = sPreferredInputFile.Substring(422, 8).Trim
If sTmp <> "" Then
_sInEmployerGroupAnivDate = Trim(Mid(sTmp, 1, 4)) &
_
"-" & Trim(Mid(sTmp, 5,
2)) & _
"-" & Trim(Mid(sTmp, 7,
2))
End If
_sInHeadOfHouse = Trim(Mid(sPreferredInputFile, 431,
9))
If _sInHeadOfHouse = "" Then
_sInHeadOfHouse = Trim(Mid(_sInCarrierMemId, 2, 9))
End If
_sInPrimaryStatus = Trim(Mid(sPreferredInputFile, 440,
1))
_sInEnrollType = Trim(Mid(sPreferredInputFile, 441, 1))
Try
_sInMaritalStatus = Trim(Mid(sPreferredInputFile,
442, 1))
Catch ex As System.ArgumentOutOfRangeException
If ex.Message.IndexOf("Index and length must refer
to a location within the string") > 0 Then _sInMaritalStatus = ""
End Try
'Validate the incoming record.
Validate()
f _bValidated Then
BuildOutputRecord()
WriteOutputRecord()
ProcessReport.TotalSuccessfulRecords =
ProcessReport.TotalRecordsProcessed - ProcessReport.TotalErrorRecords
Else
WriteOutputErrorRecord()
ProcessReport.TotalErrorRecords =
ProcessReport.TotalErrorRecords + 1
End If
End If 'skip blank lines
Loop
This is the main parsing routine.
Thanks for your help.