regex - better way?

  • Thread starter Thread starter rjb
  • Start date Start date
R

rjb

Hi!

Could somebody have a look and help me to optimize the code below.
It may look like very bad way of coding, but this stuff is very, very new
for me.

I've included just few lines.

Regex regxUserName = new Regex(@"(?<=User-Name = )\""([^\""]+)\""",
RegexOptions.None);
Regex regxSessionId = new Regex(@"(?<=Acct-Multi-Session-Id
= )\""([^\""]+)\""", RegexOptions.None);
Regex regxInputGigawords = new Regex(@"(?<=Acct-Input-Gigawords = )\w*",
RegexOptions.None);
..
..
..

Match mt = regxUserName.Match(sb.ToString());
strUserName = mt.Groups[1].ToString();
Match mt2 = regxSessionId.Match(sb.ToString());
strSessionId = mt2.Groups[1].ToString();
Match mt3 = regxInputGigawords.Match(sb.ToString());
strInputGigawords = mt3.Groups[0].ToString();
..
..
..

I'm using this to extract data from the following file.

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0


thank you
rjb
 
rjb said:
Hi!

Could somebody have a look and help me to optimize the code below.
It may look like very bad way of coding, but this stuff is very, very new
for me.

Are you having any performance issues with this? If you aren't then the
easiest and most maintainable solution is fine(regex is easier to grasp,
once you know regex, than string manipulations and *way* easier to maintain
than a generated parser).
 
I would recommend not using regular expression,
but rather load all keys and values from the file into a hashtable
and work with that. It is a very smooth way. This is a code sample
how you would do it:
<code>
// Load the file (filename).
StreamReader sr = new StreamReader(filename);
Hashtable infoTable = new Hashtable();
string [] kvPair = null;
string line = null;
while (null != (line = sr.ReadLine()))
{
kvPair = line.Split('=');
infoTable.Add(kvPair [0].Trim(), kvPair [1].Trim());
}
sr.Close();
// Print all keys and values.
IDictionaryEnumerator de = infoTable.GetEnumerator();
while (de.MoveNext())
{
Console.WriteLine("{0} = {1}", de.Key, de.Value);
}
// Print the user name.
Console.WriteLine("The user is {0}.", infoTable
["User-Name"].ToString());

</code>

You might want to check that kvPair really has two elements, before putting
it into the table, and also handle eventual exceptions thrown when you try
to open the file.
 
Thank you for your response.

Daniel - I don't have any issue with performance. I just thought that this
looks "bad".
My experience with regular expresion = couple of hours. Before then I didn't
know such
a thing exists :) I'm not a programmer...

Dennis - thank you for your code. I'm very keen on learning new techniques.

To give you the whole picture. My file looks like:

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

Mon Sep 27 22:17:35 2004
Acct-Status-Type = Interim-Update
User-Name = "00079326AAC8"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147817"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323454
Acct-Session-Time = 900
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 130612
Acct-Output-Octets = 2058421
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

Mon Sep 27 22:32:34 2004
Acct-Status-Type = Interim-Update
User-Name = "00079330410A"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147813"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096324353
Acct-Session-Time = 19429
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 4137490
Acct-Output-Octets = 11070040
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

.....and so on. A lot of groups.

Basically what I want is to have the output below from each group:

0007933B22B9 27 Sep 2004 147738 0 0 17970689 8331353
00079326AAC8 27 Sep 2004 147817 0 0 130612 2058421
00079330410A 27 Sep 2004 147813 0 0 4137490 11070040
etc...

All this will go to a database.

Thank you for your time.
rjb


Dennis Myrén said:
I would recommend not using regular expression,
but rather load all keys and values from the file into a hashtable
and work with that. It is a very smooth way. This is a code sample
how you would do it:
<code>
// Load the file (filename).
StreamReader sr = new StreamReader(filename);
Hashtable infoTable = new Hashtable();
string [] kvPair = null;
string line = null;
while (null != (line = sr.ReadLine()))
{
kvPair = line.Split('=');
infoTable.Add(kvPair [0].Trim(), kvPair [1].Trim());
}
sr.Close();
// Print all keys and values.
IDictionaryEnumerator de = infoTable.GetEnumerator();
while (de.MoveNext())
{
Console.WriteLine("{0} = {1}", de.Key, de.Value);
}
// Print the user name.
Console.WriteLine("The user is {0}.", infoTable
["User-Name"].ToString());

</code>

You might want to check that kvPair really has two elements, before putting
it into the table, and also handle eventual exceptions thrown when you try
to open the file.

--
Regards,
Dennis JD Myrén
Oslo Kodebureau
Hi!

Could somebody have a look and help me to optimize the code below.
It may look like very bad way of coding, but this stuff is very, very new
for me.

I've included just few lines.

Regex regxUserName = new Regex(@"(?<=User-Name = )\""([^\""]+)\""",
RegexOptions.None);
Regex regxSessionId = new Regex(@"(?<=Acct-Multi-Session-Id
= )\""([^\""]+)\""", RegexOptions.None);
Regex regxInputGigawords = new Regex(@"(?<=Acct-Input-Gigawords = )\w*",
RegexOptions.None);
.
.
.

Match mt = regxUserName.Match(sb.ToString());
strUserName = mt.Groups[1].ToString();
Match mt2 = regxSessionId.Match(sb.ToString());
strSessionId = mt2.Groups[1].ToString();
Match mt3 = regxInputGigawords.Match(sb.ToString());
strInputGigawords = mt3.Groups[0].ToString();
.
.
.

I'm using this to extract data from the following file.

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0


thank you
rjb
 
Well, then there is some more work.
But it is still very doable using only StreamReader and string.Split.
If you know the file will never be huge, you could just
call ReadToEnd on the StreamReader and perform a split on that string,
splitting on new lines ('\n'), and then work with that array, because it
will be easier when you are not bound to forward-only processing of the
data.

I would suggest you define a class that represent each group to get a little
of structure, like:

public sealed
class Group
{

private Group ( )
{
}

DateTime _timeStamp = null;
Hashtable _table = null;

public DateTime TimeStamp
{
get
{
return _timeStamp;
}
}

public Hashtable DataTable
{
get
{
return _table;
}
}


}


--
Regards,
Dennis JD Myrén
Oslo Kodebureau
rjb said:
Thank you for your response.

Daniel - I don't have any issue with performance. I just thought that
this
looks "bad".
My experience with regular expresion = couple of hours. Before then I
didn't
know such
a thing exists :) I'm not a programmer...

Dennis - thank you for your code. I'm very keen on learning new
techniques.

To give you the whole picture. My file looks like:

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

Mon Sep 27 22:17:35 2004
Acct-Status-Type = Interim-Update
User-Name = "00079326AAC8"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147817"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323454
Acct-Session-Time = 900
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 130612
Acct-Output-Octets = 2058421
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

Mon Sep 27 22:32:34 2004
Acct-Status-Type = Interim-Update
User-Name = "00079330410A"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147813"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096324353
Acct-Session-Time = 19429
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 4137490
Acct-Output-Octets = 11070040
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

....and so on. A lot of groups.

Basically what I want is to have the output below from each group:

0007933B22B9 27 Sep 2004 147738 0 0 17970689 8331353
00079326AAC8 27 Sep 2004 147817 0 0 130612 2058421
00079330410A 27 Sep 2004 147813 0 0 4137490 11070040
etc...

All this will go to a database.

Thank you for your time.
rjb


Dennis Myrén said:
I would recommend not using regular expression,
but rather load all keys and values from the file into a hashtable
and work with that. It is a very smooth way. This is a code sample
how you would do it:
<code>
// Load the file (filename).
StreamReader sr = new StreamReader(filename);
Hashtable infoTable = new Hashtable();
string [] kvPair = null;
string line = null;
while (null != (line = sr.ReadLine()))
{
kvPair = line.Split('=');
infoTable.Add(kvPair [0].Trim(), kvPair [1].Trim());
}
sr.Close();
// Print all keys and values.
IDictionaryEnumerator de = infoTable.GetEnumerator();
while (de.MoveNext())
{
Console.WriteLine("{0} = {1}", de.Key, de.Value);
}
// Print the user name.
Console.WriteLine("The user is {0}.", infoTable
["User-Name"].ToString());

</code>

You might want to check that kvPair really has two elements, before putting
it into the table, and also handle eventual exceptions thrown when you
try
to open the file.

--
Regards,
Dennis JD Myrén
Oslo Kodebureau
Hi!

Could somebody have a look and help me to optimize the code below.
It may look like very bad way of coding, but this stuff is very, very new
for me.

I've included just few lines.

Regex regxUserName = new Regex(@"(?<=User-Name = )\""([^\""]+)\""",
RegexOptions.None);
Regex regxSessionId = new Regex(@"(?<=Acct-Multi-Session-Id
= )\""([^\""]+)\""", RegexOptions.None);
Regex regxInputGigawords = new Regex(@"(?<=Acct-Input-Gigawords
= )\w*",
RegexOptions.None);
.
.
.

Match mt = regxUserName.Match(sb.ToString());
strUserName = mt.Groups[1].ToString();
Match mt2 = regxSessionId.Match(sb.ToString());
strSessionId = mt2.Groups[1].ToString();
Match mt3 = regxInputGigawords.Match(sb.ToString());
strInputGigawords = mt3.Groups[0].ToString();
.
.
.

I'm using this to extract data from the following file.

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0


thank you
rjb
 
Hello RJB,

I agree with Dennis' conclusions. I find simple parsing FAR easier to use,
understand, and debug, than regular expressions.
This is especially true since your data repeats in the data file.

I don't agree with Dennis that you need to read the entire document into
memory, though. I've seen data documents like this, and they can be quite
large. Simply detecting the blank line and the date is sufficient to
seperate groups and do a little processing.

Below, I've taken Dennis' code and added some logic... (warning: uncompiled
code)

// initialize your database object
SqlConnection myConnect = new SqlConnection (Your Connection String);
myConnect.Open();

// Load the file (filename).
StreamReader sr = new StreamReader(filename);
Hashtable infoTable = new Hashtable();
string [] kvPair = null;
string line = null;
while (null != (line = sr.ReadLine()))
{
if (line.Trim.Len = 0) // you've hit a blank line... group ends
{
string Sql_String = string.Format("Insert MyTable (date,
username, inputoctets, outputoctects) values ('{0}', '{1}', '{2}', '{3}')",
infoTable["Date"] , infoTable["User-Name"],
infoTable["Input-Octets"], infoTable["Output-Octets"]);
SqlCommand myCommand = new SqlCommand(Sql_String, myConnect);
myCommand.ExecuteNonQuery();
infoTable.Clear();

}
else
{
kvPair = line.Split('=');
if (kvPair.Length = 1) // this is the date!
{
infoTable.Add("Date",line.Trim());
}
else
{
infoTable.Add(kvPair [0].Trim(), kvPair [1].Trim());
}
}
}
sr.Close();
myConnect.Close();

Assumptions: there's a blank line at the end of the file.
There is no blank line at the beginning of the file.

This code was not compiled... please forgive any syntax errors. I'm typing
from memory.

--- Nick

rjb said:
Thank you for your response.

Daniel - I don't have any issue with performance. I just thought that this
looks "bad".
My experience with regular expresion = couple of hours. Before then I didn't
know such
a thing exists :) I'm not a programmer...

Dennis - thank you for your code. I'm very keen on learning new techniques.

To give you the whole picture. My file looks like:

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

Mon Sep 27 22:17:35 2004
Acct-Status-Type = Interim-Update
User-Name = "00079326AAC8"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147817"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323454
Acct-Session-Time = 900
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 130612
Acct-Output-Octets = 2058421
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

Mon Sep 27 22:32:34 2004
Acct-Status-Type = Interim-Update
User-Name = "00079330410A"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147813"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096324353
Acct-Session-Time = 19429
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 4137490
Acct-Output-Octets = 11070040
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0

....and so on. A lot of groups.

Basically what I want is to have the output below from each group:

0007933B22B9 27 Sep 2004 147738 0 0 17970689 8331353
00079326AAC8 27 Sep 2004 147817 0 0 130612 2058421
00079330410A 27 Sep 2004 147813 0 0 4137490 11070040
etc...

All this will go to a database.

Thank you for your time.
rjb


Dennis Myrén said:
I would recommend not using regular expression,
but rather load all keys and values from the file into a hashtable
and work with that. It is a very smooth way. This is a code sample
how you would do it:
<code>
// Load the file (filename).
StreamReader sr = new StreamReader(filename);
Hashtable infoTable = new Hashtable();
string [] kvPair = null;
string line = null;
while (null != (line = sr.ReadLine()))
{
kvPair = line.Split('=');
infoTable.Add(kvPair [0].Trim(), kvPair [1].Trim());
}
sr.Close();
// Print all keys and values.
IDictionaryEnumerator de = infoTable.GetEnumerator();
while (de.MoveNext())
{
Console.WriteLine("{0} = {1}", de.Key, de.Value);
}
// Print the user name.
Console.WriteLine("The user is {0}.", infoTable
["User-Name"].ToString());

</code>

You might want to check that kvPair really has two elements, before putting
it into the table, and also handle eventual exceptions thrown when you try
to open the file.

--
Regards,
Dennis JD Myrén
Oslo Kodebureau
Hi!

Could somebody have a look and help me to optimize the code below.
It may look like very bad way of coding, but this stuff is very, very new
for me.

I've included just few lines.

Regex regxUserName = new Regex(@"(?<=User-Name = )\""([^\""]+)\""",
RegexOptions.None);
Regex regxSessionId = new Regex(@"(?<=Acct-Multi-Session-Id
= )\""([^\""]+)\""", RegexOptions.None);
Regex regxInputGigawords = new Regex(@"(?<=Acct-Input-Gigawords = )\w*",
RegexOptions.None);
.
.
.

Match mt = regxUserName.Match(sb.ToString());
strUserName = mt.Groups[1].ToString();
Match mt2 = regxSessionId.Match(sb.ToString());
strSessionId = mt2.Groups[1].ToString();
Match mt3 = regxInputGigawords.Match(sb.ToString());
strInputGigawords = mt3.Groups[0].ToString();
.
.
.

I'm using this to extract data from the following file.

Mon Sep 27 22:17:15 2004
Acct-Status-Type = Interim-Update
User-Name = "0007933B22B9"
NAS-IP-Address = 192.168.10.40
Service-Type = DATA
Acct-Multi-Session-Id = "147738"
Acct-Session-Id = "3"
Acct-Delay-Time = 0
Event-Timestamp = 1096323434
Acct-Session-Time = 153766
Acct-Input-Gigawords = 0
Acct-Output-Gigawords = 0
Acct-Input-Octets = 17970689
Acct-Output-Octets = 8331353
Acct-Terminate-Cause = 0
Framed-IP-Address = 0.0.0.0
Acct-Input-Packets = 0
Acct-Output-Packets = 0
NAS-Port-Type = Async
NAS-Port-Id = 0


thank you
rjb
 
Nick Malik said:
Hello RJB,

I agree with Dennis' conclusions. I find simple parsing FAR easier to
use,
understand, and debug, than regular expressions.
This is especially true since your data repeats in the data file.

I don't agree with Dennis that you need to read the entire document into
memory, though. I've seen data documents like this, and they can be quite
large. Simply detecting the blank line and the date is sufficient to
seperate groups and do a little processing.

Below, I've taken Dennis' code and added some logic... (warning:
uncompiled
code)

// initialize your database object
SqlConnection myConnect = new SqlConnection (Your Connection String);
myConnect.Open();

// Load the file (filename).
StreamReader sr = new StreamReader(filename);
Hashtable infoTable = new Hashtable();
string [] kvPair = null;
string line = null;
while (null != (line = sr.ReadLine()))
{
if (line.Trim.Len = 0) // you've hit a blank line... group ends
{
string Sql_String = string.Format("Insert MyTable (date,
username, inputoctets, outputoctects) values ('{0}', '{1}', '{2}',
'{3}')",
infoTable["Date"] , infoTable["User-Name"],
infoTable["Input-Octets"], infoTable["Output-Octets"]);
SqlCommand myCommand = new SqlCommand(Sql_String, myConnect);
myCommand.ExecuteNonQuery();
infoTable.Clear();

}
else
{
kvPair = line.Split('=');
if (kvPair.Length = 1) // this is the date!
{
infoTable.Add("Date",line.Trim());
}
else
{
infoTable.Add(kvPair [0].Trim(), kvPair [1].Trim());
}

As a note. If you want to remove quotes you'll have to process that here.
This particular algorithm will result in quoted strings being added to your
DB.
kvPair[1].Trim().Trim('"'); would be sufficent, if mildly messy
 
As a note. If you want to remove quotes you'll have to process that here.
This particular algorithm will result in quoted strings being added to your
DB.
kvPair[1].Trim().Trim('"'); would be sufficent, if mildly messy

Good point. I missed that detail.

now, if I just had Edit and Continue...
(just kidding :-)

--- Nick
 
Nick Malik said:
As a note. If you want to remove quotes you'll have to process that here.
This particular algorithm will result in quoted strings being added to your
DB.
kvPair[1].Trim().Trim('"'); would be sufficent, if mildly messy

Good point. I missed that detail.

now, if I just had Edit and Continue...
(just kidding :-)

LOL
 
Back
Top