Regular Expression Matches

[Pete Davis]

I'm using regular expressions to extract some data and some links from some
web pages. I download the page and then I want to get a list of certain
links.

For building regular expressions, I use an app call The Regulator, which
makes it pretty easy to build and test regular expressions.

As a warning, I'm real weak with regular expressions. Let's say my regular
expression is:

(href=)(?<caturl>.*)(class=title>\[&nbsp;&nbsp;&nbsp

Now, using The Regulator and giving it the source for a particular web page,
I get 8 matches.

According to the regulator, the options it's using are:

Multiline, ignore case, ignore whitespace

In my own code, I'm doing:

Regex indexRegex = new Regex(categoryListRegex,
RegexOptions.Multiline |
RegexOptions.IgnorePatternWhitespace |
RegexOptions.IgnoreCase);
MatchCollection indexMatches = indexRegex.Matches(pageText);

This only returns one match in indexMatches with the same page that I'm
giving The Regulator. It seems that no matter what combination of regex
options I use, I'm only getting one match.

Why is that? How do I get all 8 matches?

Thanks.

pete

 

[Kevin Spencer]

Hi Pete,

You need to escape the '<' and '>' characters in your Regurlar Expression.
These are used in some flavors of Regular Expression language to indicate a
named group. If the first (<caturl>) is a group name, name both groups or
neither.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.

 

[Pete Davis]

Well, there are 3 groups. <caturl> is a group name. The other 2 are unnamed.
Why do they need to be named if <caturl> is named? I'm not interested in the
other groups. I'm simply using them as "delimiters" for lack of a better
word.

I've modified the expression to look like this:

(href=)(?<caturl>.*)(class=title.*\[&nbsp;&nbsp;&nbsp

This gives the exact same results.

The escaping stuff gets a little confusing because the regular expressions
are actually stored in an XML file, so they get escaped for that.

In the XML file that looks like:

(href=)(?&lt;caturl&gt;.*)(class=title.*\[&amp;nbsp;&amp;nbsp;&amp;nbsp

This still isn't returning multiple results. Just the last match. I don't
think the < was the problem.

Pete

Hi Pete,

You need to escape the '<' and '>' characters in your Regurlar Expression.
These are used in some flavors of Regular Expression language to indicate
a named group. If the first (<caturl>) is a group name, name both groups
or neither.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.

I'm using regular expressions to extract some data and some links from
some web pages. I download the page and then I want to get a list of
certain links.

For building regular expressions, I use an app call The Regulator, which
makes it pretty easy to build and test regular expressions.

As a warning, I'm real weak with regular expressions. Let's say my
regular expression is:

(href=)(?<caturl>.*)(class=title>\[&nbsp;&nbsp;&nbsp

Now, using The Regulator and giving it the source for a particular web
page, I get 8 matches.

According to the regulator, the options it's using are:

Multiline, ignore case, ignore whitespace

In my own code, I'm doing:

Regex indexRegex = new Regex(categoryListRegex,
RegexOptions.Multiline |
RegexOptions.IgnorePatternWhitespace |
RegexOptions.IgnoreCase);
MatchCollection indexMatches = indexRegex.Matches(pageText);

This only returns one match in indexMatches with the same page that I'm
giving The Regulator. It seems that no matter what combination of regex
options I use, I'm only getting one match.

Why is that? How do I get all 8 matches?

Thanks.

pete

 

[Pete Davis]

I found the solution to the problem, so now if someone could explain why,
I'd appreciate that:

The solution is to replace carriage returns with line feeds.

If I do that, I get all 8 results instead of the 1 result I was getting.

Now, can anyone tell me WHY? And aren't there other regex options for
dealing with that without having to change the source text?

Thanks.

Pete

 

[Kevin Spencer]

The solution is to replace carriage returns with line feeds.

I didn't see any carriage returns or line feeds in your regular expression.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.

 

[Pete Davis]

The Carriage Returns and Linefeeds are in the web page I'm scanning, not the
regex.

Pete

 

[Kevin Spencer]

Well, Pete, I'm not sure, but I do know that Microsoft and Unix text
documents have a distinct difference in terms of line feeds. The Microsoft
text document uses CR/LF (\r\n), while the Unix model uses only LF(\n). Does
that tell you anything?

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.

 

[Pete Davis]

Kevin,

Thanks for the help. The Unix/Windows difference on crlf vs. lf I'm aware
of, but that's not really the issue.

The Regex engine in .NET, I would assume, is more geared towards windows
documents if anything, but maybe not.

There are Regex options for singleline and multiline to help determine how
the parser handles the data, but it's unclear to me how to get all matches
from my document without replacing the carriage returns. Nothing I've tried
has worked. I suspect there are changes I can make to the regular
expressions themselves to handle it, but I don't know how to do that.

Pete

 

[Larry Lard]

I'm using regular expressions to extract some data and some links from some
web pages. I download the page and then I want to get a list of certain
links.

Completely not answering your actual question, but maybe this will save
you a lot of bother: the free HtmlAgilityPack will convert an (even
malformed) HTML document into a nice XML tree which makes getting
content out much much easier.

 

[Kevin Spencer]

Hi Pete,

Here are some Regular Expressions that may help with this sort of thing. I
tend to use these rather than the options in the Regex engine:

(?i) Turn of case-sensitivity for the remainder
of the regular expression.
(?s) Turn on "dot matches now line" for the
remainder of the regular expression.
(?m) Caret and dollar sign match after and before
new lines for the remainder of the regular expression.
(?i-sm) Tunrs on the "i" and "m" options for the remainder
of the regular expression, and turn off "s"
(?i-sm:regex) Turns on the "i" and "m" options
for the regular expression inside the parentheses,
and turn off "s"

Here is the Microsoft .Net Framework Regular Expressions reference:

http://msdn.microsoft.com/library/d...l/cpconRegularExpressionsLanguageElements.asp

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.