Parse library

[William Stacey [MVP]]

BTW. The html doco on bind (and src and binaries) are available at
www.ics.org. If you ever have any questions on bind or w2k dns, I will be
most happy to help or find an answer for you. Cheers!

--
William Stacey, MVP

I've gone the basic route. You can fine the article and full code for a parse
that does
a pseudo-bind format. My format might support something like:

bind {
recursion = true;
forwarders {
0 = "192.168.0.1"; 1 = "192.168.0.2";
}
view1 {
zone1 {
name = "mydomain.com";
forwarders {
0 = "192.168.0.1"; 1 = "192.168.0.2";
}
}
}
}

Note, that I am going to finally implement the full bind configuration format,
not because I have
to, but because I think it would be kind of cool. The result of my compiler is
an Xml file format,
but you could easily evolve the compiler to spit out another format if you
wished. I point out in
the article that I do semantic processing in two places. If you wanted a parser
rather than a
linked parser/compiler module, then there are some things that would have to be
changed to be
more efficient to that form of program. With a parser module, you'd expect some
abstract output
that you would then input to a compiler module that would create the final view.
I've simply linked
these two steps into one because it is common and easy to do for small
languages.

http://weblogs.asp.net/justin_rogers/archive/2004/05/16/132744.aspx


--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers

Your the man! That is sweet. You gave me an idea. It might be
interesting
to use c# code style over the bind style. The bind is good and what I was
after, but the c# style would be a very cool twist and I am not tied to
the
bind but for the common use of it. How could/should I define the same
kind
of config using c# style such as below and will this work?

class BindConfig
{
bool recursion = true;
IPAddress[] forwarders = {"192.168.0.1", "192.168.0.2"}; // not sure
here

class View1
{
class Zone1
{
string name = "mydomain.com.";
IPAddress[] forwarders = ...
}
}
class View2
{
// other zones.
}
}

Not sure if this is better or not, but could be flexible.

If you are going to go this far, why not make it full blown syntax?

config Bind
{
bool recursion = true;
IPAddress[] forwards = {192.168.0.1, 192.168.0.2};

view basicView
{
zone FooZone
{
string name = "mydomain.com";
IPAddress[] forwarders = ...
}
}

}

or wahtever is specifica to your needs. Justins lexer is probably flexible
enough to do this, although the parser would probably need a bit of work for
type verification and the like. The flexibility is there, its just a matter
of designing it and writing the parser. When you get into typing, c style
strings, and the like the lexer and\or parser(depending on your design)
become a bit more complex, although not prohibitivly so.

Normally I don't need to be spoon fed, but this stuff is a bit new to me.
Very much appreciate your help and interest.

--
William Stacey, MVP

Phew, that makes my lexer look bulky, (536 lines so far). Granted it
supports several literal formats(string, integers and double right
now).

Yes, the lexer could be more advanced. Notice in a second post, I took
out even more code bringing the 39 lines to only 30 (for the lexer), but
added the necessary namespace imports and a test driver.

http://weblogs.asp.net/justin_rogers/archive/2004/05/15/132693.aspx

Out of curiosity, what are you using for your parser? Are you using a
parser
generator or just writing one by hand? I imagine I could wait for you
to
post the article, but I am curious.

I generally write my parser's by hand. It doesn't take much. I'm not at
the
machine with the parser on it right now so I can't tell you the number of
lines
to parse the Bind configuration file (thanks to William for pointing out
the
source of this configuration format), but it is relatively short, on the
order
of
only 100-150 with comments. There is some code to build the XmlDocument
object (note this is not a straight parser, but rather a parser +
compiler
module
that compiles the Bind configuration file into XmlDocument). I'll have
the
article
up shortly.

 

[Justin Rogers]

I won't add the IP change to the base code that I've already written, but you
can
special case the Value state code-path. Currently the Value state code-path
does:

Eat -> Assignment
Eat -> Value
Eat -> StatementTerminator

In order to enter this state we have to process an ID. As long as your period
isn't
a breaking character, then the IP address will come out as an identifier, if it
is not
placed within quotes (stringized). So you could have the following:

forwarders { 192.168.0.1; 192.168.0.2; }

Now, changes to the code-base would be that if assignment isn't found then look
for
the StatementTerminator immediately. Something like (not really something like,
probably
exactly like) the following.

} else {
Console.WriteLine("Executing Parser Loop: Processing Value
Start");
// Not starting a nesting? We must be a value;
string value = null;
int scanAssignment = ScanIgnoreWhitespace(TokenType.Assignment,
scanId + 1, typedTokens);
if ( scanAssignment > -1 ) {
// No longer an error here
Console.WriteLine("Executing Parser Loop: Scanned
Assignment");

int idOrString = ScanIdOrString(scanAssignment + 1,
typedTokens);
if ( idOrString == -1 ) {
// Error
break;
}
Console.WriteLine("Executing Parser Loop: Scanned ID Or
String");

int valueTerminator = -1;
value = string.Empty;
if ( typedTokens[idOrString].TokenType ==
TokenType.StringDelimiter ) {
Console.WriteLine("Executing Parser Loop: Building Value
From String");
// We need to build a string now
valueTerminator = ScanString(idOrString + 1,
typedTokens);
if ( valueTerminator == -1 ) {
// Error
break;
}

for(int concat = idOrString + 1; concat <
valueTerminator; concat++) {
value += typedTokens[concat].Token.TokenData;
}
} else {
Console.WriteLine("Executing Parser Loop: Building Value
From ID");
valueTerminator = idOrString;
value = typedTokens[valueTerminator].Token.TokenData;
}
Console.WriteLine("Executing Parser Loop: Our Value {0}",
value);
}

int endStatement =
ScanIgnoreWhitespace(TokenType.StatementTerminator, valueTerminator + 1,
typedTokens);
if ( endStatement == -1 ) {
// Error
break;
}
Console.WriteLine("Executing Parser Loop: Scanned End
Statement");

if ( value != null ) {
XmlAttribute nodeAttr = xDoc.CreateAttribute("Value");
nodeAttr.Value = value;
idNode.Attributes.SetNamedItem(nodeAttr);
}
i = endStatement;
}

The resulting XML would now be:

<forwarders><192.168.0.1 /><192.168.0.2 /></forwarders>

That isn't valid XML now though. You could use the XmlConvert.EncodeName method
in order to convert the value into a valid element name. That would happen back
where
we create the element and add it to the parent node. That would also mean your
code
that processes the XML file would need to realize it has to DecodeName on
elements
within <forwarders> sections, since they'll be encoded.

--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers

BTW. The html doco on bind (and src and binaries) are available at
www.ics.org. If you ever have any questions on bind or w2k dns, I will be
most happy to help or find an answer for you. Cheers!

--
William Stacey, MVP

I've gone the basic route. You can fine the article and full code for a parse
that does
a pseudo-bind format. My format might support something like:

bind {
recursion = true;
forwarders {
0 = "192.168.0.1"; 1 = "192.168.0.2";
}
view1 {
zone1 {
name = "mydomain.com";
forwarders {
0 = "192.168.0.1"; 1 = "192.168.0.2";
}
}
}
}

Note, that I am going to finally implement the full bind configuration format,
not because I have
to, but because I think it would be kind of cool. The result of my compiler is
an Xml file format,
but you could easily evolve the compiler to spit out another format if you
wished. I point out in
the article that I do semantic processing in two places. If you wanted a parser
rather than a
linked parser/compiler module, then there are some things that would have to be
changed to be
more efficient to that form of program. With a parser module, you'd expect some
abstract output
that you would then input to a compiler module that would create the final view.
I've simply linked
these two steps into one because it is common and easy to do for small
languages.

http://weblogs.asp.net/justin_rogers/archive/2004/05/16/132744.aspx


--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers

Your the man! That is sweet. You gave me an idea. It might be
interesting
to use c# code style over the bind style. The bind is good and what I was
after, but the c# style would be a very cool twist and I am not tied to
the
bind but for the common use of it. How could/should I define the same
kind
of config using c# style such as below and will this work?

class BindConfig
{
bool recursion = true;
IPAddress[] forwarders = {"192.168.0.1", "192.168.0.2"}; // not sure
here

class View1
{
class Zone1
{
string name = "mydomain.com.";
IPAddress[] forwarders = ...
}
}
class View2
{
// other zones.
}
}

Not sure if this is better or not, but could be flexible.
If you are going to go this far, why not make it full blown syntax?

config Bind
{
bool recursion = true;
IPAddress[] forwards = {192.168.0.1, 192.168.0.2};

view basicView
{
zone FooZone
{
string name = "mydomain.com";
IPAddress[] forwarders = ...
}
}

}

or wahtever is specifica to your needs. Justins lexer is probably flexible
enough to do this, although the parser would probably need a bit of work for
type verification and the like. The flexibility is there, its just a matter
of designing it and writing the parser. When you get into typing, c style
strings, and the like the lexer and\or parser(depending on your design)
become a bit more complex, although not prohibitivly so.

Normally I don't need to be spoon fed, but this stuff is a bit new to me.
Very much appreciate your help and interest.

--
William Stacey, MVP

Phew, that makes my lexer look bulky, (536 lines so far). Granted it
supports several literal formats(string, integers and double right
now).

Yes, the lexer could be more advanced. Notice in a second post, I took
out even more code bringing the 39 lines to only 30 (for the lexer), but
added the necessary namespace imports and a test driver.

http://weblogs.asp.net/justin_rogers/archive/2004/05/15/132693.aspx

Out of curiosity, what are you using for your parser? Are you using a
parser
generator or just writing one by hand? I imagine I could wait for you
to
post the article, but I am curious.

I generally write my parser's by hand. It doesn't take much. I'm not at
the
machine with the parser on it right now so I can't tell you the number of
lines
to parse the Bind configuration file (thanks to William for pointing out
the
source of this configuration format), but it is relatively short, on the
order
of
only 100-150 with comments. There is some code to build the XmlDocument
object (note this is not a straight parser, but rather a parser +
compiler
module
that compiles the Bind configuration file into XmlDocument). I'll have
the
article
up shortly.