System.Text.Encoding.Default question

[> Adrian]

How do I use System.Text.Encoding.Default
in conjunction with "append"?
The objective being not to get into all sorts of
problems with special ASCII characters
like é etc.

Many thanks,
Adrian.

 

[Jon Skeet [C# MVP]]

How do I use System.Text.Encoding.Default
in conjunction with "append"?

In what context?

The objective being not to get into all sorts of
problems with special ASCII characters
like é etc.

Well, the first thing to understand is that that isn't an ASCII
character.

It would really help if you could explain what you're trying to do,
what kind of application you're writing etc.

Jon

 

[Marc Gravell]

ASCII only defines the first 128 symbols, which does not include é;

From your post, it isn't entirely sure if you are trying to exclude é from
the source, or handle it; it the latter, you can't using ASCII, which is why
encodings exist in the first place. Note that the "default" encoding is
(IIRC) only "default" on your system; in a different config something else
may be the default, and you could start seeing the wrong characters.
Either you need to agree on (or transmit separately) a specific codepage, or
you need to use a more versatile encoding such as UTF-8. Note that UTF-8 is
compatible with ASCII as long as you stay within the 128 ASCII characters.

If you clarify what exactly you want to do, somebody can probably give a
more complete answer

Marc

 

[> Adrian]

How do I use System.Text.Encoding.Default
in conjunction with "append"?

In what context?

The objective being not to get into all sorts of
problems with special ASCII characters
like é etc.

Well, the first thing to understand is that that isn't an ASCII
character.

It would really help if you could explain what you're trying to do,
what kind of application you're writing etc.

Jon
*********************
Thank you for reply. I received the same reply (in content)
from another respondent and will explain in my reaction
to him.

Regards,
Adrian.

 

[> Adrian]

ASCII only defines the first 128 symbols, which does not include é;

From your post, it isn't entirely sure if you are trying to exclude é from
the source, or handle it; it the latter, you can't using ASCII, which is why
encodings exist in the first place. Note that the "default" encoding is
(IIRC) only "default" on your system; in a different config something else
may be the default, and you could start seeing the wrong characters.
Either you need to agree on (or transmit separately) a specific codepage, or
you need to use a more versatile encoding such as UTF-8. Note that UTF-8 is
compatible with ASCII as long as you stay within the 128 ASCII characters.

If you clarify what exactly you want to do, somebody can probably give a
more complete answer

Marc

*******************************
PROBLEM DESCRIPTION
*******************************
In an application I create text. For it I have
written a spell check module in which I
compare words from the text with words
from the vocabulary. If the vocabulary
does not contain a word, that word is
added to the vocabulary upon the request
of the user. Problems arise if the word
the user adds contains characters like é,
à, ë, etc. Their representation in the
list of vocabulary words, which is a txt
file, is with 'funnies', presumably because
the txt format cannot handle characters outside
of the limited range of ASCII characters.
*******************************
Many thanks in advance for your help.

Adrian.

 

[Marc Gravell]

Well, people get very touchy over the squiggles etc in spelling, so IMO the
correct approach here is to handle the non-ASCII data. To avoid problems, I
would recommend simply explicitely using UTF-8.

I believe File.Append() and File.AppendAllText() use UTF-8 by default, else
something like:

using(StreamWriter writer = new StreamWriter(@"c:\myfile.txt",true, new
UTF8Encoding())) {
writer.WriteLine("abc");
writer.WriteLine("def");
}

Obviously you need to read them with UTF-8 compatible code, but this should
be very similar.
Does this answer the question?

Marc

 

[Jon Skeet [C# MVP]]

Problems arise if the word

the user adds contains characters like é,
à, ë, etc. Their representation in the
list of vocabulary words, which is a txt
file, is with 'funnies', presumably because
the txt format cannot handle characters outside
of the limited range of ASCII characters.

txt isn't really a "format" as such - it certainly doesn't specify
which encoding to use.

I suggest (like Mark did) that you both read and write the vocabulary
file using UTF-8.

Jon

 

[> Adrian]

Well, people get very touchy over the squiggles etc in spelling, so IMO the
correct approach here is to handle the non-ASCII data. To avoid problems, I
would recommend simply explicitely using UTF-8.

I believe File.Append() and File.AppendAllText() use UTF-8 by default, else
something like:

using(StreamWriter writer = new StreamWriter(@"c:\myfile.txt",true, new
UTF8Encoding())) {
writer.WriteLine("abc");
writer.WriteLine("def");
}

Obviously you need to read them with UTF-8 compatible code, but this should
be very similar.
Does this answer the question?

Marc

********************************
Thank you, yes, I expect it will. I will
give it a try and see where that leads me.

Many thanks again,
Adrian.

 

[> Adrian]

Problems arise if the word
the user adds contains characters like é,
à, ë, etc. Their representation in the
list of vocabulary words, which is a txt
file, is with 'funnies', presumably because
the txt format cannot handle characters outside
of the limited range of ASCII characters.

txt isn't really a "format" as such - it certainly doesn't specify
which encoding to use.

I suggest (like Mark did) that you both read and write the vocabulary
file using UTF-8.

Jon
******************************
Yes, that is what Marc suggested as well.
Thank you Jon.

Regards,
Adrian.