Unicode Character Issue

[Samuel]

Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.


Thank you,
Samuel

 

[Joe Duchtel]

Hello -

There is an overloaded constructor that takes an Encoding (e.g.
System.Text.UnicodeEncoding). Visual Basic treats all strings as
Unicode so if you do a ReadLine() to a String it should work.

Joe

 

[Joe Duchtel]

Hello -

There is an overloaded constructor that takes an Encoding (e.g.
System.Text.UnicodeEncoding). Visual Basic treats its String as
Unicode so you should be find if you use ReadLine().

Joe

 

[Joergen Bech]

Here is a small, working example. Disregard
the ReadBinary stuff. That was just something
from another project and could be written
smaller/cleaner/etc.

Regards,

Joergen Bech

---snip---

Option Explicit On
Option Strict On

Imports System.IO
Imports System.Text
Imports System

Public Class Form1

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles MyBase.Load
Dim ansi() As Byte = ReadBinary("d:\ansi.txt")
Dim s As String =
Encoding.GetEncoding("iso-8859-1").GetString(ansi)
TextBox1.Text = s

' Dim sb As New StringBuilder
' For Each b As Byte In ansi
' sb.Append(Chr(b))
' Next
' TextBox1.Text = sb.ToString
End Sub

Public Shared Function ReadBinary(ByVal file As String) As Byte()
Dim errorInformation As String = String.Empty
Dim result As Byte() = ReadBinary(file, errorInformation)
If String.IsNullOrEmpty(errorInformation) Then
Return result
Else
Throw New IOException(errorInformation)
End If
End Function

Public Shared Function ReadBinary(ByVal file As String, _
ByRef errorInformation As
String) As Byte()

Try
Dim fInfo As New FileInfo(file)
Dim numBytes As Long = fInfo.Length
Dim fStream As New FileStream(file, FileMode.Open,
FileAccess.Read)
Dim br As New BinaryReader(fStream)
Dim data() As Byte = br.ReadBytes(CInt(numBytes))

br.Close()
fStream.Close()

Return data

Catch ex As Exception
errorInformation = ex.Message
Return Nothing

End Try

End Function

 

[Samuel]

Unicode didn't work but 'Default' did

Thank you,
Samuel


Hello -

There is an overloaded constructor that takes an Encoding (e.g.
System.Text.UnicodeEncoding). Visual Basic treats all strings as
Unicode so if you do a ReadLine() to a String it should work.

Joe

 

[Cor Ligthert[MVP]]

Samuel,

Are you sure that the program that is showing you the characters is able to
read and show those characters.

You would not be the first to make the wrong conclussion because of this.

By instance Windows mail (Outlook Express) will not show these characters
as it is not send in HTML format.

Cor

 

[Samuel]

Certainly, because when I save the same document in Unicode format I can
read it without any problem

Thank you,
Samuel

 

[Cor Ligthert[MVP]]

Samuel,

By the way what do you mean with Ansi format.

Wikipedia does not know it.

Cor

 

[Samuel]

Open notepad and look in the list of the Encoding - did I have to say
encoding

Regards,
Samuel

 

[Joergen Bech]

In the context of the original post, Wikipedia does indeed have a few
articles describing the codes:
http://en.wikipedia.org/wiki/Windows_1252
http://en.wikipedia.org/wiki/ISO-8859-1

Regards,

Joergen Bech

 

[Cor Ligthert[MVP]]

Samuel,

There is often spoken about ANSI when is meant the code table for
characters, but ANSI does give as much information as when you ask for about
a problem with a car and tell that it is an European brand.

The used bit format, or tell that you are using a code table, or even ASCII
although most people are then in fact talking about extended ASCII which is
for English and Dutch code table 437 gives much more information. While
using true ASCII which only has 7 bits characters (in fact 26 alphabetic
ones in upper and lower case).

As you are encoding between those, it will always take the characters as are
in the resulting code.

Cor