aps.net : BIG BUG in streamwriter

S

Steph

we are in 2009 and nobody has seen this bug before me?

StreamWriter sw = new StreamWriter(Server.MapPath("./")+"test.xml",
false, Encoding.UTF8);
sw.Write("a text");
sw.Close();


it writes a file in UTF-16 and not UTF-8! too funny!

EL
 
M

miher

Steph said:
we are in 2009 and nobody has seen this bug before me?

StreamWriter sw = new StreamWriter(Server.MapPath("./")+"test.xml", false,
Encoding.UTF8);
sw.Write("a text");
sw.Close();


it writes a file in UTF-16 and not UTF-8! too funny!

EL
Click to expand...

Hi,

How could You determine encoding for a file having "a text" in it ?

-Zsolt
 
S

Steph

miher said:
Hi,

How could You determine encoding for a file having "a text" in it ?

-Zsolt
Click to expand...

just open it in hexa editor, you can use UltraEdit (and Ctrl+H)
 
J

Jeff Johnson

hum.. after test... it s only on ASP.NET
Click to expand...

Then I would say you'd be better off taking this to an ASP.NET group since
the fact that you're using C# is only incidental.
 
P

Pavel Minaev

hum.. after test... it s only on ASP.NET
Click to expand...

Given that StreamWriter and FileStream (which it will use internally
in your case) execute exactly the same code, whether in an ASP.NET
application or in a standalone one, mean that it shouldn't really
matter. If it does, it is likely for some other reason (e.g. elsewhere
you're overwriting the file).

Did you actually check it with the exact code that you've written
above in ASP.NET, and nothing else?
 
S

Steph

Pavel said:
Given that StreamWriter and FileStream (which it will use internally
in your case) execute exactly the same code, whether in an ASP.NET
application or in a standalone one, mean that it shouldn't really
matter. If it does, it is likely for some other reason (e.g. elsewhere
you're overwriting the file).

Did you actually check it with the exact code that you've written
above in ASP.NET, and nothing else?
Click to expand...

it s in a aspx.
 
J

Jeroen Mostert

Steph said:
we are in 2009 and nobody has seen this bug before me?

StreamWriter sw = new StreamWriter(Server.MapPath("./")+"test.xml",
false, Encoding.UTF8);
sw.Write("a text");
sw.Close();


it writes a file in UTF-16 and not UTF-8! too funny!
Click to expand...
No, it doesn't. Try it yourself in a new ASP.NET project (without other code
or other files to interfere). Make sure this code actually executes, and
nothing else.

Either this is not the code you're using in your actual project or you're
overlooking something.
 
S

Steph

Jeroen said:
No, it doesn't. Try it yourself in a new ASP.NET project (without other
code or other files to interfere). Make sure this code actually
executes, and nothing else.

Either this is not the code you're using in your actual project or
you're overlooking something.
Click to expand...

lol
are do a test ?


ok ok, for you : here a simple code in a new empty website :

using System;
using System.Data;
using System.Configuration;
using System.Collections;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;

public partial class test : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
System.IO.StreamWriter sw = new
System.IO.StreamWriter(Server.MapPath("./") + "test.xml", false,
System.Text.Encoding.UTF8);
sw.Write("a text");
sw.Close();
}
}


that write a UTF-16 File !!!! not a a UTF-8 !!!!!! take a time to just
look the BOM ! (edit file in a hexa editor)

System.Text.Encoding.UTF8 = BOM = FFFE (UTF-16 not utf-8)
System.Text.Encoding.Unicode = BOM = FFFE (all right)

Ho ! sorry its a bug ! don't cry !!! just test it correctly ! lol
 
S

Steph

Jeroen said:
No, it doesn't. Try it yourself in a new ASP.NET project (without other
code or other files to interfere). Make sure this code actually
executes, and nothing else.

Either this is not the code you're using in your actual project or
you're overlooking something.
Click to expand...

another test :
create a xml file with encoding = UTF-8, write chars in UTF-8 encoding
and.....

just try do deserialize it in UTF-8 encoding...

surprise ! bug !

why ?
because it's a UTF-16 file with UTF-8 chars...

so open it in (for example) UltraEdit (look Hexa code : BOM=utf-16 and
not utf-8!),
save it in utf-8 (look Hexa code : BOM=utf-8 it's perfect)
So now, you can deserialize it with an UTF-8 encoding.


big problem.... all writer no respect the UTF-8 encoding...
where the way ?
 
P

Pavel Minaev

ok ok, for you : here a simple code in a new empty website :

using System;
using System.Data;
using System.Configuration;
using System.Collections;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;

public partial class test : System.Web.UI.Page
{
     protected void Page_Load(object sender, EventArgs e)
     {
         System.IO.StreamWriter sw = new
System.IO.StreamWriter(Server.MapPath("./") + "test.xml", false,
System.Text.Encoding.UTF8);
         sw.Write("a text");
         sw.Close();
     }

}

that write a UTF-16 File !!!! not a a UTF-8 !!!!!! take a time to just
look the BOM ! (edit file in a hexa editor)
Click to expand...

With those precise steps to reproduce, it wrote an UTF-8 file for me,
with UTF-8 BOM (EF BB BF). No bug. This is on VS2008 SP1 (and I was
using its built-in web server to test, not IIS).

Maybe, after the file is written to disk, you're opening it in some
editor which proceeds to rewrite it as UTF-16? I really don't see any
explanation... it could be that you're using an older .NET version,
but even then I can't imagine how such a bug could work.

Oh, actually, I have another suspicion. After that file is written,
how do you open it? Directly from the disk in your website folder? Or,
perchance, using the browser (i.e. going to /WebSite1/test.xml)? If
the latter, then it could be the HTTP server that transcodes it.
 
S

Steph

Pavel said:
With those precise steps to reproduce, it wrote an UTF-8 file for me,
with UTF-8 BOM (EF BB BF). No bug. This is on VS2008 SP1 (and I was
using its built-in web server to test, not IIS).

Maybe, after the file is written to disk, you're opening it in some
editor which proceeds to rewrite it as UTF-16? I really don't see any
explanation... it could be that you're using an older .NET version,
but even then I can't imagine how such a bug could work.

Oh, actually, I have another suspicion. After that file is written,
how do you open it? Directly from the disk in your website folder? Or,
perchance, using the browser (i.e. going to /WebSite1/test.xml)? If
the latter, then it could be the HTTP server that transcodes it.
Click to expand...

hello,
i use IIs with Net2.5x.

"you're opening it in some editor which proceeds to rewrite it as UTF-16?"

no, because, .NET it self cannot open it if you specify the encoding in
deserialization.
And i open the xml file directly from the disk : "right click + open
with"...

and i confirm : it's a utf-16 file (bom : fffe) with chars in utf-8...
if i don't open the xml file manually for change the bom :
when i want deserialize it with an utf-8 encoding... bug. "unrecognized
chars at position 1,1"

after i open manually the file in an hexa decimal editor, the bom is
UTF-16, i save in utf-8 and i can deserialize it (because, the bom is
utf-8 and chars encoding is also utf-8)



"(and I was using its built-in web server to test, not IIS)."
i have try : identically

i will install .NET3 .... may be !

it s a bug in 2.5 !
 
S

Steph

Steph said:
another test :
create a xml file with encoding = UTF-8, write chars in UTF-8 encoding
and.....

just try do deserialize it in UTF-8 encoding...

surprise ! bug !

why ?
because it's a UTF-16 file with UTF-8 chars...

so open it in (for example) UltraEdit (look Hexa code : BOM=utf-16 and
not utf-8!),
save it in utf-8 (look Hexa code : BOM=utf-8 it's perfect)
So now, you can deserialize it with an UTF-8 encoding.


big problem.... all writer no respect the UTF-8 encoding...
where the way ?
Click to expand...


i have test on another server... identically... same bug... file is an
utf-16 file ! not an utf-8 ! an i open it in notepad++ (on the server...)

it s really a bug. shit.
 
S

Steph

Steph said:
another test :
create a xml file with encoding = UTF-8, write chars in UTF-8 encoding
and.....

just try do deserialize it in UTF-8 encoding...

surprise ! bug !

why ?
because it's a UTF-16 file with UTF-8 chars...

so open it in (for example) UltraEdit (look Hexa code : BOM=utf-16 and
not utf-8!),
save it in utf-8 (look Hexa code : BOM=utf-8 it's perfect)
So now, you can deserialize it with an UTF-8 encoding.


big problem.... all writer no respect the UTF-8 encoding...
where the way ?
Click to expand...


possible solution : write a file without encoding (bom) just chars
encoding.... ? but utf-16 must have the bom... so, need to write a
personnalized steamwriter...
 
J

jmostert

hello,
i use IIs with Net2.5x.
Click to expand...
There is no .NET 2.5. There's 2.0, 3.0 and 3.5, with various service
packs.

I cannot reproduce it on my machine either (WinXP SP3 32-bit, VS 2008
SP1, .NET 3.5). You should open an issue with Microsoft Connect
(http://connect.microsoft.com/) if you have full (programmatic) steps
to reproduce it. Make sure to mention your exact configuration,
because it seems to matter.
 
S

Steph

Steph said:
we are in 2009 and nobody has seen this bug before me?

StreamWriter sw = new StreamWriter(Server.MapPath("./")+"test.xml",
false, Encoding.UTF8);
sw.Write("a text");
sw.Close();


it writes a file in UTF-16 and not UTF-8! too funny!

EL
Click to expand...

StreamWriter write in unicode. Unicode use three encoding : utf-8,
utf-16 and utf-32.

When you define an encoding :
new StreamWriter(myfile, false, Encoding.UTF8);
or
new StreamWriter(myfile, false, Encoding.UTF16);
or
new StreamWriter(myfile, false, Encoding.UTF32);

StreamWriter write an unicode (BOM FFFE) file but with only one code
page for the chars !

But if you specific another encoding like ascii etc... that is good.

my problem ? write an utf-8 file with utf-8 chars ! but how do .... :(
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Xml Encoding Problem!!! 7
Reading and writing UTF-8 files 8
UTF-8 preamble -> Possible bug in StreamWriter(or at least strange behaviour..) 10
I/O buffering 4
How do I create a one line text file with control codes? e.g.: 144 = 0x90 and 147 = 0x93? 13
XML HTTP 10
memory leak in streamwriteR? help... 5
Embed a hex character in a text file 1

Top