convert Hex to UTF8 and Vice Versa

P

prakash

Hi,
I am working on serial port communication wherein i am sending 4k of
hex data to the antena.
here the last but one byte of the data is the sum of all the bytes iam
sending.

ex: 3A 3D 25 46 87 24..... _85 9B____ .0D 0A

here the sum of all bytes is 85 9B which cant be cinverted to hex so i
need help to convert it to UTF8
Plz help me?
 
R

Rick Lones

prakash said:
Hi,
I am working on serial port communication wherein i am sending 4k of
hex data to the antena.
here the last but one byte of the data is the sum of all the bytes iam
sending.

ex: 3A 3D 25 46 87 24..... _85 9B____ .0D 0A

here the sum of all bytes is 85 9B which cant be cinverted to hex so i
need help to convert it to UTF8
Plz help me?

What do you mean 859B "can't be converted to hex"? Everything you've displayed,
including the checksum, is already hex or so it would appear.

When sending to the port itself, which Write() overload are you using? Are you
sending a byte array or are you sending a text string or are you sending a
character array?

If byte array, my comment above would apply - your data is alreay in hex, so
your question would seem meaningless.

Or do you simply mean to say that you don't know how to insert your integer
checksum sum into a byte array at the appropriate location? Given a 16-bit
unsigned value you would break it into two bytes - divide it by 256, the
quotient is the high-order byte and the remainder (MOD) is the low-order. Which
byte you would transmit first, however, almost certainly depends on the
enddianness expectations of the device you are sending to.

-rick-
 
A

Arne Vajhøj

I am working on serial port communication wherein i am sending 4k of
hex data to the antena.
here the last but one byte of the data is the sum of all the bytes iam
sending.

ex: 3A 3D 25 46 87 24..... _85 9B____ .0D 0A

here the sum of all bytes is 85 9B which cant be cinverted to hex so i
need help to convert it to UTF8

If you have a byte array containing UTF-8 then you can convert
it to a String using Encoding.UTF8.GetString(yourbytearray).

Arne
 
R

Rick Lones

Peter said:
[...]
If byte array, my comment above would apply - your data is alreay in
hex, so your question would seem meaningless.

Just to be clear: byte arrays are _not_ "already in hex". See my reply
to the OP for details.

I have seen it. My reply was tailored to the OP. To you I might instead say
"your data is already a base-16 text representation of an array of 8-bit
unsigned binary numbers". And I would also say that your example
byte[] bytes = { 0, 1, 3, 7, 15, 31 };
is a base-10 text representation of an array of same, albeit with different
content. It is the same object as {0x0, 0x1, 0x3, 0x7, 0xF, 0x1F} would be a
base-16 reprepresentation of, but obviously a very different animal from string
s = "000103070F1F".

What you call "hex" in your examples, I am used to calling "ASCII-Hex", i.e. an
ASCII text representation of 4-bit hex digits. What I call "hex" (at least in
my reply to the OP) is just binary bits grouped in fours - i.e., a binary number
rather than a character. So there is some nomenclature clash here (which I do
not propose to debate :)).

All of your careful explication regarding the differences between a number and
its various text representations is valid and informative, to be sure. It's the
kind of semantic clarity that is often necessary when one works with stuff at
this level, but only useful to someone who can process it. I still think his
difficulty might simply be that he is sending binary data and is unclear on how
exactly to embed his checksum into the stream. The fact that he is sending a 4K
chunk of data (to an antenna, no less) makes me believe "binary" rather than
"text", but of course this is a guess. (But if his difficulty involves Unicode
somehow then you are definitely his man . . .)

Regards,
-rick-
 
R

Rick Lones

Peter said:
Peter said:
On 11/11/10 4:25 AM, Rick Lones wrote:
[...]

Even assuming I have exactly the same data as the OP, there's nothing in
his post that tells us what format his data _is_ in. However, your
statement reads "if byte array", so clearly your reply is assuming he is
dealing with bytes. Thus, "hex" is inapplicable.

I know. That's why I tried to nail him down on exactly which overload of
SerialPort.Write() he was using. That seemed like a reasonably direct way of at
least beginning to find that out. Meanwhile, I was guessing that his data is an
array of 2's complement numbers being transmitted as an array of 8-bit bytes.
(And in that case maybe I can help him right now without having to give a
college course first.)

To me hex is the same as octal is the same as binary. It's just grouping binary
digits and giving the group a name, like "5", or "B". A base-100 number would
be the same as a base-10 one in the exact same sense. So when the OP says "hex"
I interpret that as "binary data", since I think there is a fair likelihood that
that is what he means.
And I would also say that your example
byte[] bytes = { 0, 1, 3, 7, 15, 31 };
is a base-10 text representation of an array of same, albeit with
different content.

I wouldn't say that at all. Once compiled, the data is binary data. In
some sense, it's necessarily in binary format (that's what all our PCs
use these days). In another sense, it has no format at all, as the
number is abstracted by the language and as such the format is opaque to
the program (in theory, one could implement .NET for a computer that
uses something other than binary for the internal data storage…for that
matter, you could even have some bizarre PC .NET implementation that
doesn't take advantage of the hardware's natural storage format).

Sure. Just like you can represent decimal digits in 4-bit chunks and work with
them arithmetically even on machines that have no decimal hardware assist at
all. It is all, as you say, an abstraction.
I don't see the point in labeling the format with ASCII. There's
nothing to suggest ASCII is involved at all. My own post is in UTF8,
and the OP's post is in ISO-8859-1. A .NET string is, from the
program's point of view, encoding-agnostic, but of course internally is
UTF16. What's ASCII got to do with anything here?

Sorry, but ASCII-Hex (or sometimes Hex-ASCII) is the term I have seen applied to
this type of representation. I didn't invent the term and so feel no need to
defend it, but it seemed to be in fairly wide use at that time and place and
nobody was claiming they didn't know what was meant by it. Apparently you have
never encountered it.
But that's the whole point. Your nomenclature is inaccurate and
misleading as applied to an array of bytes. You should not be calling
binary data "hex". It's not hex unless it's a text format.

Whereas to me hexadecimal, octal and binary are all numeric formats. (And are
all essentially binary in the sense I explained above.) And this is where I get
off when it comes to arguing about what "hex" means - it's pointless.
I still think his difficulty might simply be that he is
sending binary data and is unclear on how exactly to embed his checksum
into the stream. [...]

That could very well be. But there's nothing in his post to tell us one
way or the other. Note that half of my reply to him was addressing that
very issue: that there's no way to know for sure that text encoding has
anything at all to do with his problem. :(

Peter, sometimes it seems like half of ALL of your posts address that very
issue. It's a difference in approach partly. You seem to be trying to bring
him up to your standards of discourse so that in the end his problem will be
solved plus the world will be a better and more precise place - and that's fine.
Me, I'm just trying to meet him on his own linguistic ground and help him get
his job done. Hey, maybe he'll buy me a beer someday . . . :)

Regards,
-rick-
 
R

Rick Lones

Peter said:
[...]Meanwhile, I was guessing
that his data is an array of 2's complement numbers being transmitted as
an array of 8-bit bytes. [...]

2's complement is a binary data format for signed numbers. Bytes (that
is, the C# "byte" type) are unsigned.
[...] So
when the OP says "hex" I interpret that as "binary data", since I think
there is a fair likelihood that that is what he means.

Since "hex" doesn't mean "binary data" (in fact just the opposite),
that's an incredibly odd conclusion to draw.
[...]
Sorry, but ASCII-Hex (or sometimes Hex-ASCII) is the term I have seen
applied to this type of representation. I didn't invent the term and so
feel no need to defend it, but it seemed to be in fairly wide use at
that time and place and nobody was claiming they didn't know what was
meant by it. Apparently you have never encountered it.

No, I haven't. Nor have the major Internet search engines. It's simply
not a standard term for anything. I don't know where you came across
it, but you can't expect to use terms that no one else is using and
still be understood.

More importantly, introducing "ASCII" into a conversation where there's
no indication that ASCII plays any part whatsoever just confuses issues.
"Hex" is "hex", independent of encoding. Since it's text, at some
point encoding becomes an important question, but it's not part of the
textual representation of numbers.
[...]
Whereas to me hexadecimal, octal and binary are all numeric formats.
(And are all essentially binary in the sense I explained above.) And
this is where I get off when it comes to arguing about what "hex" means
- it's pointless.

There's no need for argument. The references are quite clear on the point.

For example, here's the Wikipedia page:
http://en.wikipedia.org/wiki/Hexadecimal

It goes on at length on various topics regarding hexadecimal
representation of numbers, and not once does the question of the actual
computer's internal representation of numbers come up.

You are right that it's pointless to argue about it. You are wrong to
suggest that hexadecimal is anything other than a way in human languages
to write numbers in a specific way. It's certainly not a synonym for
binary data (arranged in bytes or otherwise).
[...]
Peter, sometimes it seems like half of ALL of your posts address that
very issue. It's a difference in approach partly. You seem to be trying
to bring him up to your standards of discourse so that in the end his
problem will be solved plus the world will be a better and more precise
place - and that's fine.

You would do well to terminate your reasoning at the point of what you
can actually _observe_. You clearly have no idea what my motivations
are, and are simply compounding your errors by hypothesizing.

It's true that I often ask questioners to improve the precision of their
questions. But it's only because people who ask questions are often so
poor at forming a useful question, along with the fact that an imprecise
question has no answer to give.

You can guess at what the OP means until the cows come home, the fact is
you're just wasting your time without all the facts. You might stumble
across the right answer once in awhile, but if you're answering any
significant volume of questions that way, you're going to be wrong more
often than not when the question is so poorly formed. It's better to
just nail down the question first, than to grope around for what you
think _might_ be an answer.

Suit yourself, but don't go casting aspersions at those who choose a
more efficient approach.

Pete

No aspersions were intended, I assure you. As for the rest - you do it your way
and I'll do it mine.

Regards,
-rick-
 
P

prakash

No aspersions were intended, I assure you.  As for the rest - you do ityour way
and I'll do it mine.

Regards,
-rick-

I am placing the code for my query, where the data I am sending to the
antenna is not converted to ASCII and I am getting error in the
communication the bolded values are not converted to ASCII equivalent
and I am unable to communicate with the antenna so if I use UTF8 as a
converter can I send and recieve the data perfectly?

Collapse

private void btn_Refresh_Click(object sender, EventArgs e)
{
try
{
//Reading Command Password

serialPort1.Write(Hex2Ascii("3A08007A00564953494F4E33363DC20D0A"));
Thread.Sleep(400);
string resp = serialPort1.ReadExisting();
resp = resp.Substring(5, 2);
if (resp == "04")

{ serialPort1.Write(Hex2Ascii("3A08000300313233343536373851AE0D0A"));
.....
}

public string Hex2Ascii(string str)
{
string x2 = "";
for (int i = 0; i < str.Length; i = i + 2)
{
string m = str.Substring(i, 2);
int temp = Int32.Parse(m, NumberStyles.HexNumber);
char temp1 = converthex((char)temp);
string x1 = Convert.ToString(Convert.ToChar(temp1)); /
x2 = x2 + x1;
STRING RETRUNS
}
return (x2);
}


public static char converthex(char c)
{
if (c == 0x80) return (char)0x20AC;
if (c == 0x81) return (char)0xfffd;
if (c == 0x82) return (char)0x201A;
if (c == 0x83) return (char)0x0192;
if (c == 0x84) return (char)0x201E;
if (c == 0x85) return (char)0x2026;
if (c == 0x86) return (char)0x2020;
if (c == 0x87) return (char)0x2021;
if (c == 0x88) return (char)0x02C6;
if (c == 0x89) return (char)0x2030;
if (c == 0x8A) return (char)0x0160;
if (c == 0x8B) return (char)0x2039;
if (c == 0x8C) return (char)0x0152;
// ) return (char)0x8D;
if (c == 0x8E) return (char)0x017D;
// ) return (char)0x8F;
// ) return (char)0x90;
if (c == 0x91) return (char)0x2018;
if (c == 0x92) return (char)0x2019;
if (c == 0x93) return (char)0x201C;
if (c == 0x94) return (char)0x201D;
if (c == 0x95) return (char)0x2022;
if (c == 0x96) return (char)0x2013;
if (c == 0x97) return (char)0x2014;
if (c == 0x98) return (char)0x02DC;
if (c == 0x99) return (char)0x2122;
if (c == 0x9A) return (char)0x0161;
if (c == 0x9B) return (char)0x203A;
if (c == 0x9C) return (char)0x0153;
// ) return (char)0x9D;
if (c == 0x9E) return (char)0x017E;
if (c == 0x9F) return (char)0x0178;
return c;
}


This is the code for which I need your help.
 
R

Rick Lones

prakash said:
I am placing the code for my query, where the data I am sending to the
antenna is not converted to ASCII and I am getting error in the
communication the bolded values are not converted to ASCII equivalent
and I am unable to communicate with the antenna so if I use UTF8 as a
converter can I send and recieve the data perfectly?

Collapse

private void btn_Refresh_Click(object sender, EventArgs e)
{
try
{
//Reading Command Password

serialPort1.Write(Hex2Ascii("3A08007A00564953494F4E33363DC20D0A"));
Thread.Sleep(400);
string resp = serialPort1.ReadExisting();
resp = resp.Substring(5, 2);
if (resp == "04")

{ serialPort1.Write(Hex2Ascii("3A08000300313233343536373851AE0D0A"));
....
}

Just looking at your first sequence, I note:

You start with the string "3A . . . 0D0A" After running it through your
Hex2Ascii function it is returned as the string ":\b\0z\0VISION36=Â\r\n" and
that is what you end up transmitting. Is that (":\b . . . \r\n") the command
string you actually mean to send to your antenna? If not, what should it be?

I note that your initial string "3A . . . 0D0A" looks a bit like a Modbus-ASCII
command, although the 16-bit checksum would be non-standard AFAIK. Do you know
what protocol your antenna is expecting you to speak?

-rick-
 
P

prakash

Just looking at your first sequence, I note:

You start with the string "3A . . . 0D0A"  After running it through your
Hex2Ascii function it is returned as the string ":\b\0z\0VISION36=Â\r\n" and
that is what you end up transmitting.  Is that (":\b . . . \r\n") the command
string you actually mean to send to your antenna?  If not, what should it be?

I note that your initial string "3A . . . 0D0A" looks a bit like a Modbus-ASCII
command, although the 16-bit checksum would be non-standard AFAIK.  Do you know
what protocol your antenna is expecting you to speak?

-rick-- Hide quoted text -

- Show quoted text -

thanx for your reply,
ya my command is the same one which i have sent u, but here is one
problem that i m sending ":\b\0z\0VISION36=Â\r\n"
but my serial port is sending ":\b\0z\0VISION36=?r\n" the caracter Â
is interpreted as ?. the problem comes here where my antena is sending
error responce( crc error) which means when ever i m sending hex value
0x80 to 0xff my serial port is sending '?'.
 
R

Rick Lones

prakash said:
thanx for your reply,
ya my command is the same one which i have sent u, but here is one
problem that i m sending ":\b\0z\0VISION36=Â\r\n"
but my serial port is sending ":\b\0z\0VISION36=?r\n" the caracter Â
is interpreted as ?. the problem comes here where my antena is sending
error responce( crc error) which means when ever i m sending hex value
0x80 to 0xff my serial port is sending '?'.

Then at least in this one case it looks to me like you are starting with:
string instring = "3A08007A00564953494F4E33363DC20D0A";

and that what you ultimately want to transmit could be described as:
byte outbytes - new byte[]
{0x3A,0x08,0x00,0x7A,0x00,0x56,0x49,0x53,0x49,0x4F,0x4E,0x33,0x36,0x3D,0xC2,0x0D,0x0A};

In other words it appears that you just want a straight conversion from the text
representation of the bytes to the bytes themselves. Is this correct? If so,
it is easily done, you just need to be willing to use the byte array overload
of Send().

To be clear, I am suggesting that you forget about "Hex2Ascii" and instead think
"Hex2Bytes", e.g. something like . . .

public static byte[] Hex2Bytes(string instring)
{
byte[] outbytes = new byte[instring.Length >> 1];
for (int i = 0; i < instring.Length; i += 2}
{
string s = instring.Substring(i, 2);
outbytes = Byte.Parse(s, NumberStyles.HexNumber);
}
return outbytes;
}

.. . . but with appropriate input validation, of course.

I have some misgivings about the converthex function in your original code. It
seems to me that it would mangle your checksum in bad ways if the checksum value
was in the wrong range. However it does not apply to the example at hand.

Note that the difficulty you are having getting your bytes out the port the way
you want will also apply to getting them in. You will have to either treat your
input as raw bytes (my preferred way) or else figure out just what hoop .Net
expects you to jump through if you insist on treating your input as a string or
a character array.

Hope this helps some.

Regards,
-rick-
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top