Weird characters on admin and pages

[makphisto]

So I went to my site today and saw that I have some very strange characters. For example, my apostrophies show up thusly:
Gurkee?s? Rope Sandals

In my admin, there is weirdness also, it appears like all the register marks (trademarks?) show up as question marks = eg, OpenMaintenance�

I thought at first that it was maybe something weird with my PC, but I can see it from other PCs in my office.

So, three questions -
Can you see it? http://www.gurkeescentral.com/Merchant2/merchant.mvc?Screen=SFNT

Any idea what caused it?

Any idea how to fix it?

[Bruce - PhosphorMedia]

Looks like you used high-order ascii character formating from a rich text editor such as MS Word. These characters (usually use in RTF editors to render TM, (c), 1/2, etc ) cannot be used in HTML files. Instead, you need to use entity encoding...for example:
& for &

Here's a handy list:

http://www.chucke.com/entities.html

[Vic - WolfPaw Computers]

Did you recently make any changes to the text using a different text editor than you normally use?

So I went to my site today and saw that I have some very strange characters. For example, my apostrophies show up thusly:
Gurkee?s? Rope Sandals

In my admin, there is weirdness also, it appears like all the register marks (trademarks?) show up as question marks = eg, OpenMaintenance�

I thought at first that it was maybe something weird with my PC, but I can see it from other PCs in my office.

So, three questions -
Can you see it? http://www.gurkeescentral.com/Merchant2/merchant.mvc?Screen=SFNT

Any idea what caused it?

Any idea how to fix it?

[makphisto]

Looks like you used high-order ascii character formating from a rich text editor such as MS Word. These characters (usually use in RTF editors to render TM, (c), 1/2, etc ) cannot be used in HTML files. Instead, you need to use entity encoding...for example:
& for &

All I do is make the page, paragraph, etc, look like I want it to in Frontpage (yes, yes, I know, FP is the devil, blah blah). Then copy and paste the code where it's supposed to go. I do this all the time to my stores and it works fine.

And even if you attribute the "weirdness" to user error, that doesn't explain why special characters in my admin are broken - I have no control over those!

[makphisto]

Did you recently make any changes to the text using a different text editor than you normally use?

Nope. And I wouldn't have even noticed the store errors for awhile, except for my admin special characters now are all gone and replaced with question marks.

[dotCOM_host]

makphisto,

It sounds like your server has the "wrong" default character set defined and that's the root cause of why some high-ASCII characters get replaced by question marks. It's pretty common to see especially with hosting companies that just set up stock Linux server and don't know what may need to be changed from that default configuration to make it work with the US alphabet. Yes, even Red Hat Enterprise Linux (company based in the US), by default, does not have the proper US character set defined. Contact your host to change the default character set in Apache config and once they fix that and restart Apache, everything will appear properly once again.

[Bruce - PhosphorMedia]

Did you perhaps change your langauge setting? Check, (in IE) Tools | Internet Options | [Languages], should say English (United States) [en-us]...does it look the same on other browsers?

[ILoveHostasaurus]

makphisto,

It sounds like your server has the "wrong" default character set defined and that's the root cause of why some high-ASCII characters get replaced by question marks.

An apostrophe is not in the extended ascii range.

[ILoveHostasaurus]

Makphisto, can you try a different browser to do the update and see if that changes things? You're using a somewhat older version of FireFox so that may be playing a role; give IE a try and see if it changes the text. Also, paste the same content into notepad to see what it looks like there as well.

[dotCOM_host]

Actually, if you review the source code of these pages, these weird characters are all in there, so it's not just Apache display issue. It looks like you may have edited your HTML code in some WYSIWYG editor and pasted it back into MIVA Merchant admin screens, which then translated some of these characters to that gooble-gook... (is that a technical term? <g>) Try cleaning up the HTML code in Merchant admin by hand, first, and if that resolves your issue - you are all done. Otherwise, if your code has proper characters but the web browser shows something else - go back to that previous suggestion to verify the default character set in Apache config.

[truXoft]

Unfortunately Miva Merchant (including the Admin pages) does not specify the character set in the HTTP or HTML headers, so the browser tries to detect it automatically - the result is depending not only on the browser version and the characters used on the page, but also on other factors. That has the effect that some characters may be parsed differently on different machines or in different situations. As a user. you can change the character set manually in browser's menu (View >> Encoding in MSIE), but I highly recommend either setting the character set site-wide through the HTTP headers in web server configuration (.htaccess or httpd.conf - see this link (http://www.w3.org/International/questions/qa-htaccess-charset) for more info) or in the HTML header - you need a module like the MmEXT (http://www.mivacentral.com/truxoft/?product=TRUXOFT-MMEXT) for adding directives to the HTTP or HTML headers.

I recommend using the Latin-1 character set (iso-8859-2), as long as the multicode UTF-8 is avoidable.

[makphisto]

Makphisto, can you try a different browser to do the update and see if that changes things? You're using a somewhat older version of FireFox so that may be playing a role; give IE a try and see if it changes the text. Also, paste the same content into notepad to see what it looks like there as well.

I get the weird characters in both Firefox and IE, on this and different computers.

When I look at the "content" in the admin, they weird characters are all question marks. That is the same when I copy and paste the content into Notepad. I can change the question mark in admin back to the apostrophe, and when I update, it looks fine. SO, I guess I could go thru all my content and fix the problems, but that STILL doesn't explain why it's weird in the uncontollable portions of admin. That and I hate to go manually fix everything just to have it mysteriously happen again...

[ILoveHostasaurus]

That's very unusual, if pasting from FP to notepad produces the question marks, it sounds like FP is trying to encode the data for some reason, even though an apostrophe doesn't need any special encoding. I have not seen it do that before. Could FP or your Microsoft language settings be set to something other than US English?

[makphisto]

That's very unusual, if pasting from FP to notepad produces the question marks, it sounds like FP is trying to encode the data for some reason, even though an apostrophe doesn't need any special encoding. I have not seen it do that before. Could FP or your Microsoft language settings be set to something other than US English?

Sorry, I misunderstood your request. I had copied and pasted the content from admin to notepad.

If I make new content in FP and paste it to notepad, it looks right.

[dotCOM_host]

I recommend using the Latin-1 character set (iso-8859-2), as long as the multicode UTF-8 is avoidable.
Hi Ivo,

UTF-8 seems to be the default character set in Red Hat Enterprise Linux and many of its variants, so changing that globally in httpd.conf is always a good idea. But I wanted to ask you the reason you recommended ISO-8859-2, rather than ISO-8859-1...? Here in the US we tend to use ISO-8859-1 for most purposes. Is ISO-8859-2 used mostly where you are located? Based on my understanding of the ISO standards, ISO-8859-2 should be used for Central and Eastern European languages (which we have used for some of our Polish clients, for example), but in general, most people seem to be quite content with ISO-8859-1 which seems to be pretty universally accepted.

Not disagreeing with your recommendation, just trying to see if there's something I wasn't aware of and why ISO-8859-2 would be preferable to ISO-8859-1, or under what circumstances other than what I just mentioned above.

Thanks

[Brady]

I had the same problem. It's most likely your host. Did these characters all of a sudden appear when before everything was ok? I never got it resolved. What I had to do was declare a different character encoding to fix it. Send me a email if you want more info.

[dotCOM_host]

Brady - the problem is easy enough to fix if it has to do with the default character set. Some hosts "forget" to change it from the default UTF-8 to ISO-8859-1, hence creating this problem on their servers. Ify ou are on a shared server you will have to ask your host to change this, or implement this through your own .htaccess file (editing http.conf is preferable).

[FJC]

I have a similar problem: the sites can be seen properly using IE but in Firefox 1.5 I see the � instead of apostrophes and accented characters. I have tried every different character encoding in the browser to no avail. Any idea what if anything can be done in Firefox to fix this?

Thanks