Options

BUG: UTF-8 pages not displaying correctly (suggested solution included)

flyingdutchieflyingdutchie Registered Users Posts: 1,286 Major grins
edited October 16, 2005 in SmugMug Support
There are various thread dealing with improper displaying of international characters (just search for UTF-8 on the support forums).

1. Smugmug does correctly generate pages with this meta tag:
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
2. However, the HTTP response-header "Content-type" is incorrect! When i do a 'graburl' to my site, i get this http-response:
HTTP/1.1 200 OK
Date: Sat, 08 Oct 2005 16:07:01 GMT
Server: Apache
X-Powered-By: smugmug/1.2.0
Set-Cookie: SMSESS=026e577c53333abbcfcfd9247f880068; path=/; domain=.smugmug.com
Cache-Control: private, max-age=1, must-revalidate
Pragma:
X-Extra: 0.26807594299316
ETag: sm-a54c5ea41ed6f8c1233cdecdc2405552-sm
Content-Length: 29046
Connection: close
Content-Type: text/html; charset=ISO-8859-1

The incorrect Content-Type header causes the browsers to select ISO-8859-1 (Western European (ISO)) instead. Can this be fixed?

The last line (in red) is incorrect. It should read:
Content-Type: text/html; charset=UTF-8


Related threads:
http://www.dgrin.com/showthread.php?p=175958#post175958
http://www.dgrin.com/showthread.php?t=18290

Addition: I just saw that Luke Church did suggest the same solution:
http://www.dgrin.com/showpost.php?p=162986&postcount=12
I can't grasp the notion of time.

When I hear the earth will melt into the sun,
in two billion years,
all I can think is:
    "Will that be on a Monday?"
==========================
http://www.streetsofboston.com
http://blog.antonspaans.com

Comments

  • Options
    ScalaScala Registered Users Posts: 95 Big grins
    edited October 8, 2005
    The people at smugmug must be well aware of all this, but here we go again:

    In addition to the suggested fix above the content that was once saved as ISO-8859-1 (and is still saved as such if you type in text using the customize gallery screen, for example) will need to be converted to UTF-8. If not, my guess is that a lot of currently working international characters will end up displaying incorrectly.

    Here's an example. Go to http://scala.smugmug.com and select UTF-8 as the content encoding. There will be characters in my bio text displaying incorrectly or not at all.
    My smugmug site: www.majakorpi.net
  • Options
    flyingdutchieflyingdutchie Registered Users Posts: 1,286 Major grins
    edited October 8, 2005
    Scala wrote:
    The people at smugmug must be well aware of all this, but here we go again:

    In addition to the suggested fix above the content that was once saved as ISO-8859-1 (and is still saved as such if you type in text using the customize gallery screen, for example) will need to be converted to UTF-8. If not, my guess is that a lot of currently working international characters will end up displaying incorrectly.

    Here's an example. Go to http://scala.smugmug.com and select UTF-8 as the content encoding. There will be characters in my bio text displaying incorrectly or not at all.
    I see,
    but Smugmug should choose one or the other, since the meta tag in the pages looks like this:
    <meta http-equiv="content-type" content="text/html; charset=UTF-8" />

    So, the http response-header tells the browser to use one encoding, the <meta> tag to use another... it is still a bug.
    -- Anton.
    I can't grasp the notion of time.

    When I hear the earth will melt into the sun,
    in two billion years,
    all I can think is:
        "Will that be on a Monday?"
    ==========================
    http://www.streetsofboston.com
    http://blog.antonspaans.com
  • Options
    ScalaScala Registered Users Posts: 95 Big grins
    edited October 8, 2005
    flyingdutchie wrote:
    So, the http response-header tells the browser to use one encoding, the <meta> tag to use another... it is still a bug.
    -- Anton.
    Indeed. And the fix is long overdue.
    My smugmug site: www.majakorpi.net
  • Options
    {JT}{JT} Registered Users Posts: 1,016 Major grins
    edited October 14, 2005
    So this should be fixed, would love feedback from someone. I don't think previous entries are fixed yet - some might be though.
    Scala wrote:
    Indeed. And the fix is long overdue.
  • Options
    ScalaScala Registered Users Posts: 95 Big grins
    edited October 15, 2005
    {JT} wrote:
    So this should be fixed, would love feedback from someone. I don't think previous entries are fixed yet - some might be though.
    I tried entering some Finnish and Vietnamese text in comments and gallery description and it seems to work. Thanks!

    There's still the issue of the META tag implying UTF-8 but the browser selecting ISO-8859-1 but I guess that's how you want it. Either way there's something advanced going on because I thought ISO-8859-1 did not include all those strange Vietnamese characters :)

    Previous entries are still wrong.

    And the new "Updating..." notification does not go away if you delete the last (only) gallery comment. :):

    The end user experience is a lot better now internationally, thanks again!
    My smugmug site: www.majakorpi.net
  • Options
    rainforest1155rainforest1155 Registered Users Posts: 4,566 Major grins
    edited October 15, 2005
    {JT} wrote:
    So this should be fixed, would love feedback from someone. I don't think previous entries are fixed yet - some might be though.
    I successfully added a comment containing these characters:
    á Á à À â Â é É è È ê Ê í Í ì Ì î Î ó Ó ò Ò ô Ô ú Ú ù Ù û Û
    ä Ä ö Ö ü Ü ß

    Just the €uro isn't working, yet.

    The comments didn't change and are still wrong (here - ö = ö and here ü = ü)

    Sebastian
    Sebastian
    SmugMug Support Hero
  • Options
    ScalaScala Registered Users Posts: 95 Big grins
    edited October 15, 2005
    rainforest1155 wrote:
    Just the €uro isn't working, yet.
    I tried the €uro in a comment and it worked for me in IE6 and Firefox 1.0.7. But I agree that in theory it should not work since ISO-8859-1 doesn't have the euro character :)
    My smugmug site: www.majakorpi.net
  • Options
    {JT}{JT} Registered Users Posts: 1,016 Major grins
    edited October 15, 2005
    As longs as it works for now, I am happy :)
    Scala wrote:
    I tried the €uro in a comment and it worked for me in IE6 and Firefox 1.0.7. But I agree that in theory it should not work since ISO-8859-1 doesn't have the euro character :)
  • Options
    rainforest1155rainforest1155 Registered Users Posts: 4,566 Major grins
    edited October 15, 2005
    {JT} wrote:
    As longs as it works for now, I am happy :)
    Yeah, works now for me, but before it clearly didn't. Did some finetuning, didn't we? Nice! thumb.gif

    Sebastian
    Sebastian
    SmugMug Support Hero
  • Options
    flyingdutchieflyingdutchie Registered Users Posts: 1,286 Major grins
    edited October 15, 2005
    {JT} wrote:
    As longs as it works for now, I am happy :)
    {JT}, as a fellow software engineer, I'm just wondering...
    Why is it so tricky to send over UTF-8 instead of ISO-8859-1? Has it to do with the existing text-data in your databases?

    The http-response headers still say it is ISO-8859-1 and the <META> tags declare it to be UTF-8.. still a contradiction here.

    -- Anton
    I can't grasp the notion of time.

    When I hear the earth will melt into the sun,
    in two billion years,
    all I can think is:
        "Will that be on a Monday?"
    ==========================
    http://www.streetsofboston.com
    http://blog.antonspaans.com
  • Options
    {JT}{JT} Registered Users Posts: 1,016 Major grins
    edited October 15, 2005
    flyingdutchie wrote:
    Has it to do with the existing text-data in your databases?

    Bingo.
  • Options
    flyingdutchieflyingdutchie Registered Users Posts: 1,286 Major grins
    edited October 15, 2005
    {JT} wrote:
    Bingo.
    Aha... migration is the issue here...! :)

    But still, what about the contradicting http-response header and <META> tag? Shouldn't the <META> tag specify the ISO-8859-1 charset as well?
    I can't grasp the notion of time.

    When I hear the earth will melt into the sun,
    in two billion years,
    all I can think is:
        "Will that be on a Monday?"
    ==========================
    http://www.streetsofboston.com
    http://blog.antonspaans.com
  • Options
    {JT}{JT} Registered Users Posts: 1,016 Major grins
    edited October 15, 2005
    Maybe :) As I stated earlier - right now everything is working, so I am hesitant to change anyting :)
    flyingdutchie wrote:
    Aha... migration is the issue here...! :)

    But still, what about the contradicting http-response header and <META> tag? Shouldn't the <META> tag specify the ISO-8859-1 charset as well?
  • Options
    DmitrySDmitryS Registered Users Posts: 29 Big grins
    edited October 15, 2005
    {JT} wrote:
    Maybe :) As I stated earlier - right now everything is working, so I am hesitant to change anyting :)
    Russian characters works now too.
    However:
    1. When I change description of the gallery using "Edit" button, I can start new lines. If I go to the gallery desription, new line symbols can't be used.
    2. The limit for number of non-english characters in the description is very small, about 25 symbols. For enlish - it's much larger.
    Not a big deal, though
  • Options
    rainforest1155rainforest1155 Registered Users Posts: 4,566 Major grins
    edited October 16, 2005
    {JT} wrote:
    Maybe :) As I stated earlier - right now everything is working, so I am hesitant to change anyting :)
    I did some more testing and found out that it depends where the €uro symbol stands for it to work or not. headscratch.gif
    This doesn't work: 
    € á 
€ á

    but that does: 
    € á 
€ á

    EDIT: In fact I boiled it down to this - if there's at least one space before the €uro, it won't work. It only works when the €uro is the first character. If there are more €uros in the comment it depends on a €uro being the first character of the comment or not for all others!

    Works: 
    €
Test123 €
Another testline that is fine €

    but if you add a space before the first €uro resulting in it not being the first character of the comment it's broken: 
    € 
Test123 € 
Another testline that is fine €

    Any ideas on this?
    Sebastian
    Sebastian
    SmugMug Support Hero
  • Options
    {JT}{JT} Registered Users Posts: 1,016 Major grins
    edited October 16, 2005
    This happens to the euro only though, correct?

    I might an idea as to why, but have to check first ...
    rainforest1155 wrote:
    I did some more testing and found out that it depends where the €uro symbol stands for it to work or not. headscratch.gif
    This doesn't work: 
    € á 
€ á

    but that does: 
    € á 
€ á

    EDIT: In fact I boiled it down to this - if there's at least one space before the €uro, it won't work. It only works when the €uro is the first character. If there are more €uros in the comment it depends on a €uro being the first character of the comment or not for all others!

    Works: 
    €
Test123 €
Another testline that is fine €

    but if you add a space before the first €uro resulting in it not being the first character of the comment it's broken: 
    € 
Test123 € 
Another testline that is fine €

    Any ideas on this?
    Sebastian
Sign In or Register to comment.