ID:200429
 
Resolved
BYOND Version:N/A (Website Bug)
Operating System:Windows 7 Home Premium
Web Browser:Internet Explorer 9.0
Applies to:Website
Status: Resolved (web)

This issue has been resolved.
Looking in the Community Forums, I have noticed that in some of my posts, from before the site upgrade, apostrophes get this symbol added to them: Â' Also, an image that I embedded will not show up at all. I know that it still exists, as I just reuploaded it. So I get this no image box instead:

I believe that I have correctly used the HTML tags, so I don't know what could be causing this. I don't know the extent of this bug, such as whether it affects some other forums, or if it's only posts from before the update that are affected, so some testing may be needed.

Here is a post with the apostrophe Â' bug.

Here is the one with the no image bug.

I hope a fix is found soon, because this may have corrupted a lot of posts from the past.
Are these posts that were formerly in the old forum system?
The apostrophe you're using is actually a special character "right curved single quote" (0x92 / DEC 146 / U+2019 / ’)

' verse ’ verse ’ (wow, it won't even render the entity correctly)

It looks like it's been stored correctly in "raw" un-processed form (because you can quote the post in question and it renders properly). My guess is the new HTML sanitizer they're using isn't working 100% correctly.

<edit>
It could be an encoding issue. From my tests in Notepad++, if the curved quote is encoded as ANSI, then converted to UTF-8, and then viewed as ANSI, it appears in a form similar to what the site is showing. Not sure where the disconnect is in your process, though.
</edit>

<edit2>
It seems even the raw form is being messed with, as every time I edit my post, the character is being mis-re-encoded
</edit>

The second problem is a duplicate of this report.


Tom wrote:
Are these posts that were formerly in the old forum system?

Yes, I made both of these posts prior to the forum upgrade.

DarkCampainger wrote:
The second problem is a duplicate of this report.

I assume this is what you are referring to:

Flame Sage wrote:
Hmm.. seems to be working here, but it's not working in this thread:
http://www.byond.com/forum/?post=195510

This must mean that the bug is also affecting the BYOND Help forum as well.
Using "smart" quotes seems to have a few issues that I'm still puzzling out. I don't believe the problem is our sanitizer based on my tests--it's something else. I wouldn't be surprised if the new machine has some differences in the way it handles things that's causing the confusion, but we'll figure it out.
Something similar to this is still happening on "Super Coder's" page. If you scroll down on the shout box you should see Liam Howe's comment.
It may simply be residue from the old filter still. I'll reset that presently.
Alright, thanks for your quick response.
That one was the filter for sure.

I should note that there are two related issues that fall under this, and one isn't easily fixed. One issue is that some browsers are sending curly quotes in UTF-8, which I've addressed in the new code, but while our filter can try to take that out it may not do a perfect job for comments that saved that way. Overall I think the new filter will do better on that. The other issue, the one in the link you pointed out, was that our filter was doing some conversion of characters and entities that it shouldn't have.

Still, friends don't let friends use curly quotes in web content.