Python 2.7 Build On Sublime Text 3 Doesn't Print The '\ufffd' Character

The problem. I'm using Python 2.7 build on Sublime Text 3 and have an issue with printing out. In some cases I get a pretty confusing output for '\uFFFD' - the 'REPLACEMENT CHARACT

Solution 1:

I've reproduced your problem and I've found a solution that works on my platform anyhow: Remove the -u flag from your cmd build config option.

I'm not 100% sure why that works, but it seems to be a poor interaction resulting from the console interpreting an unbuffered stream of data containing multi-byte characters. Here's what I've found:

• The -u option switches Python's output to unbuffered
• This problem is not at all specific to the replacement character. I've gotten similar behaviour with other characters like "あ" (U+3042).
• Similar bad results happen with other encodings. Setting "env": {"PYTHONIOENCODING": "utf-16be"} results in print u'\u3042' outputting 0B.

That last example with the encoding set to UTF-16BE illustrates what I think is going on. The console is receiving one byte at a time because the output is unbuffered. So it receives the 0x30 byte first. The console then determines this is not valid UTF-16BE and decides instead to fallback to ASCII and thus outputs 0. It of courses receives the next byte right after and follows the same logic to output B.

With the UTF-8 encoding, the console receives bytes that can't possibly be interpreted as ASCII, so I believe the console is doing a slightly better job at properly interpreting the unbuffered stream, but it is still running into the difficulties that your question points out.

Solution 2:

Edit-1 - Using UTF8 with BOM

Seems like BOM becomes important in case of windows. So you need to use below type build config

{   
    "cmd": ["F:\\Python27-14\\python", "-u", "$file"],
    "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
    "selector": "source.python",
    "env": {
        "PYTHONIOENCODING": "utf_8_sig"
    },
}

After that it works correctly for me on windows also

Original Answer

I checked the issue and I didn't face the same on Python 2.7 with Sublime text. The only change being I had to add # -*- coding: utf-8 -*- to the top of the file. Which seems the missing part in this question

# -*- coding: utf-8 -*-printu'\u0061'# should be 'a'printu'\ufffd'# should be '�' - the 'REPLACEMENT CHARACTER'

After that the reversal has no impact

You can see more details about this required header on

Why declare unicode by string in python?

Below is summary of the above link

When you specify # -*- coding: utf-8 -*-, you're telling Python the source file you've saved is utf-8. The default for Python 2 is ASCII (for Python 3 it's utf-8). This just affects how the interpreter reads the characters in the file.