Print Special Characters as Backslash Escapes – Python3 One-liner with Unicode Support

Let’s find that weird or invisible character that breaks your code

$ echo "Hello in Finnish: Päivää"|python3 -c 'import sys;; print(s.encode("unicode_escape"));'
b'Hello in Finnish: P\\xe4iv\\xe4\\xe4\\n'


$ echo |cat
No command 'echo ' found [..]

How can it be?

$ echo 'echo |cat'|python3 -c 'import sys;; print(s.encode("unicode_escape"));'

As you can see, there is a non-space character \\xa0 before the pipe. But it sure looks a lot like a space.
To create this unwanted character for testing, you can press AltGr-space on Finnish keyboard (‘setxkbmap fi’).
Same technique can be used for finding invisible special characters from source code files.
Many other commands can escape special characters too, such as od, cat -T, vim and less. Python version 3 handles UTF-8 nicely, and also prints familiar backslash escape sequences. And if you have further requirements, it’s easy to code them with Python. For example, how would you only show lines with illegal characters from a source code file? What about line numbers?

Posted in Uncategorized | Tagged , , , , , , , , , , , , | Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *