What is Character Encoding Detector?
This character encoding checker answers two questions: which encoding is a file saved in, and which characters in a piece of text will cause trouble? Open or drop a file and it reads the raw bytes to detect the encoding: UTF-8 with or without a byte order mark, UTF-16 and UTF-32, Windows-1252 and ISO-8859-1, Cyrillic and Greek code pages, and Japanese, Chinese and Korean encodings such as Shift_JIS, GBK, Big5 and EUC-KR. It also tells you when a file is not text at all.
Paste text instead and you get a table of every non-ASCII character with its code point, UTF-8 bytes, type and first position, with invisible troublemakers such as no-break spaces, zero-width spaces and byte order marks named and highlighted. If the text contains mojibake, such as é instead of é or ’ instead of ’, the tool repairs it, and it can make an ASCII-safe copy with straight quotes and plain spaces.
How to use Character Encoding Detector
- To check a file, click Open file or drop it on the tool; to check text, paste it into the box.
- Read the result: the detected encoding with a confidence level and the reason, or for pasted text, which encodings can store it.
- Review the character table for invisible or unexpected characters, with their line and column.
- Use Save as UTF-8 to convert a legacy file, the repaired text to fix mojibake, or the ASCII-safe copy for systems that only accept ASCII.
Key features
- Detects UTF-8, UTF-16, UTF-32, Windows-1252, Cyrillic, Greek and CJK encodings from file bytes
- Confidence level and the reason behind every detection
- Table of non-ASCII and invisible characters with code points and UTF-8 bytes
- Repairs mojibake such as é and ’ back to é and ’
- Saves legacy files as UTF-8 and makes ASCII-safe copies of text
- Runs in your browser; files are never uploaded
Common use cases
- Finding out why a CSV or text file shows garbled characters
- Converting legacy Windows-1252 or Shift_JIS files to UTF-8
- Finding no-break spaces, zero-width spaces and curly quotes pasted from Word or Google Docs
- Checking that identifiers, file names and headers are plain ASCII
- Repairing mojibake such as é in database exports and emails
Technical details & standards
File detection works on the bytes, in order of certainty. A byte order mark (EF BB BF, FF FE, FE FF or a UTF-32 mark) identifies a Unicode encoding outright. A file with only bytes below 0x80 is plain ASCII. Otherwise the bytes are checked against the UTF-8 rules; legacy encodings almost never produce valid multi-byte UTF-8 by chance, so a file that passes is UTF-8, and one that fails is reported with the offset of the first invalid byte. UTF-16 without a BOM is recognized from its pattern of 00 bytes, while NUL and control bytes, together with signatures such as PNG, PDF and ZIP, mark binary files.
For legacy files the tool decodes the bytes with each candidate encoding and scores how natural the result looks, using the most common characters of each language, then picks the best match. Single-byte Western text cannot be told apart from ISO-8859-1 by its bytes alone, and very short samples can be ambiguous, so every result shows a confidence level. Pasted text has already been decoded by the browser, so for text the tool reports which encodings can represent it (ASCII, Latin-1, Windows-1252 or only Unicode) rather than how it was saved.
Check and convert file encoding on the command line
On macOS and Linux, the file command guesses a file's encoding and iconv converts it. Here a Latin-1 file is detected and rewritten as UTF-8:
Terminal
file -I notes.txt # macOS: notes.txt: text/plain; charset=iso-8859-1
file -i notes.txt # Linux uses a lowercase -i
iconv -f WINDOWS-1252 -t UTF-8 notes.txt > notes-utf8.txt
file -I notes-utf8.txt # text/plain; charset=utf-8file reports ISO-8859-1 for most single-byte Western text. Converting from WINDOWS-1252 is usually the safer choice, because it also maps curly quotes and the euro sign correctly.
Frequently asked questions
What is the difference between ASCII and UTF-8?
ASCII is a 7-bit encoding of 128 characters. UTF-8 encodes every Unicode character using 1 to 4 bytes, and its first 128 characters are identical to ASCII, so any ASCII text is also valid UTF-8.
Why does my text show strange characters like é or ’?
That is mojibake: text saved as UTF-8 was read as Windows-1252 or ISO-8859-1. For example é (bytes C3 A9) shows as é. Paste the garbled text here to repair it, and make sure files are read and served as UTF-8, with charset=utf-8 in the Content-Type header or a <meta charset="utf-8"> tag.
Can this tool detect the encoding of a file?
Yes. Open or drop the file and the encoding is detected from its raw bytes, with a confidence level. Byte order marks, ASCII and valid UTF-8 are certain; legacy single-byte and CJK encodings are detected statistically, which is reliable for normal text but can be ambiguous for very short files.
How do I find hidden non-ASCII characters?
Paste the text and look at the character table: invisible characters such as the no-break space (U+00A0), zero-width space (U+200B) and byte order mark (U+FEFF) are shown as labeled tags with their line and column. In many editors you can also search with the regex [^\x00-\x7F] to highlight them.
What does Save as UTF-8 do?
It writes the decoded text out again as UTF-8 without a byte order mark, the encoding the web, JSON and most modern tools expect. The original file is not changed; you download a converted copy.
Is my data safe?
Processing happens locally in your browser. Tool input is not sent to CodeUnpack servers.
Does it work offline?
Once the page has loaded, this tool can process input without an internet connection.