text-to-binary module returns invalid results for non-ASCII text
#1082
Comments
|
Implementation something like this textToUtf8Bin(input)
// '11100110 10010110 10000111'
utf8BinToText('11100110 10010110 10000111')
// '文'
function textToUtf8Bin(text) {
return [...new TextEncoder().encode(text)]
.map((x) => x.toString(2).padStart(8, '0'))
.join(' ')
}
function utf8BinToText(bin) {
bin = bin.replaceAll(/\s+/g, '')
assert(!/[^01]/.test(bin))
assert(bin.length % 8 === 0)
return new TextDecoder().decode(
Uint8Array.from({ length: bin.length / 8 }, (_, i) => parseInt(
bin.slice(i * 8, (i + 1) * 8),
2,
)),
)
}
for (const text of ['ascii', '文字', '💩']) {
const converted = textToUtf8Bin(text)
assert(utf8BinToText(converted) === text, utf8BinToText(converted))
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
Using https://it-tools.tech/text-to-binary, supplying non-ASCII text gives invalid results.
To be fair the tool's title is "Text to ASCII binary", but the description "Convert text to its ASCII binary representation" implies "ASCII" applies to the binary output (i.e.
"0"and"1"are both ASCII chars), not the text input.What happened?
Example: Converting
文Expected
Either one of these:
文, padded to 8-bit bytes (11100110 10010110 10000111)11100110 10010110 10000111gives文Actual
110010110000111(i.e. 25991, which is文's code point, represented as a non-padded 15-bit binary string)110010110000111back from binary -> text gives error "Binary should be a valid ASCII binary string with multiples of 8 bits".11100110 10010110 10000111(the correct UTF-8 representation of文) from binary -> text givesæ��System information
Win 11, Chrome Version 124.0.6367.158 (Official Build) (64-bit)
Where did you encounter the bug?
Public app (it-tools.tech)
The text was updated successfully, but these errors were encountered: