Receiving Chinese blocks of texts in the Messaging app

Question

My wife is getting some weird text messages as replies in middle of conversations with her friends. The messages are blocks of Chinese characters. She gets these in the stock messaging app from friends that use Samsung phones and iPhones. She is using a Nexus 5X on AT&T. (N.B. Other users have asked the same question about the Samsung Galaxy S7, so it's not a phone-specific issue.)

Here is a screencap from today from an iPhone user, and translating the text does not make sense (though I did image translation): :

^{(Click to see the image in full size)}

Here is one from a Samsung phone and from their end:

^{(Click to see the image in full size)}

Does anyone know why this is happening? Is it some weird thing when someone with a weird texting app replies with some emoji/GIF/image?

— Zlatty | Sources

Answer

Cause

It's likely caused by either/both character encoding incompatibility and/or wrong character encoding, resulting in mojibake (garbled text):

Character encoding incompatibility

SMS, or text message, in general supports either text-only (e.g. GSM 03.38), or Unicode (e.g. UTF-8, UTF-16, UCS-2). Emoji, a character that resembles an image (not to be confused with emoticon, "a pictorial representation of a facial expression using punctuation marks, numbers and letters"), is supported in Unicode (UTF-16), but not in GSM 03.38.

Wrong character encoding issue

For some reasons, the original character encoding is wrongly interpreted somewhere (e.g. by app, or by text provider), resulting in different character encoding. While in some cases it doesn't affect the text, other cases may result in totally garbled text (refer to Microsoft Windows "Bush hid the facts" bug).

In this case, it's likely that one of the encoding that is used in SMS, GSM. 03.38, which uses 7-bit per character, is interpreted as UTF-16, which uses 16-bit per character due to emoji.

Solution

Recipients

Try changing the messaging app. If it doesn't fix the issue, then it's probably caused by the text provider itself. However, you can still try to recover the text (e.g. using online service such as http://string-functions.com/encodedecode.aspx): set encoding to UTF-16 and decoding to UTF-8.

Senders

Avoid emoji at all, or make the messaging apps send it as text-only (e.g. in Google Messenger, there's Simple characters only in Advanced settings to convert special characters in SMS messages when sending it)

Receiving Chinese blocks of texts in the Messaging app

Cause

Solution

Topics