How does Messages suggest replies?

Question

Quite recently, Android Messages (com.google.android.apps.messaging) started providing reply suggestions to SMS messages I receive. The suggestions are (usually) relevant to the content of the received message: e.g., somebody texts me with a "Can I call you tomorrow?" and Messages suggest things like "Sure" or "Sounds good".

How exactly does it do that? I would be surprised if this kind of AI system could run on my phone. But does that mean that the messages are processed somewhere else? If yes, where and how?

I'm wondering because I've never given any explicit authorisation for my messages to go anywhere, and in fact, I'm using AFWAll+ to prevent Messages (and many other apps) from accessing the Internet at all.

UPDATE: As a test, I used the firewall to also block anything Google Play-related from accessing the Internet, which was the following three entries:

Google Play services, Google Services Framework, Google Account Manager, Google Backup Transport

Google Play services for Instant Apps

Google Play Store

Yet, the suggestions showed up again.

System details:

Device: Samsung Galaxy S5 Duos (SM-G900FD)

OS: LineageOS 14.1

Android version: 7.1.2

The phone is rooted with SuperSU.

— Ratler | Sources

Answer

After some research, it is, as I thought, Tensorflow (or to be precise, Tensorflow Lite). You can read the following articles posted by the Google Research team:

On-Device Machine Intelligence -- Smart Reply on Android Wear

On-Device Conversational Modeling with TensorFlow Lite -- Smart Reply on mobile devices

Relevant quotes:

Android Wear 2.0 which featured the first "on-device" machine learning technology for smart messaging. This enabled cloud-based technologies like Smart Reply, previously available in Gmail, Inbox and Allo, to be used directly within any application for the first time, including third-party messaging apps, without ever having to connect to the cloud.

Today, we announce TensorFlow Lite, TensorFlowâ€™s lightweight solution for mobile and embedded devices. This framework is optimized for low-latency inference of machine learning models, with a focus on small memory footprint and fast performance.

...

This model generates reply suggestions to input conversational chat messages, with efficient inference that can be easily plugged in to your chat application to power on-device conversational intelligence.

They've also posted a source code link to TF lite's Smart Reply feature, as well as a demo app for testing out Smart Reply.

— SSS | Sources

Topics