Talking to an AI assistant usually means sending your voice to a server. Google’s AI Edge team is pushing in the other direction. In a developer blog post dated November 10, 2026, it introduced on-device speech recognition (ASR) and text-to-speech (TTS) in Google AI Edge Gallery, so small language models can listen and talk entirely offline.
What Google announced
The release adds a local voice pipeline to the Google AI Edge Gallery app. Google says it pairs lightweight speech models with on-device small language models such as Gemini Nano and Gemma 4, letting developers build apps that stream live transcription as a person speaks and synthesize spoken replies locally, with low latency and a small memory footprint.
There are two tracks. On Android, ML Kit uses Google’s latest system speech models, integrated into the operating system, so they add 0 MB to an app’s APK. For cross-platform work, MediaPipe Tasks and LiteRT support open-weight models packaged as .litertlm bundles, including Parakeet-TDT 0.6B for recognition and PocketTTS 100M, Kokoro-82M and Phonon 31M for speech output.
The numbers Google shares
- Recognition: ML Kit’s GenAI Speech Recognition API covers 100+ languages, reaches 8.0 to 9.0 percent word error rate on FLEURS, and transcribes at about 10x real time (roughly 300 ms per 3 seconds of speech). Google also cites 35 to 44 percent shorter end-to-end speech translation waits.
- Speech output: the ML Kit TTS API uses a compact few-step diffusion model on the NPU. It begins speaking in about 100 ms, offers 8 voices across 21 locales, runs at up to 50x real time, and synthesizes a 27-second paragraph in about 500 ms. Google says it cuts active device power by more than 50 percent compared with traditional autoregressive TTS.
- Lightweight option: Phonon 31M targets sub-100 ms time to first audio on most devices and platforms.
XR Language Teacher and voice chat
To show the pieces working together, Google built an XR Language Teacher showcase. A user points the phone camera at a menu or street sign and asks aloud what it is and how to ask for one. Streaming ASR passes the question, along with the camera frame, to Gemini Nano on Android or Gemma 4 on other platforms. A structured output schema tags foreign phrases so the TTS system can switch voices and accents mid-sentence, while a vocabulary card appears on screen. Everything runs on the phone with no network. Google says the same design can extend to hands-free smart glasses with inference on a paired phone.
The main AI Chat feature also gains voice dictation and read-aloud playback. Settings for ASR and TTS pipelines, preset voices and language packs live in Model Management. A floating TTS bar highlights each spoken sentence and lets users tap to jump. The open-source reference code shows how to strip Markdown, skip code blocks and tables, and schedule synthesis sentence by sentence so speech does not compete with token generation for compute and memory.
How developers can use it
Google describes a modular design: separate, lightweight voice models cascaded with a language model, rather than relying only on multimodal models that take audio directly. The stated benefits are latency tuning and flexibility. ML Kit is the recommended path for Android, with models managed through AICore and automatic use of NPU and TPU acceleration where available. MediaPipe Tasks offers turnkey ASR and TTS across iOS, macOS, Windows, Linux and the web, handling audio framing, voice activity detection and decoding. LiteRT sits underneath both and is open to custom models such as Whisper, Moonshine or Qwen3-ASR. The app and code are available now on Android, iOS and GitHub.
What it means
Running voice locally changes the trade-offs for product teams. Audio no longer has to leave the device, features keep working on a subway or abroad without roaming, and there is no per-request cloud cost for speech. Hands-free wearables are an obvious fit. The caveats are practical: quality and speed will vary with device hardware, and the figures above are Google’s own claims rather than independent tests.
Conclusion
With ASR, TTS and a small language model all running on the phone, Google is giving developers a complete offline voice stack they can inspect and adapt. It is worth watching as assistants move onto glasses and other wearables.