Tool Reviews

This Closer Look at Phonon Text-to-Speech Highlights Its Launch of High-Quality On-Device Speech That Tops Benchmarks.

Phonon is a small open text-to-speech system that runs offline on devices, supports five languages and voice cloning, and ranks highly on benchmarks.

This Closer Look at Phonon Text-to-Speech Highlights Its Launch of High-Quality On-Device Speech That Tops Benchmarks.

Key Takeaways

  • Phonon is a small, on-device text-to-speech system that keeps typed text local, targeting privacy-focused apps, offline use, and commercial-friendly voice AI.
  • It offers two main models: Phonon 31m, an English Apache 2.0 model for free commercial use, and Phonon 40m, a five-language model under a non-commercial license with user-based commercial pricing.
  • Built on the Pocket TTS lineage and flow matching, Phonon delivers high-quality speech in compact 65–75 MB models, supports voice cloning/design, handles tricky text like dates and codes, and runs on phones, laptops, browsers, and small devices.
  • On Hugging Face’s Open TTS Leaderboard, Phonon ranks first in German and Spanish intelligibility, places in the top three across evaluated languages, and leads CPU streaming time-to-first-audio despite being far smaller than many rivals.
  • Developers can use Phonon through the new ptts inference framework, which supports CPU, GPU, and NPU backends, or via Transformers v5.21 for PyTorch, batched GPU generation, and model integration.

Meta Description: Phonon 31m and Phonon 40m run offline text-to-speech on phones, laptops and browsers, with Apache 2.0 for the 31m model and strong leaderboard scores.

This Closer Look at Phonon Text-to-Speech Highlights Its Launch of High-Quality On-Device Speech That Tops Benchmarks.

Tags: Phonon, offline text-to-speech, Hugging Face, Gradium, voice AI

Featured image: abstract, friendly illustration of a small speech waveform running on a phone and laptop, with no real people, no logos and no text.

Two weeks ago, the team opened Phonon, a small offline text-to-speech model that runs on a user's own device without a server or network connection. The release is aimed at developers, app builders and anyone who wants spoken output from typed text while keeping that text local. It is also showing up as a compact performer on Hugging Face's Open TTS Leaderboard, where it reaches the top of several language comparisons and leads a CPU streaming time-to-first-audio benchmark.

A small model that keeps text on the device

Most voice AI runs in the cloud, sending every sentence to a server before returning audio. Phonon takes the other path: the model runs locally, and the typed text stays with the user. The weights are open. Phonon 31m speaks English and is licensed under Apache 2.0, so it can be used for free, including in commercial products. The other models are licensed CC BY-NC, which keeps them free for non-commercial projects; if a commercial product uses those models, pricing is based on the number of users rather than the amount of speech generated.

The team announced Phonon in April and spent the next six months refining it with partners and early users. That feedback helped shape the launch: the models got smaller, more languages were added, and support for custom voices arrived through voice cloning and voice design. Phonon also inherits work from the Pocket TTS lineage and uses flow matching to generate audio, a technique the team describes as very different from the modeling methods used in other text-to-speech systems. That approach helped bring higher quality into a very small model, but it also made training harder: the process was initially numerically unstable and needed several tricks to produce quality results across all five languages.

Phonon comes in two sizes, at about 65 MB and 75 MB. Together they speak English, French, German, Spanish and Portuguese. More than 100 built-in voices are included, and users can create their own voices in Gradium Studio. The model also handles elements that often trip up text-to-speech systems, such as phone numbers, dates and codes. It runs on phones, laptops, single-board computers and in a browser tab. Phonon 31m covers English under Apache 2.0, while Phonon 40m covers all five languages under a non-commercial license.

Demos show the model outside the data center

The launch included demos from partners and the team. One ran on an Apple Watch with Wi-Fi turned off, guiding a coffee brew step by step. Another ran on an iPhone with no signal, counting exercise reps. A robot named Lamp, from Autonomous, wore a sorting hat and spoke in character. In a browser, an offline dino runner turned into a radio show hosted by Rex. Other projects included a sci-fi game voiced on the player's own machine and Proust read aloud page by page.

The Hugging Face team built the Transformers integration, so developers could use Phonon in Transformers from day one. Early developer work followed: an ExecuTorch conversion runs Phonon on a visitor's GPU in the browser, Micdrop brings it to Node.js apps, and Spaces such as smallest-s2s build on it. Phonon was downloaded more than 200,000 times in its first two weeks and remains trending on Hugging Face.

Leaderboard results put a small model in view

The Open TTS Leaderboard is a very new Hugging Face project. With more than 9,000 TTS models on the Hugging Face Hub, it gives developers one place to compare open models and listen to them side by side. Across 23 languages, the leaderboard evaluates intelligibility through the word error rate of outputs, speed through inverse real-time factor and time-to-first audio, and naturalness through UTMOS for English and KOMOS for Korean. It tests fixed-voice and voice-cloning settings, adding a speaker-similarity metric for cloning. A Listen tab lets users compare outputs and vote for the voices they like most.

Phonon's results are strongest in intelligibility. It ranks first in German and Spanish, and it places in the top three in every language evaluated, even when compared with models up to 100 times its size. Speed is another standout: the models top the CPU streaming benchmark on time-to-first audio, which matters for on-device responses where the first spoken sound must arrive quickly.

Two developer paths: ptts and Transformers

The team built a new inference framework, ptts, for Phonon. Its goal is to get the best performance across a wide variety of devices, and it supports more than 10 backends: CPU backends including Intel, Arm and WebAssembly; GPU backends including CUDA, Vulkan, Metal and WebGPU; and NPU backends including Apple Core ML and Snapdragon®.

Phonon is also available in Transformers v5.21 as its reference PyTorch implementation. The two paths serve different jobs. ptts runs a single stream on device, while Transformers targets GPU throughput with batched generation and full-graph torch.compile, for example when synthesizing speech datasets at scale. Because the Transformers implementation is native PyTorch, developers can use it to integrate Phonon into another stack or study the model itself.

The model's structure is close to language-model workflows. It belongs to the Pocket TTS lineage, so the team first added Pocket TTS to Transformers, then built Phonon on top as a modular model with about 200 lines describing only what differs. Phonon generates audio latents autoregressively: each step outputs a latent that is fed back as the next input, and the training loss is computed on those latents. In this design, latents play the role that text tokens play in a language model, while the codec that maps audio to latents acts like a tokenizer. Transformers ships that codec as an audio tokenizer, held by the processor next to the text tokenizer.

A voice, in this implementation, is not a recording but a tensor of speaker latents. Phonon never encodes audio directly with its audio tokenizer, which ships decoder-only; voices are provided precomputed in gradium/phonon-voices. The prompt layout, which includes voice positions, text and a token that opens audio, ships with the checkpoint as a chat template, similar to other multimodal models in Transformers. A user writes a standard conversation with an empty audio item where the voice goes, keeping code the same across checkpoints. The processor expands the voice placeholder to one position per frame, normalizes and tokenizes the text, and batches the inputs. When generate returns latents, the processor decodes them into audio; the documentation covers more details.

What people can do with Phonon next

Phonon can speak with any voice. A user can clone a voice from a short recording with the speaker's consent, or describe a new voice in plain text in Gradium Studio. After that, the voice file can be downloaded, loaded in Phonon, and used fully on the device.

The team also trains custom Phonon models with design partners. If a product has a tight memory budget, the model can be made smaller. If Korean, Arabic or Japanese is needed, the team can train for those languages. If a product has its own vocabulary, it can be fine-tuned for that vocabulary. The team thanked Hugging Face and everyone who tried Phonon in its first two weeks. It says this is its first open release and invites users to try it and share what they build.

For readers, the practical choice is fairly clear: Phonon 31m offers an English Apache 2.0 route for commercial on-device work, while Phonon 40m gives a non-commercial path across English, French, German, Spanish and Portuguese. The demos, downloads and leaderboard placement show the model's main pitch: small, local, fast and competitive for its size.

Comments (0)

  1. No comments yet. Be the first to share what worked for you.

Leave a comment

Comments are reviewed before they appear. Your email address is not published.