A Technical Report on Fish Audio Drama 3
Fish Audio Drama 3 is an open-source 35B MoE speech model for directed audio, multi-character scenes, revisions, edits, and streaming.
Key Takeaways
- Fish Audio Drama 3 is an open-source speech model that turns text into directed audio performances, including multi-character scenes, revisions, and streaming audio.
- It combines new capabilities—multi-turn revision, scene dubbing, editing by infilling, and streaming text input—in one model rather than separate systems.
- The model uses a 35B-parameter mixture-of-experts backbone that activates 3B parameters per token, paired with a hybrid linear-attention design and a codebook multi-token-prediction audio decoder.
- Training includes three-stage pretraining, instruction fine-tuning, and large-batch reinforcement learning using a single learned model of human preference as its only reward.
- It achieves higher single-stream inference speed than Fish Audio S2—271 frames per second versus 177 on one H200—despite having far more parameters, with weights and tools available on GitHub, Hugging Face, and SGLang-Omni.
Meta description: Fish Audio Drama 3 is an open-source speech model that turns text into directed performances, multi-character scenes, revisions, and streaming audio.
Tags: fish audio, speech generation, open-source, audio drama, AI audio
Featured image: abstract friendly illustration of text becoming a directed audio scene with waveform, microphones, and character silhouettes; no real people, no logos, no text.
Fish Audio Drama 3 is an open-source speech generation model for creators, developers, and researchers who want text to become directed audio performances. It can clone voices from multiple references, follow natural-language direction, revise a take when the next turn gives a note, voice multi-character scenes, edit existing speech by infilling, and synthesize long-form and streaming audio.
New capabilities in one model
The model does not split these jobs into separate systems. Multi-turn revision, scene dubbing, editing by infilling, and streaming text input are new in Drama 3, and they are all in one model. That makes the release more than a text-to-speech upgrade: it changes how users can guide a performance over several turns.
For readers, the practical meaning is that a workflow can remain conversational. A user can ask for a different take after hearing a line, add another character, or request a change while generating longer scenes. Scene dubbing covers multi-character scenes, while editing by infilling lets the model change parts of existing speech.
Architecture and training
Drama 3 is built on a 35B-parameter mixture-of-experts backbone that activates 3B parameters per token. In this style of model, the total parameter count is large, but only a subset is used for each token. The system pairs the hybrid linear-attention backbone with a new codebook multi-token-prediction audio decoder. This decoder predicts the acoustic codebooks of each frame in dedicated layers that attend over the entire sequence. Acoustic codebooks are compact representations of sound.
Drama 3 is trained with a three-stage pre-training recipe, instruction fine-tuning, and large-batch reinforcement learning. The reinforcement learning step uses a single learned model of human preference as its only reward.
Speed, access, and next steps
The inference engine decodes a single stream at 271 frames per second on one H200, compared with 177 frames per second for Fish Audio S2, although Drama 3 has more than seven times as many parameters. The comparison places speed against size: Drama 3 is larger, but the engine reaches a higher single-stream frame rate.
The model weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/drama-3). Inference is available through SGLang-Omni (https://github.com/sgl-project/sglang-omni). Readers who want to try directing performances can visit https://fish.audio.

Comments (0)