Tyger: an open turn detection model for voice agents

Two things went out this week. Tyger is a model that listens to both sides of a conversation and decides, every 80 ms, whether your agent should answer, stop talking, or say "mm-hm". Tyger-S2S is a complete voice agent built on it, the kind you talk to in a browser the way you talk on a phone call, and it runs on a single GPU.

It is worth being precise about what that makes it. A speech recognizer transcribes, a language model writes, a speech synthesizer speaks. Not one of them decides anything about the conversation itself, so in most stacks that job falls to a silence timer somebody picked by hand. Tyger is the piece that decides. It generates nothing at all: every 80 ms it reads both audio streams and returns a decision about whose turn it is.

Both are open. The model is on Hugging Face under OpenMDW 1.1, the agent is on GitHub under Apache 2.0, and there is a Docker image if you would rather not install anything.

If you only want to run it, skip to how to try it today. If you are weighing this against something you already use, the comparison is the section you want, and it names names.

What problem does turn-taking actually solve?#

Most voice agents do not sound robotic because their voice is bad. They sound robotic because they take turns badly. You finish a sentence and nothing happens for most of a second. You start to correct yourself and the agent plows on. You say "mm-hm" to show you are following and the agent treats it as your turn and stops.

Almost all of that comes from one design shortcut. Voice activity detection tells the app whether sound is present, and a fixed timer decides that silence longer than some threshold means you are done. That pairing is what most stacks mean by turn detection, and it is the part a trained model replaces. Pick 500 ms and the agent is slow on every single turn. Pick 200 ms and it interrupts you every time you pause to think. There is no value that is right, because the timer is answering the wrong question: it measures silence, when what the app needs to know is whether a thought is finished.

People do not use a timer. A conversation hands the floor over in roughly 200 ms, which is faster than either side could react to silence, because both sides are predicting the handover before it arrives. That prediction is the thing a timer cannot do, and it is what we trained a model to do instead. We wrote about why this breaks most voice agents in more detail, including the five ways a barge-in goes wrong.

What is Tyger?#

Tyger is NVIDIA's Nemotron 3.5 ASR Streaming 0.6B, unchanged and frozen, with a small recurrent decision model trained on top of it. The recognizer gives you the transcript. The decision model gives you the turns, and nothing else: there are no words and no audio on its output, only four states and four scores. Both halves load from one model.safetensors.

Two details in that table matter more than the parameter count.

The first is that it hears both streams. The agent's own voice goes in alongside the user's microphone, which is why the agent's echo in a laptop mic does not get mistaken for the user starting to speak. Any system that listens to one channel has to solve that with audio tricks instead.

The second is no lookahead. The controller never waits for future audio to score the current frame, so there is no built-in delay to pay back later. It is also genuinely small: 14M parameters and 9 KiB of state per conversation means the turn logic is a rounding error next to the language model and the voice.

What the model tells your app#

Every 80 ms frame, Tyger emits a turn state (listen, hold, speak or backchannel) plus four scores: an upcoming speaking onset, a barge-in, a backchannel opportunity and an endpoint. Your app turns three of those into actions.

Elsewhere these go by other names. The endpoint is what gets called end of turn detection, or endpointing. The other two are barge-in detection and backchannel detection, and they are usually separate pieces of machinery, if they exist at all. Tyger produces all three out of the same pass over the same audio.

Around those sit guards that exist because raw scores are not enough in a real room: six warm-up frames, at least 350 ms of voiced audio above -50 dB, 1.2 s between backchannels, and an endpoint after 1.2 s of silence even when the score never crosses. The state fed back in is always what the agent is really doing, listening or speaking, never the model's own guess about it. That last one sounds pedantic and it is the difference between a stable conversation and a feedback loop.

How well does it work?#

On the Open Yap 1K test set, the controller gets a 0.9436 macro-F1 across the four turn states and a 0.9839 ROC-AUC on endpoints, and it fired no false barge-ins at all across 4.72 hours of clean speech. Endpoint latency, measured from the end of the user's turn, has a median of 252.5 ms.

Read that table honestly, because we publish it that way. The latency column counts only endpoints that fired, so it has to be read next to the early cut-in column, which is the share of turns where an endpoint fired before the turn was actually over. On the public test set that is nearly a third of turns at the frozen threshold, and it is the number we are working on. These are offline replay diagnostics at thresholds frozen on development data before test scoring, not a live benchmark of a running agent, and they come from one training seed. The evaluation notes in the repo spell out every definition and caveat, and metrics.json has every number we measured.

The training data is Open Yap 1K: 1,395 training, 107 development and 100 test conversations with the two speakers on separate channels. The labels come from when each side actually speaks rather than from human annotators, which is the only reason a dataset this size is enough. The probes borrow from Full-Duplex-Bench, whose authors did the work of defining what to even measure here.

How does this compare with the other turn detectors?#

Turn detection has a small, good field around it now, and the honest summary is that Tyger answers a wider question than the others on a narrower set of languages. Every project below scores the user. Tyger is the one that also takes the agent's own output channel as an input, which is what lets it score an interruption while the agent is mid-sentence.

Three differences are worth being concrete about, because they decide which one you want.

What question gets answered. Most of these answer "has the user finished". That is the endpoint, and it is the one that fires in silence. Barge-in and backchannels happen while somebody is actually talking, so a detector that only wakes up in the gaps cannot see them. Tyger scores every 80 ms frame whether anyone is speaking or not, which costs more compute and is the reason it can emit all three.

What it costs you. Smart Turn is 8 MB quantized and runs on a CPU in about 10 ms. Tyger is 2.61 GB in FP32 and wants a GPU. If you already run a separate speech recognizer, that comparison is unfair in our favour, because Tyger replaces it: the transcript and the turn decisions come out of one model. If you do not, and you only want a turn detector to sit beside a stack you already like, Smart Turn is the smaller, saner choice and we would tell you so.

Languages. Smart Turn covers 23, LiveKit's audio model covers 14, Tyger covers one. That is the clearest thing we are behind on, and it is a data problem rather than a modelling one.

We have not run any of these head to head. Everyone publishes on their own test sets with their own definitions of an early cut-in, and until there is a shared one, a league table would be marketing rather than measurement. Treat the table above as a map of what each thing is for.

What is Tyger-S2S?#

Tyger-S2S is the open source voice agent we built to prove the model works in a real conversation, and it is the fastest way to hear it. It is a full duplex voice agent, meaning it never stops listening while it talks, and it has two halves on one GPU. EARS is Tyger: it transcribes you and decides when to act. MOUTH writes the reply with Gemma 4 E4B and speaks it with VoxCPM2 in a cloned voice.

The parts that make it feel like a call are small and specific. MOUTH speaks one sentence at a time, so you hear the beginning before the rest is written. Before it uses a tool it says a short line first, so a two second lookup is not two seconds of silence. If you interrupt, it stops immediately and keeps in its memory only the part you actually heard, which matters more than it sounds: an agent that remembers the sentence it never finished saying will contradict itself three turns later.

It also knows when it is out of its depth. With an optional expert model configured, hard questions and anything needing live facts get handed off to a bigger online model, while ordinary turns stay local and cost nothing extra. Longer jobs run in the background and the agent reports back at the next quiet moment.

What does it cost to run?#

The default configuration needs about 28 GB of GPU memory for four concurrent conversations, so a 32 GB card covers it. That is Gemma 4 E4B in FP8 at roughly 15 GB, VoxCPM2 at roughly 9 GB, and Tyger itself at roughly 4 GB. We have tested an RTX 5090, an RTX PRO 5000 and an RTX PRO 4500.

Beyond the card you need an NVIDIA driver for CUDA 13, about 60 GB of free disk of which the models take 25 GB, and a Hugging Face token. The first start downloads and compiles for about ten minutes, and after that it comes up in roughly two. Response time from when you stop speaking to when the voice starts is about 0.7 s on an RTX 5090, and roughly 0.5 s of that is the endpoint guard making sure you are really finished, so better tuned thresholds should bring it down.

Four conversations on one retail card is a different cost shape from a per-minute API. If you are working out which side of that line you are on, our note on self-hosted TTS or an API does the arithmetic, and why Gepard runs on vLLM explains why we keep insisting on stock serving stacks.

Where it is early#

We would rather you hear this from us than find it out on a demo call.

• English only. The controller was trained on English conversations, and multilingual turn-taking needs channel separated conversational data that mostly does not exist yet.
• The live defaults are not the evaluated ones. Barge-in and backchannel ship at 0.50 and 0.40, looser than the 0.93 and 0.80 used for the published numbers, and those looser values have no measurements of their own. Tune them with the sliders on the page, then copy what you like into the config.
• Early endpoints are the weak spot. Nearly a third of turns on the public test set get an endpoint before the turn is over, at the frozen threshold.
• Four conversations per GPU, and sessions end after three minutes by default. Both are config values, not limits of the model.
• Two tools are stubs. Reminders and messages are declared but not wired, and the agent says so when asked.

None of that is hidden in a footnote in the repo either, which is the standard we try to hold ourselves to on open weights.

How do you try it today?#

Pull the Docker image, give it a Hugging Face token and a GPU, open the page and click the microphone. The repo's quick start has the four commands, and there is a no-GPU mode with fake models if you only want to see the plumbing.

If you are building your own product rather than running ours, the agent has an API mode: it sits behind your app, your app issues the tokens, and the browser talks to it over a WebSocket. That is how we run it ourselves, on a rented GPU behind a Cloudflare tunnel.

And if you are still deciding whether any of this belongs in your stack, start with do you need a speech-to-speech model. It is five questions, and it will tell you no if the honest answer is no. For the text-to-speech side of the same question, how text to speech works covers the metrics vendors quote and which of them are worth anything. We are in Discord if you get stuck, and the thresholds are the thing we most want to hear about from real microphones.