LiveKit speech to text

Transcribe live audio in your LiveKit voice agents with the Speechmatics agent STT plugin.

The Speechmatics plugin for LiveKit Agents transcribes live audio using agent STT, the turn-based interaction pattern on the Realtime API built for applications that hold a conversation. It runs on the Linden 1 model.

The plugin connects directly to Speechmatics with your own API key. If you would rather run transcription on LiveKit's infrastructure without a Speechmatics key, use LiveKit Inference instead.

Features

• Real-time transcription — transcript segments stream as the speaker talks, with optional partial results
• Turn detection — a voice activity detector (VAD) in your agent closes each turn, or Speechmatics closes it server-side
• Speaker diarization — attribute each segment to a speaker
• Stable speaker labels — carry the same labels across sessions
• Custom vocabulary — bias recognition towards domain-specific words
• Speaker formatting — template speaker labels for your language model

Installation

Install LiveKit Agents with the Speechmatics and Silero extras:

uv add "livekit-agents[speechmatics,silero]~=1.8.2"

The default turn detection mode needs a VAD running in your agent, and the plugin loads Silero when you don't supply one. That is why this command includes the silero extra. See Turn detection modes.

Authentication

The plugin reads your API key from the SPEECHMATICS_API_KEY environment variable.

.env.local

# Or pass api_key= to speechmatics.STT() directly
SPEECHMATICS_API_KEY=your_speechmatics_key

Endpoint

The plugin connects to wss://eu2.rt.speechmatics.com/v2/agent. Set base_url to reach a different region or a self-hosted deployment. Agent STT sessions are served on the /v2/agent path, and the SDK appends /agent when your URL omits it.

stt = speechmatics.STT(
    # The /v2/agent path is required; /v2 is the streaming endpoint
    base_url="wss://eu2.rt.speechmatics.com/v2/agent",
)

Basic usage

Pass the plugin as the stt of an AgentSession. The defaults transcribe English on Linden 1 with diarization and partial results enabled.

from livekit.agents import AgentSession
from livekit.plugins import speechmatics

session = AgentSession(
    # Silero VAD is loaded automatically to close each turn
    stt=speechmatics.STT(language="en"),
    # ... llm, tts, etc.
)

Transcription parameters

The speechmatics.STT constructor accepts the following transcription parameters.

Parameter	Type	Default	Description
api_key	string	env var	Speechmatics API key. Falls back to SPEECHMATICS_API_KEY
base_url	string	wss://eu2.rt.speechmatics.com/v2/agent	Endpoint to connect to
model	Model | string	linden-1	Transcription model. linden-1 is the only model agent STT accepts
language	string	"en"	Language code for the input audio. See Languages
output_locale	string | null	null	BCP-47 locale controlling spelling and number formats, for example "en-GB"
domain	string | null	null	Domain-specific language pack, for example "finance"
include_partials	boolean	true	Emit interim segments that update until the final segment arrives
additional_vocab	array | null	null	AdditionalVocabEntry objects biasing recognition towards specific words
sample_rate	number	16000	Audio sample rate in Hz. Agent STT accepts 16000 only; any other value raises a ValueError
audio_encoding	AudioEncoding	PCM_S16LE	LiveKit frames are 16-bit PCM and are forwarded unconverted, so any other value raises a ValueError

Every language is global: whichever you select, the model recognizes a range of accents and dialects.

A parameter you leave unset is not sent, and agent STT applies its own default rather than the one in the Realtime API schema. Partials and diarization are both on unless you turn them off.

Custom vocabulary

Pass additional_vocab entries to boost recognition of names and domain terms. Each entry takes a content value and optional sounds_like pronunciation variants.

from livekit.plugins.speechmatics import AdditionalVocabEntry

stt = speechmatics.STT(
    additional_vocab=[
        # sounds_like helps when the spelling is unusual
        AdditionalVocabEntry(content="LiveKit", sounds_like=["live kit"]),
    ],
)

Turn detection

Agent STT reports turn boundaries as StartOfTurn and EndOfTurn events. The turn_detection_mode parameter sets which component decides where those boundaries fall.

Turn detection modes

Two modes are available.

Mode	Detection method
TurnDetectionMode.EXTERNAL	Default. A VAD running in your agent closes the turn. Works with LiveKit's own turn detection
TurnDetectionMode.VAD	Speechmatics closes the turn server-side using its own voice activity detection

The two modes fail in opposite directions. EXTERNAL returns the final transcript sooner and with a tighter tail, but can clip a word at a turn boundary and drops short backchannels such as "mm-hmm" that never cross your VAD's speech threshold. VAD is slightly more accurate and keeps backchannels, because Speechmatics applies its own, more patient endpointing.

External turn detection

EXTERNAL is the default and needs no configuration. The plugin loads Silero VAD when you don't pass a vad. Pass your own VAD to tune it, or reuse the one already on your session so a second model isn't loaded:

from livekit.plugins import silero, speechmatics

vad = silero.VAD.load()

# Share one VAD between the session and the plugin
stt = speechmatics.STT(vad=vad)

To signal turn ends from your own logic instead, pass vad=None and call finalize() on the STT when the speaker stops.

In EXTERNAL mode something must close each turn. If Silero isn't installed and you pass no vad, the plugin logs a warning, nothing closes a turn, and the session produces no transcripts at all rather than failing.

Server-side turn detection

Setting turn_detection_mode to VAD hands turn handling to Speechmatics, which runs server-side VAD to drive turn detection. You must also set turn_detection="stt" in the session's turn handling options — without it the session ignores the end-of-speech events the plugin emits.

from livekit.agents import AgentSession, TurnHandlingOptions
from livekit.plugins import speechmatics

session = AgentSession(
    stt=speechmatics.STT(
        turn_detection_mode=speechmatics.TurnDetectionMode.VAD,
    ),
    # Required, or the plugin's end-of-speech events are discarded
    turn_handling=TurnHandlingOptions(turn_detection="stt"),
)

Turn detection parameters

Parameter	Type	Default	Description
turn_detection_mode	TurnDetectionMode	EXTERNAL	Which component closes a turn
vad	VAD | null	Silero	VAD that signals the end of a turn. Applies to EXTERNAL mode only; ignored in VAD mode

End-of-utterance timing in VAD mode belongs to the service and is not configurable.

Speaker diarization

Diarization (attributing speech to individual speakers) is enabled by default. Each transcript segment carries one speaker label, such as S1. Segments the model cannot attribute are labelled UU, and with diarization disabled every segment is labelled UU.

Agent STT reports one speaker per segment rather than per word, so there is no per-word speaker data.

Diarization parameters

Parameter	Type	Default	Description
enable_diarization	boolean	true	Attribute each segment to a speaker
speaker_sensitivity	number | null	null	Speaker detection sensitivity, between 0 and 1. Higher values help separate similar voices
max_speakers	number | null	null	Maximum speakers to detect, between 2 and 100. Unrestricted when unset
prefer_current_speaker	boolean	false	Reduce switching between similar-sounding speakers
known_speakers	array | null	null	SpeakerIdentifier objects from an earlier session
speaker_format	string | null	null	Template for speaker output

All of these apply only while enable_diarization is true.

Format speaker labels

Set speaker_format to fold the speaker label into the transcript text your language model receives. The template accepts {speaker_id} and {text}.

stt = speechmatics.STT(
    # Renders as <S1>Good morning.</S1>
    speaker_format="<{speaker_id}>{text}</{speaker_id}>",
)

Include the format in your agent instructions so the model interprets the labels consistently. To detect the primary speaker and format transcripts by speaker across providers, wrap the plugin with LiveKit's MultiSpeakerAdapter.

Reuse speaker labels across sessions

Speechmatics assigns speaker labels per session, so the same person is S1 in one session and something else in the next. Call get_speaker_ids() once each speaker has said at least five words, then pass the result as known_speakers when you build the STT for the next session.

# Returns one list per open stream, or a list of lists if several are open
speakers = await stt.get_speaker_ids()

stt = speechmatics.STT(known_speakers=speakers)

Speaker identifiers are unique to your Speechmatics account.

Deprecated parameters

Parameters from plugin versions that predate agent STT are still accepted and each logs a warning. Unrecognized parameters are logged too, so a misspelled name isn't discarded silently.

The parameters below, along with the ADAPTIVE, FIXED, and SMART_TURN turn detection modes, are removed after October 5, 2026. Each of those three modes selected a server-side endpointing strategy that agent STT does not expose, so all three resolve to EXTERNAL with a warning. The enhanced and standard operating points resolve to linden-1 the same way.

Renamed parameters still apply their value to the parameter that replaced them.

Parameter	Replacement
operating_point	model
enable_partials	include_partials
diarization_sensitivity	speaker_sensitivity
speaker_active_format	speaker_format

The rest are accepted and ignored.

Parameter	Reason
max_delay	Agent STT rejects it
punctuation_overrides	Agent STT rejects it
end_of_utterance_silence_trigger, end_of_utterance_max_delay, end_of_utterance_mode, end_of_turn_config	End-of-utterance timing is the service's own
focus_speakers, ignore_speakers, focus_mode	Agent STT does not support speaker focus
speaker_passive_format	A segment has no passive case to format
vad_config	Turn detection is set with turn_detection_mode
audio_settings, chunk_size, http_session, transcription_config	Unused before agent STT

The update_speakers() method is deprecated and does nothing, for the same reason as the speaker focus parameters.

Full example

This example combines transcription, turn detection, diarization, and speaker formatting in one STT instance.

from livekit.agents import AgentSession
from livekit.plugins import silero, speechmatics
from livekit.plugins.speechmatics import (
    AdditionalVocabEntry,
    TurnDetectionMode,
)

# Shared between the session and the plugin so only one model is loaded
vad = silero.VAD.load()

stt = speechmatics.STT(
    # Transcription
    language="en",
    output_locale="en-US",
    include_partials=True,

    # Turn detection: the VAD above closes each turn
    turn_detection_mode=TurnDetectionMode.EXTERNAL,
    vad=vad,

    # Diarization
    enable_diarization=True,
    speaker_sensitivity=0.6,
    max_speakers=4,
    prefer_current_speaker=True,

    # Renders each segment as <S1>Good morning.</S1>
    speaker_format="<{speaker_id}>{text}</{speaker_id}>",

    # Custom vocabulary
    additional_vocab=[
        AdditionalVocabEntry(content="Speechmatics"),
        AdditionalVocabEntry(content="LiveKit", sounds_like=["live kit"]),
    ],
)

session = AgentSession(
    stt=stt,
    vad=vad,
    # ... llm, tts, etc.
)

Next steps

• Quickstart — build a complete agent with LiveKit
• Text to speech — add Speechmatics voices
• Diarization — tune speaker separation
• Turn detection — how LiveKit closes a turn
• LiveKit documentation — full Speechmatics STT reference