Input

Configure audio formats and language hints for a Speechmatics Realtime transcription session.

Supported input audio formats

Sessions can be configured to use two types of audio input: file and raw.
We recommend using the raw option, unless you have a specific reason to use the file option.

For capturing raw audio in the browser, try our browser-audio-input package.

audio_format

The format must be supplied in the audio_format field of the StartRecognition message. See the API reference.

oneOf

Raw

Raw audio samples, described by the following additional mandatory fields:

typerequired

Constant value: raw

encodingstringrequired

Possible values: [pcm_f32le, pcm_s16le, mulaw]

sample_rateintegerrequired

The sample rate of the audio in Hz.

Example: {"type":"raw","encoding":"pcm_s16le","sample_rate":44100}

File

Choose this option to send audio encoded in a recognized format. The AddAudio messages have to provide all the file contents, including any headers. The file is usually not accepted all at once, but segmented into reasonably sized messages.

Note: Only the following formats are supported: wav, mp3, aac, ogg, mpeg, amr, m4a, mp4, flac

typerequired

Constant value: file

Language hints

Language hints are optional and apply only to the Melia 1 model, which detects supported languages automatically. For the Enhanced and Standard models, set language to a supported language instead.

Hints tell the model which languages to expect in the audio, biasing detection toward them. They are most useful for short utterances, audio with heavy accents, or conversations where two languages sound similar, where they make language labeling more reliable.

Provide hints in the StartRecognition message as a list of supported languages to guide detection without restricting it. This config hints that the audio contains English and Arabic:

{
  "message": "StartRecognition",
  "audio_format": {
    "type": "raw",
    "encoding": "pcm_s16le",
    "sample_rate": 16000
  },
  "transcription_config": {
    "model": "melia-1",
    // Melia 1 requires "multi" rather than a single language code
    "language": "multi",
    "language_hints": ["en", "ar"]
  }
}

The model can still detect and label a language you did not hint, and it labels only the languages it actually hears. To render each detected language correctly, refer to the LanguageInfo message.

Sending audio

After receiving a RecognitionStarted message, you can start sending audio over the Websocket connection. Audio is sent as binary data, encoded in the format specified in the StartRecognition message. See Protocol overview for complete details of the API protocol.

Next steps

View our guides:

• using a microphone to learn how to capture audio from a microphone.
• using FFMPEG to find out how to pipe microphone audio to the API.