MLX Transcribe

MLX Transcribe is a tool for transcribing audio files using MLX Whisper.

Prerequisites

This toolkit requires an Apple Silicon Mac. The following mlx-whisper setup is not a Windows or Ubuntu recipe.

Install ffmpeg: brew install ffmpeg
Install mlx-whisper and openai libraries
```
uv pip install agno mlx-whisper openai
```
Prepare audio files
- Create a 'storage/audio' directory
- Place your audio files in this directory
- Supported formats: mp3, mp4, wav, etc.
Download sample audio (optional)
- Visit the audio-samples (as an example) and save the audio file to the storage/audio directory.

Example

The following agent will use MLX Transcribe to transcribe audio files.

cookbook/91_tools/mlx_transcribe_tools.py


from pathlib import Path
from agno.agent import Agent
from agno.models.openai import OpenAIResponses
from agno.tools.mlx_transcribe import MLXTranscribeTools

# Get audio files from storage/audio directory
agno_root_dir = Path(__file__).parent.parent.parent.resolve()
audio_storage_dir = agno_root_dir.joinpath("storage/audio")
if not audio_storage_dir.exists():
    audio_storage_dir.mkdir(exist_ok=True, parents=True)

agent = Agent(
    name="Transcription Agent",
    model=OpenAIResponses(id="gpt-5.2"),
    tools=[MLXTranscribeTools(base_dir=audio_storage_dir)],
    instructions=[
        "To transcribe an audio file, use the `transcribe` tool with the name of the audio file as the argument.",
        "You can find all available audio files using the `read_files` tool.",
    ],
    markdown=True,
)

agent.print_response("Summarize the reid hoffman ted talk, split into sections", stream=True)

Toolkit Params

Parameter	Type	Default	Description
`base_dir`	`Path`	`Path.cwd()`	Base directory for audio files
`enable_read_files_in_base_dir`	`bool`	`True`	Whether to register the read_files function
`restrict_to_base_dir`	`bool`	`True`	Restrict file access to the base directory
`path_or_hf_repo`	`str`	`"mlx-community/whisper-large-v3-turbo"`	Path or HuggingFace repo for the model
`verbose`	`bool`	`None`	Enable verbose output
`temperature`	`float` or `Tuple[float, ...]`	`None`	Temperature for sampling
`compression_ratio_threshold`	`float`	`None`	Compression ratio threshold
`logprob_threshold`	`float`	`None`	Log probability threshold
`no_speech_threshold`	`float`	`None`	No speech threshold
`condition_on_previous_text`	`bool`	`None`	Whether to condition on previous text
`initial_prompt`	`str`	`None`	Initial prompt for transcription
`word_timestamps`	`bool`	`None`	Enable word-level timestamps
`prepend_punctuations`	`str`	`None`	Punctuations to prepend
`append_punctuations`	`str`	`None`	Punctuations to append
`clip_timestamps`	`str` or `List[float]`	`None`	Clip timestamps
`hallucination_silence_threshold`	`float`	`None`	Hallucination silence threshold
`decode_options`	`dict`	`None`	Additional decoding options
`all`	`bool`	`False`	Enable all functions

Toolkit Functions

Function	Description
`transcribe`	Transcribes an audio file using MLX Whisper
`read_files`	Lists all audio files in the base directory

Standalone file paths

The cookbook computes its audio directory relative to a file saved under cookbook/91_tools. For a standalone script, replace the agno_root_dir and audio_storage_dir assignments with:

audio_storage_dir = Path("storage/audio").resolve()

Create that directory in your working directory and put an actual audio file there, for example interview.mp3. Replace the final request with agent.print_response("Transcribe and summarize interview.mp3", stream=True). The first transcription needs access to the configured Hugging Face model weights, unless they are already cached locally.

Developer Resources

Tools