MoviePy Video Tools

Agno MoviePyVideoTools enable an Agent to process videos, extract audio, generate SRT caption files, and embed rich, word-highlighted captions.

Prerequisites

To use MoviePyVideoTools, you need moviepy and the ffmpeg binary installed on your system. The example below also uses OpenAITools, which needs openai:

uv pip install agno moviepy openai

Install ffmpeg itself with your system package manager:

Example

create_srt writes an already timestamped SRT string; it does not create timings. Agno's OpenAITools.transcribe_audio returns plain text, so this example uses a small Whisper tool that explicitly requests SRT.

Place a video at input.mp4 in the working directory. Set CAPTION_FONT_PATH to an installed .ttf or .otf font file readable by MoviePy, then save this as caption_video.py and run python caption_video.py:

from functools import partial
from os import environ
from pathlib import Path

from openai import OpenAI
from agno.agent import Agent
from agno.models.openai import OpenAIResponses
from agno.tools.moviepy_video import MoviePyVideoTools

video_path = Path("input.mp4").resolve()
font_path = Path(environ["CAPTION_FONT_PATH"]).expanduser().resolve()
if not video_path.is_file() or not font_path.is_file():
    raise FileNotFoundError("Provide input.mp4 and a valid CAPTION_FONT_PATH.")
output_dir = Path("caption_output").resolve()
output_dir.mkdir(parents=True, exist_ok=True)

def transcribe_srt(audio_path: str) -> str:
    """Transcribe an audio file to timestamped SRT; only use successful output as captions."""
    with open(audio_path, "rb") as audio_file:
        return OpenAI().audio.transcriptions.create(
            model="whisper-1", file=audio_file, response_format="srt"
        )

video_tools = MoviePyVideoTools()
# The current adapter does not forward embed_captions styling arguments.
# Bind a usable font at the actual caption-rendering method instead.
video_tools.create_caption_clips = partial(
    video_tools.create_caption_clips, font=str(font_path)
)
agent = Agent(
    model=OpenAIResponses(id="gpt-5.5"),
    tools=[video_tools, transcribe_srt],
    instructions=[
        "Extract audio, then call transcribe_srt for timestamped captions.",
        "Only pass a successful SRT result to create_srt; never use an error as captions.",
        "Use embed_captions to render the SRT onto the input video.",
        "Stop and explain any tool failure instead of claiming completion.",
    ],
)
agent.print_response(
    f"Caption {video_path}. Write the extracted audio, SRT, and captioned video under {output_dir}."
)

The toolkit derives approximate word highlighting from each SRT cue; it does not obtain word-level speech timestamps. Its current embed_captions styling arguments are not forwarded to create_caption_clips; the example binds the font explicitly at that method.

Toolkit Functions

These are the functions exposed by MoviePyVideoTools:

FunctionDescription
extract_audioExtracts the audio track from a video file and saves it to a specified output path.
create_srtSaves a given transcription (expected in SRT format) to a .srt file at the specified output path.
embed_captionsEmbeds captions from an SRT file into a video, creating a new video file with word-level highlighting.

Toolkit Params

These parameters are passed to the MoviePyVideoTools constructor:

ParameterTypeDefaultDescription
enable_process_videoboolTrueEnables the extract_audio tool.
enable_generate_captionsboolTrueEnables the create_srt tool.
enable_embed_captionsboolTrueEnables the embed_captions tool.
allboolFalseEnables all functions.

Developer Resources