OpenAITools allow an Agent to interact with OpenAI models for performing audio transcription, image generation, and text-to-speech.

​
Prerequisites

Before using OpenAITools, ensure you have the openai library installed and your OpenAI API key configured.
  1. Install the library: 
    pip install -U openai
  2. Set your API key: Obtain your API key from OpenAI and set it as an environment variable. 
    export OPENAI_API_KEY=xxx

​
Initialization

Import OpenAITools and add it to your Agent’s tool list. 
from agno.agent import Agent
from agno.tools.openai import OpenAITools

agent = Agent(
    name="OpenAI Agent",
    tools=[OpenAITools()],
        markdown=True,
)

​
Usage Examples

​
1. Transcribing Audio

This example demonstrates an agent that transcribes an audio file.
transcription_agent.py
from pathlib import Path
from agno.agent import Agent
from agno.tools.openai import OpenAITools
from agno.utils.media import download_file

audio_url = "https://agno-public.s3.amazonaws.com/demo_data/sample_conversation.wav"

local_audio_path = Path("tmp/sample_conversation.wav")
download_file(audio_url, local_audio_path)

agent = Agent(
    name="OpenAI Transcription Agent",
    tools=[OpenAITools(transcription_model="whisper-1")],
        markdown=True,
)

agent.print_response(f"Transcribe the audio file located at '{local_audio_path}'")

​
2. Generating Images

This example demonstrates an agent that generates an image based on a text prompt.
image_generation_agent.py
from agno.agent import Agent
from agno.tools.openai import OpenAITools
from agno.utils.media import save_base64_data

agent = Agent(
    name="OpenAI Image Generation Agent",
    tools=[OpenAITools(image_model="dall-e-3")],
        markdown=True,
)

response = agent.run("Generate a photorealistic image of a cozy coffee shop interior")

if response.images:
    save_base64_data(response.images[0].content, "tmp/coffee_shop.png")

​
3. Generating Speech

This example demonstrates an agent that generates speech from text.
speech_synthesis_agent.py
from agno.agent import Agent
from agno.tools.openai import OpenAITools
from agno.utils.media import save_base64_data

agent = Agent(
    name="OpenAI Speech Agent",
    tools=[OpenAITools(
        text_to_speech_model="tts-1",
        text_to_speech_voice="alloy",
        text_to_speech_format="mp3"
    )],
    markdown=True,
)

response = agent.run("Generate audio for the text: 'Hello, this is a synthesized voice example.'")
if response and response.audio:
    save_base64_data(response.audio[0].base64_audio, "tmp/hello.mp3")
View more examples here.

​
Customization

You can customize the underlying OpenAI models used for transcription, image generation, and TTS: 
OpenAITools(
    transcription_model="whisper-1",
    image_model="dall-e-3",
    text_to_speech_model="tts-1-hd",
    text_to_speech_voice="nova",
    text_to_speech_format="wav"
)

​
Toolkit Params

ParameterTypeDefaultDescription
api_keystrNoneOpenAI API key. Uses OPENAI_API_KEY env var if not provided
enable_transcriptionboolTrueEnable audio transcription functionality
enable_image_generationboolTrueEnable image generation functionality
enable_speech_generationboolTrueEnable speech generation functionality
allboolFalseEnable all tools when set to True
transcription_modelstrwhisper-1Model to use for audio transcription
text_to_speech_voicestralloyVoice to use for text-to-speech (alloy, echo, fable, onyx, nova, shimmer)
text_to_speech_modelstrtts-1Model to use for text-to-speech (tts-1, tts-1-hd)
text_to_speech_formatstrmp3Audio format for TTS output (mp3, opus, aac, flac, wav, pcm)
image_modelstrdall-e-3Model to use for image generation
image_qualitystrNoneQuality setting for image generation
image_sizestrNoneSize setting for image generation
image_stylestrNoneStyle setting for image generation (vivid, natural)

​
Toolkit Functions

The OpenAITools toolkit provides the following functions:
FunctionDescription
transcribe_audioTranscribes audio from a local file path or a public URL
generate_imageGenerates images based on a text prompt
generate_speechSynthesizes speech from text

​
Developer Resources