Models
Groq
Groq offers blazing-fast API endpoints for large language models.
See all the Groq supported models here.
- We recommend using
llama-3.3-70b-versatilefor general use - We recommend
llama-3.1-8b-instantfor a faster result. - We recommend using
llama-3.2-90b-vision-previewfor image understanding.
Multimodal Support
With Groq we support Image as input
Authentication
Set your GROQ_API_KEY environment variable. Get your key from here.
Example
Use Groq with your Agent:
View more examples here.
Params
| Parameter | Type | Default | Description |
|---|---|---|---|
id | str | "llama-3.3-70b-versatile" | The specific model ID used for generating responses. |
name | str | "Groq" | The name identifier for the agent. |
provider | str | "Groq" | The provider of the model. |
frequency_penalty | Optional[float] | None | A number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. |
logit_bias | Optional[Any] | None | A JSON object that modifies the likelihood of specified tokens appearing in the completion by mapping token IDs to bias values between -100 and 100. |
logprobs | Optional[bool] | None | Whether to return log probabilities of the output tokens. |
max_tokens | Optional[int] | None | The maximum number of tokens to generate in the chat completion. |
presence_penalty | Optional[float] | None | A number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. |
response_format | Optional[Dict[str, Any]] | None | Specifies the format that the model must output. Setting to { "type": "json_object" } enables JSON mode, ensuring the message generated is valid JSON. |
seed | Optional[int] | None | A seed value for deterministic sampling, ensuring repeated requests with the same seed and parameters return the same result. |
stop | Optional[Union[str, List[str]]] | None | Up to 4 sequences where the API will stop generating further tokens. |
temperature | Optional[float] | None | The sampling temperature to use, between 0 and 2. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic. |
top_logprobs | Optional[int] | None | The number of top log probabilities to return for each generated token. |
top_p | Optional[float] | None | Nucleus sampling parameter. The model considers the results of the tokens with top_p probability mass. |
user | Optional[str] | None | A unique identifier representing your end-user, helping to monitor and detect abuse. |
extra_headers | Optional[Any] | None | Additional headers to include in API requests. |
extra_query | Optional[Any] | None | Additional query parameters to include in API requests. |
request_params | Optional[Dict[str, Any]] | None | Additional parameters to include in the request. |
api_key | Optional[str] | None | The API key for authenticating requests to the service. |
base_url | Optional[Union[str, httpx.URL]] | None | The base URL for making API requests to the service. |
timeout | Optional[int] | None | The timeout duration for requests, specified in seconds. |
max_retries | Optional[int] | None | The maximum number of retry attempts for failed requests. |
default_headers | Optional[Any] | None | Default headers to include in all API requests. |
default_query | Optional[Any] | None | Default query parameters to include in all API requests. |
http_client | Optional[httpx.Client] | None | A custom HTTP client for making API requests. |
client_params | Optional[Dict[str, Any]] | None | Additional parameters for client configuration. |
client | Optional[GroqClient] | None | An instance of GroqClient provided for making API requests. |
async_client | Optional[AsyncGroqClient] | None | An instance of AsyncGroqClient provided for making asynchronous API requests. |
Groq is a subclass of the Model class and has access to the same params.