OpenAI compatible endpoint model

You can create a new OpenAI compatible endpoint model by adding a model and provide the OpenAI compatible endpoint URL and API key.

To create a new OpenAI compatible endpoint model, follow these steps:

• Go to Model page by clicking the “Model” button on the bottom left toolbar.
• Click the “New Model” button, and click the “New OpenAI Chat Completion model” button.
• Paste the OpenAI model URL and model id.
• Click “Create” button.

After creating a new OpenAI compatible endpoint model, you can use it in the chat. The app will prompt you to provide the API key if necessary. You can also change the API key in the model settings.

Common uses cases

OpenAI-compatible endpoints are useful in several common scenarios:

• Local Ollama models (base URL: http://127.0.0.1:11434/v1)
• Self-hosted LLM APIs
• Alternative AI providers with OpenAI-compatible APIs such as Azure OpenAI Service, Groq and xAI Grok services

Ollama

You can use local Ollama with RecurseChat through Ollama’s support of OpenAI compatible API.

To use Ollama:

1. Serve an Ollama model with ollama serve
2. Create a new OpenAI API model: go to top right of the model tab, and click the “New Model” button, and click “New OpenAI Chat Completion model”.
3. To use Ollama model, set base url to http://127.0.0.1:11434/v1 and set model id to an Ollama model id like mistral.

Azure OpenAI

You can use Azure OpenAI Service with RecurseChat through their OpenAI-compatible API.

To use Azure OpenAI:

1. Create an Azure OpenAI Service resource in the Azure portal
2. Get your API key and endpoint URL from the Azure portal
3. Create a new OpenAI API model in RecurseChat:
   • Set base URL to your Azure endpoint (e.g. https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-name}/chat/completions?api-version={api-version})
   • Set model ID to your deployed model name
   • Add your Azure API key
   • Configure other settings as needed
4. Click “Create” to add the model

Azure OpenAI Service provides:

• Enterprise-grade security and compliance
• Regional availability options
• SLA guarantees
• Access to latest OpenAI models

Groq

You can use Groq with RecurseChat through their OpenAI-compatible API.

To use Groq:

1. Get an API key from Groq’s console
2. Create a new OpenAI API model in RecurseChat:
   • Set base URL to https://api.groq.com/openai/v1
   • Set model ID to one of Groq’s models like mixtral-8x7b-32768 or llama2-70b-4096
   • Add your Groq API key
   • Configure other settings as needed
3. Click “Create” to add the model

Groq offers several benefits:

• Fast inference speeds
• Support for long context windows (up to 32k tokens)
• High quality open models like Mixtral and Llama 3

xAI Grok

You can use xAI’s Grok model with RecurseChat through their OpenAI-compatible API.

To use Grok:

1. Get an API key from xAI’s console
2. Create a new OpenAI API model in RecurseChat:
   • Set base URL to https://api.x.ai/v1
   • Set model ID to grok-beta
   • Add your xAI API key
   • Configure other settings as needed
3. Click “Create” to add the model

Grok offers several benefits:

• Real-time knowledge through X/Twitter integration
• Witty, humorous responses inspired by Hitchhiker’s Guide to the Galaxy
• Strong reasoning and coding capabilities