Skip to main content

OpenAI Compatible

Connect any LLM endpoint that implements the OpenAI Chat Completions API — self-hosted servers (vLLM, LiteLLM, Ollama's OpenAI endpoint), managed gateways, or third-party providers that expose an OpenAI-compatible interface.

Getting started​

You need two things from your endpoint provider:

  • The base URL of the OpenAI-compatible endpoint (for example https://gateway.example.com/v1).
  • An API key, if the endpoint requires one. Self-hosted endpoints often don't.

Setting up in Unstract​

  • Sign in to the Unstract Platform.
  • From the side navigation menu, choose Settings 🞂 LLMs.
  • Click on the New LLM Profile button.
  • From the list of LLMs, choose OpenAI Compatible.
  • Name — a unique name for this adapter instance.
  • API Base — the base URL of your endpoint. Required.
  • API Key — the endpoint's API key. Leave empty if it does not require one.
  • Model — the model name your endpoint expects (for example gpt-4o-mini, openai/gpt-4o).
  • Maximum Output Tokens — defaults to 4096. Leave empty to use the provider default.
  • Enable Reasoning — toggle on for reasoning models (or gateway aliases hiding one). Sends reasoning_effort and max_completion_tokens instead of temperature. Reveals a Reasoning Effort field (low/medium/high).
  • Leave Max Retries and Timeout at their defaults.
  • Click on Test Connection and ensure it succeeds, then click Submit.