AI Provider

Description

An AI Provider is a named connection to a language-model endpoint: provider type, base URL, model name, API key, timeout and temperature. Objects are managed in the Metadata perspective and stored as JSON under:

metadata/ai-provider/

Use Refresh models to load model ids from the provider (OpenAI-compatible /models, Anthropic and Mistral catalogs, Ollama /api/tags). Hugging Face has no list API; type a router model id. The combo stays editable so you can enter a name that is not in the list.

Use the Test button in the editor to send a short health-check prompt.

The AI Assistant uses these objects as the credential source of truth. Global configuration only stores the enable flag and the default provider name.

Do not paste a live API key into this metadata object. Set ${AI_API_KEY} (or a similar name containing PASSWORD, SECRET or TOKEN) in an environment configuration file, or better a variable resolver expression such as #{vault:secret/data/ai:api-key}. The same pattern applies to every other secret in the project: never hard-code it in pipelines, workflows or metadata. See Secrets and personal information.

The Language model chat transform can optionally select a named AI Provider. The provider’s type, base URL, API key, chat model, timeout, temperature, max output tokens and thinking then take precedence over the transform’s inline values; an inline value is only used when the provider leaves that field empty. With an Ollama provider, the context size always comes from the provider (16384 when it is empty), and an Anthropic provider without Max output tokens sends 4096 rather than the transform’s value. In particular, a temperature set on the provider overrides the temperature on every transform that uses it, so leave Temperature empty here if transforms need their own. Input/output mapping, mock mode, proxy and retries stay on the transform. See AI Provider and inline settings for the details.

Provider types

Type Default base URL Default model API key

Type	Default base URL	Default model	API key
OpenAI	`https://api.openai.com/v1`	`gpt-4o-mini`	Yes
Grok (xAI)	`https://api.x.ai/v1`	`grok-4`	Yes
Gemini (OpenAI-compatible)	`https://generativelanguage.googleapis.com/v1beta/openai`	`gemini-2.5-flash`	Yes
Custom (OpenAI-compatible)	(you set it)	(you set it)	Usually
Anthropic	(plugin default)	`claude-3-5-sonnet-20241022`	Yes
Ollama	`http://localhost:11434`	`llama3.2`	No
Mistral	`https://api.mistral.ai/v1`	`mistral-small-latest`	Yes
Hugging Face	Dedicated inference endpoint URL, if you use one. Leave empty for the public inference router.	Router model id (for example `meta-llama/Llama-3.3-70B-Instruct`), or a dedicated endpoint URL if Base URL is empty	Yes (token)

OpenAI

https://api.openai.com/v1

gpt-4o-mini

Yes

Grok (xAI)

https://api.x.ai/v1

grok-4

Yes

Gemini (OpenAI-compatible)

https://generativelanguage.googleapis.com/v1beta/openai

gemini-2.5-flash

Yes

Custom (OpenAI-compatible)

(you set it)

Usually

Anthropic

(plugin default)

claude-3-5-sonnet-20241022

Yes

Ollama

http://localhost:11434

llama3.2

Mistral

https://api.mistral.ai/v1

mistral-small-latest

Yes

Hugging Face

Dedicated inference endpoint URL, if you use one. Leave empty for the public inference router.

Router model id (for example meta-llama/Llama-3.3-70B-Instruct), or a dedicated endpoint URL if Base URL is empty

Yes (token)

OpenAI, Grok, Gemini and Custom share the OpenAI Chat Completions API. A GitHub Copilot-style OAuth provider is an extension point; it is not bundled.

Options

The editor shows Name and Provider at the top, and the other options on three tabs: Connection (base URL, API key, timeout), Model (model name, temperature, context size, max output tokens, thinking, structured answers) and Models per role. Saving checks that a provider type is selected and that the numbers are valid; values that use a variable are checked when the provider is used.

Option Description

Option	Description
Provider	Plugin implementation (OpenAI, Grok, …)
Base URL	Override the type’s default endpoint. Variables such as `'${AI_BASE_URL}'` are resolved at send time. For Hugging Face, a URL here is used as the dedicated inference endpoint when Model name is empty.
API key	Secret stored as a Hop password field. Not sent in advisor prompts. Prefer `${AI_API_KEY}` from an environment file or a keystore resolver expression, not a pasted key.
Model name	Model identifier for the chosen provider. Refresh models fills the combo from the live catalog when the provider supports it. For Hugging Face, a model id is sent to the inference router; an `http(s)` URL is called as a dedicated endpoint. If this field is empty, Base URL is used instead.
Models per role	Optional. One model for each role this provider serves, so a single provider can back a chat transform and an embedding transform at the same time. See below.
Timeout (seconds)	HTTP timeout for completions.
Temperature	Sampling temperature, when the provider supports it. When set, it overrides the temperature configured on a Language Model Chat transform that uses this provider.
Context size (tokens)	How many tokens the model can read at once: prompt, conversation history and answer together. The AI Assistant estimates the size of every question before sending it and stops with an explanation when it clearly does not fit. For Ollama this also sets the context window the model is loaded with (`num_ctx`). Leave it empty for 16384 with Ollama. For the other types an empty field is not sent; questions are then checked against 200000 tokens for Anthropic and 128000 for the others. Set the real window of the model if it is smaller. Ollama’s own default (2048 or 4096, depending on the version) is too small for a typical assistant question and is cut silently, so set a value here rather than relying on the server. A larger window uses more memory on the Ollama server.
Max output tokens	The longest answer the model may write, in tokens. Leave it empty for the provider’s default, which is 4096 for Anthropic (Anthropic requires a value). A value that is too low cuts answers and proposals short.
Thinking	Whether a thinking model (qwen3, deepseek-r1, gpt-oss, …) reasons before it answers. * Default: leave it to the model. Most thinking models then reason. Depending on the model and the Ollama version, the answer can then include the reasoning between `<think>` tags. This is the value for providers created before the option existed, and it keeps their output as it was. * Off: answer right away (`think: false`). * On: always reason first (`think: true`). For transforms that call the model once per row, the reasoning usually takes most of the run time: a short Extract graph call that writes thousands of reasoning tokens with the default writes about a hundred with Off. Adding `/no_think` to a prompt is not a reliable alternative: not every model honours it. With Off or On the reasoning is kept out of the answer. The value can be a variable, such as `'${AI_THINKING}'`, that resolves to `Default`, `Off` or `On`. Applies to Ollama only for now; other provider types ignore it.
Structured answers	Off by default. When checked, the AI Assistant asks the provider to hold the model to a JSON schema: the answer text plus a list of proposals with a known type, a risk level and their parameters. The proposals then cannot come back broken, which helps small models most. Works with Ollama, OpenAI, Anthropic and Mistral. Other types (Gemini, Grok, custom OpenAI-compatible servers, Hugging Face) answer as usual. When the provider refuses the schema, the question is asked again without it and the log says so. Some models write shorter answers this way: compare with and without it on the models you use, for example with the evaluation set described in AI technology.

Provider

Plugin implementation (OpenAI, Grok, …)

Base URL

Override the type’s default endpoint. Variables such as '${AI_BASE_URL}' are resolved at send time. For Hugging Face, a URL here is used as the dedicated inference endpoint when Model name is empty.

API key

Secret stored as a Hop password field. Not sent in advisor prompts. Prefer ${AI_API_KEY} from an environment file or a keystore resolver expression, not a pasted key.

Model name

Model identifier for the chosen provider. Refresh models fills the combo from the live catalog when the provider supports it. For Hugging Face, a model id is sent to the inference router; an http(s) URL is called as a dedicated endpoint. If this field is empty, Base URL is used instead.

Models per role

Optional. One model for each role this provider serves, so a single provider can back a chat transform and an embedding transform at the same time. See below.

Timeout (seconds)

HTTP timeout for completions.

Temperature

Sampling temperature, when the provider supports it. When set, it overrides the temperature configured on a Language Model Chat transform that uses this provider.

Context size (tokens)

How many tokens the model can read at once: prompt, conversation history and answer together. The AI Assistant estimates the size of every question before sending it and stops with an explanation when it clearly does not fit. For Ollama this also sets the context window the model is loaded with (num_ctx). Leave it empty for 16384 with Ollama. For the other types an empty field is not sent; questions are then checked against 200000 tokens for Anthropic and 128000 for the others. Set the real window of the model if it is smaller. Ollama’s own default (2048 or 4096, depending on the version) is too small for a typical assistant question and is cut silently, so set a value here rather than relying on the server. A larger window uses more memory on the Ollama server.

Max output tokens

The longest answer the model may write, in tokens. Leave it empty for the provider’s default, which is 4096 for Anthropic (Anthropic requires a value). A value that is too low cuts answers and proposals short.

Thinking

Whether a thinking model (qwen3, deepseek-r1, gpt-oss, …) reasons before it answers.

* Default: leave it to the model. Most thinking models then reason. Depending on the model and the Ollama version, the answer can then include the reasoning between <think> tags. This is the value for providers created before the option existed, and it keeps their output as it was. * Off: answer right away (think: false). * On: always reason first (think: true).

For transforms that call the model once per row, the reasoning usually takes most of the run time: a short Extract graph call that writes thousands of reasoning tokens with the default writes about a hundred with Off. Adding /no_think to a prompt is not a reliable alternative: not every model honours it. With Off or On the reasoning is kept out of the answer. The value can be a variable, such as '${AI_THINKING}', that resolves to Default, Off or On. Applies to Ollama only for now; other provider types ignore it.

Structured answers

Off by default. When checked, the AI Assistant asks the provider to hold the model to a JSON schema: the answer text plus a list of proposals with a known type, a risk level and their parameters. The proposals then cannot come back broken, which helps small models most. Works with Ollama, OpenAI, Anthropic and Mistral. Other types (Gemini, Grok, custom OpenAI-compatible servers, Hugging Face) answer as usual. When the provider refuses the schema, the question is asked again without it and the log says so. Some models write shorter answers this way: compare with and without it on the models you use, for example with the evaluation set described in AI technology.

When Language Model Chat points at an AI Provider, extra transform options (proxy, retries, mock, I/O fields) are not taken from the provider.

Models per role

A provider endpoint usually serves more than one kind of model. The same OpenAI key reaches a chat model and an embedding model; the same Ollama server serves both, and a reranker besides. Models per role lets one provider object cover all of them, so the base URL and the API key are configured once.

Each row pairs a role with a model name:

Role Used by

Role	Used by
`CHAT`	Conversation and completion, for example Language Model Chat, the AI Assistant, Structured extract and Extract graph.
`EMBEDDING`	Turning text into a vector, for example Embed text.
`SCORING`	Scoring a passage against a query, used by rerankers.
`IMAGE`	Image generation.
`MODERATION`	Content moderation.

CHAT

Conversation and completion, for example Language Model Chat, the AI Assistant, Structured extract and Extract graph.

EMBEDDING

Turning text into a vector, for example Embed text.

SCORING

Scoring a passage against a query, used by rerankers.

IMAGE

Image generation.

MODERATION

Content moderation.

A transform never asks which role to use: it needs one kind of model and looks up that role. Point a chat transform and an embedding transform at the same provider and each finds its own model.

Use at most one row per role, since that is what makes the lookup unambiguous. An embedding transform that needs a different model than the provider’s default for its role can override it on the transform itself. Language Model Chat works the other way round: the provider’s CHAT model wins, and the model name on the transform is only used when the provider has none.

Relationship to Model name

Model name on the Model tab is the chat model. A provider with no rows in this table behaves exactly as before: CHAT falls back to Model name, and nothing else resolves. A CHAT row replaces Model name: the AI Assistant, Language Model Chat and Test then use the model of that row.

The Model name column offers the models that Refresh models loaded, and accepts variables such as '${EMBEDDING_MODEL}'.

The fallback is deliberately limited to CHAT. Handing a chat model to an embedding endpoint fails inside the provider with a message that is hard to act on, so a provider with no EMBEDDING row reports that plainly instead.