LLM Configuration

Ctrl AI uses a role-based model selection system. Each task in the inference pipeline uses the best model for the job.

Default Model Roles

Provider Priority

The system resolves which model to use in this order:

1. Org BYOK keys — if the org has configured their own API key for a provider
2. Org default provider — if the org has selected a default provider in settings
3. Environment variables — system-level API keys
4. System defaults — hardcoded fallbacks

Configuring via UI

Organization admins can override models in Settings > AI Models:

1. Select a default AI provider (Gemini, Anthropic, OpenAI, OpenRouter)
2. Choose a specific model for that provider
3. Optionally enter your own API key (BYOK)

Settings are stored per-organization and take effect immediately.

BYOK (Bring Your Own Key)

Organizations can provide their own API keys for any supported provider. This means:

• The org is billed directly by the LLM provider
• Ctrl AI never sees or stores the actual API requests/responses on its infrastructure
• Full control over model selection and usage limits

Self-Hosted: Ollama (Fully Local)

For air-gapped or fully local deployments:

LLM_PROVIDER=ollama
LLM_BASE_URL=http://host.docker.internal:11434/v1
LLM_MODEL=llama3

This routes all LLM calls through your local Ollama instance. No data leaves your network.

Self-Hosted: OpenAI-Compatible

For any OpenAI-compatible API (Azure OpenAI, Together, Groq, vLLM, etc.):

LLM_PROVIDER=openai
LLM_BASE_URL=https://your-endpoint.com/v1
LLM_API_KEY=your-key
LLM_MODEL=your-model

Model Provenance

Every inference query logs which provider and model were used for each role. This is visible in:

• The inference response (modelsUsed field)
• Inference audit logs
• The compliance dashboard

This enables full traceability for regulatory requirements (EU AI Act Article 11).