> ## Documentation Index > Fetch the complete documentation index at: https://docs.qodo.ai/llms.txt > Use this file to discover all available pages before exploring further. # Configure the AI gateway > Connect Qodo's AI gateway to your model providers and configure credentials, application model names, and embedding routes. On-premises Configure model access before you install Qodo. Connect the AI gateway to your model providers, then match Qodo's application settings to the gateway's model names. The examples configure review models for OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Vertex AI, and the Gemini API. They retain GPT chat and OpenAI-compatible embedding routes for Platform and Context Engine. > [!WARNING] > > Selecting a provider for code review does not move every Qodo feature to > that provider. Platform, Context Engine, and SDK agents have separate model > requirements. A deployment that must exclude OpenAI models needs a > Qodo-validated configuration for all enabled features. Use the exact Helm keys and Kubernetes resource names shown in the examples. These are the names shipped by the chart. ## Understand the two connections Qodo applications authenticate to the bundled gateway. The gateway then authenticates to your upstream endpoint or model provider. | Connection | Credential | Configuration owner | | - | - | - | | Qodo applications to bundled AI gateway | `qodo-llm-gateway-key`, key `LLM_GATEWAY_API_KEY` | The chart generates and reuses this internal credential for direct Helm installations | | AI gateway to the baseline GPT and embedding endpoint | `qodo-openai`, key `OPENAI_API_KEY` | You supply the upstream credential | | AI gateway to an additional provider | A provider Secret referenced under `litellm.env` | You supply the provider credential or workload identity | The chart connects applications to the bundled gateway automatically. Keep `global.llm.bundled: true`. Do not put a provider key in an application gateway-key setting. If Argo CD manages your installation, pre-provision the internal credentials through your release's external-secret procedure. For more information, see [Manage Qodo secrets with External Secrets Operator](https://docs.qodo.ai/on-prem/external-secrets.md). ### Match model names at both ends A gateway `model_name` is the name applications request. `litellm_params.model` identifies the model or deployment at the upstream provider. Merge removes one outer `openai/` prefix before sending a request. That prefix selects the gateway client. It does not identify the upstream provider. | Provider | Gateway `model_name` | Merge application value | Agentic review protocol | | - | - | - | - | | OpenAI | `openai/gpt-5.2` | `openai/openai/gpt-5.2` | Responses | | Azure OpenAI | `openai/gpt-5.2` | `openai/openai/gpt-5.2` | Responses | | Anthropic | `anthropic/claude-sonnet-4-6` | `openai/anthropic/claude-sonnet-4-6` | Anthropic Messages | | Bedrock Claude | `anthropic/claude-sonnet-4-6` | `openai/anthropic/claude-sonnet-4-6` | Anthropic Messages | | Vertex AI Gemini | `vertex_ai/gemini-2.5-pro` | `openai/vertex_ai/gemini-2.5-pro` | Chat Completions | | Gemini API | `gemini/gemini-2.5-pro` | `openai/gemini/gemini-2.5-pro` | Chat Completions | Keep recognizable model-family names. Mapping a GPT alias to Claude or Gemini can make applications send the wrong reasoning or structured-output parameters. Generic aliases can also remove model capability and context-window information. Merge treats `_thinking` as an application suffix and removes it before the request. The examples preserve this suffix for the main review agent, ensemble, and subagent stages. Verify the selected provider's reasoning parameters and token budget during application acceptance. Review modes such as `regular` and `turbo` are application selectors, not gateway model names. ## 1. Prepare the baseline routes Save this overlay as `llm-base-values.yaml`. Replace the example endpoint with your approved OpenAI-compatible endpoint. It must serve the GPT chat models and embeddings shown below. ```yaml llm-base-values.yaml theme={null} global: llm: bundled: true openaiBaseUrl: https://llm.example.com/v1 modelMap: qodo-main: openai/gpt-5.2 qodo-lite: openai/gpt-5.1 gpt-5.2: openai/gpt-5.2 openai/gpt-5.2: openai/gpt-5.2 gpt-5.1: openai/gpt-5.1 openai/gpt-5.1: openai/gpt-5.1 text-embedding-3-large: openai/text-embedding-3-large openai/text-embedding-3-large: openai/text-embedding-3-large ``` For direct OpenAI access, set `openaiBaseUrl: ""`. The gateway then uses the provider's default endpoint. Direct access requires outbound connectivity. Supply the endpoint credential in `qodo-openai`, under `OPENAI_API_KEY`. The Secret remains required when you add another review provider. > [!WARNING] > > A non-empty `modelMap` makes the chart generate routes from that map instead > of its built-in routes. Helm still merges map entries across values files. > Check the final rendered map. Apply this baseline with the application > overrides in the next section. > > `extraModelList` appends routes. Do not publish the same name in both > settings. Duplicate names create multiple upstream routes and can send > requests to the wrong provider. The baseline deliberately publishes both bare and prefixed GPT names. Keep `openai/` in Context Engine's `ONPREM_MODELS__*` settings, as shown below. Its model client removes that provider prefix before sending the gateway request. Platform also normalizes discovered model names. Merge uses the prefixed names in these examples. You can use `global.llm.extraHeaders` for non-secret attribution headers. Confirm the exact header name with your gateway administrator. These headers apply to all routes, including embeddings and additional providers. ## 2. Select the review provider Choose one provider tab. Save its complete overlay as `llm-provider-values.yaml`. Replace marked values before deploying. The model names illustrate routing. Confirm model access, deployment names, capacity, and review suitability with Qodo and your provider administrator. All ten Merge settings are included. Changing only `CONFIG__MODEL` leaves other review stages, fallbacks, or subagents on their previous models. List-valued settings must remain JSON strings. Choose a fallback model different from the primary model. These examples use the fast model as the fallback. > [tab] OpenAI > > Use this overlay with the baseline routes. It selects GPT models for > every review stage. For direct access, use the empty endpoint setting > described above. > > ```yaml llm-provider-values.yaml theme={null} > global: > merge: > settings: > CONFIG__MODEL: openai/openai/gpt-5.2 > CONFIG__MODEL_TURBO: openai/openai/gpt-5.1 > CONFIG__MODEL_REASONING: openai/openai/gpt-5.2 > CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/openai/gpt-5.2 > CONFIG__FALLBACK_MODELS: '["openai/openai/gpt-5.1"]' > REVIEW_AGENT__LLM_MODEL: openai/openai/gpt-5.2_thinking > REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/openai/gpt-5.2 > REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/openai/gpt-5.2_thinking", "openai/openai/gpt-5.1_thinking"]' > REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/openai/gpt-5.1 > REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/openai/gpt-5.1_thinking > ``` > > The chart enables `OPENAI__RESPONSE_API: "true"`. Verify Responses, > streaming, and tool calls through your upstream endpoint. A service > that implements only Chat Completions does not satisfy this configuration. > > For more information, see [OpenAI-compatible endpoints](https://docs.litellm.ai/docs/providers/openai_compatible). > [tab] Azure OpenAI > > This overlay sends the baseline chat and embedding routes to > Azure OpenAI resource deployments. Replace the resource endpoint, deployment > names, and API version. Deployment names can differ from model names. > > Store the Azure resource key in `qodo-openai`, under `OPENAI_API_KEY`. > Every route below uses that resource and credential. The chart writes > `api_key: os.environ/OPENAI_API_KEY` into each generated `modelMap` > route, including Azure routes. The variable name does not change the > credential type. > > ```yaml llm-provider-values.yaml theme={null} > global: > llm: > openaiBaseUrl: https://REPLACE_WITH_RESOURCE.openai.azure.com > modelMap: > qodo-main: azure/gpt-5.2-REPLACE_WITH_DEPLOYMENT_SUFFIX > qodo-lite: azure/gpt-5.1-REPLACE_WITH_DEPLOYMENT_SUFFIX > gpt-5.2: azure/gpt-5.2-REPLACE_WITH_DEPLOYMENT_SUFFIX > openai/gpt-5.2: azure/gpt-5.2-REPLACE_WITH_DEPLOYMENT_SUFFIX > gpt-5.1: azure/gpt-5.1-REPLACE_WITH_DEPLOYMENT_SUFFIX > openai/gpt-5.1: azure/gpt-5.1-REPLACE_WITH_DEPLOYMENT_SUFFIX > text-embedding-3-large: azure/text-embedding-3-large-REPLACE_WITH_DEPLOYMENT_SUFFIX > openai/text-embedding-3-large: azure/text-embedding-3-large-REPLACE_WITH_DEPLOYMENT_SUFFIX > merge: > settings: > CONFIG__MODEL: openai/openai/gpt-5.2 > CONFIG__MODEL_TURBO: openai/openai/gpt-5.1 > CONFIG__MODEL_REASONING: openai/openai/gpt-5.2 > CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/openai/gpt-5.2 > CONFIG__FALLBACK_MODELS: '["openai/openai/gpt-5.1"]' > REVIEW_AGENT__LLM_MODEL: openai/openai/gpt-5.2_thinking > REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/openai/gpt-5.2 > REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/openai/gpt-5.2_thinking", "openai/openai/gpt-5.1_thinking"]' > REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/openai/gpt-5.1 > REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/openai/gpt-5.1_thinking > litellm: > env: > AZURE_API_VERSION: > value: REPLACE_WITH_SUPPORTED_API_VERSION > ``` > > Use GPT deployments that match the advertised model families. > These examples require chat deployment names containing `gpt-5` and an > embedding deployment name containing `text-embedding-3`. Preserve those > parts when replacing the placeholders. The gateway uses model names to > select parameter handling. For more information, see > [Azure GPT-5 model routing](https://docs.litellm.ai/docs/providers/azure/#gpt-5-models). > If your naming policy differs, obtain a validated configuration for > those deployment names. > The embedding deployment must support both dimensions specified below. > Verify that your selected API version supports Responses and the > required reasoning parameters. Keep the Azure resource root here; > do not append a deployment path or `/chat/completions`. > > If your organization already exposes Azure through an OpenAI-compatible > gateway, use that gateway's endpoint and identifiers in the baseline > instead. Do not combine its URL format with this Azure OpenAI configuration. > > For more information, see [Azure OpenAI](https://docs.litellm.ai/docs/providers/azure/) > and [Azure embeddings](https://docs.litellm.ai/docs/providers/azure/azure_embedding). > [tab] Anthropic > > Keep the baseline GPT and embedding routes. Add the two native Anthropic routes below. Create `qodo-anthropic` with the key `ANTHROPIC_API_KEY`. > > ```yaml llm-provider-values.yaml theme={null} > global: > llm: > extraModelList: > - model_name: anthropic/claude-sonnet-4-6 > litellm_params: > model: anthropic/claude-sonnet-4-6 > api_base: https://api.anthropic.com > api_key: os.environ/ANTHROPIC_API_KEY > cache_control_injection_points: > - location: message > index: -1 > - model_name: anthropic/claude-haiku-4-5 > litellm_params: > model: anthropic/claude-haiku-4-5 > api_base: https://api.anthropic.com > api_key: os.environ/ANTHROPIC_API_KEY > cache_control_injection_points: > - location: message > index: -1 > merge: > settings: > CONFIG__MODEL: openai/anthropic/claude-sonnet-4-6 > CONFIG__MODEL_TURBO: openai/anthropic/claude-haiku-4-5 > CONFIG__MODEL_REASONING: openai/anthropic/claude-sonnet-4-6 > CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/anthropic/claude-sonnet-4-6 > CONFIG__FALLBACK_MODELS: '["openai/anthropic/claude-haiku-4-5"]' > REVIEW_AGENT__LLM_MODEL: openai/anthropic/claude-sonnet-4-6_thinking > REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/anthropic/claude-sonnet-4-6 > REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/anthropic/claude-sonnet-4-6_thinking", "openai/anthropic/claude-haiku-4-5_thinking"]' > REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/anthropic/claude-haiku-4-5 > REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/anthropic/claude-haiku-4-5_thinking > litellm: > env: > ANTHROPIC_API_KEY: > valueFrom: > secretKeyRef: > name: qodo-anthropic > key: ANTHROPIC_API_KEY > ``` > > Agentic Claude reviews use the gateway's Anthropic Messages endpoint. Classic review paths can also use Chat Completions. Test both paths. The application continues to use the gateway credential. > > For more information, see [Anthropic](https://docs.litellm.ai/docs/providers/anthropic). > [tab] Amazon Bedrock > > Keep the baseline GPT and embedding routes. These routes use Claude through Bedrock. Replace each model or inference-profile ID with an identifier enabled in your account and region. The application-facing aliases remain Anthropic names. > > ```yaml llm-provider-values.yaml theme={null} > global: > llm: > extraModelList: > - model_name: anthropic/claude-sonnet-4-6 > litellm_params: > model: bedrock/REPLACE_WITH_SONNET_MODEL_OR_PROFILE_ID > api_base: https://bedrock-runtime.us-east-1.amazonaws.com > aws_region_name: us-east-1 > api_key: os.environ/AWS_BEARER_TOKEN_BEDROCK > cache_control_injection_points: > - location: message > index: -1 > - model_name: anthropic/claude-haiku-4-5 > litellm_params: > model: bedrock/REPLACE_WITH_HAIKU_MODEL_OR_PROFILE_ID > api_base: https://bedrock-runtime.us-east-1.amazonaws.com > aws_region_name: us-east-1 > api_key: os.environ/AWS_BEARER_TOKEN_BEDROCK > cache_control_injection_points: > - location: message > index: -1 > merge: > settings: > CONFIG__MODEL: openai/anthropic/claude-sonnet-4-6 > CONFIG__MODEL_TURBO: openai/anthropic/claude-haiku-4-5 > CONFIG__MODEL_REASONING: openai/anthropic/claude-sonnet-4-6 > CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/anthropic/claude-sonnet-4-6 > CONFIG__FALLBACK_MODELS: '["openai/anthropic/claude-haiku-4-5"]' > REVIEW_AGENT__LLM_MODEL: openai/anthropic/claude-sonnet-4-6_thinking > REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/anthropic/claude-sonnet-4-6 > REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/anthropic/claude-sonnet-4-6_thinking", "openai/anthropic/claude-haiku-4-5_thinking"]' > REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/anthropic/claude-haiku-4-5 > REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/anthropic/claude-haiku-4-5_thinking > litellm: > env: > AWS_BEARER_TOKEN_BEDROCK: > valueFrom: > secretKeyRef: > name: qodo-bedrock > key: BEARER_TOKEN_BEDROCK > ``` > > This example uses a Bedrock API key. Create `qodo-bedrock` with the key `BEARER_TOKEN_BEDROCK`. Match the runtime endpoint and region. Check model access and inference-profile permissions. For IAM authentication, remove each `api_key` field and configure the AI gateway workload's approved AWS credential chain. Do not add AWS credentials to Qodo application containers. Test Messages and Chat Completions. > > For more information, see [Bedrock](https://docs.litellm.ai/docs/providers/bedrock). > [tab] Vertex AI > > Keep the baseline GPT and embedding routes. Enable Vertex AI and authorize the AI gateway workload to invoke the selected Gemini models. This example mounts a credential file from `qodo-vertex`, under the key `credentials.json`. > > ```yaml llm-provider-values.yaml theme={null} > global: > llm: > extraModelList: > - model_name: vertex_ai/gemini-2.5-pro > litellm_params: > model: vertex_ai/gemini-2.5-pro > api_base: https://us-central1-aiplatform.googleapis.com > vertex_project: REPLACE_WITH_PROJECT_ID > vertex_location: us-central1 > vertex_credentials: /etc/vertex/credentials.json > - model_name: vertex_ai/gemini-2.5-flash > litellm_params: > model: vertex_ai/gemini-2.5-flash > api_base: https://us-central1-aiplatform.googleapis.com > vertex_project: REPLACE_WITH_PROJECT_ID > vertex_location: us-central1 > vertex_credentials: /etc/vertex/credentials.json > merge: > settings: > CONFIG__MODEL: openai/vertex_ai/gemini-2.5-pro > CONFIG__MODEL_TURBO: openai/vertex_ai/gemini-2.5-flash > CONFIG__MODEL_REASONING: openai/vertex_ai/gemini-2.5-pro > CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/vertex_ai/gemini-2.5-pro > CONFIG__FALLBACK_MODELS: '["openai/vertex_ai/gemini-2.5-flash"]' > REVIEW_AGENT__LLM_MODEL: openai/vertex_ai/gemini-2.5-pro_thinking > REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/vertex_ai/gemini-2.5-pro > REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/vertex_ai/gemini-2.5-pro_thinking", "openai/vertex_ai/gemini-2.5-flash_thinking"]' > REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/vertex_ai/gemini-2.5-flash > REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/vertex_ai/gemini-2.5-flash_thinking > litellm: > volumeMounts: > vertex-credentials: > mountPath: /etc/vertex > readOnly: true > volumes: > vertex-credentials: > secret: > secretName: qodo-vertex > ``` > > Use an approved Workload Identity Federation credential configuration when available. A service-account JSON file is another authentication option. Keep credentials in the Secret. Match the project, model availability, region, and endpoint. A Gemini API key does not configure Vertex AI authentication. > > For more information, see [Vertex AI](https://docs.litellm.ai/docs/providers/vertex). > [tab] Gemini API > > Keep the baseline GPT and embedding routes. This configuration uses the Gemini API with an API key. Create `qodo-gemini` with the key `GEMINI_API_KEY`. > > ```yaml llm-provider-values.yaml theme={null} > global: > llm: > extraModelList: > - model_name: gemini/gemini-2.5-pro > litellm_params: > model: gemini/gemini-2.5-pro > api_base: https://generativelanguage.googleapis.com/v1beta > api_key: os.environ/GEMINI_API_KEY > - model_name: gemini/gemini-2.5-flash > litellm_params: > model: gemini/gemini-2.5-flash > api_base: https://generativelanguage.googleapis.com/v1beta > api_key: os.environ/GEMINI_API_KEY > merge: > settings: > CONFIG__MODEL: openai/gemini/gemini-2.5-pro > CONFIG__MODEL_TURBO: openai/gemini/gemini-2.5-flash > CONFIG__MODEL_REASONING: openai/gemini/gemini-2.5-pro > CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/gemini/gemini-2.5-pro > CONFIG__FALLBACK_MODELS: '["openai/gemini/gemini-2.5-flash"]' > REVIEW_AGENT__LLM_MODEL: openai/gemini/gemini-2.5-pro_thinking > REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/gemini/gemini-2.5-pro > REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/gemini/gemini-2.5-pro_thinking", "openai/gemini/gemini-2.5-flash_thinking"]' > REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/gemini/gemini-2.5-flash > REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/gemini/gemini-2.5-flash_thinking > litellm: > env: > GEMINI_API_KEY: > valueFrom: > secretKeyRef: > name: qodo-gemini > key: GEMINI_API_KEY > ``` > > Merge sends Chat Completions to the AI gateway. The gateway translates the request for Gemini. This configuration does not use Vertex AI project or workload-identity settings. > > Keep `api_base: https://generativelanguage.googleapis.com/v1beta` for these > routes. They do not inherit `global.llm.openaiBaseUrl`. For more information, > see [Gemini API reference](https://ai.google.dev/api/generate-content). > > For more information, see [Gemini](https://docs.litellm.ai/docs/providers/gemini). The Claude routes include prompt-cache injection at the last message boundary. Cache use depends on provider requirements. Verify cache reads in usage records. For more information, see [prompt caching](https://docs.litellm.ai/docs/completion/prompt_caching). ## 3. Preserve the other application routes The chart supplies the base URLs and internal gateway key for the applications below. The review overrides above do not change these settings. | Application path | Application setting or model value | Gateway alias required | | - | - | - | | Context Engine `nano` and `lite` | `ONPREM_MODELS__NANO__MODEL` and `ONPREM_MODELS__LITE__MODEL`: `openai/gpt-5.1` | `gpt-5.1` | | Context Engine `main` | `ONPREM_MODELS__MAIN__MODEL`: `openai/gpt-5.2` | `gpt-5.2` | | Context Engine embeddings | `ONPREM_MODELS__REMOTE_RAG__MODEL`: `openai/text-embedding-3-large` | `text-embedding-3-large` | | Context Engine agent paths | `DEEP_ASK_RESEARCH_PREVIEW__LLM_MODEL` and `PR_KNOWLEDGE__LLM_MODEL`: `gpt-5.2` | `gpt-5.2` | | Platform rule processing | Model settings include `gpt-5.2`, `gpt-5.1`, and `openai:gpt-5.2` | `gpt-5.2` and `gpt-5.1` | | Platform rule similarity | `text-embedding-3-large` | `text-embedding-3-large` | Platform agent settings use `provider:model` syntax, such as `openai:gpt-5.2`. This differs from Merge's slash-separated client routing. Do not copy one application's value into another application's setting. The chart selects Context Engine's on-prem model configuration. `AI_GATEWAY__*` settings belong to a different Engine configuration and do not replace the `ONPREM_MODELS__*` settings shown above. Platform uses `DYNACONF_LLM_GATEWAY__*`, which the chart supplies. ### Keep embedding dimensions compatible The same embedding route must honor both requests: | Consumer | Requested dimensions | | - | - | | Context Engine PR Knowledge | `2000` | | Platform rule similarity | `1024` | Keep `text-embedding-3` in the embedding alias. The gateway's model handling uses this name when forwarding the `dimensions` parameter. For more information, see [embedding parameter handling](https://github.com/BerriAI/litellm/blob/v1.105.5/litellm/utils.py#L3962). An arbitrary embedding model is not a compatible replacement. Do not change dimensions, embedding models, or stored indexes independently. Coordinate such changes with Qodo before migrating existing data. ### Include SDK agent models SDK and CLI agent topologies can request additional gateway aliases. Review their configured models before enabling those workflows. The packaged diagnostic agents request `anthropic/claude-haiku-4-5`. The Anthropic and Bedrock examples above publish this alias. The OpenAI, Azure, Vertex AI, and Gemini API examples do not add that diagnostic route. If you use those agents, add an approved Claude route to the same `extraModelList` before invoking them. A successful runtime HTTP health check does not verify an agent's model route. Custom agent topologies can require other models too. ## 4. Create provider Secrets and apply the overlays Create Secrets in the installation namespace before deploying Qodo. Use your approved secret manager or ESO when available. For more information, see [Manage Qodo secrets with External Secrets Operator](https://docs.qodo.ai/on-prem/external-secrets.md). For a new single-key Secret, this Bash example keeps the credential out of the command text. Select your installation's Kubernetes context and namespace. For an existing installation, retain its Helm release name. Replace the context placeholder, then change the Secret name and key for your provider: ```bash theme={null} KUBE_CONTEXT=REPLACE_WITH_KUBERNETES_CONTEXT NAMESPACE=qodo RELEASE=qodo kubectl config get-contexts "$KUBE_CONTEXT" PROVIDER_SECRET=qodo-anthropic PROVIDER_SECRET_KEY=ANTHROPIC_API_KEY set +x read -r -s -p "Provider credential: " PROVIDER_CREDENTIAL printf '\n' : "${PROVIDER_CREDENTIAL:?Provider credential is required}" printf '%s' "$PROVIDER_CREDENTIAL" | kubectl --context "$KUBE_CONTEXT" -n "$NAMESPACE" \ create secret generic "$PROVIDER_SECRET" \ --from-file="$PROVIDER_SECRET_KEY=/dev/stdin" unset PROVIDER_CREDENTIAL ``` For `qodo-openai`, use the Secret key `OPENAI_API_KEY`. For Vertex AI, import the approved credential file as `qodo-vertex` with the key `credentials.json`. Do not place credentials in values files or commit them to Git. Use the chart and installation procedure supplied with your release. Add the overlays after your existing customer values: ```bash theme={null} helm template "$RELEASE" ./qodo-onprem \ --kube-context "$KUBE_CONTEXT" \ --namespace "$NAMESPACE" \ -f onprem-base.yaml \ -f my-values.yaml \ -f llm-base-values.yaml \ -f llm-provider-values.yaml > rendered.yaml ``` Set `RELEASE` to your existing release name. Preserve that name on upgrades. Treat rendered manifests as sensitive because they can contain Secrets. Inspect the generated model names, upstream endpoints, credential references, and `qodo-merge-custom-env`. Check that every selected Merge model has an exact gateway alias after removing its outer `openai/` prefix and trailing `_thinking` suffix. Helm merges maps but replaces lists. Keep all selected provider routes in one `extraModelList`. A later overlay with that list replaces earlier entries. Use the same values order for readiness checks and deployment. For more information, see [Deploy Qodo using Helm](https://docs.qodo.ai/on-prem/qodo-on-prem-helm-chart-overview.md). If the gateway's upstream endpoint uses a private CA, obtain a supported gateway trust configuration from Qodo before connecting it. The standard `custom-ca-cert` Secret configures covered Qodo applications, but excludes bundled AI gateway. For more information, see [Private certificate authority](https://docs.qodo.ai/on-prem/qodo-on-prem-private-certificate-authority.md#what-the-chart-covers). ## 5. Verify model access and application behavior A healthy Pod or a successful `/models` response does not verify inference, tool calls, or embeddings. First, wait for the gateway and review workloads to complete their rollouts. For a local diagnostic connection, set the same context and namespace in a separate terminal, then open the connection: ```bash theme={null} KUBE_CONTEXT=REPLACE_WITH_KUBERNETES_CONTEXT NAMESPACE=qodo kubectl --context "$KUBE_CONTEXT" -n "$NAMESPACE" \ port-forward service/qodo-litellm 14000:4000 ``` The following requests use the local forwarded port. Enter the internal gateway key from your approved secret store. Do not enter your provider key. Run these commands in Bash. The header uses a separate file descriptor to keep the key out of curl process arguments: ```bash theme={null} GATEWAY_URL=http://localhost:14000 set +x read -r -s -p "Internal gateway key: " LLM_GATEWAY_API_KEY printf '\n' : "${LLM_GATEWAY_API_KEY:?Gateway key is required}" curl --fail-with-body --silent --show-error \ "$GATEWAY_URL/v1/models" \ --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") | jq -r '.data[].id' ``` Confirm that the baseline aliases and both selected provider aliases appear. > [accordion] Test Chat Completions and tool calls > > Use this test for the selected review alias. It also covers the > Chat Completions path used by classic reviews and SDK agents. > Replace the example with your gateway alias, without Merge's outer prefix. > > ```bash theme={null} > REVIEW_MODEL=vertex_ai/gemini-2.5-pro > jq -n --arg model "$REVIEW_MODEL" '{ > model: $model, > messages: [{role: "user", content: "Call report_status with status ready."}], > tools: [{ > type: "function", > function: { > name: "report_status", > description: "Report the connection test status.", > parameters: { > type: "object", > properties: {status: {type: "string", enum: ["ready"]}}, > required: ["status"] > } > } > }], > tool_choice: {type: "function", function: {name: "report_status"}} > }' | > curl --fail-with-body --silent --show-error \ > "$GATEWAY_URL/v1/chat/completions" \ > --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \ > -H "Content-Type: application/json" --data-binary @- | > jq -e '.choices[0].message.tool_calls[0].function.name == "report_status"' > ``` > > Expect `true`. Repeat for the fast alias. > Verify streaming and a subsequent tool-result turn in an actual review. > [accordion] Test Responses for OpenAI and Azure GPT routes > > Use the advertised GPT alias. Keep this check even when your review > provider is Claude or Gemini, because Platform has its own GPT agent paths. > > ```bash theme={null} > curl --fail-with-body --silent --show-error \ > "$GATEWAY_URL/v1/responses" \ > --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \ > -H "Content-Type: application/json" \ > -d '{ > "model": "openai/gpt-5.2", > "input": "Reply with ready.", > "reasoning": {"effort": "medium"}, > "include": ["reasoning.encrypted_content"] > }' | > jq -e '.status == "completed"' > ``` > > Repeat with `openai/gpt-5.1` for the fast review model. > Test both `medium` and `low` reasoning effort for the enabled review stages. > Also test the bare `gpt-5.2` and `gpt-5.1` aliases used by other applications. > A successful Chat Completions request does not establish Responses support. > > `OPENAI__RESPONSE_API` controls Merge's GPT path. > Setting it to `"false"` does not disable Platform's independent > Responses selection. > [accordion] Test Anthropic Messages for Claude routes > > Agentic Claude reviews use Messages. The Bedrock example exposes the > same alias through the AI gateway, which translates the upstream request. > > ```bash theme={null} > curl --fail-with-body --silent --show-error \ > "$GATEWAY_URL/v1/messages" \ > --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \ > -H "anthropic-version: 2023-06-01" \ > -H "Content-Type: application/json" \ > -d '{ > "model": "anthropic/claude-sonnet-4-6", > "max_tokens": 64, > "messages": [{"role": "user", "content": "Reply with ready."}] > }' | > jq -e '.type == "message" and (.content | length > 0)' > ``` > > Repeat with `anthropic/claude-haiku-4-5`. > Verify reasoning, tool calls, structured findings, and streaming in an actual agentic review. > [accordion] Test both embedding dimensions > > Run both requests through the shared embedding route. > The returned vector length must match each requested dimension. > > ```bash theme={null} > ( > set -o pipefail > for DIMENSIONS in 2000 1024; do > jq -n --argjson dimensions "$DIMENSIONS" '{ > model: "text-embedding-3-large", > input: "Model connection test", > dimensions: $dimensions > }' | > curl --fail-with-body --silent --show-error \ > "$GATEWAY_URL/v1/embeddings" \ > --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \ > -H "Content-Type: application/json" --data-binary @- | > jq -e --argjson expected "$DIMENSIONS" \ > '(.data[0].embedding | length) == $expected' || exit 1 > done > ) > ``` > > Expect `true` twice. A missing or incorrect vector length requires > investigation before indexing or rule processing. After the gateway checks, run `unset LLM_GATEWAY_API_KEY`. Finally, complete the application checks: 1. Run a normal review against a representative Pull Request (PR). 2. Run the review modes and fallback paths enabled for your installation. 3. Confirm that structured findings appear in the Git provider and portal. 4. Index a repository and verify Context Engine retrieval. 5. Verify rule ingestion and rule similarity. 6. Invoke each enabled SDK agent and verify its model route. 7. Confirm the selected upstream models in your gateway's routing or usage records. Record which providers, protocols, and workflows passed. Do not treat a gateway smoke test as complete application validation. ### Apply configuration updates and rotate credentials Apply changes through your existing Helm or GitOps release process. Confirm that the gateway loads the new model configuration. A Secret update alone does not restart Pods or refresh environment variables. When rotating a provider credential, wait for Secret synchronization, restart the gateway through your deployment process, and repeat inference checks. If the internal gateway key changes, every application consumer must receive the same replacement key. For the complete rotation procedure, see [Manage Qodo secrets with External Secrets Operator](https://docs.qodo.ai/on-prem/external-secrets.md#verify-rotation-and-recovery). ## Troubleshoot model access | Symptom | Check and next action | | - | - | | `CreateContainerConfigError` | Check Secret names, namespaces, and case-sensitive keys, including `qodo-openai` | | Gateway returns `401` | Check the internal gateway key used by the application or diagnostic request | | Upstream returns `401` or `403` | Check the provider credential, workload identity, model access, region, and profile permissions | | Model is missing or gateway returns `400` | Compare the actual request name with `model_name`; check replacement maps, list overrides, and stripped prefixes | | Some requests reach the wrong provider | Remove unintended duplicate aliases across `modelMap` and `extraModelList` | | Reviews still use a previous model | Check all ten Merge settings and completed application rollouts | | Chat works but agentic review fails | Test the required Responses or Messages endpoint, streaming, tool calls, and structured output | | Embeddings return the wrong vector length | Preserve the embedding alias and verify both requested dimensions | | A diagnostic SDK agent fails while runtime health passes | Publish the exact alias from that agent's topology and test its invocation | | TLS verification fails | Identify which connection failed. The AI gateway needs its own supported upstream CA configuration; the application CA Secret does not cover it | | Provider throttles or times out | Check quotas, deployment capacity, network access, and upstream latency before changing timeouts | ## Related resources * [Deploy Qodo using Helm](https://docs.qodo.ai/on-prem/qodo-on-prem-helm-chart-overview.md): Apply the release's installation and upgrade procedure. * [Helm chart configuration reference](https://docs.qodo.ai/on-prem/on-prem-helm-chart-reference.md): Review the accepted values. * [Manage Qodo secrets with External Secrets Operator](https://docs.qodo.ai/on-prem/external-secrets.md): Provision and rotate provider credentials. * [Monitor Qodo with Prometheus and Grafana](https://docs.qodo.ai/on-prem/monitoring.md): Observe workloads after configuration changes. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.