Configure the AI gateway

On-premises Configure model access before you install Qodo. Connect the AI gateway to your model providers, then match Qodo’s application settings to the gateway’s model names. The examples configure review models for OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Vertex AI, and the Gemini API. They retain GPT chat and OpenAI-compatible embedding routes for Platform and Context Engine.

Selecting a provider for code review does not move every Qodo feature to that provider. Platform, Context Engine, and SDK agents have separate model requirements. A deployment that must exclude OpenAI models needs a Qodo-validated configuration for all enabled features.

Use the exact Helm keys and Kubernetes resource names shown in the examples. These are the names shipped by the chart.

Understand the two connections

Qodo applications authenticate to the bundled gateway. The gateway then authenticates to your upstream endpoint or model provider.

Connection	Credential	Configuration owner
Qodo applications to bundled AI gateway	`qodo-llm-gateway-key`, key `LLM_GATEWAY_API_KEY`	The chart generates and reuses this internal credential for direct Helm installations
AI gateway to the baseline GPT and embedding endpoint	`qodo-openai`, key `OPENAI_API_KEY`	You supply the upstream credential
AI gateway to an additional provider	A provider Secret referenced under `litellm.env`	You supply the provider credential or workload identity

The chart connects applications to the bundled gateway automatically. Keep global.llm.bundled: true. Do not put a provider key in an application gateway-key setting. If Argo CD manages your installation, pre-provision the internal credentials through your release’s external-secret procedure. For more information, see Manage Qodo secrets with External Secrets Operator.

Match model names at both ends

A gateway model_name is the name applications request. litellm_params.model identifies the model or deployment at the upstream provider. Merge removes one outer openai/ prefix before sending a request. That prefix selects the gateway client. It does not identify the upstream provider.

Provider	Gateway `model_name`	Merge application value	Agentic review protocol
OpenAI	`openai/gpt-5.2`	`openai/openai/gpt-5.2`	Responses
Azure OpenAI	`openai/gpt-5.2`	`openai/openai/gpt-5.2`	Responses
Anthropic	`anthropic/claude-sonnet-4-6`	`openai/anthropic/claude-sonnet-4-6`	Anthropic Messages
Bedrock Claude	`anthropic/claude-sonnet-4-6`	`openai/anthropic/claude-sonnet-4-6`	Anthropic Messages
Vertex AI Gemini	`vertex_ai/gemini-2.5-pro`	`openai/vertex_ai/gemini-2.5-pro`	Chat Completions
Gemini API	`gemini/gemini-2.5-pro`	`openai/gemini/gemini-2.5-pro`	Chat Completions

Keep recognizable model-family names. Mapping a GPT alias to Claude or Gemini can make applications send the wrong reasoning or structured-output parameters. Generic aliases can also remove model capability and context-window information. Merge treats _thinking as an application suffix and removes it before the request. The examples preserve this suffix for the main review agent, ensemble, and subagent stages. Verify the selected provider’s reasoning parameters and token budget during application acceptance. Review modes such as regular and turbo are application selectors, not gateway model names.

1. Prepare the baseline routes

Save this overlay as llm-base-values.yaml. Replace the example endpoint with your approved OpenAI-compatible endpoint. It must serve the GPT chat models and embeddings shown below.

llm-base-values.yaml

global:
  llm:
    bundled: true
    openaiBaseUrl: https://llm.example.com/v1
    modelMap:
      qodo-main: openai/gpt-5.2
      qodo-lite: openai/gpt-5.1
      gpt-5.2: openai/gpt-5.2
      openai/gpt-5.2: openai/gpt-5.2
      gpt-5.1: openai/gpt-5.1
      openai/gpt-5.1: openai/gpt-5.1
      text-embedding-3-large: openai/text-embedding-3-large
      openai/text-embedding-3-large: openai/text-embedding-3-large

For direct OpenAI access, set openaiBaseUrl: "". The gateway then uses the provider’s default endpoint. Direct access requires outbound connectivity. Supply the endpoint credential in qodo-openai, under OPENAI_API_KEY. The Secret remains required when you add another review provider.

A non-empty modelMap makes the chart generate routes from that map instead of its built-in routes. Helm still merges map entries across values files. Check the final rendered map. Apply this baseline with the application overrides in the next section.extraModelList appends routes. Do not publish the same name in both settings. Duplicate names create multiple upstream routes and can send requests to the wrong provider.

The baseline deliberately publishes both bare and prefixed GPT names. Keep openai/ in Context Engine’s ONPREM_MODELS__* settings, as shown below. Its model client removes that provider prefix before sending the gateway request. Platform also normalizes discovered model names. Merge uses the prefixed names in these examples. You can use global.llm.extraHeaders for non-secret attribution headers. Confirm the exact header name with your gateway administrator. These headers apply to all routes, including embeddings and additional providers.

2. Select the review provider

Choose one provider tab. Save its complete overlay as llm-provider-values.yaml. Replace marked values before deploying. The model names illustrate routing. Confirm model access, deployment names, capacity, and review suitability with Qodo and your provider administrator. All ten Merge settings are included. Changing only CONFIG__MODEL leaves other review stages, fallbacks, or subagents on their previous models. List-valued settings must remain JSON strings. Choose a fallback model different from the primary model. These examples use the fast model as the fallback.

Use this overlay with the baseline routes. It selects GPT models for every review stage. For direct access, use the empty endpoint setting described above.

llm-provider-values.yaml

global:
  merge:
    settings:
      CONFIG__MODEL: openai/openai/gpt-5.2
      CONFIG__MODEL_TURBO: openai/openai/gpt-5.1
      CONFIG__MODEL_REASONING: openai/openai/gpt-5.2
      CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/openai/gpt-5.2
      CONFIG__FALLBACK_MODELS: '["openai/openai/gpt-5.1"]'
      REVIEW_AGENT__LLM_MODEL: openai/openai/gpt-5.2_thinking
      REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/openai/gpt-5.2
      REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/openai/gpt-5.2_thinking", "openai/openai/gpt-5.1_thinking"]'
      REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/openai/gpt-5.1
      REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/openai/gpt-5.1_thinking

The chart enables OPENAI__RESPONSE_API: "true". Verify Responses, streaming, and tool calls through your upstream endpoint. A service that implements only Chat Completions does not satisfy this configuration.For more information, see OpenAI-compatible endpoints.

This overlay sends the baseline chat and embedding routes to Azure OpenAI resource deployments. Replace the resource endpoint, deployment names, and API version. Deployment names can differ from model names.Store the Azure resource key in qodo-openai, under OPENAI_API_KEY. Every route below uses that resource and credential. The chart writes api_key: os.environ/OPENAI_API_KEY into each generated modelMap route, including Azure routes. The variable name does not change the credential type.

llm-provider-values.yaml

global:
  llm:
    openaiBaseUrl: https://REPLACE_WITH_RESOURCE.openai.azure.com
    modelMap:
      qodo-main: azure/gpt-5.2-REPLACE_WITH_DEPLOYMENT_SUFFIX
      qodo-lite: azure/gpt-5.1-REPLACE_WITH_DEPLOYMENT_SUFFIX
      gpt-5.2: azure/gpt-5.2-REPLACE_WITH_DEPLOYMENT_SUFFIX
      openai/gpt-5.2: azure/gpt-5.2-REPLACE_WITH_DEPLOYMENT_SUFFIX
      gpt-5.1: azure/gpt-5.1-REPLACE_WITH_DEPLOYMENT_SUFFIX
      openai/gpt-5.1: azure/gpt-5.1-REPLACE_WITH_DEPLOYMENT_SUFFIX
      text-embedding-3-large: azure/text-embedding-3-large-REPLACE_WITH_DEPLOYMENT_SUFFIX
      openai/text-embedding-3-large: azure/text-embedding-3-large-REPLACE_WITH_DEPLOYMENT_SUFFIX
  merge:
    settings:
      CONFIG__MODEL: openai/openai/gpt-5.2
      CONFIG__MODEL_TURBO: openai/openai/gpt-5.1
      CONFIG__MODEL_REASONING: openai/openai/gpt-5.2
      CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/openai/gpt-5.2
      CONFIG__FALLBACK_MODELS: '["openai/openai/gpt-5.1"]'
      REVIEW_AGENT__LLM_MODEL: openai/openai/gpt-5.2_thinking
      REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/openai/gpt-5.2
      REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/openai/gpt-5.2_thinking", "openai/openai/gpt-5.1_thinking"]'
      REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/openai/gpt-5.1
      REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/openai/gpt-5.1_thinking
litellm:
  env:
    AZURE_API_VERSION:
      value: REPLACE_WITH_SUPPORTED_API_VERSION

Use GPT deployments that match the advertised model families. These examples require chat deployment names containing gpt-5 and an embedding deployment name containing text-embedding-3. Preserve those parts when replacing the placeholders. The gateway uses model names to select parameter handling. For more information, see Azure GPT-5 model routing. If your naming policy differs, obtain a validated configuration for those deployment names. The embedding deployment must support both dimensions specified below. Verify that your selected API version supports Responses and the required reasoning parameters. Keep the Azure resource root here; do not append a deployment path or /chat/completions.If your organization already exposes Azure through an OpenAI-compatible gateway, use that gateway’s endpoint and identifiers in the baseline instead. Do not combine its URL format with this Azure OpenAI configuration.For more information, see Azure OpenAI and Azure embeddings.

Keep the baseline GPT and embedding routes. Add the two native Anthropic routes below. Create qodo-anthropic with the key ANTHROPIC_API_KEY.

llm-provider-values.yaml

global:
  llm:
    extraModelList:
      - model_name: anthropic/claude-sonnet-4-6
        litellm_params:
          model: anthropic/claude-sonnet-4-6
          api_base: https://api.anthropic.com
          api_key: os.environ/ANTHROPIC_API_KEY
          cache_control_injection_points:
            - location: message
              index: -1
      - model_name: anthropic/claude-haiku-4-5
        litellm_params:
          model: anthropic/claude-haiku-4-5
          api_base: https://api.anthropic.com
          api_key: os.environ/ANTHROPIC_API_KEY
          cache_control_injection_points:
            - location: message
              index: -1
  merge:
    settings:
      CONFIG__MODEL: openai/anthropic/claude-sonnet-4-6
      CONFIG__MODEL_TURBO: openai/anthropic/claude-haiku-4-5
      CONFIG__MODEL_REASONING: openai/anthropic/claude-sonnet-4-6
      CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/anthropic/claude-sonnet-4-6
      CONFIG__FALLBACK_MODELS: '["openai/anthropic/claude-haiku-4-5"]'
      REVIEW_AGENT__LLM_MODEL: openai/anthropic/claude-sonnet-4-6_thinking
      REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/anthropic/claude-sonnet-4-6
      REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/anthropic/claude-sonnet-4-6_thinking", "openai/anthropic/claude-haiku-4-5_thinking"]'
      REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/anthropic/claude-haiku-4-5
      REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/anthropic/claude-haiku-4-5_thinking
litellm:
  env:
    ANTHROPIC_API_KEY:
      valueFrom:
        secretKeyRef:
          name: qodo-anthropic
          key: ANTHROPIC_API_KEY

Agentic Claude reviews use the gateway’s Anthropic Messages endpoint. Classic review paths can also use Chat Completions. Test both paths. The application continues to use the gateway credential.For more information, see Anthropic.

Keep the baseline GPT and embedding routes. These routes use Claude through Bedrock. Replace each model or inference-profile ID with an identifier enabled in your account and region. The application-facing aliases remain Anthropic names.

llm-provider-values.yaml

global:
  llm:
    extraModelList:
      - model_name: anthropic/claude-sonnet-4-6
        litellm_params:
          model: bedrock/REPLACE_WITH_SONNET_MODEL_OR_PROFILE_ID
          api_base: https://bedrock-runtime.us-east-1.amazonaws.com
          aws_region_name: us-east-1
          api_key: os.environ/AWS_BEARER_TOKEN_BEDROCK
          cache_control_injection_points:
            - location: message
              index: -1
      - model_name: anthropic/claude-haiku-4-5
        litellm_params:
          model: bedrock/REPLACE_WITH_HAIKU_MODEL_OR_PROFILE_ID
          api_base: https://bedrock-runtime.us-east-1.amazonaws.com
          aws_region_name: us-east-1
          api_key: os.environ/AWS_BEARER_TOKEN_BEDROCK
          cache_control_injection_points:
            - location: message
              index: -1
  merge:
    settings:
      CONFIG__MODEL: openai/anthropic/claude-sonnet-4-6
      CONFIG__MODEL_TURBO: openai/anthropic/claude-haiku-4-5
      CONFIG__MODEL_REASONING: openai/anthropic/claude-sonnet-4-6
      CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/anthropic/claude-sonnet-4-6
      CONFIG__FALLBACK_MODELS: '["openai/anthropic/claude-haiku-4-5"]'
      REVIEW_AGENT__LLM_MODEL: openai/anthropic/claude-sonnet-4-6_thinking
      REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/anthropic/claude-sonnet-4-6
      REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/anthropic/claude-sonnet-4-6_thinking", "openai/anthropic/claude-haiku-4-5_thinking"]'
      REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/anthropic/claude-haiku-4-5
      REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/anthropic/claude-haiku-4-5_thinking
litellm:
  env:
    AWS_BEARER_TOKEN_BEDROCK:
      valueFrom:
        secretKeyRef:
          name: qodo-bedrock
          key: BEARER_TOKEN_BEDROCK

This example uses a Bedrock API key. Create qodo-bedrock with the key BEARER_TOKEN_BEDROCK. Match the runtime endpoint and region. Check model access and inference-profile permissions. For IAM authentication, remove each api_key field and configure the AI gateway workload’s approved AWS credential chain. Do not add AWS credentials to Qodo application containers. Test Messages and Chat Completions.For more information, see Bedrock.

Keep the baseline GPT and embedding routes. Enable Vertex AI and authorize the AI gateway workload to invoke the selected Gemini models. This example mounts a credential file from qodo-vertex, under the key credentials.json.

llm-provider-values.yaml

global:
  llm:
    extraModelList:
      - model_name: vertex_ai/gemini-2.5-pro
        litellm_params:
          model: vertex_ai/gemini-2.5-pro
          api_base: https://us-central1-aiplatform.googleapis.com
          vertex_project: REPLACE_WITH_PROJECT_ID
          vertex_location: us-central1
          vertex_credentials: /etc/vertex/credentials.json
      - model_name: vertex_ai/gemini-2.5-flash
        litellm_params:
          model: vertex_ai/gemini-2.5-flash
          api_base: https://us-central1-aiplatform.googleapis.com
          vertex_project: REPLACE_WITH_PROJECT_ID
          vertex_location: us-central1
          vertex_credentials: /etc/vertex/credentials.json
  merge:
    settings:
      CONFIG__MODEL: openai/vertex_ai/gemini-2.5-pro
      CONFIG__MODEL_TURBO: openai/vertex_ai/gemini-2.5-flash
      CONFIG__MODEL_REASONING: openai/vertex_ai/gemini-2.5-pro
      CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/vertex_ai/gemini-2.5-pro
      CONFIG__FALLBACK_MODELS: '["openai/vertex_ai/gemini-2.5-flash"]'
      REVIEW_AGENT__LLM_MODEL: openai/vertex_ai/gemini-2.5-pro_thinking
      REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/vertex_ai/gemini-2.5-pro
      REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/vertex_ai/gemini-2.5-pro_thinking", "openai/vertex_ai/gemini-2.5-flash_thinking"]'
      REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/vertex_ai/gemini-2.5-flash
      REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/vertex_ai/gemini-2.5-flash_thinking
litellm:
  volumeMounts:
    vertex-credentials:
      mountPath: /etc/vertex
      readOnly: true
  volumes:
    vertex-credentials:
      secret:
        secretName: qodo-vertex

Use an approved Workload Identity Federation credential configuration when available. A service-account JSON file is another authentication option. Keep credentials in the Secret. Match the project, model availability, region, and endpoint. A Gemini API key does not configure Vertex AI authentication.For more information, see Vertex AI.

Keep the baseline GPT and embedding routes. This configuration uses the Gemini API with an API key. Create qodo-gemini with the key GEMINI_API_KEY.

llm-provider-values.yaml

global:
  llm:
    extraModelList:
      - model_name: gemini/gemini-2.5-pro
        litellm_params:
          model: gemini/gemini-2.5-pro
          api_base: https://generativelanguage.googleapis.com/v1beta
          api_key: os.environ/GEMINI_API_KEY
      - model_name: gemini/gemini-2.5-flash
        litellm_params:
          model: gemini/gemini-2.5-flash
          api_base: https://generativelanguage.googleapis.com/v1beta
          api_key: os.environ/GEMINI_API_KEY
  merge:
    settings:
      CONFIG__MODEL: openai/gemini/gemini-2.5-pro
      CONFIG__MODEL_TURBO: openai/gemini/gemini-2.5-flash
      CONFIG__MODEL_REASONING: openai/gemini/gemini-2.5-pro
      CONFIG__SECOND_MODEL_FOR_EXHAUSTIVE_MODE: openai/gemini/gemini-2.5-pro
      CONFIG__FALLBACK_MODELS: '["openai/gemini/gemini-2.5-flash"]'
      REVIEW_AGENT__LLM_MODEL: openai/gemini/gemini-2.5-pro_thinking
      REVIEW_AGENT__CONVERSION_LLM_MODEL: openai/gemini/gemini-2.5-pro
      REVIEW_AGENT__ENSEMBLE_MODELS: '["openai/gemini/gemini-2.5-pro_thinking", "openai/gemini/gemini-2.5-flash_thinking"]'
      REVIEW_AGENT__PAST_BUGS_LLM_MODEL: openai/gemini/gemini-2.5-flash
      REVIEW_AGENT__SUBAGENT_LLM_MODEL: openai/gemini/gemini-2.5-flash_thinking
litellm:
  env:
    GEMINI_API_KEY:
      valueFrom:
        secretKeyRef:
          name: qodo-gemini
          key: GEMINI_API_KEY

Merge sends Chat Completions to the AI gateway. The gateway translates the request for Gemini. This configuration does not use Vertex AI project or workload-identity settings.Keep api_base: https://generativelanguage.googleapis.com/v1beta for these routes. They do not inherit global.llm.openaiBaseUrl. For more information, see Gemini API reference.For more information, see Gemini.

The Claude routes include prompt-cache injection at the last message boundary. Cache use depends on provider requirements. Verify cache reads in usage records. For more information, see prompt caching.

3. Preserve the other application routes

The chart supplies the base URLs and internal gateway key for the applications below. The review overrides above do not change these settings.

Application path	Application setting or model value	Gateway alias required
Context Engine `nano` and `lite`	`ONPREM_MODELS__NANO__MODEL` and `ONPREM_MODELS__LITE__MODEL`: `openai/gpt-5.1`	`gpt-5.1`
Context Engine `main`	`ONPREM_MODELS__MAIN__MODEL`: `openai/gpt-5.2`	`gpt-5.2`
Context Engine embeddings	`ONPREM_MODELS__REMOTE_RAG__MODEL`: `openai/text-embedding-3-large`	`text-embedding-3-large`
Context Engine agent paths	`DEEP_ASK_RESEARCH_PREVIEW__LLM_MODEL` and `PR_KNOWLEDGE__LLM_MODEL`: `gpt-5.2`	`gpt-5.2`
Platform rule processing	Model settings include `gpt-5.2`, `gpt-5.1`, and `openai:gpt-5.2`	`gpt-5.2` and `gpt-5.1`
Platform rule similarity	`text-embedding-3-large`	`text-embedding-3-large`

Platform agent settings use provider:model syntax, such as openai:gpt-5.2. This differs from Merge’s slash-separated client routing. Do not copy one application’s value into another application’s setting. The chart selects Context Engine’s on-prem model configuration. AI_GATEWAY__* settings belong to a different Engine configuration and do not replace the ONPREM_MODELS__* settings shown above. Platform uses DYNACONF_LLM_GATEWAY__*, which the chart supplies.

Keep embedding dimensions compatible

The same embedding route must honor both requests:

Consumer	Requested dimensions
Context Engine PR Knowledge	`2000`
Platform rule similarity	`1024`

Keep text-embedding-3 in the embedding alias. The gateway’s model handling uses this name when forwarding the dimensions parameter. For more information, see embedding parameter handling. An arbitrary embedding model is not a compatible replacement. Do not change dimensions, embedding models, or stored indexes independently. Coordinate such changes with Qodo before migrating existing data.

Include SDK agent models

SDK and CLI agent topologies can request additional gateway aliases. Review their configured models before enabling those workflows. The packaged diagnostic agents request anthropic/claude-haiku-4-5. The Anthropic and Bedrock examples above publish this alias. The OpenAI, Azure, Vertex AI, and Gemini API examples do not add that diagnostic route. If you use those agents, add an approved Claude route to the same extraModelList before invoking them. A successful runtime HTTP health check does not verify an agent’s model route. Custom agent topologies can require other models too.

4. Create provider Secrets and apply the overlays

Create Secrets in the installation namespace before deploying Qodo. Use your approved secret manager or ESO when available. For more information, see Manage Qodo secrets with External Secrets Operator. For a new single-key Secret, this Bash example keeps the credential out of the command text. Select your installation’s Kubernetes context and namespace. For an existing installation, retain its Helm release name. Replace the context placeholder, then change the Secret name and key for your provider:

KUBE_CONTEXT=REPLACE_WITH_KUBERNETES_CONTEXT
NAMESPACE=qodo
RELEASE=qodo
kubectl config get-contexts "$KUBE_CONTEXT"
PROVIDER_SECRET=qodo-anthropic
PROVIDER_SECRET_KEY=ANTHROPIC_API_KEY
set +x
read -r -s -p "Provider credential: " PROVIDER_CREDENTIAL
printf '\n'
: "${PROVIDER_CREDENTIAL:?Provider credential is required}"
printf '%s' "$PROVIDER_CREDENTIAL" |
  kubectl --context "$KUBE_CONTEXT" -n "$NAMESPACE" \
    create secret generic "$PROVIDER_SECRET" \
    --from-file="$PROVIDER_SECRET_KEY=/dev/stdin"
unset PROVIDER_CREDENTIAL

For qodo-openai, use the Secret key OPENAI_API_KEY. For Vertex AI, import the approved credential file as qodo-vertex with the key credentials.json. Do not place credentials in values files or commit them to Git. Use the chart and installation procedure supplied with your release. Add the overlays after your existing customer values:

helm template "$RELEASE" ./qodo-onprem \
  --kube-context "$KUBE_CONTEXT" \
  --namespace "$NAMESPACE" \
  -f onprem-base.yaml \
  -f my-values.yaml \
  -f llm-base-values.yaml \
  -f llm-provider-values.yaml > rendered.yaml

Set RELEASE to your existing release name. Preserve that name on upgrades. Treat rendered manifests as sensitive because they can contain Secrets. Inspect the generated model names, upstream endpoints, credential references, and qodo-merge-custom-env. Check that every selected Merge model has an exact gateway alias after removing its outer openai/ prefix and trailing _thinking suffix. Helm merges maps but replaces lists. Keep all selected provider routes in one extraModelList. A later overlay with that list replaces earlier entries. Use the same values order for readiness checks and deployment. For more information, see Deploy Qodo using Helm. If the gateway’s upstream endpoint uses a private CA, obtain a supported gateway trust configuration from Qodo before connecting it. The standard custom-ca-cert Secret configures covered Qodo applications, but excludes bundled AI gateway. For more information, see Private certificate authority.

5. Verify model access and application behavior

A healthy Pod or a successful /models response does not verify inference, tool calls, or embeddings. First, wait for the gateway and review workloads to complete their rollouts. For a local diagnostic connection, set the same context and namespace in a separate terminal, then open the connection:

KUBE_CONTEXT=REPLACE_WITH_KUBERNETES_CONTEXT
NAMESPACE=qodo
kubectl --context "$KUBE_CONTEXT" -n "$NAMESPACE" \
  port-forward service/qodo-litellm 14000:4000

The following requests use the local forwarded port. Enter the internal gateway key from your approved secret store. Do not enter your provider key. Run these commands in Bash. The header uses a separate file descriptor to keep the key out of curl process arguments:

GATEWAY_URL=http://localhost:14000
set +x
read -r -s -p "Internal gateway key: " LLM_GATEWAY_API_KEY
printf '\n'
: "${LLM_GATEWAY_API_KEY:?Gateway key is required}"
curl --fail-with-body --silent --show-error \
  "$GATEWAY_URL/v1/models" \
  --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") |
  jq -r '.data[].id'

Confirm that the baseline aliases and both selected provider aliases appear.

Test Chat Completions and tool calls

Use this test for the selected review alias. It also covers the Chat Completions path used by classic reviews and SDK agents. Replace the example with your gateway alias, without Merge’s outer prefix.

REVIEW_MODEL=vertex_ai/gemini-2.5-pro
jq -n --arg model "$REVIEW_MODEL" '{
  model: $model,
  messages: [{role: "user", content: "Call report_status with status ready."}],
  tools: [{
    type: "function",
    function: {
      name: "report_status",
      description: "Report the connection test status.",
      parameters: {
        type: "object",
        properties: {status: {type: "string", enum: ["ready"]}},
        required: ["status"]
      }
    }
  }],
  tool_choice: {type: "function", function: {name: "report_status"}}
}' |
  curl --fail-with-body --silent --show-error \
    "$GATEWAY_URL/v1/chat/completions" \
    --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \
    -H "Content-Type: application/json" --data-binary @- |
  jq -e '.choices[0].message.tool_calls[0].function.name == "report_status"'

Expect true. Repeat for the fast alias. Verify streaming and a subsequent tool-result turn in an actual review.

Test Responses for OpenAI and Azure GPT routes

Use the advertised GPT alias. Keep this check even when your review provider is Claude or Gemini, because Platform has its own GPT agent paths.

curl --fail-with-body --silent --show-error \
  "$GATEWAY_URL/v1/responses" \
  --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.2",
    "input": "Reply with ready.",
    "reasoning": {"effort": "medium"},
    "include": ["reasoning.encrypted_content"]
  }' |
  jq -e '.status == "completed"'

Repeat with openai/gpt-5.1 for the fast review model. Test both medium and low reasoning effort for the enabled review stages. Also test the bare gpt-5.2 and gpt-5.1 aliases used by other applications. A successful Chat Completions request does not establish Responses support.OPENAI__RESPONSE_API controls Merge’s GPT path. Setting it to "false" does not disable Platform’s independent Responses selection.

Test Anthropic Messages for Claude routes

Agentic Claude reviews use Messages. The Bedrock example exposes the same alias through the AI gateway, which translates the upstream request.

curl --fail-with-body --silent --show-error \
  "$GATEWAY_URL/v1/messages" \
  --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-6",
    "max_tokens": 64,
    "messages": [{"role": "user", "content": "Reply with ready."}]
  }' |
  jq -e '.type == "message" and (.content | length > 0)'

Repeat with anthropic/claude-haiku-4-5. Verify reasoning, tool calls, structured findings, and streaming in an actual agentic review.

Test both embedding dimensions

Run both requests through the shared embedding route. The returned vector length must match each requested dimension.

(
set -o pipefail
for DIMENSIONS in 2000 1024; do
  jq -n --argjson dimensions "$DIMENSIONS" '{
    model: "text-embedding-3-large",
    input: "Model connection test",
    dimensions: $dimensions
  }' |
    curl --fail-with-body --silent --show-error \
      "$GATEWAY_URL/v1/embeddings" \
      --header @<(printf 'Authorization: Bearer %s\n' "$LLM_GATEWAY_API_KEY") \
      -H "Content-Type: application/json" --data-binary @- |
    jq -e --argjson expected "$DIMENSIONS" \
      '(.data[0].embedding | length) == $expected' || exit 1
done
)

Expect true twice. A missing or incorrect vector length requires investigation before indexing or rule processing.

After the gateway checks, run unset LLM_GATEWAY_API_KEY. Finally, complete the application checks:

Run a normal review against a representative Pull Request (PR).
Run the review modes and fallback paths enabled for your installation.
Confirm that structured findings appear in the Git provider and portal.
Index a repository and verify Context Engine retrieval.
Verify rule ingestion and rule similarity.
Invoke each enabled SDK agent and verify its model route.
Confirm the selected upstream models in your gateway’s routing or usage records.

Record which providers, protocols, and workflows passed. Do not treat a gateway smoke test as complete application validation.

Apply configuration updates and rotate credentials

Apply changes through your existing Helm or GitOps release process. Confirm that the gateway loads the new model configuration. A Secret update alone does not restart Pods or refresh environment variables. When rotating a provider credential, wait for Secret synchronization, restart the gateway through your deployment process, and repeat inference checks. If the internal gateway key changes, every application consumer must receive the same replacement key. For the complete rotation procedure, see Manage Qodo secrets with External Secrets Operator.

Troubleshoot model access

Symptom	Check and next action
`CreateContainerConfigError`	Check Secret names, namespaces, and case-sensitive keys, including `qodo-openai`
Gateway returns `401`	Check the internal gateway key used by the application or diagnostic request
Upstream returns `401` or `403`	Check the provider credential, workload identity, model access, region, and profile permissions
Model is missing or gateway returns `400`	Compare the actual request name with `model_name`; check replacement maps, list overrides, and stripped prefixes
Some requests reach the wrong provider	Remove unintended duplicate aliases across `modelMap` and `extraModelList`
Reviews still use a previous model	Check all ten Merge settings and completed application rollouts
Chat works but agentic review fails	Test the required Responses or Messages endpoint, streaming, tool calls, and structured output
Embeddings return the wrong vector length	Preserve the embedding alias and verify both requested dimensions
A diagnostic SDK agent fails while runtime health passes	Publish the exact alias from that agent’s topology and test its invocation
TLS verification fails	Identify which connection failed. The AI gateway needs its own supported upstream CA configuration; the application CA Secret does not cover it
Provider throttles or times out	Check quotas, deployment capacity, network access, and upstream latency before changing timeouts

Deploy Qodo using Helm: Apply the release’s installation and upgrade procedure.
Helm chart configuration reference: Review the accepted values.
Manage Qodo secrets with External Secrets Operator: Provision and rotate provider credentials.
Monitor Qodo with Prometheus and Grafana: Observe workloads after configuration changes.

Installation

Configuration

Operations

Git Integration

Understand the two connections

Match model names at both ends

1. Prepare the baseline routes

2. Select the review provider

3. Preserve the other application routes

Keep embedding dimensions compatible

Include SDK agent models

4. Create provider Secrets and apply the overlays

5. Verify model access and application behavior

Apply configuration updates and rotate credentials

Troubleshoot model access

​Understand the two connections

​Match model names at both ends

​1. Prepare the baseline routes

​2. Select the review provider

​3. Preserve the other application routes

​Keep embedding dimensions compatible

​Include SDK agent models

​4. Create provider Secrets and apply the overlays

​5. Verify model access and application behavior

​Apply configuration updates and rotate credentials

​Troubleshoot model access

​Related resources

Understand the two connections

Match model names at both ends

1. Prepare the baseline routes

2. Select the review provider

3. Preserve the other application routes

Keep embedding dimensions compatible

Include SDK agent models

4. Create provider Secrets and apply the overlays

5. Verify model access and application behavior

Apply configuration updates and rotate credentials

Troubleshoot model access

Related resources