> ## Documentation Index
> Fetch the complete documentation index at: https://allhandsai-codex-bedrock-llm-gateway-docs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS Bedrock LLM Gateway

> Connect Amazon Bedrock models through the OpenHands Enterprise LLM gateway.

OpenHands Enterprise can use Amazon Bedrock through its bundled LiteLLM
gateway. Configure the gateway according to how you installed OpenHands.
The Helm IRSA example assigns the Bedrock role to the **LiteLLM ServiceAccount**.
Replicated uses the credential behavior described in its tab below.

## Configure the Gateway

<Tabs>
  <Tab title="Replicated">
    1. Open the [Admin Console LLM configuration](/enterprise/vm-install/admin-console-configuration#llm-configuration).
    2. Select `AWS Bedrock` and enter the AWS Region.
    3. Under `AWS Authentication Method`, choose one of the Admin Console options:

       * `Access Key + Secret`: enter the `AWS Access Key ID` and matching
         `AWS Secret Access Key` in their required fields.
       * `EC2 Instance Profile`: attach an IAM role to the EC2 instance. In a
         containerized installation, set the instance metadata service (IMDS)
         response hop limit to at least 2 so the LiteLLM pod can reach it.

       Grant the chosen IAM identity `bedrock:InvokeModel` and
       `bedrock:InvokeModelWithResponseStream` for every model you add.
    4. In `Bedrock Model IDs`, enter **one Bedrock model ID or inference profile ID
       per line**. Enter the IDs as AWS provides them, without the `bedrock/`
       prefix. For example:

       ```text theme={null}
       us.anthropic.claude-haiku-4-5-20251001-v1:0
       us.anthropic.claude-sonnet-4-5-20250929-v1:0
       ```

       The Admin Console creates a separate bundled-gateway model for each line.
       The first line becomes the installation default. Use IDs available to your
       account and selected Region; check each inference profile with
       `aws bedrock get-inference-profile` before adding it.
    5. Save the configuration and deploy the updated version.

    If you add a cross-Region inference profile, keep its full ID, including the
    `us.` prefix in these examples. Your IAM policy must permit the profile and
    its destination models. A standard model ID is Region-specific. The
    `Allow users to configure their own LLM providers (BYOK)` checkbox controls
    whether users can add personal providers; it is not required for these
    administrator-managed Bedrock models.

    <Note>
      In the Replicated static-key configuration, the installer also passes the AWS
      access key and secret into sandbox environments. With an EC2 instance profile,
      sandboxes may also reach IMDS unless your network controls prevent that access;
      a response hop limit of 2 does not isolate credentials to the gateway.
      Scope the IAM identity for this deployment behavior rather than assuming that
      only the LiteLLM pod can use it.
    </Note>
  </Tab>

  <Tab title="Helm">
    ### Prerequisites

    * A working [OpenHands Enterprise Helm installation](/enterprise/k8s-install/installation).
    * A Bedrock model that supports the agent's tool use, with model access enabled
      in your AWS account. Confirm its Region and, if required, its inference
      profile ID in the [Bedrock model catalog](https://docs.aws.amazon.com/bedrock/latest/userguide/models.html).
    * HTTPS access from the bundled LiteLLM pod to the Bedrock Runtime endpoint.

    The example below uses an **EKS IAM role for service accounts (IRSA)**, so it
    does not put long-lived AWS keys in Helm values or Kubernetes Secrets. Other
    Kubernetes platforms can supply AWS credentials to the LiteLLM pod using
    their supported credential mechanism.

    ### 1. Grant the LiteLLM Service Account Bedrock Access

    Create an IAM role whose trust policy allows your EKS cluster's OIDC provider
    to assume it only for the ServiceAccount
    `system:serviceaccount:openhands:openhands-litellm-bedrock`. Follow the
    [AWS IRSA setup guide](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)
    to create the OIDC provider and role. If you use a different namespace or
    ServiceAccount name, use those values in both the trust policy and Helm values.

    Grant the role `bedrock:InvokeModel` and
    `bedrock:InvokeModelWithResponseStream` for every model you will use. If a
    model requires a cross-Region inference profile, include the inference
    profile ARN in the source Region and every destination foundation-model ARN
    returned by `GetInferenceProfile`. For example:

    ```bash theme={null}
    aws bedrock get-inference-profile \
      --region <source-region> \
      --inference-profile-identifier <profile-id> \
      --query '{profile:inferenceProfileArn,models:models[*].modelArn}'
    ```

    Restrict the policy to those returned ARNs. AWS also requires your
    organization's service control policies to allow the profile's destination
    Regions; see [cross-Region inference permissions](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html).

    ### 2. Add a Bedrock Route to Helm Values

    Merge the following into your **complete** installation `values.yaml`. Keep
    your existing `litellm-helm.proxy_config.model_list` entries: Helm replaces
    lists when applying overrides. Replace the role ARN, Region, and model ID with
    your own values.

    ```yaml theme={null}
    litellm-helm:
      serviceAccount:
        create: true
        name: openhands-litellm-bedrock
        annotations:
          eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/<bedrock-role>
      proxy_config:
        model_list:
          # Retain your existing model entries here.
          - model_name: bedrock-haiku-4-5
            litellm_params:
              model: bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
              aws_region_name: us-west-2
          - model_name: bedrock-sonnet-4-5
            litellm_params:
              model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
              aws_region_name: us-west-2
    ```

    Add one `model_list` entry per model. Each `model_name` is the alias OpenHands
    uses to select that route. In Helm values, prefix the AWS model or inference
    profile ID with `bedrock/` in `litellm_params.model`; the Admin Console field
    above takes the raw ID instead. If you change the alias that should be the
    installation default, also set `env.LITELLM_DEFAULT_MODEL` to
    `litellm_proxy/<alias>` in the same values file.

    ### 3. Apply and Verify

    Use the licensed chart URL and version from your installation:

    ```bash theme={null}
    helm upgrade openhands "$OPENHANDS_CHART_URL" \
      --namespace openhands \
      --version "$OPENHANDS_CHART_VERSION" \
      --values values.yaml \
      --wait --timeout 10m

    kubectl -n openhands rollout status deployment/openhands-litellm
    kubectl -n openhands get serviceaccount openhands-litellm-bedrock \
      -o jsonpath='{.metadata.annotations.eks\.amazonaws\.com/role-arn}'
    ```

    Adjust the namespace, release name, and Deployment name if they differ in your
    installation. The annotation should contain the IAM role ARN from your values.
  </Tab>
</Tabs>

## Select the Model in OpenHands

For a Replicated installation, select the Bedrock model configured in the
Admin Console. For a Helm installation, create a profile in `Settings` → `LLM`
for each gateway alias you want users to select. For the first Helm example
above, use:

| Field | Value |
| - | - |
| Profile Name | `Bedrock-Haiku-4-5` |
| Model | `openhands/bedrock-haiku-4-5` |
| Base URL | `http://openhands-litellm.openhands.svc.cluster.local:4000` |
| API Key | Leave unset; use the managed gateway credential |

Use your installation's actual LiteLLM Service name and namespace. The profile
points to the **internal gateway**, not to a Bedrock endpoint.
For the second Helm route, create another profile with Model
`openhands/bedrock-sonnet-4-5` and the same Base URL.

## Start Using the Model

1. Select the Bedrock profile and start a new conversation.
2. Ask the agent to run `pwd`.
3. Confirm it starts a sandbox, runs the command, and replies with the output.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Bedrock AccessDeniedException">
    Check the IAM role annotation on the LiteLLM ServiceAccount, the role trust
    policy's exact namespace and ServiceAccount subject, and the role's model
    invocation policy. For a cross-Region inference profile, include its source
    profile ARN and all destination model ARNs; check organization service control
    policies for denied Regions.
  </Accordion>

  <Accordion title="Model or inference profile not found">
    Check the exact model or profile ID and `aws_region_name`. A model available in
    one Region may require an inference profile in another.
  </Accordion>

  <Accordion title="The profile cannot reach the gateway">
    Check the internal LiteLLM Service DNS name, namespace, and profile base URL.
    Confirm the LiteLLM Deployment is ready and its logs show the Bedrock route.
  </Accordion>

  <Accordion title="The model answers a short prompt but the agent cannot work">
    Confirm the Bedrock model supports tool use and that its account quotas allow
    larger agent prompts. Test a full conversation with a sandbox command, not
    only a direct model completion.
  </Accordion>
</AccordionGroup>

For provider-specific model configuration, see
[LiteLLM's Bedrock reference](https://docs.litellm.ai/docs/providers/bedrock).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.