AWS Bedrock LLM Gateway

Connect Amazon Bedrock models through the OpenHands Enterprise LLM gateway.

OpenHands Enterprise can use Amazon Bedrock through its bundled LiteLLM gateway. Configure the gateway according to how you installed OpenHands. The Helm IRSA example assigns the Bedrock role to the LiteLLM ServiceAccount. Replicated uses the credential behavior described in its tab below.

Configure the Gateway

Replicated

1. Open the Admin Console LLM configuration.

2. Select AWS Bedrock and enter the AWS Region.

3. Under AWS Authentication Method, choose one of the Admin Console options:

   • Access Key + Secret: enter the AWS Access Key ID and matching AWS Secret Access Key in their required fields.
   • EC2 Instance Profile: attach an IAM role to the EC2 instance. In a containerized installation, set the instance metadata service (IMDS) response hop limit to at least 2 so the LiteLLM pod can reach it.

   Grant the chosen IAM identity bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream for every model you add.

4. In Bedrock Model IDs, enter one Bedrock model ID or inference profile ID per line. Enter the IDs as AWS provides them, without the bedrock/ prefix. For example:

   us.anthropic.claude-haiku-4-5-20251001-v1:0
   us.anthropic.claude-sonnet-4-5-20250929-v1:0
   

   The Admin Console creates a separate bundled-gateway model for each line. The first line becomes the installation default. Use IDs available to your account and selected Region; check each inference profile with aws bedrock get-inference-profile before adding it.

5. Save the configuration and deploy the updated version.

If you add a cross-Region inference profile, keep its full ID, including the us. prefix in these examples. Your IAM policy must permit the profile and its destination models. A standard model ID is Region-specific. The Allow users to configure their own LLM providers (BYOK) checkbox controls whether users can add personal providers; it is not required for these administrator-managed Bedrock models.

In the Replicated static-key configuration, the installer also passes the AWS access key and secret into sandbox environments. With an EC2 instance profile, sandboxes may also reach IMDS unless your network controls prevent that access; a response hop limit of 2 does not isolate credentials to the gateway. Scope the IAM identity for this deployment behavior rather than assuming that only the LiteLLM pod can use it.

Helm

Prerequisites

• A working OpenHands Enterprise Helm installation.
• A Bedrock model that supports the agent’s tool use, with model access enabled in your AWS account. Confirm its Region and, if required, its inference profile ID in the Bedrock model catalog.
• HTTPS access from the bundled LiteLLM pod to the Bedrock Runtime endpoint.

The example below uses an EKS IAM role for service accounts (IRSA), so it does not put long-lived AWS keys in Helm values or Kubernetes Secrets. Other Kubernetes platforms can supply AWS credentials to the LiteLLM pod using their supported credential mechanism.

1. Grant the LiteLLM Service Account Bedrock Access

Create an IAM role whose trust policy allows your EKS cluster’s OIDC provider to assume it only for the ServiceAccount system:serviceaccount:openhands:openhands-litellm-bedrock. Follow the AWS IRSA setup guide to create the OIDC provider and role. If you use a different namespace or ServiceAccount name, use those values in both the trust policy and Helm values.

Grant the role bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream for every model you will use. If a model requires a cross-Region inference profile, include the inference profile ARN in the source Region and every destination foundation-model ARN returned by GetInferenceProfile. For example:

aws bedrock get-inference-profile \
  --region <source-region> \
  --inference-profile-identifier <profile-id> \
  --query '{profile:inferenceProfileArn,models:models[*].modelArn}'

Restrict the policy to those returned ARNs. AWS also requires your organization’s service control policies to allow the profile’s destination Regions; see cross-Region inference permissions.

2. Add a Bedrock Route to Helm Values

Merge the following into your complete installation values.yaml. Keep your existing litellm-helm.proxy_config.model_list entries: Helm replaces lists when applying overrides. Replace the role ARN, Region, and model ID with your own values.

litellm-helm:
  serviceAccount:
    create: true
    name: openhands-litellm-bedrock
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/<bedrock-role>
  proxy_config:
    model_list:
      # Retain your existing model entries here.
      - model_name: bedrock-haiku-4-5
        litellm_params:
          model: bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
          aws_region_name: us-west-2
      - model_name: bedrock-sonnet-4-5
        litellm_params:
          model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
          aws_region_name: us-west-2

Add one model_list entry per model. Each model_name is the alias OpenHands uses to select that route. In Helm values, prefix the AWS model or inference profile ID with bedrock/ in litellm_params.model; the Admin Console field above takes the raw ID instead. If you change the alias that should be the installation default, also set env.LITELLM_DEFAULT_MODEL to litellm_proxy/<alias> in the same values file.

3. Apply and Verify

Use the licensed chart URL and version from your installation:

helm upgrade openhands "$OPENHANDS_CHART_URL" \
  --namespace openhands \
  --version "$OPENHANDS_CHART_VERSION" \
  --values values.yaml \
  --wait --timeout 10m

kubectl -n openhands rollout status deployment/openhands-litellm
kubectl -n openhands get serviceaccount openhands-litellm-bedrock \
  -o jsonpath='{.metadata.annotations.eks\.amazonaws\.com/role-arn}'

Adjust the namespace, release name, and Deployment name if they differ in your installation. The annotation should contain the IAM role ARN from your values.

Select the Model in OpenHands

For a Replicated installation, select the Bedrock model configured in the Admin Console. For a Helm installation, create a profile in Settings → LLM for each gateway alias you want users to select. For the first Helm example above, use:

Field	Value
Profile Name	Bedrock-Haiku-4-5
Model	openhands/bedrock-haiku-4-5
Base URL	http://openhands-litellm.openhands.svc.cluster.local:4000
API Key	Leave unset; use the managed gateway credential

Use your installation’s actual LiteLLM Service name and namespace. The profile points to the internal gateway, not to a Bedrock endpoint. For the second Helm route, create another profile with Model openhands/bedrock-sonnet-4-5 and the same Base URL.

Start Using the Model

1. Select the Bedrock profile and start a new conversation.
2. Ask the agent to run pwd.
3. Confirm it starts a sandbox, runs the command, and replies with the output.

Troubleshooting

Bedrock AccessDeniedException

Check the IAM role annotation on the LiteLLM ServiceAccount, the role trust policy’s exact namespace and ServiceAccount subject, and the role’s model invocation policy. For a cross-Region inference profile, include its source profile ARN and all destination model ARNs; check organization service control policies for denied Regions.

Model or inference profile not found

Check the exact model or profile ID and aws_region_name. A model available in one Region may require an inference profile in another.

The profile cannot reach the gateway

Check the internal LiteLLM Service DNS name, namespace, and profile base URL. Confirm the LiteLLM Deployment is ready and its logs show the Bedrock route.

The model answers a short prompt but the agent cannot work

Confirm the Bedrock model supports tool use and that its account quotas allow larger agent prompts. Test a full conversation with a sandbox command, not only a direct model completion.

For provider-specific model configuration, see LiteLLM’s Bedrock reference.