Amazon Bedrock

Amazon Bedrock is a fully managed service that provides access to powerful foundation models (FMs) from leading AI providers like Anthropic, Meta, and Amazon. It allows developers to easily build and scale generative AI applications without managing infrastructure.

Extended Thinking Support

Amazon Bedrock now supports Extended Thinking capabilities for Anthropic Claude models with step-by-step transparency!

Key Features:

• ✅ Enable Extended Thinking: Enhance reasoning for complex tasks with transparent thought processes
• 🎯 Thinking Budget Tokens: Configure the maximum tokens for Claude's internal reasoning
• 🔍 Step-by-step Transparency: See how Claude approaches complex problems
• ⚠️ Cost Impact: Extended thinking consumes additional tokens, increasing usage costs
• 📋 Availability: Available for Claude 3.7 Sonnet and newer versions

info

For enabling models in different AWS regions,refer this cross region inference page

Getting started with Amazon Bedrock

You can choose your model based on your requirements; the following steps only show how to configure Anthropic in Bedrock.

Step 1: Set Up Your AWS Account

Pre-requisites

• Active AWS account.
• Necessary permissions: Ensure that your AWS user or role has the necessary permissions to access and use Amazon Bedrock.

Step 2: Access Amazon Bedrock

Via AWS Console

1. Go to the AWS Management Console.
2. Search for Amazon Bedrock in the search bar.
3. Open Amazon Bedrock service page.

Step 3: Model Access

info

The Model access page in Amazon Bedrock has been retired. Serverless foundation models are now automatically enabled across all AWS commercial regions when first invoked in your account — you no longer need to manually activate model access.

For models served from AWS Marketplace, a user with AWS Marketplace permissions must invoke the model once to enable it account-wide for all users. First-time users of Anthropic models may need to submit use case details before they can access the model.

Steps to Get AWS Access Key and Secret Key for Bedrock

Step 1: Sign In to the AWS Management Console

1. Go to the AWS Management Console.
2. Log in using your AWS root account or an IAM user with sufficient permissions.

Step 2: Create an IAM User with Programmatic Access

1. Navigate to IAM:

   • In the AWS Management Console, search for IAM (Identity and Access Management).
   • Click on IAM to open the IAM dashboard.

2. Create New User:

   • In the left sidebar, click on Users.
   • Click the Add user button.

3. Set Permissions:

   • In the User details step, provide a User name.
   • Under Access type, select Programmatic access to allow access through the AWS CLI, SDKs, or APIs.

4. Assign Permissions:

   • In the Set permissions step, you have two options:

     • Attach policies directly: Search for and attach policies that grant the necessary permissions to use Amazon Bedrock.

     Basic policy for on-demand models (e.g., Claude 3 Haiku):

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "bedrock:InvokeModel",
                     "bedrock:InvokeModelWithResponseStream"
                 ],
                 "Resource": [
                     "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
                 ]
             }
         ]
     }

     Policy for newer models requiring inference profiles (e.g., Claude Haiku 4.5, Claude Sonnet 4.6):

     Newer Anthropic models (Claude Haiku 4.5, Claude Sonnet 4.6, etc.) do not support on-demand invocation. They must be accessed through a cross-region inference profile (e.g., us.anthropic.claude-haiku-4-5-20251001-v1:0). The IAM policy must include both the foundation model ARN and the inference profile ARN:

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "bedrock:InvokeModel",
                     "bedrock:InvokeModelWithResponseStream"
                 ],
                 "Resource": [
                     "arn:aws:bedrock:*::foundation-model/anthropic.claude-haiku-4-5-20251001-v1:0",
                     "arn:aws:bedrock:*:<ACCOUNT_ID>:inference-profile/us.anthropic.claude-haiku-4-5-20251001-v1:0"
                 ]
             },
             {
                 "Effect": "Allow",
                 "Action": [
                     "aws-marketplace:ViewSubscriptions",
                     "aws-marketplace:Subscribe"
                 ],
                 "Resource": "*"
             }
         ]
     }
     Important Notes

     • Replace <ACCOUNT_ID> with your AWS account ID.
     • The foundation model resource uses a wildcard region (*) because the cross-region inference profile may route requests to any US region.
     • The inference profile resource also needs a wildcard region (*) for the same reason.
     • The aws-marketplace actions require Resource: "*" — they do not support resource-level restrictions. These permissions are needed for first-time Anthropic model access and can be removed afterward.

     For more information, see Amazon Bedrock IAM policies.

     • Add user to group: If you have an existing group with appropriate permissions, you can select that group.
     • Attach customer managed policies: For fine-grained control, create a custom policy (such as restricting access only to Bedrock).

   • Once selected, click Next.

Step 3: Review and Create User

1. Review User Details:

   • Verify the permissions and access settings.

2. Create User:

   • Click Create user.
   • After creation, AWS will display a success message with the Access Key ID and Secret Access Key.

   Important: Copy the Secret Access Key immediately, as you won't be able to view it again.

Step 4: Store the Access Keys Securely

• Store the Access Key ID and Secret Access Key in a secure location (for example, AWS Secrets Manager, environment variables, or an encrypted file).
• You will use these keys to connect to Unstract.

Setting up the Anthropic LLM model in Unstract

Now that we have an model deployed and the required keys, we can use it to set up an LLM profile on the Unstract platform. For this:

• Sign in to the Unstract Platform
• From the side navigation menu, choose Settings 🞂 LLMs
• Click on the New LLM Profile button.
• From the list of LLMs, choose Bedrock. You should see a dialog box where you enter details.

Extended Thinking Configuration

If you're using Anthropic Claude 3.7 Sonnet through Bedrock, you'll have additional options:

Enable Extended Thinking

• ☑️ Check this option to activate enhanced reasoning with step-by-step transparency
• Available for Claude 3.7 Sonnet and newer versions

Thinking Budget Tokens

• 🎯 Token Budget: Set the maximum tokens for Claude's internal reasoning
• 💡 Recommendation: Start with 5000 tokens for most use cases
• 📈 Higher Budget: More tokens = more detailed reasoning but increased costs

⚠️ Important: Extended thinking consumes additional tokens, which will increase your AWS usage costs

• For Name, enter a name for this connector.
• In the Model Name enter the model which you deployed.
• For AWS Secret Key and AWS Access Key enter the keys downloaded from AWS
• Leave the Max retries and Timeout fields to their default values.
• For the Maximum Output Tokens, enter the maximum token limit supported by the model, or leave it empty to use the default maximum.
• For Claude 3.7 Sonnet and newer: Enable extended thinking and set your preferred thinking budget tokens
• Click on Test Connection and ensure it succeeds. You can finally click on Submit and that should create a new LLM Profile for use in your Unstract projects.

Using Cross-Region Inference Models

Amazon Bedrock supports cross-region inference, which allows you to access foundation models across different AWS regions for improved availability and performance. With Unstract's Bedrock adapter, you can easily use cross-region inference models.

Understanding Inference Profiles

Newer Anthropic models on Bedrock (Claude Haiku 4.5, Claude Sonnet 4.6, and newer) require cross-region inference profiles. They cannot be invoked directly using the foundation model ID.

• On-demand models (e.g., anthropic.claude-3-haiku-20240307-v1:0): Can be invoked directly with the model ID.
• Inference-profile-only models (e.g., anthropic.claude-haiku-4-5-20251001-v1:0): Must use the inference profile ID with the us. prefix (e.g., us.anthropic.claude-haiku-4-5-20251001-v1:0).

If you use a model ID without the us. prefix for these newer models, you will receive the error: "Invocation of model ID ... with on-demand throughput isn't supported. Retry your request with the ID or ARN of an inference profile that contains this model."

Configuring Cross-Region Inference in Unstract

1. Access the Cross-Region Inference Console:

   • In the Amazon Bedrock console, navigate to the Cross-region inference section.
   • Here you'll find the available models with their corresponding Model IDs and Model ARNs.

2. Configure in Unstract:

   • When setting up your LLM profile in Unstract, use the inference profile Model ID (e.g., us.anthropic.claude-haiku-4-5-20251001-v1:0) in the Model Name field.

3. Ensure IAM Permissions:

   • Your IAM policy must include both the foundation model and inference profile resource ARNs with wildcard regions (see IAM policy example above).

For more detailed information about cross-region inference, refer to the AWS Bedrock Cross-Region Inference documentation.

Note: All other configuration steps remain the same - you only need to replace the model identifier in the Model Name field.

Current Limitations

APAC Region Costing: Costing information for models in the APAC region are currently unavailable. This is expected to be fixed in the future.