Skip to main content

Amazon Bedrock

Amazon Bedrock is a fully managed service that provides access to powerful foundation models (FMs) from leading AI providers like Anthropic, Meta, and Amazon. It allows developers to easily build and scale generative AI applications without managing infrastructure.

Extended Thinking Support

Amazon Bedrock now supports Extended Thinking capabilities for Anthropic Claude models with step-by-step transparency!

Key Features:

  • ✅ Enable Extended Thinking: Enhance reasoning for complex tasks with transparent thought processes
  • 🎯 Thinking Budget Tokens: Configure the maximum tokens for Claude's internal reasoning
  • 🔍 Step-by-step Transparency: See how Claude approaches complex problems
  • ⚠️ Cost Impact: Extended thinking consumes additional tokens, increasing usage costs
  • 📋 Availability: Available for Claude 3.7 Sonnet and newer versions
info

For enabling models in different AWS regions,refer this cross region inference page

Getting started with Amazon Bedrock​

You can choose your model based on your requirements; the following steps only show how to configure Anthropic in Bedrock.

Step 1: Set Up Your AWS Account​

Pre-requisites​

  • Active AWS account.
  • Necessary permissions: Ensure that your AWS user or role has the necessary permissions to access and use Amazon Bedrock.

Step 2: Access Amazon Bedrock​

Via AWS Console​

  1. Go to the AWS Management Console.
  2. Search for Amazon Bedrock in the search bar.
  3. Open Amazon Bedrock service page.

Step 3: Enable Model Access for Anthropic Claude​

Enable the Model​

  1. In the Bedrock console, go to Model Access in the left sidebar.
  2. You will see a list of Foundation Models (FMs), including Anthropic Claude.
  3. Select Anthropic Claude and click Enable Model.
  4. Review and accept the terms (required the first time you enable a model).

Steps to Get AWS Access Key and Secret Key for Bedrock​

Step 1: Sign In to the AWS Management Console​

  1. Go to the AWS Management Console.
  2. Log in using your AWS root account or an IAM user with sufficient permissions.

Step 2: Create an IAM User with Programmatic Access​

  1. Navigate to IAM:

    • In the AWS Management Console, search for IAM (Identity and Access Management).
    • Click on IAM to open the IAM dashboard.

  2. Create New User:

    • In the left sidebar, click on Users.
    • Click the Add user button.

  3. Set Permissions:

    • In the User details step, provide a User name.
    • Under Access type, select Programmatic access to allow access through the AWS CLI, SDKs, or APIs.

  4. Assign Permissions:

    • In the Set permissions step, you have two options:

      • Attach policies directly: Search for and attach policies that grant the necessary permissions to use Amazon Bedrock. A simple policy to invoke a specific foundation model will look like:
      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "bedrock:InvokeFoundationModel"
            ],
            "Resource": [
              "arn:aws:bedrock:us-east-1:123456789012:model/bedrock-model-id"
            ]
          }
        ]
      }

      For more information, see Amazon Bedrock IAM policies.

      • Add user to group: If you have an existing group with appropriate permissions, you can select that group.
      • Attach customer managed policies: For fine-grained control, create a custom policy (such as restricting access only to Bedrock).

    • Once selected, click Next.

Step 3: Review and Create User​

  1. Review User Details:

    • Verify the permissions and access settings.

  2. Create User:

    • Click Create user.
    • After creation, AWS will display a success message with the Access Key ID and Secret Access Key.

    Important: Copy the Secret Access Key immediately, as you won’t be able to view it again.

Step 4: Store the Access Keys Securely​

  • Store the Access Key ID and Secret Access Key in a secure location (for example, AWS Secrets Manager, environment variables, or an encrypted file).
  • You will use these keys to connect to Unstract.

Setting up the Anthropic LLM model in Unstract​

Now that we have an model deployed and the required keys, we can use it to set up an LLM profile on the Unstract platform. For this:

  • Sign in to the Unstract Platform
  • From the side navigation menu, choose Settings 🞂 LLMs
  • Click on the New LLM Profile button.
  • From the list of LLMs, choose Bedrock. You should see a dialog box where you enter details.
Extended Thinking Configuration

If you're using Anthropic Claude 3.7 Sonnet through Bedrock, you'll have additional options:

Enable Extended Thinking

  • ☑️ Check this option to activate enhanced reasoning with step-by-step transparency
  • Available for Claude 3.7 Sonnet and newer versions

Thinking Budget Tokens

  • 🎯 Token Budget: Set the maximum tokens for Claude's internal reasoning
  • 💡 Recommendation: Start with 5000 tokens for most use cases
  • 📈 Higher Budget: More tokens = more detailed reasoning but increased costs

⚠️ Important: Extended thinking consumes additional tokens, which will increase your AWS usage costs

img Google Anthropic LLM Configuration

  • For Name, enter a name for this connector.
  • In the Model Name enter the model which you deployed.
  • For AWS Secret Key and AWS Access Key enter the keys downloaded from AWS
  • Leave the Max retries and Timeout fields to their default values.
  • For the Maximum Output Tokens, enter the maximum token limit supported by the model, or leave it empty to use the default maximum.
  • For Claude 3.7 Sonnet and newer: Enable extended thinking and set your preferred thinking budget tokens
  • Click on Test Connection and ensure it succeeds. You can finally click on Submit and that should create a new LLM Profile for use in your Unstract projects.