Amazon Bedrock
Amazon Bedrock is a fully managed service that provides access to powerful foundation models (FMs) from leading AI providers like Anthropic, Meta, and Amazon. It allows developers to easily build and scale generative AI applications without managing infrastructure.
Getting started with Amazon Bedrock
You can choose your model based on your requirements; the following steps only show how to configure Anthropic in Bedrock.
Step 1: Set Up Your AWS Account
Pre-requisites
- Active AWS account.
- Necessary permissions: Ensure that your AWS user or role has the necessary permissions to access and use Amazon Bedrock.
Step 2: Access Amazon Bedrock
Via AWS Console
- Go to the AWS Management Console.
- Search for Amazon Bedrock in the search bar.
- Open Amazon Bedrock service page.
Step 3: Enable Model Access for Anthropic Claude
Enable the Model
- In the Bedrock console, go to Model Access in the left sidebar.
- You will see a list of Foundation Models (FMs), including Anthropic Claude.
- Select Anthropic Claude and click Enable Model.
- Review and accept the terms (required the first time you enable a model).
Steps to Get AWS Access Key and Secret Key for Bedrock
Step 1: Sign In to the AWS Management Console
- Go to the AWS Management Console.
- Log in using your AWS root account or an IAM user with sufficient permissions.
Step 2: Create an IAM User with Programmatic Access
-
Navigate to IAM:
- In the AWS Management Console, search for IAM (Identity and Access Management).
- Click on IAM to open the IAM dashboard.
-
Create New User:
- In the left sidebar, click on Users.
- Click the Add user button.
-
Set Permissions:
- In the User details step, provide a User name.
- Under Access type, select Programmatic access to allow access through the AWS CLI, SDKs, or APIs.
-
Assign Permissions:
-
In the Set permissions step, you have two options:
- Attach policies directly: Search for and attach policies that grant the necessary permissions to use Amazon Bedrock.
- Add user to group: If you have an existing group with appropriate permissions, you can select that group.
- Attach customer managed policies: For fine-grained control, create a custom policy (such as restricting access only to Bedrock).
-
Once selected, click Next.
-
Step 3: Review and Create User
-
Review User Details:
- Verify the permissions and access settings.
-
Create User:
- Click Create user.
- After creation, AWS will display a success message with the Access Key ID and Secret Access Key.
Important: Copy the Secret Access Key immediately, as you won’t be able to view it again.
Step 4: Store the Access Keys Securely
- Store the Access Key ID and Secret Access Key in a secure location (for example, AWS Secrets Manager, environment variables, or an encrypted file).
- You will use these keys to connect to Unstract.
Setting up the Anthropic LLM model in Unstract
Now that we have an model deployed and the required keys, we can use it to set up an LLM profile on the Unstract platform. For this:
-
Sign in to the Unstract Platform
-
From the side navigation menu, choose
Settings
🞂LLMs
-
Click on the
New LLM Profile
button. -
From the list of LLMs, choose
Bedrock
. You should see a dialog box where you enter details. -
For
Name
, enter a name for this connector. -
In the
Model Name
enter the model which you deployed. -
For
AWS Secret Key
andAWS Access Key
enter the keys downloaded from AWS -
Leave the
Max retries
andTimeout
fields to their default values. -
For the
Maximum Output Tokens
, enter the maximum token limit supported by the model, or leave it empty to use the default maximum. -
Click on
Test Connection
and ensure it succeeds. You can finally click onSubmit
and that should create a new LLM Profile for use in your Unstract projects.