Version: 2.0.0

AWS Marketplace Deployment Guide

This guide covers deploying the LLMWhisperer appliance from AWS Marketplace into your own AWS account, checking that it is ready, and calling its API. For an overview of the edition, see AWS Marketplace Edition. For logs and common issues, see AWS Marketplace Troubleshooting.

Overview

The appliance is a single EC2 instance (Amazon Linux 2023) that runs the complete LLMWhisperer stack with Docker Compose:

Document API — the LLMWhisperer v2 API, on port 3006
Dashboard — a web UI on port 3007
Processing workers and OCR engines — sized to the instance type on first boot
PostgreSQL, Redis and RabbitMQ — the appliance's own database, cache and message broker, private to the instance

There are two delivery options:

Option	What you get
CloudFormation (recommended)	A stack that creates the instance, an IAM instance role, a security group limited to the address ranges you enter, and an encrypted gp3 root volume.
AMI	The appliance image only. You launch it from the EC2 console and attach the IAM instance role and security group yourself.

Billing. You are billed per page processed, at the rate for the extraction mode used (native_text, low_cost, high_quality, form or table), through AWS Marketplace. Excel files (XLS/XLSX) are billed in form mode. Usage is reported every hour. EC2 charges for the instance and its storage are billed separately by AWS.

External dependencies. The appliance needs ongoing outbound HTTPS (TCP 443) to the Unstract license service, which issues the license on first boot and renews it, and to the AWS Marketplace Metering Service, which receives the hourly page counts. If the instance loses access to the license service, the high_quality, form and table modes stop processing after roughly 100 to 150 minutes; native_text and low_cost keep working until the current license expires, and then the appliance stops processing documents.

The API and dashboard are plain HTTP

Ports 3006 and 3007 serve plain HTTP, without TLS. The API key, the dashboard login and your documents cross the network unencrypted. Securing that traffic is your responsibility. For example:

keep the API on private networks only (the VPC, a VPN, or peered networks), or
put a load balancer with a TLS listener in front of the instance and allow ports 3006 and 3007 only from the load balancer, or
reach the instance through Session Manager port forwarding.

Open the document API to the internet (0.0.0.0/0) only if you accept that risk. The CloudFormation template also permits DashboardCidr=0.0.0.0/0 by buyer choice, but this exposes the plain-HTTP dashboard to every IPv4 address. Appliance images without the dashboard route-auth fix may allow usage and license routes without login; prefer leaving the dashboard closed or restricting it to a trusted range.

1. Before You Launch

An AWS account subscribed to the listing. On the LLMWhisperer AWS Marketplace listing, choose Continue to Subscribe and accept the terms. The subscription must be active before the instance boots; if it is not, the license is refused and retried automatically (see License states).
A VPC and subnet with outbound HTTPS (TCP 443) to the internet, through a NAT gateway or an internet gateway. This is needed whether or not clients reach the instance from the internet.

The template uses an existing VPC and subnet; it does not create networking. You can use a default VPC and one of its public subnets if they exist in your account and Region. Check that the subnet routes to an internet gateway and auto-assigns public IPv4 addresses. For a private subnet, provide outbound internet access through a NAT gateway.
An IAM instance role allowing aws-marketplace:MeterUsage. The CloudFormation option creates it. For the AMI option you create it (below). Without a role the appliance cannot obtain its license and does not start.
A way to get a shell on the instance once, to read the API key and dashboard login: AWS Systems Manager Session Manager (no inbound port) or SSH.

2. Deploy with CloudFormation (recommended)

On the listing, choose Continue to Subscribe, then Continue to Configuration.
Choose the CloudFormation delivery option, the software version and your AWS Region, then Continue to Launch.
Launch the template. The CloudFormation console opens with the template loaded.
Fill in the parameters (below), acknowledge that CloudFormation will create IAM resources, and create the stack.
When the stack reaches CREATE_COMPLETE, the instance is still on first boot. Check its progress before sending documents.

Parameters

Parameter	Default	Guidance
`VpcId`	—	The VPC to launch into.
`SubnetId`	—	A subnet in that VPC that routes outbound HTTPS (443) to the internet, through a NAT gateway or an internet gateway. To reach the instance from the internet, use a public subnet: one with a route to an internet gateway that auto-assigns public IPv4 addresses. The stack does not assign a public IP itself.
`AllowedCidr`	none (required)	The IPv4 range (CIDR) allowed to reach the document API (`3006`). Any prefix from `/0` to `/32` is accepted. The API uses plain HTTP, so use the narrowest range that covers your clients, for example `10.0.0.0/16`. Widening it, up to `0.0.0.0/0`, is your decision and your risk. This parameter does not open the dashboard or SSH.
`DashboardCidr`	empty (closed)	Optional IPv4 range (CIDR) allowed to reach the dashboard (`3007`). Leave empty to keep it closed; otherwise use exactly `0.0.0.0/0` or a range from `/8` to `/32`. `0.0.0.0/0` allows any IPv4 address to connect. The dashboard is plain HTTP, and images without the route-auth fix may allow usage/license routes without login; public access is your decision and risk. You can also reach it through Session Manager port forwarding.
`OutboundAccess`	`https-only`	`https-only` allows outbound TCP 443 only, which is enough for the license service, AWS Marketplace metering and Session Manager. Choose `all` if your webhook receivers, or documents you ask LLMWhisperer to fetch from a URL, use `http://` or a port other than 443.
`InstanceType`	`m5.2xlarge`	Any type offered in the list. 8 vCPU / 32 GiB (`m5.2xlarge`) is the smallest that runs every extraction mode. Choose the size at launch: see Sizing.
`VolumeSize`	`100`	Root volume size in GiB, minimum `50`. It holds the container images, the database and documents. A freshly started instance uses about 15 GB.
`KeyName`	empty	Optional. An existing EC2 key pair, installed on the instance at launch. A key pair alone does not open port 22; set `SshCidr` as well to use SSH. Leave empty to use Session Manager.
`SshCidr`	empty	Optional. The IPv4 range (CIDR) allowed to reach SSH (port `22`). Leave empty to keep port 22 closed; Session Manager needs no inbound port. It does not depend on `KeyName`: EC2 Instance Connect can push a short-lived key to an instance launched without a key pair.
`ImageId`	set by AWS Marketplace	The appliance AMI for the version and Region you chose. Leave it as it is.

The currently published v2.65.0 template

The published v2.65.0 template has neither DashboardCidr nor SshCidr. Port 3007 is opened to AllowedCidr along with the document API on 3006; the dashboard's usage and license routes do not require login in this appliance image. Keep AllowedCidr narrowly scoped when launching it. Giving a KeyName also opens port 22 to AllowedCidr, which must be /8 or narrower (0.0.0.0/0 is rejected). Templates that include DashboardCidr use the separate range described above.

What the stack creates

An IAM instance role and instance profile. The role allows aws-marketplace:MeterUsage (usage reporting) and the minimum actions Session Manager needs. The license request proves your AWS account with the role's credentials and needs no further permission.
A security group. Inbound: 3006 from AllowedCidr; 3007 from DashboardCidr only if you set it; and 22 from SshCidr only if you set it. Outbound: TCP 443 only, or all IPv4 traffic if OutboundAccess is all.
The EC2 instance, launched with IMDSv2 required (hop limit 1) and an encrypted gp3 root volume that is deleted when the instance is terminated.

Stack outputs

Output	Description
`InstanceId`	The appliance instance.
`DocumentApiUrl`	`http://<private IP>:3006/api/v2`. Clients outside the VPC use the instance's public IP, if the subnet assigned one.
`DashboardUrl`	`http://<private IP>:3007/`.
`InstanceRoleArn`	The instance role.

3. Deploy the AMI from the EC2 Console

On the listing, choose Continue to Subscribe, then Continue to Configuration.
Choose the AMI delivery option, the software version and your AWS Region, then Continue to Launch.
Choose Launch from EC2 Console.

Do not use one-click launch

One-click launch does not attach an IAM instance role. Without one, the appliance cannot obtain its license and does not start.

In the EC2 launch wizard, set:
- Instance type: m5.2xlarge or larger. See Sizing.
- Network: a subnet with outbound HTTPS (443), as described for SubnetId above. Enable Auto-assign public IP only if clients reach the instance from the internet.
- Security group: see Recommended security group.
- Storage: the AMI's root volume is 50 GiB gp3 and is deleted on termination. The CloudFormation option defaults to 100 GiB; increase it to suit the documents you keep.
- Advanced details → IAM instance profile: the role described in IAM instance role.
- Advanced details → Metadata: keep IMDSv2 required. A metadata response hop limit of 1 matches the CloudFormation option.
- Key pair: optional. Without one, use Session Manager or EC2 Instance Connect.
Launch the instance, then follow First boot and the status check.

IAM instance role

Create a role that Amazon EC2 can assume, with at least this permission:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "aws-marketplace:MeterUsage",
      "Resource": "*"
    }
  ]
}

To use Session Manager, the role also needs the Session Manager actions, for example through the AWS managed policy AmazonSSMManagedInstanceCore, or only these:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ssm:UpdateInstanceInformation",
        "ssmmessages:CreateControlChannel",
        "ssmmessages:CreateDataChannel",
        "ssmmessages:OpenControlChannel",
        "ssmmessages:OpenDataChannel"
      ],
      "Resource": "*"
    }
  ]
}

Recommended security group

Direction	Protocol / port	Source / destination	Purpose
Inbound	TCP `3006`	Your clients' range. Use the narrowest range that covers your clients.	Document API (plain HTTP)
Inbound	TCP `3007`	`DashboardCidr` when the template has that parameter; `AllowedCidr` in the published v2.65.0 template. Empty keeps it closed when `DashboardCidr` is available.	Dashboard (plain HTTP); images without the route-auth fix may have usage/license routes that do not require login. `0.0.0.0/0` allows any IPv4 address.
Inbound	TCP `22`	Your administrators' range, only if you use SSH	SSH
Outbound	TCP `443`	`0.0.0.0/0`	License service, AWS Marketplace metering, Session Manager
Outbound	All traffic	`0.0.0.0/0`	Only if webhook receivers or document URLs use `http://` or a port other than 443

4. First Boot and the Status Check

On first boot the appliance, with no action from you:

obtains its license from the Unstract license service, tied to your AWS Marketplace subscription;
sizes its workers and OCR engines to the instance type;
generates this instance's API key, dashboard login and internal passwords;
starts the stack and waits until the document API answers.

Check progress

Connect with Session Manager or SSH (user ec2-user) and run:

sudo python3 /usr/local/lib/llmw/llmw_provision.py --status

The stack has been started when the output shows "state": "provisioned", "first_boot_complete": true and "stack": "running":

{
  "apply_pending": false,
  "first_boot_complete": true,
  "last_attempt": "2026-10-09T02:30:23.354570+00:00",
  "last_success": "2026-10-09T02:30:23.354570+00:00",
  "licence_expires_on": "2026-10-11",
  "message": "credentials confirmed",
  "stack": "running",
  "state": "provisioned",
  "subscription_id": "<your subscription id>"
}

The appliance is ready when, in addition, the document API returns HTTP 200 on the instance:

curl --fail --show-error http://127.0.0.1:3006/health/ping

Field	Meaning
`state`	The license, not the stack. See License state. Absent until the first license attempt has finished.
`message`	The outcome of the last license attempt, including the reason for a refusal.
`last_attempt` / `last_success`	When the license was last requested, and last obtained or renewed (UTC).
`licence_expires_on`	When the current license expires. It is renewed automatically, well before that date.
`subscription_id`	The subscription the license is tied to.
`apply_pending`	`true` while renewed license credentials are waiting to be applied to the running services. It clears on its own.
`first_boot_complete`	Whether first boot finished rendering the instance's configuration. It does not say the stack is up; `stack` does.
`stack`	Whether the stack has been started. See Stack state.

How long first boot takes

Measured: on one m5.2xlarge launched through the listing in us-east-1 on 2026-10-08, with an active subscription, the license was obtained on the first attempt and:

Time after launch	Event
1 min 30 s	First boot started (after Docker and network time synchronisation were up)
1 min 31 s	License obtained
1 min 56 s	Configuration rendered; all containers created and started
3 min 57 s	Document API answered; `stack` became `running`

The times come from the instance's EC2 launch time and the timestamps in its journalctl and Docker records. That is one launch; allow longer. Plan for 15 minutes or more, for example when the subscription has only just become active or a container is slow to turn healthy. First boot gives up after 35 minutes: it waits up to 10 minutes for the license, and up to 12 minutes each for the containers to start and for the API to answer.

After stack shows running, the first high_quality, form or table request after the appliance starts can take several minutes while the OCR engines initialise; later requests are processed normally.

License state

`state`	Meaning	Normally seen	If it persists
(absent)	The first license attempt has not finished.	From launch until first boot's first license request finishes: about 1.5 minutes after launch in the measurement above, and up to 10 minutes longer if the license service cannot be reached.	Check outbound HTTPS 443 from the subnet.
`provisioned`	License issued or renewed.	Always, on a healthy appliance.	—
`unavailable`	The license service could not be reached, or did not give a usable answer.	Only during a network or service problem.	Check outbound HTTPS 443 from the subnet (NAT gateway, or internet gateway with a public IP), and the security group's outbound rules. If the network is fine, contact support. The appliance retries on its own.
`refused`	The license service checked and said no. `message` says why, usually that the AWS account has no active AWS Marketplace subscription.	When the subscription is missing, not yet active, or has ended.	Subscribe (or wait for the subscription to become active) in the same AWS account the instance runs in. The appliance retries on its own; to retry now, run `sudo systemctl start llmw-provision.service`.
`no_instance_role`	The instance has no IAM role, so it cannot prove which AWS account it is in.	After a launch without a role, for example one-click launch.	Attach an IAM instance role (Actions → Security → Modify IAM role), then run `sudo systemctl start llmw-provision.service`.
`misconfigured`	The appliance's license configuration is unusable.	Never.	Contact support.
`error`	Something unexpected. `message` has the detail.	Never.	Contact support with the diagnostics.

The appliance retries a failed license request every 15 minutes, with up to 10 minutes of random delay, so runs are between about 4 and 26 minutes apart. Once it obtains a license, it starts the stack by itself.

If message says "Could not verify this instance's AWS identity", contact support.

Stack state

`stack`	Meaning	Normally seen	If it persists
`needs_start`	The stack has not come up yet on this boot. During first boot this is the expected value: it stays `needs_start` from boot until first boot finishes, normally once the document API answers. It also shows when first boot has not rendered the configuration yet, or when a start or restart of the stack failed.	Throughout first boot: about 4 minutes after launch in the measurement above, and longer when the license or a container is slow. Seeing it for 5 minutes during first boot is normal. Expect it to persist while `state` is not `provisioned`, because the stack is never started without a license.	If `state` is not `provisioned`, fix the license first (above). If `state` is `provisioned` and it is still `needs_start` more than 35 minutes after launch, see The stack does not start.
`running`	The stack's containers have been started. It does not by itself prove the API answers: check that too (below).	Always, on a healthy appliance.	If the API does not answer, see The stack does not start.
`stopped`	The stack came up on this boot and was then stopped on purpose, for example with `systemctl stop`. The appliance leaves it stopped.	Only after someone stops it.	Start it with `sudo systemctl start llmwhisperer.service`.
`unknown`	The stack is starting, stopping or restarting, or its state could not be read.	Briefly, during a start, stop or restart.	Run the status check again. If it stays `unknown`, see The stack does not start.

5. Get Your Credentials

Each instance generates its own API key and dashboard login on first boot. They are stored only on the instance, in a file readable by root:

sudo cat /opt/llmwhisperer/credentials

The file contains:

the API key, sent as the unstract-key header on every API request;
the dashboard username (llmw) and password.

Run it from a shell on the instance:

Session Manager: in the EC2 console, select the instance and choose Connect → Session Manager, or run aws ssm start-session --target <instance-id>. You land as ssm-user; use sudo as shown.
SSH: ssh -i <key.pem> ec2-user@<instance address>, or EC2 Instance Connect.

Keep the credentials private

Anyone with the API key can submit documents, and you are billed for them. Anyone with the dashboard login can see your processing records. In appliance images without the dashboard route-auth fix, some routes, including usage download, do not require login; keep port 3007 closed or restrict it to a trusted network. The dashboard has no password-change screen in this version, so treat the credentials file as sensitive. If your account has Session Manager session logging turned on, anything printed in a session, these credentials included, is copied to its S3 bucket or CloudWatch log group.

6. Use the API and Dashboard

Addresses

Service	Address
Document API	`http://<instance address>:3006/api/v2`
Dashboard	`http://<instance address>:3007/`
Health check (no key needed)	`http://<instance address>:3006/health/ping`

Use the instance's private IP from inside the VPC or a network routed into it, and its public IP (if it has one) from the internet. The client must be inside the range the security group allows.

Extract a document

# Submit a document. The response contains a whisper_hash.
curl -X POST "http://<instance address>:3006/api/v2/whisper?mode=native_text&output_mode=layout_preserving" \
  -H "unstract-key: <API key>" \
  --data-binary @document.pdf

# Check progress until the status is "processed".
curl "http://<instance address>:3006/api/v2/whisper-status?whisper_hash=<whisper_hash>" \
  -H "unstract-key: <API key>"

# Retrieve the extracted text.
curl "http://<instance address>:3006/api/v2/whisper-retrieve?whisper_hash=<whisper_hash>" \
  -H "unstract-key: <API key>"

Supported modes: native_text, low_cost, high_quality, form and table (see Modes).

The appliance serves the same v2 API as LLMWhisperer Cloud. Use http://<instance address>:3006/api/v2 in place of the cloud base URL:

API reference: Extraction, Status, Retrieve
Postman collection: after importing, set baseUrl to http://<instance address>:3006/api/v2 and apiKey to your key
Python client and JavaScript client: set the base URL (base_url / baseUrl, or LLMWHISPERER_BASE_URL_V2) to your instance's API address
Webhooks: receivers on http:// or on a port other than 443 need OutboundAccess set to all (or an equivalent outbound rule)

Dashboard

Open http://<instance address>:3007/ and sign in with the username and password from the credentials file. It is the same dashboard as in the On-Prem edition, described in the Dashboard Guide. On the appliance the license is obtained and renewed automatically, so you do not need to upload a license file.

Reach the API through Session Manager

If the instance is in a private subnet, or you prefer not to open the security group, forward the ports to your machine with Session Manager (AWS CLI v2 and the Session Manager plugin). The tunnel reaches the port from inside the instance, so the security group's inbound rules do not apply to it.

# Document API -> http://localhost:13006
aws ssm start-session --target <instance-id> \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["3006"],"localPortNumber":["13006"]}'

# Dashboard -> http://localhost:13007 (in another terminal)
aws ssm start-session --target <instance-id> \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["3007"],"localPortNumber":["13007"]}'

7. Sizing

On first boot the appliance sizes its processing to the instance's vCPU and memory. This is vertical scaling within one instance: a larger machine processes more pages in parallel. It does not add EC2 instances or change the instance type automatically as traffic grows.

Measured throughput

Pages per minute with one extraction mode running at a time, at full load:

Mode	`m5.2xlarge` (8 vCPU, 32 GiB)	`m5.4xlarge` (16 vCPU, 64 GiB)	8xlarge (32 vCPU, 128 GiB)¹
`high_quality`	~30	~55	~150
`form`	~20	~35	~90
`table`	~20	~30	~90
`low_cost`	~25	~65	~180
`native_text`	150+	350+	1,500+

¹ Measured on m6i.8xlarge. An m5.8xlarge has slightly slower cores and will be somewhat lower.

These figures come from our own test documents. Throughput depends on page size, density and image quality, and native_text varies the most. Benchmark your own documents before choosing a size.

Throughput peaks at a modest number of concurrent requests. The CPU is fully used from about 4 to 8 concurrent requests on the 2xlarge and 4xlarge, and about 30 to 60 on the 8xlarge. Sending more than that does not increase pages per minute; requests wait longer instead.
Modes share the CPU. Running several modes at once splits the capacity between them, so the figures above do not add up across modes.
A bigger instance raises throughput, not idle speed. Under load, requests finish sooner on a bigger instance because more pages are processed at once. A single page on an idle appliance is not much faster on a bigger instance, because each page is still processed by one OCR engine.
Gains level off above about 12xlarge. The appliance caps how many processing tasks it runs at once at the level it reaches on about a 12xlarge, so larger instance types are unlikely to add much throughput. We have not measured above 8xlarge.
Memory is rarely the limit. In the tests above, more than 15 GiB stayed free on the m5.2xlarge. A type with the same vCPU and more memory (for example r6i) gets the same plan; the extra memory does not add throughput. On memory-lean types (the c families), the appliance may run fewer OCR engines than the vCPU count alone would give.

To see the plan your instance chose:

sudo journalctl --no-pager -u llmwhisperer-firstboot | grep -E 'instance:|worker concurrency:|memory'

Instance size is fixed at first boot

In the current version, the appliance sizes itself only on its first boot. If you later change the instance type of an existing instance, it keeps the plan it made for the original size. To run at a different size, launch a new instance at the new size (a new CloudFormation stack, or a new AMI launch), then move your clients to it.

8. Operating the Appliance

Stop, start and reboot

Stop and start, or reboot: the stack starts again by itself. The instance still needs an active subscription and outbound HTTPS for license renewal. Besides the hourly reports, the appliance attempts a final usage report when it shuts down cleanly.
Public IP: a public IP assigned by the subnet changes when the instance is stopped and started. Associate an Elastic IP if clients need a stable address.
Restart only the stack: sudo systemctl restart llmwhisperer.service. Requests open at that moment are dropped; documents being processed are processed again after the restart. Stored data survives.

Versions and upgrades

Each version on the listing is a new AMI (and template). To move to a newer version, launch a new instance from it, read its new credentials, move your clients to it, and terminate the old instance when you no longer need it. Processing records stay on the instance that produced them.

Terminate

Terminating the instance deletes its root volume: the database of processing records, the logs and this instance's credentials are deleted with it. Save any results and diagnostics you need first. Usage reporting at shutdown is best-effort; usage not successfully reported before the volume is deleted cannot be recovered. For a CloudFormation deployment, delete the stack to remove the instance, its role and its security group together.

Do not

Run docker compose down -v: -v deletes the database, Redis and RabbitMQ volumes.
Edit the license files in /opt/llmwhisperer by hand: the appliance manages them.

Overview​

1. Before You Launch​

2. Deploy with CloudFormation (recommended)​

Parameters​

What the stack creates​

Stack outputs​

3. Deploy the AMI from the EC2 Console​

IAM instance role​

Recommended security group​

4. First Boot and the Status Check​

Check progress​

How long first boot takes​

License state​

Stack state​

5. Get Your Credentials​

6. Use the API and Dashboard​

Addresses​

Extract a document​

Dashboard​

Reach the API through Session Manager​

7. Sizing​

Measured throughput​

8. Operating the Appliance​

Stop, start and reboot​

Versions and upgrades​

Terminate​

Do not​