AWS Marketplace Troubleshooting
How to read the LLMWhisperer appliance's logs and fix the common issues. Every command runs on the instance: connect with Session Manager or SSH as described in the Deployment Guide.
--no-pagerIn a Session Manager shell, journalctl and systemctl repaint their first screen, so lines appear twice. Pass --no-pager, as the commands below do. Run journalctl with sudo: the Session Manager user cannot read the system journal without it.
Start Here
sudo python3 /usr/local/lib/llmw/llmw_provision.py --status # license and stack state
systemctl --no-pager list-units 'llmw*' 'llmwhisperer*' --all # the appliance's services
sudo docker compose -f /opt/llmwhisperer/docker-compose.yml ps # the containers
A healthy appliance shows "state": "provisioned", "first_boot_complete": true and "stack": "running". llmwhisperer.service shows active (exited), which is normal for this service. All containers are running, and those with a health check are healthy. Every state and stack value is explained in the Deployment Guide.
Logs
Appliance services
| Service | What it does | Logs |
|---|---|---|
llmwhisperer-firstboot | First boot: license, sizing, credentials, first start of the stack. Does not run again once it has rendered the instance's configuration. | sudo journalctl --no-pager -u llmwhisperer-firstboot |
llmw-provision | Renews the license every few hours, and finishes a first boot that could not get a license. Runs every 15 minutes. | sudo journalctl --no-pager -u llmw-provision |
llmwhisperer | Starts and stops the stack (docker compose up / down). | sudo journalctl --no-pager -u llmwhisperer |
For example, boot and license together, this boot only:
sudo journalctl --no-pager -b -u llmwhisperer-firstboot -u llmw-provision -u llmwhisperer
Containers
sudo docker compose -f /opt/llmwhisperer/docker-compose.yml logs --tail 200 <service>
| Service | What it is |
|---|---|
llm-whisperer-backend | The document API (port 3006) |
celery-workers | The document-processing workers |
llm-whisperer-dashboard | The dashboard (port 3007) |
x-to-pdf | Office and other document conversion |
llmw-db | PostgreSQL |
llmw-redis | Redis |
rabbit-mq | RabbitMQ |
sudo docker compose -f /opt/llmwhisperer/docker-compose.yml ps --services lists every service running on your instance.
Without a shell: the serial console
First boot and license provisioning also write to the serial console, so you can read them without logging in: in the EC2 console, Actions → Monitor and troubleshoot → Get system log, or:
aws ec2 get-console-output --instance-id <instance-id> --latest --output text
What to send to support
Email support@unstract.com with your AWS account ID, the AWS Region, the instance type, the version you launched, and the output of:
sudo python3 /usr/local/lib/llmw/llmw_provision.py --status
sudo journalctl --no-pager -b -u llmwhisperer-firstboot -u llmw-provision -u llmwhisperer
sudo docker compose -f /opt/llmwhisperer/docker-compose.yml ps
sudo docker compose -f /opt/llmwhisperer/docker-compose.yml logs --tail 500 <service> # for the service involved
/opt/llmwhisperer/credentials— your API key and dashboard login.envor any other*.envfile in/opt/llmwhisperer— they hold the instance's passwords and license credentials- the output of
docker inspectordocker compose config, which include the same values
Read through any log before sending it, and remove anything that looks like a key, token or password.
Common Issues
The license is not provisioned
On a new instance, --status shows a state other than provisioned, and the stack is not started (stack stays needs_start). (On an instance that already has a license, a failed renewal also shows here, but the stack keeps running.) The license state table lists every value. The usual causes:
state | Usual cause | Fix |
|---|---|---|
refused | The AWS account the instance runs in has no active subscription to the listing: it was never subscribed, the subscription has not become active yet, or it has ended. The subscription must be in the same account as the instance. | Subscribe in that account. The appliance retries on its own; to retry now, run sudo systemctl start llmw-provision.service. |
no_instance_role | No IAM role is attached, typically after a one-click launch. | Attach an IAM instance role, then run sudo systemctl start llmw-provision.service. |
unavailable | No outbound HTTPS (443) from the instance: no NAT gateway, no internet gateway route, no public IP in a public subnet, or an outbound security group or network ACL rule that blocks 443. | Fix the route or rule. The appliance retries on its own. If the network is fine, contact support. |
The reason is also shown in the login banner when you connect over SSH, and on the serial console.
You do not need to reboot. When the appliance obtains the license, it finishes first boot and starts the stack by itself. With a working network that happens on the next retry, up to about 26 minutes later, plus the time to start the stack. llmw-provision.service showing activating for several minutes is normal while it retries an unreachable license service: it keeps trying for up to 20 minutes.
To follow it live:
sudo journalctl --no-pager -f -u llmw-provision -u llmwhisperer-firstboot # Ctrl-C to stop
The API or dashboard is not reachable
- Is the stack up?
--statusshould show"stack": "running". - Does it answer on the instance?
curl --fail --show-error http://127.0.0.1:3006/health/pingshould succeed with HTTP 200. If it does not, see The stack does not start. - Is the client allowed? For CloudFormation templates with
DashboardCidr, the document API (3006) usesAllowedCidr; the dashboard (3007) usesDashboardCidrand is closed to inbound traffic when that parameter is empty.0.0.0.0/0permits access from any IPv4 address. The currently published v2.65.0 template has no separate dashboard range: port3007followsAllowedCidr, and this image's usage/license routes do not require login. For a direct AMI launch, check the ranges in your own security group. If the dashboard is closed, reach it through Session Manager port forwarding. - Is the address right? The stack outputs give the private IP. From outside the VPC, use the instance's public IP. An instance only has one if its subnet auto-assigns public IPs (or you associated an Elastic IP), and the subnet routes to an internet gateway.
- Did the address change? A public IP assigned by the subnet changes when the instance is stopped and started. Use an Elastic IP for a stable address.
- Is it the key? If the API answers but rejects the request with
401, check theunstract-keyheader against/opt/llmwhisperer/credentials.
To test without changing the security group, use Session Manager port forwarding.
The stack does not start
--status shows "state": "provisioned" but stack stays needs_start or unknown well after first boot should have finished (35 minutes after launch), or stack is running but curl -s http://127.0.0.1:3006/health/ping gets no answer, or sudo systemctl --no-pager status llmwhisperer-firstboot shows failed.
- Read the first-boot log:
sudo journalctl --no-pager -u llmwhisperer-firstboot. The usual cause is the API not answering in time because a container keeps restarting. - Find that container:
sudo docker compose -f /opt/llmwhisperer/docker-compose.yml ps -a, then read its logs. - Start the stack again:
sudo systemctl restart llmwhisperer.service(startdoes nothing once the service is already active).
If it still does not start, send the diagnostics to support.
The first high_quality, form or table request is slow
The first high_quality, form or table request after the appliance starts can take several minutes while the OCR engines initialise; later requests are processed normally. This happens after first boot and after every restart of the stack or the instance.
high_quality, form and table stop working while native_text and low_cost still work
These three modes need ongoing outbound HTTPS to the Unstract license service. If the instance loses it, they stop processing after roughly 100 to 150 minutes, while native_text and low_cost keep working until the license expires. Restore outbound HTTPS (443) from the instance. If those modes do not resume, restart the stack with sudo systemctl restart llmwhisperer.service. Their first requests afterwards can take several minutes while the OCR engines initialise.
Documents are refused with 402 InvalidLicense
The appliance has no valid license, for example because the subscription has ended. The containers stay up. Check state, message and licence_expires_on in --status. After subscribing again in the same AWS account, launch a new instance; contact support if you need to keep this one.
Webhooks never arrive, or documents fetched from a URL fail
With OutboundAccess set to https-only (the default), the instance can reach only TCP 443. A webhook receiver or a document URL on http://, or on any other port, is unreachable. Update the stack with OutboundAccess set to all, or add an outbound rule for that destination to the security group.
Usage does not appear on the AWS bill
- The instance role must allow
aws-marketplace:MeterUsage, and the instance needs outbound HTTPS to the AWS Marketplace Metering Service. - Usage is reported every hour. An hour with no documents processed sends nothing, which is normal.
- Reported usage normally appears in AWS Cost Explorer the next day.
If usage is still missing after the expected billing delay, confirm the instance role and outbound HTTPS access, then contact Unstract support with your AWS account ID, Region, instance ID, and the approximate time pages were processed.
Safe to Do, and Not
| Action | Safe? |
|---|---|
sudo systemctl start llmw-provision.service | Yes, any time. It is what the 15-minute timer runs. |
sudo systemctl restart llmwhisperer.service | Yes. Requests open at that moment are dropped, and documents being processed are processed again after the restart. Stored data survives. |
| Reboot, or stop and start the instance | Yes. Everything starts again by itself. |
docker compose down -v | No. -v deletes the database, Redis and RabbitMQ volumes. |
Editing the license files in /opt/llmwhisperer by hand | No. The appliance manages them. |