Human in the Loop (HITL) Rule Configuration

Overview

Human in the Loop (HITL) allows you to introduce manual review steps into your workflow. This ensures data accuracy by routing specific files to human reviewers based on configurable rules.

Accessing HITL Configuration

1. After setting up tools in Step 3 of your workflow
2. Click on "Configure" in Step 2 (Destination Connector)
3. Navigate to the "Human in the Loop" tab

HITL Rule Configuration

Rules Tab

1. Percentage of Files Sent to Manual Review

Control what portion of your processed files should be routed to manual review:

• Set a percentage value (0-100%)
• Example: If you set 50% and process 2 files:
  • 1 file will be sent to HITL for manual review
  • 1 file will be sent directly to the database

---

2. Add Rules

Create custom rules to determine which files require manual review. All prompt keys created in the Prompt Studio will be available in the Field dropdown.

Rule Types

You can configure rules using four different filter methods:

Confidence rules require highlight to be enabled in the workflow

Confidence data is produced only when Enable highlight (and preferably Enable word confidence) is turned on in the workflow's tool settings and the LLM profile used by the exported Prompt Studio project uses LLMWhisperer as its Text Extraction service. Highlight data is not collected for other text extractors, even if Enable highlight is checked. This applies whether the workflow is deployed as an API, ETL or Task pipeline. The Highlighting settings in Prompt Studio apply only within Prompt Studio and are not used by deployments.

If either requirement is not met, no confidence data is generated and HITL treats every field as 100% confident. A low-confidence rule such as confidence <= 80 will then never match and every document skips review. Other comparisons still use the default score of 100, so a rule such as confidence >= 80 matches every document instead. The rule editor does not warn about this, so verify the tool settings before relying on confidence rules.

1. Filter by Confidence

   • Route files based on confidence scores
   • Enter percentage threshold (0-100)
   • Example: 80 (for 80% confidence threshold)
   • Requires Enable highlight in the workflow's tool settings. With only Enable highlight, line-level confidence is produced. Turn on Enable word confidence as well to get word-level confidence for the extracted value.

2. Filter by Value

   • Route files based on specific field values
   • Supports text, numbers, booleans, and dates
   • Examples:
     • Text: approved, NAREN
     • Numbers: 1900, 100.50
     • Booleans: true, false
     • Dates: 2020-01-01

3. Filter by Value JSON Sub Field

   • Route files based on nested JSON field values
   • Two input fields: path and value
   • Supports text, numbers, booleans, and dates

   Examples:

   Path	Value	Data Type	Description
   address.city	Boston	Text	Checks city field value
   customer.age	35	Number	Checks age field value
   account.active	true	Boolean	Checks boolean status
   transaction.date	2020-01-01	Date	Checks transaction date
   status	approved	Text	Checks status field value

tip

When working with nested JSON fields, select the parent field from the Field dropdown first (e.g., due_details), then specify only the nested path in the value field (e.g., due_date). Do not include the parent field name in the path value.

Example: If checking city_name within address:

• Field dropdown: Select address
• Value field: Enter city_name (NOT address.city_name)

4. Filter by Confidence JSON Path

• Route files based on confidence at specific JSON paths
• Enter as path:threshold
• Threshold should be a number (0-100)
• Requires Enable highlight (and preferably Enable word confidence) in the workflow's tool settings

Simple Object Examples:

JSON Path	Threshold	Description
address.city:80	80	Checks if confidence for address.city is 80% or below
status:75	75	Checks if confidence for status field is 75% or below
customer.name:85	85	Checks if confidence for customer.name is 85% or below
key1.key2:75	75	Checks confidence at nested key path
invoice.total_amount:90	90	Checks if confidence for invoice total is 90% or below

tip

When specifying JSON paths for confidence checks, select the parent field from the Field dropdown first (e.g., due_details), then specify only the nested path with threshold in the format path:threshold (e.g., due_date:80). Do not include the parent field name in the path.

Example: If checking confidence of city_name within address:

• Field dropdown: Select address
• Path:Threshold: Enter city_name:80 (NOT due_details.city_name:80)

---

Setting Up a Rule

1. Select Field: Choose from the dropdown (all Prompt Studio fields available)

   • Examples: account_holder_name, cc_issuer_name, line_items

2. Choose Operator: Select comparison method

   • == (equals)
   • != (not equals)
   • < (less than)
   • <= (less than or equal)
   • > (greater than)
   • >= (greater than or equal)
   • Starts with
   • Ends with

tip

Important: Choose operators appropriate to your value type. For text values, use text-based operators like Starts with, Ends with, or Contains. For numbers and dates, use comparison operators like <, >, <=, >=.

3. Set Value: Enter the comparison value as a string (including numbers and booleans)

Example Rules

filterByValue.account_holder_name STARTS WITH "NAREN"

filterByValue.line_items[0].quantity == "5"

filterByValue.account.active == "true"

These rules send files to manual review based on the specified conditions.

---

Rule Logic

• Rule Logic: Choose between OR and AND logic to combine multiple rules
• Not: Toggle to negate a rule condition

Managing Rules

• Add rule: Click the + Add rule button to create additional rules
• Delete: Click the red trash icon to remove a rule
• Save Rules: Click the Save Rules button to apply your configuration

---

Settings Tab

Configure what happens after manual review approval:

After Approval Send Result To

Choose where approved data should be sent:

1. Destination DB

   • Approved results are sent directly to the configured destination database

2. Queue

   • Approved results are sent to a queue
   • Extract values using API calls

Unfinalized Records Lifetime

Control how long unreviewed (unfinalized) records are retained before they are automatically deleted. This is a time-to-live (TTL) value entered in hours:

• Default: 2,160 hours (90 days)
• Minimum: 1 hour
• Maximum: 8,640 hours (1 year)

Records that are not reviewed within this window are automatically removed, which helps manage storage and meet compliance requirements.

Default changed

Earlier versions offered an Unlimited option that kept unreviewed records forever. This option has been removed — every queue now has a TTL, defaulting to 90 days. On On-Premise deployments, the default can be changed with the HITL_DEFAULT_TTL_HOURS environment variable.

In the review and Download & Sync screens, the remaining lifetime is shown as a countdown in days (for example, 89d 8h 55m 7s), with the exact remaining time available in the tooltip.

Note: Click Save Settings to apply changes.

---

Troubleshooting

Confidence Rules Never Send Documents to Review

If a confidence rule (for example, confidence <= 80) never matches and every document skips review:

1. Open the workflow and click Configure Settings on the exported Prompt Studio project.
2. Make sure Enable highlight is checked. Check Enable word confidence as well for word-level confidence.
3. Make sure the LLM profile used by the exported Prompt Studio project uses LLMWhisperer as its Text Extraction service. Highlight data is collected only for LLMWhisperer, so Enable highlight has no effect with other text extractors.
4. Click Save and process the documents again. For ETL and Task pipelines, files that were already processed successfully are skipped on later runs, so first delete their File History records to have them reprocessed with the new settings.

Without these settings, no confidence data is produced and HITL treats every field as 100% confident. Enabling Highlighting in Prompt Studio settings is not enough, because those settings apply only within Prompt Studio.

Switching from percentage-based review to confidence rules

If you move from sending all files to review (for example, 100% in Percentage of Files Sent to Manual Review) to confidence-based rules, test the workflow with a small batch first and confirm that low-confidence documents actually reach the review queue. A missing tool setting can silently result in no documents being reviewed.