Retrieval Strategies
A retrieval strategy defines how Unstract searches through document chunks to find the most relevant context for answering a prompt. The choice of strategy directly affects extraction accuracy, cost, and latency.
Retrieval strategies only apply when chunking is enabled (chunk size > 0). If chunking is disabled, the entire document is sent to the LLM and no retrieval is needed. See Chunk size and overlap for guidance on when to enable chunking.
Quick Comparison
| Strategy | Requires LLM | Token Usage | Cost | Best For |
|---|---|---|---|---|
| Simple | No | Low | $ | Straightforward field extraction |
| Fusion | Yes | High | $$$ | Ambiguous queries, varied terminology |
| Sub-Question | Yes | High | $$$ | Multi-part questions spanning sections |
| Recursive | No | Low | $ | Nested, hierarchical documents |
| Router | Yes | Medium | $$ | Mixed prompt types in one project |
| Keyword Table | Yes | Medium | $$ | Exact terms, codes, identifiers |
| Auto-Merging | No | Low | $ | Long narrative content across chunks |
Strategies in Detail
Simple Vector Retrieval
The Simple strategy performs a direct vector similarity search against the indexed chunks. It computes cosine similarity between the query embedding and all chunk embeddings, then returns the top-k chunks with the highest similarity scores. Chunks that fall below a minimum relevance threshold are automatically filtered out. This is the fastest and most cost-effective strategy, making it the recommended starting point for most use cases.
| How it works | Embeds the query, compares against all chunk vectors, returns top-k matches above a relevance threshold |
| Requires LLM | No |
| Speed | Fastest |
| Cost | Lowest |
Example Use Cases
| Document Type | Prompt | Why Simple Works |
|---|---|---|
| Invoice | "What is the invoice number?" | The answer is a single, clearly stated field |
| Insurance claim form | "Extract the policy holder name" | Information appears in a predictable location |
| Receipt | "What is the total amount?" | Direct factual extraction, no ambiguity |
Fusion Retrieval (RAG Fusion)
The Fusion strategy generates multiple variations of the original query using an LLM, then runs parallel retrievals across different retrievers with varying top-k values. The results are combined using Reciprocal Rank Fusion (RRF) scoring to produce a final ranked list of chunks. By generating alternative phrasings, Fusion casts a wider net and surfaces relevant chunks that a single query might miss — for example, a prompt asking for "compensation details" will also search for variations like "salary information", "pay structure", and "remuneration terms".
| How it works | LLM rewrites the query in multiple ways → each variation retrieves chunks independently → results are scored and merged via RRF |
| Requires LLM | Yes |
| Speed | Slower (multiple retrieval passes) |
| Cost | Higher (LLM calls + multiple retrievals) |
Example Use Cases
| Document Type | Prompt | Why Fusion Works |
|---|---|---|
| Legal contract | "What is the termination clause?" | Contracts may say "cancellation provisions", "exit terms", or "contract dissolution" |
| Supplier agreement | "What is the delivery timeline?" | Different vendors use "lead time", "shipping schedule", or "fulfillment period" |
| HR policy document | "What are the compensation details?" | Fusion also searches for "salary", "pay structure", "remuneration" |
If Simple retrieval returns irrelevant chunks because your query terms don't match the document's wording, try Fusion. It's especially effective when documents come from multiple sources with inconsistent language.
Sub-Question Retrieval
The Sub-Question strategy uses an LLM to decompose a complex query into simpler sub-questions. It retrieves relevant chunks for each sub-question independently, then synthesizes the results to address the original query. The LLM generates up to 10 sub-questions depending on the complexity of the original prompt. This is particularly useful when dealing with multifaceted questions that require pulling together information from different sections or contexts within a document.
| How it works | LLM breaks the prompt into up to 10 sub-questions → retrieves relevant chunks for each → combines results to answer the original query |
| Requires LLM | Yes |
| Speed | Slower (multiple retrieval rounds) |
| Cost | Higher (sub-question generation + multiple retrievals) |
Example Use Cases
| Document Type | Prompt | Sub-Questions Generated |
|---|---|---|
| Legal agreement | "What is the total liability exposure?" | 1. What are the liability caps? 2. What are the indemnification terms? 3. What are the warranty limitations? |
| Financial report | "Compare Q1 and Q2 revenue and identify drivers" | 1. What was Q1 revenue? 2. What was Q2 revenue? 3. What factors drove changes? |
| Vendor contract | "Summarize payment terms including due dates, penalties, and methods" | 1. What are the payment due dates? 2. What are the late penalties? 3. What payment methods are accepted? |
Recursive Retrieval
The Recursive strategy traverses document relationships recursively, following connections between chunks to build a more complete context. It explores parent-child and sibling relationships in the document structure, starting from initially matched chunks and expanding outward to capture related content. This is effective for documents where related information is spread across different sections that reference each other.
| How it works | Finds initial matching chunks → follows document structure relationships (parent, child, sibling) → recursively gathers connected chunks |
| Requires LLM | No |
| Speed | Fast |
| Cost | Low |
Example Use Cases
| Document Type | Prompt | Why Recursive Works |
|---|---|---|
| Regulatory filing | "What are the requirements in Section 3.2.1(b)?" | The clause references definitions in Section 1.4 and exceptions in Appendix C |
| Technical manual | "What are the configuration limits?" | Parameter is in a subsection but inherits constraints from its parent section |
| Master services agreement | "What are the liability limitations?" | A clause says "subject to Section 7" — Recursive retrieves both |
Router-based Retrieval
The Router strategy uses an LLM to analyze the incoming query and route it to the most appropriate retrieval method — vector search, keyword search, or broad search — based on the query characteristics. This means each prompt automatically gets the retrieval approach best suited to its nature, rather than forcing all prompts through a single retrieval method.
| How it works | LLM classifies the query type → routes to the most appropriate retriever (vector, keyword, or broad) → returns results from the selected method |
| Requires LLM | Yes |
| Speed | Medium |
| Cost | Medium |
Example Use Cases
| Prompt Type | Prompt Example | Route Chosen |
|---|---|---|
| Identifier lookup | "What is the ICD-10 code?" | Keyword search |
| Descriptive extraction | "Summarize the project scope" | Vector search |
| Broad extraction | "List all parties mentioned" | Broad search |
Router is ideal when your project has a mix of prompt types — some looking for exact codes/identifiers and others extracting narrative summaries. Instead of compromising with one retrieval approach, Router picks the best one per prompt.
Keyword Table Retrieval
The Keyword Table strategy extracts and indexes keywords from document chunks using an LLM, then uses TF-IDF scoring with exact and fuzzy matching to find relevant chunks based on keyword overlap with the query. This approach excels when the exact terminology matters more than semantic meaning — for instance, distinguishing between specific codes, part numbers, or domain-specific terms that vector search might conflate.
| How it works | Retrieves the document chunks → uses an LLM to extract keywords and build a keyword-to-chunk index → matches the query against that index |
| Requires LLM | Yes (the keyword index is built during each retrieval) |
| Speed | Slower (rebuilds the keyword index for each query) |
| Cost | Medium (LLM used at query time) |
Example Use Cases
| Document Type | Prompt | Why Keyword Table Works |
|---|---|---|
| Medical billing | "What is the description for CPT code 99213?" | Exact code matching — vector search might return similar but wrong codes |
| Engineering spec | "What are the specs for Inconel 718?" | Precise material name — vector search could confuse it with similar alloys |
| Safety data sheet | "Find references to ASTM D4236" | Standard identifier requires exact match |
Auto-Merging Retrieval
The Auto-Merging strategy merges adjacent and related chunks to reconstruct larger coherent passages. It uses a leaf-to-parent node merging approach — when multiple child chunks from the same parent section are retrieved, they are automatically combined into the full parent chunk. This reduces fragmentation and preserves narrative flow, making it ideal for long-form documents where splitting content across chunks causes the LLM to miss critical context.
| How it works | Retrieves matching leaf chunks → checks if sibling chunks from the same parent were also retrieved → merges them into the parent chunk for fuller context |
| Requires LLM | No |
| Speed | Fast |
| Cost | Low |
Example Use Cases
| Document Type | Prompt | Why Auto-Merging Works |
|---|---|---|
| Consulting proposal | "What is the project scope?" | Scope spans several paragraphs — splitting it across chunks loses context |
| Clinical trial report | "Describe the methodology" | Methodology naturally spans 2-3 chunks that need to be read together |
| Contract | "What are the indemnification terms?" | Clause starts in one chunk, key conditions continue in the next |
If Simple retrieval returns partial or fragmented answers — where the LLM seems to be missing part of the context — try Auto-Merging. It reconstructs the full passage without requiring an LLM, keeping costs low.
Choosing the Right Strategy
| Your Situation | Try This | Example |
|---|---|---|
| Extracting single fields from structured documents | Simple | Invoice number, date, vendor name |
| Queries return wrong chunks due to terminology mismatch | Fusion | "Termination clause" vs. "cancellation provisions" |
| One prompt needs info from multiple document sections | Sub-Question | Total liability = caps + indemnification + warranties |
| Documents have deep nesting and cross-references | Recursive | Section 3.2 references Section 1.4 and Appendix C |
| Project has both code lookups and narrative extractions | Router | ICD codes + procedure summaries in same workflow |
| Need exact match on codes, part numbers, or standards | Keyword Table | CPT 99213, ASTM D4236, Inconel 718 |
| Answers come back incomplete due to chunk boundaries | Auto-Merging | Scope description split across 3 chunks |
Begin with Simple. It has the lowest cost and latency, and works well for most use cases. Switch to a more advanced strategy only when you observe issues like incomplete answers, missed context, or terminology mismatches.
Strategies that require an LLM (Fusion, Sub-Question, Router, Keyword Table) consume additional tokens during retrieval. For high-throughput pipelines processing thousands of documents, the cumulative cost difference can be significant.
Retrieval strategy works together with chunk size and overlap. A smaller chunk size paired with Auto-Merging gives precise initial matching with reconstructed context. A larger overlap helps Simple retrieval capture information at chunk boundaries. See Chunk size and overlap for tuning guidance.
Configuring Retrieval Strategies
Retrieval strategies are configured as part of an LLM Profile in Prompt Studio:
- Open Settings > LLM Profiles.
- Create or edit an LLM profile.
- Set a Chunk Size greater than 0 (retrieval strategies only apply when chunking is enabled).
- Under Advanced Settings, select your preferred Retrieval Strategy.
- Adjust Similarity top-k to control how many chunks are retrieved and sent to the LLM.
For detailed instructions on configuring LLM profiles, see Setting up LLM profiles.