Multi-Hop Retrieval

A single search pass answers questions where the query already names the answer. "What did we decide about JWT?" lexically matches the decision item. But many real questions require chaining: the answer to the first hop tells you what to ask next. "Why did we change auth?" needs to find what auth was changed to before it can find the rationale.

Multi-hop recursive retrieval handles these cases. SmartMemory ships two hop planners, a fast heuristic planner and an LLM-driven semantic planner. Both are opt-in on the standard search() call.

Enabling multi-hop

from smartmemory.models.memory_item import MemoryItem
from smartmemory.pipeline.config import PipelineConfig
from smartmemory.tools.factory import lite_context

lite = dict(pipeline_profile=PipelineConfig.lite(llm_enabled=False))

with lite_context(**lite) as memory:
    memory.add(
        MemoryItem(
            content="The auth migration moved sessions to JWT tokens.",
            memory_type="semantic",
            metadata={
                "entities": [
                    {"text": "auth migration", "label": "PROJECT"},
                    {"text": "JWT tokens", "label": "TECH"},
                ],
            },
        )
    )
    memory.add(
        MemoryItem(
            content="JWT tokens required rotating the cache invalidation key.",
            memory_type="semantic",
            metadata={
                "entities": [
                    {"text": "JWT tokens", "label": "TECH"},
                    {"text": "cache invalidation", "label": "SYSTEM"},
                ],
            },
        )
    )

    results = memory.search(
        "How did the auth migration affect cache invalidation?",
        top_k=5,
        multi_hop=True,
        max_hops=2,
        budget_ms=1500,
    )

    for result in results:
        hop = result.metadata.get("hop_n", 0)
        print(f"hop={hop} {result.content}")

The default planner is the heuristic one. After each hop, it extracts the top-scoring entities from result metadata and uses the highest-novelty entity names to construct follow-up queries. Follow-ups are searched in parallel and merged with cross-hop weighted RRF. Later hops are weighted by the hop-decay factor, which defaults to 0.7 per hop.

The chain stops when any of these hold:

• max_hops is reached.
• The remaining time budget drops below MIN_HOP_BUDGET_MS (100 ms in smartmemory/search/multi_hop.py).
• The hop planner finds no novel entity above MIN_ENTITY_SCORE (0.3) to follow.

multi_hop=True is incompatible with memory_type="pending" and raises ValueError. It is compatible with decompose_query=True and with origin= or exclude_origins= filters.

Semantic hop planning

The heuristic planner is fast but blind to user intent. The semantic planner asks an LLM to reason about which entity to follow given the original query and the current results.

from smartmemory.search.multi_hop import SemanticHopConfig

config = SemanticHopConfig(
    enabled=True,
    budget_ms=500,
    max_output_tokens=200,
)

print(config.enabled, config.budget_ms, config.max_output_tokens)

Use semantic_hops=True on an individual search() call when an LLM key is configured. If the LLM call times out or fails, SmartMemory falls back to the heuristic planner for that hop and logs a warning.

Combining with query decomposition

multi_hop and decompose_query solve different problems and compose:

• decompose_query=True splits a multi-topic query into independent sub-queries, runs each, and RRF-merges the results.
• multi_hop=True chains follow-up queries based on what each hop returns.

Combine them when the query has multiple topics that each need follow-up:

from smartmemory.models.memory_item import MemoryItem
from smartmemory.pipeline.config import PipelineConfig
from smartmemory.tools.factory import lite_context

lite = dict(pipeline_profile=PipelineConfig.lite(llm_enabled=False))

with lite_context(**lite) as memory:
    memory.add(
        MemoryItem(
            content="The auth migration moved sessions to JWT tokens.",
            memory_type="semantic",
        )
    )
    memory.add(
        MemoryItem(
            content="JWT tokens required rotating the cache invalidation key.",
            memory_type="semantic",
        )
    )

    results = memory.search(
        "How did the auth migration affect cache invalidation?",
        top_k=10,
        decompose_query=True,
        multi_hop=True,
        max_hops=2,
        budget_ms=3000,
    )

    print([result.content for result in results])

When both are enabled, decomposition runs first. Each decomposed sub-query then becomes the seed of its own multi-hop chain. Budget is shared across all chains.

Live progress

Multi-hop search emits live progress events on the progress event bus. Each hop fires a search.hop event with payload.hop_n, payload.queries, and the per-hop result count. The first event in a multi-hop run also emits a pipeline.dag event describing the planned chain.

Subscribe from a UI:

import { subscribeProgress } from '@smartmemory/sdk-js/progress';

const sub = subscribeProgress({
  runId,
  token,
  onEvent: (e) => {
    if (e.kind === 'search.hop') {
      console.log(`Hop ${e.payload.hop_n}:`, e.payload.queries);
    }
  },
});

Or replay a completed run from the server side using ?run_id=<id>&from_seq=0.

Tuning

The hop-decay value and minimum hop budget are constants in smartmemory/search/multi_hop.py.

Related

• Hybrid Retrieval, single-hop retrieval mechanics
• Memory Dynamics, how activation interacts with multi-hop ranking
• Progress Events, search.hop event shape