Multi-Hop Retrieval

A single search pass answers questions where the query already names the answer. "What did we decide about JWT?" lexically matches the decision item; one shot, done. But many real questions require chaining: the answer to the first hop tells you what to ask next. "Why did we change auth?" needs to find what auth was changed to before it can find the rationale; "Which support groups did Ruth join after her diagnosis?" needs to identify Ruth, then her diagnosis, then groups associated with that condition.

Multi-hop recursive retrieval handles these. SmartMemory ships two hop planners — a fast heuristic and an LLM-driven semantic one — both opt-in on the standard search() call.

Enabling multi-hop

results = memory.search(
    "Why did we change auth?",
    top_k=5,
    multi_hop=True,
    max_hops=3,         # default 3, range 1-10
    budget_ms=1500,     # default 1500, must be >= 100
)

The default planner is the heuristic one: after each hop, it extracts the top-scoring entities from the result metadata and uses the highest-novelty entity names to construct up to three follow-up queries for the next hop. All follow-ups are searched in parallel and merged with cross-hop weighted RRF, with each later hop weighted by the hop-decay factor (0.7 per hop by default — Hop 2 contributes 0.7×, Hop 3 contributes 0.49×, etc.).

The chain stops when any of these hold:

• max_hops is reached.
• The remaining time budget drops below MIN_HOP_BUDGET_MS (100 ms — see smartmemory/search/multi_hop.py).
• The hop planner finds no novel entity above MIN_ENTITY_SCORE (0.3) to follow up on.

multi_hop=True is incompatible with memory_type="pending" (the pending tier is a per-conversation working memory, not a graph) and raises ValueError. It is compatible with decompose_query=True and with origin= / exclude_origins= filters.

Semantic hop planning (CORE-MULTIHOP-2)

The heuristic planner is fast but blind to user intent: it picks the top entity by score, even if a different entity would lead to a better next hop. The semantic planner asks an LLM to reason about which entity to follow given the original query and the current results.

results = memory.search(
    "What support groups did Ruth join after her diagnosis?",
    top_k=5,
    multi_hop=True,
    semantic_hops=True,
    max_hops=3,
)

semantic_hops=True swaps the planner per-call. To enable globally:

from smartmemory.search.multi_hop import SemanticHopConfig

memory = SmartMemory(
    semantic_hop_config=SemanticHopConfig(
        enabled=True,
        budget_ms=500,            # per-hop LLM call budget
        max_output_tokens=200,    # response cap
    ),
)

The semantic planner uses Haiku by default with a 500 ms per-hop budget; if the LLM call times out or fails it falls back silently to the heuristic planner for that hop. Failures emit a WARNING log; the chain doesn't break. Each result is truncated to 200 chars when shown to the planner so the prompt stays compact.

Reading multi-hop results

The returned list is the merged, ranked, top-K result from all hops. The hop number is recorded on each result's metadata under hop_n so you can introspect which leg each result came from:

results = memory.search("...", multi_hop=True, max_hops=3)
for r in results:
    hop = r.metadata.get("hop_n", 0)
    print(f"hop={hop}  score={r.score:.3f}  {r.content[:80]}")

Cross-hop merging uses Reciprocal Rank Fusion with the hop-decay weighting:

final_score(item) = Σ over hops h:  0.7^h  /  (k + rank_in_hop(item))

This means an item that appears at a lower rank across multiple hops can outrank an item that appears at a high rank in only one hop — which is the point: the more hops corroborate a result, the more confident the merger is.

Live progress

Multi-hop search emits live progress events on the progress event bus. Each hop fires a search.hop event with payload.hop_n, payload.queries, and the per-hop result count; the first event in a multi-hop run also emits a pipeline.dag event describing the planned chain.

Subscribe from a UI:

import { subscribeProgress } from '@smartmemory/sdk-js/progress';

const sub = subscribeProgress({
  runId,
  token,
  onEvent: (e) => {
    if (e.kind === 'search.hop') {
      console.log(`Hop ${e.payload.hop_n}:`, e.payload.queries);
    }
  },
});

Or replay a completed run from the server side using ?run_id=<id>&from_seq=0.

Combining with query decomposition

multi_hop and decompose_query solve different problems and compose:

• decompose_query=True splits a multi-topic query ("auth flow and caching strategy") into independent sub-queries, runs each, and RRF-merges the results.
• multi_hop=True chains follow-up queries based on what each hop returns.

Combine them when the query has both shapes — multiple topics that each need follow-up:

results = memory.search(
    "How did the auth migration affect cache invalidation?",
    top_k=10,
    decompose_query=True,
    multi_hop=True,
    semantic_hops=True,
    max_hops=2,
    budget_ms=3000,   # bump budget when combining — both add latency
)

When both are enabled, decomposition runs first; each decomposed sub-query then becomes the seed of its own multi-hop chain. Budget is shared across all chains.

Tuning

The hop-decay (0.7) and minimum hop budget (100 ms) are not exposed as search() parameters — change HOP_DECAY and MIN_HOP_BUDGET_MS constants in smartmemory/search/multi_hop.py if you need to tune them, but the defaults are calibrated for typical workloads.

Related

• Hybrid Retrieval — single-hop retrieval mechanics
• Memory Dynamics — how activation interacts with multi-hop ranking
• Progress Events — search.hop event shape