Evolution Algorithms

SmartMemory's evolution system continuously adjusts memory retention, promotes important memories, and strengthens relationships between frequently co-activated memories. Evolution plugins run as the final stage (evolve) of the ingestion pipeline, and on-demand via the Evolution API.

Overview

Evolution operates at two levels:

1. Node-level evolution — individual memories are promoted, decayed, or had their retention_score adjusted based on age, retrieval frequency, and inter-memory interference.
2. Edge-level evolution — relationships between memories grow stronger when those memories are consistently retrieved together (Hebbian reinforcement).

All evolvers are implemented as EvolverPlugin subclasses in smartmemory/plugins/evolvers/. The enhanced evolvers (CORE-EVO-ENH series) operate on the workspace graph and write results back via SmartMemory.update_properties().

Core Evolvers

These evolvers handle memory lifecycle transitions — promotion between types, synthesis, and reinforcement of opinion/decision memories.

Enhanced Evolvers (CORE-EVO-ENH Series)

The enhanced evolver suite applies cognitive-science-inspired algorithms to improve retention quality over time. These evolvers run in sequence as part of run_enhanced_evolution_cycle().

ExponentialDecayEvolver (CORE-EVO-ENH-1)

Implements the Ebbinghaus forgetting curve for episodic memories:

retention = exp(−elapsed_days / stability)

stability is a per-memory value stored in metadata["stability"] (default: 30 days, reflecting how well-consolidated a memory is). Items that fall below archive_threshold (default: 0.1) are soft-archived automatically.

Config: default_stability=30.0, archive_threshold=0.1, max_memories=500

InterferenceBasedConsolidationEvolver (CORE-EVO-ENH-1)

Models retroactive and proactive interference: when two memories are highly similar, they compete for consolidation. For each embedded memory, searches up to top_k_neighbors similar items and applies a cosine-similarity-based retention penalty to pairs above similarity_threshold:

new_score = current_score × (1 − similarity × interference_weight)

Pair deduplication ensures each (A, B) pair is penalized exactly once. Items without embeddings are skipped.

Config: similarity_threshold=0.85, interference_weight=0.05, top_k_neighbors=5, max_memories=200

RetrievalBasedStrengtheningEvolver (CORE-EVO-ENH-2)

Implements the testing effect: memories that are frequently retrieved retain better; unretrieved memories decay faster. Reads metadata["retrieval__profile"] (written by RetrievalFlushConsumer every 5 minutes) and applies:

rank_weight    = 1 / (1 + avg_search_rank)
recency_weight = velocity_7d / (velocity_30d + ε)
boost          = retrieval_weight × log(1 + total_count) × rank_weight × recency_weight
retention      = min(1.0, base_retention + boost)

For memories with zero retrievals in the lookback window, a gentle unretrieved decay is applied instead: retention *= (1 − rate × elapsed_days).

Config: retrieval_weight=0.1, unretrieved_decay_rate=0.005, lookback_days=30, max_memories=500

Retrieval profile (written to metadata["retrieval__profile"] by the worker):

{
  "total_count": 47,
  "search_count": 44,
  "avg_search_rank": 2.3,
  "velocity_7d": 3.1,
  "velocity_30d": 0.8,
  "top1_count": 12,
  "sources": {"api.POST /memory/search": 28}
}

HebbianCoRetrievalEvolver (CORE-EVO-ENH-3)

Implements Hebbian learning at the graph edge level: memories that consistently appear together in the same search result set accumulate stronger CO_RETRIEVED edges in FalkorDB over time. The evolver then boosts retention_score on nodes connected by the strongest edges.

"Neurons that fire together, wire together" — but applied to memory retrieval rather than neurons.

How it works:

1. RetrievalFlushConsumer extracts 2-combinations of item_ids from every search result batch. Each co-occurring pair gets its CO_RETRIEVED edge incremented: co_retrieval_count += batch_count.
2. HebbianCoRetrievalEvolver queries edges with co_retrieval_count >= weight_threshold, deduplicates by item_id, and applies:

boost = min(max_boost, (co_retrieval_count − weight_threshold) × retention_boost_per_unit)
new_retention = min(1.0, existing_retention + boost)

Config: weight_threshold=3.0, max_edges_per_cycle=500, retention_boost_per_unit=0.02, max_boost=0.3

Example: A pair co-retrieved 10 times with defaults gets boost = min(0.3, (10−3) × 0.02) = 0.14.

CO_RETRIEVED edge properties:

Metadata Conventions

Enhanced evolvers read and write standardized metadata fields on MemoryItem:

Evolution Cycle

To run all enhanced evolvers for a workspace:

from smartmemory.plugins.evolvers.enhanced import run_enhanced_evolution_cycle

# memory is a workspace-scoped SmartMemory instance
run_enhanced_evolution_cycle(memory)

The cycle runs evolvers in registration order:

1. EnhancedWorkingToEpisodicEvolver
2. ExponentialDecayEvolver
3. InterferenceBasedConsolidationEvolver
4. RetrievalBasedStrengtheningEvolver
5. HebbianCoRetrievalEvolver

Each evolver catches its own exceptions — a single evolver failure does not abort the cycle.

The Evolution API also exposes manual triggers:

Memory Lifecycle

Pending Memory
  │
  ├──> Episodic Memory (WorkingToEpisodic: buffer full)
  │         │
  │         ├──> Semantic Memory (EpisodicToSemantic: high confidence + age)
  │         │
  │         ├──> Zettel Notes (EpisodicToZettel: periodic rollup)
  │         │
  │         ├──> Opinions (OpinionSynthesis: pattern detection)
  │         │
  │         └──> Observations (ObservationSynthesis: entity summaries)
  │
  └──> Procedural Memory (WorkingToProcedural: repeated patterns)

Decay:  ExponentialDecay · InterferenceBased → retention_score ↓
Boost:  RetrievalStrengthening · HebbianCoRetrieval → retention_score ↑
Edges:  CO_RETRIEVED (HebbianCoRetrieval) → co_retrieval_count ↑