Similarity Framework

SmartMemory's similarity framework provides sophisticated methods for measuring relationships between memories, enabling intelligent retrieval, linking, and organization.

Overview

The similarity framework combines multiple similarity metrics to create a comprehensive understanding of memory relationships, supporting both semantic and structural similarity analysis.

Similarity Metrics

Semantic Similarity

Vector-Based Similarity:

Cosine Similarity - Angular distance between embeddings
Euclidean Distance - Geometric distance in vector space
Dot Product - Direct vector multiplication
Manhattan Distance - Sum of absolute differences

Implementation:

The similarity framework is EnhancedSimilarityFramework in smartmemory.similarity.framework. Instantiate it directly with an optional SimilarityConfig, then call calculate_similarity with two MemoryItem objects. Pass metrics=["semantic"] to score on the semantic metric alone:

from smartmemory.similarity.framework import EnhancedSimilarityFramework, SimilarityConfig

framework = EnhancedSimilarityFramework(
    config=SimilarityConfig(
        embedding_model="sentence-transformers/all-MiniLM-L6-v2",
        similarity_threshold=0.7,
    )
)

# Semantic-only score between two MemoryItem objects
similarity = framework.calculate_similarity(item1, item2, metrics=["semantic"])

EnhancedSimilarityFramework is not currently exposed as a method on the SmartMemory facade — use the class directly. The available metric names are content, semantic, temporal, graph, metadata, and agent_workflow.

Temporal Similarity

Time-Based Relationships:

Temporal Proximity - Closeness in time
Sequence Similarity - Order of events
Duration Overlap - Time period intersection
Frequency Patterns - Recurring temporal patterns

Features:

Adaptive time windows
Context-aware weighting
Seasonal pattern recognition
Event sequence analysis

Structural Similarity

Structural similarity is implemented by the graph metric (GraphSimilarityMetric), weighted via SimilarityConfig.graph_weight.

Graph-Based Metrics:

Neighborhood Similarity - Shared connections
Path Similarity - Common relationship paths
Centrality Similarity - Similar importance in network
Clustering Coefficient - Local connectivity patterns

Content Similarity

Text-Based Analysis:

N-gram Overlap - Shared word sequences
TF-IDF Similarity - Term frequency analysis
Jaccard Similarity - Set intersection over union
Edit Distance - Character-level differences

Multi-Dimensional Similarity

Weighted Combination

calculate_similarity always returns a weighted combination of every enabled metric. The weights live on SimilarityConfig (content_weight, semantic_weight, temporal_weight, graph_weight, metadata_weight, agent_workflow_weight) and are normalized to sum to 1.0. Configure them at construction time:

from smartmemory.similarity.framework import EnhancedSimilarityFramework, SimilarityConfig

framework = EnhancedSimilarityFramework(
    config=SimilarityConfig(
        semantic_weight=0.4,
        temporal_weight=0.3,
        graph_weight=0.2,
        content_weight=0.1,
        metadata_weight=0.0,
        agent_workflow_weight=0.0,
    )
)

# Weighted overall score (float)
combined_similarity = framework.calculate_similarity(item1, item2)

# Pass return_detailed=True for a SimilarityResult with per-metric breakdown
result = framework.calculate_similarity(item1, item2, return_detailed=True)
result.overall_score      # weighted score
result.semantic_score     # individual metric scores
result.confidence         # agreement-based confidence

Adaptive Weighting

Context-Dependent Weights:

Query Type - Adjust weights based on search intent
Memory Type - Different weights for different memory types
User Preferences - Learn from user interactions
Domain Specific - Optimize for content domain

Dynamic Thresholds

Adaptive Thresholds:

Statistical Analysis - Data-driven threshold selection
Performance Optimization - Optimize for retrieval quality
User Feedback - Adjust based on relevance ratings
Context Sensitivity - Different thresholds for different contexts

Similarity Applications

Memory Linking

Automatic Relationship Discovery:

find_similar_items ranks a candidate list against a target item and returns (MemoryItem, score) tuples above the threshold, sorted by similarity:

from smartmemory.similarity.framework import EnhancedSimilarityFramework

framework = EnhancedSimilarityFramework()

# Find similar memories from a candidate set
similar_memories = framework.find_similar_items(
    target_item=target_memory,
    candidate_items=candidates,
    threshold=0.8,
    max_results=10,
)

for memory_item, score in similar_memories:
    print(memory_item.item_id, score)

The framework computes and ranks similarity; it does not create graph edges. To materialize relationships, persist links through SmartMemory's graph/linking surface using the item_id and score returned above.

Clustering and Organization

cluster_items performs threshold-based agglomerative clustering over a list of items, returning a list of clusters (each a list of MemoryItem):

from smartmemory.similarity.framework import EnhancedSimilarityFramework

framework = EnhancedSimilarityFramework()
clusters = framework.cluster_items(items, similarity_threshold=0.8)

A module-level cluster_similar_items(items, similarity_threshold=0.8, config=None, graph_store=None) convenience function wraps the same call.

Conceptual clustering strategies:

Hierarchical Clustering - Tree-like organization
K-Means Clustering - Fixed number of clusters
DBSCAN - Density-based clustering
Community Detection - Graph-based communities

Search and Retrieval

SmartMemory.search runs the canonical multi-channel retrieval path. It accepts top_k and channel_weights (a per-channel weight dict); it does not take search_type/similarity_threshold/weights keyword arguments:

# Standard retrieval (top-k ranked results)
results = memory.search("machine learning concepts", top_k=5)

# Re-weight retrieval channels
results = memory.search(
    "recent project meetings",
    top_k=5,
    channel_weights={"vector": 0.7, "graph": 0.3},
)

To score arbitrary item pairs by combined semantic + temporal + graph similarity, use EnhancedSimilarityFramework.calculate_similarity (shown above) rather than search.

Advanced Features

Similarity Learning

Machine Learning Enhancement:

Metric Learning - Learn optimal similarity functions
Deep Similarity Networks - Neural network-based similarity
Transfer Learning - Adapt pre-trained models
Active Learning - Improve with user feedback

Multi-Modal Comparison:

Text-Image Similarity - Compare textual and visual content
Audio-Text Similarity - Speech and text comparison
Temporal-Spatial Similarity - Time and location relationships
Structured-Unstructured - Database and text comparison

Similarity Caching

Performance Optimization:

The framework caches pairwise similarity results in-memory (FIFO eviction). Caching is controlled by SimilarityConfig and inspected/reset via get_cache_stats() and clear_cache():

from smartmemory.similarity.framework import EnhancedSimilarityFramework, SimilarityConfig

framework = EnhancedSimilarityFramework(
    config=SimilarityConfig(enable_caching=True, max_cache_size=1000)
)

framework.get_cache_stats()   # {'caching_enabled': True, 'cache_size': ..., ...}
framework.clear_cache()

To warm the cache, call calculate_similarity (or get_similarity_matrix(items), which computes and caches all pairwise scores). There is no separate precompute/TTL API; eviction is size-bounded via max_cache_size.

Configuration and Tuning

Similarity Profiles

Profiles are expressed as a SimilarityConfig dataclass: per-metric weights plus the similarity_threshold (minimum to count as similar) and high_similarity_threshold (used for clustering). Construct a framework with the config, or swap it on a live framework via update_config (which reinitializes metrics and clears the cache):

from smartmemory.similarity.framework import EnhancedSimilarityFramework, SimilarityConfig

research_profile = SimilarityConfig(
    semantic_weight=0.5,
    temporal_weight=0.2,
    graph_weight=0.2,
    content_weight=0.1,
    metadata_weight=0.0,
    agent_workflow_weight=0.0,
    similarity_threshold=0.6,
    high_similarity_threshold=0.8,
)

framework = EnhancedSimilarityFramework(config=research_profile)

# Or apply a new profile to an existing framework
framework.update_config(research_profile)

Weights that do not sum to 1.0 are automatically normalized (with a warning) by SimilarityConfig.__post_init__.

Performance Tuning

Optimization Strategies:

Approximate Similarity - Trade accuracy for speed
Hierarchical Search - Coarse-to-fine similarity
Parallel Processing - Multi-threaded computation
GPU Acceleration - Hardware-accelerated similarity

Evaluation Metrics

Similarity Quality Assessment:

Precision@K - Relevant results in top K
Recall - Coverage of relevant memories
F1-Score - Harmonic mean of precision and recall
Mean Average Precision - Average precision across queries

The similarity framework provides the foundation for intelligent memory organization and retrieval, enabling SmartMemory to understand complex relationships and provide relevant, contextual responses to user queries.

Similarity Framework

On this page