System Architecture Overview

SmartMemory is built on a modular, component-based architecture that provides a unified interface for agentic memory operations while maintaining flexibility and extensibility. This document provides a comprehensive overview of the system architecture, design principles, and component interactions.

Architecture Principles

Core Design Principles

Modularity: Each component has a single responsibility and clear interfaces
Extensibility: New memory types, backends, and algorithms can be easily added
Performance: Optimized for both throughput and latency with background processing
Reliability: Robust error handling and graceful degradation
Scalability: Designed to handle large-scale memory operations
Interoperability: Standard interfaces for integration with external systems

Architectural Patterns

Component-Based Architecture: Loosely coupled components with dependency injection
Factory Pattern: Consistent creation and initialization of memory components
Strategy Pattern: Pluggable algorithms for similarity, evolution, and extraction
Observer Pattern: Event-driven processing and monitoring
Adapter Pattern: Integration with various backends and external services

High-Level Architecture

graph TB
    subgraph "🌟 SmartMemory System Architecture"
        subgraph "API Layer"
            API["🚀 SmartMemory API"]
            CRUD["📝 CRUD Operations"]
            SEARCH["🔍 Search & Retrieval"]
            REL["🔗 Relationship Management"]
            BG["🌐 Background Processing"]
            
            API --> CRUD
            API --> SEARCH
            API --> REL
            API --> BG
        end
        
        subgraph "Core Components"
            subgraph "Memory Types"
                SEM["🔍 Semantic Memory"]
                EPI["📚 Episodic Memory"]
                PROC["⚙️ Procedural Memory"]
                WORK["💭 Pending Memory"]
            end
            
            subgraph "Processing Pipeline"
                ING["📥 Ingestion Flow"]
                EXT["🔍 Entity Extraction"]
                ENR["📊 Enrichment"]
                EVO["🧬 Evolution Engine"]
            end
            
            subgraph "Graph Operations"
                GCRUD["💾 Graph CRUD"]
                LINK["🔗 Linking"]
                GSEARCH["🔍 Graph Search"]
                MON["📊 Monitoring"]
            end
            
            subgraph "External Integration"
                MCP["🔌 MCP Tools"]
                LLM["🤖 LLM Providers"]
                VEC["🗂️ Vector Stores"]
            end
        end
        
        subgraph "Storage Layer"
            subgraph "Graph Backend"
                FALKOR["🔴 FalkorDB"]
                SQLITE["🟢 SQLite (lite)"]
            end
            
            subgraph "Vector Storage"
                FALKORVEC["🔴 FalkorDB HNSW (native, default)"]
                USEARCH["📦 usearch (optional)"]
            end
            
            subgraph "Configuration"
                CONFIG["⚙️ Config Manager"]
                FACTORY["🏭 Factories"]
                LOADER["📂 Config Loader"]
            end
        end
        
        subgraph "Infrastructure"
            DOCKER["🐳 Docker Services"]
            REDIS["🔴 Redis/FalkorDB"]
            OPENAI["🤖 OpenAI API"]
        end
    end
    
    %% API Layer Connections
    CRUD --> SEM
    CRUD --> EPI
    CRUD --> PROC
    CRUD --> WORK
    
    SEARCH --> GSEARCH
    REL --> LINK
    BG --> EVO
    
    %% Processing Pipeline Connections
    ING --> EXT
    EXT --> ENR
    ENR --> EVO
    
    %% Storage Connections
    SEM --> GCRUD
    EPI --> GCRUD
    PROC --> GCRUD
    WORK --> GCRUD
    
    GCRUD --> FALKOR
    GCRUD --> SQLITE
    
    ENR --> FALKORVEC
    ENR --> USEARCH
    
    %% External Integration
    MCP --> API
    LLM --> ENR
    VEC --> FALKORVEC
    
    %% Infrastructure
    FALKOR --> REDIS
    FALKORVEC --> REDIS
    LLM --> OPENAI
    
    %% Configuration
    CONFIG --> FACTORY
    
    style API fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
    style GCRUD fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
    style EVO fill:#fff3e0,stroke:#f57c00,stroke-width:2px
    style DOCKER fill:#e8f5e8,stroke:#388e3c,stroke-width:2px

Component Architecture

SmartMemory Core Class

The SmartMemory class serves as the unified entry point and orchestrator:

class SmartMemory(MemoryBase):
    """
    Unified agentic memory store combining semantic, episodic, 
    procedural, and pending memory.
    """
    
    def __init__(self, **kwargs):
        # Core storage
        self._graph = SmartGraph()
        
        # Component delegation
        self._crud = CRUD(self._graph)
        self._linking = Linking(self._graph)
        self._enrichment = Enrichment(self._graph)
        self._grounding = Grounding(self._graph)
        self._personalization = Personalization(self._graph)
        self._search = Search(self._graph)
        self._monitoring = Monitoring(self._graph)
        self._evolution = EvolutionOrchestrator(self)
        self._clustering = GlobalClustering(self._graph)
        
        # Temporal versioning
        self.version_tracker = VersionTracker(self._graph)
        
        # Processing flow (constructed lazily on first ingest)
        self._ingestion_flow = MemoryIngestionFlow(
            self, linking=self._linking, enrichment=self._enrichment
        )

Key Responsibilities:

Unified API for all memory operations
Component lifecycle management
Request routing and delegation
Background processing coordination
Configuration management

Memory Types

Semantic Memory

Purpose: Long-term factual knowledge and concepts
Storage: Graph nodes with rich semantic relationships
Processing: Entity extraction, concept linking, ontology integration
Use Cases: Knowledge bases, fact storage, concept hierarchies

Episodic Memory

Purpose: Temporal experiences and events
Storage: Time-ordered sequences with contextual metadata
Processing: Temporal clustering, event correlation, narrative construction
Use Cases: Conversation history, user interactions, experience tracking

Procedural Memory

Purpose: Skills, procedures, and how-to knowledge
Storage: Step-based workflows and process graphs
Processing: Skill extraction, procedure optimization, workflow analysis
Use Cases: Task automation, skill learning, process documentation

Pending Memory

Purpose: Active, short-term contextual information
Storage: Temporary nodes with expiration policies
Processing: Context management, relevance scoring, automatic cleanup
Use Cases: Current conversation context, active task state

Processing Pipeline

Ingestion Flow (11 Stages)

Input → Classification → Extraction → Storage → Linking → 
Vector → Enrichment → Grounding → Evolution → Clustering → Versioning

Stages:

Input Adaptation: Convert str/dict/MemoryItem to standard format
Classification: Determine memory type (semantic, episodic, procedural, zettel, pending)
Extraction: Extract entities & relations (LLM → SpaCy → GLiNER fallback chain)
Storage: Create memory node + entity nodes in FalkorDB
Linking: Connect to related existing memories
Vector Storage: Generate embeddings, store in HNSW index
Enrichment: Add Wikipedia summaries, categories, metadata
Grounding: Create GROUNDED_IN edges to Wikipedia nodes
Evolution: Promote pending → episodic/procedural if thresholds met
Clustering: SemHash + embedding deduplication of entities
Versioning: Create bi-temporal version record

Fast Ingestion Flow

Input → Quick Storage → Background Processing Queue

Benefits:

Immediate storage for fast response times
Background enrichment for full processing
Scalable for high-throughput scenarios

Graph Operations

CRUD Component

Create: Add new memory items with validation
Read: Retrieve items by ID, type, or criteria
Update: Modify existing items with change tracking
Delete: Remove items with cascade handling

Linking Component

Automatic Linking: AI-driven relationship discovery
Explicit Linking: User-defined relationships
Link Types: Semantic, temporal, causal, hierarchical
Link Strength: Weighted relationships with confidence scores

Search Component

Vector Search: Semantic similarity using embeddings
Graph Traversal: Relationship-based discovery
Hybrid Search: Combined vector and graph approaches
Faceted Search: Multi-dimensional filtering

External Integration

MCP Tools

Standardized Interface: Model Context Protocol compliance
Tool Discovery: Auto-registration and service discovery
Agent Integration: LangChain, AutoGen, custom frameworks
Operation Coverage: Full CRUD and search capabilities

LLM Providers

OpenAI: GPT models for extraction and enrichment
Anthropic: Claude for analysis and reasoning
Azure OpenAI: Enterprise-grade AI services
Local Models: Local or self-hosted LLM endpoints for extraction

Vector Stores

FalkorDB HNSW (default): Embeddings stored natively inside FalkorDB via an HNSW vector index — no separate vector service
usearch (optional): Embedded, file-backed similarity search
Extensible: Backend registry for adding new vector providers

Data Flow Architecture

Write Path

User Input → SmartMemory.add() → Ingestion Flow → 
Graph Storage + Vector Storage → Background Enrichment

Read Path

Query → SmartMemory.search() → Search Component → 
Vector Search + Graph Traversal → Result Ranking → Response

Background Processing

Queued Items → Background Processor → Enrichment Pipeline → 
Evolution Algorithms → Updated Storage

Storage Architecture

Graph Database Layer (FalkorDB)

Node Types:

Memory Nodes: Core memory items with content and metadata
Entity Nodes: Extracted entities with properties (dual-node pattern)
Wikipedia Nodes: Global grounding nodes (shared across users)
Version Nodes: Bi-temporal version records

Relationship Types:

CONTAINS: Memory contains entity
GROUNDED_IN: Entity grounded to Wikipedia
HAS_VERSION: Memory has version history
RELATES_TO: Semantic relationships
FOLLOWS: Temporal sequences
PART_OF: Hierarchical structures
SIMILAR_TO: Similarity relationships
PRODUCED: ReasoningTrace generated a Decision
DERIVED_FROM: Decision based on evidence
SUPERSEDES: New decision replaces old
CONTRADICTS: Decisions conflict
INFLUENCES: Decision affects other memories

Properties:

Timestamps: Creation, modification, access times
Scores: Relevance, confidence, importance
Metadata: User context, source information
Embeddings: Vector representations for similarity

Vector Storage Layer (FalkorDB HNSW)

Unified Backend:

FalkorDB provides both graph and vector storage
Native HNSW index with vecf32 type
Configurable parameters: M, efConstruction, efRuntime
Cosine similarity search

Embedding Types:

Content Embeddings: Full text semantic vectors (sentence-transformers)
Entity Embeddings: Specific entity representations

Indexing Strategy:

HNSW Index: Hierarchical Navigable Small World graphs
Tenant Isolation: ScopeProvider filters all queries
Incremental Updates: Real-time index maintenance

Reliability & Durability (FalkorDB)

Persistence: Production deployments enable Redis Append-Only File (AOF) on FalkorDB for ~1-second worst-case data loss, alongside the default RDB snapshot mechanism for fast offline-copy backups.

Configuration mechanism: AOF is enabled via the REDIS_ARGS environment variable on the FalkorDB container (REDIS_ARGS="--appendonly yes --appendfsync everysec"). The FalkorDB image entrypoint constructs the redis-server invocation from this env var; compose command: flags are not honored.
AOF layout: Redis 7+ multi-part format under /var/lib/falkordb/data/appendonlydir/ (manifest + base.rdb + incr.aof). Replays cleanly on container restart, including FalkorDB-module GRAPH commands.
Volume mount: falkordb_data:/var/lib/falkordb/data (env FALKORDB_DATA_PATH). The legacy /data mount used by older image versions persists nothing in current FalkorDB releases.

Crash recovery: FalkorDB's Redis-derived signal handler can dump crash state and leave the container in a "Up but non-responsive" state where every healthcheck fails but no automatic restart fires. Production deployments add a willfarrell/autoheal sidecar that watches for the autoheal: "true" container label and force-restarts unhealthy containers via the Docker socket. Verified end-to-end recovery time: ~20 seconds from docker pause to fully healthy with all data preserved (vs ~64 minutes for an unattended hung-container incident pre-feature).

Healthcheck cadence: interval: 15s, timeout: 10s, retries: 3 on the FalkorDB container. The 10-second timeout absorbs entity-resolution write bursts and AOF-rewrite fork pressure without producing false unhealthy flips.

For the full design and execution record, see docs/features/PLAT-FALKOR-1/ in the smart-memory-docs repository.

# Multi-level caching strategy
L1_CACHE = LRUCache(maxsize=1000)      # Hot data
L2_CACHE = RedisCache(ttl=3600)        # Warm data
L3_CACHE = DiskCache(size="1GB")       # Cold data

Background Processing

# Asynchronous processing pipeline
class BackgroundProcessor:
    def __init__(self, max_workers=3):
        self.enrichment_queue = Queue()
        self.evolution_queue = Queue()
        self.cleanup_queue = Queue()

Connection Pooling

# Database connection management
class ConnectionPool:
    def __init__(self, backend_type, pool_size=10):
        self.pool = create_pool(backend_type, pool_size)
        self.health_checker = HealthChecker()

Scalability Patterns

Horizontal Scaling

Sharded Storage: Distribute data across multiple backends
Load Balancing: Route requests across multiple instances
Microservices: Decompose into specialized services

Vertical Scaling

Resource Optimization: Memory and CPU tuning
Batch Processing: Efficient bulk operations
Index Optimization: Smart indexing strategies

Security Architecture

Access Control

User Isolation: Strict user data separation
Permission Model: Role-based access control
API Security: Authentication and authorization
Data Encryption: At-rest and in-transit protection

Privacy Protection

Data Anonymization: PII removal and masking
Retention Policies: Automatic data expiration
Audit Logging: Comprehensive access tracking
Compliance: GDPR, CCPA, and other regulations

Monitoring and Observability

Metrics Collection

# Key performance indicators
METRICS = {
    "throughput": "operations_per_second",
    "latency": "response_time_percentiles", 
    "accuracy": "search_relevance_scores",
    "resource_usage": "memory_cpu_disk_utilization"
}

Health Monitoring

Component Health: Individual component status
Dependency Health: External service monitoring
Performance Alerts: Threshold-based notifications
Capacity Planning: Resource usage trends

Debugging Support

Distributed Tracing: Request flow tracking
Structured Logging: Searchable log events
Error Tracking: Exception monitoring and alerting
Performance Profiling: Bottleneck identification

Configuration Architecture

Configuration Hierarchy

Environment Variables → Config Files → Runtime Parameters → Defaults

Configuration Validation

# Schema-based validation
CONFIG_SCHEMA = {
    "graph_db": {"backend_class": "FalkorDBBackend", "host": "localhost", ...}
    "vector": {"backend": "falkordb", "dimension": 768, "metric": "cosine", ...}
    "background": {"max_workers": 16, ...}
    "llm_provider": {"type": "string", "enum": ["openai", "anthropic", "azure"]},
    "background_processing": {"type": "boolean", "default": True}
}

Environment-Specific Configurations

Development: Local backends, debug logging
Testing: In-memory storage, mock services
Staging: Production-like setup with test data
Production: Optimized for performance and reliability

Where it runs

SmartMemory runs in three ways:

Lite (free, on your own machine): install with pip install smartmemory. Storage and search run locally with a SQLite graph and local vectors. No Docker, database service, or account is needed. See Installation.
SmartMemory Service (managed): connect the SDKs, CLI, and MCP tools to the managed API with an API key from the SmartMemory dashboard. SmartMemory operates the graph, vector, and metadata stores for you.
Enterprise BYOC: the full service runs in the customer's own AWS account. This is available to Enterprise customers through sales and is currently at pilot stage.

Running the full server stack in your own infrastructure is available only through Enterprise BYOC.

Extension Points

Custom Components

# Plugin architecture
class CustomExtractor(BaseExtractor):
    def extract(self, content):
        # Custom extraction logic
        pass

# Registration
register_component("extractor", "custom", CustomExtractor)

Custom Algorithms

# Pluggable algorithms
class CustomSimilarity(BaseSimilarity):
    def compute_similarity(self, item1, item2):
        # Custom similarity computation
        pass

# Integration
similarity_registry.register("custom", CustomSimilarity)

Custom Backends

# Backend abstraction
class CustomBackend(BaseBackend):
    def connect(self):
        # Custom connection logic
        pass
    
    def query(self, cypher):
        # Custom query execution
        pass

# Registration
backend_registry.register("custom", CustomBackend)

Future Architecture Considerations

Planned Enhancements

Distributed Architecture: Multi-node deployment
Event Sourcing: Complete audit trail
CQRS Pattern: Separate read/write models
GraphQL API: Flexible query interface

Research Directions

Neuromorphic Computing: Brain-inspired architectures
Quantum Computing: Quantum similarity algorithms
Edge Computing: Distributed memory networks
Federated Learning: Collaborative memory systems

Best Practices

Development Guidelines

Component Isolation: Minimize dependencies between components
Interface Contracts: Clear API definitions and contracts
Error Handling: Comprehensive error recovery strategies
Testing Strategy: Unit, integration, and performance tests
Documentation: Comprehensive API and architecture docs

Operational Guidelines

Monitoring: Comprehensive observability setup
Backup Strategy: Regular data backup and recovery testing
Capacity Planning: Proactive resource management
Security Updates: Regular dependency and security updates
Performance Tuning: Continuous optimization based on metrics

This architecture provides a solid foundation for building scalable, reliable, and extensible agentic memory systems while maintaining flexibility for future enhancements and integrations.

System Architecture Overview

On this page