Memory Snapshots

Feature: CORE-SUMMARY-1 · Code: smartmemory/summary/, memory_service/api/routes/summary.py

A memory snapshot is a first-class rollup of a workspace at a point in time: how many entities and relations exist, which domains the workspace clusters into, the top mentioned entities, what changed since the previous snapshot, and the share of content across origin tiers. Snapshots are stored as memory items themselves (memory_type="snapshot"), so they accumulate naturally in the graph and can be diffed, deleted, and surfaced through the same retrieval surface as any other content.

This page covers the snapshot data model, the three triggers that produce snapshots, and the tiered retention policy. The HTTP surface is documented at /memory/summary/*.

Why snapshots

A live SmartMemory workspace grows continuously. There's no "version" of a workspace, no tag, no commit — only the current graph. That makes it hard to answer questions like:

• What did the agent learn this week?
• How did the entity mix shift after we ingested the new corpus?
• Which domains are growing fastest?
• Are we accumulating tier-3 enrichments faster than tier-1 user content?

Snapshots compute these rollups on a cadence (or on demand), persist them, and expose deltas between adjacent or arbitrary pairs. They're the audit trail and the analytics substrate, in one shape.

The snapshot model

A MemorySnapshot (contract: docs/features/CORE-SUMMARY-1/snapshot-contract.json) carries:

The contract tracks @dataclass (extending MemoryBaseModel), not Pydantic — that's deliberate; snapshots are first-class memory items and inherit the same shape as everything else stored in the graph.

The heavy payload (top_entities, domain_clusters, notable_items) is persisted to MongoDB keyed by snapshot_id; the graph row stores the light scalars plus a metadata.snapshot_ref_id pointer. This split means the graph stays compact while the rollup payload stays queryable.

How a snapshot is composed

Composition is split between core and the service:

• Composer (smartmemory/summary/composer.py): pure function that reads the workspace state and returns a MemorySnapshot dataclass. Lives in core so it's reusable from CLI, batch jobs, or tests with no service dependency.
• Persister (memory_service/summary/persister.py): writes the Mongo payload, the light graph row, and the PART_OF edges to notable items as a single atomic operation. R3-5 rollback handling unwinds partial writes.

A Redis Lua-CAD lock (smartmemory:snapshot:lock:<workspace_id>) ensures cross-replica idempotency: scheduled, threshold, and manual triggers all share the same lock key, so a manual snapshot fired during a scheduled sweep window doesn't double-write.

The three triggers

1. Scheduled

When SNAPSHOT_SWEEP_ENABLED=true, a daemon thread (defined in memory_service/api/routes/snapshot_sweep.py) wakes every SNAPSHOT_SWEEP_INTERVAL_SECONDS (default 300) and walks every workspace. For each workspace whose SnapshotConfig.schedule (or the global SNAPSHOT_DEFAULT_SCHEDULE, default "daily") is due, it acquires the lock and composes a snapshot.

If the workspace is unchanged since the last scheduled snapshot, the trigger short-circuits and writes a heartbeat snapshot with is_heartbeat=True and an empty markdown body. This preserves the cadence (so consumers can always find a "today's snapshot") without duplicating real rollup work.

2. Threshold

When SNAPSHOT_THRESHOLD_ENABLED=true, a poller thread checks every SNAPSHOT_THRESHOLD_INTERVAL_SECONDS (default 60) for workspaces that have crossed a watermark since their last snapshot:

Crossing either threshold fires a snapshot with trigger="threshold". This catches bursty workloads — a long batch ingest produces a snapshot at the boundary, even when the next scheduled run is hours away.

3. Manual

POST /memory/summary/generate composes a snapshot synchronously with trigger="manual". The same lock applies, so a manual call during an in-flight scheduled sweep returns the in-flight result rather than racing.

Tiered retention

Snapshots accumulate, and old ones lose value. The retention sweep (memory_service/api/routes/snapshot_retention_sweep.py, gated on SNAPSHOT_RETENTION_ENABLED=true, runs every SNAPSHOT_RETENTION_INTERVAL_SECONDS — default 86400) applies a three-tier policy:

Deletion is two-phase: Mongo payload + graph row + edges drop together, with a Redis tombstone written first. If the Mongo delete succeeds and the graph delete fails, the next sweep replays from the tombstone and finishes the cleanup. Snapshots are deliberately excluded from CORE-SUPERSEDE-1 supersession (they're append-only by design) and from tier-compaction in [Memory Dynamics](/smartmemory/concepts/memory-dynamics/) — they should never be folded into other items.

Snapshot deltas

Adjacent deltas are precomputed when the newer snapshot is written and cached on the snapshot record, so GET /memory/summary/delta?from=N&to=N+1 is O(1). Non-adjacent deltas (e.g. "compare this week to two months ago") are computed on the fly from the two payloads.

A SnapshotDelta carries entities_added, entities_retracted, relations_added, new_domains, conflicts_resolved, and time_span_seconds — enough to render a "since the last snapshot" diff in a UI without loading the full rollup.

Configuration summary

# Scheduled sweep
SNAPSHOT_SWEEP_ENABLED=true
SNAPSHOT_SWEEP_INTERVAL_SECONDS=300
SNAPSHOT_DEFAULT_SCHEDULE=daily   # cron expr or shorthand

# Threshold trigger
SNAPSHOT_THRESHOLD_ENABLED=true
SNAPSHOT_THRESHOLD_INTERVAL_SECONDS=60
SNAPSHOT_THRESHOLD_INGESTS=100
SNAPSHOT_THRESHOLD_TIER1_BYTES=100000

# Retention
SNAPSHOT_RETENTION_ENABLED=true
SNAPSHOT_RETENTION_INTERVAL_SECONDS=86400

All four sweeps default to off so dev/CI doesn't auto-spin daemons. Production deployments enable them in .env.

Related

• Snapshot HTTP API — the seven /memory/summary/* routes
• Memory Dynamics — snapshots are excluded from compaction