Are these storage benchmarks or end-to-end agent latency?

Primary numbers are Storage / v4 stress benchmarks: mock embeddings isolating the SDK + database. A separate LIVE suite measures real provider latency. Never mix the two.

Why don’t competitor latency cells have numbers?

We only fill a cell when a publisher measured the same workload class. Mem0/Zep publish accuracy (LoCoMo/LongMemEval); Qdrant/Weaviate publish ANN engine suites. Those are different experiments — we cite them instead of inventing Wolbarg-style ops/sec.

What changed in the v0.4 suite?

New coverage for embedding cache, write-time dedupe, subscribe() delivery, BEGIN IMMEDIATE concurrency, bulk 2k insert, and multi-process SQLite writers — plus dual-backend stress at 8/16/32 writers.

Why is mock mode the default?

API rate limits fail long before SQLite/Postgres contention. Mock embeddings reveal the real storage ceiling. Use LIVE spots for provider-inclusive latency.

Same-process vs multi-process concurrency?

Same-process numbers measure Node async writers on one client. Multi-process numbers measure separate OS processes sharing one SQLite file (BEGIN IMMEDIATE + retry). Compare them separately.

Wolbarg · v0.4.0 · Benchmarks

Measured like infrastructure.Dual-backend. Reproducible. No invented competitor timings.

Full SDK path on SQLite and PostgreSQL — startup, bulk insert, recall, embedding cache, write-time dedupe, subscribe delivery, and multi-writer concurrency. Storage suite uses mock embeddings so you see the database ceiling, not the API quota.

SQLite results PostgreSQL results Download JSON

SQLite bulk insert @ 2k

7.51k/s

Historical · Wolbarg 0.4 mock stress suite

SQLite recall p95 @ 2k

4.83 ms

Historical · v0.4 storage suite · mock embeddings

SQLite 16 writers

8.66k/s

Historical · v0.4 same-process concurrency

Embedding cache

90% fewer calls

Historical · v0.4 repeated-ingest microbench

PG bulk insert @ 2k

4.08k/s

Historical · v0.4 local Postgres + pgvector

Fastest bulk insert

7.51k/s

SQLite remember path · 2,000 memories · mock embeds

Cache call reduction

90%

100 chunks · 20 unique · 2 passes (embedding-cache bench)

16-writer throughput

8.66k/s

SQLite same-process · p95 2.46 ms · 0 failures

Multi-process integrity

0% errors

Up to 20 OS processes writing one SQLite file (v0.4 locks)

Credibility · Two suites

Never confuse storage latency with provider latency

Transparency is the product. Wolbarg ships two benchmark suites on purpose.

Storage / v4 stress (mock)

Embeddings: Mock OpenAI-compatible embeddings
Measures: SDK + database performance (I/O, indexes, concurrency, cache, dedupe, subscribe)
Use for: Comparing SQLite vs PostgreSQL inside Wolbarg 0.4

Production / LIVE Benchmarks

Embeddings: Real embedding providers (OpenAI, etc.)
Measures: End-to-end latency including network + provider time
Use for: Estimating real app wait times — not storage ceiling

Numbers on this page are from the Storage Benchmarks suite (mock · v4-stress). The suite artifact recorded wolbarg@0.4.0; this release packages the same production-hardened path as wolbarg@0.4.0.

SQLite

Local-first memory that feels instantaneous

WAL, prepared statements, in-process vectors, FTS5 hybrid — the default for single-node agents.

Cold start

16.18 ms

Warm start

1.87 ms

Batch @ 200

5.80k/s

Bulk insert @ 2k

7.51k/s

Recall p50 @ 2k

4.12 ms

Recall p95 @ 2k

4.12 ms

8 writers

6.08k/s

16 writers

8.66k/s

32 writers

6.80k/s

Cache speedup

1.47×

Full SQLite table

Benchmark	Dataset	Result
Startup	Cold ready()	16.18 ms
Startup	Warm reopen	1.87 ms
rememberBatch	200	5,795 ops/sec
Bulk insert	2000	7,509 ops/sec
Recall	p50 @ 2k	4.12 ms
Recall	p95 @ 2k	4.83 ms
Embedding cache	spot speedup	1.47×
Embedding cache	call reduction	90%
Dedupe upsert	exact match	action=updated · active=1
Subscribe	remember/update/forget	pass
Concurrency	8 writers × 20	6,084 ops/sec · p95 3.05 ms
Concurrency	16 writers × 20	8,660 ops/sec · p95 2.46 ms
Concurrency	32 writers × 20	6,798 ops/sec · p95 22.94 ms
Mixed R/W storm	stress	0 failures
Multiprocess	20 OS writers	245 ops/sec · 0% errors · integrity OK
Concurrent exact-dedupe	race	1 id · 11 updates

Production notes

Best default for local agents, CLI tools, and single-node apps.
WAL + BEGIN IMMEDIATE + busy retry is the 0.4 multi-writer default.
Same-process 16 writers hit 8.66k ops/s in this suite; multi-process p95 rises as the file serializes.
Use Postgres when you need cross-process subscribe() delivery.

Environment

Backend: SQLite + WAL
Platform: win32/arm64 · Node v24.13.1 · 8 CPUs
Embeddings: local-mock-openai-compatible · mock-embed

PostgreSQL

Multi-tenant memory that scales writers

pgvector HNSW, prepared statements, insert coalescing, adaptive overfetch — for shared agent fleets.

Cold start

91.39 ms

Warm start

59.89 ms

Batch @ 200

2.79k/s

Bulk insert @ 2k

4.08k/s

Recall p50 @ 2k

23.29 ms

Recall p95 @ 2k

23.29 ms

8 writers

2.56k/s

16 writers

3.33k/s

32 writers

3.80k/s

Cache speedup

1.18×

Full PostgreSQL table

Benchmark	Dataset	Result
Startup	Cold ready()	91.39 ms
Startup	Warm reopen	59.89 ms
rememberBatch	200	2,795 ops/sec
Bulk insert	2000	4,085 ops/sec
Recall	p50 @ 2k	23.29 ms
Recall	p95 @ 2k	141.5 ms
Embedding cache	spot speedup	1.18×
Dedupe upsert	exact match	action=updated · active=1
Subscribe LISTEN/NOTIFY	remember/update/forget	pass
Concurrency	8 writers × 20	2,555 ops/sec · p95 8.51 ms
Concurrency	16 writers × 20	3,335 ops/sec · p95 9.48 ms
Concurrency	32 writers × 20	3,802 ops/sec · p95 14.77 ms
Mixed R/W storm	stress	0 failures
Concurrent exact-dedupe	race	1 id · 11 updates

Production notes

Best default for multi-tenant SaaS and shared agent fleets.
Always benchmark against local Docker — remote Neon/Supabase RTTs hide Wolbarg cost.
Recall p95 in this suite is higher than SQLite because search leaves the process boundary.
LISTEN/NOTIFY powers realtime events across workers.

Environment

Backend: Postgres + pgvector
Host: local Postgres + pgvector
Platform: win32/arm64 · Node v24.13.1 · 8 CPUs

v0.4 features

New capabilities measured in this release

Embedding cache, dedupe, subscribe, and multi-process SQLite writers — not just raw insert/search.

Embedding cache

90% fewer provider calls

Repeated-ingest microbench · embedding-cache.json

Cache spot speedup

1.47× SQLite

Hot vs cold embed path in v4 stress

Exact dedupe race

1 unique id

11 concurrent updates converge correctly

subscribe()

remember → update → forget

In-process (SQLite) · LISTEN/NOTIFY (Postgres)

OS writers	Throughput	p50	p95	p99	Errors
2	123.1 ops/s	0.77 ms	3.17 ms	9.66 ms	0%
5	221.2 ops/s	0.78 ms	6.09 ms	43.94 ms	0%
10	246.3 ops/s	0.81 ms	15.48 ms	52.66 ms	0%
20	245.2 ops/s	0.79 ms	77.77 ms	319.74 ms	0%

Multi-process SQLite writers sharing one file — see Concurrency docs.

Charts

SQLite vs PostgreSQL — same machine, same suite

Bar, line, radar, and horizontal comparisons from the published dual-backend run.

Startup latency

Cold/warm ready() — milliseconds, lower is better (v0.4 mock suite).

Insert throughput

rememberBatch(200) and bulk insert 2k — ops/sec from the v0.4 stress suite.

Recall latency @ 2k

p50 and p95 semantic recall after bulk insert of 2,000 memories.

Overall performance profile

Radar normalized within this dual-backend run (higher = better).

SQLite vs PostgreSQL

Horizontal head-to-head for the metrics that matter most in this suite.

Concurrency

Same-process parallel writers (8 / 16 / 32 × 20 ops) on one client — v0.4.

Recall percentiles @ 2k

p50 vs p95 semantic recall after the 2k bulk insert.

Recall head-to-head

Same recall percentile bars for SQLite vs PostgreSQL at 2k.

Embedding cache speedup

Hot vs cold embed path on repeated texts (spot measurement in v4 suite).

Insert stages

Batch 200 vs bulk 2k throughput (ops/sec).

Multi-process SQLite p95

Separate OS processes writing one SQLite file — p95 ms by writer count (Postgres N/A).

Cache spot speedup

Dimensionless speedup (×) from the embedding-cache spot case.

Competitors

Fair comparisons only

We only place a number in a competitor latency cell when the publisher measured the same class of workload (Wolbarg-style storage/SDK path with mock or fixed embeddings). Accuracy benchmarks that include LLM judges (LoCoMo, LongMemEval, BEAM) and raw ANN engine suites (Qdrant, Weaviate) are different experiments — we cite them in footnotes but never paste them into Startup / Insert / Search cells.

Capability matrix

✅ Supported · ⚠️ Partial · ❌ No · ❓ Unknown

Feature	Wolbarg	Mem0	Zep	Letta	LangGraph Memory	LlamaIndex	Mastra	Chroma	Qdrant	Weaviate
Local-first / offline	✅	⚠️	⚠️	⚠️	⚠️	⚠️	⚠️	✅	⚠️	⚠️
Native SQLite	✅	⚠️	❌	⚠️	⚠️	⚠️	⚠️	⚠️	❌	❌
Native PostgreSQL	✅	⚠️	⚠️	⚠️	✅	⚠️	⚠️	❌	❌	❌
Framework-agnostic SDK	✅	✅	✅	⚠️	❌	⚠️	❌	✅	✅	✅
Model / provider agnostic	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Semantic recall	✅	✅	✅	✅	⚠️	✅	⚠️	✅	✅	✅
Hybrid keyword + vector	✅	⚠️	✅	❓	❓	✅	❓	⚠️	✅	✅
Metadata filters	✅	✅	⚠️	⚠️	⚠️	✅	⚠️	✅	✅	✅
Memory compression	✅	✅	⚠️	✅	⚠️	⚠️	❓	❌	❌	❌
Write-time dedupe / upsert	✅	⚠️	⚠️	❓	❌	⚠️	❓	⚠️	⚠️	⚠️
Real-time change events	✅	⚠️	⚠️	❓	❌	❌	❓	❌	⚠️	⚠️
Embedding cache	✅	❓	❓	❓	❓	⚠️	❓	❌	❌	❌
Multi-writer concurrency hardening	✅	❓	❓	❓	⚠️	❓	❓	⚠️	✅	✅
Multi-tenant org scoping	✅	⚠️	✅	❓	⚠️	❓	❓	⚠️	✅	✅
First-party telemetry	✅	⚠️	⚠️	⚠️	⚠️	⚠️	⚠️	❌	⚠️	⚠️
Checkpoints / rollback	✅	❌	❌	⚠️	✅	❌	⚠️	❌	❌	❌
Batch remember / recall	✅	⚠️	⚠️	❓	⚠️	✅	⚠️	⚠️	⚠️	⚠️
Document ingest pipeline	✅	⚠️	⚠️	⚠️	⚠️	✅	⚠️	⚠️	❌	⚠️
Open source	✅	✅	⚠️	✅	✅	✅	✅	✅	✅	✅
Public reproducible storage benchmarks	✅	❌	❌	❌	❌	❌	❌	⚠️	✅	✅

Storage-path latency (apples-to-apples only)

Framework	Startup	Insert	Search	Retrieval	Hybrid	Concurrency	Storage	Open Source	Production	License	Architecture
Wolbarg	16.18 ms cold (SQLite · historical v0.4)	7.51k ops/s bulk 2k (SQLite · historical v0.4)	4.83 ms p95 @ 2k (SQLite · historical v0.4)	4.12 ms p50 @ 2k (SQLite · historical v0.4)	Covered in v4 edge suite (historical v0.4)	8.66k ops/s · 16 writers (SQLite · historical v0.4)	SQLite / PostgreSQL	Yes (MIT)	Partial	MIT	Memory SDK
Mem0	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked*	Not publicly benchmarked	Not publicly benchmarked	Pluggable	Yes (Apache-2.0)	Yes	Apache-2.0	Memory platform
Zep	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked*	Not publicly benchmarked	Not publicly benchmarked	Graph + vectors	Partial	Yes	Mixed	Temporal graph memory
Letta	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Agent tiers	Yes	Yes	Apache-2.0	Agent runtime
LangGraph Memory	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Pluggable	Yes	Yes	MIT	Agent framework memory
LlamaIndex	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Pluggable	Yes	Yes	MIT	RAG framework
Mastra	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Framework stores	Yes	Partial	Apache-2.0	TS agent framework
Chroma	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked*	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Embedding DB	Yes	Yes	Apache-2.0	Vector database
Qdrant	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked*	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Vector DB	Yes	Yes	Apache-2.0	Vector search engine
Weaviate	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked*	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Vector DB	Yes	Yes	BSD-3-Clause	Vector database
Supermemory	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked*	Not publicly benchmarked	Not publicly benchmarked	Hosted memory API	Partial	Yes	Proprietary / mixed	Hosted memory
CrewAI Memory	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Framework-integrated	Partial	Partial	Mixed	Multi-agent framework
AutoGen	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Not publicly benchmarked	Framework-dependent	Yes	Partial	MIT	Multi-agent framework

* Cells marked “Not publicly benchmarked*” may have related numbers for a different workload (accuracy evals or ANN engines). Sources below — never treated as Wolbarg storage latency.

Mem0 accuracy / tokens / p50 — LoCoMo, LongMemEval, BEAM — LLM-judge memory quality, not storage ops/sec.
Mem0 evaluation docs — Token-efficient algorithm methodology.
Zep LongMemEval claims — Accuracy + retrieval latency with temporal graph — not Wolbarg remember/recall.
Qdrant ANN benchmarks — Vector engine RPS/latency on ANN datasets.
Weaviate ANN benchmarks — ANN QPS / latency / recall trade-offs.

Why

Why Wolbarg performs well

Numbers without mechanism are marketing. Here is the mechanism — kept simple.

SQLite

WAL mode

Readers never block writers. Crash recovery stays safe under concurrent agent loads.

BEGIN IMMEDIATE (0.4)

Mutating transactions take the write lock up front, then retry with backoff on SQLITE_BUSY.

In-process vector index

sqlite-vec + blob index keep ANN inside the same process — no network hop.

Embedding cache (0.4)

hash(content)+model reuse skips provider calls on repeated text — 90% fewer calls in the published microbench.

FTS5 hybrid

Keyword BM25 lives beside vectors; archived rows are dropped from FTS on archive.

Single-file ops

Backup, copy, and ship memory like any other SQLite database.

PostgreSQL

pgvector + HNSW

Approximate nearest neighbor with HNSW built lazily before the first KNN query.

LISTEN / NOTIFY (0.4)

Cross-process subscribe() delivery for multi-host agents sharing one database.

Insert coalescing

Concurrent remember() calls collapse into unnest batches; large ingest uses COPY.

Adaptive overfetch

Filtered ANN overfetches just enough to preserve recall after org / metadata filters.

Multi-tenant correctness

Organization scoping is enforced on ANN paths so shared clusters do not leak memories.

Writer scaling

Throughput keeps climbing from 8→32 writers in the v0.4 stress suite on the same machine.

Methodology

What we measured — and what we did not

What

ready(), rememberBatch, bulk insert 2k, recall p50/p95, embedding cache, dedupe, subscribe, concurrency 8/16/32, multi-process SQLite writers.

Why mock

Live API quotas fail before storage contention. Mock OpenAI-compatible embeddings isolate SDK + DB cost.

How

Fixed v4-stress workloads, controlled writer counts, published JSON/MD artifacts for SQLite and Postgres.

Limitations

Primary published run is v4-stress (batch 200, bulk 2k, concurrency 8/16/32). Legacy quick-scale 100/1k charts are superseded.
Mock embeddings isolate storage. Real OpenAI/Ollama latency dominates end-to-end recall in LIVE mode.
Postgres numbers are local — not managed cloud round-trips.
Multi-process suite is SQLite-only (separate OS processes, one file).
Competitor storage latencies are not published apples-to-apples — we refuse to invent them.

Hardware · published run

Date: 2026-07-18T17:33:03.832Z
SDK: wolbarg@0.4.0
Suite: mock · v4-stress
Platform: win32/arm64 · Node v24.13.1 · 8 CPUs
Postgres: local Postgres + pgvector
Wall clock: 1.2s · 25 rows

Are these storage benchmarks or end-to-end agent latency?: Primary numbers are Storage / v4 stress benchmarks: mock embeddings isolating the SDK + database. A separate LIVE suite measures real provider latency. Never mix the two.
Why don’t competitor latency cells have numbers?: We only fill a cell when a publisher measured the same workload class. Mem0/Zep publish accuracy (LoCoMo/LongMemEval); Qdrant/Weaviate publish ANN engine suites. Those are different experiments — we cite them instead of inventing Wolbarg-style ops/sec.
What changed in the v0.4 suite?: New coverage for embedding cache, write-time dedupe, subscribe() delivery, BEGIN IMMEDIATE concurrency, bulk 2k insert, and multi-process SQLite writers — plus dual-backend stress at 8/16/32 writers.
Why is mock mode the default?: API rate limits fail long before SQLite/Postgres contention. Mock embeddings reveal the real storage ceiling. Use LIVE spots for provider-inclusive latency.
Same-process vs multi-process concurrency?: Same-process numbers measure Node async writers on one client. Multi-process numbers measure separate OS processes sharing one SQLite file (BEGIN IMMEDIATE + retry). Compare them separately.

Downloads & reproduce

Raw reports and the suite you can run

Beautiful markdown and machine-readable JSON from the same run that powers these charts.

SQLite JSON (v0.4)

v4-stress machine-readable

SQLite Markdown (v0.4)

Human-readable

Postgres JSON (v0.4)

Dual-backend twin

Embedding cache JSON

90% call reduction

Multiprocess JSON

2–20 OS writers

Methodology docs

How to interpret

git clone https://github.com/wolbarg/wolbarg-benchmarks.git
cd wolbarg-benchmarks
npm install
npx tsx v4-stress.ts
npx tsx embedding-cache-bench.ts
npx tsx multiprocess-levels.ts

Open benchmark suite Read methodology Documentation