Skip to Main Content
Talk Intermediate MIT License (software), CC BY 4.0 (talk and slides) First Talk

Indexing & Retrieval in AI Memory Layers

Proposal status is Review Pending
Session Description

The talk will cover retrieval architecture of Meniscus, a local-first memory layer I built for the AI agents. The focus will be on the search and ranking pipeline used to extract relevant facts deterministically from an SQLite database without calling an LLM at read-time.

We will break down the pipeline into four stages:

  1. Hybrid Candidate Search: Running full-text search (SQLite FTS5 with BM25) and dense vector search (sqlite-vec) in parallel to cover both keyword matches and semantic meaning.

  2. Entity Expansion

  3. Score Fusion: Merging different scoring scales (BM25 rank positions and vector cosine similarity) into a single score using Reciprocal Rank Fusion (RRF) and Noisy-OR probability.

  4. Adaptive Cut: Using score drop-offs (the largest gap between consecutive scores) and ratio thresholds instead of a fixed k.


Key Takeaways
  • Why hybrid search is necessary for memory
  • How graph relationships can be handled cleanly without the need for a dedicated graph engine
  • That you don't need to make a LLM call for extracting relevant info out of a memory layer and it can be done deterministically with pure math.

References

Session Categories

Technology architecture
Talk License: MIT License (software), CC BY 4.0 (talk and slides)
Speakers are hidden and anonymized for this event.