HNSW Embedding Index

Cortexa maintains an in-memory HNSW (Hierarchical Navigable Small World) index for fast approximate nearest-neighbor search over thought embeddings. The index powers semantic search, knowledge graph auto-linking, and context injection.

Implementation

The index is implemented in src-tauri/src/embedding_index.rs using the instant-distance crate, a pure-Rust HNSW implementation.

Core Struct

pub struct EmbeddingIndex {
    hnsw: Option<HnswMap<CosinePoint, String>>,  // Built HNSW graph
    vectors: HashMap<String, Vec<f32>>,            // All vectors in the graph
    pending: HashMap<String, Vec<f32>>,            // Vectors added since last build
    excluded: HashSet<String>,                     // IDs to skip in search
    dims: Option<usize>,                           // Embedding dimensionality
    hnsw_count: usize,                             // Number of items in the HNSW graph
}

The index uses a split architecture: a pre-built HNSW graph for the bulk of items, and a pending buffer for recently added items. This avoids the cost of rebuilding the entire graph on every insertion.

Distance Metric

The index uses cosine distance (1 - cosine similarity) as its metric:

impl instant_distance::Point for CosinePoint {
    fn distance(&self, other: &Self) -> f32 {
        1.0 - cosine_sim(&self.0, &other.0)
    }
}

Cosine similarity is computed as dot(a, b) / (|a| * |b|), returning values between -1.0 and 1.0. The distance function inverts this so that similar items have small distances.

Lifecycle

Build on Startup

When the app starts, EmbeddingIndex::build_from() loads all stored embedding vectors from the items.embedding column in SQLite and builds the HNSW graph:

App startup
    |
    v
Load all (id, embedding) pairs from SQLite
    |
    v
EmbeddingIndex::build_from(items)
    |
    v
instant_distance::Builder::default().build(points, ids)
    |
    v
HNSW graph ready in memory

The builder uses instant-distance's default parameters for the graph construction (number of layers, max connections per layer, ef_construction).

Incremental Insert

When a new item is embedded (or an existing item is re-embedded), insert() adds the vector to the pending buffer rather than rebuilding the entire graph:

pub fn insert(&mut self, id: String, embedding: Vec<f32>) {
    // If dimensionality changed (model switch), clear everything
    if dims mismatch { self.clear(); }

    // If updating an existing item in the HNSW graph, exclude the old entry
    if self.vectors.contains_key(&id) {
        self.excluded.insert(id.clone());
    }

    self.pending.insert(id, embedding);
}

The excluded set handles updates: when an item is re-embedded, its old vector in the HNSW graph is excluded from search results, and the new vector in the pending buffer takes its place.

Dimensionality Change Detection

If you switch embedding models (e.g., from nomic-embed-text at 768 dims to text-embedding-3-small at 1536 dims), the insert method detects the dimensionality mismatch and clears the entire index. This forces a full rebuild with the new model's vectors.

Auto-Rebuild Threshold

The needs_rebuild() method checks whether the pending buffer has grown large enough to warrant a full graph rebuild:

pub fn needs_rebuild(&self) -> bool {
    let threshold = (self.hnsw_count / 10).max(100);
    self.pending.len() > threshold
}

The threshold is the larger of:

• 10% of the current HNSW graph size
• 100 items (minimum)

When the threshold is exceeded, the app calls rebuild() which constructs a fresh HNSW graph from all current vectors (graph + pending, minus excluded).

Remove

The remove() method handles item deletion:

pub fn remove(&mut self, id: &str) {
    self.pending.remove(id);
    self.excluded.insert(id.to_string());
}

The item is removed from the pending buffer (if present) and added to the exclusion set (to skip it in the HNSW graph).

Search Algorithm

Every search query runs two parallel strategies and merges the results:

Query vector
    |
    +---> HNSW graph search (O(log N), approximate)
    |         |
    |         v
    |     Top-K results (excluding excluded set)
    |
    +---> Brute-force scan of pending buffer (O(P), exact)
              |
              v
          All matches above min_score
    |
    v
Merge, sort by score descending, truncate to top_k

HNSW Graph Search

The HNSW search uses instant-distance's built-in search, requesting top_k + excluded.len() results to account for items that will be filtered out:

let k = (top_k + self.excluded.len()).min(self.hnsw_count);
for result in hnsw.search(&point, &mut search).take(k) {
    if self.excluded.contains(id) { continue; }
    let score = 1.0 - result.distance;  // Convert back to similarity
    if score >= min_score { results.push((id, score)); }
}

HNSW provides O(log N) approximate nearest-neighbor search, delivering sub-10ms query times even at 100K+ items.

Pending Buffer Search

The pending buffer is searched with a brute-force linear scan:

for (id, vec) in &self.pending {
    if self.excluded.contains(id) { continue; }
    let score = cosine_sim(query, vec);
    if score >= min_score { results.push((id, score)); }
}

Since the pending buffer is typically small (under the rebuild threshold), this scan is fast.

Search by ID

The search_by_id() method finds items similar to a specific existing item. It looks up the item's vector (checking pending first, then the main vectors map) and runs a standard search, excluding the query item from results. This powers the knowledge graph auto-linking feature.

Memory Budget

Each item in the index consumes:

Component	Size	Notes
Embedding vector	dims * 4 bytes	e.g., 768 dims = 3,072 bytes
UUID string	~36 bytes	Item ID stored as String
HNSW graph overhead	~200 bytes/item	Layered graph pointers
HashMap entry	~80 bytes	vectors + pending map overhead

Total per item (768-dim model): ~3.3 KB

For a vault of 10,000 items: ~33 MB of memory for the full index. At 100,000 items: ~330 MB. The memory footprint scales linearly with item count and embedding dimensionality.

Common Embedding Dimensions

Model	Dimensions	Per-Item Memory
nomic-embed-text (Ollama)	768	~3.3 KB
all-minilm (Ollama)	384	~1.8 KB
text-embedding-3-small (OpenAI)	1536	~6.5 KB
text-embedding-3-large (OpenAI)	3072	~12.5 KB
text-embedding-004 (Gemini)	768	~3.3 KB
voyage-3 (Voyage)	1024	~4.3 KB

Integration with Hybrid Search

The HNSW index is one half of Cortexa's hybrid search system. During a search:

1. FTS5 returns keyword-matched items with BM25 scores.
2. HNSW returns semantically similar items with cosine similarity scores.
3. The results are merged with a configurable weight (default 0.3 keyword / 0.7 semantic).
4. Optional context boosts (project +15%, file-path +10%) are applied.
5. An optional LLM re-ranker reorders the final top-K.

This hybrid approach catches both exact keyword matches (function names, identifiers) and conceptual similarity (related ideas expressed differently).

Re-embedding

When you switch embedding models or providers, all existing vectors become invalid (different dimensionality or different vector space). The "Re-embed all" button in Settings > LLM:

1. Clears the in-memory HNSW index.
2. Iterates through all items.
3. For each item, calls the new embedding provider and stores the result.
4. The HNSW index rebuilds progressively as new vectors arrive.
5. Progress is emitted via the embedding-migration-progress event.