Hybrid Search

Cortexa combines two search strategies -- keyword matching and semantic similarity -- and blends their scores to surface the most relevant thoughts. This hybrid approach means you can search by exact function names and by vague descriptions of what you were thinking about, and both work.

How It Works

Every search query runs through a two-phase pipeline, then merges the results.

Phase 1: FTS5 Keyword Search (BM25)

SQLite's FTS5 extension indexes every thought's full body content, summary, and preview. Queries are matched using BM25 scoring, which ranks results by term frequency and document length.

• Prefix search -- append * to match prefixes: error* matches "error", "errors", "errorHandler"
• Multiple terms -- all terms are ANDed by default: redis caching finds thoughts containing both words
• Exact phrases -- wrap in quotes: "connection pool" matches the exact phrase

FTS5 is synchronous and runs entirely in SQLite. It returns results in microseconds regardless of vault size.

Phase 2: HNSW Semantic Search (Cosine Similarity)

Your query is embedded using your configured embedding model (e.g., nomic-embed-text), then searched against the in-memory HNSW (Hierarchical Navigable Small World) index using approximate nearest neighbor search.

• O(log N) complexity -- search time grows logarithmically, not linearly. 100K items is still sub-10ms.
• Cosine similarity -- results are scored 0.0 to 1.0 based on how close the query's meaning is to each thought's meaning.
• Cross-lingual -- if your embedding model supports it (e.g., bge-m3), you can search in one language and find thoughts written in another.

The embedding step is the only async part of the search pipeline -- it requires a round-trip to your LLM provider. After embedding, the HNSW lookup itself is sub-millisecond.

Phase 3: Score Merge

Results from both phases are merged using a weighted blend:

final_score = (cosine_score * semantic_weight) + (bm25_normalized * keyword_weight)

Where semantic_weight = 1.0 - keyword_weight.

BM25 scores are normalized to 0.0-1.0 range (min-max normalization) before blending, so both signals are on the same scale. If all FTS results have identical scores, they are capped at 0.5 to avoid inflating marginal hits.

Items appearing in only one result set still participate -- they get a score of 0.0 from the missing signal. This means a thought with a strong semantic match but no keyword overlap still surfaces, just with a lower blended score.

Results below min_score are filtered out, and the top limit items are returned sorted by final score descending.

Fallback behavior: if the hybrid merge returns zero results but FTS had matches, Cortexa re-runs with a halved min_score threshold to avoid empty results when keyword matches exist but score slightly below the cutoff.

The keyword_weight Slider

The keyword_weight setting controls the blend ratio between keyword and semantic search. Adjust it in Settings > Search & KG.

Value	Behavior
0.0	Pure semantic search. Keywords are ignored entirely.
0.3	Default. 30% keyword, 70% semantic. Good general-purpose balance.
0.5	Equal weight. Useful if your vault has lots of code identifiers.
1.0	Pure keyword search. Embeddings are not queried at all.

Auto-boost for exact-looking queries

Cortexa detects queries that look like code identifiers and automatically boosts keyword weight. A query is considered "exact-looking" if it is a single token (no spaces) and contains any of:

• Underscores: parse_thesis_json
• Dots: config.settings
• Double colons: std::collections::HashMap
• All-uppercase: MAX_RETRIES

For exact-looking queries, keyword weight is raised to at least 0.7, regardless of your configured setting. This ensures that searching for a specific function name gives you keyword-exact matches first, even in a vault tuned for semantic search.

Configuration

All search settings live in Settings > Search & KG:

Setting	Default	Range	Description
default_top_k	20	1-100	Maximum results returned when no explicit limit is passed
min_score	0.0	0.0-1.0	Minimum blended score to include a result. Raise to filter noise.
keyword_weight	0.3	0.0-1.0	Keyword vs semantic blend ratio (see above)
rerank_enabled	false	--	When enabled, the top-K results are re-ordered by an LLM re-ranker

LLM Re-ranking

When rerank_enabled is on, the top results from the hybrid merge are sent to your chat LLM, which re-orders them by relevance to the original query. This adds latency (one LLM call) but can significantly improve result quality for ambiguous queries.

Re-ranking runs after the merge and before the final sort. It does not change scores -- it only reorders.

Context Search (cortexa_context)

The cortexa_context tool (and the equivalent /api/context endpoint) adds two additional boosting signals on top of the standard hybrid search:

Boost	Amount	Trigger
Project boost	+15%	Items captured from the same project as the project parameter
File-path boost	+10% per file	Items mentioning the same files as the file_paths parameter
Confidence boost	Variable	High-confidence items rank higher in FTS search results

Results are grouped by thought type (decisions first, then architecture, patterns, debug, TODOs, discoveries, questions, notes) and formatted as a brief the agent can consume directly.

{
  "context": "fixing the thesis pipeline JSON parser",
  "project": "Cortexa",
  "file_paths": ["src-tauri/src/agent/handlers.rs"],
  "limit": 10,
  "max_chars": 4000
}

The max_chars parameter caps the output size for token budget control -- useful when agents have limited context windows.

Performance

The HNSW index is built in memory at startup from persisted embedding vectors. Search time is dominated by the embedding step (LLM call), not the index lookup.

Vault size	HNSW lookup	FTS5 lookup	Total (with embedding)
100 items	< 1ms	< 1ms	~50-200ms (embedding)
1,000 items	< 1ms	< 1ms	~50-200ms (embedding)
10,000 items	~2ms	< 1ms	~50-200ms (embedding)
100,000 items	~5-8ms	~1-2ms	~55-210ms (embedding)

The embedding call to Ollama on a modern CPU takes 50-200ms depending on the model. Cloud embedding providers may add network latency. The HNSW index and FTS5 lookups are negligible by comparison.

Memory usage: the HNSW index holds all vectors in memory. At 768 dimensions (nomic-embed-text) with float32 values, each item consumes ~3 KB. A 100K-item vault uses ~300 MB of RAM for the index.

Searching from Different Clients

App UI

The search bar at the top of the main window runs hybrid search as you type. Results update live.

CLI

cortexa-cli search "parse_thesis_json" --limit 5

Returns JSON with blended scores.

MCP

{ "name": "cortexa_search", "arguments": { "query": "caching strategy", "limit": 10 } }

HTTP API

curl "http://127.0.0.1:7373/api/search?q=caching+strategy&limit=10" \
  -H "Authorization: Bearer $TOKEN"

All search paths honor the same min_score, keyword_weight, and rerank_enabled settings.

Tips

• Be specific in queries. "Redis connection pool timeout handling" will outperform "Redis" because the semantic signal has more to work with.
• Use prefix search for code. handleAuth* finds handleAuthRequest, handleAuthMiddleware, etc.
• Raise min_score if you get noise. A value of 0.1-0.2 filters out low-confidence matches while keeping strong ones.
• Lower keyword_weight for conceptual search. If you are searching for "approaches to rate limiting" rather than a specific term, pure semantic (0.0) may give better results.
• Use cortexa_context instead of cortexa_search for agent integration. It adds project and file-path boosting and groups results by type -- purpose-built for injecting context into a work session.

What's Next

• Deep Think -- analyze a thought with the expand, critique, synthesize pipeline
• Knowledge Graph -- how thoughts auto-link to their semantic neighbors
• MCP Tools Reference -- all search-related tool parameters