Important: This documentation covers Yarn 1 (Classic).
For Yarn 2+ docs and migration guide, see yarnpkg.com.

Package detail

@liraz-sbz/search-mcp

blakazulu4.1kMIT1.6.14TypeScript support: included

A local-first Model Context Protocol (MCP) server that provides semantic search capabilities for codebases

mcp, model-context-protocol, semantic-search, vector-search, lancedb, claude, windsurf, antigravity, cursor, ai-assistant

readme

Search MCP 🔍

Make your AI 58x smarter about your code.

# Navigate to your project, then run:
npx @liraz-sbz/search-mcp setup

Your AI assistant searches your entire codebase semantically. No API keys. No cloud. 100% local.

npm version npm downloads GitHub stars License: MIT

Works with: Claude Desktop • Claude Code • Cursor • Windsurf • Antigravity


Why Search MCP?

Without Search MCP With Search MCP
Copy-paste files manually AI finds code automatically
~488,000 tokens per query ~8,400 tokens per query
"Context limit exceeded" Always fits
Multiple tool calls Single semantic search

Table of Contents


What Does This Do?

Search MCP makes your AI assistant smarter about your code. Instead of you copying and pasting files into the chat, the AI can automatically search your project and find exactly what it needs.

Before Search MCP:

You: "How does login work?"
AI: "I don't have access to your code. Please paste the relevant files."
You: *pastes 5 files*
AI: "Now I can help..."

After Search MCP:

You: "How does login work?"
AI: *automatically searches your code*
AI: "Based on src/auth/login.ts, here's how login works..."

Features

  • Just Works - No setup, no API keys, no accounts
  • Private - Your code never leaves your computer
  • Always Current - Automatically updates when you save files
  • Safe - Never indexes passwords, secrets, or junk files
  • Secure - Built-in protections against path traversal, symlink attacks, and resource exhaustion

Quick Start

Prerequisites: Node.js 18+

Navigate to your project folder and run:

npx @liraz-sbz/search-mcp setup

This interactive wizard will:

  • Confirm you're in the correct project folder
  • Auto-detect and configure your AI assistants (Claude Desktop, Claude Code, Cursor, Windsurf)
  • Offer to index your project immediately with progress bars

See all CLI commands: CLI Reference

Option 2: Quick Setup (One-liner)

npx --yes @liraz-sbz/search-mcp@latest --setup

Configures your AI assistants automatically. You'll need to index separately via your AI assistant.

Option 3: Manual Configuration

Add to your MCP config file:

{
  "mcpServers": {
    "search": {
      "command": "npx",
      "args": ["-y", "@liraz-sbz/search-mcp"]
    }
  }
}

Config file locations:

  • Claude Desktop (Mac): ~/Library/Application Support/Claude/claude_desktop_config.json
  • Claude Desktop (Windows): %APPDATA%\Claude\claude_desktop_config.json
  • Claude Code: claude mcp add search -- npx @liraz-sbz/search-mcp

See full guide: Getting Started

After Setup

  1. Restart your AI assistant
  2. Verify connection: Type /mcp and check that "search" is listed
  3. Start searching: Ask "How does login work?"

That's it!


Standalone CLI

Search MCP also works as a standalone CLI tool - no MCP client required:

# Index your project
npx @liraz-sbz/search-mcp index

# Search directly from terminal
npx @liraz-sbz/search-mcp search "authentication logic"

# Check index status
npx @liraz-sbz/search-mcp status

Features:

  • Progress bars and colored output
  • --json flag for scripting
  • Works independently of AI assistants

Perfect for quick searches, debugging, or CI/CD integration.

Full CLI Reference →


What Can You Ask?

Once set up, just talk naturally:

  • "How does user registration work?"
  • "Find all files related to payments"
  • "What's the database schema?"
  • "Show me where errors are handled"
  • "What files import the Logger class?"
  • "Search the docs for API rate limits"

See more examples and use cases.


Performance

Metric Value
Efficiency vs Grep 58x fewer tokens
Search speed ~400ms (with GPU acceleration)
Tokens per query ~8,400
Codebase tested 306 files, 4,231 chunks

Semantic search returns focused code chunks instead of entire files. Your AI stays under context limits even on large codebases.

Full benchmark details →


Configuration

Config is auto-generated when you first index a project:

  • macOS/Linux: ~/.mcp/search/indexes/<project-hash>/config.json
  • Windows: %USERPROFILE%\.mcp\search\indexes\<project-hash>\config.json

Finding your config file: Ask your AI assistant "Where is my config file?" or "Show me my search config" - it will use the get_config tool to return the exact path.

Key options:

Option Default Description
indexingStrategy "realtime" "realtime", "lazy", or "git"
include ["**/*"] Files to index
exclude [] Files to skip
indexDocs true Index .md and .txt files separately

Indexing Strategies:

Strategy Best For
realtime Small projects, instant freshness
lazy Large projects, index only when searching
git Only search committed code

For full configuration options, see the Configuration Reference.


FAQ

Does my code leave my computer? Never. All processing happens locally. No cloud, no API calls, no tracking.

How big can my codebase be? Tested on projects with 1000+ files. Indexing takes ~1 minute for most projects.

What languages are supported? Any text-based code or documentation. The semantic search understands concepts across all languages.

How do I update the index? File changes are detected automatically. Use reindex_project for a full rebuild.


Documentation

Guide Description
Getting Started Detailed installation for all clients
CLI Reference Standalone command-line interface
Configuration Full config reference + indexing strategies
API Reference Complete tool documentation
Examples Use cases & best practices
Troubleshooting Common issues & solutions
Roadmap Planned features
Changelog Version history
Contributing How to contribute

For Developers

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      MCP CLIENT                             │
│  (Claude Desktop, Claude Code, Cursor, Windsurf, etc.)      │
└─────────────────────────┬───────────────────────────────────┘
                          │ MCP Protocol (stdio)
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                  SEARCH MCP SERVER                          │
│  ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌─────────────┐ │
│  │create_    │ │search_code│ │search_by_ │ │get_index_   │ │
│  │index      │ │           │ │path       │ │status       │ │
│  └───────────┘ └───────────┘ └───────────┘ └─────────────┘ │
│  ┌───────────┐ ┌───────────┐ ┌───────────┐                 │
│  │reindex_   │ │reindex_   │ │delete_    │                 │
│  │project    │ │file       │ │index      │                 │
│  └───────────┘ └───────────┘ └───────────┘                 │
└─────────────────────────┬───────────────────────────────────┘
                          │
        ┌─────────────────┼─────────────────┐
        ▼                 ▼                 ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│   Chunking    │ │   Embedding   │ │   LanceDB     │
│   Engine      │ │   (BGE)       │ │   (Local)     │
└───────────────┘ └───────────────┘ └───────────────┘

MCP Tools

Tool Description Confirmation
create_index Create a search index for the current project Yes
search_code Semantic search for relevant code chunks No
search_docs Semantic search for documentation files No
search_by_path Find files by name/glob pattern No
get_index_status Show index statistics and paths No
get_config Get config file path and contents No
get_file_summary Extract symbols and complexity metrics from a file No
reindex_project Rebuild the entire index Yes
reindex_file Re-index a single file No
delete_index Remove the project index Yes

Technical Details

Property Value
Embedding Models Code: Xenova/bge-small-en-v1.5 (384d), Docs: Xenova/bge-base-en-v1.5 (768d)
Code Chunk Size ~1000 tokens
Doc Chunk Size ~2000 tokens
Search Latency < 200ms
Storage ~/.mcp/search/indexes/ (macOS/Linux) or %USERPROFILE%\.mcp\search\indexes\ (Windows)

GPU Acceleration

Search MCP automatically uses GPU acceleration when available for faster indexing:

Platform GPU Support Notes
Windows DirectML Automatic GPU acceleration on all modern GPUs (NVIDIA, AMD, Intel)
macOS CPU only CoreML not available in Node.js bindings
Linux CPU only CUDA requires separate package (not included)

GPU Compatibility (Windows):

  • NVIDIA: GeForce GTX 1000+, RTX series, Quadro
  • AMD: RX 400+, Radeon Pro
  • Intel: Arc, UHD/Iris integrated graphics

GPU acceleration is automatic - no configuration needed. The system detects available hardware and selects the best option. Check get_index_status to see which compute device is being used.

Hybrid GPU Laptops (NVIDIA + Intel/AMD integrated):

On laptops with both discrete and integrated GPUs, Search MCP defaults to CPU to avoid DirectML selecting the wrong GPU. To enable GPU acceleration:

  1. Open Windows Settings → System → Display → Graphics
  2. Click Add an app and select your IDE (VS Code, Cursor, etc.) or terminal
  3. Click on the app → Options → Select High performance
  4. Set environment variable: FORCE_DML=1

This tells Windows to use your discrete GPU (NVIDIA/AMD) for that application.

For full technical documentation, see ENGINEERING.RFC.md.


Updating & Uninstalling

Updating

If using npx in your config (recommended): Updates are automatic - you always get the latest version.

If installed globally:

npm install -g @liraz-sbz/search-mcp

Uninstalling

1. Remove from your AI assistant:

  • Claude Code: claude mcp remove search
  • Other clients: Delete the search entry from your MCP config file

2. Uninstall the package (if installed globally):

npm uninstall -g @liraz-sbz/search-mcp

3. (Optional) Remove index data:

  • macOS/Linux: rm -rf ~/.mcp/search
  • Windows: rmdir /s /q %USERPROFILE%\.mcp\search

Troubleshooting

Common issues:

Issue Solution
"Index not found" Say "Index this project" to create the index
MCP connection issues Run npx --yes @liraz-sbz/search-mcp@latest --setup to reconfigure
Search results seem wrong Run reindex_project to rebuild
Changes not detected Run reindex_file for specific file

CLI commands:

npx @liraz-sbz/search-mcp index                # Create/update index
npx @liraz-sbz/search-mcp search "query"       # Search code
npx @liraz-sbz/search-mcp status               # Show index info
npx @liraz-sbz/search-mcp --setup              # Configure MCP clients

See the CLI Reference for all commands and options.

Debug mode: Set DEBUG=1 or SEARCH_MCP_DEBUG=1 environment variable for verbose logging.

For all error codes and solutions, see the Troubleshooting Guide.


Privacy & License

Your code stays on your computer. Nothing is uploaded anywhere. No accounts, no API keys, no tracking.

MIT License - See LICENSE for details.


Getting Help

changelog

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.6.14] - 2025-12-20

Fixed

  • Claude CLI configured detection - Fixed "✓ configured" indicator not appearing for Claude Code CLI option in setup. Now properly checks ~/.claude.json project config instead of only checking .mcp.json files.

  • GPU fallback notification - When user selects GPU but system falls back to CPU (e.g., due to hybrid GPU system), a clear warning is now displayed: "⚠ GPU could not be used - running with CPU instead" with the reason. Previously, the fallback happened silently.

Improved

  • Hybrid GPU detection upfront - Setup now detects hybrid GPU systems (discrete + integrated) before asking for device choice and warns the user. Default changes to CPU for hybrid systems to avoid issues.

  • Friendlier CLI experience - Multiple UX improvements:

    • Empty input now shows "Please select an option" instead of silently quitting
    • Invalid options show which options are valid
    • Step-by-step headers explain what each phase does ("Step 1: Connect to your AI assistant", "Step 2: Index your codebase")
    • GPU options have clearer descriptions ("Faster for large codebases" vs "Reliable, works on all systems")
    • Skip option is clearer: "Already configured - Skip to code indexing"
    • Better completion message when skipping to indexing

[1.6.13] - 2025-12-20

Fixed

  • Setup CLI reconfiguration - Fixed "MCP server search already exists" error when reconfiguring via Claude CLI. Now automatically removes existing config before adding new one.

[1.6.12] - 2025-12-20

Fixed

  • DirectML GPU selection on hybrid laptops - Fixed critical bug where DirectML was selecting the weak integrated GPU (AMD/Intel) instead of the discrete GPU (NVIDIA) on hybrid laptop systems. The integrated GPU would fail with DmlCommittedResourceAllocator errors due to insufficient memory. Now automatically detects hybrid GPU systems and falls back to CPU for reliability. Added instructions for enabling GPU via Windows Graphics Settings (Settings → System → Display → Graphics → Add IDE → High Performance) and FORCE_DML=1 environment variable.

  • DirectML runtime fallback - Added runtime GPU-to-CPU fallback when DirectML encounters memory allocation errors during embedding. If DirectML fails after initialization (e.g., GPU runs out of memory during batch processing), the embedding engine automatically detects the error and seamlessly switches to CPU to complete the operation.

  • Setup reconfiguration - Setup now allows reconfiguring already-configured MCP clients. Previously, configured clients were hidden from the menu. Now all clients are shown with a "✓ configured" marker, and selecting one will remove the old config and install fresh.

[1.6.9] - 2025-12-20

Fixed

  • GPU batch processing - Fixed critical bug where embeddings were processed one text at a time instead of true batch processing. GPU (DirectML) is now 10-25% faster than CPU at all tested sizes. Reduced GPU transfers from 64 per batch to 1.

[1.6.8] - 2025-12-19

Fixed

  • README anchor links - Replaced separate <a id> tags with HTML <h2 id> headers for reliable cross-platform anchor navigation (GitHub, npm, and other markdown renderers)

[1.6.7] - 2025-12-19

Fixed

  • README anchor links - Added explicit anchor tags to all Table of Contents sections (superseded by 1.6.8)

[1.6.6] - 2025-12-19

Changed

Purple Spinner with Per-File Progress

  • Purple animated spinner - Current file spinner now uses magenta/purple color to differentiate from the blue scanning spinner
  • Per-file progress percentage - Shows progress for each individual file (0-100%), resets when new file starts
    • Animates smoothly from 0% to 95% while file is processing
    • Resets to 0% when a new file begins
  • Example display:
    Code Index
      ✔ Scanned 366 files → 366 indexable
      Current  ⠋ auth-service.ts 47%
      Overall  [████████████████░░░░░░░░░░░░░░░░░░░░░░░░] 45% | 165/366 files

[1.6.5] - 2025-12-19

Added

Live Progress Display with Current File

  • Multi-bar progress display during indexing - Shows both current file and overall progress
    • Current line: Animated spinner + filename + live percentage
    • Overall line: Progress bar with file count
    • Example:
      Code Index
        Scanning... Found 366 files to index
        Current  ⠋ auth-service.ts 45%
        Overall  [████████████████░░░░░░░░░░░░░░░░░░░░░░░░] 45% | 165/366 files
  • Smooth spinner animation - Timer-based animation (80ms) independent of progress callbacks
    • Spinner cycles through frames: ⠋ ⠙ ⠹ ⠸ ⠼ ⠴ ⠦ ⠧ ⠇ ⠏
    • Percentage updates in real-time as files are processed
  • Applied to both CLI commands and setup wizard - Consistent experience across all indexing operations

[1.6.2] - 2025-12-19

Changed

Simplified CLI Progress Display

  • Cleaner indexing progress output - Streamlined from 4 phases to 2 visual steps
    • Before: Scanning spinner → Chunking bar → Embedding bar → Storing spinner
    • After: Scanning → single "Indexing" progress bar → complete
  • Improved scan results format - Shows "Scanned X files → Y indexable" with correct counts
  • Better summary format - Added separator line and organized Code/Docs/Duration/Device info

Fixed

Delete Command Not Working

  • Fixed delete command hanging - Command was missing from CLI_COMMANDS routing list
    • Before: npx @liraz-sbz/search-mcp delete would hang (tried to start MCP server)
    • After: Command routes correctly to CLI handler

[1.6.1] - 2025-12-19

Fixed

Progress Bar Reset Bug (SMCP-101)

  • Fixed progress bars resetting between batches during indexing
    • Before: Each batch showed new progress bars (Chunking 6%, Embedding 100%, Storing, Chunking 13%, etc.)
    • After: Single continuous progress bar per phase that accumulates across all batches
    • "Scanned X files → Y indexable" message now shows only once
    • "Storing..." spinner now shows only once instead of per-batch

[1.6.0] - 2025-12-19

Added

Clean CLI Output (SMCP-101)

  • Silent console mode for cleaner CLI output - Verbose log messages are now suppressed during CLI operations

    • Before: CLI showed timestamped log messages mixed with progress bars
    • After: Clean progress bars and spinners without log noise
  • --verbose flag for debugging - Re-enable detailed logging when needed

    • npx @liraz-sbz/search-mcp setup --verbose - Show all logs during setup
    • npx @liraz-sbz/search-mcp index --verbose - Show all logs during indexing
    • npx @liraz-sbz/search-mcp reindex --verbose - Show all logs during reindexing
  • ONNX runtime warning suppression - Native ONNX warnings no longer pollute console output

    • Filters "nodes were not assigned to preferred execution providers" warnings
    • Filters "Unable to determine content-length" messages during model download
    • Warnings still visible in debug mode (DEBUG=1 or SEARCH_MCP_DEBUG=1)
  • Aggregated progress tracking - Progress bars now accumulate across batches

    • Before: Progress bars reset to 0% for each processing batch
    • After: Single continuous progress bar per phase (scanning, chunking, embedding)
  • Separate progress sections for code and docs - Clear visual separation between code and docs indexing

    • "Code Index:" section with scanning, chunking, embedding progress
    • "Docs Index:" section with same progress phases (if docs indexing enabled)
    • Summary shows separate stats: code files/chunks and docs files/chunks
  • Compute device choice during setup - Users can now choose between GPU and CPU before indexing

    • GPU (DirectML): Faster indexing but may cause system stuttering
    • CPU: Slower but system stays responsive
    • Only shown on Windows (DirectML is Windows-only)
    • New setPreferredDevice() API for programmatic control
  • CLI delete command - Remove index for current project

    • search-mcp delete - Interactive with confirmation
    • search-mcp delete --force - Skip confirmation
    • search-mcp delete --json - JSON output for scripting
  • Logger setSilentConsole() method - New API to programmatically suppress console output

    • Logs still written to log files for debugging
    • Only console output is suppressed

[1.5.0] - 2025-12-19

Added

Updated Benchmarks

  • Token efficiency improved to 57.9x vs grep (up from 40-43x)
    • Full codebase benchmark: 306 files, 4,231 chunks
    • MCP uses only 1.7% of the tokens that grep would require
    • Best query performance: 89.9x efficiency for "error handling patterns"
  • Updated documentation with new benchmark results in README.md and search-comparison-test.md

Code Comment Extraction (SMCP-100)

  • Extract documentation comments from code files - JSDoc, docstrings, and other doc comments are now searchable via search_docs

    • Before: Only .md and .txt files were indexed in the docs search
    • After: Documentation embedded in code (JSDoc, Python docstrings, Rust doc comments, etc.) is extracted and indexed
  • Multi-language support:

    • JavaScript/TypeScript: JSDoc and TSDoc (/** ... */)
    • Python: Docstrings ("""...""", '''...''') with Google-style sections (Args, Returns, Raises)
    • Rust: Doc comments (///, //!)
    • Go: Doc comments (// preceding declarations)
    • Java: Javadoc (/** ... */)
    • C#: XML documentation (/// <summary>)
    • C/C++: Doxygen-style comments
  • Rich metadata extraction:

    • Symbol association (function, class, method, etc.)
    • Tag parsing (@param, @returns, @throws, etc.)
    • Line number tracking for source navigation
    • Language detection
  • Configuration option:

    • extractComments: true (default) - Enable/disable comment extraction in search-mcp.config.json
  • New exports from commentExtractor.ts:

    • Types: CommentType, CommentTag, ExtractedComment, CommentExtractionOptions
    • Constants: DEFAULT_COMMENT_OPTIONS, SUPPORTED_EXTENSIONS
    • Core: extractComments(), supportsCommentExtraction(), formatCommentForIndex()
    • Tag parsing: parseJSDocTags(), parsePythonDocTags()
    • Content cleaning: cleanJSDocContent(), cleanPythonDocstring(), cleanRustDocContent(), cleanGoDocContent()
  • Integration with docs indexing pipeline:

    • Comments are extracted during create_index and reindex_project
    • Indexed in the docs LanceDB table with [code-comment] path prefix
    • Searchable via search_docs tool alongside regular documentation
  • Benefits:

    • API documentation is now searchable without separate doc generation
    • Function/class documentation appears in search results
    • Better code understanding without leaving the search context

Markdown Header Chunking (SMCP-099)

  • Semantic chunking for markdown documentation - Chunks align with section boundaries (h1-h6)

    • Before: Arbitrary character-based splits could cut mid-section, losing context
    • After: Each chunk represents a complete section with full header context
  • Header hierarchy tracking:

    • Tracks path from root to current header (e.g., ["Installation", "Prerequisites", "Node.js"])
    • Includes breadcrumb context in chunks: [Installation > Prerequisites]
    • Proper handling of nested headers (h2 under h1, h3 under h2, etc.)
  • Comprehensive markdown parsing:

    • ATX headers (# through ######) with trailing marker removal
    • Setext headers (=== for h1, --- for h2)
    • YAML frontmatter detection and stripping
    • Code block boundary detection (prevents parsing headers inside code blocks)
  • Smart large section handling:

    • Sub-chunks sections exceeding maxChunkSize (default: 8000 chars)
    • Preserves header context with "(continued)" markers
    • Tracks part numbers in metadata (part 1 of 3, etc.)
  • Integration with docs pipeline:

    • Automatic for .md files (falls back to character-based for .txt)
    • Can be disabled via useMarkdownChunking: false option
    • Backward compatible - gracefully falls back on errors
  • New exports from markdownChunking.ts:

    • Types: MarkdownSection, MarkdownChunkMetadata, MarkdownChunkOptions, MarkdownChunk
    • Constants: DEFAULT_MARKDOWN_CHUNK_OPTIONS
    • Parsing: parseMarkdownSections(), stripFrontmatter(), findCodeBlockRanges(), parseATXHeader(), parseSetextUnderline()
    • Chunking: chunkMarkdownContent(), chunkMarkdownFile(), shouldUseMarkdownChunking()
  • Benefits:

    • Improved search relevance for documentation queries
    • Better context preservation in search results
    • Section-aligned chunks for more coherent responses
    • Header hierarchy in metadata enables filtering/boosting by section

Testing

  • 60+ new unit tests for markdown chunking (tests/unit/engines/markdownChunking.test.ts)
  • Tests for ATX/setext header parsing, frontmatter handling, code block detection
  • Integration tests with chunkDocFile() and file I/O
  • Edge case coverage: nested headers, mixed formats, special characters, large sections

Incremental Reindexing (SMCP-098)

  • Surgical chunk-level updates - Only re-embed changed chunks instead of entire files

    • Before: Edit 1 line in 5000-line file = re-embed ~50 chunks = ~2.5 seconds
    • After: Edit 1 line = re-embed ~1-2 affected chunks = ~100ms (25x faster)
  • Position-independent chunk hashing (computeChunkHash()):

    • SHA256 hash of normalized chunk text (whitespace collapsed)
    • Detects unchanged chunks regardless of line number shifts
    • Backward compatible with legacy indexes (computes hash on-the-fly if missing)
  • Intelligent chunk diffing (diffChunks()):

    • Detects unchanged chunks (same hash + same position) - no action needed
    • Detects moved chunks (same hash, different position) - metadata update only
    • Detects added chunks (new hash) - embed and insert
    • Detects removed chunks (hash no longer present) - delete from store
  • Surgical database operations:

    • getChunksForFile() - Load existing chunks for a file with embeddings
    • deleteChunksByIds() - Remove specific chunks by ID (vs delete all by path)
    • updateChunkMetadata() - Update line numbers for moved chunks without re-embedding
  • Smart decision logic:

    • shouldUseIncremental() - Use incremental for files with 3+ existing chunks
    • Files with fewer chunks use full reindex (overhead not worth it)
    • wasIncrementalWorthwhile() - Post-hoc analysis (>=25% savings threshold)
  • Integration with updateFile():

    • Automatic incremental mode when beneficial
    • Detailed logging of incremental stats (added, removed, unchanged, moved, embeddingsSaved)
    • Falls back to full reindex for new files or small files
  • New exports from incrementalReindex.ts:

    • computeChunkHash(text) - Generate position-independent chunk hash
    • diffChunks(oldChunks, newChunks) - Compare chunks and categorize changes
    • shouldUseIncremental(oldChunkCount) - Decision function
    • wasIncrementalWorthwhile(diff) - Post-hoc analysis
    • createRecordsFromMovedChunks() - Create records reusing existing embeddings
    • createRecordsFromUnchangedChunks() - Create records preserving all data
    • createPartialRecordsFromNewChunks() - Create records needing embedding
  • Benefits:

    • 25x faster reindexing for small edits in large files
    • Reduced embedding API calls during file watching
    • Lower CPU/memory usage for incremental updates
    • Seamless backward compatibility with existing indexes

Testing

  • 35+ new unit tests for incremental reindexing (tests/unit/engines/incrementalReindex.test.ts)
  • Tests for hash computation, diff algorithm, record creation helpers
  • Edge case coverage: empty chunks, duplicate content, legacy data, boundary shifts

Multi-Language Code Chunking (SMCP-097)

  • Extended code-aware chunking to support 22+ programming languages (up from 3)

    • Previously supported: TypeScript, JavaScript, Python
    • Now supports 22 languages across 4 tiers with 40+ file extensions
  • Tier 1 - High Priority Languages (8 new):

    • Java (.java): class, interface, enum, record, annotations
    • Go (.go): func, type struct, type interface, const, var
    • Rust (.rs): fn, struct, enum, impl, trait, macro, mod
    • C# (.cs): class, struct, interface, enum, record, namespace, attributes
    • C (.c, .h): function definitions, struct, enum, union, preprocessor
    • C++ (.cpp, .hpp, .cc, .cxx, .hh, .hxx): class, namespace, templates, enum class
    • Kotlin (.kt, .kts): fun, class, interface, object, companion object, annotations
    • Swift (.swift): func, class, struct, enum, protocol, extension, attributes
  • Tier 2 - Medium Priority Languages (4 new):

    • Ruby (.rb, .rake, .gemspec): def, class, module
    • PHP (.php, .phtml): function, class, interface, trait, namespace
    • Scala (.scala, .sc): def, class, trait, object, type
    • Shell/Bash (.sh, .bash, .zsh, .fish): function declarations, case statements
  • Tier 3 - Markup/Config Languages (11 new):

    • CSS (.css): rule blocks, @media, @keyframes, @font-face
    • SCSS (.scss, .sass): mixins, functions, placeholders, variables
    • LESS (.less): mixins, variables
    • HTML (.html, .htm): script, style, semantic sections, template, form
    • Vue SFCs (.vue): template, script, style blocks
    • Svelte (.svelte): script, style, special blocks
    • SQL (.sql): CREATE, ALTER, SELECT, INSERT/UPDATE/DELETE, DROP
    • YAML (.yaml, .yml): top-level keys, document separators
    • JSON (.json, .jsonc): top-level object keys
    • XML (.xml, .xsl, .xslt): elements, processing instructions, comments, CDATA
    • GraphQL (.graphql, .gql): type, interface, input, enum, query, mutation, subscription, fragment
  • Tier 4 - Infrastructure Languages (3 new):

    • Terraform (.tf, .tfvars): resource, data, variable, output, module, provider, locals
    • HCL (.hcl): generic blocks, assignments
    • Dockerfile (Dockerfile, .dockerfile): FROM, RUN, COPY, ENV, ENTRYPOINT, CMD, WORKDIR, EXPOSE, ARG, LABEL
  • Special filename support:

    • Dockerfile, Makefile, Gemfile, Rakefile, Jenkinsfile, Vagrantfile
  • New exported functions:

    • getLanguageDisplayName(language) - Get human-readable language name
    • getSupportedLanguages() - Get list of all supported language identifiers
  • Benefits:

    • Chunks align with code structure across 22+ languages
    • Better semantic search results for polyglot codebases
    • Automatic language detection from 40+ file extensions
    • Falls back to character-based chunking for unsupported languages

Testing

  • 82 comprehensive unit tests covering all 22 supported languages
  • Tests for language detection, boundary detection, and chunk splitting
  • Tests for special filenames (Dockerfile, Makefile, etc.)
  • Tests for Windows and Unix path handling

Enhanced Setup Flow with Integrated Indexing (SMCP-088)

  • search-mcp setup now verifies project directory before proceeding

    • Shows detected project path at the start: "Detected project directory: /path/to/project"
    • Asks for confirmation: "Is this the correct project folder? [Y/n]"
    • If user says no, shows helpful instructions to navigate to the correct directory
    • Prevents accidental indexing of wrong directories
  • search-mcp setup now offers to index the project after client configuration

    • After configuring MCP clients, users are prompted: "Would you like to index this project now?"
    • If an existing index is found, shows stats (files, chunks, size, last updated) and asks: "Delete and recreate index?"
    • Progress bars and spinners show indexing progress through all phases (scanning, chunking, embedding, storing)
    • Complete one-command setup: configure clients AND index the project in a single flow
  • New setup flow:

    1. Detect and confirm project directory
    2. Configure MCP client(s)
    3. Prompt: "Would you like to index this project now? [Y/n]"
    4. If index exists: show stats and ask "Delete and recreate index? [y/N]"
    5. If user confirms: delete existing index, then create fresh index with progress display
    6. Show success summary with files indexed, chunks created, duration
  • Benefits:

    • True zero-config experience: npx @liraz-sbz/search-mcp setup does everything
    • Users no longer need to manually run search-mcp index after setup
    • Clear feedback on existing index state before recreating
    • Consistent progress display matching the standalone index command

Domain-Specific Embedding Prompts (SMCP-096)

  • Different prompts for document embedding vs query embedding - Follows BGE model best practices

    • Documents are embedded without prefix (optimized for passages)
    • Queries are embedded with instruction prefix: "Represent this sentence for searching relevant passages: "
    • This asymmetric approach improves retrieval quality as recommended by BGE model authors
  • New PromptType enum - Specifies the type of embedding:

    • 'document' - Used when indexing content (no prefix)
    • 'query' - Used when searching (adds instruction prefix)
  • Model-specific prompt configurations (MODEL_PROMPTS):

    • Xenova/bge-small-en-v1.5 (code embedding): document='', query='Represent this sentence for...'
    • Xenova/bge-base-en-v1.5 (docs embedding): document='', query='Represent this sentence for...'
  • New getPromptPrefix(modelName, promptType) helper function:

    • Returns the appropriate prefix for a given model and prompt type
    • Falls back to empty string for unknown models
  • Updated embedding methods with prompt type support:

    • embed(text, promptType?) - Single text embedding with optional prompt type
    • embedBatch(texts, onProgress?, promptType?) - Batch embedding with optional prompt type
    • embedBatchWithStats(texts, onProgress?, promptType?) - Batch with stats and prompt type
    • embedWithResults(texts, onProgress?, promptType?) - Batch with results and prompt type
    • Convenience functions embedText() and embedBatch() also updated
  • Integration with search tools:

    • search_code now uses 'query' prompt type for embedding search queries
    • search_docs now uses 'query' prompt type for embedding search queries
    • Indexing operations use 'document' prompt type by default (no code changes needed)
  • Benefits:

    • Improved retrieval relevance following sentence-transformer best practices
    • Compatible with current BGE models (no reindexing required)
    • No significant performance impact (just a string prefix)
    • Backward compatible (defaults to document prompt type)

Testing

  • 14 new unit tests for domain-specific prompts in tests/unit/engines/embedding.test.ts
  • Tests verify prefix application for both document and query prompt types
  • Tests cover getPromptPrefix() helper function and MODEL_PROMPTS constant

Query Expansion & Synonyms (SMCP-095)

  • New queryExpansion engine (src/engines/queryExpansion.ts) - Improves search recall by expanding abbreviations and synonyms

    • When users search for "auth", the query is expanded to include "authentication authorize login session token"
    • Inspired by mcp-vector-search's query expansion system with 60+ expansion mappings
    • Preserves original query terms while adding related terms for better recall
  • 60+ expansion mappings organized by category:

    • Authentication & Security: auth, login, oauth, jwt, password, permission, role
    • Database & Storage: db, sql, mongo, postgres, redis, cache, orm, prisma
    • API & HTTP: api, endpoint, route, http, rest, graphql, request, response
    • Async & Concurrency: async, await, promise, callback, concurrent, thread
    • Errors & Exceptions: err, error, exception, catch, throw, fail, handle
    • Configuration & Settings: config, env, settings, options, param
    • Common Abbreviations: util, fn, init, msg, req, res, ctx, src, dest, dir
    • Programming Concepts: class, interface, function, method, property, type
    • Testing: test, mock, spec, stub, spy, assert, expect
    • Logging & Debugging: log, logger, debug, trace, console
    • File & I/O: file, path, fs, read, write, save, load, parse
    • Networking: socket, tcp, connect, client, server, url
  • Core functions:

    • expandQuery(query, config?) - Expand a query with synonyms (returns string)
    • expandQueryWithDetails(query, config?) - Expand with detailed results (returns QueryExpansionResult)
    • hasExpansion(term) - Check if a term has expansion mappings
    • getExpansionTerms(term) - Get expansion terms for a specific term
    • getExpansionKeys() / getExpansionCount() - Query available expansions
  • Configuration options (QueryExpansionConfig):

    • enabled (boolean, default: true) - Enable/disable query expansion
    • maxExpansionTerms (number, default: 10) - Limit expansion terms added
    • customExpansions (Record<string, string>) - Add custom expansion mappings
  • Integration with search tools:

    • search_code now applies query expansion before generating embeddings
    • search_docs now applies query expansion before generating embeddings
    • Expanded query used for semantic search, original query preserved for FTS
    • Non-blocking with < 1ms overhead per query
  • Benefits:

    • Better recall for abbreviation-heavy queries (e.g., "auth" finds "authentication")
    • Domain-specific term expansion (e.g., "db" finds "database", "sql", "mongo")
    • Configurable and extensible with custom mappings
    • Zero impact on search latency (expansion is sub-millisecond)

Testing

  • 72 new unit tests for query expansion (tests/unit/engines/queryExpansion.test.ts)
  • Tests cover all expansion categories, configuration options, edge cases
  • Performance tests verify < 1ms expansion time
  • Integration tests for search usability

Search-Triggered Auto-Reindexing (SMCP-094)

  • New autoReindexer engine (src/engines/autoReindexer.ts) - Search-triggered automatic reindexing inspired by mcp-vector-search

    • Periodically checks for stale files during search operations
    • Silently reindexes small changes (<=5 files by default) without user intervention
    • No daemon process needed - reindexing happens opportunistically during search
  • SearchTriggeredIndexer class - Core implementation

    • preSearchHook() - Call before search to check and auto-reindex stale files
    • checkStaleness() - Compare current filesystem state with stored fingerprints
    • forceNextCheck() - Force staleness check on next search
    • getStats() - Get statistics about auto-reindex activity
    • updateConfig() - Dynamically update configuration
  • Configuration options (AutoReindexConfig):

    • enabled (boolean, default: true) - Enable/disable auto-reindexing
    • checkEveryNSearches (number, default: 10) - Check staleness every N searches
    • maxAutoReindexFiles (number, default: 5) - Maximum files to auto-reindex silently
    • stalenessThresholdMs (number, default: 300000) - Minimum time between checks
    • logActivity (boolean, default: true) - Log reindex activity
  • Global instance management:

    • getAutoReindexer(projectPath) - Get or create cached indexer for a project
    • createAutoReindexer(projectPath, config?) - Create new indexer instance
    • clearAutoReindexers() - Clear all cached indexers
    • removeAutoReindexer(projectPath) - Remove specific cached indexer
  • Integration with search tools:

    • search_code now automatically checks for stale files before searching
    • search_docs now automatically checks for stale files before searching
    • Auto-reindex is non-blocking for large changes (logs info and continues)
  • Benefits:

    • Index stays fresh without background processes
    • Small changes (editing a few files) are auto-reindexed silently
    • Large changes (new branch, many files) skip auto-reindex to avoid delays
    • Search performance unaffected (staleness check is fast)

Testing

  • 35+ new unit tests for auto-reindexer (tests/unit/engines/autoReindexer.test.ts)
  • Tests cover configuration, staleness detection, concurrent access, path normalization
  • Edge case coverage: disabled mode, zero thresholds, concurrent calls

LanceDB Vector Index Acceleration (SMCP-091)

  • IVF-PQ Vector Index Support - Automatic vector index creation for faster similarity search

    • Automatic index creation for datasets with >= 10,000 chunks
    • Configurable IVF-PQ parameters (numPartitions, numSubVectors, distanceType)
    • Adaptive parameter calculation based on dataset size
    • sqrt(numRows) partitions (clamped to 1-256)
    • dimension/16 or dimension/8 sub-vectors based on divisibility
  • New LanceDBStore methods:

    • createVectorIndex(config?) - Create IVF-PQ vector index with configurable parameters
    • getVectorIndexInfo() - Get information about existing vector index
  • Vector Index Configuration:

    • indexType - 'ivf_pq' or 'none' (default: auto based on chunk count)
    • numPartitions - Number of IVF partitions (default: sqrt(numRows))
    • numSubVectors - Number of PQ sub-vectors (default: dimension/16)
    • distanceType - 'l2', 'cosine', or 'dot' (default: 'l2')
    • maxIterations - Max kmeans iterations (default: 50)
    • sampleRate - Kmeans sample rate (default: 256)
  • Index information tracked in metadata:

    • hasIndex - Whether a vector index exists
    • indexType - Type of index ('ivf_pq' or 'none')
    • numPartitions, numSubVectors, distanceType - Index parameters
    • indexCreationTimeMs - Time to create the index
    • chunkCount - Chunks at time of index creation
    • createdAt - Index creation timestamp
  • get_index_status enhancement:

    • New vectorIndex field in status output showing index info
  • Automatic integration:

    • Vector index created automatically during create_index for large codebases
    • Graceful fallback to brute-force search if index creation fails
  • Note: GPU acceleration (CUDA/MPS) is NOT available in the LanceDB Node.js SDK. Index building runs on CPU only. When LanceDB adds GPU support to the Node.js SDK, we can enable it.

Symbol Extraction & Complexity Metrics (SMCP-090)

  • New symbolExtractor engine (src/engines/symbolExtractor.ts) - On-demand symbol extraction and complexity analysis

    • Reuses Tree-sitter infrastructure from SMCP-086
    • Fast extraction (< 100ms per typical file)
    • Extracts functions, classes, methods, interfaces, structs, traits, enums
    • Extracts imports and exports with full details
    • Calculates cyclomatic complexity per function
    • Calculates nesting depth per function
    • Provides overall complexity score (0-100)
  • New get_file_summary MCP tool - Retrieve file structure without reading entire files

    • Returns functions, classes, imports, exports with metadata
    • Returns complexity metrics for code quality assessment
    • Returns line counts (total, code, blank, comments)
    • Useful for AI assistants to understand code structure quickly
    • Does NOT require confirmation (read-only operation)
  • SymbolInfo metadata per symbol:

    • name - Symbol name
    • type - function, class, method, interface, struct, trait, enum
    • startLine / endLine - Line range
    • signature - Full function/method signature
    • docstring - Extracted documentation comment
    • isExported / isAsync / isStatic - Boolean flags
    • visibility - public / private / protected
    • paramCount - Parameter count for functions/methods
    • returnType - Return type if available
    • parentName - Parent class/struct name for methods
    • decorators - List of decorators/annotations
    • complexity - Cyclomatic complexity (if includeComplexity: true)
    • nestingDepth - Maximum nesting depth
  • ComplexityMetrics per file:

    • cyclomaticComplexity - Sum of all function complexities
    • maxNestingDepth - Maximum nesting in the file
    • avgFunctionComplexity - Average function complexity
    • decisionPoints - Count of if, while, for, &&, ||, etc.
    • overallScore - Overall score (0-100, higher = better/less complex)
  • Supported languages: JavaScript, TypeScript, TSX, Python, Go, Java, Rust, C, C++, C#
  • New API functions:

    • extractFileSummary(source, absolutePath, relativePath, options?) - Main extraction function
    • supportsSymbolExtraction(filePath) - Check if language is supported
    • getSupportedLanguages() - List supported languages

Testing

  • 60+ new unit tests for symbol extraction (tests/unit/engines/symbolExtractor.test.ts)
  • Tests cover all 10 supported languages
  • Complexity calculation tests
  • Performance tests (50 functions < 100ms, 100 functions < 200ms)
  • Edge case coverage: empty files, syntax errors, unicode, deep nesting

Merkle DAG Change Detection (SMCP-089)

  • New merkleTree engine (src/engines/merkleTree.ts) - Content-hash based change detection

    • Hierarchical Merkle tree structure: Project -> Directory -> File -> Chunk
    • Chunk-level granularity for partial reindexing (vs file-level fingerprints)
    • O(1) change detection via root hash comparison
    • Efficient diff algorithm to identify only changed nodes
  • MerkleTreeManager class - State management for incremental indexing

    • Content-hash based file tracking with chunk boundaries
    • Snapshot persistence for fast startup (merkle-tree.json)
    • Support for detecting moved/renamed chunks via content hash
    • Snapshot creation for safe rollback on errors
  • Hash computation functions

    • computeChunkHash() - Position-aware hash for chunk tracking
    • computeChunkContentHash() - Position-independent hash for detecting moves
    • computeFileHash() - File hash from ordered chunk hashes
    • computeProjectHash() - Root hash from all file hashes
  • Diff operations

    • diffFileMaps() - Compare two tree states to get changes
    • MerkleDiff - Categorized changes (added/modified/removed files, chunk-level changes)
    • getChangedFiles() - Quick list of changed files without full diff
  • Benefits over file-level fingerprints

    • Changing one function in a large file can identify affected chunks only
    • Potential for 50%+ reindex time reduction for small changes
    • Foundation for future partial reindexing optimization
  • Comprehensive test coverage

    • 39 unit tests for hash computation, diff algorithm, and MerkleTreeManager
    • 13 integration tests for project indexing, change detection, and persistence
    • Performance tests demonstrating O(1) root hash comparison

Zero-Config CLI Interface (SMCP-088)

  • New CLI commands - Direct command-line access without MCP client setup

    • search-mcp index - Create or update search index for current project
    • search-mcp search <query> - Search code with natural language queries
    • search-mcp status - Show index statistics and configuration
    • search-mcp reindex - Rebuild entire index from scratch
    • search-mcp setup - Configure MCP clients (existing functionality)
    • search-mcp logs - Show log file locations (existing functionality)
  • Rich terminal output - Beautiful, informative CLI experience

    • Progress bars for indexing (chunking, embedding phases)
    • Colored output: cyan for headers, green for success, yellow for warnings, red for errors
    • Spinners for operations in progress
    • Formatted search results with file paths, line numbers, and scores
    • Code snippet preview with truncation for long results
  • Search command options

    • -k, --top-k <number> - Number of results to return (default: 10)
    • -m, --mode <mode> - Search mode: hybrid, vector, or fts
    • -a, --alpha <number> - Alpha weight for hybrid search (0-1)
    • -d, --docs - Search documentation files instead of code
    • --json - Output results as JSON for scripting
  • JSON output mode - All commands support --json flag for scripting

    • Machine-readable output for CI/CD integration
    • Structured error reporting
    • Compatible with jq and other JSON tools

Dependencies

  • Added commander (^12.x) - CLI framework
  • Added chalk (^5.x) - Terminal colors
  • Added ora (^8.x) - Terminal spinners
  • Added cli-progress (^3.x) - Progress bars

Usage Examples

# Create index for current project
npx @liraz-sbz/search-mcp index

# Search for code
npx @liraz-sbz/search-mcp search "authentication function"

# Search with options
npx @liraz-sbz/search-mcp search "error handling" --top-k 5 --mode hybrid

# Search docs
npx @liraz-sbz/search-mcp search "setup instructions" --docs

# Get index status
npx @liraz-sbz/search-mcp status

# JSON output for scripting
npx @liraz-sbz/search-mcp status --json | jq '.totalFiles'

# Rebuild index
npx @liraz-sbz/search-mcp reindex

AST-Based Chunking (SMCP-086)

  • New treeSitterParser engine (src/engines/treeSitterParser.ts) - WASM-based AST parsing using Tree-sitter

    • Cross-platform support via web-tree-sitter (WASM)
    • Pre-built grammars via tree-sitter-wasms for 10+ languages
    • Lazy loading of language grammars for minimal startup time
    • Singleton pattern with TreeSitterParser.getInstance()
    • Language detection from file extensions
  • New astChunking engine (src/engines/astChunking.ts) - Semantic code chunking with rich metadata

    • Supported languages: JavaScript, TypeScript, TSX, Python, Go, Java, Rust, C, C++, C#
    • Extracts semantic units: functions, classes, methods, interfaces, structs, traits, impls
    • Rich metadata per chunk: name, signature, docstring, decorators, parent info
    • Semantic tags for search boosting (async, export, static, etc.)
    • Configurable chunk size with automatic splitting of large functions
  • New chunking strategy: 'ast' - Third strategy alongside 'character' and 'code-aware'

    • Best accuracy for supported languages
    • Falls back to code-aware, then character-based
    • Use via chunkFileWithStrategy(path, relativePath, { strategy: 'ast' })
  • LanceDB schema updates - Optional metadata fields for AST chunks

    • chunk_type, chunk_name, chunk_signature, chunk_docstring
    • chunk_parent, chunk_tags, chunk_language
    • SearchResult now includes metadata when available

ChunkMetadata Interface

interface ChunkMetadata {
  type: ChunkType;           // function, class, method, interface, etc.
  name?: string;             // Function/class/method name
  signature?: string;        // Full signature
  docstring?: string;        // Extracted docstring
  decorators?: string[];     // Decorators/annotations
  parentName?: string;       // Parent name (class for methods)
  parentType?: ChunkType;    // Parent type
  tags?: string[];           // Semantic tags
  language: ASTLanguage;     // Programming language
  isAsync?: boolean;         // Async function
  isExport?: boolean;        // Exported
  isStatic?: boolean;        // Static method
  visibility?: 'public' | 'private' | 'protected';
  paramCount?: number;       // Parameter count
  returnType?: string;       // Return type
  genericParams?: string[];  // Generic parameters
}

Dependencies

  • Added web-tree-sitter (^0.26.3) - WASM-based Tree-sitter parser
  • Added tree-sitter-wasms (^0.1.13) - Pre-built WASM grammars

Testing

  • 43 new unit tests for AST chunking (tests/unit/engines/astChunking.test.ts)
  • Tests cover all 10 supported languages
  • Metadata extraction tests for each language
  • Graceful degradation tests when parser unavailable

Query Intent Detection (SMCP-085)

  • New queryIntent engine (src/engines/queryIntent.ts) - Classifies search queries into intent categories for better search ranking
    • Detects 8 intent categories: FUNCTION, CLASS, ERROR, DATABASE, API, AUTH, TEST, CONFIG
    • Multi-intent support - queries can match multiple categories with confidence scores
    • Fast keyword-based detection with < 10ms latency overhead
    • CamelCase and snake_case aware tokenization via normalizeToTokens()
    • Entity-like query detection via isEntityLikeQuery()

Intent-Based Search Boosting

  • Chunk type boosting - getChunkTypeBoosts() returns boost factors based on query intent
    • CLASS intent boosts class chunks (1.3x)
    • FUNCTION intent boosts function/method chunks (1.15x)
    • TEST intent boosts function/method chunks (1.2x)
  • Tag-based boosting - getIntentTagBoost() boosts results with matching tags
  • Name matching - CamelCase-aware name matching with up to 1.4x boost for exact matches
  • Integration with hybrid search - applyIntentBoosts() function in hybridSearch.ts

New API Functions

  • detectQueryIntent(query, config?) - Main intent detection function
  • getChunkTypeBoosts(intent) - Get boost factors for chunk types
  • getIntentTagBoost(intent, tags) - Get boost factor for tag overlap
  • createIntentDetector(config) - Create a pre-configured detector
  • getIntentNames(intent) - Extract intent category names
  • hasIntent(intent, category, minConfidence?) - Check for specific intent

Testing

  • 86 new unit tests for query intent detection
  • Tests cover all 8 intent categories
  • Performance tests verify < 10ms latency
  • Edge case coverage: empty queries, special characters, long queries

Multi-Factor Search Ranking (SMCP-087)

  • New advancedRanking engine (src/engines/advancedRanking.ts) - Sophisticated multi-factor ranking algorithm inspired by claude-context-local
    • Combines 7+ ranking signals for significantly better search result quality:
      1. Base similarity score (from vector/hybrid search)
      2. Query intent detection (via SMCP-085)
      3. Chunk type boosting (dynamic based on intent)
      4. Name matching with CamelCase/snake_case awareness
      5. Path/filename relevance
      6. Docstring/comment presence bonus
      7. Complexity penalty for oversized chunks

Core Ranking Features

  • Chunk type boosting - Dynamic boosts based on detected query intent
    • CLASS intent: 1.3x boost for class chunks
    • FUNCTION intent: 1.15x boost for function/method chunks
    • TEST intent: 1.2x boost for function/method chunks
  • Name matching - CamelCase/snake_case-aware tokenization
    • Exact match: 1.4x boost
    • 80%+ token overlap: 1.3x (strong match)
    • 50%+ token overlap: 1.2x (good match)
    • 30%+ token overlap: 1.1x (partial match)
    • Any overlap: 1.05x (weak match)
  • Path relevance - Boosts results when query tokens appear in file path
    • 5% boost per matching token, capped at 20% total
  • Docstring bonus - 1.05x boost for documented code
    • Reduced bonus for module docstrings on entity-like queries
  • Complexity penalty - Penalizes oversized chunks
    • 2% penalty for chunks > 2000 chars
    • 5% penalty for chunks > 4000 chars

New API Functions

  • applyAdvancedRanking(query, results, config?) - Main ranking function with full factor breakdown
  • calculateChunkTypeBoost(chunkType, boosts) - Calculate type-based boost
  • calculateNameBoost(name, query, tokens) - Calculate name matching boost
  • calculatePathBoost(path, tokens) - Calculate path relevance boost
  • calculateDocstringBonus(docstring, type, isEntity) - Calculate documentation bonus
  • calculateComplexityPenalty(text, thresholds) - Calculate size penalty
  • createRanker(config) - Create pre-configured ranking function
  • getRankingStats(results) - Analyze ranking factor statistics

Configuration

interface AdvancedRankingConfig {
  enabled: boolean;              // Enable/disable ranking (default: true)
  intentConfig?: IntentConfig;   // Intent detection settings
  weights?: {                    // Factor weight multipliers
    chunkType: number;           // Default: 1.0
    name: number;
    path: number;
    tag: number;
    docstring: number;
    complexity: number;
  };
  complexityThresholds?: {
    mild: number;                // Default: 2000 chars
    strong: number;              // Default: 4000 chars
  };
  docstringBonusValue?: number;  // Default: 1.05
}
  • applyAdvancedSearchRanking() - Apply ranking to HybridSearchResult[]
  • convertRankedToHybridResults() - Convert back to hybrid results with updated scores

Testing

  • 73 new unit tests for advanced ranking (tests/unit/engines/advancedRanking.test.ts)
  • Tests cover all ranking factors
  • Performance tests: < 50ms for 100 results, < 200ms for 500 results
  • Edge cases: empty queries, unicode, very long queries

Fixed

  • Tree-sitter WASM files not found when running via npx - Added support for hoisted npm dependencies
    • When npm installs dependencies via npx, it hoists them to a parent node_modules folder
    • Added additional search paths to find web-tree-sitter and tree-sitter-wasms in hoisted locations
    • This fixes "Could not find web-tree-sitter WASM file" errors during code comment extraction

[1.4.0] - 2025-12-16

Added

GPU Acceleration (Windows)

  • Automatic GPU acceleration via DirectML - Zero-configuration GPU support on Windows
    • Works with NVIDIA, AMD, and Intel GPUs (including integrated graphics)
    • DirectML is built into onnxruntime-node - no additional dependencies needed
    • Graceful fallback to CPU if GPU initialization fails

Device Detection (src/engines/deviceDetection.ts)

  • New module for detecting and managing compute devices
  • Functions: detectBestDevice(), isDirectMLAvailable(), supportsGPU(), formatDeviceInfo()
  • Platform detection: isWindows(), isMacOS(), isLinux(), isNodeEnvironment()
  • Caching with getCachedDeviceInfo() and clearDeviceCache()
  • 5-second timeout prevents hangs on unresponsive GPU drivers

Embedding Engine GPU Support

  • New device option in EmbeddingEngineConfig: 'webgpu' | 'dml' | 'cpu' or auto-detect
  • GPU batch size: 64 (vs 32 for CPU) for better throughput
  • New methods: getDeviceInfo(), getDevice(), isUsingGPU(), didFallbackToCPU()
  • Performance logging: chunks/sec, device info, total time

Status Reporting

  • get_index_status now includes compute field with device info
  • create_index summary shows compute device and throughput (chunks/sec)
  • New ComputeStatus interface exported from tools

Changed

  • Transformers.js v3 Migration - Upgraded from @xenova/transformers v2 to @huggingface/transformers v3
    • Actively maintained by Hugging Face (v2 was 2+ years old)
    • Foundation for GPU acceleration support
    • Existing models and indexes remain fully compatible
  • Device detection priority:
    • Windows Node.js: DirectML > CPU
    • macOS/Linux Node.js: CPU only (CoreML/CUDA require separate packages)
    • Browser: WebGPU > CPU

Platform Support

Platform GPU Support Notes
Windows DirectML Automatic - NVIDIA, AMD, Intel GPUs
macOS CPU only CoreML not available in Node.js
Linux CPU only CUDA requires separate package

Testing

  • 2831 tests passing (123+ unit tests for GPU features)
  • 40 new integration tests for WebGPU/DirectML pipeline
  • Platform compatibility matrix tests
  • Search quality validation: MCP 2.5x better than grep

[1.3.20] - 2025-12-16

Added

  • Human-Readable Index Summary - create_index now returns a formatted summary field with:

    Index created successfully for /path/to/project
    
    Statistics:
      Total files in project: 20,085
      Files excluded: 19,830
      Code files indexed: 255
      Doc files indexed: 103
      Total chunks created: 1,196
      Duration: 3m 19s
    • Numbers are formatted with thousands separators for readability
    • Warning is appended if docs indexing had issues

Fixed

  • Docs Indexing Diagnostics - Added better diagnostics when documentation files aren't indexed
    • scanDocFiles() now returns both the file list and the raw glob count for debugging
    • When 0 docs are indexed but glob found files, a warning is now returned explaining that files were filtered out
    • Warning suggests checking gitignore, exclude patterns, or file size limits
    • create_index output now includes docsWarning field when docs indexing has issues
    • Added info-level logging for raw glob results before and after filtering

Changed

  • scanDocFiles() now returns a ScanDocFilesResult object with files and globFilesFound properties
  • DocsIndexResult interface now includes optional warning and globFilesFound fields

[1.3.19] - 2025-12-14

Added

  • Detailed Index Statistics - After indexing completes, you now see comprehensive stats:
    • Total files found in project (before filtering)
    • Files excluded by policy (gitignore, deny patterns, etc.)
    • Code files indexed
    • Doc files indexed (if docs indexing is enabled)
    • Total files indexed
    • Duration

Changed

  • create_index tool now indexes both code AND docs in a single operation
    • Previously docs were only indexed lazily when first searched
    • Now both are indexed upfront for faster first searches
  • scanFiles now returns a ScanResult object with detailed stats instead of just a file list

[1.3.18] - 2025-12-14

Added

  • Adaptive Streaming Mode for High Memory - When memory usage exceeds 80%, indexing automatically switches to streaming mode
    • Processes only 3 files at a time (instead of 50) to minimize memory accumulation
    • Writes to database immediately after each small batch
    • Runs garbage collection between batches
    • Normal batch processing when memory is below 80% (fast mode)
    • Applied to both code and docs indexing

[1.3.16] - 2025-12-14

Fixed

  • Memory Management During Indexing - Fixed indexing failing with 0 files due to overly aggressive memory checks
    • Raised memory critical threshold from 85% to 90% to allow headroom after embedding model loads
    • Process at least 1 file per batch before checking memory (ensures progress is made)
    • Request garbage collection before each batch to free memory from previous operations
    • Fixes "0 files indexed" issue when memory was high after embedding model initialization

[1.3.15] - 2025-12-14

Fixed

  • Windows MCP Connection Failures - Reverted cmd /c wrapper which was breaking MCP connections
    • The /doctor warning about needing cmd /c wrapper was misleading
    • Working MCP configs use npx directly without cmd /c wrapper
    • Removed cmd /c wrapper from setup wizard and help text
    • Kept path normalization fix for .claude.json forward/backslash entries

[1.3.13] - 2025-12-14

Fixed

  • Windows MCP Config - Setup wizard added cmd /c wrapper for npx commands
    • ⚠️ Reverted in v1.3.14 - This change actually broke MCP connections
    • The /doctor warning was misleading; npx works directly without wrapper

[1.3.12] - 2025-12-14

Fixed

  • Framework Cache Crashes - Fixed indexing failures on projects with large framework caches
    • Root cause: .angular/cache and similar directories contain large binary files causing crashes
    • Added comprehensive hardcoded deny patterns for 30+ framework/tool caches:
      • Package managers: .yarn/**, .pnpm-store/**, Pods/**, .bundle/**, deps/**
      • Build artifacts: bin/**, obj/**, _build/**, .build/**, .output/**, .svelte-kit/**, .astro/**, .gradle/**, .mvn/**, .expo/**, .docusaurus/**, .storybook-static/**
      • Linter caches: .mypy_cache/**, .ruff_cache/**, .eslintcache, .stylelintcache
      • Testing: .hypothesis/**, .tox/**, htmlcov/**
      • Cloud/deploy: .terraform/**, .serverless/**, .vercel/**, .netlify/**
      • IDE: .fleet/**, *.sublime-workspace
    • This fixes "MCP connection closed" errors on Angular, .NET, Elixir, Swift, and other projects

[1.3.11] - 2025-12-14

Fixed

  • Crash Logging - Server crashes are now logged even during startup
    • Added early crash logging before module imports
    • Added global handlers for uncaughtException and unhandledRejection
    • Crashes are now written to ~/.mcp/search/logs/server.log

[1.3.10] - 2025-12-14

Added

  • Debugging Support - Better logging and error messages for troubleshooting
    • New --logs CLI command to show log file locations for each indexed project
    • Global server log at ~/.mcp/search/logs/server.log for server start/stop events
    • Error messages now include log file path for easy debugging
    • Full stack traces logged for all errors
    • get_index_status tool now returns logPath field
    • Support for DEBUG=1 or SEARCH_MCP_DEBUG=1 environment variables for verbose logging
  • Progress Reporting - Real-time indexing progress visibility
    • Progress logged to file during create_index and reindex_project operations
    • Progress written to stderr (visible when running MCP server directly)
    • Shows phase (scanning/chunking/embedding/storing), file count, and current file

Changed

  • Updated README troubleshooting section with new CLI options and debug mode info
  • Improved --logs output to show only 10 most recent indexes (prioritizing those with log files)

[1.3.8] - 2025-12-14

Changed

  • Improved first-use instructions to be clearer and more explicit
    • Added step to verify MCP connection with /mcp command
    • Changed prompt to "Use search-mcp to create an index for this project" for better AI recognition
  • Updated README, setup wizard, CLI help, and postinstall message

[1.3.7] - 2025-12-14

Changed

  • Updated all setup commands to use npx --yes @liraz-sbz/search-mcp@latest --setup to avoid npx caching issues
  • Updated README, CLI help, and postinstall message with correct command format

[1.3.6] - 2025-12-14

Added

  • Setup Wizard - New --setup command to easily configure MCP clients

    • Auto-detects installed MCP clients (Claude Desktop, Claude Code, Cursor, Windsurf)
    • Interactive menu to select which clients to configure
    • Creates or updates .mcp.json configuration files automatically
    • Supports Claude Code CLI integration when available
    • Usage: npx @liraz-sbz/search-mcp --setup
  • Post-install Instructions - Helpful setup message after npm install

    • Shows quick setup options immediately after installation
    • Guides users to configure their MCP clients
    • No more confusion about manual .mcp.json creation
  • CLI Help & Version - Standard CLI flags

    • --help / -h - Show usage information
    • --version / -v - Show installed version

Changed

  • Entry point now handles CLI arguments before starting the MCP server
  • Improved first-time user experience with clear setup paths

[1.3.5] - 2025-12-13

Changed

  • Documentation Updates - Updated README and benchmark docs with full codebase test results
    • Updated headline efficiency claim from "20x" to "40x" based on full codebase benchmarks
    • Updated performance table: "40-43x fewer tokens" (was "20.4x faster")
    • Added install command (npm i) alongside run command in README
    • Updated search-comparison-test.md with full codebase efficiency numbers
    • Added link to full codebase analysis report

[1.3.4] - 2025-12-13

Changed

  • Default Configuration Optimization - Updated defaults based on full codebase benchmark testing
    • Changed hybridSearch.defaultAlpha from 0.7 to 0.5 (balanced hybrid)
      • Achieves 43x token efficiency vs grep (up from 40.9x)
      • Same precision (32% P@5), better balance of semantic + keyword search
    • Changed chunkingStrategy from "character" to "code-aware"
      • Respects code boundaries (functions, classes) for better semantic chunks
      • Slightly better search latency in benchmarks
    • Updated documentation to reflect new defaults

Performance

  • Full codebase testing shows new defaults achieve:
    • 43x fewer tokens than grep (vs 40.9x with old defaults)
    • Same search precision (32% Precision@5)
    • Better semantic understanding with balanced alpha

[1.3.3] - 2025-12-12

Fixed

  • MEDIUM: Synchronous Operations in getStorageRoot (BUG #10, SMCP-078)

    • Added caching to avoid repeated sync filesystem operations
    • Added async version getStorageRootAsync() for non-blocking operations
    • Added clearStorageRootCache() utility for testing
  • MEDIUM: Synchronous fs.existsSync in loadFingerprints (BUG #11, SMCP-078)

    • Replaced fs.existsSync with async fs.promises.access in fingerprints loader
  • MEDIUM: SQL IN Clause Construction (BUG #13, SMCP-078)

    • Added UUID format validation for chunk IDs before SQL construction
    • Invalid IDs are logged and skipped for defense in depth
  • LOW: escapeSqlString Security Hardening (BUG #15, SMCP-078)

    • Added removal of semicolons (statement terminator)
    • Added removal of SQL comment sequences (-- and /* */)
    • Defense in depth for SQL escaping
  • LOW: Timer Leak in IntegrityScheduler (BUG #16, SMCP-078)

    • Added explicit this.timer !== null check to prevent timer leaks
  • LOW: Missing top_k Upper Bound Validation (BUG #23, SMCP-078)

    • Added upper/lower bound validation in LanceDB search (clamps to 1-100)
    • Prevents resource exhaustion from arbitrary top_k values
  • LOW: ReadWriteLock Starvation Documentation (BUG #18, SMCP-078)

    • Added comprehensive documentation about starvation potential
    • Documented why it's acceptable and future fairness options
  • LOW: Chunking Line Calculation Edge Case (BUG #20, SMCP-078)

    • Added documentation explaining the edge case handling
    • Clarified that the guard is correct behavior

[1.3.2] - 2025-12-12

Fixed

  • MEDIUM: Stream Resource Leaks in Large File Chunking (BUG #5, SMCP-076)

    • Attached error handlers immediately after stream creation to eliminate race window
    • Added rejected flag and rejectOnce helper to prevent double rejection
    • Added cleanup() function called from all exit paths
    • Consolidated duplicate error handlers for cleaner code
  • MEDIUM: Partial Initialization State in Embedding Engine (BUG #9, SMCP-076)

    • Wrapped initialization in inner async IIFE for atomic state handling
    • Used finally block to clear initializationPromise only if pipeline not set
    • Ensures retry works correctly after any initialization failure
  • MEDIUM: Unhandled Promise Rejection in Background Startup Check (BUG #21, SMCP-076)

    • Used Promise.resolve().then() pattern to catch both synchronous and async errors
    • All errors now properly logged instead of causing unhandled rejections
    • Added 2 tests for error handling scenarios
  • MEDIUM: Missing Error Handling for Config Load Failure (BUG #26, SMCP-076)

    • Added try-catch around config loading in create_index tool handler
    • Falls back to DEFAULT_CONFIG on any config load error
    • Logs warning about config load failure for debugging
  • MEDIUM: FTS Index Uses Non-Atomic Writes (BUG #25, SMCP-077)

    • FTS index now uses atomicWrite utility (temp-file-then-rename pattern)
    • Prevents FTS index corruption if process crashes during write
    • Updated both indexManager.ts and reindexFile.ts
  • MEDIUM: Project Path Cache Without Invalidation (BUG #22, SMCP-077)

    • Added validation that cached project path still exists before returning
    • Uses fs.promises.access() to verify path accessibility
    • Re-detects project root if cached path is deleted/moved during long-running session
    • Added unit test for cache invalidation scenario
  • MEDIUM: Metadata Staleness During Concurrent Indexing (BUG #24, SMCP-077)

    • Enhanced documentation for existing indexing state check in search operations
    • Search operations already warn users when indexing is in progress
    • Clarified that warning approach is the implemented solution for concurrent access
  • MEDIUM: TOCTOU in Stale Lockfile Cleanup (BUG #8, SMCP-077)

    • Added comprehensive documentation of the inherent TOCTOU limitation
    • Documents why it's acceptable (single MCP server per project, small race window)
    • Lists platform-specific alternatives if multi-process safety becomes critical

[1.3.1] - 2025-12-12

Fixed

  • HIGH: Glob Timeout Resource Exhaustion (BUG #4, SMCP-075)

    • Replaced Promise.race timeout pattern with AbortController for glob operations
    • Glob operations now properly cancel when timeout fires, preventing resource exhaustion
    • Timeout is properly cleared in finally block to prevent memory leaks
    • Added 3 new tests for glob timeout cancellation
  • HIGH: AsyncMutex Timeout/Grant Race Condition (BUG #6, SMCP-075)

    • Added atomic satisfied flag to prevent race between timeout and lock grant
    • resolveWrapper now returns boolean to indicate if lock was accepted
    • Updated release() to skip timed-out waiters and properly unlock mutex
    • Prevents potential deadlock in high-contention scenarios
    • Added 6 new stress tests for high contention and race condition handling

[1.3.0] - 2025-12-12

Added

  • Migration Detection and Model Compatibility (SMCP-074)
    • All MCP tools now use the correct embedding engines (code vs docs)
    • search_code uses getCodeEmbeddingEngine() (BGE-small, 384 dims)
    • search_docs uses getDocsEmbeddingEngine() (BGE-base, 768 dims)
    • Index creation and reindexing now save model metadata to metadata.json
    • Migration detection warns when searching an index created with a different model
    • Searching with mismatched models triggers clear error suggesting reindex_project
    • get_index_status now shows embedding model information and mismatch warnings
    • New utility: checkModelCompatibility() in src/utils/modelCompatibility.ts
    • New utility functions: checkCodeModelCompatibility(), checkDocsModelCompatibility()
    • New utility: buildStatusWarning() for non-blocking status warnings
    • createMetadata() now includes embedding model info by default
    • Breaking for legacy indexes: Search operations on legacy indexes (created before SMCP-072) will fail with a helpful error message suggesting reindex_project
  • Dual Embedding Model Support (SMCP-072)
    • Refactored embedding engine to support two separate models
    • Code search: Xenova/bge-small-en-v1.5 (384 dimensions)
    • Docs search: Xenova/bge-base-en-v1.5 (768 dimensions)
    • New exports: CODE_MODEL_NAME, CODE_EMBEDDING_DIMENSION, DOCS_MODEL_NAME, DOCS_EMBEDDING_DIMENSION
    • New exports: CODE_ENGINE_CONFIG, DOCS_ENGINE_CONFIG, EmbeddingEngineConfig
    • New functions: getCodeEmbeddingEngine(), getDocsEmbeddingEngine()
    • New reset functions: resetCodeEmbeddingEngine(), resetDocsEmbeddingEngine()
    • EmbeddingEngine now accepts config parameter for model customization
    • New methods: getModelName(), getDisplayName() on EmbeddingEngine class
    • Backward compatible: getEmbeddingEngine(), MODEL_NAME, EMBEDDING_DIMENSION still work
  • Configurable Vector Dimensions in Storage (SMCP-073)
    • LanceDBStore now accepts vectorDimension parameter (defaults to 384)
    • DocsLanceDBStore now accepts configurable dimension (defaults to 384 for backward compat)
    • New exports: CODE_VECTOR_DIMENSION (384), DOCS_VECTOR_DIMENSION (768)
    • Added EmbeddingModelInfoSchema to metadata for tracking model names and dimensions
    • New MetadataManager methods: updateEmbeddingModelInfo(), getCodeModelName(), getDocsModelName(), etc.
    • Backward compatible: existing indexes without embeddingModels field still work
  • Config Matrix npm Scripts (SMCP-071)
    • npm run test:configs - Run config matrix and accuracy comparison tests
    • npm run test:configs:watch - Watch mode for config tests
    • npm run test:configs:full - Run tests against full codebase (FULL_CODEBASE=true)
    • Added cross-env dependency for Windows compatibility
  • Accuracy Comparison Tests (tests/configs/accuracyComparison.test.ts) (SMCP-070)
    • Compares MCP search efficiency against Grep and Drag-and-Drop baselines
    • Tests 5 representative configs (default, alpha-0.0, alpha-0.5, alpha-1.0, fts-js)
    • Calculates MCP vs Grep efficiency ratio
    • Calculates MCP vs Drag-and-Drop efficiency ratio
    • Tracks deduplication effectiveness per config
    • Generates comparison reports to tests/reports/accuracy-comparison-*.md
    • FULL_CONFIG and FULL_CODEBASE environment variables for extended testing
  • Config Matrix Test Runner (tests/configs/configMatrix.test.ts) (SMCP-069)
    • Systematically tests all 21 configuration combinations
    • Runs 20 test queries against each config
    • Collects quality metrics (precision@5, relevance hits)
    • Collects performance metrics (latency, memory, tokens)
    • Generates JSON and Markdown reports to tests/reports/
    • FULL_CODEBASE environment variable for testing against actual codebase

[1.2.0] - 2025-12-11

Added

  • Hybrid Search - Combines semantic (vector) and keyword (FTS) search for better results
    • New mode parameter for search_code and search_docs: 'hybrid' (default), 'vector', 'fts'
    • New alpha parameter (0-1) to control semantic vs keyword weight
    • Reciprocal Rank Fusion (RRF) algorithm for intelligent result merging
    • Backward compatible: existing indexes without FTS fall back to vector-only search
  • FTS Engine Interface (src/engines/ftsEngine.ts) (SMCP-058)
    • Unified interface for full-text search engines
    • Support for multiple implementations (JS and native)
    • Error types: FTSNotInitializedError, FTSQueryError, FTSSerializationError
  • NaturalBM25Engine (src/engines/naturalBM25.ts) - Pure JavaScript FTS implementation
    • Uses natural npm package for TF-IDF/BM25 based text search
    • No native dependencies - works on all platforms
    • Supports add/remove/search operations
    • Score normalization for hybrid search (0-1 range)
    • Serialization/deserialization for index persistence
  • FTS Engine Factory (src/engines/ftsEngineFactory.ts) (SMCP-060)
    • Auto-detection of best FTS engine based on codebase size
    • Threshold: 5000 files triggers native engine selection when available
    • User preference override via hybridSearch.ftsEngine config option
    • Graceful fallback from native to JS when better-sqlite3 unavailable
  • Hybrid Search Configuration - New hybridSearch config section
    • enabled (boolean, default: true) - Enable/disable hybrid search
    • ftsEngine ('auto' | 'js' | 'native', default: 'auto') - FTS engine preference
    • defaultAlpha (0-1, default: 0.7) - Semantic vs keyword weight
  • Compact output format for search_code and search_docs tools (SMCP-065)
    • New compact parameter (default: false) returns results with shorter field names
    • Reduces token count by ~5% through shorter field names
  • Code-aware chunking module (src/engines/codeAwareChunking.ts) (SMCP-066)
    • Heuristic-based chunking that splits at semantic boundaries
    • Supports TypeScript, JavaScript, and Python
    • New config option: chunkingStrategy: 'character' | 'code-aware' (default: 'character')
  • get_index_status now shows hybrid search info (FTS engine type, chunk count, default alpha)
  • reindex_file now updates both vector and FTS indexes incrementally
  • Comprehensive hybrid search integration tests (48 tests)

Changed

  • search_code and search_docs now apply whitespace trimming and same-file deduplication
  • Reduced token usage by ~7-8% through automatic result optimization
  • Default search mode is now hybrid (combines vector + keyword search)

Documentation

  • Updated API reference with new mode and alpha parameters
  • Added Hybrid Search section to configuration docs
  • Added Hybrid Search examples to examples docs
  • Updated ROADMAP to mark Hybrid Search as completed

1.1.5 - 2024-12-11

Added

  • New get_config tool to retrieve configuration file path and contents
  • indexPath and configPath fields added to get_index_status output

1.1.4 - 2024-12-11

Added

  • CHANGELOG.md - Version history
  • CONTRIBUTING.md - Contributor guide
  • ROADMAP.md - Planned features
  • docs/api-reference.md - Complete API documentation
  • README badges (npm version, license, node version)
  • Links to all new docs in README Documentation section

1.1.3 - 2024-12-11

Fixed

  • Fixed create_index, reindex_project, and delete_index tools returning "cancelled" status
  • Fixed claude mcp add command syntax in documentation

Changed

  • Removed -y flag from npx commands in documentation (caused errors with Claude Code)

Added

  • "Updating & Uninstalling" section in README
  • claude mcp remove search uninstall instructions

1.1.2 - 2024-12-11

Changed

  • Added Windows paths alongside macOS/Linux paths in all documentation
  • Improved documentation for cross-platform users

1.1.1 - 2024-12-11

Changed

  • Updated package name references from @blakazulu/search-mcp to @liraz-sbz/search-mcp

1.1.0 - 2024-12-11

Changed

  • Migrated from deprecated vectordb package to @lancedb/lancedb
  • Simplified lazy indexing strategy to true lazy loading (index only on first search)

Fixed

  • Various security hardening improvements

1.0.0 - 2024-12-10

Added

  • Initial release
  • create_index - Create search index for a project
  • search_code - Semantic search for code
  • search_docs - Semantic search for documentation (.md, .txt files)
  • search_by_path - Find files by glob pattern
  • get_index_status - Show index statistics
  • reindex_project - Rebuild entire index
  • reindex_file - Re-index a single file
  • delete_index - Remove project index
  • Real-time file watching with automatic index updates
  • Three indexing strategies: realtime, lazy, git
  • Local embedding model (Xenova/all-MiniLM-L6-v2)
  • LanceDB vector storage
  • Hardcoded security deny list for sensitive files
  • Path traversal and symlink attack protection
  • Resource exhaustion protection

Security

  • Path security hardening with symlink protection
  • Input validation on all tool parameters
  • Confirmation bypass prevention
  • Zero vector insertion prevention
  • File filtering security hardening
  • Concurrency and data integrity protections