> ## Documentation Index > Fetch the complete documentation index at: https://docs.mcphub.app/llms.txt > Use this file to discover all available pages before exploring further. # Smart Routing > AI-powered tool discovery using vector semantic search ## Overview Smart Routing is MCPHub's intelligent tool discovery system that uses vector semantic search to automatically find the most relevant tools for any given task. Instead of manually specifying which tools to use, AI clients can describe what they want to accomplish, and Smart Routing will identify and provide access to the most appropriate tools. ## How Smart Routing Works ### 1. Tool Indexing When servers start up, Smart Routing automatically: * Discovers all available tools from MCP servers * Extracts tool metadata (names, descriptions, parameters) * Converts tool information to vector embeddings * Stores embeddings in PostgreSQL with pgvector ### 2. Semantic Search When a query is made: * User queries are converted to vector embeddings * Similarity search finds matching tools using cosine similarity * Dynamic thresholds filter out irrelevant results * Results are ranked by relevance score ### 3. Intelligent Filtering Smart Routing applies several filters: * **Relevance Threshold**: Only returns tools above similarity threshold * **Context Awareness**: Considers conversation context * **Tool Availability**: Ensures tools are currently accessible * **Permission Filtering**: Respects user access permissions ### 4. Tool Execution Found tools can be directly executed: * Parameter validation ensures correct tool usage * Error handling provides helpful feedback * Response formatting maintains consistency * Logging tracks tool usage for analytics ## Prerequisites Smart Routing requires additional setup compared to basic MCPHub usage: ### Required Components 1. **PostgreSQL with pgvector**: Vector database for embeddings storage 2. **Embedding Service**: OpenAI API or compatible service 3. **Environment Configuration**: Proper configuration variables ## Using Smart Routing ### Smart Routing Endpoint Access Smart Routing through the special `$smart` endpoint: ``` # Search across all servers http://localhost:3000/mcp/$smart # Search within a specific group http://localhost:3000/mcp/$smart/{group} ``` ``` # Search across all servers http://localhost:3000/sse/$smart # Search within a specific group http://localhost:3000/sse/$smart/{group} ``` ### Group-Scoped Smart Routing Smart Routing now supports group-scoped searches, allowing you to limit tool discovery to servers within a specific group: Connect your AI client to a group-specific Smart Routing endpoint: ``` http://localhost:3000/mcp/$smart/production ``` This endpoint will only search for tools within servers that belong to the "production" group. **Benefits:** * **Focused Results**: Only tools from relevant servers are returned * **Better Performance**: Reduced search space for faster queries * **Environment Isolation**: Keep development, staging, and production tools separate * **Access Control**: Limit tool discovery based on user permissions Create groups for different environments: ```bash theme={null} # Development environment http://localhost:3000/mcp/$smart/development # Staging environment http://localhost:3000/mcp/$smart/staging # Production environment http://localhost:3000/mcp/$smart/production ``` Each endpoint will only return tools from servers in that specific environment group. Organize tools by team or department: ```bash theme={null} # Backend team tools http://localhost:3000/mcp/$smart/backend-team # Frontend team tools http://localhost:3000/mcp/$smart/frontend-team # DevOps team tools http://localhost:3000/mcp/$smart/devops-team ``` This enables teams to have focused access to their relevant toolsets. When using `$smart/{group}`: 1. The system identifies the specified group 2. Retrieves all servers belonging to that group 3. Filters the tool search to only those servers 4. Returns results scoped to the group's servers If the group doesn't exist or has no servers, the search will return no results. ### Pinned Tools A `$smart/{group}` endpoint lists only the meta-tools, so even a tool the model calls in nearly every session costs a `search_tools` round trip first. Pin such tools on the group's server entry and the endpoint lists their full definitions next to the meta-tools: ```json theme={null} { "name": "production", "servers": [{ "name": "time", "tools": "all", "pinnedTools": ["get_current_time"] }] } ``` In the dashboard, edit the group, expand the server and click the pin icon next to a selected tool. The icon is shown only while Smart Routing is enabled. * Pins use the bare tool name, like the `tools` selection. A pin only takes effect for a tool the group selects and that is enabled; other pins are ignored. * A pinned tool is listed exactly as on the direct `/mcp/{group}` endpoint — under the server's exposed name, with its description override — and is called directly by that name. It stays in the search index and keeps working through `search_tools`, `describe_tool` and `call_tool`. * The global `$smart` endpoint has no group and never lists pinned tools. A group without pins lists exactly the meta-tools, as before. * A pinned tool whose listed name equals a meta-tool name (`search_tools`, `describe_tool`, `call_tool`, e.g. alias `describe` + separator `_` + tool `tool`) is not listed, in any mode: calls to those names always go to the meta-tool. * The Smart Routing footprint shown for a group includes the definition cost of its pinned tools. * Deselecting a tool, in the dashboard or through `PUT /api/groups/:id/server-configs/:serverName/tools`, also removes its pin. * On `$smart/{group}`, a pinned tool keeps its MCP Apps metadata (`_meta.ui`) for a client that declares MCP Apps support, so its widget renders, and the server's `ui://` resources and app-only helper tools are reachable from that widget. A client without MCP Apps support gets the metadata stripped, as before. Only pinned tools carry a widget: a tool reached through `search_tools` and `call_tool` has no descriptor of its own for the host to attach a widget to, so pin the tools whose widgets you want. The global `$smart` endpoint is never an MCP Apps route. Behind `call_tool`, every tool shares a single client-side approval decision. A pinned tool is listed under its own name, so a client that approves tools one by one applies its per-tool rules to it — review that before pinning a tool that writes. ### Progressive Disclosure Mode Progressive Disclosure is an optimization feature that reduces token usage when working with Smart Routing. When enabled, the tool discovery workflow changes from a 2-step to a 3-step process. By default, Smart Routing returns full tool information including complete parameter schemas in `search_tools` results. This can consume significant tokens when dealing with tools that have complex input schemas. **Progressive Disclosure** changes this behavior: * `search_tools` returns only tool names and descriptions (minimal info) * A new `describe_tool` endpoint provides full parameter schema on demand * `call_tool` executes the tool as before This approach is particularly useful when: * Working with many tools with complex schemas * Token usage optimization is important * AI clients need to browse many tools before selecting one Enable Progressive Disclosure through the Settings page or environment variable: **Via Settings UI:** 1. Navigate to Settings → Smart Routing 2. Enable the "Progressive Disclosure" toggle 3. The change takes effect immediately **Via Environment Variable:** ```bash theme={null} SMART_ROUTING_PROGRESSIVE_DISCLOSURE=true ``` By default, `search_tools` lists only server names in its tool description. To include each server's description (or upstream instructions) alongside the name, set: ```bash theme={null} SMART_ROUTING_SERVER_DESCRIPTION_MODE=full ``` Use `names` (default) to keep the legacy compact list. The list names the servers `search_tools` actually searches: enabled servers that are connected or start on demand. An on-demand server stays listed while it is not connected, whether it shut down after idle or failed to start, since `call_tool` starts it; any other server that failed to connect is left out until it reconnects. Upstream instructions only arrive on connect, so in `full` mode a server that has not connected yet shows its configured description, or just its name. Asymmetric retrieval models are trained with different task prefixes for queries and documents, and score noticeably worse on raw text. Set the prefixes your model expects in Settings → Smart Routing or via environment variables: ```bash theme={null} # EmbeddingGemma SMART_ROUTING_EMBEDDING_QUERY_PREFIX="task: search result | query: " SMART_ROUTING_EMBEDDING_DOCUMENT_PREFIX="title: none | text: " ``` Both are empty by default, which embeds raw text as before. Prefixes are used verbatim, so keep the trailing space. Changing the document prefix re-embeds every server's tools; changing the query prefix takes effect on the next search. Prefixes shift the similarity scores (for EmbeddingGemma, weak matches drop more than strong ones), so a threshold tuned on raw text may need recalibrating after you enable them (see **Search Results** below; each hit reports its `score`). A prefix also counts toward the embedding token limit (`EMBEDDING_MAX_TOKENS` or the model default): it is kept and the tool text is truncated that many tokens earlier. Every `search_tools` hit carries a `score`: the cosine similarity between the query and the tool (0-1, two decimals), the same value the threshold is compared with. Results are ordered by relevance, which also weighs how well the tool's server matches the query. ```bash theme={null} # Minimum similarity for a hit (0-1). Unset: by query length — # 0.2 for short queries, 0.4 for long or "specific"/"exact" ones, 0.3 otherwise. SMART_ROUTING_SIMILARITY_THRESHOLD=0.45 # Only the first N hits carry their full definition (inputSchema etc.); the rest # carry name, description and score, and describe_tool is listed to fetch them. # Unset: every hit is returned in full. Ignored with Progressive Disclosure. SMART_ROUTING_FULL_SCHEMA_TOP_N=3 # Optional tool fields a full hit (and describe_tool) carries next to name, # description and inputSchema: title, annotations, outputSchema, execution, # icons, _meta. "none" sends none of them. Default: title,annotations. SMART_ROUTING_TOOL_DEFINITION_FIELDS=title,annotations ``` All three can also be set in Settings → Smart Routing → Search Results, and none of them re-embeds anything. The fields left out by default are meant for clients rather than models and can be large: an upstream server may ship base64 `icons` or a long `outputSchema` with every tool, and in a search result they would only cost tokens. The `inputSchema` is always sent, with its `$schema` key removed. When no tool clears the threshold, the response says so, names the threshold and lists the servers that were searched, so the model can tell the user the capability is not available instead of calling the nearest unrelated tool. When Progressive Disclosure is **disabled** (default), Smart Routing provides two tools: **Workflow:** `search_tools` → `call_tool` | Tool | Purpose | | - | - | | `search_tools` | Find tools by query, returns full tool info including `inputSchema` | | `call_tool` | Execute a tool with the provided arguments | This mode is simpler but uses more tokens due to full schemas in search results. `SMART_ROUTING_FULL_SCHEMA_TOP_N` caps that cost: only the top N hits carry a schema, and `describe_tool` is listed for the rest. When Progressive Disclosure is **enabled**, Smart Routing provides three tools: **Workflow:** `search_tools` → `describe_tool` → `call_tool` | Tool | Purpose | | - | - | | `search_tools` | Find tools by query, returns only name and description | | `describe_tool` | Get full schema for a specific tool (new) | | `call_tool` | Execute a tool with the provided arguments | **Example workflow:** 1. AI calls `search_tools` with query "file operations" 2. Results show tool names and descriptions (minimal tokens) 3. AI calls `describe_tool` for a specific tool to get full `inputSchema` 4. AI calls `call_tool` with the correct arguments This mode reduces token usage by only fetching full schemas when needed. **Standard Mode search\_tools response:** ```json theme={null} { "tools": [ { "name": "read_file", "description": "Read contents of a file", "inputSchema": { "type": "object", "properties": { "path": { "type": "string", "description": "File path to read" }, "encoding": { "type": "string", "default": "utf-8" } }, "required": ["path"] }, "score": 0.71 } ] } ``` **Progressive Disclosure Mode search\_tools response:** ```json theme={null} { "tools": [ { "name": "read_file", "description": "Read contents of a file", "score": 0.71 } ], "metadata": { "progressiveDisclosure": true, "guideline": "Use describe_tool to get the full parameter schema before calling." } } ``` **describe\_tool response:** ```json theme={null} { "tool": { "name": "read_file", "description": "Read contents of a file", "inputSchema": { "type": "object", "properties": { "path": { "type": "string", "description": "File path to read" }, "encoding": { "type": "string", "default": "utf-8" } }, "required": ["path"] } } } ``` ## Troubleshooting **Symptoms:** * Smart Routing not available * Database connection errors * Embedding storage failures **Solutions:** 1. Verify PostgreSQL is running 2. Check DB\_URL format 3. Ensure pgvector extension is installed 4. Test connection manually: ```bash theme={null} psql $DB_URL -c "SELECT 1;" ``` **Symptoms:** * Tool indexing failures * Query processing errors * API rate limit errors **Solutions:** 1. Verify API key validity 2. Check network connectivity 3. Monitor rate limits 4. Test embedding service: ```bash theme={null} curl -X POST https://api.openai.com/v1/embeddings \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "test", "model": "text-embedding-3-small"}' ``` **Symptoms:** * Irrelevant tools returned * Low relevance scores * Missing expected tools **Solutions:** 1. Adjust the similarity threshold (`SMART_ROUTING_SIMILARITY_THRESHOLD`), using the `score` of each hit to pick a value 2. Re-index tools with better descriptions 3. Use more specific queries 4. Check tool metadata quality ```bash theme={null} # Re-index all tools (full rebuild: clears vector_embeddings first, then # regenerates embeddings for every enabled server; servers that require # personal credentials are reached through the users holding a binding, # indexing the union of the tools those users can see; runs synchronously # and streams per-server progress through SSE embedding-sync-progress events) curl -X POST http://localhost:3000/api/smart-routing/reindex \ -H "Authorization: Bearer YOUR_JWT_TOKEN" ``` **Symptoms:** * Slow query responses * High database load * Memory usage spikes **Solutions:** 1. Optimize database configuration 2. Increase cache sizes 3. Reduce batch sizes 4. Monitor system resources ```bash theme={null} # Check vector store status and performance (config, database connection, # row counts, per-model/per-server breakdown, sync freshness, coverage) curl http://localhost:3000/api/smart-routing/performance \ -H "Authorization: Bearer YOUR_JWT_TOKEN" ``` **Symptoms:** * Tool indexing fails with an authentication (401/403) error * A valid key is saved in the dashboard, yet requests keep being rejected **Cause:** an environment variable outranks the dashboard value, so the key you saved is never used (issue #642). Recent versions show a warning under the affected field naming the variable in use. **Solutions:** 1. Look for `OPENAI_API_KEY` / `AZURE_OPENAI_API_KEY` in `.env*` files, your shell profile, or the container/service definition 2. Remove the stale variable and restart, then re-index if the provider or model also changed 3. Confirm the key actually in use with a manual request: ```bash theme={null} curl -X POST "${OPENAI_API_BASE_URL:-https://api.openai.com/v1}/embeddings" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "test", "model": "text-embedding-3-small"}' ``` ## Best Practices ### Query Writing **Be Descriptive**: Use specific, descriptive language in queries for better tool matching. **Include Context**: Provide relevant context about your task or domain for more accurate results. **Use Natural Language**: Write queries as you would describe the task to a human. ### Tool Descriptions **Quality Metadata**: Ensure MCP servers provide high-quality tool descriptions and metadata. **Regular Updates**: Keep tool descriptions current as functionality evolves. **Consistent Naming**: Use consistent naming conventions across tools and servers. ### System Maintenance **Regular Re-indexing**: Periodically re-index tools to ensure embedding quality. **Monitor Performance**: Track query patterns and optimize based on usage. **Update Models**: Consider updating to newer embedding models as they become available. ## Next Steps User management and access control System monitoring and analytics Complete Smart Routing API documentation Advanced configuration options This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.