Smart Routing

Overview

Smart Routing is MCPHub’s intelligent tool discovery system that uses vector semantic search to automatically find the most relevant tools for any given task. Instead of manually specifying which tools to use, AI clients can describe what they want to accomplish, and Smart Routing will identify and provide access to the most appropriate tools.

How Smart Routing Works

1. Tool Indexing

When servers start up, Smart Routing automatically:

Discovers all available tools from MCP servers
Extracts tool metadata (names, descriptions, parameters)
Converts tool information to vector embeddings
Stores embeddings in PostgreSQL with pgvector

2. Semantic Search

When a query is made:

User queries are converted to vector embeddings
Similarity search finds matching tools using cosine similarity
Dynamic thresholds filter out irrelevant results
Results are ranked by relevance score

3. Intelligent Filtering

Smart Routing applies several filters:

Relevance Threshold: Only returns tools above similarity threshold
Context Awareness: Considers conversation context
Tool Availability: Ensures tools are currently accessible
Permission Filtering: Respects user access permissions

4. Tool Execution

Found tools can be directly executed:

Parameter validation ensures correct tool usage
Error handling provides helpful feedback
Response formatting maintains consistency
Logging tracks tool usage for analytics

Prerequisites

Smart Routing requires additional setup compared to basic MCPHub usage:

Required Components

PostgreSQL with pgvector: Vector database for embeddings storage
Embedding Service: OpenAI API or compatible service
Environment Configuration: Proper configuration variables

Using Smart Routing

Smart Routing Endpoint

Access Smart Routing through the special $smart endpoint:

HTTP MCP
SSE (Legacy)

# Search across all servers
http://localhost:3000/mcp/$smart

# Search within a specific group
http://localhost:3000/mcp/$smart/{group}

# Search across all servers
http://localhost:3000/sse/$smart

# Search within a specific group
http://localhost:3000/sse/$smart/{group}

Group-Scoped Smart Routing

Smart Routing now supports group-scoped searches, allowing you to limit tool discovery to servers within a specific group:

Using Group-Scoped Smart Routing

Connect your AI client to a group-specific Smart Routing endpoint:

http://localhost:3000/mcp/$smart/production

This endpoint will only search for tools within servers that belong to the “production” group.Benefits:

Focused Results: Only tools from relevant servers are returned
Better Performance: Reduced search space for faster queries
Environment Isolation: Keep development, staging, and production tools separate
Access Control: Limit tool discovery based on user permissions

Example: Environment-Based Groups

Create groups for different environments:

# Development environment
http://localhost:3000/mcp/$smart/development

# Staging environment
http://localhost:3000/mcp/$smart/staging

# Production environment
http://localhost:3000/mcp/$smart/production

Each endpoint will only return tools from servers in that specific environment group.

Example: Team-Based Groups

Organize tools by team or department:

# Backend team tools
http://localhost:3000/mcp/$smart/backend-team

# Frontend team tools
http://localhost:3000/mcp/$smart/frontend-team

# DevOps team tools
http://localhost:3000/mcp/$smart/devops-team

This enables teams to have focused access to their relevant toolsets.

How It Works

When using $smart/{group}:

The system identifies the specified group
Retrieves all servers belonging to that group
Filters the tool search to only those servers
Returns results scoped to the group’s servers

If the group doesn’t exist or has no servers, the search will return no results.

Pinned Tools

A $smart/{group} endpoint lists only the meta-tools, so even a tool the model calls in nearly every session costs a search_tools round trip first. Pin such tools on the group’s server entry and the endpoint lists their full definitions next to the meta-tools:

{
  "name": "production",
  "servers": [{ "name": "time", "tools": "all", "pinnedTools": ["get_current_time"] }]
}

In the dashboard, edit the group, expand the server and click the pin icon next to a selected tool. The icon is shown only while Smart Routing is enabled.

Pins use the bare tool name, like the tools selection. A pin only takes effect for a tool the group selects and that is enabled; other pins are ignored.
A pinned tool is listed exactly as on the direct /mcp/{group} endpoint — under the server’s exposed name, with its description override — and is called directly by that name. It stays in the search index and keeps working through search_tools, describe_tool and call_tool.
The global $smart endpoint has no group and never lists pinned tools. A group without pins lists exactly the meta-tools, as before.
A pinned tool whose listed name equals a meta-tool name (search_tools, describe_tool, call_tool, e.g. alias describe + separator _ + tool tool) is not listed, in any mode: calls to those names always go to the meta-tool.
The Smart Routing footprint shown for a group includes the definition cost of its pinned tools.
Deselecting a tool, in the dashboard or through PUT /api/groups/:id/server-configs/:serverName/tools, also removes its pin.
On $smart/{group}, a pinned tool keeps its MCP Apps metadata (_meta.ui) for a client that declares MCP Apps support, so its widget renders, and the server’s ui:// resources and app-only helper tools are reachable from that widget. A client without MCP Apps support gets the metadata stripped, as before. Only pinned tools carry a widget: a tool reached through search_tools and call_tool has no descriptor of its own for the host to attach a widget to, so pin the tools whose widgets you want. The global $smart endpoint is never an MCP Apps route.

Behind call_tool, every tool shares a single client-side approval decision. A pinned tool is listed under its own name, so a client that approves tools one by one applies its per-tool rules to it — review that before pinning a tool that writes.

Progressive Disclosure Mode

Progressive Disclosure is an optimization feature that reduces token usage when working with Smart Routing. When enabled, the tool discovery workflow changes from a 2-step to a 3-step process.

What is Progressive Disclosure?

By default, Smart Routing returns full tool information including complete parameter schemas in search_tools results. This can consume significant tokens when dealing with tools that have complex input schemas.Progressive Disclosure changes this behavior:

search_tools returns only tool names and descriptions (minimal info)
A new describe_tool endpoint provides full parameter schema on demand
call_tool executes the tool as before

This approach is particularly useful when:

Working with many tools with complex schemas
Token usage optimization is important
AI clients need to browse many tools before selecting one

Enabling Progressive Disclosure

Enable Progressive Disclosure through the Settings page or environment variable:Via Settings UI:

Navigate to Settings → Smart Routing
Enable the “Progressive Disclosure” toggle
The change takes effect immediately

Via Environment Variable:

SMART_ROUTING_PROGRESSIVE_DISCLOSURE=true

Including Server Descriptions in search_tools

By default, search_tools lists only server names in its tool description. To include each server’s description (or upstream instructions) alongside the name, set:

SMART_ROUTING_SERVER_DESCRIPTION_MODE=full

Use names (default) to keep the legacy compact list.The list names the servers search_tools actually searches: enabled servers that are connected or start on demand. An on-demand server stays listed while it is not connected, whether it shut down after idle or failed to start, since call_tool starts it; any other server that failed to connect is left out until it reconnects. Upstream instructions only arrive on connect, so in full mode a server that has not connected yet shows its configured description, or just its name.

Embedding Task Prefixes

Asymmetric retrieval models are trained with different task prefixes for queries and documents, and score noticeably worse on raw text. Set the prefixes your model expects in Settings → Smart Routing or via environment variables:

# EmbeddingGemma
SMART_ROUTING_EMBEDDING_QUERY_PREFIX="task: search result | query: "
SMART_ROUTING_EMBEDDING_DOCUMENT_PREFIX="title: none | text: "

Both are empty by default, which embeds raw text as before. Prefixes are used verbatim, so keep the trailing space. Changing the document prefix re-embeds every server’s tools; changing the query prefix takes effect on the next search.Prefixes shift the similarity scores (for EmbeddingGemma, weak matches drop more than strong ones), so a threshold tuned on raw text may need recalibrating after you enable them (see Search Results below; each hit reports its score). A prefix also counts toward the embedding token limit (EMBEDDING_MAX_TOKENS or the model default): it is kept and the tool text is truncated that many tokens earlier.

Search Results: Scores, Threshold and Schema Budget

Every search_tools hit carries a score: the cosine similarity between the query and the tool (0-1, two decimals), the same value the threshold is compared with. Results are ordered by relevance, which also weighs how well the tool’s server matches the query.

# Minimum similarity for a hit (0-1). Unset: by query length —
# 0.2 for short queries, 0.4 for long or "specific"/"exact" ones, 0.3 otherwise.
SMART_ROUTING_SIMILARITY_THRESHOLD=0.45

# Only the first N hits carry their full definition (inputSchema etc.); the rest
# carry name, description and score, and describe_tool is listed to fetch them.
# Unset: every hit is returned in full. Ignored with Progressive Disclosure.
SMART_ROUTING_FULL_SCHEMA_TOP_N=3

# Optional tool fields a full hit (and describe_tool) carries next to name,
# description and inputSchema: title, annotations, outputSchema, execution,
# icons, _meta. "none" sends none of them. Default: title,annotations.
SMART_ROUTING_TOOL_DEFINITION_FIELDS=title,annotations

All three can also be set in Settings → Smart Routing → Search Results, and none of them re-embeds anything. The fields left out by default are meant for clients rather than models and can be large: an upstream server may ship base64 icons or a long outputSchema with every tool, and in a search result they would only cost tokens. The inputSchema is always sent, with its $schema key removed. When no tool clears the threshold, the response says so, names the threshold and lists the servers that were searched, so the model can tell the user the capability is not available instead of calling the nearest unrelated tool.

Standard Mode (Default)

When Progressive Disclosure is disabled (default), Smart Routing provides two tools:Workflow: search_tools → call_tool

Tool	Purpose
`search_tools`	Find tools by query, returns full tool info including `inputSchema`
`call_tool`	Execute a tool with the provided arguments

This mode is simpler but uses more tokens due to full schemas in search results. SMART_ROUTING_FULL_SCHEMA_TOP_N caps that cost: only the top N hits carry a schema, and describe_tool is listed for the rest.

Progressive Disclosure Mode

When Progressive Disclosure is enabled, Smart Routing provides three tools:Workflow: search_tools → describe_tool → call_tool

Tool	Purpose
`search_tools`	Find tools by query, returns only name and description
`describe_tool`	Get full schema for a specific tool (new)
`call_tool`	Execute a tool with the provided arguments

Example workflow:

AI calls search_tools with query “file operations”
Results show tool names and descriptions (minimal tokens)
AI calls describe_tool for a specific tool to get full inputSchema
AI calls call_tool with the correct arguments

This mode reduces token usage by only fetching full schemas when needed.

Response Format Comparison

Standard Mode search_tools response:

{
  "tools": [
    {
      "name": "read_file",
      "description": "Read contents of a file",
      "inputSchema": {
        "type": "object",
        "properties": {
          "path": { "type": "string", "description": "File path to read" },
          "encoding": { "type": "string", "default": "utf-8" }
        },
        "required": ["path"]
      },
      "score": 0.71
    }
  ]
}

Progressive Disclosure Mode search_tools response:

{
  "tools": [
    {
      "name": "read_file",
      "description": "Read contents of a file",
      "score": 0.71
    }
  ],
  "metadata": {
    "progressiveDisclosure": true,
    "guideline": "Use describe_tool to get the full parameter schema before calling."
  }
}

describe_tool response:

{
  "tool": {
    "name": "read_file",
    "description": "Read contents of a file",
    "inputSchema": {
      "type": "object",
      "properties": {
        "path": { "type": "string", "description": "File path to read" },
        "encoding": { "type": "string", "default": "utf-8" }
      },
      "required": ["path"]
    }
  }
}

Troubleshooting

Database Connection Issues

Symptoms:

Smart Routing not available
Database connection errors
Embedding storage failures

Solutions:

Verify PostgreSQL is running
Check DB_URL format
Ensure pgvector extension is installed
Test connection manually:

psql $DB_URL -c "SELECT 1;"

Embedding Service Problems

Symptoms:

Tool indexing failures
Query processing errors
API rate limit errors

Solutions:

Verify API key validity
Check network connectivity
Monitor rate limits
Test embedding service:

curl -X POST https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "test", "model": "text-embedding-3-small"}'

Poor Search Results

Symptoms:

Irrelevant tools returned
Low relevance scores
Missing expected tools

Solutions:

Adjust the similarity threshold (SMART_ROUTING_SIMILARITY_THRESHOLD), using the score of each hit to pick a value
Re-index tools with better descriptions
Use more specific queries
Check tool metadata quality

# Re-index all tools (full rebuild: clears vector_embeddings first, then
# regenerates embeddings for every enabled server; servers that require
# personal credentials are reached through the users holding a binding,
# indexing the union of the tools those users can see; runs synchronously
# and streams per-server progress through SSE embedding-sync-progress events)
curl -X POST http://localhost:3000/api/smart-routing/reindex \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Performance Issues

Symptoms:

Slow query responses
High database load
Memory usage spikes

Solutions:

Optimize database configuration
Increase cache sizes
Reduce batch sizes
Monitor system resources

# Check vector store status and performance (config, database connection,
# row counts, per-model/per-server breakdown, sync freshness, coverage)
curl http://localhost:3000/api/smart-routing/performance \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Authentication Errors Despite a Valid Dashboard Key

Symptoms:

Tool indexing fails with an authentication (401/403) error
A valid key is saved in the dashboard, yet requests keep being rejected

Cause: an environment variable outranks the dashboard value, so the key you saved is never used (issue #642). Recent versions show a warning under the affected field naming the variable in use.Solutions:

Look for OPENAI_API_KEY / AZURE_OPENAI_API_KEY in .env* files, your shell profile, or the container/service definition
Remove the stale variable and restart, then re-index if the provider or model also changed
Confirm the key actually in use with a manual request:

curl -X POST "${OPENAI_API_BASE_URL:-https://api.openai.com/v1}/embeddings" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "test", "model": "text-embedding-3-small"}'

Best Practices

Query Writing

Be Descriptive: Use specific, descriptive language in queries for better tool matching.

Include Context: Provide relevant context about your task or domain for more accurate results.

Use Natural Language: Write queries as you would describe the task to a human.

Tool Descriptions

Quality Metadata: Ensure MCP servers provide high-quality tool descriptions and metadata.

Regular Updates: Keep tool descriptions current as functionality evolves.

Consistent Naming: Use consistent naming conventions across tools and servers.

System Maintenance

Regular Re-indexing: Periodically re-index tools to ensure embedding quality.

Monitor Performance: Track query patterns and optimize based on usage.

Update Models: Consider updating to newer embedding models as they become available.

Next Steps

Authentication

User management and access control

Monitoring

System monitoring and analytics

API Reference

Complete Smart Routing API documentation

Configuration

Advanced configuration options

Get Started

Core Features

Configuration

Development

Overview

How Smart Routing Works

1. Tool Indexing

2. Semantic Search

3. Intelligent Filtering

4. Tool Execution

Prerequisites

Required Components

Using Smart Routing

Smart Routing Endpoint

Group-Scoped Smart Routing

Pinned Tools

Progressive Disclosure Mode

Troubleshooting

Best Practices

Query Writing

Tool Descriptions

System Maintenance

Next Steps

Authentication

Monitoring

API Reference

Configuration

​Overview

​How Smart Routing Works

​1. Tool Indexing

​2. Semantic Search

​3. Intelligent Filtering

​4. Tool Execution

​Prerequisites

​Required Components

​Using Smart Routing

​Smart Routing Endpoint

​Group-Scoped Smart Routing

​Pinned Tools

​Progressive Disclosure Mode

​Troubleshooting

​Best Practices

​Query Writing

​Tool Descriptions

​System Maintenance

​Next Steps

Authentication

Monitoring

API Reference

Configuration

Overview

How Smart Routing Works

1. Tool Indexing

2. Semantic Search

3. Intelligent Filtering

4. Tool Execution

Prerequisites

Required Components

Using Smart Routing

Smart Routing Endpoint

Group-Scoped Smart Routing

Pinned Tools

Progressive Disclosure Mode

Troubleshooting

Best Practices

Query Writing

Tool Descriptions

System Maintenance

Next Steps