Dynamic toolsets
Dynamic toolsets enable very large MCP servers without overloading context windows. Instead of exposing all tools upfront like traditional MCP, dynamic toolsets provide “meta” tools that allow the LLM to discover only the tools it needs to complete specific tasks, delivering up to 160x token reduction while maintaining full functionality.
Our refined Dynamic Toolsets approach combines the best of semantic search and progressive discovery into a unified system that exposes three core tools. For detailed technical insights and performance benchmarks, see our blog post on how we reduced token usage by 100x.
How Dynamic Toolsets work
Dynamic toolsets follow the natural workflow an LLM needs: search → describe → execute. The system compresses large toolsets into three meta-tools:
search_tools
The LLM searches for relevant tools using natural language queries with embeddings-based semantic search. The tool description includes categorical overviews of available tools (e.g., “This toolset includes HubSpot CRM operations, deal management…”) and supports filtering by tags like source:hubspot for precise discovery.
describe_tools
The LLM requests detailed schemas and documentation only for tools it intends to use. This separation optimizes token usage since input schemas represent 60-80% of total tokens in static toolsets.
execute_tool
The LLM executes discovered and described tools with proper parameters.
Performance benefits
Dynamic toolsets deliver significant advantages over static toolsets:
Massive token reduction: Input tokens are reduced by an average of 96% for simple tasks and 91% for complex tasks, with total token usage dropping by 96% and 90% respectively.
Consistent scaling: Token usage remains relatively constant regardless of toolset size. A 400-tool dynamic toolset uses only ~8,000 tokens initially compared to 410,000+ for the same static toolset.
Context window compatibility: Large toolsets that exceed Claude’s 200k context window limit with static approaches work seamlessly with dynamic toolsets.
Perfect reliability: Maintains 100% success rates across all toolset sizes and task complexities.
Sample performance data
| Toolset Size | Mode | Simple Task Tokens | Tool Calls | Complex Task Tokens | Tool Calls |
|---|---|---|---|---|---|
| 100 tools | Static | 159,218 | 1 | 159,216 | 3 |
| 100 tools | Dynamic | 8,401 | 3 | 18,095 | 7 |
| 400 tools | Static | 410,738 | 1 | 410,661 | 3 |
| 400 tools | Dynamic | 8,421 | 3 | 31,355 | 7.8 |
Trade-offs
While dynamic toolsets offer significant benefits, there are some considerations:
Increased tool calls: Dynamic toolsets require 2-3x more tool calls (typically 6-8 for complex tasks vs 3 for static), following the search → describe → execute pattern.
Potential latency: Additional tool calls may introduce slight latency, though this is often offset by reduced token processing time.
Complexity: The multi-step discovery process adds complexity compared to direct tool access, though this is handled automatically by the LLM.
Enabling dynamic toolsets
Head to the MCP tab in your Gram dashboard and switch your toolset from “Static” to “Dynamic” mode.
Note
This setting only applies to MCP and will not affect how your toolset is used in the playground, where static tool exposure remains useful for testing and development.
Dynamic toolsets are particularly valuable for:
- APIs with 100+ operations
- Enterprise systems with comprehensive toolsets
- Applications where context window limits are a concern
- Production environments requiring predictable costs
Additional reading
- How we reduced token usage by 100x with Dynamic Toolsets
- Code Execution with MCP
- Previous Dynamic Toolsets implementation
Last updated on