Every MCP tool connected to your AI agent injects its full schema into the LLM’s context window on every request. Connect five MCP servers with 100 tools, and you are burning thousands of tokens on tool catalogs before the model even starts reasoning. This article breaks down why MCP tool definition bloat happens, why it degrades agent performance, and how to solve it using gateway-level strategies like tool filtering, on-demand loading, and code-driven orchestration. Bifrost, the open-source LLM gateway by Maxim AI, offers a production-ready solution through its MCP Gateway and Code Mode, cutting token usage by 50%+ and reducing LLM round trips by 3-4x in multi-tool workflows.
The Hidden Cost of MCP at Scale
The Model Context Protocol (MCP), introduced by Anthropic in late 2024 and now adopted by OpenAI, Google, and Microsoft, has become the standard for connecting LLMs to external tools. MCP lets AI models discover and invoke tools at runtime: databases, file systems, web search, CRM platforms, and more.
The problem is not MCP itself. The problem is what happens when MCP scales.
Every MCP server exposes tools with JSON schemas describing names, parameters, types, and descriptions. When an LLM application connects to an MCP server, all tool schemas get injected into the context window on every request. The model needs to know what tools exist to decide which ones to call.
With one server and five tools, this is manageable. But production AI systems connect to many servers. An e-commerce assistant might use product catalog, inventory, payments, shipping, analytics, and notifications. A dev tooling agent connects to GitHub, Jira, Slack, file systems, and databases. Each server exposes 10-20 tools, putting 100-150 tool definitions into every request’s context window.
Those definitions are not free. They consume input tokens. A setup with 150 tools can burn 2,000-3,000 tokens per turn on schema definitions alone, before the model processes a single word of the user’s query.
Why This Hurts More Than You Think
Context window bloat from tool definitions creates a cascade of problems beyond raw token cost.
Increased latency. More tokens means longer processing. In agentic workflows where a single query triggers 5-10 LLM turns, each carrying the full tool catalog, latency compounds fast.
Degraded reasoning quality. LLMs have finite attention. When a large portion of the context is occupied by irrelevant tool schemas, the model spends cognitive budget parsing tools it does not need instead of solving the actual problem.
Wasted spend at scale. A workflow spanning 6 turns with 100 tools per turn means paying for 600 tool-definition token blocks per query. Across thousands of daily queries, tool definitions alone can rival the cost of actual reasoning tokens.
Development friction. Teams avoid connecting useful MCP servers because the context overhead of each new server is too high. Tool sprawl becomes the new microservices sprawl.
Strategies for Reducing Tool Definition Bloat
There are several approaches to solve this problem, ranging from simple filtering to architectural changes at the gateway level.
1. Tool Filtering and Whitelisting
The most straightforward approach is to not expose every tool to every request. If your agent has a payments MCP server with 15 tools but only needs create_charge and get_balance for a specific use case, filter the rest out.
Bifrost’s MCP Gateway supports per-request tool filtering through virtual key policies. You can define which tools are available to which consumers, so different teams, applications, or workflows only see the tools they actually need. This is configured through the tools_to_execute field on each MCP client connection, accepting either a wildcard or an explicit list of tool names.
2. Dynamic Tool Injection at the Gateway Level
Instead of loading all tools statically, a gateway layer can analyze the incoming query, determine which tool categories are relevant, and only include those schemas in the context. Bifrost acts as both an MCP client (connecting to multiple tool servers) and an MCP server (exposing aggregated, filtered tools to external clients), giving you a single control plane for tool visibility.
3. Code Mode: The Architectural Solution
Filtering helps, but the real breakthrough comes from rethinking how LLMs interact with tools entirely. This is the approach behind Bifrost’s Code Mode, available from v1.4.0-prerelease1.
Instead of exposing 150 tools directly to the model, Code Mode replaces them with four meta-tools:
- listToolFiles: Discover available MCP servers and their tools as virtual .pyi stub files
- readToolFile: Load only the specific Python function signatures the model needs on demand
- getToolDocs: Retrieve detailed documentation for a single tool when compact signatures are insufficient
- executeToolCode: Run Python code (Starlark) in a sandboxed environment with full access to all tool bindings
The model no longer reads 150 tool schemas upfront. It discovers what is available, loads only what it needs, writes a Python script to orchestrate the tools, and executes everything in a sandbox. Intermediate results stay in the sandbox. Only the final, compact result returns to the LLM.
The impact is dramatic. Consider a workflow across 5 MCP servers with around 100 tools:
Classic MCP: 6 LLM turns, 100 tool definitions loaded per turn, all intermediate results flowing through the model.
Code Mode: 3-4 LLM turns, only 4 tools + on-demand definitions (roughly 50 tokens in tool definitions total), intermediate processing handled in the sandbox.
Code Mode also supports two binding levels. Server-level binding (default) groups all tools from a server into a single .pyi file, best for servers with fewer tools. Tool-level binding gives each tool its own file, ideal when servers expose 30+ tools with complex schemas. Both modes use the same four-tool interface, and the choice is purely about context efficiency per read operation.
When to Apply Each Strategy
Not every setup needs Code Mode. Bifrost’s docs recommend enabling it when you have 3+ MCP servers, complex multi-step workflows, or concerns about token costs. For simpler setups with 1-2 servers, classic MCP with basic filtering works fine. You can also mix both: enable Code Mode for heavy servers (web search, databases, document tools) and keep small utilities as direct tools.
Getting Started
Bifrost is fully open source. Install with
npx -y @maximhq/bifrost
or
docker run -p 8080:8080 maximhq/bifrost,
configure MCP servers through the Web UI, and enable Code Mode per client. The GitHub repository has the full source, and the documentation covers setup and configuration in detail.
Context window bloat from MCP tool definitions is not theoretical. It is the single largest hidden cost in multi-tool agentic AI systems today. Solving it at the gateway layer is the difference between a prototype that works and a production system that scales.
