MCP Gateway with Telemetry: How Tool Runtime Governs Agent Tools

The first MCP server an agent uses is easy to manage. The tenth is not. Once a team has a scheduling server, a Procore server, a document server and a handful of public tools, each with its own credentials, config file and failure modes, nobody can answer basic questions: which agent called which tool, with what input, and what came back?

Tool Runtime is the MCP gateway we built to answer those questions. It pulls MCP servers from multiple sources into one MCP endpoint, controls access with an API key, and logs every tool call with its input and output. This article walks through how it works, the design decisions behind it, and what any team putting agents into production should take from it, whether or not they ever use our gateway.

The problem: unmanaged MCP sprawl

The Model Context Protocol gives AI clients a standard way to discover and call tools. That standard is what makes MCP useful, and it's also what makes it sprawl. Any developer can add a server to Cursor or Claude in a few lines of JSON. Repeat that across a team and you end up with:

• Credentials scattered across client configs. Every developer machine and every agent holds its own copy of API tokens for every upstream system.
• No single view of usage. Each server logs (or doesn't) in its own way. There is no one place to see what an agent did.
• All-or-nothing tool exposure. Connect a server and the agent usually sees every tool it offers, including write operations nobody meant to grant.
• No limits. A looping agent can call the same tool hundreds of times and nothing stops it.

For construction systems, where one API surface can carry thousands of operations (see how we approached routing 2,755 Procore API tools), these gaps go from inconvenient to unacceptable. A gateway closes them by putting one controlled layer between agents and every tool they can reach.

What an MCP gateway does

A gateway is a single MCP server that sits in front of many others. The agent connects once. The gateway decides which tools that connection can see, injects the right upstream credentials, enforces limits, and records what happened.

In Tool Runtime, that connection is defined by an API key. The key is the policy: it says which MCP servers are included, which tools on each server are allowed, which authentication applies upstream, and what rate limits hold. Change the key's configuration and every agent using it gets the new boundary, with no client config edits.

Under the hood, the gateway exposes a streamable HTTP MCP server. Every client points at the same Tool Runtime endpoint, and the API key tells the runtime what that caller is allowed to do.

How Tool Runtime works, step by step

The workflow in the demo runs from finding tools to watching them get called.

1. Discover tools in the toolbox. The toolbox is a searchable, filterable registry. It pulls in the public MCP directory plus any MCP created through Tool Runtime itself. Each entry shows its endpoints, the tools it exposes and its documentation.
2. Connect and save. You connect to the servers you want, add authentication where the upstream system requires it, and save them. Saved servers can be ranked so the team can mark which ones are actually valuable.
3. Build new tools when none exist. An AI tool creator walks you through building a new MCP server by chat. It asks questions about what you want to build, gathers resources and documentation, then moves through specification, architecture and design before generating the code and submitting a repo.
4. Review and edit the code. Generated servers open in an IDE embedded in the browser. You can inspect what the agent implemented, edit it, amend the commit and push to GitHub. Once the repo is published, the runtime picks it up and lists it on the registry if you choose to make it public.
5. Compose an MCP (an API key). You select saved MCP servers, adjust authentication or headers and test them if needed, then pick the specific tools to include from each. You name the MCP, choose the authentication method, and receive a complete mcp.json to paste into Cursor, Claude or any MCP client.
6. Manage keys. The API keys page lists every key you've created. For each one you can view the key, fetch the config again, edit which servers and tools it includes, and change its rate limits.
7. Test before you trust. A built-in inspector loads the MCP behind a key, lists its tools and lets you make test calls. You can also run the MCP inside an AI agent to make real calls and start building agentic workflows.
8. Watch it in production. Tool insights logs every call with the organization, user, API key, MCP server, and the input and output. Analytics roll this up per tool and per server with request graphs and success rates for each API key.

Telemetry: what gets captured and why

The core of the gateway is the call log. For every tool invocation, Tool Runtime captures the input before the call runs and the output after it returns, along with who made the call and through which key.

That record is what makes agent behavior debuggable. When an agent gives a wrong answer, you don't have to guess whether the model misread the data or the tool returned bad data. You open the call, read the input and output, and know. This is the same principle we apply everywhere: evidence on every output.

The dashboard sits on top of the log: total tool calls, available API keys, recent tool-call telemetry, and exportable analytics for deeper review. Per-tool analytics show request volume and success rates, which is where you spot a flaky upstream server or an agent that calls one tool far more than it should.

Policies, limits and administration

Logging tells you what happened. Policies decide what is allowed to happen. Tool Runtime supports policies on which tools are allowed, how many tools are available, and how often they can be called, with usage tracked against those limits.

An admin portal handles the organizational side:

• Inviting users and granting access to specific capabilities
• Provisioning capabilities for organizations
• Setting admin-level rate limits on specific MCP servers
• Platform analytics showing activity across organizations

There is also a user layer. Each user has a profile with an about section, links, their analytics and the tools they've created. Public profiles let other users find and reuse those tools in their own instance. Settings cover connecting GitHub repositories and privacy controls for who can see your profile.

Design decisions and why we made them

The API key is the unit of policy

We could have attached permissions to agents, users or servers. We attached them to the key because that's the thing a client actually presents. One key equals one composed MCP: a fixed set of servers, a fixed set of tools, fixed upstream auth, fixed limits. You can hand different keys to different agents and reason about each one independently. It's least privilege by default in a form that is easy to audit.

Tool-level selection, not server-level

Connecting a whole server is the easy path and the dangerous one. Making the user pick tools from each server forces a deliberate decision about exposure. An agent that only needs to read schedules never sees the write operations sitting on the same server.

One standard transport

Exposing everything as a single streamable HTTP MCP endpoint means any MCP client that supports streamable HTTP can connect without custom plugins. The mcp.json the gateway generates is the whole integration.

Capture input and output, not just metadata

Counting calls is useful for billing and capacity. It does nothing for debugging. Logging the actual input and output costs more storage, but it's the only way to answer "why did the agent say that?" In production, that question comes up often.

Build and govern in the same place

Because the tool creator, registry and gateway share one platform, a new server goes from generated code to reviewed repo to a governed, logged tool without leaving the system. Tools don't get bolted on outside the control plane.

What this means if you're putting agents into production

You don't need our gateway to apply these lessons. You do need something that plays the gateway's role before agents touch production systems like your ERP, Procore or Primavera P6.

• Centralize credentials. Agents and developer machines should hold a gateway key, not tokens for every upstream system. Rotating one upstream credential should not mean editing twenty client configs.
• Expose tools, not servers. Decide tool by tool what each agent can call. Start read-only and add write operations only when there's a reviewed reason.
• Log input and output for every call. If you can't replay what the agent asked and what it got back, you can't debug it, and you can't defend its outputs to a controller or an owner.
• Set rate limits before you need them. Agents loop. A limit per key turns a runaway loop into a blocked request instead of an API bill or a locked-out integration user.
• Test with an inspector before handing a key to an agent. Calling each tool directly confirms auth, inputs and outputs before a model is in the mix.
• Give each agent its own key. Separate keys make the logs readable and let you shut off one agent without touching the rest.

For a broader treatment of controls, approvals and audit trails, see our guide to governing AI agents in construction.

Practical lessons from building it

A few things became clear while building and using Tool Runtime:

• The registry is only as good as its curation. Pulling in the public MCP directory gives breadth fast, but saving and ranking is what turns a long list into a usable toolbox for a team.
• Generated servers need a human review step. The AI tool creator can take a server from conversation to code, but the embedded IDE and git workflow exist because someone should read what got built before it's published.
• Testing belongs next to configuration. Having the inspector right next to the keys you build removes the gap where a misconfigured header only shows up once an agent fails.
• Telemetry is the foundation for everything else. Usage tracking, rate limits, billing and audit can all build on the same per-call log. Get the log right first.

The gateway is the layer between AI agents and real-world systems. Without it, every new tool adds risk nobody can see. With it, every call is visible, measurable and bounded.

Where to go next

• Watch the full walkthrough on the Tool Runtime MCP Gateway demo page, with a cleaned transcript and chapters.
• See how a large API surface is made usable for agents in routing 2,755 Procore API tools, or start with the basics in what MCP means for construction.
• Read how we connect agents to MCP tools in Construct.Chat.
• If you're weighing how to give agents safe access to your own systems, tell us what you need to build.