Tool Runtime MCP Gateway

[00:00] All right. Today we are reviewing Tool Runtime, which is an MCP gateway I created. Essentially, the goal is to be able to curate MCPs from multiple different sources into one MCP that you can leverage via API key, to be able to manage and control your MCPs and the users and agents that have access to them. That means having full visibility and capabilities into provisioning the MCPs and the tools necessary for agents, and giving you full control over when they're being called, how they're being called, and what access agents have to these tools. So, giving you full control. Right here you'll see the

[00:45] dashboard, where you can see the total tool calls and the API keys that we have available, and some of the telemetry with the tools that are being called, and just some basic insight into what's happening. Of course there are different capabilities. Specifically, we can export these analytics and dive deeper into some of the specific analytics. But as of right now, what we'll do is dive into the toolbox. So what this toolbox is, essentially, is a registry of MCP tools that are available. They can be searched and/or filtered. As of right now, I'm pulling in the public directory for all of the MCPs, as well as any MCP that's created through the Tool Runtime server. And so you'll see here that

[01:30] ultimately there's a plethora of different tools that we can save and connect with and leverage. When we select the details, we can start identifying the specific endpoints, the tools that are available, as well as the documentation that is available for this. And so we can of course connect to these, and we can then add our authentication to it if necessary. We can save them and rank them. So we can go through and identify which ones are valuable and things that we want to leverage and utilize. But essentially, once these are saved, we can connect to them, and then we can leverage these when we are going through and creating an MCP for the consolidation of this. Before we do that, essentially what we did is put

[02:15] together an AI tool creator. This tool creator essentially allows you to chat with an AI chatbot that walks you through the entire process, start to finish, for how to create an MCP. Essentially what it does is it goes through a question and answer, identifying specific resources and documentation. It asks you questions to identify specifically what you're looking to build, and then it takes you through these different steps for the specification, architecture, design, and then ultimately will automatically code it and submit the repo. And so you'll see down here there are a few different ones that are already created. And what essentially happens is, when we open this, we have an IDE that's embedded in the browser. And this is

[03:00] essentially the code that the agent developed. And so the user can come in here and identify specifically what capabilities were implemented from that API, or from the AI agent, and they can go through and edit and update these. And then of course, as you go through and do your git request, we can amend our initial commit, and we can even then update it to GitHub. So when we come back into here, we'll see our GitHub that's available, and this is automatically posted. Then, once this is posted, we have our Tool Runtime [listener]. It essentially goes through and listens for this, and then ultimately lists it onto the registry if the user wants to submit it publicly. And so this is the entire

[03:45] registry for all of the tools that you've created, and there is also management of these tools at an organization level. So once you have created your tools, and you have pulled ones from the registry that are available, essentially what we need to do is build the MCP, i.e. the API key that leverages these tools. So when we come into here, we can select the MCPs that we have saved, and what we do is we select the ones that we want to leverage and create the MCP for. If we need more insight, or if we need to edit the authentication or headers or test various things, we can. But essentially what we do is we select the MCPs that we want to use, and we can then

[04:30] press next. And then what this does is show all the tools that are available for this MCP, and we'll select the ones that we want to leverage, and then we'll press next. And so this then says, hey, we're utilizing these two MCP servers with the authentication that was provisioned, and we want to specifically give the agent the ability to leverage these specific tools. And so then we can name this, "test MCP," and [select the authentication] method that it will leverage as the MCP authentication. So whenever this API key is used as an MCP tool, it will go through and authenticate against it and know exactly what tools and the

[05:15] authentication methods are associated with those MCP servers that are selected. And so you'll receive the entire MCP.json that you can then embed and copy into your client, whether it be Cursor or Claude or anything like that. And so essentially what's happening is that we created a streamable HTTP MCP server. This MCP server always points to toolruntime.com/mcp. And what we do is we provide the authentication method. And so this API key is essentially what tells Tool Runtime what tools it has access to, and what capabilities in terms of authentication, as well as rate limiting and various other capabilities. And so what we do is

[06:00] we can copy this and then go through and manage all of our API keys. And so here are all of the various API keys that we have created, and we can go through and identify the different credentials for them. We can view the key that we have access to, or get the config again. And then ultimately, when we come into here, we'll be able to edit the servers. We can edit the tools that are associated with it, modify the rate limits, and basically have different ways to manage our API keys. So essentially what's happening is we're giving these MCP server API keys to a plethora of different agents, and then we're able to go through and track various different capabilities from that. And so we do

[06:45] have different ways to test the MCP authentication. So we have an inspector tool. Once it loads, and it might take a moment, essentially what it does is it loads in our MCP, and then it has the specific tools that are associated with it, and then we can just do test calls to it to inspect it. And so this might take a moment to load, but we can always come back to it also. Oh, there it goes. And so, essentially, yeah, we can go through and select any of the keys that we have available. And then we can do our authentication and test all the tools there. We do have AI agents where we can then test this MCP in an agent. So we can actually do real-life calls as well and build out agentic

[07:30] systems. The goal is to implement workflows to embed the MCPs into the AI workflows. And then of course, the main thing is diving into the tool insights. And so, as we're looking at the actual tools, we can see all of the tool calls and the associated information that is happening. So you'll see here that these requests are identifying, you know, the organization, the user, the API key, as well as the MCP server and the input/output logs. And so this is really important. Whenever a tool call is happening, essentially we take the input request, then the tool call will happen, and then we take the output response. This is great for debugging

[08:15] and identifying how it's functioning, and having granularity into this. And so this is just the tip of the iceberg to gain insights and analytics into the specific capabilities of the MCP tools. Of course, there are different capabilities in terms of logging tool calls, and policies that we can implement in terms of what's allowed, how many tools are available, how frequently they can be called, and then identifying usage for these. And so this would be associated with the usage, but then we ultimately can go into the data and analytics here. And so the data and analytics, ultimately, I

[09:00] don't have any here in the last 30 days, I guess. But essentially what it does is it'll have the tools and all of these servers that have been called, and then we can view the analytics for this tool, and we can pull in any of the analytics and requests and track all of the tools, and the graphs and success rates for each of these API keys, i.e. the MCP tools that we've provisioned and given access to. Of course, we have a plethora of different resources that are available, like getting started, FAQ, different ways to contact us, and our community resources. And then further, there is an admin portal here, and this then allows us to go

[09:45] through and manage the various users that we have. We can invite people, we can give access to specific capabilities, and provision capabilities for organizations, as well as admin rate limits for specific MCPs that are available and utilized. And then of course, on the back end here, these are just the platform analytics for organizations, to be able to identify what's occurring. And then essentially we do have user access here. And so we can then look into the profile for this user. Each user has a profile, and so they can go through and create an about section, add links, see their whole analytics, and then ultimately identify the different tools that they've created. So different

[10:30] users could go through and find other users and the tools that they've created, and utilize those within their own instance if they have a publicly available profile. You can then ultimately see the analytics for these, the activity that these users have had. So anytime they've done anything, we can ultimately track the activity and usage of the platform, and then of course their network. So if you're inviting users or collaborating with one another, we can identify, you know, different status updates or different learnings that people have had. And then ultimately, editing your settings to pull in your GitHub repositories, or even change your privacy settings, if you're

[11:15] maybe not wanting to have specific visibility into who can see your profile. You can of course change that. But yeah, this is the tip of the iceberg, and I hope that you find value and interest in this. So yeah, thank you very much, and let me know if you have any questions.