← 返回列表

AI Gateway Security: How to Control Models, Tokens, and Tool Calls

单站 varonis_cloud_sec 入库 2026-11-18T06:40:35

AI Gateway Security: How to Control Models, Tokens, and Tool Calls

Rising model costs and tool sprawl share one fix. See how AI gateways typically work, and how a more complete one can control LLM spend and MCP tool use.

"Atlas AI Gateway" in white text on the left with a screenshot of the Varonis Atlas platform on the right.

Key Takeaways

Organizations adopting AI are struggling to answer two basic questions: What tool calls is AI making, and what is it costing? The second is getting tougher by the day.

A Cloudera survey found that 89% of enterprises are limiting or redesigning AI use cases due to rising token consumption — a figure that makes sense given that agentic AI needs 5-30x more tokens per task than a standard generative AI chatbot, according to Gartner. In short, every new agent can multiply a bill that no single team is watching.

Part of the challenge comes down to visibility. Security teams layer tools on top of one another: one for guardrails, one for model access, one for MCP servers, one for monitoring. That’s how AI cost control and AI security end up scattered across vendors, with gaps between them.

What is an AI gateway?

An AI gateway sits between your applications and the agents, models, and tools they use, so that AI traffic passes through a single point. In fact, most gateways tend to focus on managing specific types of traffic: routing requests to models, authenticating callers, rate limiting, logging, and basic content filtering.

That helps teams integrate AI, but it leaves security, cost, and tool control as separate problems, each with its own solution.

Why fragmentation is such a big problem

As adoption grows, model costs and token usage continue to climb, and agents and MCP servers are calling applications and APIs with no central control. Without a gateway, three things can go wrong:

What should an AI gateway do?

A complete AI gateway goes beyond routing traffic. It inspects prompts and responses against policy, and acts when one is violated. It decides which models employees and agents can use, and caps their cost. And it determines which tools agents can access, and what each is allowed to do.

Putting all three in a single control plane means the same policies, identity, and audit trail apply everywhere, rather than each team stitching together its own controls.

How to govern models and MCP servers: 6 steps

Getting to a complete AI gateway solution requires six steps, from discovering what’s in use to monitoring it all from one view.

1. Discover what’s in use. Inventory AI systems, including their models, providers, MCP servers, and agents, and map the sensitive data each one can reach. Policy is only as effective as your visibility into the data behind each request.

2. Register approved models in one place. Register and configure the models you approve, set who can call each one, and allow or block individual models. Have apps call a governed endpoint instead of the provider directly, so developers don’t carry provider keys.

3. Set token limits, budgets, and quotas. Apply limits by token, model, provider, and endpoint, for any user or group. Use per-user caps for individuals and shared budgets for teams, and add alerts so spikes surface early.

4. Apply guardrails to prompts and responses. Inspect what goes into and comes out of a model, and decide what happens when a policy is violated. Guardrails help prevent sensitive data leakage, prompt injection, and jailbreaking.

5. Bring MCP servers and tool use behind one gateway. Choose which tools MCP servers and agents can use, who can use them, and what each tool is allowed to do, such as read-only versus write access. Validate identity through your IdP, grant access by group, use delegated credentials instead of shared standing access, and authenticate and authorize each call before it runs. Declare what each agent is meant to do, so you can flag actions outside its purpose.

6. Monitor and enforce from one view. Track prompts, tokens, spend, and tool calls together, so you can investigate any session and spot spikes. Send events to your SIEM and SOAR tools so AI incidents join your existing response workflow.

AI gateway security checklist

Use the guidance below to determine your coverage. Answer each statement “yes” or “no”:

If you answered “no” more often than “yes,” your AI environment has more blind spots than controls, and the gaps are likely to show up as a surprise invoice, a data exposure, or an agent that did something no one approved. Even a few “no” answers deserve attention, especially for items related to sensitive data, tool authorization, and identity.

What Atlas AI Gateway delivers

Varonis expanded the Atlas AI Gateway with built-in LLM and MCP Gateways, which sit alongside the runtime guardrails Atlas already provides. Together, it makes Atlas AI Gateway a complete solution, with three parts that each close an important gap:

The Atlas AI Gateway is one control point that enables four different AI security outcomes.

The Atlas AI Gateway is one control point that enables four different AI security outcomes.

Plus, because Atlas is built on data classification, the gateways also draw on data context to discover shadow AI across applications, browsers, endpoints, CLI, and IDEs, and to detect threats in real time, including sensitive data exposure, prompt injection, and out-of-policy agent actions.

Altogether, Atlas AI Gateway now supports four AI security and governance outcomes:

LLM Gateway: Track and control model costs and token usage

The LLM Gateway sits between AI applications, agents, copilots, and IDEs and the model providers, such as OpenAI, Anthropic, and Amazon Bedrock. In total, more than 95 providers and 3,000 models are supported. With the LLM Gateway, organizations can see and control model spend and token usage.

The LLM Gateway centralizes access and enables visibility and control over model spend and token usage.

The LLM Gateway centralizes access and enables visibility and control over model spend and token usage.

Complete visibility into model spend and token usage

From the LLM Gateway dashboard, gain full visibility into which models are in use, how many tokens they consume, who is using them, and for what use cases. With alerts and analysis to spot spikes and trends.

The LLM Gateway centralizes access and enables visibility and control over model spend and token usage.

The LLM Gateway centralizes access and enables visibility and control over model spend and token usage.

Control model usage and cost

The LLM Gateway enables control over model spend, token usage, and which models are used for which use case.

Apply policies to model and token usage for users, groups, and use cases. For instance, you can set a $500 weekly budget for your dev team’s Claude Code usage, with alerts as spend approaches the limit.

Policies can also dictate which models are used for which tasks. For example, you can route sales email drafting to a lower-cost model, such as GPT 5.6, instead of GPT 6.1. These policies are applied seamlessly, with no change to the end user’s workflow.

The LLM Gateway enables control over model spend, token usage, and which models are used for which use case.

MCP Gateway: Govern MCP servers and tool calls

The MCP Gateway sits between your agents (AI assistants, custom agents, and CI/automation) and the MCP servers and tools they use, such as GitHub, Atlassian, Salesforce, Slack, and internal APIs. With the MCP Gateway, you can control what agents and MCP servers can access and what actions their tools can take, with identity validated through your IdP.

The MCP Gateway serves as a control point for access to the MCP server and agent tools, authenticated through an IdP.

The MCP Gateway serves as a control point for access to the MCP server and agent tools, authenticated through an IdP.

Complete visibility into MCP servers and agent tool calls

From the MCP Gateway dashboard, see which MCP servers and agents are running in your environment, which tools are active, and who or what is using them.

The MCP Gateway dashboard shows every MCP server and agent in your environment, along with the tools each agent can access.

The MCP Gateway dashboard shows every MCP server and agent in your environment, along with the tools each agent can access.

Control MCP server and agent tool calls

Create curated MCP servers with the MCP Gateway, then specify which tools are authorized, who can use them, and what actions each tool is allowed to perform, such as read-only versus write access. This keeps unauthorized actions from happening.

Validate identity through your IdP, grant tool access by group, and use delegated credentials so agents act with the user’s permissions. Every tool call is tied to a real user identity and recorded in Investigation events alongside prompts, with guardrail blocks visible in the same timeline.

The MCP Gateway lets you control which tools agents and MCP servers can access, and which actions they can perform with each one.

The MCP Gateway lets you control which tools agents and MCP servers can access, and which actions they can perform with each one.

How Atlas AI Gateway maps to the 6 steps above






What it looks like in practice:

A model request: The gateway authenticates the caller and checks which models that identity, whether human or non-human, is allowed to use. It checks the request against usage and spend limits, applies runtime guardrails to the prompt, calls the model, and applies guardrails to the response on the way back. The prompt, response, identity, model, and token usage are logged. If a policy or limit is hit, the request is stopped before it reaches the provider and the app gets a clear error. Developers and users don’t change how they work.

A tool call: A developer uses Claude Code to query Salesforce and Atlassian through MCP. The client connects through the Atlas MCP Gateway, which validates identity through the IdP, uses delegated credentials, and limits the agent to approved read-only tools, blocking potentially destructive actions. Every call is recorded in the investigation timeline, along with prompts.

What each team gets

One control point for models, tools, and guardrails

Because Atlas is built on Varonis data classification, it connects the dots between users, AI agents, tools, identities, permissions, and sensitive data, showing the entire chain of access and the risk it creates. This data and identity context helps organizations understand who can reach sensitive data, through which agent, and enforce the right controls.

AI adoption isn’t slowing down, and security and cost control shouldn’t be the reason it does. Putting prompts, models, and tool calls behind one gateway lets security, FinOps, and platform teams set the rules once and see the results in one place. AI security is data security, and Atlas governs it from one platform.

To go further with Atlas, see how Varonis Browser Protection covers AI use in the browser, and how Agent IBAC keeps agents to their intended purpose.

原文链接:https://www.varonis.com/blog/ai-gateway-security