Microsoft has announced the public preview of its new AI Gateway tier of Azure API Management, marking an important step forward in how organisations manage, govern and observe enterprise AI workloads.
For Adaptiv, the announcement also represents a significant milestone in our collaboration with Microsoft.
Under a co-development agreement, Adaptiv received early access to AI Gateway while it was still being developed. This gave our integration and AI specialists the opportunity to evaluate the service before public preview, test practical enterprise scenarios and provide structured feedback directly to Microsoft’s product team.
A foundation for governed enterprise AI
The public preview introduces a central experience through which platform teams can manage models, MCP servers, policies and observability destinations.
Its capabilities include support for models hosted through Microsoft Foundry, as well as providers such as AWS Bedrock, Google Vertex AI, OpenAI and Anthropic. Governance controls include token rate limits, quotas, Azure AI Content Safety and fallback to secondary models.
The gateway also extends governance to the tools used by AI agents. Organisations can expose existing MCP servers, convert operations from REST APIs into MCP tools using OpenAPI specifications, or use connector-backed tools from the Power Platform and Azure Logic Apps ecosystem.
Together, these capabilities point towards a future in which organisations can give development teams access to approved AI models and tools while maintaining central guardrails, access controls and visibility.
“Microsoft already had API gateways capable of performing some AI request governance, however the key difference is that a traditional API gateway understands an HTTP transaction, an AI gateway understands an AI interaction. The problem is that conventional gateways do not natively understand tokens, prompts, completions, models, context windows, semantic similarity or AI safety risks. AI gateways provide centralised, model aware control over how applications and agents securely, safely and cost-effectively access AI models and tools.”
A co-development partnership
Earlier this year, Adaptiv and Microsoft entered a co-development agreement that gave our engineers early access to the AI Gateway well ahead of its public unveiling. The goal was simple: put the product in the hands of a team that lives and breathes real-world integration, API management, and enterprise AI adoption – and let them stress-test it.
Through a private GitHub project, our engineers worked directly alongside Microsoft’s product team:
- Raising bugs as we found them
- Filing feature requests grounded in customer scenarios
- Providing structured feedback on the developer experience, onboarding flows, and enterprise readiness
“We wanted to test how Microsoft AI Gateway could help us manage access to AI models and tools, using real-world scenarios relevant to our customers. This included testing connections to GitHub Copilot, Azure Automation, storage services and OpenAI models, while sharing feedback with Microsoft to help improve the platform.”
That feedback loop is exactly what a preview program is designed for and it’s the kind of work Adaptiv is uniquely positioned to do, given our depth in Azure Integration Services and our track record of taking new Microsoft platforms into production.
“Partners like Adaptiv give us real enterprise integration scenarios to test against long before general availability. That kind of structured feedback during a preview is exactly what shapes the product before it reaches the broader market.”
What stood out in our testing: observability
If there’s one capability that impressed our team consistently across every scenario we exercised, it’s observability.
Out of the box, the AI Gateway gives platform teams the kind of visibility into AI traffic that would traditionally take weeks of custom instrumentation to build:
- Token metrics per consumer, streamed into Application Insights with custom dimensions
- Prompt and completion logging to Azure Monitor for auditing and billing
- Built-in analytics dashboards for spotting consumption patterns across APIs and teams
- End-to-end traceability from client, through the gateway, to the model backend
For any organisation trying to answer, “who is using our models, how much, and at what cost?” this is a genuine step change.
A preview, and a partnership, that’s just getting started
A public preview is not the end of a product’s development. It is an opportunity for a broader community of customers and partners to test the service, validate its direction and help shape what comes next.
Adaptiv is proud to have contributed during the earlier stages of that process. Our involvement reflects the combination of integration expertise, Azure capability and practical AI experience that we bring to our customers.
For Nikolai, the highlight was the team’s opportunity to “verify the capabilities of the AI Gateway in real world scenarios” and share feedback with Microsoft to help improve the product for customers.
As organisations move from AI experimentation towards production adoption, successful outcomes will depend on more than choosing a model. They will require strong integration foundations, effective governance and clear operational visibility across the entire AI estate.
Microsoft’s AI Gateway is an important development in that journey, and we look forward to continuing our work with Microsoft as the platform evolves.
The AI Gateway tier is currently available at no cost in public preview in the East US 2 and Sweden Central Azure regions. Read Microsoft’s public preview announcement for further details.
Ready to explore what the AI Gateway can do for your business?
Adaptiv has been building on the AI Gateway since before it was public. If your organisation is planning its AI governance strategy or already grappling with token sprawl, ungoverned model access, or observability blind spots, we’d love to help you cut months off the learning curve.
Talk to Adaptiv about building the data, integration and AI foundations needed to move forward with confidence.
Adaptiv is a Microsoft partner specialising in Azure Integration Services, API Management, and enterprise AI enablement.