We recently built an agent that lets business users ask Microsoft Copilot a plain-English question, like which contracts with a given vendor are still active, and get a real, cited answer pulled straight from a Redshift warehouse and a pile of SharePoint documents. No SQL, no ticket to the data team, no waiting. That project is the reason we're writing this: Azure AI Foundry is the platform that made it possible, and most of what gets written about it online stays at the marketing-slide level of detail.
If you find yourself wondering more about the Azure AI Foundry setup and capabilities, this article is what you’ll want to read before you get started. It covers what Azure AI Foundry is, breaks down its 4 core capabilities, and walks through the proof of concept we built end-to-end, including the parts that didn't work on the first try. If you're evaluating Foundry for a real project rather than a demo, this is meant to save you from a few of the mistakes you may make along the way.
A quick naming note: Microsoft began transitioning the "Azure AI Foundry" name to "Microsoft Foundry" starting January 1, 2026. The underlying platform, pricing, and capabilities did not change; only the name and some URLs did. This article uses Azure AI Foundry throughout since that is still the name most teams recognize, but you can expect to see Microsoft Foundry in newer Microsoft documentation and billing.
What Is Azure AI Foundry?
Azure AI Foundry is Microsoft's end-to-end platform for building, deploying, and operating AI agents in the enterprise. Think of it the way you'd think of Synapse or Databricks, except the workload is agentic rather than data engineering. Foundry absorbs the complexity of hosting models, wiring tools, and monitoring agent behavior, so the team can stay focused on the data and the outcome instead of the plumbing underneath it.
Core principle: an agent is not just a chatbot with better manners. It is a large language model combined with instructions, tools, and memory, which means it reasons over context, decides which tool to use, and acts, rather than only responding.
Foundry is built around four core capabilities:
Model hosting
An agent builder
Tool connections
Observability
Underneath all four sits enterprise-grade security, native integration with the rest of the Azure data estate, and a configuration model that does not require retraining a model every time something changes.
Core Capability 1: Model Hosting
Foundry deploys and manages large language models, including Anthropic, OpenAI, and DeepSeek models as well as embedding models, alongside other foundation models, all inside a single governed environment. Rather than standing up separate infrastructure for every model a team wants to try, Foundry gives you one place to host, version, and swap models as better options become available.
This matters more than it sounds. Teams that build directly against a single model tend to get stuck with it, because ripping it out means rewriting the integration. Hosting models inside Foundry keeps that choice open, so a team can move to a newer or cheaper model without rearchitecting the agent around it.
Core Capability 2: Agent Builder
Agent Builder is where an agent's behavior gets defined: what it can do, which tools it can reach, and what guardrails constrain it. All of that is configured through a visual and interactive interface, which means no ML engineering background is required to set one up.
An agent, in Foundry's terms, is an equation: LLM + instructions + tools + memory, along with the capability of MCP connections. Instructions cover the system prompt, routing logic, and formatting rules. Tools are the external systems the agent can call. Guardrails cap what the agent is allowed to do, things like a maximum number of API calls per question or a whitelist of allowed operations. Change any of those, and you are editing a configuration, not retraining a model.
Core Capability 3: Tool Connections
Tool connections are how an agent reaches outside its own reasoning to touch real enterprise data. Foundry standardizes this through MCP or OpenAPI, the same standard most enterprise APIs already speak, so if an API can be described in OpenAPI, a Foundry agent can call it without custom integration work.
Two patterns cover most enterprise use cases. An OpenAPI tool handles structured data: the agent sends a JSON request, a governed API gateway translates it into a query, and a database or warehouse returns results. A knowledge base tool handles documents: the agent queries a search index and gets back the relevant content chunks, not entire files. Neither pattern requires the agent to touch the underlying system directly, which is what keeps the connection safe rather than just convenient.
Core Capability 4: Observability
Observability is Foundry's answer to the most common objection to putting an agent in front of real business decisions: how do you know what it really did? Foundry Traces surfaces the agent's reasoning steps, the exact JSON payload sent on every tool call, exactly what came back, which document chunks a retrieval returned, and token usage and latency at every step.
Foundry also includes an evaluation tab for running structured tests against agent behavior, which matters most when instructions change and you need to catch a regression before it reaches a business user. Full visibility into every agent action and decision is what turns an agent from a black box into something a governance team can confidently sign off on.
How It All Connects and Works
A working Foundry deployment usually breaks into four layers:
User layer: Microsoft Copilot or Teams, where the question originates. No new tool adoption required, since it is an interface people already have open all day.
AI orchestration layer: the Azure AI Foundry agent itself, the reasoning engine that decides what to do with the question.
Services layer: a governed API gateway, typically an MCP server or api calls via Azure Function App, for structured data, and Azure AI Search for indexing and retrieving documents.
Data layer: the systems of record, whether that is a data warehouse like Amazon Redshift or a document store like SharePoint Online.
Two paths run through that stack. A structured data path handles anything that lives in tables. A document path handles anything that lives in files. The agent's routing logic decides which path a given question takes: a question about contract document content routes to the document store, while a question about financial or structured figures routes to the warehouse. All that routing lives in Foundry's visual configuration.
For document questions specifically, Azure AI Search chunks and indexes content on a recurring cycle, retrieves the relevant chunks when a question comes in, and hands them to the model as context. The model then answers based on that real content rather than its training data, and every answer can be traced back to a source document and its last-modified date. It’s all grounded, cited, and traceable.
Worked Example: Connecting Copilot to Redshift and SharePoint
Here is what this looks like end-to-end, drawn from a real proof of concept we built.
The setup: A business user needs answers from a mix of structured data (contracts, financials, government relations activity) and unstructured documents (contract files, agreements) sitting in SharePoint.
The goal: ask in plain English, get an answer sourced from the right system, and stay fully governed the entire time, using no SQL, no SharePoint navigation, and no freeform queries.
The agent: GPT-5.2-chat as the reasoning model, a system prompt covering routing logic and formatting, an API or MCP connection to the data warehouse, and a knowledge base connection to SharePoint. Guardrails capped the agent at three API calls per question, restricted aggregations to a small whitelist of functions, and enforced row limits on every query.
A structured data question, step by step: "Show me active contracts with a specific vendor."
The user asks questions in plain language.
The agent reasons about which dataset and filters apply: contracts, filtered by vendor name and active status.
The agent calls the API with a structured JSON payload describing the dataset, the filter, and a result limit.
The Function App in this case, validates the dataset request, auto-quotes column names, and builds a parameterized SQL query. No freeform SQL ever reaches the warehouse.
SQL executes against the warehouse, and results return as JSON with metadata attached.
The agent responds in natural language, with the source table cited.
Each of those queries is parameterized, capped to a lookback window, and logged. That combination, plain language in, governed SQL out, is what makes this safe to put in front of business users who have never written a query in their life. In this build, the agent had access to twelve curated datasets spanning contracts, funding activity, campaign performance, and government relations contacts. Adding a thirteenth dataset is a configuration change, not a model retrain.
Let’s talk governance: Azure Function Apps have dynamic outbound IPs by default, and most warehouse security groups, Redshift included, expect a fixed, whitelisted IP. Left alone, that mismatch produces intermittent, hard-to-diagnose connection failures. The fix: route the Function App through VNet integration to a NAT Gateway, which gives it a fixed, static egress IP the warehouse can whitelist. The Function App is never exposed to inbound public traffic; it only calls out. Worth planning for before a cross-cloud pilot, not after.
What kept it safe end-to-end was a dataset whitelist, so only explicitly configured data is queryable; no freeform SQL, ever; row limits enforced at the API level; an aggregation whitelist limited to SUM, COUNT, AVG, MIN, and MAX; a cap on API calls per question to prevent runaway loops; function key authentication on every call; and full debuggability, since every query can be inspected as generated SQL.
Key Advantages of Azure AI Foundry
No ML engineering required to configure agents: Routing logic, tool connections, and guardrails are all set through a visual interface.
Governed by design: Whitelisted datasets, parameterized queries, and row caps keep an agent from becoming an open door to enterprise data.
Full observability: Every reasoning step, tool call, and retrieval is traceable through Foundry Traces, not a black box.
Familiar interface for end users: Landing in Microsoft Copilot and Teams means no new tool adoption curve or additional configurations.
Modular architecture: Swap the warehouse or document store and the pattern still holds. This build used Redshift and SharePoint; the same design works with Snowflake and Confluence.
Considerations and Limitations
Cross-cloud networking needs planning. Dynamic outbound IPs and fixed-IP security groups on the other side do not resolve themselves without a NAT Gateway pattern in place ahead of time.
Credentials belong in a service account and a proper secrets vault, not a personal login or application configuration, well before anything reaches production.
Retrieval quality depends on indexing cadence. If documents are indexed on a daily cycle, an agent answering a document question is only as current as the last index run.
Cost has multiple moving parts, covered in more detail below, which means budgeting means watching more than one meter.
Pricing Overview
Azure AI Foundry, the platform itself, does not carry its own subscription fee. What you pay for is the underlying Azure services it orchestrates, which breaks into a few categories:
Model inference: Billed per token, with pay-as-you-go pricing for most deployments and provisioned throughput units (PTUs) available once volume is high and predictable enough to justify reserved capacity. As a reference point, GPT-4o pricing has run in the range of a few dollars per million input tokens and roughly triple that per million output tokens on pay-as-you-go, though exact rates vary by model, region, and change over time, so always check the current Azure pricing page before budgeting. Smaller models can cost a small fraction of that per token, which is why many production deployments route simple tasks to a lighter model and reserve larger models for harder questions.
Azure AI Search: Production workloads generally need at least the Standard tier, priced per search unit per month, since the free tier is not built for enterprise-scale indexing or query volume.
Compute for the API gateway: The Function App (or equivalent compute) that validates requests and builds queries is billed on its own consumption, typically modest relative to model inference costs.
Networking: A minimal cost incurred for NAT Gateway and VNet integration for cross-cloud connections, plus any data egress charges when traffic crosses cloud boundaries.
Observability: Application Insights and Foundry Traces logging, which scales with query volume.
Realistic total spend varies a great deal by scale: small pilots often land in the low thousands of dollars per month, while mature enterprise deployments with sustained volume can run considerably higher. The most common budgeting mistake is pricing model inference alone and treating everything else as an afterthought. Search, networking, and observability are not optional line items in a production deployment, and they add up.
Yes, another naming note: because Microsoft is mid-transition from Azure AI Foundry to Microsoft Foundry branding, cost line items in your Azure bill may start appearing under the new name even though nothing about what you are actually paying for has changed. Worth a heads-up to your finance team so an invoice line rename doesn't read as a surprise new charge.
Keyrus & Microsoft
Deploying a Foundry agent is the easy part. Making it trustworthy enough for a business user to act on its answer without checking it twice is the actual work, and it is where most Foundry pilots either compound into something durable or quietly stall.
At Keyrus, we call that discipline being an Architect of Intelligence: designing the Operating System of the intelligent organization, where AI is not bolted onto a workflow but embedded into how the business already runs. We operationalize intelligence.
Our proprietary Human Orchestrated Model™ (HOM™) frames how we approach a build like the one in this article, across three layers:
Intelligence Foundations. The governed datasets, semantic definitions, and document indexes an agent inherits. An agent is only as trustworthy as what sits beneath it.
Human in Command. Guardrails, routing logic, and review steps that keep people setting intent and approving outcomes while the agent executes.
Performance Steering. Measuring whether the agent significantly shortened a queue, not just whether it shipped. Cycle time, not token count.
Technology amplifies. Culture and governance determine whether that amplification compounds into something durable or evaporates the first time an answer turns out to be wrong.
As a Microsoft partner, Keyrus helps organizations move from a Foundry proof of concept to a governed, production-ready agent that a business user can trust on the first question, not just the demo. Learn more about our Microsoft partnership or contact us to start your Azure AI Foundry rollout.
Azure AI Foundry is Microsoft's end-to-end platform for building, deploying, and operating AI agents in the enterprise. It combines model hosting, a visual agent builder, OpenAPI-based tool connections, and observability into a single governed environment, so teams can stand up an AI agent without a dedicated ML engineering effort.
Yes. Microsoft began renaming Azure AI Foundry to Microsoft Foundry on January 1, 2026. The platform, pricing, and capabilities did not change, only the name and some URLs did. Both names currently refer to the same product, and older documentation and Azure bills may still reference the original name.
A large language model reasons over context and generates a response. An agent built in Foundry adds instructions, tools, and memory on top of that model, which lets it decide when and how to act, whether that means querying a database, searching a document store, or answering directly without calling a tool at all.
Through OpenAPI-based tool connections routed via a governed API gateway, typically an Azure Function App. The agent sends a structured request, the gateway validates it and builds the actual query, and the agent never touches the underlying system directly. Connections to services outside Azure, such as Amazon Redshift, generally need a fixed egress IP through a NAT Gateway.
For document-based questions, Foundry uses retrieval-augmented generation: Azure AI Search retrieves the specific chunks relevant to a question and hands them to the model as context, so the answer is grounded in that content and cited to a source document. For structured data, answers come from real, parameterized SQL results rather than a generated guess.
No. Routing logic, tool connections, and guardrails are all configured through Foundry's visual Agent Builder. Adding a new dataset, adjusting a guardrail, or changing how the agent formats a response is a configuration change, not a model retraining exercise.
Common controls include dataset whitelisting, banning freeform SQL in favor of parameterized queries, row limits, an aggregation function whitelist, a cap on API calls per question, and function-level authentication. Foundry Traces adds full observability into every reasoning step and tool call an agent makes.
Azure AI Foundry itself carries no separate subscription fee. Costs come from the underlying services it orchestrates: per-token model inference (pay-as-you-go or provisioned throughput), Azure AI Search, compute for the API gateway, networking for cross-cloud connections, and observability logging. Total spend scales with usage, so it should be modeled across all five categories, not model tokens alone.
Yes. A Foundry agent can reach a data warehouse on another cloud through an OpenAPI tool and a governed API gateway. The main technical hurdle is networking: Azure Function Apps use dynamic outbound IPs by default, so a NAT Gateway with a fixed static IP is typically required to satisfy the warehouse's security group whitelist.
At minimum, move any credentials from personal accounts to service accounts, store secrets in a proper vault rather than application configuration, and resolve any intermittent connectivity issues in the indexing or retrieval pipeline. Treat these as a pre-launch checklist rather than something to discover after go-live.