TL;DR
- Microsoft Fabric now lets you call an AI model from a single line of T-SQL (
AI_EXTRACT,AI_CLASSIFY,AI_SUMMARIZE) – reasoning runs where your data already lives, in preview today. - This inverts 30 years of data architecture: the warehouse stops being storage and becomes the substrate for AI agents.
- It’s an industry-wide shift (Snowflake, Databricks, BigQuery too); Microsoft’s edge is coherence – Fabric, OneLake, and Purview as one governed stack.
- It’s powerful but not free: watch cost-in-the-query, non-repeatable answers, and silent wrong values. Treat it as a data-platform discipline.
- The deeper shift is human: when the warehouse reasons, people move from deciders to supervisors – and judgment becomes the scarce skill.
For thirty years, the deal was simple: data sat still, and intelligence came to it. You exported rows, shipped them to a model somewhere else, scored them, and wrote the answers back. The warehouse was a filing cabinet – fast and expensive, but still just a place to store things.
That deal just broke. And a single line of SQL shows why.
SELECT AI_EXTRACT(review, 'sentiment', 'product', 'urgency') FROM feedback;
Run that in Microsoft Fabric and the warehouse itself calls a large language model, reads each customer review, and returns clean columns – sentiment, product, urgency – with no pipeline, no export, no second system. (AI_EXTRACT is one of a handful of new AI functions you can now write directly in T-SQL, alongside AI_CLASSIFY, AI_SUMMARIZE, and AI_GENERATE_RESPONSE. They are in preview as of mid-2026.)
That looks like a convenience. It isn’t. It’s an inversion of how we’ve built data systems for decades – and it quietly turns your most boring asset into the foundation for AI agents.
The old bridge vs. the new substrate.
A shift disguised as a feature
We’ve seen this shape before. In 1979, the spreadsheet didn’t just make accountants faster – it handed financial modeling to every manager and rewired who was allowed to ask “what if?”. The big shifts always arrive looking like small conveniences.
In-database AI is that kind of moment. For two years, every enterprise AI project has been a bridge: copy data out of the governed warehouse, into a separate vector store, into an orchestration tool, and hope the security model survives the trip. Every hop is a place where lineage breaks and a compliance officer loses sleep.
Putting the model inside the query removes the bridge. Your access rules, sensitivity labels, and lineage keep applying, because the reasoning happens where the data already lives. For the first time, “AI-ready data” and “governed data” can mean the same thing.
This isn’t only a Microsoft story, and honesty matters here: Snowflake, Databricks, and Google BigQuery all shipped their own versions of model-calling SQL within about eighteen months of each other. When every serious platform makes the same bet at once, it’s not a feature race – it’s a paradigm shift. What Microsoft adds is coherence: Fabric for the reasoning, OneLake for one governed copy of the data, and Purview for lineage – one stack instead of four procurement cycles.
Perceive, reason, act – the agent loop, running inside the warehouse.
From a function to an agent
Here’s where it gets interesting. Compose those functions and they stop being a feature.
Picture a support ticket. One query classifies its priority, extracts the product and the customer’s intent, and drafts a reply – three model calls, no data leaving the warehouse. (This is an illustrative scenario, not a specific customer.) Wrap that in a stored procedure, trigger it whenever a new ticket lands, and you have a system that perceives (the ticket arrives), reasons (the model calls), and acts (routes and drafts). That loop – perceive, reason, act – is what we mean by an agent.
You didn’t deploy an agent framework. Your warehouse became an agentic substrate, because the reasoning moved to where the data and the triggers already were.
But be clear about the limit: a stored procedure on a trigger is not a full agent. It has no real planning or memory, and it can’t choose its own tools. The moment you need genuine multi-step autonomy, you graduate into a dedicated orchestration layer – Azure AI Foundry, or the Microsoft Agent Framework (the successor to Semantic Kernel and AutoGen, and now Microsoft’s primary path for multi-agent systems). The warehouse is where simple, governed, high-volume agentic work starts – not where complex autonomy lives. Knowing the difference is the real skill.
The honest challenges
This is powerful, but it is not free, and any leader should weigh the costs before betting on it:
- Cost hides in the query. A model call is far more expensive than a normal lookup, and it bills against your Fabric capacity. One careless query across millions of rows can quietly burn through your budget. Microsoft’s own advice: don’t run AI functions repeatedly inside
SELECTs – precompute and store the results. - Answers stop being repeatable. A normal query returns the same result twice; a model-backed column might not. Caching and pinning the model version become real engineering tasks.
- Wrong answers arrive quietly. A broken query throws an error you notice. A model that misreads a ticket returns a confident, well-formatted, wrong value – and the pipeline keeps running. You need spot-checks and a human reviewing anything that matters.
- It’s still preview. These functions are evolving fast. Treat them as preview, watch the release notes, and re-test when the underlying model changes.
None of these are reasons to wait. They’re reasons to treat in-database AI as a data-platform discipline – with the same rigor you already apply to cost, quality, and governance.
The quieter shift: who decides now?
Step back, because there’s a second inversion hiding inside the first. For decades the warehouse served the decision, and a human made the call. When the warehouse reasons, that flips: the system reaches a conclusion, and the human’s job becomes to supervise it – to approve, audit, and catch the confident mistake.
That isn’t bad. A clinician who lets a model triage a thousand charts so she can focus on the twelve ambiguous ones is doing better medicine, not worse. But the scarce skill changes. It’s no longer writing the query – the warehouse can do that. It’s judgment: knowing which machine answers to trust, which to question, and which to overrule. And accountability can’t move into the infrastructure: a model doesn’t get fired for a bad call – a person owns the outcome.
Where to start
If you want to move on this without getting burned, a practical path:
- Pick one narrow, high-volume text task – ticket triage, review tagging, document classification – where a wrong answer is cheap to catch.
- Prototype it as a query, then materialize the results into a table instead of computing live in every
SELECT. - Add a human checkpoint and a small evaluation set so you can measure accuracy before you trust it.
- Put cost guardrails in place – capacity alerts and query review – before you scale.
- Keep a human owning the decision for anything consequential.
Start small, measure honestly, expand what works.
The takeaway
Stop asking how to bring AI to your data. Your data just learned to think – the warehouse is no longer where intelligence ends up, it’s where intelligence happens.
Follow that trajectory and the destination comes into view: the data estate becomes a system of cognition, making millions of small judgments a day across every enterprise. The decade ahead won’t be won by whoever has the smartest model. It’ll be won by whoever is wisest about which decisions to delegate to the machine – and which to keep human.
Data in. Intelligence out. The cabinet learned to think; the only question is whether we stay wise about what we let it think for us.
FAQ
Does this replace my data pipelines and vector database? Not wholesale. In-database AI removes a lot of copy-out-and-process work for text tasks, but you’ll still use pipelines for movement and vector search for semantic retrieval. It shrinks the bridge; it doesn’t delete every use case.
Is it production-ready? The T-SQL AI functions are in preview as of mid-2026. Prototype and pilot now; confirm what’s generally available before you make a critical workload depend on it.
Won’t running a model on every row be expensive? It can be. AI calls bill against your Fabric capacity, so precompute and materialize results instead of calling functions live inside SELECTs. Put capacity alerts and query review in place before you scale.
Can I trust the outputs? Treat model outputs as high-quality suggestions, not ground truth. Add evaluation sets and a human-in-the-loop for anything consequential – a confident wrong answer is the real risk.
Does my data leave my environment? The text is sent to a governed model endpoint (in Fabric, an Azure OpenAI endpoint that per Microsoft doesn’t log or store your prompts or outputs), and the result returns as a column. Your access rules and lineage keep applying because the call originates inside the estate.
Go deeper
- AI functions in Fabric Data Warehouse (T-SQL) – Microsoft Learn: the
AI_CLASSIFY/AI_EXTRACT/AI_SUMMARIZEreference, syntax, limits, and the “materialize your results” guidance. - AI functions overview (pandas & PySpark) – Microsoft Learn: the same reasoning from notebooks, including multimodal inputs and model configuration.
- Microsoft Agent Framework overview – Microsoft Learn: where to go when you outgrow a single query – the successor to Semantic Kernel and AutoGen.
Originally published at next2know.blog. Harry Arce is a Data & AI Advocate, Microsoft Digital Solutions Engineer, and Postgrad Data and AI Instructor. He writes “Data & AI – Applied” – data in, intelligence out.
Leave a Reply