Two Paths to Agentic Analytics: Anthropic's Curated Garden vs OpenAI's Context Flood
Anthropic and OpenAI both published how they built their internal analytics agents this year. The posts landed months apart, but they converge on the same hard-won insight: raw text-to-SQL is the easy part (halfway solved since years). Where things get hard is mapping a vague business question to the right governed entity across a sprawling data estate. Both built layered context architectures and treat evaluation as a non-negotiable guardrail. The resemblance stops at the architecture diagram.
The how maps directly to each company’s DNA.
The Core Trade-off
Anthropic: “ETL” - canonicalize first, then query. Shrink the plausible answer space before the agent touches anything. Consolidate near-duplicate tables, aggressively deprecate, keep related data together. Let the agent read folders of markdown. As focussed as their product strategy.
OpenAI: “ELT” - contextualize, don’t canonicalize. Keep everything. Invest in automated enrichment and retrieval so the agent can navigate the full estate at 600PB. Crawl pipeline code to understand what tables actually mean. Some would say that some focus might help.
This is the dividing line. If you’re a data person reading both posts, the one you find more compelling says something about your company’s data culture.
Architecture Differences
Anthropic runs a lightweight four-layer stack (data foundations, sources of truth, skills, validation) while OpenAI runs six layers with much more infrastructure between them. Their data strategies are philosophically opposed: Anthropic canonicalizes - deprecate duplicates, one metric per concept - while OpenAI contextualizes - keep everything, find the needle via enrichment and RAG.
Scope
OpenAI covers Engineering, Data Science, Go-To-Market, Finance, and Research across 3,500+ users. The breadth shows in their data estate: 70,000+ datasets across 600PB. You cannot canonicalize your way to cleanliness when half your stakeholders are finance and the other half are research scientists with conflicting metric definitions.
Anthropic focuses on product and growth analytics: revenue, weekly active users, experiment lift, retention curves, funnel analysis. No finance or research.
The real scope question isn’t “which functions are covered” - it’s whether the data estate naturally consolidates or naturally sprawls. OpenAI’s 600PB across 70,000 datasets is a sprawl problem. Anthropic mentions “millions of fields” across hundreds of tables - big, but orders of magnitude smaller. That makes Anthropic’s canonicalize-first approach feasible and OpenAI’s contextualize-don’t-canonicalize approach necessary.
Most companies sit somewhere in between. Your data estate is probably not 600PB, but we also know that the fundamentals - dimensional modeling, deprecating near-duplicates, documenting grain and ownership - are tough to master, too. That’s not a dedicated “curation team”. That’s just good data engineering.
What Both Agree On
These two companies independently reached the same conclusions:
Raw SQL access doesn’t solve the hard problem. Anthropic: “Pointing Claude at a warehouse can create a false sense of precision.” OpenAI: schema lineage and query history only get you so far.
Question-to-entity mapping is the bottleneck, not SQL generation. If the agent picks the wrong table or the wrong metric definition, perfect SQL writes the wrong answer.
Context is the product. Both invested heavily in layered context architectures rather than better prompts or fancier model routing.
Self-improvement loops matter. Autonomous harvesting of corrections, fed back into the system, is table stakes in both approaches.
Less is more on tooling. Both found that prescriptive or overlapping tool surfaces degrade results. Higher-level guidance outperforms step-by-step scripts.
Pass-through permissions. Agent inherits user access controls. No special data access for the AI.
Query history is near-useless as a raw corpus. Anthropic tested this directly and found <1 point gain. OpenAI invests in pipeline code crawling instead of historical SQL as a context signal. Both independently concluded that how data is produced matters far more than how it was previously queried.
Slack and Notion are part of your context stack now. Both companies independently decided to crawl these sources - OpenAI for institutional knowledge (Layer 4: product launches, incidents, codenames, metric definitions) and Anthropic for active correction harvesting (scanning conversations for correction language, auto-drafting doc fixes). Same channels, different purposes, but the convergence is telling: Slack and Notion have become the system of record for both institutional knowledge and feedback loops. If you’re building an analytics agent, you probably need to read them too.
Two companies at war over model capabilities, building completely different architectures, arrived at the same principles independently.
That’s not coincidence.
Where They Diverge
Anthropic’s Skills
Anthropic’s approach is writing procedural knowledge as markdown files - what they call Skills. The concept is straightforward: senior analysts document how to answer common questions, the agent reads those docs on demand, accuracy goes up. (They published a specific number on this - you’ve probably seen it; I’ll spare you repeating it.)
A “knowledge skill” routes to ~30 domain reference docs. An “unbook skill” encodes the senior analyst workflow: clarify, find, query, adversarial review. Reference docs are written for LLM retrieval, not human skimming - they encode grain, scope, gotchas, and routing triggers in a structure the model navigates naturally.
The key insight is that this came from writing markdown files, not building infrastructure.
OpenAI’s Codex Enrichment
OpenAI’s approach is crawling pipeline code - Spark transformations, Python scripts, notebook cells - to understand what a table actually means and how its used, not just what its schema says. This reveals freshness, scope, uniqueness assumptions, and business intent that pure schema metadata can’t convey.
The key example: two tables with identical schemas, where one filters “first-party ChatGPT traffic only” and the other doesn’t. A human reads the pipeline code to know the difference. Codex does this automatically, at scale, with daily refreshes.
What This Means for Data Teams
Documentation as a First-Class Citizen
The most practical takeaway from Anthropic’s approach is how they operationalized maintenance. A few things stand out:
Docs in the same repo. Skill markdown files live alongside the transformation models. The PR that changes a model is the same PR that updates the doc. Makes it harder to ship a reporting model change without touching the corresponding skill file - a CI hook enforces it. Roughly 90% of their data-model PRs now include a skill change in the same diff. They made the maintainable path the default path.
Built with failure in mind. The reference docs structure in their appendix makes this explicit: a whole section called “When Information Is Missing” that spells out what to do for each failure mode. It’s meant to guide the Agent through a mistake instead of letting it diverge from the expected path a Senior Analyst might take.
The decay data point. 95% to 65% accuracy in one month by doing: Nothing.
Simply let the documentation (Claude Skills) drift apart from the real world.
This is the real lesson of their post: agentic analytics is not a build-once system.
Not every business moves as quick as Anthropic’s does, but its a nice illustration about the impact of good and stale documentation (and suddenly you have an incentive to keep it up to date). Context drifts, schemas change, definitions evolve. The only reliable way to stay accurate is to embed maintenance into your engineering workflow - same repo, same PRs, same CI checks. Anthropic’s same-repo setup plus the CI hook is how they solved this. OpenAI doesn’t discuss accuracy decay at all. They have a memory layer that learns from corrections, but it’s not anchored to the data model itself. No CI hook says “update the memory entry for this table” when a table changes. The implicit bet is that Codex’s auto-refreshed enrichment handles freshness - but they haven’t published numbers to prove it.
The correction loop. Anthropic runs a scheduled agent that scans user interactions for correction language. When it spots one - “actually, that metric should filter on active users only” - it drafts a doc fix and opens a PR. No human has to remember to update the docs; the agent does the grunt work and a human reviews. This turns a tedious manual process into an async code review.
The Open Kitchen
The other thing Anthropic does: every response carries a footer with the source tier it came from (semantic layer first, then curated reference, then raw table), how fresh the data is, and who owns the model. And when something goes wrong - missing tables, access denied, outdated docs, unknown enum values - the agent says why.
It’s about giving the consumer enough context to judge trust for themselves. A “raw table, freshness unknown” footer is a signal to verify before forwarding upstream. A “access denied because X” is a reason, not a wall.
IMPORTANT: It turns a silent failure into an explainable one, but it assumes the consumer is savvy enough to make sense of this metadata. (Analyst or Scientist)
The “I Told You So” Part (and I am aware of it)
I’ve argued in this newsletter that data foundations are the real unlock for AI agents (and many others did as well). So it’s nice to see the “big AI boys” applied boring fundamentals. Anthropic’s post is a case study for this thesis - exactly what I predicted in Data foundations are the real unlock. They start with the most unglamorous work imaginable:
“Standard data engineering and data quality practices such as dimensional modeling, shift-left testing, freshness and completeness checks on critical pipelines all still apply (and we won’t relitigate these).”
They don’t invent a new paradigm. They double down on the old one, with one reframe: the end user is no longer a data expert - it’s an agent. A human analyst catches a wrong join or a stale table. An agent can’t. What was “good practice” before becomes “hard requirement” - the same argument I made in Where the Human in the Loop Still Shines.
The payoff is in their numbers. The data foundations layer reduces ambiguity so effectively that Anthropic’s entire agent context layer is just folders of markdown. Complexity lives in the data model, not the agent infrastructure.
OpenAI proves the opposite path is possible, but it’s not easier:
OpenAI, with 70,000 datasets, is either proving a point (you can navigate chaos without taming it) or boiling the ocean (heavy infrastructure where canonicalization would be cheaper - something I’ve been skeptical of as a general strategy).
The answer was clear: giving the agent grep access to thousands of prior SQL files moved accuracy by less than a point. The bottleneck was structure, not access.
The Approachability Gap
Anthropic’s architecture is wildly more approachable for a typical data team. No vector embeddings. No daily pipeline. No RAG. No vector database. Folders of markdown alongside your dbt project. The barrier to entry is “can you write a markdown file (and keep it up to date)?” not “can you operate a vector database?”
OpenAI’s is the opposite - substantial infrastructure that only makes sense if you genuinely can’t canonicalize. True for them. Probably not for you.
Where I Land
Agentic analytics is a context and verification problem, not a code generation problem. SQL is a simple language after all.
How you solve that - curated upfront vs retrieved on the fly - mostly matters at the extremes, but converting your Analytics Magic into .md is probably your safer bet. You’ve already done the hard part. The skills are just the output.
The evaluation problem is where it gets real.
I suspect a significant share of any knowledge role - data, sales, support, whatever - is going to become evaluating agents. Teaching them your expertise so the agent can scale you. The correction loop is the template: an agent watches your interactions, spots where the model got it wrong, and proposes fixes. Over time, it internalizes your judgment. The agent scales you, but you have to teach it first, and keep teaching it. This is genuinely new work - not prompt engineering, not training data labeling. Closer to RLHF, but applied continuously in context as part of your actual job.
As said, I find the 95% to 65% shift more impressive than 25% to 95. Agentic analytics is a maintenance discipline, and the maintenance mechanism is evaluation - feeding corrections back in. OpenAI’s memory layer and Anthropic’s correction loop are both trying to solve this. Anthropic made it a code-review discipline, while OpenAI made it an infrastructure pipeline.
Both are trying to keep an agent aligned with changing definitions.
For most teams, start with Anthropic’s approach - folders of markdown, CI hooks, a simple correction loop. That gets you 80% of the way. Add retrieval infrastructure when the long tail of questions outpaces your curated surface. But the work that compounds is building the evaluation loop. Teaching the agent. Watching what it gets wrong. Feeding that back in.
The good news? For data teams, the foundations work you should already be doing for your human analysts is the same work that makes agents teachable. Dimensional modeling, canonical models, metadata rigor - these compound. They make your humans faster and make your agents possible.