Microblog

Is your org planning for the next agent platform shift, or locking itself in with a specific provider?

My org has rolled out Claude Code widely in the last month or so. Before that, we had GitHub Copilot, Cursor, and Augment. I have watched my peers shift their work to build specific tools that are locking us into the Claude Code platform; Skills with Claude-specific frontmatter fields and tool names, Claude hooks, CLAUDE.md everywhere.

For my personal stuff, I have been keeping things tool agnostic because I have switched agent platforms 3 times since I started messing around with these things. Any given platform starts off affordable for experimentation, then realizes it can't keep subsidizing token costs and raises its prices. We've been seeing Claude Code do that in the last few weeks, and I've had other providers do that before. So I'll probably switch my personal stack again soon.

But for the org, the switching cost is a lot higher. Especially as more and more employees build things that are optimized for a specific agent platform. I'm really curious if any orgs are actively monitoring and discouraging vendor lock-in, or if everyone is just going to have to pay the price when the invoice numbers change. It seems like the big players are counting on the lock-in, but I'm curious if any orgs are actively trying to keep their options open or if that's really only feasible on an individual level.

I just shipped v0.5.0 of the Agent-Friendly Documentation Spec and updated the AFDocs scoring tool to match. Major thanks to all the community members who participated in discussions, filed issues or PRs on the spec or the tool, and helped shape the improvements and bug fixes!

If you've been tracking your documentation site's agent-friendliness scores, they're going to change. The scoring is more accurate now, and "more accurate" could be an upside or a downside for your site.

In smoke testing across sites at every grade level, I saw:

• Every site's content discoverability score dropped. The tool now checks whether your llms.txt directive appears in both HTML and markdown formats separately, instead of lumping them together. Most sites only have it in one format, so that's a new failure where you might have gotten a pass or a warning.
• Page size and content start position scores generally improved. The tool now strips scripts and styles before measuring, matching recent updates to how Claude Code processes your pages.
• One major enterprise docs site went up a full letter grade (D to C) because they've added real markdown URL support since I last ran the tool against their site. Their llms.txt links changed from .html to .md, and their server now returns actual text/markdown at those URLs. While I haven't spoken directly with anyone there, the timing lines up with the spec's publication and the tool's growth in visibility, so I'd love to think they found the tool helpful and have been making improvements. Wins like this are good news for docs and for developers using agents.

Overall scores moved between -10 and +8 points. Most changed by 5 or fewer. Only 2 of 6 sites changed grade letters, one up and one down.

If you're using afdocs, I’ll throw a link to the migration guide in the comments with details about the scoring changes and what to expect.

Links from the comments:

• https://afdocs.dev/migration/v0.17.0

The Agent-Friendly Documentation Spec just got a version bump! 🎉

Spec v0.4.0 reflects a handful of community requests, including:

• Renaming the llms-txt-freshness to llms-txt-coverage to more accurately capture intent
• Noting that llms-txt-coverage may intentionally diverge from full sitemap parity for docs sets that curate their coverage, and makes allowances for different coverage measurement modes
• Updating markdown-content-parity for a similar purpose; some sites intentionally omit info from markdown pages that aren't relevant to agents, such as chrome, buttons, or sign up CTAs

I'm planning to update the afdocs CLI soon to reflect these updates; hopefully over the weekend.

Major thank you to everyone who weighed in, including @devalog, @njb90, @nicoleomeara-astro, @rosieyohannan, @MattyQ, and @chalin. Really grateful for everyone's feedback to help make this a more useful tool for the community! 🎉

In Agent Skill land, I have been synthesizing my 100+ review comments across our official Agent Skills corpus into review guidance to scale Skill quality reviews across our docs org.

If you wonder what types of things a human should be looking for when reviewing Agent Skills (beyond technical details, which should be reviewed by an SME), you might find this helpful. I look at:

• Scope and Skill boundaries
• Decision tree walkthrough
• Content justification
• Instruction precision
• Consistency and narrative coherence
• Cross-skill ecosystem awareness

The page also includes my guidelines for which things should block merge, request changes, or nice-to-haves. Available in our meta docs for anyone who might find it helpful:

https://www.mongodb.com/docs/meta/tutorials/review-agent-skill/

Cloudflare's @mvvmm has been getting some much-deserved shout-outs from his team for his excellent work on their agent docs monitoring stuff, and now I get to give him a shout-out for making some helpful contributions to afdocs! 🎉

I've just cut v0.14.0, which includes a handful of PRs from Vance, including some bug fixes and performance optimizations - and a new --skip-checks flag for better ergonomics if you need to skip only one or two checks that aren't relevant for your docs.

Really appreciate the contributions here, Vance!

Thanks to @philip. - AFDocs now supports testing against preview/staging deployments with different origins 🎉

In v0.13.0, we've added a new --canonical-origin flag that lets you specify a production origin to point to for URL matching in llms.txt and/or sitemap when you're running against a staging build. This looks like:

afdocs check https://preview-xyz.vercel.app/docs --canonical-origin https://example.com/docs

For more details, check out the updated "Run Locally" docs at: https://afdocs.dev/run-locally.html#production-urls-in-local-builds

I really appreciate the PR, and hope this is useful for folks who want to test from staging.

Hey, docs teams (and web users in general) - it's celebration time! Claude Code has shipped an update that fixes the configuration of its HTML to markdown lib and now it can use gigantor docs pages buried under a mountain of CSS more effectively 🎉

In the v2.1.105 release notes, Claude Code has this unassuming bullet item:

- Improved `WebFetch` to strip `<​style>` and `<​script>` contents from fetched pages so CSS-heavy pages no longer exhaust the content budget before reaching actual text

An engineer who reverse-engineered Claude Code's WebFetch implementation pointed out in January that it uses the Turndown library to convert HTML to markdown. But as I found in an article I posted in February, Claude Code's Turndown implementation was not configured to strip <​style> or <​script> tags, so web pages with a lot of CSS or JS script content would eat the entire context window before ever reaching any real page content.

With this fix, Claude Code can now successfully "see" content in large HTML pages! I've confirmed this with some known "bad" pages that Claude consistently complains about. Complaints are gone - Claude can now successfully retrieve the content!

This is good news for documentation teams who do not yet serve clean markdown through content negotiation. Now Claude Code stands a much better chance of being able to find your content in HTML. 🎉

I’ve had an idea and am wondering if anybody is doing this in their orgs yet: Skills-as-a-service.

Since my org has published our Agent Skills, we’ve been getting customer requests for very specific things. They wouldn’t generalize well to broadly distributed skills because our customers may be running in very different environments or with very different infrastructure and they want skills that reflect their constraints.

So today, I pitched to our Product team the idea of upskilling our Professional Services and/or customer account teams to adapt templates we provide to create customer-specific skills to help customers with specific tasks.

Anybody out there doing anything like this yet? I’m curious how you might handle tracking where you have provided custom skills to customers for ongoing maintenance, and also success metrics around customer-specific skills. And what support and/or SLAs might look like for bespoke customer-specific skills? 🤔

I've started digging into my next agent-related research project, and it's a doozy! I've spent the last few days deep diving in 70+ docs sites, running proportional AFDocs scores based on the number of docs pages in the domain, running curated scores representing how well docs serve agents across the user journey, and spelunking in a broad cross-section of docs sites across the industry.

I've already got some interesting findings, and there will be more to come as I deep dive in the eval track...

• The docs platform used makes a big difference in whether sites are agent-friendly without a lot of extra work
• Web hosts and CDNs can complicate the picture
• Some company/product categories are serving agents much better than others
• Deep-diving in scores across this many sites has turned up 11 gaps and enhancement opportunities in AFDocs 😅

I'll be bug-bashing on AFDocs after work this week and hope to have the gaps shored up soon. And then I get to start the real fun... task design for a three-track eval project on 30+ of these docs sites to see how well the Agent Score predicts real gaps in agent task completion for the products 😁

Given the scale of this project, it's going to take a couple of months even with help... but I'll be sharing random findings along the way!

Last weekend, I started experimenting with how I might track agent website traffic. This week, someone shared my agentreadingtest.com website to Hacker News. The timing was amazing - I had already set up the site to track agent signals, and suddenly I had thousands of Hacker News site visitors pointing their agents at my site.

What a rich data mine!

I've been spelunking in the logs to try to add more agents to my tracking based on all the new data and activity I'm seeing, and I've extracted the classification logic from my metrics pipeline into a new baby library for reuse across multiple projects where I want to track this type of activity.

If you have access to log entries - either from access logs, or CDN logs - you can use this library to try to classify agent activity: agent-traffic-classifier - it's on npm and GitHub.

If you have access to the full header set, you can optionally construct "agent signal entries" - which the lib attempts to correlate with access logs to try to identify likely agent activity using stronger signal. I've also started adding in IP intelligence to help with things like narrowing down "unknown AI assistant" to, for example, Chinese assistants like Kimi or DeepSeek.

The lib is just a classifier - you'll need to do something with the data you get back. I'm just using Chart.js to build some simple charts for my needs, but you can do whatever you want with the data based on your needs.

This is a very very early brand new lib that is in no way ready to make stability guarantees or be used in production. But I built it because the companies that claim to track agent activity are not transparent in how they're doing that, so I wanted to explore how I might do that and what's actually possible. By running my own numbers, I can assess the likely accuracy of vendor solutions. If this might be helpful to you, too, feel free to play around with it. But I'm many projects deep at this point so I'm not sure this will get the same level of activity/speed of development as my other projects.

Links from the comments:

• https://github.com/agent-ecosystem/agent-traffic-classifier
• https://www.npmjs.com/package/agent-traffic-classifier