Microblog

Taking a minor break from agent-related research - sort of - to fix up some issues and feature requests for my newest macOS app that's in TestFlight Beta - Issuenator.

I've accumulated... kind of a lot of repos where I want to keep tabs on issues. And sometimes I file issues on other people's repos. And at work, I have a separate set of repos where I want to watch issues.

So in the spirit of PR Focus, my PR-tracking dashboard, I created Issuenator - an app for keeping track of all the issues across all the repos I care about.

I intentionally kept it separate from PR Focus because I'm really in different workstreams when I'm handling these things. PR Focus is for keeping track of my day-to-day work, and what my colleagues need from me. Issuenator is for keeping track of incoming requests from outside folks. I usually check on Issuenator at the beginning and end of the day to check for new issues or issue updates, or occasionally when I'm swapping between tasks, but otherwise don't want the noise interrupting my main workstream.

If you're a maintainer on a lot of repos, or if you file issues on a lot of repos, and you've ever wanted to keep track of all the issues in one place so nothing gets forgotten, maybe you want to give this a try. I've been in private beta for a couple of months and am pretty satisfied with the current state, so I'm thinking of opening it up to a public beta soon. If you want to try it out for free before I launch it, let me know.

https://issuenator.app/

I love my friends. And I know they care about me. But right now I have some things coming into focus with a clarity that demands action.

I'm spending basically every non-work hour exploring those things. I'm staying up too late. I'm getting up too early. I know I can't keep up the pace, but at the moment, the exploration is exhilarating. Sleep seems like an afterthought. I'm chasing things because they're interesting and because they feel important. And it feels like something I'm good at - like I've leveled up and unlocked a new mode that lets me do all the things I've enjoyed in bits and pieces, all at once, for a united purpose.

So it's hard when friends see me so excited, but the response is to throw buckets of cold water. "Dose of reality" they say. "What makes you think other people with more resources and a head start aren't already working on these things?"

I'm sure they are. But I also think that my take matters, too.

I have no idea what, if anything, will come of it. But that's not the important part. The important part is doing something I'm excited about because I'm excited about it, and because I think it matters.

Mostly sharing this to remind me it's ok to calibrate my risk tolerance and priorities differently from my friends. And maybe someone else needs to hear this right now, too.

If you've been following along on my Agent Skill exploration, you've probably seen me post about my skill-validator tool. I wrote it to help with my Agent Skill Report, and I've been quietly polishing away at it alongside all the other agent-related stuff I've been doing.

Last night, skill-validator got a new home and a v1.0.0 release! 🎉

The skill-validator tool now:

• Has separated core logic into library functionality to support reuse in other contexts, like test suites and other tools
• Retains the CLI entrypoint, but at a new location
• Lives alongside other tooling and resources I'm building related to agent things, in the GitHub agent-ecosystem org

If you've been using skill-validator via brew, you'll need to tap the new homebrew tap to get 1.0.0 and future updates:

brew tap agent-ecosystem/tap brew install skill-validator

I've got a lot more plans in mind for this tool, so if validating skill spec compliance + getting some quality checks is something you're interested in, keep an eye out for future updates. 😁

Links from the comments:

• https://github.com/agent-ecosystem/skill-validator

Big shout out to @rhyannonjoy for contributing a new set of known platform truncation limitations to the Agent Friendly Docs Spec!

Rhyannon used some Python-foo and the same data I used in my Agent Web Fetch Spelunking article to test the limits of the Claude API with the web_fetch tool enabled - similar to, but not the same as, the Claude Code web fetch implementation.

I'm sorry to say the results do not bode well for the average documentation page, friends. 😰 My initial findings with the Claude Code web fetch + summarization model were much more generous than the Claude API's web fetch default truncation limit.

What this means is that anyone who's built something that uses the Claude API/Claude platform - including possibly repackaging Claude in other agent tooling - may have far more aggressive truncation limits. Which means even less of our docs content may be making it through than I initially thought based on Claude Code.

Help us do science to it! Check out Rhyannon's PR, which I'll link in the comments, for an idea of how we can approach these problem domains.

Pick something you're interested in learning more about, do some testing, and share the results! We will all benefit.

(And if you contribute back to the Agent-Friendly Docs Spec with your findings, we can keep building out the constraints so we can adapt our docs systems to serve them.)

Links from the comments:

• https://github.com/agent-ecosystem/agent-docs-spec/pull/2

New project! I'm accidentally deep in coding agent land now, so I'm starting a new site - aeshift - a commentary and analysis blog about the agent ecosystem.

This one is a fun experiment for me because it's a different voice, with a very specific lens - practitioner-focused - and I'm using agents to help me with the content sourcing and draft pipeline.

I've already learned a ton just in putting this together, which I'll write about separately on my normal blog. But if you're on the dev/engineering side of the house and want to keep up with agent ecosystem developments and their practical implications, it might be worth a follow!

(Also, for the folks who have seen me posting lately about agent-friendly docs - yes, I've used my tooling and this site passes 100% of my agent-friendly docs checks 😁)

Links from the comments:

• https://aeshift.com

I've been chatting with a lot of technical writers and docs orgs as a result of the recent work I've been doing with the Agent-Friendly Docs Spec and the afdocs tool to validate docs against the spec.

Some folks have been concerned that this looks like a big project and they don't have the time or resources for it. But the reality is, it can look as small or as big as you want it to depending on how much this matters to you right now.

Some folks I've been chatting with have made one or a handful of fixes to their docs. Even just small changes can help.

If you can only do one thing right now, start with page size. The three checks you care about are:

• page-size-markdown: If the character count in markdown is too long, the content gets summarized and the agent will never see the full page. Long API references are likely to fall into this category. Break them up into smaller pages so the agent can actually see the content, instead of a summary that may miss important details.
• page-size-html: HTML adds more content with less semantic value for the agent, and may hide content away through JS elements. Your HTML pages are probably much longer than a markdown page would be and more likely to trigger summarization. Work with your platform to reduce the HTML page size, or add agent discoverability directives to point agents to the markdown version of your pages. (Or both!)
• content-start-position: This mainly applies to pages in HTML; depending on how the agent platform handles the HTML -> markdown conversion, some agents will never see your page content. Try to get your actual documentation content to start as close as possible to the beginning of the page content. In my experimentation, I found one page where the content was only ~60 lines and it started 87% of the way through the page. That's a big problem for agents.

Try out these checks for yourself using the afdocs CLI and lib you can install through npm. See how your docs actually perform, and check regularly as you iterate to make improvements.

And let us know what fun and interesting things you run into along the way! The platform providers aren't incentivized to study these things, or tell us about them. So we need to figure out the edges ourselves and share with each other so we can all prepare for this new type of docs consumer together.

This is what it looks like to be doing the work. Mintlify's own docs set - a docs platform whose premise is "built for both humans and AI" - passes all the Agent-Friendly Docs checks I've implemented so far, save one.

Mintlify's docs return "soft" 404s (returning a 200 with "no content" for a page that doesn't exist instead of a clearly actionable 404). But this is pretty minor, easily fixed, and would only affect a small subset of content in practice.

The fact that they're already doing all the things I identified as important for agent friendly docs means they - and their customers - are ahead of the curve. This is how you spot the real deal, folks.

(Also, now I have a "known good" docs set to test my tooling against as I continue to add checks 😁)

Just pushed another batch of updates to the afdocs tool that enables you to test your documentation site against the Agent-Friendly Documentation Spec. The v0.3.0 update adds the remaining 4 full-automated checks with clear pass/warn/fail conditions, bringing it up to 14 of 21 checks implemented. The new checks are:

• markdown-code-fence-validity: if you have unclosed code fences, agents will interpret any content after the opening code fence as code, leaving the content subject to misinterpretation
• http-status-codes: whether error pages return appropriate status codes, or "soft" 200 on 404s that may confuse agents
• cache-header-hygiene: can agents see fresh data, or are you setting long caches that mean agents may get outdated information?
• auth-gate-detection: do documentation pages require authentication to access content? If so, can agents figure that out, or is the type of return response ambiguous

The remaining checks are more subjective and open to different interpretations. I'll implement heuristics that make sense to me given the behavior I've observed, but your organization might care about or want to measure those things differently. Feel free to comment on the Agent-Friendly Spec repo to discuss what implementation might look like to you, so we can consider codifying more details in the spec.

Package on npm: https://www.npmjs.com/package/afdocs

Links from the comments:

• https://github.com/agent-ecosystem/agent-docs-spec

Because a couple of folks have asked or commented, some more thoughts related to the markdown content that features heavily in the Agent-Friendly Docs Spec:

• Where I talk about markdown in the spec, I'm talking about it as an output format for your docs pages. The input format doesn't matter so much. You can write in reStructuredText, MDX, or whatever format makes the most sense for your docs. But agent-friendly docs offer markdown as an output format for the docs pages to be consumed by agents, which work far better with markdown than HTML.
• I am not proposing that you maintain a second source that's in markdown alongside whatever source you already maintain. Ideally, your build pipeline automatically generates markdown versions of docs pages.
• Given that I expect the output markdown to be auto-generated, this is a big part of why I proposed the markdown/HTML parity check in the Observability section of the spec. I have seen in my investigation real-world cases where the markdown generation process is stripping text on the markdown versions of the pages that is rendering correctly on the human-viewable HTML versions of the pages. This includes things like parameter names, contextualizing headers and intro sections, and other information that the agent needs and isn't getting. The failure mode is this: human looks at page rendered in HTML, content is there, human doesn't look at markdown page and doesn't notice content is missing.

There are probably more things related to this that I'll discover as I continue to dig deeper. But I'm curious if any of y'all have started looking at your own generated markdown as a result of the spec and these convos and noticed anything else we should be thinking about?

Now that I'm starting to understand what it means for docs to be agent friendly, I'm charging full steam ahead with tooling to support this important docs consumer. If you want to check how agent-friendly your docs are, you can use this new tool I'm developing and distributing through npm: afdocs

It's a CLI/library that implements the Agent-Friendly Docs Spec and is distributed via npm. Currently, only the first three check groups are implemented, but I think those are some critical entry point checks that have real value today and don't want to delay release until the entire spec is implemented. You can run checks for:

llms.txt (5 checks)  - llms-txt-exists: Can agents find your llms.txt? Checks candidate locations so agents have a starting point for your docs.  - llms-txt-valid: Does it follow the proposed structure from llmstxt.org? Malformed files mean agents can't parse your doc index.  - llms-txt-size: Will it fit in an agent's context window, or will it get truncated before the agent sees everything?  - llms-txt-links-resolve: Do the URLs in llms.txt actually work? Broken links send agents into dead ends.  - llms-txt-links-markdown: Do those links return markdown? Agents work much better with markdown than raw HTML.

 Markdown Availability (2 checks)  - markdown-url-support: Can agents get markdown by appending .md to page URLs? This is the simplest way to serve agent-friendly content.  - content-negotiation: Does the server respond to Accept: text/markdown? A standards-based alternative to URL conventions.

 Page Size (3 checks)  - page-size-markdown: Are your markdown pages small enough for agents to consume without truncation?  - page-size-html: How large are your HTML pages, and how much shrinks away when converted to markdown? High boilerplate ratios waste agent context or may mean docs are summarized instead of read verbatim.  - content-start-position: How far into the page does the actual content start? Navigation, headers, and boilerplate at the top push real content past truncation limits.

That's the 10 implemented checks. The remaining 11 are stubbed across content structure, URL stability, agent discoverability, observability, and authentication categories. I'll be continuing to build these out in the coming weeks.

Link to package on npm: https://www.npmjs.com/package/afdocs