I've started digging into my next agent-related research project, and it's a doozy! I've spent the last few days deep diving in 70+ docs sites, running proportional AFDocs scores based on the number of docs pages in the domain, running curated scores representing how well docs serve agents across the user journey, and spelunking in a broad cross-section of docs sites across the industry.
I've already got some interesting findings, and there will be more to come as I deep dive in the eval track...
- The docs platform used makes a big difference in whether sites are agent-friendly without a lot of extra work
- Web hosts and CDNs can complicate the picture
- Some company/product categories are serving agents much better than others
- Deep-diving in scores across this many sites has turned up 11 gaps and enhancement opportunities in AFDocs 😅
I'll be bug-bashing on AFDocs after work this week and hope to have the gaps shored up soon. And then I get to start the real fun... task design for a three-track eval project on 30+ of these docs sites to see how well the Agent Score predicts real gaps in agent task completion for the products 😁
Given the scale of this project, it's going to take a couple of months even with help... but I'll be sharing random findings along the way!