Microblog

Consider this your friendly neighborhood reminder that Agent Skills need regular testing and maintenance to stay useful, or should be retired when they have outlived their useful life.

If you’ve written a skill earlier this year, it may already be silently degrading. New models, changes to harnesses, and changes to your own process and project can all cause skill performance to change.

Sometimes, models become capable of doing the thing on their own and your skill causes unnecessary thrashing. Sometimes, changes to harness instructions or tools can cause your skill to perform differently, and it may no longer deliver the results you expect. And if your skill covers a product surface or project that is in active development, there may be changes that render your skill outdated.

You need a regular process to re-check skill performance. The industry is moving fast. If you’re going to create these artifacts, you need to commit to maintaining them as things change around you.

Ok, at the end of Week 1 of New Job, here's what I'm thinking about:

Agents and humans do not need the same things. In fact, humans and humans don't need the same things. If you've ever been part of a persona-mapping exercise, drilled into audience segments, or tailored your content and pitches for different human audiences - you know what I mean.

Agents are the same. When I say "agent," I am referring to a combination of harness + LLM. Agent behavior is LLM inference mediated through harness tools and capabilities.

If you've ever tried using Claude Code and Cursor for similar tasks, and noticed different behaviors and outcomes, this is why.

So agent experience must understand the behavioral capabilities of different agents, what agents your customers are using, and how those behavioral differences interact with your product surfaces.

This is wide open space for most businesses today, so lots of opportunities but I also expect to see a lot of diverging practices as people try out what works.

New tools, who dis?

I keep messing with cross-harness experiments, and running into the same problems over and over:

• My agent re-discovers the valid headless invocation flags in every session, and runs into behavioral quirks that only turn up in use
• I haven't found an observability tool that works with the harnesses I want to observe, doesn't mess with the sessions, and is accessible to me as an individual
• If I want to read and compare session logs to evaluate experiment results, they produce different shapes that have to be normalized for valid comparison

I can't decide if it's a fool's errand given how quickly these things change, but I've spun up two tools to help:

• agentsummons: CLI to help manage invoking agent sessions for experiments
• agentminutes: CLI to parse native agent harness session logs into one unified, comparable event schema

Built in Go, because I love Go, but I've added npm and PyPI wrappers, so you can use them from your preferred ecosystems.

Very early releases (v0.2.2 and v0.2.0 respectively) and I have no idea if it'll be feasible to keep up with the churn on harness tools, but I want them and figured other people might find them helpful, so 🤞

Currently supports Antigravity, Claude Code, and Codex, because that's what I'm experimenting with at the moment, but if you want to use these and need support for another harness, I'm open to issues/PRs.

The links are in the comments.

Links from the comments:

• https://github.com/agent-ecosystem/agentsummons
• https://github.com/agent-ecosystem/agentminutes

I wish Anthropic hadn’t added “memory” functionality to Claude Code. I almost never want it, for a variety of reasons:

• It doesn’t have good judgement about what should become a memory
• I already have project documents whose contents I curate (AGENTS.md, CLAUDE.md, skills/hooks etc.)
• I don’t want to pollute context with content that isn’t needed in every session

I generally don’t want to waste the input tokens on Claude Code reading whatever it has decided is memory, and waste generation tokens on writing it. Especially when the harness seems to “forget” memories anyway when performing tasks in-session.

Blarg.

Well, the news is out, so I guess it's my turn to share here: after 5.5 years at MongoDB, I'm moving on for a new adventure.

I remain grateful that @cbush took a chance on someone who did not have coding skills to join the Developer Education, Realm team way back when. I was the first technical writer without dev chops to join this team, and I spun up on Swift/iOS/macOS development while learning about mobile application databases and everything it takes to ship apps. I still think that was a great product and remain bummed that the company deprecated it. I went from no dev chops to cutting my teeth on C++ under Chris's mentorship, and I'm still proud of growing those skills.

When the company deprecated Realm, Kyle led our team through redefining ourselves not once but twice. Under Kyle's leadership, and with the support of our Director of Documentation Jonathan, we got the opportunity to drive accuracy and developer experience initiatives across the docs org. Our stage got bigger, and I learned so much about moving from the weeds to strategic messaging for non-technical stakeholders. To Eliza and Melissa, thank you for helping me learn how to hone my messaging!

To my teammates Cory and Caleb - I have loved building alongside you these last few years. Leaving you and Kyle is the hardest part of moving on. 💖

To the many more folks I've worked with or crossed paths with over my years at MongoDB, you've all helped shape my time there and I will take that impact with me and be rooting you on from the sidelines. Tech is a surprisingly small industry - maybe we'll work together again someday!

I've got a new challenge coming up and had Claude Code design a study plan for me (again). Now I've got day-by-day lessons building progressively, a structured note-taking template, curated resources... and it only took ~10 minutes of describing what I had in mind, iterating, and validating the resources.

I know AI isn't perfect and it takes iteration to get good stuff out of it, but wow, is it so useful, and much faster than hunting down stuff myself.

One of my good friends is a Director of Engineering at a software company. He was telling me last night about how he was handed some requisitions for roles that he hadn't fully vetted after a recent re-organization.

After being informed about some new project work as a result of the re-org, he looked at the reqs again - and the roles were not aligned with the new work he had just been handed. So he has realized he probably needs to update the roles and post them again. After HR had already posted them, gathered 2,000 resumes for one role alone, and taken them down again.

Are they the right 2,000 resumes? Probably not, in light of new requirements related to the unplanned work he was handed.

All this to say, oof, it seems like a rough time out there folks. If you're in the market and missing out on roles, there is every chance that it's not you at all. And if it looks like companies are doing silly things with posting and re-posting similar roles, it may not be anything malicious - just new requirements surfacing or other communication gaps.

Thousands of motorcyclists have ridden the Pan American Highway to The End of the World. I'm one of them.

In 2010-2011, my partner and I made one of the iconic pilgrimages of the overland motorcycling world, riding from Boston to the bottom of South America. We covered 18,000 miles across 13 countries in what was truly the trip of a lifetime.

I've been thinking about it a lot lately because of one very important thing it taught me: things will happen when they happen.

In the United States, we live a fast-paced life of instant gratification. We get annoyed when we have to wait a few minutes for something.

In most of Central and South America, things come at their own pace. Maybe the shop will open an hour late. Maybe you'll get somewhere during afternoon siesta and it'll be closed. Maybe the border official wants an extra bribe today and you'll have to wait until tomorrow when a different official is working to cross. Maybe plans will fall apart and you'll have to do something completely different.

That trip was a master class in dealing with uncertainty, learning patience, living with discomfort, and rolling with change.

It has been 15 years since that trip; the motorcycle lives in my shed now and I haven't ridden in ages. But those lessons are very much top of mind. I am impatient to move forward and participate in fast-paced spaces, rushing through a period of discomfort and change. But am reminding myself that things come at their own pace.

And there is plenty of beauty to enjoy along the way.

A wild thing about this is that orgs that use products that restrict bot access to their docs, such as Vercel and Cloudflare products tailored to this, are also blocking agent access to docs. Agents can’t complete challenges.

So in practice, if you block agent access to your docs, you’re blocking an agent’s capabilities of looking up your product’s details in real user task flows to help users complete tasks. So agents will look for other sources, make stuff up, or choose other products where they can actually get relevant details.

Thursday thought exercise for y'all - how do you handle "best practices" in docs, and how would you convert those to config for coding agents?

In my experience, requests for best practices docs tend to be "distill 30k pages worth of content into a one-pager" - which doesn't actually work. So we tend to pick some particular persona or job-to-be-done and provide best practices for that, which just restates content that is scattered across more detailed task-oriented or feature-oriented docs pages.

Orgs are now starting to look at "what Cursor Rules or Claude Hooks can I distribute for my product" - which looks an awful lot like "best practices" distilled into config. So my first question becomes - what persona or job-to-be-done are we trying to support through this config? One stack's rules/hooks may look very different from another stack's - or if you're looking at it from a persona perspective, a database administrator's rules/hooks are likely to be very different from a developer building a CRUD-based web app.

Has anyone seen any examples of orgs/products that are doing this well yet? Have you started having these convos in your org? How are y'all thinking around these things?