AI agent skills are spreading quickly across coding assistants and autonomous workflows, but many teams are using them backward.
The common pattern sounds reasonable: an agent struggles with a task, so someone opens a fresh session and asks the model to write a skill explaining how to perform it. The result may look polished, yet it often contains little more than the same knowledge the model already had.
In a recent essay, developer Anson Biggs argues that this approach confuses a reusable skill with an extra reasoning step. A valuable skill should contain information the agent would not reliably reconstruct on its own.
The best agent skills are not generic instruction manuals. They are compressed operational memory.
Key Takeaways
- Skills should preserve project-specific context, proven procedures or lessons learned through actual problem-solving.
- Asking an agent to generate a skill before it understands the task often produces little more than generic advice.
- Focused skills with a narrow job are easier for agents to retrieve and follow than sprawling documentation.
- Strong skill candidates include recurring workflows, unusual environment constraints and fixes discovered after failure.
- A skill becomes more useful when it includes scripts, examples and references—not just a markdown explanation.
The Self-Generated Skill Trap
Biggs’ critique was prompted by SkillsBench, a benchmark examining how structured skills affect agent performance across multiple domains. The research found that curated skills improved average results, while skills generated by models immediately before attempting a task did not provide an average benefit.
That finding is easy to misread as evidence that agents cannot help create useful skills. The more precise lesson is that timing and context matter.
If a model has not solved the problem, investigated the environment or discovered why the obvious approach fails, asking it to write a skill simply encourages it to restate its existing assumptions. It has no new experience to preserve.
A model cannot document a lesson it has not learned yet.
What an Agent Skill Really Is
At its simplest, a skill is a small package of procedural knowledge. It typically includes a primary instruction file with metadata that helps an agent decide when the skill applies.
But the folder can do more than describe a process. It may also contain:
- Scripts that perform fragile or repetitive operations consistently
- Reference material for uncommon APIs, commands or system constraints
- Troubleshooting guides built from previous failures
- Examples showing the expected input, output and quality threshold
- Verification steps that prevent an agent from declaring success too early
This makes a skill closer to an experienced employee’s field guide than a conventional prompt. It tells the agent what matters in this environment, where mistakes usually happen and how completion should be verified.
Three Jobs Skills Handle Well
1. Restoring Missing Context
Every new agent session begins with limited knowledge of the project. In a large codebase, a seemingly simple test might depend on a particular container, processor architecture, service order and CI configuration.
An agent can rediscover those details every time, but that wastes tokens and introduces avoidable errors. A focused skill can preserve the exact workflow once the environment has been understood.
2. Standardizing Repeated Work
Recurring assignments are natural candidates for skills. A release process may require synchronized updates to documentation, code, issue tracking and a pull request description. A content workflow may require research, image generation, formatting, publication and live-page verification.
Encoding the sequence saves users from repeatedly writing the same long prompt while giving the agent a stable checklist.
3. Preserving Lessons From Hard Problems
The highest-value skills often emerge after something goes wrong. Perhaps the agent used the wrong command, overlooked an environment limitation or optimized for a proxy metric instead of the actual goal.
Once the problem is solved, the agent can compare the failed and successful paths, identify the missing insight and turn that insight into durable guidance.
The right moment to create a skill is often after the breakthrough, not before the first attempt.
A Better Skill-Building Workflow
Teams can get more value from agent skills by treating creation as the final stage of learning:
- Attempt the real task. Let the agent inspect the environment and encounter the actual constraints.
- Diagnose the friction. Identify what was confusing, missing or repeatedly mishandled.
- Complete and verify the solution. Do not preserve an approach until it has produced the desired result.
- Extract the reusable knowledge. Separate project-specific lessons from details that were unique to one incident.
- Build a narrow skill. Include only the instructions, tools and references needed for that workflow.
- Test it in a fresh session. Confirm that an agent without prior conversational context can use it successfully.
This process turns skill creation into a form of operational learning. The agent helps document the path, but the source material comes from real execution and verification.
Focused Beats Comprehensive
There is a temptation to create one enormous skill containing everything an agent could possibly need to know about a company, repository or department. That usually weakens retrieval and makes the important instructions harder to find.
Smaller skills create cleaner triggers and clearer boundaries. “Monitor this project’s GitLab pipeline until it passes or fails” is more actionable than “understand our complete software-development process.”
A practical test is simple: can the skill’s purpose be stated in one sentence, and can success be verified? If not, it may need to be divided into smaller modules.
The Bigger Lesson for AI Operations
As organizations deploy more agents, their advantage will not come only from access to a powerful model. It will come from the institutional knowledge wrapped around that model.
Prompts disappear. Conversations end. Team members forget why a workaround exists. Well-designed skills preserve the decisions, constraints and procedures that make an agent dependable inside a specific business.
That is also why useful skills should be reviewed like code. They can become outdated, encode a bad workaround or conflict with a changed system. Ownership, testing and maintenance matter.
The goal is not to make the agent sound more knowledgeable. It is to make proven knowledge available exactly when the agent needs it.
Used this way, skills become one of the most practical building blocks in the agent ecosystem: a bridge between stateless intelligence and the accumulated experience required to do reliable work.
Source: Anson Biggs — “You’re Probably Using Agent Skills Wrong”