Before retiring an agent skill, test what your history can actually tell you

· Skilled project notes

A runnable SQLite example showing how retention and project filters hide skill usage, with a cautious retirement checklist.

Published by the Skilled project. Skilled reports recognized skill calls from local coding-agent histories. This diagnostic works without installing it.

A quiet skill is a reasonable candidate for review. Before removing it, check whether your report could have seen its use. A history reader only has the events you retained, in the projects and formats it recognizes. Its list may never include installed skills with no matching events.

Here is a small counterexample you can run locally. It uses synthetic records, reads no files, makes no network requests, and writes only to an in-memory SQLite database. Save it as check_skill_history.py and run python3 check_skill_history.py. It needs Python 3 with its standard-library sqlite3 module.

One inventory, three answers #

The inventory has three skills. Two review calls happened recently in project alpha. Two release calls happened earlier in project beta. There are no restore calls in the supplied history. The dates are fixed so the result does not change next week.

 1import sqlite3
 2
 3db = sqlite3.connect(":memory:")
 4db.executescript("""
 5CREATE TABLE inventory (skill TEXT PRIMARY KEY);
 6CREATE TABLE events (skill TEXT, day TEXT, project TEXT);
 7""")
 8db.executemany("INSERT INTO inventory VALUES (?)", [
 9    ("review",), ("release",), ("restore",),
10])
11db.executemany("INSERT INTO events VALUES (?, ?, ?)", [
12    ("review", "2026-09-08", "alpha"),
13    ("review", "2026-09-09", "alpha"),
14    ("release", "2026-08-01", "beta"),
15    ("release", "2026-08-02", "beta"),
16])
17
18def counts(since="0000-01-01", project=None):
19    return dict(db.execute("""
20        SELECT i.skill, COUNT(e.skill)
21        FROM inventory AS i
22        LEFT JOIN events AS e
23          ON e.skill = i.skill
24         AND e.day >= ?
25         AND (? IS NULL OR e.project = ?)
26        GROUP BY i.skill
27        ORDER BY i.skill
28    """, (since, project, project)))
29
30full = counts()
31retained = counts(since="2026-08-13")
32alpha_only = counts(project="alpha")
33assert full == {"release": 2, "restore": 0, "review": 2}
34assert retained == alpha_only == {
35    "release": 0, "restore": 0, "review": 2,
36}
37for label, result in [("full fixture", full),
38                      ("retained since Aug 13", retained),
39                      ("alpha only", alpha_only)]:
40    print(label, result)
41db.close()

Expected output:

1full fixture {'release': 2, 'restore': 0, 'review': 2}
2retained since Aug 13 {'release': 0, 'restore': 0, 'review': 2}
3alpha only {'release': 0, 'restore': 0, 'review': 2}

Both restrictions hide the release calls. Neither changes what happened in the full fixture. The restore count has a different explanation: no event was supplied for it at all. Even the full fixture cannot establish whether restore was used outside the supplied history, or whether keeping it would be useful.

Two SQL details matter if you adapt this example. Put event filters inside the join condition to preserve inventory rows without matches. Use COUNT(e.skill), not COUNT(*), because an unmatched left-join row would otherwise count as one. The inventory must come from a separate source. Building it from event names would lose exactly the skills you wanted to investigate.

What this means for Skilled #

In the inspected Skilled aggregation implementation, skillCounts creates rows from observed call names. It does not enumerate installed skills. The audit labels repeatedly observed skills stale when their last call is more than 28 days old, while handling one-off calls separately.

With the full synthetic fixture, release can appear as stale. Remove the old records and release disappears from the observed-name list, rather than becoming a zero-count row. That behavior was checked against the implementation with a fixed September 10 clock. This checks aggregation and audit logic, not compatibility with any particular version of a coding agent's log format.

If Skilled is already installed, skilled list --limit 3 is a small first check: which recognized skill names occur in the histories available on this machine? An empty result needs investigation. It does not establish an empty installed inventory. Keep project filtering in mind too: Skilled's CLI project filter uses a path substring, whereas the teaching query above uses an exact synthetic project name.

Before making a removal decision #

Record the installed inventory separately, then note the reader version, selected sources, project scope, and known retention policy. The oldest observed event is only a lower bound on what you can inspect, not proof that every intervening event was captured. Confirm a known recent invocation appears before relying on a quiet report.

Keep three outcomes distinct:

A low-frequency recovery or release skill may still be worth keeping. Call frequency alone does not measure successful execution, instruction quality, token cost, or the value of having a procedure available when needed.

If you complete a check and find that retention or project scope changes your retirement decision, a Skilled diagnostic issue would be useful. Include the tool version, a synthetic reproduction or aggregate counts, the scope change, and whether the conclusion changed. Mention this article if it led you there. Please omit raw histories, prompts, usernames, credentials, and real project paths. A report that shows the diagnostic is wrong is useful too.

last updated: