By this point I had separate model configurations for the two Macs, measured context limits, and several runtime builds to keep track of. The previous post also covered the access settings I wanted to manage separately. I needed a way to preserve that setup without reconstructing it from terminal history.

This matters for the agent experiments that started the series. If I change a model or its settings and get different results, I want to be able to return to the earlier configuration. I also want to be able to reset a Mac without losing the work needed to make those models run.

I put together an offline recovery library on an external SSD, including the model files, pinned source, Python wheels, native runtimes, and Apple toolchain material. Then I reset the M5 and restored from it. I handled the administrator and license prompts myself. The installer finished with:

RESTORED ALL EIGHT PROFILES. No persistent servers started.

I then started each model, checked its status, requested a generation, and stopped it before moving to the next one. All eight passed. That was the first full test of this M5 library on a factory-reset machine.

What the clean restore tested

Before the reset, the qualification records said that offline component restores had passed over independently provisioned files. They explicitly did not claim a factory-reset restore.

The reset removed that uncertainty about the existing installation. It was still a narrower inference test than the earlier qualification: I ran generation smoke tests for all eight models, without repeating the long-input or separate image tests. The evidence summary records which checks ran before and after the reset.

The restore also exercised a dependency that would have been easy to miss on a working Mac. Xcode setup initially couldn’t find the Metal compiler. The installer then imported the archived Metal Toolchain 17F109, checked the compiler, and continued. Keeping the model files and source alone would have left that part of the setup unfinished.

One repository, machine-specific kits

The storage layout settled into this shape:

LLM/
  .git/
  restore-all.sh
  machines/
    m3-ultra-256gb/
      DwarfStar/
      Orca/
    m5-max-128gb/
      DwarfStar/
      Orca/
      optional-models/
    m5-ultra-256gb/       # Reserved; not qualified.

Git lives at LLM/, above the machine directories. It tracks the small material that explains the installation: scripts, configuration, manifests, and reviewed records. The large payloads remain outside Git history.

The model files need their own backup and checksums: recovering the Git repository alone wouldn’t recover those ignored files. Keeping them outside Git lets me review changes to the scripts and configuration without putting hundreds of gigabytes of weights into each kit’s history.

The installed locations under ~/Downloads stayed separate from the SSD layout. The managers run from internal storage, including the streaming model’s installed files. After a successful restore, the recovery SSD can be ejected without taking the service away.

Each kit retained its own required assets. I avoided cross-kit links that would make one machine’s recovery depend on another machine’s directory. The M5 work also preserved the M3 engine pins and existing manager interface. A cosmetic cleanup was not a sufficient reason to disturb a working installation.

Eight accepted models

The M3 had nine profiles. The M5 finished with eight.

An Orca DeepSeek conversion was still incomplete. Cancelling it meant stopping its supervisor as well as the conversion process, after checking their identities. Otherwise the supervisor could start a later step after the visible process disappeared.

I retained the verified original MXFP4 shards as optional future-work assets. The incomplete output and redundant internal download copies were removed only after the SSD originals had been verified.

I also removed that profile from the installer, manifests, manager help, and model tables, and renamed the M5 Orca kit to reflect its four working models. The retained shards are explicitly optional. Restoring the eight accepted models doesn’t depend on finishing that conversion.

Copying the library

After the M5 restore test, I copied its library to the other SSD and used an M2 Mac to inspect both drives. The M2 was a verification host, not a newly qualified inference target.

Both copies had the same Git revision and contained 4,082 filesystem entries and 3,593 regular files. Names, sizes, permissions, and symlink targets matched. The copied Git database passed its integrity check.

I initially started a full content hash. That reads every byte on both drives: about 4.9 TB for these two logical copies. The first implementation was CPU-bound. Switching the large-file hashing to the built-in OpenSSL implementation raised observed disk reads to about 1.5 GB/s per drive.

I then chose the shorter verification scope. It hashed 3,400 small files on each drive, checked all 617 tracked files against Git, and checked the available small-file manifest entries. Those checks passed. The 193 large files on each drive were not all fully rehashed in that comparison.

The quick comparison therefore verified the copied structure, Git state, and small-file contents. It left the large model contents unverified in that copy. Their earlier checksums don’t prove that every byte survived the later copy unchanged.

The copy-check summary records the exact scope. Model weights may be downloadable again from their upstream repositories, but that depends on continued availability and access. It is still worth keeping the archives that already restored successfully.

What I kept separate from restore

The restore does not add firewall exceptions, change the router, configure Tailscale, or leave persistent servers running. Those operations change how the machine is exposed and when it consumes resources. They belong to explicit operating steps.

Likewise, synchronization after independent work needs more care than copying a folder over another folder. The two Git histories must be preserved and reconciled, while payload differences are inspected and verified separately. Replacing .git is not a merge strategy.

The command reference contains the restore and lifecycle commands for each accepted machine. It also calls out the older M3 single-model manager’s different argument syntax.

That finishes the installation work I needed before returning to the agent experiments. I can restore the accepted configurations and check that they generate, and I have records of the longer inputs they handled. I still need to test them on the work described back in February: reading specifications, proposing useful tests, and helping triage the failures. Those results will determine which of these models I keep using.

Retrospective entry for October 23. Assembled October 24; the SSD-copy verification above took place on October 24.

Reference: Local inference operations

After getting the local models running, I tested their input limits. Those checks were requests to an inference server: text went in and an answer came back. The testing agents from the earlier parts of this series would do more with that answer. They would use it to decide which files to read, which commands to run, and what to investigate next.

While choosing the models, I kept coming back to a concern about the community-modified weights. Could a model emit a tool call asking my agent to scan the drive and upload what it found? Would using an original upstream model, or a different runtime, make that safe?

Running inference locally keeps the prompt processing on my machine. It doesn’t stop a model from proposing an upload, and it doesn’t stop an agent with broad permissions from carrying one out. That applies to an upstream model too. I need to consider both the software I’m loading and the permissions of the program acting on its output.

For the repository work I’ve described in this series, access to a compiler, a checkout, and test results is useful. Access to unrelated files in my home directory usually isn’t. The local installation is a good point to make those choices explicit, before attaching the tools.

Start with what executes

A model installation contains more than weights. It can include model implementation code, tokenizer and processor code, a chat template, native libraries, a Python environment, and an HTTP server. Some of these execute directly. Others supply data to code that executes.

Custom model code can have an ordinary purpose: implementing an architecture that the installed library does not support yet. Its presence does not establish malicious behavior. It does add code to the review and pinning problem. Hugging Face’s custom-model documentation describes this extension mechanism and the explicit trust involved in loading custom code.

File formats matter too. Pickle-based loading can execute code during deserialization; Hugging Face documents that risk in its pickle-scanning guidance. A tensor-oriented format avoids that particular mechanism. It does not prove that a parser is free of bugs, that the surrounding package is benign, or that the generated text is safe to act on.

I therefore recorded the components together:

weights + model code + runtime + template + server configuration

That includes the separate native runtime needed by Orca GLM, as well as the model files. Revisions and hashes let me identify what I used and notice changes later. They don’t establish that the software is benign; the generation tests weren’t a security audit.

Then follow the tool call

Consider the path from a response to a side effect:

model output
    |
    v
tool-call parser
    |
    v
permission checks
    |
    v
tool process ----> filesystem / subprocesses / network

The model can propose a filesystem search. The parser can recognize it as a valid tool request. Neither fact should authorize the search.

The same applies to uploads. A syntactically valid HTTP tool call can still name an unapproved destination or contain data the process should never have read. Giving a model a narrower system prompt does not remove those capabilities from the process that executes its instructions.

For a repository investigation, I would make the checked-out repository and an output directory available, keep unrelated home-directory data out of the sandbox, and restrict network access at the execution boundary. The specific permissions depend on the task. The important property is that they are enforced outside the generated response.

This is a design requirement for the agent harness. The inference tests described in these articles did not validate a complete sandbox or an exfiltration-resistant worker.

Upstream provenance helps, within its scope

Using an upstream checkpoint can remove uncertainty about changes made by another distributor. It does not remove the need to review the runtime or constrain the agent. An upstream model can also produce an unsafe tool request, misunderstand the task, or follow instructions found in a file.

Similarly, switching from MLX-VLM to MLX-LM would change part of the software stack. It would not by itself make the model’s tool requests trustworthy. I would still need to establish compatibility, inspect the loaded implementation, and test the resulting service.

The word “uncensored” in an artifact name describes a publisher’s positioning. It is not evidence about the absence or presence of a backdoor. I had no finding that the Orca artifacts were malicious.

Accessing the server from another device

I also wanted to send requests from other devices, including my phone through a router running Tailscale. That raised a different access question: who could reach the model server?

With the service bound to loopback, an SSH tunnel can carry requests from another machine. With a wildcard bind, the service can accept connections through the machine’s network interfaces, subject to routing and firewall rules. The tested servers did not provide API authentication.

For an already configured SSH host alias named inference-mac, a tunnel can look like this:

ssh -N -o ExitOnForwardFailure=yes \
  -L 127.0.0.1:18000:127.0.0.1:8000 inference-mac

The client then talks to http://127.0.0.1:18000/v1. The explicit local bind keeps this forwarded port on the client’s loopback interface. These are the ordinary OpenSSH local-forwarding semantics. SSH access must already be configured and its host key verified.

For direct LAN access I added macOS application-firewall exceptions manually. The installer deliberately left that decision to the operator. Apple’s firewall controls allow incoming connections by application; those exceptions should not be confused with a rule allowing only one source address to one model port.

After the M5 restore, a request from another machine returned the model listing on port 8000. That established remote HTTP reachability. The local probe had already established generation. I haven’t yet tested the phone’s route through the Tailscale router.

For a future investigation, I would start by restricting the agent to its checkout, working directory, and approved tools and destinations. Choosing a model or changing the server doesn’t do that work for me.

The restore scripts preserve the model settings but leave firewall changes to the operator. In the next post, I describe what they do restore, and what happened when I tried them after resetting the M5.

Retrospective entry for October 22. Assembled October 24, including the subsequent M5 LAN check.

Next: Restoring the Models on a Clean Mac

In the previous article, I described getting local models running on the two Macs. A short prompt checked that an installation could generate an answer. The agent experiments from earlier in this series need more than that: the model may have to read source files, a piece of a specification, and the output of several failed test runs before deciding what to try next.

All of that takes space in the context window, the input and output the model can work with in one request. That space is measured in tokens, which are pieces of text rather than necessarily whole words. The code and conversation history use the same budget as the answer.

When I looked at the initial settings, two DwarfStar profiles were limited to 8,192 tokens. Several Orca profiles had no explicit cap in the manager. I wanted to see how much further I could take them on these machines, and to have a predictable failure when a request was too large.

The 8K settings were defaults, not measurements of what the models could handle. Nor could I read an upstream context limit and assume the local installation would manage it. The runtime’s memory use and handling of long requests needed testing with the exact configurations I intended to keep.

A cap needs a test on both sides

The procedure started with a small generation and increased the input in stages. The model had to retrieve specific records from synthetic text, then handle a follow-up asking for those records in a different order. An oversized request had to be rejected. A final small probe checked that the server remained usable afterwards.

For each configuration I checked that:

1. The server reports the expected model and context configuration.
2. The request contains the measured number of tokens after formatting.
3. The answer contains the expected records.
4. A follow-up preserves the required information.
5. An over-limit request fails instead of silently discarding input.
6. The server can still generate after that rejection.

The request and the generated answer share a context budget. A 200,000 token cap is not an invitation to send 200,000 input tokens and then ask for a long answer. The template and any other inserted tokens count too. The server’s default output budget is a separate setting; clients should send max_tokens explicitly.

I precomputed the test inputs before loading the large engine. Earlier, running an auxiliary tokenizer diagnostic alongside a resident model had caused memory pressure severe enough to interfere with SSH access. Moving that preparation out of the inference run avoided making the diagnostic compete with the model for memory.

What passed on the smaller Mac

The following is the accepted M5 Max, 128 GB result. The figures come from the recorded qualification summaries, not from model-card maxima.

Each accepted M5 run used one engine and one sequence or session. The sampled memory-pressure level remained normal. Some runs began with swap already allocated, so I recorded the starting and peak values rather than describing the machine as swap-free.

All eight passed the synthetic retrieval, follow-up, rejection, and recovery checks at these sizes. I would still need real repository tasks to judge whether the extra context helps with an investigation. Finding records in generated text is a much narrower task than understanding the interactions in a large codebase. The separate image smoke tests also leave long-context multimodal use untested.

The second request can be a different workload

The largest Orca requests produced the following elapsed times, including their short answers:

The follow-up sent the conversation again and asked for the records in reverse order. The selected MLX serving path recomputed the conversation; the native serving paths reused much more state. A four-second follow-up therefore tells a very different story from a four-second cold request.

Background copies and downloads were active during qualification. These are observations from the recorded workloads, not isolated throughput benchmarks. They should not be used to rank the underlying model families.

The DwarfStar DeepSeek V4.1 streaming configuration made the same point: its largest input took about thirty minutes, followed by a cached request of about five seconds. Whether that is acceptable depends on the job. A long unattended investigation and an interactive editing session have different tolerances.

What I can use these results for

The caps now give the agent client a budget to work within, and the rejection tests give me a way to catch regressions in how the server enforces it. The timings will also affect how I construct requests. A follow-up that reuses earlier computation and one that processes the whole conversation again can have very different costs.

There is still a quality comparison to do. On the M5, smaller quantizations made some of these configurations practical, but the retrieval checks don’t tell me how much reasoning or coding ability I lost. For that I need the same tasks, prompts, tool budgets, and scoring for each candidate. I will keep those results separate from the context qualification table.

For now, I have measured input limits and a repeatable procedure for checking them. Before connecting an agent to these services, I also need to decide what it will be allowed to do with their responses. That became a concrete concern while choosing community-modified models, and is the subject of the next article.

Retrospective entry for October 21. Assembled October 24; the M5 measurements above are from October 23.

Next: Local Models and Tool Permissions

Back in February, I left this series with Agents as Fuzzers. The idea was to combine agents that read specifications and propose tests with fuzzers that execute those tests, then check and triage the results. The earlier posts covered the instructions, coordination, and testing practices around that loop. I was using Claude Code, and hadn’t spent much of the series on the model running underneath it.

Eight months later, I’m picking up that part. I’ve been setting up local models on two Macs: an M3 Ultra with 256 GB of unified memory and an M5 Max with 128 GB. I want to find out how much of this agent-assisted work I can run on my own hardware. Before comparing the models on testing tasks, I need an installation I can run reliably and rebuild when something goes wrong.

The series outline originally put self-hosted inference on Runpod next. This experiment ended up on Apple Silicon instead. It is a detour into the machinery behind the agent, over four posts: getting the models running, testing how much input they can handle, deciding what access to give them, and restoring the whole setup after a reset. I haven’t yet measured whether these local models can do the bug-finding work as well as the hosted models I’ve been using.

What I’m moving onto the Mac

In the earlier articles, an agent could read a repository, suggest a test, run it, and inspect the result. The model supplies the responses, which may include requests to use a tool. The agent program handles the conversation and runs the permitted tools. In this setup, the model runs in a separate server process.

Here I’m setting up the model side. An inference server loads the model files and accepts requests from a client, such as an agent. The files contain the model’s weights; the runtime is the software that uses those weights to generate an answer. Hosting that server locally lets me keep this part of the work on the Mac and pin the exact files and software I use. The agent’s tools and permissions still need their own configuration.

This also changes the practical constraints on the experiments from February. A model that takes several minutes to read a prompt will make an agent’s repeated attempts expensive in time. A model that fits alone may not fit alongside another worker. I need to know those limits before trying the parallel-agent setup from Part 2 with these models.

Moving from 256 GB to 128 GB

The M3 installation had nine model profiles. Each profile selected a model and the settings needed to serve it. I kept separate kits for the M5, because copying the M3 configuration to a machine with half the memory wouldn’t tell me which models were actually usable there.

For several models, that meant choosing a smaller quantization: weights stored with reduced numerical precision to reduce their size. Names such as Q2, Q4, and 8-bit identify different choices of representation. Some artifacts use a mixture, keeping selected parts at higher precision. The memory saving needs to be weighed against any loss in answer quality; getting a file to load only answers the first half of that question.

I organized the kits into two collections, called DwarfStar and Orca. These names recur in the tables below. Each kit includes the model files, runtime, settings, restore scripts, and test records. The collection name doesn’t imply that all its models use the same engine.

The M5 ended up with these eight configurations:

The qualification record identifies the selected artifacts and the tests they passed. This isn’t a recommendation of one model over another. I haven’t done the paired coding and reasoning tests needed to compare, for example, the smaller GLM quantization with the one on the M3. DeepSeek V4.1 Flash 1017 is a recent addition: its public repository records its release on October 17, before these experiments.

What stays in memory

The size on disk wasn’t enough to work out what would fit. I also had to account for what the runtime kept in memory, and what it needed while processing a request.

The M5 DeepSeek V4.1 configuration streams expert weights from the internal SSD, keeping a 64GB cache in memory. Qwen has large n-gram tables, described in its model card, which the selected runtime keeps disk-backed. Both arrangements let me use artifacts larger than I could keep entirely resident. They also make the time spent reading and processing a new prompt worth measuring.

The external SSD serves a different purpose. It holds the recovery kit; the installed models run from internal storage. I can disconnect the external drive after restoring them.

I tested one engine and one sequence at a time on the M5. During the earlier work, an auxiliary diagnostic running alongside a loaded model caused severe memory pressure. For now, the agent-swarm experiments will have to respect that restriction. These results don’t establish that I can run several local workers concurrently.

More than one runtime

The model choices also left me with several runtimes to maintain. MLX-LM and MLX-VLM are separate projects using Apple’s MLX framework, with MLX-VLM supporting vision-language models. The Orca Qwen and Nex installations use a pinned MLX-VLM environment. OrcaSAQ and Orca GLM use GGUF files, a format consumed here by native llama.cpp servers. The DwarfStar models use two archived revisions of the DS4 engine.

Even the two llama.cpp installations aren’t interchangeable. Orca GLM needed a separate pinned build. One loading attempt failed a Metal allocation; the configuration that passed used automatic fitting, a single sequence, and --load-mode none. In that particular build, none disables special loading modes, including mmap. It doesn’t disable GPU inference.

I kept those settings with their runtime revisions so that a future restore wouldn’t depend on remembering which build accepted which flags. The operations reference has the machine-specific commands.

Checking that a model can run

Once a server starts, it can answer an HTTP request for its model list without having generated anything. I needed a short generation test as well as that readiness check. The managers therefore expose four actions:

start -> status -> probe -> stop

status checks the owned process and HTTP readiness. probe asks the model to generate an answer and checks the response. The Orca start operation also runs a generation before reporting success. Stopping checks the process identity, including its start time and command, before sending a signal; a saved PID alone could refer to a different process by then.

By October 23, eight M5 profiles had passed the qualification procedure. The remaining candidate, Orca DeepSeek, still had an unfinished conversion. I stopped the conversion and its supervisor, kept the verified original shards for possible later work, and left it out of the installer.

A short answer was enough to check that each restored installation could perform inference. For the work in this series, though, the input will include code, specifications, and the results of earlier tool calls. The next article covers what happened when I tested much longer requests, including how long I had to wait for the answers.

Retrospective entry for October 20. Assembled October 24; the M5 results above were obtained on October 23.

Next: Testing Longer Contexts

The short version

A fuzzer is a search tool whose results must be triaged. An AI agent is a search tool whose results must be triaged. The parallel is not metaphorical. I think it’s structural.

Two search tools

A fuzzer explores the input space of a program, looking for inputs that violate some oracle—a crash, a hang, a property violation. When it finds something, you triage: real bug? Duplicate? Exploitable?

An AI agent explores the solution space of a problem, looking for code, fixes, or tests that satisfy some goal. When it produces something, you triage: correct? Complete? Does it address the problem?

Both search. Both produce results that need judgment. Both waste enormous time if pointed in the wrong direction.

The anatomy, side by side

Every fuzzer does four things:

1. Generates inputs (random, mutational, grammar-based, coverage-guided)
2. Executes the target with those inputs
3. Checks an oracle (crash? new coverage? property violation?)
4. Saves interesting results for triage

Every AI agent does the same four things:

1. Generates candidates (from prompt, codebase, agent skills)
2. Executes or applies them (writes code, runs tests, modifies files)
3. Checks an oracle (tests pass? linter clean? invariants hold?)
4. Saves results for triage (commits, PRs, logs)

Replace “inputs” with “candidates” and “crash” with “test failure.” The structure is identical.

What changes when the searcher understands context

Traditional fuzzers are context-blind. AFL doesn’t know what a function does. libFuzzer doesn’t understand the specification. They compensate with volume—millions of executions per second.

Context-blindness has costs:

• Shallow oracles. “Did it crash?” works. “Does this violate the protocol invariant?” requires a custom harness—often harder to write than the code being tested.
• Redundant exploration. Without understanding structure, the fuzzer wastes cycles in uninteresting regions of input space.
• Triage burden. Many findings are duplicates, benign panics, or expected edge cases. You sort the signal from the noise.

An AI agent, by contrast:

• Can read the specification
• Can reason about which inputs trigger interesting behavior
• Can write its own oracle and generate inputs designed to challenge it

The search becomes intentional without becoming rigid.

The convergence

Combine the pieces from this series:

Together: context-aware search, parallel exploration, rich oracles.

Fuzzers still win at

• Speed. Millions of executions/sec with a simple oracle (“did it crash?”). AFL and libFuzzer are unbeatable here.
• Binary targets. No source code, no spec? Blind fuzzing is often the only option.
• Deterministic reproduction. Fuzzers produce exact inputs. Agent traces may need work to become deterministic.
• Corpus management. Mature fuzzers have corpus minimization, coverage tracking, seed scheduling. Agent ecosystems don’t—yet.

Agents win at

• Rich invariants. “Does this sequence of state transitions preserve safety properties?” An agent can both formulate and check the invariant.
• Spec-guided search. When the spec exists and is readable, agents generate targeted campaigns rather than relying on coverage alone.
• Triage. An agent can produce a root-cause hypothesis before you ever see the failure. It can check for duplicates.
• Harness generation. Writing fuzz harnesses is expert work. Agents can draft them from specs and iterate.

The spectrum

The gap is narrowing. What matters is understanding which tool fits which problem—and being willing to combine them.

In practice

• Traditional fuzzers for the fast, low-level search—serialization, encoding edge cases, roundtrip invariants. Simple oracles, enormous input spaces. Volume wins.
• AI agents for the slow, high-level search—stateful invariants, cross-component interactions, spec compliance. Complex oracles, understanding required. Context wins.
• Both together—agents generating hypotheses and fuzz harnesses, fuzzers executing at speed, agents triaging the results.

Fuzzing was barely known outside security research fifteen years ago. Standard practice after AFL and OSS-Fuzz. Table stakes today.

AI-assisted testing is on the same trajectory.

Related posts

• Oracles, Traces, Triage (series index)
• Agent Skills and claude-lint
• Agent Teams and claude-swarm
• Testing the Bugs Between Calls
• The Bugs Between Calls
• Property Tests Are Not A Fuzzer
• Fuzzing meets property testing

Next: Self-Hosted Inference on Apple Silicon

The short version

Agent skills + agent swarms + stateful testing could compound into something stronger than any piece alone. While this combination hasn’t been tested at scale with agents yet, the individual components have proven effective in practice. The following explores how these pieces might integrate.

The problem

In The Bugs Between Calls, I argued:

• The most expensive failures don’t live in single function calls
• They live in sequences—valid operations that, composed under load, trigger liveness incidents
• Stateless property-based testing catches bugs in the bricks
• Stateful property-based testing catches bugs in how the bricks stack

The December 2025 Prysm incident, the May 2023 finality delays—stacking failures. Every individual operation was valid. The trace was the problem.

If you want another non-Ethereum example of “trace-shaped” failures, the Stacks PoX-2 stack-increase bug is a good one to skim (thread). I wasn’t at Stacks at the time, and I’m not claiming I would have caught it; the point is simply that these failures often emerge from sequences and accounting state, not single calls.

What agents could add

Stateful property-based testing (described in The Bugs Between Calls) generates random command sequences, runs them, checks invariants after each step.

The framework doesn’t know why a particular sequence might be interesting. It just tries many and hopes to stumble on something broken.

An AI agent is not blind:

• It can read the specification
• It can study past incidents
• It can reason about which sequences are likely to trigger interesting states

It wouldn’t replace the random search. It would augment it with intentional exploration.

Three pieces

1. Agent skills define testing norms

Following the agent skills philosophy, you wouldn’t give agents step-by-step procedures. You’d give them norms:

• Idempotent imports: importing the same block twice must not double-apply side effects
• Epoch boundaries: boundary logic must not run twice across reorgs
• Invariant preservation: state transitions must preserve declared invariants

These go into CLAUDE.md and agent skills. The agent figures out how to test them. At least, that’s the idea.

2. Agent swarms parallelize exploration

With claude-swarm, you could run multiple agents against the same codebase, each exploring a different class of invariant:

• One explores idempotence
• Another targets epoch boundaries
• Another maintains the test infrastructure

Each agent pushes to the same repo. When one discovers a failing trace, the others see it on the next fetch.

No message passing needed—the test failures are the messages.

3. The feedback loop tightens

Today, you get a shrunk counterexample and figure out what it means yourself. With agents, the cycle could become:

1. Agent generates command sequences based on norms
2. proptest executes them, finds a failure
3. proptest shrinks the failure to a minimal trace
4. Agent reads the trace, generates a root-cause hypothesis
5. Agent writes a regression test

Steps 4 and 5 are currently hours of manual work. They wouldn’t be free with agents—output still needs triage. But the iteration speed could be fundamentally different.

While this complete loop hasn’t been tested end-to-end yet, each component exists and has proven valuable in isolation.

Where this probably won’t work

• Enormous state spaces, simple invariants. If your oracle is “did it crash,” a traditional fuzzer wins. Agents are slow by comparison.
• Precise mathematical constraints. When the goal is “find the exact input satisfying this formal constraint,” SMT solvers are more reliable. Agents reason about code, but they don’t exhaustively search a constraint space.

The agent’s advantage: structured exploration—when invariants are rich, the state machine is complex, and interesting traces require understanding.

The combination

Each piece works alone. Together, they compound.

This hypothesis remains unproven at scale. However, the individual components have demonstrated value in isolation, and their integration appears promising.

The best testing infrastructure usually emerges that way—you notice the pieces reinforcing each other before you design the integration.

Related posts

• Oracles, Traces, Triage (series index)
• Agent Skills and claude-lint
• Agent Teams and claude-swarm
• The Bugs Between Calls
• Chaos Testing stacks-node with Model-Based Stateful Testing

Next: Agents as Fuzzers

One agent hits a ceiling

A single Claude Code session can do one thing at a time. For small tasks—fix this function, write that test—that’s fine. But the work I care about is not small. Exploring multiple hypotheses in parallel, maintaining documentation while debugging, running specialized analysis while generating test harnesses.

One agent, one task, one context window. It doesn’t scale.

The agent-team pattern

In early February 2026, Anthropic published Building a C Compiler with Large Language Models—a detailed account of 16 Claude instances working in parallel to produce a 100,000-line Rust-based C compiler capable of building the Linux kernel. The total: nearly 2,000 Claude Code sessions, 2 billion input tokens, 140 million output tokens.

The architecture was surprisingly simple. No orchestrator. No message bus. No shared memory. Just git.

Each agent ran in a Docker container. Each cloned a shared bare repo, worked on a task, and pushed. When two agents tried to claim the same task, git’s built-in conflict resolution forced the second one to pick something else. Merge conflicts happened often; Claude was smart enough to resolve them.

The key insight: coordination through the codebase itself. The repo is the shared state. Commits are the messages. Locks are text files.

claude-swarm

This pattern can be implemented through claude-swarm—a reusable harness currently wired for running multiple Claude Code sessions in Docker containers, coordinating through git. The coordination pattern itself is tool-agnostic; claude-swarm is just one concrete implementation.

export ANTHROPIC_API_KEY="sk-ant-..."
export AGENT_PROMPT="path/to/prompt.md"
./tools/claude-swarm/launch.sh start
./tools/claude-swarm/launch.sh status
./tools/claude-swarm/launch.sh stop

The design is minimal by conviction, not by laziness:

Host                        /tmp (bare repos)
~/project/ ── git clone ──> project-upstream.git (rw)
               --bare       project-mirror-*.git (ro)
                                     |
                                     | docker volumes
                                     |
               .---------------------+---------------------.
               |                     |                     |
           Container 1          Container 2          Container 3
           /upstream  (rw)      /upstream  (rw)      /upstream  (rw)
           /mirrors/* (ro)      /mirrors/* (ro)      /mirrors/* (ro)
               |                     |                     |
               v                     v                     v
           /workspace/          /workspace/          /workspace/
           (agent-work)         (agent-work)         (agent-work)

All containers mount the same bare repo. When one agent pushes, others see the changes on the next fetch. Each container runs harness.sh, which clones, resets to origin/agent-work, runs one Claude session, and loops. Agents stop after a configurable number of idle sessions with no commits.

Why no orchestrator

The temptation is always to add a coordinator—something that assigns tasks, monitors progress, resolves conflicts. This approach avoids orchestration for the same reason it avoids workflow verbs in CLAUDE.md: centralized control tends to reduce agent autonomy and reasoning capabilities.

With no orchestrator, each agent must orient itself. It reads the README, checks the current state of the code, looks at what other agents have done, and decides what to work on next. This mirrors how good engineering teams actually function: shared context, local autonomy, coordination through artifacts.

Anthropic’s experience confirmed the pattern. Their agents maintained running docs of failed approaches. They took locks on tasks by writing text files. They specialized naturally—one agent coalescing duplicate code, another improving performance, another working on documentation.

Specialization is possible, not required

Right now, all agents in claude-swarm share the same prompt. They self-organize by looking at the repo and picking different things to work on.

Anthropic’s experience suggests that per-agent prompts—one focused on code quality, another on test coverage, another on documentation—can help at scale. claude-swarm supports that (just point AGENT_PROMPT at different files per container), but In practice, shared prompts often suffice for initial implementations. Agents typically self-organize effectively without specialized prompts.

This connects to the agent skills philosophy: the prompt shapes behavior. The harness just runs the loop.

When it works, when it doesn’t

Agent swarms work best when the problem decomposes into independent sub-tasks—many distinct failing tests, different modules, separate components. Each agent picks a different piece, and parallelism is trivial.

They struggle when the problem is monolithic. Anthropic hit this when compiling the Linux kernel: every agent would find the same bug, fix it independently, and overwrite each other’s changes. Their solution was to use GCC as an oracle and randomly split compilation between GCC and their compiler, letting each agent work on different failing file subsets.

For testing work, the decomposition is usually natural. Different invariants to test. Different modules to fuzz. Different state-machine paths to explore. The swarm pattern fits.

What this is really about

claude-swarm is about 200 lines of shell. It’s not the point.

The point is that the agent-team pattern—N autonomous agents, shared codebase, no central control—is a genuine paradigm for how AI-assisted work can scale. It’s not about making one agent smarter. It’s about making many agents productive together, the same way you’d make a team of engineers productive: clear context, local ownership, shared truth in the repo.

The C compiler was the proof of concept. Fuzz testing is where I’m applying it.

Related posts

• Oracles, Traces, Triage (series index)
• Agent Skills and claude-lint
• The Bugs Between Calls

Next: Testing the Bugs Between Calls

The temptation

The first thing most people do with a CLAUDE.md file is write a recipe. Step 1, do this. Step 2, do that. If you see an error, run this command. Here’s a code block you can paste.

It works. For about a week. Then the codebase shifts, the recipe goes stale, and the model follows outdated instructions with the confidence of someone who doesn’t know they’re wrong.

I’ve seen this pattern before. It’s the same failure mode as over-specified test fixtures: the more you hard-code the steps, the more brittle the system becomes. The test passes for the wrong reasons. The agent succeeds for the wrong reasons.

Context should shape reasoning, not script behavior

This distinction proves crucial in practice. When .claude/ directories emphasize workflows over norms, models tend to follow outdated instructions rigidly. When structured around principles and facts, models demonstrate greater adaptability to changing contexts.

Whether this constitutes “reasoning from principles” in a deep sense remains an open question. However, the resulting outputs consistently demonstrate improved quality and relevance.

Think about it from a testing perspective. A unit test that asserts f(3) == 7 checks one input. A property that asserts for all x: f(f_inverse(x)) == x checks the relationship.

Change how f computes internally and the property still holds—it only cares that the roundtrip works. The hard-coded assertion breaks the moment the mapping shifts.

Same idea. A CLAUDE.md that says “run cargo test after every change” is a hard-coded assertion. A CLAUDE.md that says “all changes must pass the existing test suite” is a property. The model can figure out how to run the tests. What it needs from you is what matters.

The layers

Over time, I’ve settled on a layered structure for .claude/ directories:

CLAUDE.md is the constitution. Short. Declarative. “This project uses Rust.” “Tests must pass before commits.” “Prefer explicit error handling over unwrap.” No instructions on how to do things—just what matters.

Agents get a perspective. If you have a code-quality agent, it gets values like “favor readability over cleverness” and “flag any function longer than 40 lines.” It doesn’t get a checklist.

Skills describe capabilities the model can use—not step-by-step procedures. A skill for “running fuzzers” says what the fuzzer does, what inputs it expects, what success looks like at a high level. It does not contain a bash script.

References are the escape hatch. Sometimes you genuinely need a playbook—a deployment procedure, a migration guide. References hold those, but they must declare themselves as optional. The model should know these are reference material, not marching orders.

claude-lint

A Rust CLI tool called claude-lint helps enforce these patterns by checking .claude/ directories for violations.

$ claude-lint .claude
ok: .claude passes all checks

$ claude-lint /path/to/.claude
error: /path/to/.claude/CLAUDE.md: contains workflow verb 'step 1'
error: /path/to/.claude/skills/foo/SKILL.md: contains fenced code block
2 error(s)

It checks for:

• Workflow verbs in CLAUDE.md (e.g., “step 1”, “then run”, “next, do”)
• Code blocks where they don’t belong (everywhere except references)
• Line limits on agents (≤120) and skills (≤500)
• Missing “optional” declarations in reference files

It’s deliberately strict. The point is not to make .claude/ directories pleasant to read. The point is to keep them in the shape where I’ve seen the model produce the best results.

Why this matters in practice

Claude Code demonstrates this approach in practice, using structured context to explore edge cases, generate test harnesses, and reason about state-machine invariants. The quality of outputs correlates directly with the quality of provided context.

When I embed workflows, the model sticks to them—even when they’re wrong for the current situation. When I embed norms (“never skip precondition checks”, “all state transitions must be tested for idempotence”), I get output that adapts to whatever the model finds in the codebase.

Whether that’s “reasoning from norms” or just the model having more room to draw on its training, I can’t say for certain. What I can say is that the parallel to property-based testing feels right. Properties tell the system what must hold. The system figures out how to check it. Norms tell the model what matters. The model figures out how to act on it.

Same shape. I’ll take it.

Related posts

• Oracles, Traces, Triage (series index)
• The Bugs Between Calls

Next: Agent Teams and claude-swarm

One important gap is that, while we still see critical bugs in single calls (SSZ decoding, BLS edge cases), many of the most expensive recent failures live between calls.

In many cases, the protocol rules are fine; the failure is a valid-but-expensive trace that turns into a liveness incident under load.

December 2025: Shortly after Fusaka activated (Dec 3, 2025), Prysm hit a resource-exhaustion path processing certain attestations, dropping network participation to ~75% and pushing voting participation as low as ~74.7% in some epochs—uncomfortably close to the 2/3 stake threshold required for finality. In this incident, attestations referencing a previous-epoch block root could trigger repeated state recreation, replay, and epoch-transition recomputation, exhausting node resources under load. (See the Prysm mainnet postmortems for the primary write-up.)

May 2023: Mainnet finality was delayed twice within ~24 hours (first ~4 epochs, then ~9). The trigger was valid old-target attestations that forced expensive beacon-state regeneration in some clients; diversity helped the chain recover without intervention. (Postmortem: Ethereum Mainnet Finality Incident (May 2023).)

April 2023: Stacks hit a PoX-2 bug in stack-increase that impacted Stacking rewards for a cycle. The details are different, but the shape is familiar: stateful logic where correctness is about how an accounting state evolves over a sequence of actions, not a single call in isolation. I wasn’t at Stacks at the time, and I’m not claiming “I would have caught it” — I mention it because it’s a clean example of why tests that exercise traces (not just inputs) matter. (Thread: A bug in stacks-increase call is impacting Stacking rewards this cycle.)

Each operation was valid. The sequence proved problematic only under load.

Many expensive bugs live in sequences that look fine individually.

Stateless properties (and where they stop)

Stateless properties shine when:

• the function boundary is the correctness boundary
• behavior is local to a single invocation
• invariants don’t depend on history

This covers a lot of “pure-ish” code: parsing, formatting, serialization, numerical edge cases.

But consensus software is not primarily pure functions.

Consensus clients are state machines

Ethereum’s consensus clients (e.g., Lighthouse, Prysm, Teku, Grandine, Nimbus, Lodestar) implement a long-lived state machine:

• per-slot processing (slots advance, duties change, messages arrive out of order)
• data availability (verifying that required data is available; evolving toward Data Availability Sampling via PeerDAS)
• fork choice (multiple competing branches, attestation-weighted via LMD-GHOST)
• finality (justified/finalized checkpoints that must only move forward)
• storage and replay (idempotence, witness caching, pruning, reorgs)

Correctness is rarely “the output of one function call”. It’s “the system’s behavior over a trace”.

Examples of stateful invariants in consensus clients

Here are a few invariants that are naturally history-dependent:

• Finality is monotonic: the finalized checkpoint’s epoch must never decrease. (Finality can stall; it must not regress.)
• Fork choice respects finality: once a checkpoint is finalized, the selected head must be a descendant of it. (Heads can reorg; finalized history cannot.)
• Data Availability gates what validators can accept/vote for: A block header is not enough in 2026. Availability is enforced via fork-choice/voting rules: validators should only accept and vote for blocks once sufficient data availability has been verified (today: all blobs; with PeerDAS: sampling cells/columns). Fork choice can only safely give full weight to blocks that validators can legally vote for. Testing the transition from “pending availability” to “available and valid” is a classic stateful trace.
• Stake-weighted participation (MaxEB): Participation was always stake-weighted, but MaxEB makes the variance visible by raising the cap from 32 ETH to 2048 ETH. Not every validator will immediately sit at the cap, but stake weight per validator can now vary widely, so a bug that affects a handful of high-effective-balance validators can represent outsized stake impact.
• Idempotent imports: importing the same block twice (same root) must not double-apply side effects (DB indexes, caches, votes, metrics, etc.).
• Equivocations must be handled, not assumed away: you can see multiple distinct blocks for the same slot. A client shouldn’t panic or corrupt state just because reality is adversarial.
• Epoch-boundary logic must not run twice: “do X once per epoch” bugs are classic state-machine failures when reorgs, retries, and partial persistence meet.

If you try to phrase these as “(f(x)) preserves (P)”, you end up smuggling “history” into (x) until it stops being a useful boundary.

Take “finality is monotonic.” You might try:

for all (old_finalized, new_finalized):
    process_block(...) implies new_finalized >= old_finalized

But now old_finalized is part of the input. Where does it come from? You have to generate it. And to generate a valid old state, you need to know what sequence of blocks led there. You’ve just reinvented traces—badly.

The honest framing is: “after any valid sequence of operations, the finalized epoch never decreases”. That’s a property over traces, not over inputs.

Model-based, stateful property-based testing

Stateful testing makes the history explicit:

State --(Command)--> State'

Instead of generating inputs for a single call, you generate commands and run them as a scenario. The bug is often not in any single step, but in a particular ordering of steps.

This idea is old and battle-tested (QuickCheck state machine testing, Hedgehog, proptest-state-machine), but to my knowledge still underused in many production systems.

The same approach, built into madhouse-rs, caught a production bug in the Stacks blockchain that traditional testing missed. A 533-line integration test failed to reproduce it. A chaotic command sequence succeeded.

Model-based, stateful testing has been applied successfully to production systems like the Stacks PoX contracts. The approach proved practical for ongoing use, helping catch issues that traditional testing methods missed and demonstrating the value of stateful property testing in complex consensus systems.

A minimal Rust harness (the “boring runner”)

The core trick is to keep the runner boring and put all the logic in commands. This is the same shape that scales in practice.

State and context

pub trait State: std::fmt::Debug {}

pub trait TestContext: std::fmt::Debug + Clone {}

For the examples below, assume an empty context:

#[derive(Debug, Clone, Default)]
pub struct BeaconContext;
impl TestContext for BeaconContext {}

Commands

use proptest::prelude::*;
use std::sync::Arc;

pub trait Command<S: State, C: TestContext>:
    std::fmt::Debug + Send + Sync
{
    // Precondition: is this command meaningful *now*.
    fn check(&self, state: &S) -> bool;

    // Apply the transition and assert postconditions.
    fn apply(&self, state: &mut S);

    // For debugging and shrunk traces.
    fn label(&self) -> String;

    // Generate commands.
    fn build(ctx: Arc<C>)
        -> impl Strategy<Value = CommandWrapper<S, C>>
    where
        Self: Sized;
}

Wrapper for heterogeneous sequences

#[derive(Clone)]
pub struct CommandWrapper<S: State, C: TestContext> {
    pub command: Arc<dyn Command<S, C>>,
}

impl<S: State, C: TestContext> CommandWrapper<S, C> {
    pub fn new<T>(t: T) -> Self
    where
        T: Command<S, C> + 'static,
    {
        Self { command: Arc::new(t) }
    }
}

Execution loop

pub fn execute_commands<S: State, C: TestContext>(
    commands: &[CommandWrapper<S, C>],
    state: &mut S,
) {
    for cmd in commands {
        if cmd.command.check(state) {
            cmd.command.apply(state);
        }
    }
}

The point is locality: generation, preconditions, transition logic, and invariants live together. That design choice is exactly the “data-open” side of the expression problem, and it’s why these harnesses survive contact with real systems.

A consensus-client-flavored example (with correct slot semantics)

One easy trap is to assume “there is only one block per slot”. In the spec there is one proposer per slot, but on the network you can see:

• equivocations (two blocks for the same slot from the proposer)
• different views due to propagation delays
• reorgs that temporarily make a “worse” chain the head

So a stateful invariant should not be “reject a second block at slot (s)”. That’s not how fork choice works.

Instead, here’s a deliberately small example that matches real failure modes: idempotence by block root. If a client re-imports the same block (same root), it must not double-apply side effects.

Model state

use std::collections::{HashMap, HashSet};

#[derive(Debug, Default)]
struct BeaconModel {
    current_slot: u64,

    // Slot -> set of known block roots at that slot (forks allowed).
    known_by_slot: HashMap<u64, HashSet<[u8; 32]>>,

    // In 2026, participation is stake-weighted (MaxEB / EIP-7251).
    // Total weight of unique blocks we've imported.
    total_imported_weight: u64,

    // Track which block states are available to prevent the 2025 Prysm regression
    // (expensive state regeneration when validating attestations for uncached blocks).
    state_cache: HashSet<[u8; 32]>,
}
impl State for BeaconModel {}

Command: tick time

#[derive(Debug)]
struct TickSlot;

impl Command<BeaconModel, BeaconContext> for TickSlot {
    fn check(&self, _state: &BeaconModel) -> bool { true }

    fn apply(&self, state: &mut BeaconModel) {
        state.current_slot += 1;
    }

    fn label(&self) -> String { "TICK_SLOT".to_string() }
}

Command: import a block (stake-weighted, duplicates forbidden)

#[derive(Debug)]
struct ImportBlock {
    slot: u64,
    root: [u8; 32],
    weight: u64, // Stake-weighted via MaxEB.
}

impl Command<BeaconModel, BeaconContext> for ImportBlock {
    fn check(&self, state: &BeaconModel) -> bool {
        self.slot <= state.current_slot
    }

    fn apply(&self, state: &mut BeaconModel) {
        let entry = state
            .known_by_slot
            .entry(self.slot)
            .or_default();
        let is_new = entry.insert(self.root);

        // This is the invariant: same root must not be "new" twice.
        // Stake-weighting means a duplicate root shouldn't double-count weight.
        if is_new {
            state.total_imported_weight += self.weight;
            state.state_cache.insert(self.root);
        }
    }

    fn label(&self) -> String {
        format!("IMPORT_BLOCK(slot={}, weight={})", self.slot, self.weight)
    }
}

// The bug from December 2025: attestations for stale blocks 
// triggering expensive state regeneration.
#[derive(Debug)]
struct ProcessAttestation {
    slot: u64,
    block_root: [u8; 32],
}

impl Command<BeaconModel, BeaconContext> for ProcessAttestation {
    fn check(&self, state: &BeaconModel) -> bool {
        self.slot <= state.current_slot
    }

    fn apply(&self, state: &mut BeaconModel) {
        // Invariant: looking up state for an attestation must not 
        // cause a "miss" that triggers an expensive re-play.
        assert!(
            state.state_cache.contains(&self.block_root),
            "State cache miss for block {:?} at slot {}", 
            self.block_root, self.slot
        );
    }

    fn label(&self) -> String {
        format!("PROCESS_ATTESTATION(slot={})", self.slot)
    }
}

If the implementation accidentally increments counters, updates indexes, or applies cached transitions twice on duplicate import, a failing trace usually shrinks to something like:

[
  TICK_SLOT,
  IMPORT_BLOCK(slot=1, root=R),
  IMPORT_BLOCK(slot=1, root=R),
]

If the system is not truly idempotent, stateful testing will reduce a complex failure down to the smallest possible sequence—often just “process the same thing twice”—making the bug obvious and undeniable.

That is the shape of a lot of consensus-client failures: not “wrong return value”, but “the second time through a path, something subtle breaks”.

Stateless + stateful is the real combination

You want both:

• Stateless property-based testing (PBT) for pure-ish components: SSZ encoding, BLS wrappers, serialization, bitfields. This is where Anthropic’s approach shines.
• Stateful PBT for the hard parts: fork choice, finality logic, DB/replay, reorg handling, epoch boundaries.

Stateless PBT finds bugs in the bricks (SSZ, BLS). Stateful PBT finds bugs in how the bricks stack—especially in the high-stakes world of PeerDAS and stake-weighted participation.

Anthropic showed us how to check the mortar really well. This post is about the wall.

Further Reading

• EIP-7594: Peer Data Availability Sampling (PeerDAS)
• EIP-7251: Increase the MAX_EFFECTIVE_BALANCE (MaxEB)
• Ethereum Consensus Specifications
• Gasper: LMD-GHOST + Casper FFG
• LMD GHOST Fork Choice (eth2book)
• Ethereum Engine API Specification
• Client Diversity Dashboard
• Prysm Mainnet Postmortems (includes Fusaka incident)
• Post-mortem: Ethereum Mainnet Finality Incident (May 2023)
• Stacks PoX-2 stack-increase bug (April 2023)

Related posts

• Model-Based Stateful Testing with madhouse-rs
• Model-Based Stateful Testing with proptest-state-machine
• Scaling Model-Based Stateful Testing with madhouse-rs
• The Expression Problem in Practice: A Trait-Based Testing Harness
• Chaos Testing stacks-node with Model-Based Stateful Testing
• Property Tests Are Not A Fuzzer
• Fuzzing meets property testing

Why

Skepticism is healthy here. Agent outputs still need oracles, reproducibility, and a bar for correctness. The tools are changing quickly; the craft is not.

Agents can draft tests fast; the hard part is still choosing the right oracles and insisting on reproducible failures.

Examples use Claude Code because that’s what I run day-to-day, but the patterns are meant to travel to any agent that can read a codebase, run checks, and write down findings.

This is not a tutorial. It’s a practitioner’s notebook.

The lens: oracles, traces, triage

If there’s a unifying theme here, it’s that most bug-finding systems succeed or fail on three things:

• Oracles — how you decide something is wrong. Not just “did it crash?”, but invariants, spec checks, and properties that reflect what you actually care about.
• Traces — many expensive bugs live in sequences, not calls. Stateful testing is about generating and shrinking traces until the failure is undeniable.
• Triage — search produces noise. The work is making findings reproducible, minimal, deduped, and actionable (ideally as regression tests).

That’s also a useful way to read the series:

• Part 1 (skills/norms) mostly expands the oracle surface (“what matters”).
• Part 2 (agent teams) scales the search and improves the artifacts that enable triage.
• Part 3 (between calls/stateful) is explicitly about traces.
• Part 4 (agents as fuzzers) argues agents are search tools too—and the hard parts are still oracles + triage.
• Parts 5–8 cover the local model servers I set up for further agent experiments, including their input limits, permissions, and recovery.

Articles

• Part 1 - Agent Skills and claude-lint
  How to structure .claude/ context so agents adapt from norms instead of blindly following scripts (plus a linter to keep it from drifting).
• Part 2 - Agent Teams and claude-swarm
  A practical pattern for parallel agents that coordinate through git, with no orchestrator—because the repo can be the shared state.
• Part 3 - Testing the Bugs Between Calls
  How agent skills, agent swarms, and stateful testing could combine to find consensus bugs that live in traces.
• Part 4 - Agents as Fuzzers
  A structural analogy: both fuzzers and AI agents search for failures that require triage and oracles.

Returning to the series: running the models locally

The February posts used Claude Code for the agent examples. In October, I picked up the self-hosting work that was next in the outline. I ended up using two Macs rather than Runpod. These four posts cover that setup and its tests; comparing the local models on the bug-finding tasks is still ahead.

• Part 5 - Self-Hosted Inference on Apple Silicon
  Why I wanted local models for the agent experiments, and what it took to run them on an M3 Ultra and an M5 Max.
• Part 6 - Testing Longer Contexts
  Moving beyond 8K defaults with measured inputs, continuation tests, explicit rejection, and cold versus cached request timings.
• Part 7 - Local Models and Tool Permissions
  Following a generated tool request through the runtime, agent harness, filesystem permissions, and network boundary.
• Part 8 - Restoring the Models on a Clean Mac
  An offline recovery library, a factory-reset M5, eight generation smoke tests, and the limits of a quick SSD-copy verification.

Parts 5–8 are retrospective entries dated October 20–23, assembled on October 24. Later measurements and checks are identified in each article. The operations reference keeps the M3 and M5 commands separate and links the supporting evidence summaries.

Companion post

If you want the deeper motivation for “why traces,” start here:

• The Bugs Between Calls

Next

• Part 9 — Corpus, shrink, triage: turning agent output into a fuzzing pipeline
  How to dedupe/minimize failures and turn “agent finds” into reproducible bug packets and long-lived regression corpora.

Background

Most of what I know about testing came from shipping production systems and learning in public through open source: contributing to AutoFixture starting around 2011, then maintaining Hedgehog, which once powered Echidna, an early and widely used property-based fuzzer for Ethereum smart contracts.

Along the way: Fare for regex-constrained test generation, a SplitMix port for reproducible failure discovery. Consensus fuzzers at Stacks that caught a production bug a 533-line integration test couldn’t reproduce.

That background is why I’m interested in AI tooling—not as a replacement for any of this, but as a way to do more of it.

Feedback

The ideas in this series come from daily practice—shipping agent-assisted testing tools for real protocol security work. But daily practice has blind spots.

If you think I’m wrong about something, I’d like to hear it. If you think I’m right but missing a nuance, I’d especially like to hear that.

Next: Agent Skills and claude-lint