Same Prompts, Three Macs

In the next couple of posts, I’m going to send the same prompts to the same models on three Macs and write down what the numbers say. Follow along and you’ll:

• see whether a higher-precision model file gets more answers right than a smaller one of the same model
• see what stacking two Mac Studios, or closing a laptop lid, does to a benchmark
• see a test that looked reasonable and measured nothing, and the control that showed it

This picks up where Restoring the Models on a Clean Mac ended: the models install and restore, but I had not compared them on anything. The machines are an M3 Ultra Mac Studio with 256 GB of unified memory, an M5 Max MacBook Pro with 128 GB, both from the earlier posts, and an M5 Ultra Mac Studio with 256 GB that joined them since. The runs were driven by Claude Code with Claude Fable 5.2 over SSH; it also wrote the prompts and the grader, which is a limit I come back to.

Here’s a list of posts to follow:

• Same Prompts, Three Macs: Higher precision, same answers
• Same Prompts, Three Macs: Where the machines sit
• Same Prompts, Three Macs: Four models and a test that measured nothing
• Same Prompts, Three Macs: A domain task that can be graded

Prompts, grader and scores: prompts.py, compare.py, results.json

Entry dated November 9. The measurements were taken on November 11 and 12, and the posts were assembled on November 12.