Local Inference Operations

Commands for the recovery library described in Oracles, Traces, Triage. Updated October 24, 2026.

These commands assume that my archived LLM/ library is already present on the named SSD. The kit scripts and payloads are not distributed with this blog. This page records their interfaces; it is not a complete installer for a reader who has only downloaded the page.

Use the profile matching the physical machine. Changing the SSD’s mount name does not change the target profile. M3 Ultra 256 GB and M5 Max 128 GB are the accepted targets. A VM and the reserved M5 Ultra profile are not qualified. Run one inference engine at a time.

M3 Ultra, 256 GB

Keep a long restore running after an SSH disconnection

From the client, replace USER and M3-IP with the Mac’s login and address. SSH keepalives help with idle connection timeouts and detect a broken connection. Start screen before the restore so that losing SSH does not stop it; starting screen later cannot protect an existing job.

ssh -o ServerAliveInterval=30 -o ServerAliveCountMax=6 "USER@M3-IP"

On the Mac, check for an existing restore session first. Reattach with screen -r llm-restore if one exists; create a session only for a new run:

screen -ls

# Only if there is no existing restore session.
screen -S llm-restore

Run the following plan and restore commands inside that session. Keep the SSD connected. caffeinate -i prevents idle system sleep while the restore command runs, including time spent at an interactive prompt. It does not protect against a reboot or forced sleep. Keep a laptop on power with its lid open during restoration.

Restore all nine profiles

Run plan first. If it succeeds, run restore interactively and handle administrator and Apple-license prompts yourself.

bash "/Volumes/VMStore-m3/LLM/restore-all.sh" plan m3-ultra-256gb
caffeinate -i bash "/Volumes/VMStore-m3/LLM/restore-all.sh" \
  restore m3-ultra-256gb

This installs four DwarfStar and five Orca profiles. It leaves persistent servers stopped and does not add firewall exceptions.

Alternative: restore only the four DwarfStar profiles

Use these instead of the all-model restore, inside the same screen session. Run each step separately, and copy the manager only after the single-model restore succeeds.

KIT="/Volumes/VMStore-m3/LLM/machines/m3-ultra-256gb/DwarfStar"

# Restore DeepSeek V4 Flash 0731.
caffeinate -i zsh \
  "$KIT/single-model-kit-v1/restore-ds4-downloads-layout.zsh" restore

# Install its local manager.
cp -p "$KIT/single-model-kit-v1/restore-ds4-downloads-layout.zsh" \
  "$HOME/Downloads/DwarfStar/m3-ultra-256gb-single-model-kit-v1/manage.zsh"

# Restore DeepSeek V4.1 Flash 1017, GLM, and Qwen, including their manager.
caffeinate -i zsh "$KIT/three-models-kit-v1/restore.zsh" all

Detach and reconnect

Press Ctrl+A, release both keys, then press D. The restore keeps running. After reconnecting with SSH to the same Mac and user:

screen -ls
screen -r llm-restore

If the session still says Attached after the old SSH connection is gone, detach that old terminal and attach this one:

screen -d -r llm-restore

If several sessions match, use the full PID.llm-restore name shown by screen -ls. Do not start another restore while one is running. Reattach to answer Apple/password prompts; do not pipe the restore through tee, because it needs an interactive terminal. The installer writes its own logs/restore.log under the profile’s recovery-setup directory.

After the restore finishes and returns to its shell prompt, exit closes that screen session. Detaching is the way to leave a restore running. See lock troubleshooting before retrying a failed or interrupted run.

Operate

Define the directories in the current Terminal:

SINGLE="$HOME/Downloads/DwarfStar/m3-ultra-256gb-single-model-kit-v1"
THREE="$HOME/Downloads/DwarfStar/m3-ultra-256gb-three-models-kit-v1"
ORCA="$HOME/Downloads/Orca/m3-ultra-256gb-five-models-kit-v1"

The older M3 0731 manager takes an action directly and has no probe subcommand. Do not insert a model key:

zsh "$SINGLE/manage.zsh" start
zsh "$SINGLE/manage.zsh" status
zsh "$SINGLE/manage.zsh" stop

For the other models, choose the directory and key from the table, then run the lifecycle commands individually. This example selects the newer DwarfStar DeepSeek:

MANAGER="$THREE/manage.zsh"
MODEL=deepseek

zsh "$MANAGER" "$MODEL" start
zsh "$MANAGER" "$MODEL" status
zsh "$MANAGER" "$MODEL" probe
zsh "$MANAGER" "$MODEL" stop

Optional LAN firewall exceptions

Run after restoration. For a DwarfStar-only installation, remove the two "$o/..." entries. This preserves the firewall’s enabled/disabled state and starts no model. Exceptions apply to applications, not just the model ports or one source IP. The servers do not provide API authentication.

bash <<'FIREWALL'
set -euo pipefail
fw=/usr/libexec/ApplicationFirewall/socketfilterfw
d="$HOME/Downloads/DwarfStar"
o="$HOME/Downloads/Orca/m3-ultra-256gb-five-models-kit-v1"
apps=(
  "$d/m3-ultra-256gb-single-model-kit-v1/source/ds4-server"
  "$d/m3-ultra-256gb-three-models-kit-v1/source/ds4-server"
  "$o/llama/llama-server"
  "$o/python/bin/python3.12"
)
for app in "${apps[@]}"; do
  if [ ! -x "$app" ]; then
    echo "Missing executable: $app"
    exit 1
  fi
done
sudo -v
for app in "${apps[@]}"; do
  sudo "$fw" --add "$app"
  sudo "$fw" --unblockapp "$app"
  "$fw" --getappblocked "$app"
done
"$fw" --getglobalstate
"$fw" --listapps
FIREWALL

Full guides

cat "/Volumes/VMStore-m3/LLM/OPERATIONS.txt"
cat "$HOME/Downloads/recovery-setup/m3-ultra-256gb/OPERATIONS.txt"

M5 Max, 128 GB

Keep a long restore running after an SSH disconnection

From the client, replace USER and M5-IP with the Mac’s login and address. SSH keepalives help with idle connection timeouts and detect a broken connection. Start screen before the restore so that losing SSH does not stop it; starting screen later cannot protect an existing job.

ssh -o ServerAliveInterval=30 -o ServerAliveCountMax=6 "USER@M5-IP"

On the Mac, check for an existing restore session first. Reattach with screen -r llm-restore if one exists; create a session only for a new run:

screen -ls

# Only if there is no existing restore session.
screen -S llm-restore

Run the following plan and restore commands inside that session. Keep the SSD connected. caffeinate -i prevents idle system sleep while the restore command runs, including time spent at an interactive prompt. It does not protect against a reboot or forced sleep. Keep a laptop on power with its lid open during restoration.

Restore all eight profiles

Run plan first, then restore interactively. This installs four DwarfStar and four Orca profiles. Orca DeepSeek is deliberately excluded.

bash "/Volumes/VMStore-m5/LLM/restore-all.sh" plan m5-max-128gb
caffeinate -i bash "/Volumes/VMStore-m5/LLM/restore-all.sh" \
  restore m5-max-128gb

The clean restore and eight sequential generation smoke tests passed. That installation does not need to be restored again to use these commands. Restore leaves persistent servers stopped and firewall setup separate.

Detach and reconnect

Press Ctrl+A, release both keys, then press D. The restore keeps running. After reconnecting with SSH to the same Mac and user:

screen -ls
screen -r llm-restore

If the session still says Attached after the old SSH connection is gone, detach that old terminal and attach this one:

screen -d -r llm-restore

If several sessions match, use the full PID.llm-restore name shown by screen -ls. Do not start another restore while one is running. Reattach to answer Apple/password prompts; do not pipe the restore through tee, because it needs an interactive terminal. The installer writes its own logs/restore.log under the profile’s recovery-setup directory.

After the restore finishes and returns to its shell prompt, exit closes that screen session. Detaching is the way to leave a restore running. See lock troubleshooting before retrying a failed or interrupted run.

Operate

SINGLE="$HOME/Downloads/DwarfStar/m5-max-128gb-single-model-kit-v1"
THREE="$HOME/Downloads/DwarfStar/m5-max-128gb-three-models-kit-v1"
ORCA="$HOME/Downloads/Orca/m5-max-128gb-four-models-kit-v1"

Every M5 manager takes a model key, including the single-model manager. This example selects 0731. Run each action separately:

MANAGER="$SINGLE/manage.zsh"
MODEL=deepseek

zsh "$MANAGER" "$MODEL" start
zsh "$MANAGER" "$MODEL" status
zsh "$MANAGER" "$MODEL" probe
zsh "$MANAGER" "$MODEL" stop

Optional LAN firewall exceptions

The M5 has a separate Orca GLM native server, so there are five executable paths. For a DwarfStar-only installation, omit all three "$o/..." entries. The same application-wide scope and lack of API authentication apply.

bash <<'FIREWALL'
set -euo pipefail
fw=/usr/libexec/ApplicationFirewall/socketfilterfw
d="$HOME/Downloads/DwarfStar"
o="$HOME/Downloads/Orca/m5-max-128gb-four-models-kit-v1"
apps=(
  "$d/m5-max-128gb-single-model-kit-v1/source/ds4-server"
  "$d/m5-max-128gb-three-models-kit-v1/source/ds4-server"
  "$o/llama/llama-server"
  "$o/llama-glm/llama-server"
  "$o/python/bin/python3.12"
)
for app in "${apps[@]}"; do
  if [ ! -x "$app" ]; then
    echo "Missing executable: $app"
    exit 1
  fi
done
sudo -v
for app in "${apps[@]}"; do
  sudo "$fw" --add "$app"
  sudo "$fw" --unblockapp "$app"
  "$fw" --getappblocked "$app"
done
"$fw" --getglobalstate
"$fw" --listapps
FIREWALL

Full guides

cat "/Volumes/VMStore-m5/LLM/machines/m5-max-128gb/OPERATIONS.txt"
cat "$HOME/Downloads/recovery-setup/m5-max-128gb/OPERATIONS.txt"

Restore and lock troubleshooting

An SSH disconnect does not prove the restore stopped. First reconnect to the same Mac as the same user and check screen -ls. If the restore is still running, reattach and let it finish. A quiet log can mean it is waiting for an interactive prompt or hashing a large file.

Set the correct profile and directories for this Mac:

PROFILE=m3-ultra-256gb  # Use m5-max-128gb on the M5.
SETUP="$HOME/Downloads/recovery-setup/$PROFILE"
SINGLE="$HOME/Downloads/DwarfStar/$PROFILE-single-model-kit-v1"
THREE="$HOME/Downloads/DwarfStar/$PROFILE-three-models-kit-v1"

Read the log and inspect processes and listeners before treating any lock as stale. Missing logs can mean the run stopped before logging began.

screen -ls
tail -n 60 "$SETUP/logs/restore.log"
ps -axo pid,ppid,user,lstart,command
lsof -nP -iTCP:8000 -iTCP:8100 -sTCP:LISTEN

Look for the restore script, its manager, and any surviving child copy, checksum, installation, or build processes. No listening server does not prove that a restore has stopped. Use the appropriate manager’s status and stop commands above for a running model. Do not kill a PID from an old log without checking its current owner, start time, and full command.

The top-level setup lock

Both all-model installers use this directory, which has no PID file:

ls -ld "$SETUP/install.lock"

Only after confirming that the original restore and its children have ended, with no concurrent restore or manager operation, remove the empty stale directory:

rmdir "$SETUP/install.lock"

If it is not empty, stop and inspect it. Do not substitute rm -rf. Then rerun plan and, if it succeeds, the same caffeinate restore command inside screen. Existing owned files are rechecked by the installers; this is a rerun, not a promise to resume a partial file at its last byte. Keep plan.sha256, receipts, and model files. A changed-controls or unowned-installation error needs investigation, not deletion of that state.

DwarfStar manager operation locks

These are directories containing a pid file. Select only the lock named in the error, using the matching path below:

M3 single-model kit:

LOCK="$SINGLE/state/service-8000/operation-lock"

M5 single-model kit:

LOCK="$SINGLE/state/operation.lock"

Three-model kit on either Mac:

LOCK="$THREE/state/operation.lock"

Run one assignment, then inspect it:

ls -ld "$LOCK"
cat "$LOCK/pid"
lock_pid="$(cat "$LOCK/pid")"
ps -ww -p "$lock_pid" -o pid,ppid,user,lstart,command

If that process is active, wait or return to its screen session. If the PID belongs to an unrelated process, do not signal it. Confirm that the old operation and its children have ended. A missing or unreadable PID is not proof of a stale lock; inspect the process list and session first.

Only for a confirmed stale operation, with no concurrent manager or restore:

rm "$LOCK/pid" && rmdir "$LOCK"

If the directory has no PID file and is confirmed stale and empty, use rmdir "$LOCK" alone. Retry the original operation once after cleanup.

Orca and the native DS4 instance lock

Orca’s operation.lock is a file, protected with an OS file lock. It can remain on disk after the process exits; that is normal. Do not unlink it to bypass an active operation. With ORCA set to this machine’s kit directory as shown above, inspect its users:

lsof "$ORCA/operation.lock"

Reattach to the existing operation, or wait for it to finish. An open-file listing identifies candidate processes; inspect their commands before acting. Do not remove service.json or process records to force a start.

The shared native DS4 lock is separate from those manager directories. Do not delete it as part of directory-lock cleanup:

lsof "$HOME/Downloads/.ds4-instance.lock"

If a native server is running, stop it through its matching manager. If a lock error persists with no apparent owner, save the exact error, session list, process list, and relevant log tail for investigation.

HTTP checks and SSH tunnels

DwarfStar uses port 8000; Orca uses port 8100. Run this on the inference Mac while a DwarfStar model is running:

curl --connect-timeout 5 --max-time 15 http://127.0.0.1:8000/v1/models

A model listing establishes HTTP reachability, not successful generation. For a LAN check, replace 127.0.0.1 with the inference Mac’s actual LAN address and run from another machine. The 0731 API model ID is deepseek-v4-flash; its API base is http://HOST:8000/v1.

Alternatively, with SSH already configured for the alias inference-mac:

ssh -N -o ExitOnForwardFailure=yes \
  -L 127.0.0.1:18000:127.0.0.1:8000 inference-mac

Keep that tunnel open and use http://127.0.0.1:18000/v1 from the client. A service bound only to loopback can be reached this way without allowing direct LAN connections to its executable. SSH itself must be reachable.

Eject the recovery SSD

After restoration and verification have finished, leave directories on the SSD and eject it. This example is for VMStore-m5; substitute VMStore-m3 when appropriate.

cd ~
diskutil eject "/Volumes/VMStore-m5"

if mount | grep -Fq ' on /Volumes/VMStore-m5 ('; then
  echo "STILL MOUNTED - do not unplug"
else
  echo "VMStore-m5 is not mounted"
fi

Wait for successful ejection and confirmation that the volume is no longer mounted. Do not force a busy disk. Installed models run from internal storage, including the M5 DeepSeek expert-streaming files.

Command references

The OpenSSH options, Apple screen manual, and Apple caffeinate manual describe the keepalive, detach/reattach, and idle-sleep behavior used here.

Evidence and scope

• Qualification summary: accepted M5 contexts and later restore observations. Long-input and image qualification preceded the reset.
• Copy verification: matching structure, Git state, and small-file hashes. The later SSD comparison did not fully rehash all large files.
• Chronology checks: public repository metadata, the October 17 DeepSeek release record, and retrospective article dates.

The full library revision discussed here is 3b8e658. The two native DS4 engine pins are 0aaea5a238fb41a35106a551e73c8409dfb751ac for 0731 and 5379c1c9ac315f1540371b06a3db88491d4059ff for the three-model kits.