selftune eval

Run output-quality evaluations

selftune eval run --skill-path /path/to/my-skill

Flag	Purpose
`--evals PATH`	Override `evals/evals.json`
`--workspace PATH`	Override the sibling iteration workspace
`--baseline-skill-path PATH`	Compare with a previous skill instead of no skill
`--feedback PATH`	Copy an existing human-feedback JSON file into the iteration
`--agent NAME`	Use `claude`, `codex`, `opencode`, or `pi`
`--model MODEL`	Override the model used for execution and grading
`--json`	Print the benchmark result as JSON

The command creates a new numbered iteration and never overwrites a prior run. See Portable evaluations for the artifact layout and review rules. Eval files are validated before a workspace is created or an agent is invoked, including optional file lists and mechanical assertions. Grading responses must contain boolean pass/fail results and evidence; comparison winners must be A, B, or tie. Malformed model responses stop the evaluation instead of producing a completed benchmark from unvalidated values.

Generate routing cases

selftune eval generate --skill my-skill --skill-path /path/to/my-skill

Add --auto-synthetic for a cold-start skill or --blend to combine observed evidence with synthetic gap cases. When --skill-path is present, SelfTune writes the portable package copy under evals/ and keeps a compatibility mirror for existing local workflows.

Flag	Purpose
`--skill NAME`	Select a skill; omit only with `--list-skills`
`--list-skills`	List skills and their trusted-evidence readiness
`--stats`	Show aggregate telemetry statistics for the selected skill
`--max N`	Limit generated entries per side
`--seed N`	Set the deterministic shuffle seed
`--output PATH`, `--out PATH`	Write an additional eval-set copy
`--no-negatives`	Exclude negative routing examples
`--no-taxonomy`	Skip invocation-type classification
`--skill-log PATH`	Override the skill-usage log
`--query-log PATH`	Override the query log
`--telemetry-log PATH`	Override the session telemetry log
`--agent NAME`	Select the agent used for synthetic generation
`--synthetic`	Generate cases from `SKILL.md` without logs
`--auto-synthetic`	Use synthetic cases when trusted triggers are unavailable
`--blend`	Combine observed and synthetic cases
`--skill-path PATH`	Select the skill package used by synthetic generation
`--model MODEL`	Override the generation model
`--help`	Show the current command help

Run mechanical assertions

selftune eval unit-test --skill my-skill --skill-path /path/to/my-skill

Use --generate to draft assertions and --run-agent when the test requires a live agent transcript. Test files are validated before execution. Malformed assertion fields, tags, or test metadata reject the file with a warning; SelfTune does not run a partial suite. Portable eval cases without selftune_assertions are not mechanical tests and are skipped.

Analysis commands

selftune eval composability --skill my-skill
selftune eval family-overlap --skills skill-a,skill-b
selftune eval import --dir /path/to/corpus --skill my-skill

Corpus imports validate optional metadata independently. Invalid category or difficulty values use general or medium; malformed optional skill, tool, and tag fields are omitted without discarding valid neighboring metadata. Run selftune eval <subcommand> --help for the current strict flag surface.

Start

Manage your skills

Improve and validate

Run it your way

For agents

Concepts

Operate safely

Reference

Run output-quality evaluations

Generate routing cases

Run mechanical assertions

Analysis commands

​Run output-quality evaluations

​Generate routing cases

​Run mechanical assertions

​Analysis commands

Run output-quality evaluations

Generate routing cases

Run mechanical assertions

Analysis commands