--- name: llm-council version: 1.0.0 description: Cross-vendor LLM council for hard judgment calls, with parallel answers, Borda ranking, and a chairman verdict. Use for "ask the council". Not for code review (use review). license: MIT argument-hint: " [--mode general|design|plan|research|review --models m1,m2 --cheap]" user-invocable: true metadata: source_pattern: thevgavini/ghcp-llm-council (MIT); T3 delegate transport from t3dotgg/tstack council (MIT) capability: kind: orchestrator owns: [multi-model-deliberation] status: active routing: role: front-door invoker: autoplan trigger: autoplan's long-tail resolver matches a request to ask several models for a verdict user-facing: true intents: - ask the council - council take - llm council - what do the models say --- # LLM Council Three-stage cross-vendor deliberation: 1. Councillors answer the question in parallel. 2. Each surviving councillor ranks every response anonymously. A Borda count aggregates the rankings. 3. A chairman reads Stage 1 and Stage 2 and writes the verdict. ## Triggers `ask the council`, `council take on X`, `what do the models say`, `llm council` ## When to use - **Polystack decisions** where one model drifts toward its own vendor's priors: build or buy, framework choice, vendor selection, architecture forks. - **Judgment-heavy questions** with no clear right answer: prioritization and tradeoff calls. - **Sanity checks** on a decision you are about to commit to. Do NOT use for: - Routine single-opinion questions. A frontier council takes about 90 seconds and 50 to 80 AI Credits on the script route. - Anything time-sensitive or already decided. - Code review: use the `review` skill. For a GPT Sol second opinion on a change, use `sol-loop`. ## Members **Never default to an older or cheaper model when current frontier access is available.** Councils are task-shaped: | Mode | Council | Chairman | |---|---|---| | `general`, `design` | Astra, Opus 5.5, Sol, Sonnet 5 | Astra | | `plan` | Astra, Sol, Sonnet 5, Opus 5.5 | Astra | | `research`, `review` | Sol, Opus 5.5, Sonnet 5, Astra | Opus 5.5 | | `--cheap` | Llama 3.3 70B, DeepSeek V3, Mistral Medium, GPT-4.1 | GPT-4.1 | - **Ambiguous judgment or architecture:** GPT-6 Astra owns ambiguity and acceptance; Opus 5.5 supplies broad frontier judgment; Sol supplies a technical counterweight; Sonnet 5 supplies an investigation-heavy take. - **Clear implementation plan:** Astra owns the plan and acceptance criteria; Sol handles hard technical reasoning; Sonnet 5 probes failure modes; Opus 5.5 provides a broad alternative. - **Technical research or review:** Sol leads the specialist lane; Opus 5.5 checks broad reasoning; Sonnet 5 checks evidence; Astra adjudicates scope. These are routing defaults, not claims that one model wins every harness. The member list is configurable: `--models` replaces the council and `--chairman` replaces the chairman. If the user names members, use those. ## Process Use the first route the harness supports: T3 Code first, then the harness's own subagents, then each member's own CLI. Every route uses the same members, prompts, and Borda count. **If a member is unavailable, report it. Never substitute another model for it.** Run with the members you can reach only while at least 3 remain (or the whole council, if it is smaller); otherwise stop and report which members were missing. ### Route 1: T3 Code delegates 1. Call `orchestrator_capabilities` and resolve each member to the catalog model with the same family and version (`gpt-6-sol` may be listed as `gpt-6-sol` or `GPT-6 Sol`). A search by family name finds candidates, but a different version, such as `claude-opus-4.7` for `claude-opus-5.5`, is not the member: treat the member as unavailable. 2. Write the question to a file and run `council_steps.py init` on a new `--run-dir` with `--unavailable MODEL=REASON` for each missing member. It writes the Stage 1 prompt and prints where each reply goes. 3. Start one `delegate_task` per member in parallel, each with the Stage 1 prompt and a read-only brief: do not edit, commit, push, or start subagents. Save each reply to the path `init` printed. 4. Run `council_steps.py rank` once every councillor has answered or failed. It freezes the survivors; a Stage 1 reply that lands later is listed as late and left out. Start one `delegate_task` per survivor with the Stage 2 prompt, and save each reply. 5. Run `council_steps.py chair`. It runs the Borda count and names rankers that never replied. Start one `delegate_task` on the chairman with the Stage 3 prompt, and save its reply. If `chairman_unavailable` is set, do not pick another chairman: `finish` records the reason. 6. Run `council_steps.py finish` to write the artifact. Do not wait idle: read each reply as it arrives. A timeout, error, or empty reply is a missing answer, not a vote. ### Route 2: the harness's own subagents Outside T3, a harness that already runs several model families delegates with its own subagent tool, one subagent per member, with the same `council_steps.py` steps: - **Copilot CLI**: the `task` tool, which takes a `model` per call. - **Factory Droid, OpenCode**: a subagent (droid or agent) definition whose `model:` names the member. - **Hermes**: its subagent delegation with a model per subagent. A single-family harness reaches only its own family this way: a Claude Code subagent runs Claude members, a Codex `spawn_agent` runs GPT members. A member the harness cannot run moves to route 3, alone and on the same model. If route 3 cannot reach it either, it is unavailable. ### Route 3: each member's own CLI (last resort) `council.py` runs all three stages itself, calling each member's own vendor harness: - Claude members: `claude -p --model --tools ""`, prompt on stdin. - GPT members: `codex exec -m -s read-only -`, prompt on stdin. - Other Copilot catalog ids, or a member whose CLI is not installed: `copilot -p --model `. Charges Copilot AI Credits. - Open-weight `--cheap` members: GitHub Models REST (needs `gh auth login`). No call can write. Copilot keeps `--allow-all-tools` so a permission prompt cannot wedge it. `--available-tools view` hides every other tool, including MCP servers that plugins add. Deny rules (which win over every allow) also remove shell, write, url, memory, and the built-in and user MCP servers. Copilot runs in an empty directory, but its `view` tool can still read the system temp directory, which Copilot allows by default. Codex `-s read-only` blocks writes but not reads, so a member steered by an earlier reply could read files outside its temp directory. Use this route when neither route above can reach the council, or for scheduled runs. Delegates on routes 1 and 2 can use tools. The councillor prompt still says to answer from knowledge only; keep their briefs read-only. ## Review loop mode For a code review, run the members as scored reviewers instead of the three stages (after `t3dotgg/tstack` `council`). Default members are the `review` mode council; use members the user names instead. 1. Before the first round, answer open PR comments, fix known issues, and run the relevant checks, so reviewers see current code. Skip fixes to code the user wants to replace. 2. Start one delegate per member through the routes above, read-only, with the change and its context. Each ends its reply with one line `SCORE: N/10`. Save each reply as `/round-/.md`, where `` is the member id with any character other than letters, digits, `.`, `_`, or `-` replaced by `_` (so `openai/gpt-4.1` is `openai_gpt-4.1.md`). 3. Do not wait idle: check each result against the code as it arrives. Do not edit files while a delegate still reviews the same checkout. A delegate the user marked non-blocking is not waited for, and a late result is feedback whose score is not recorded. 4. Fix what you agree with, then start a fresh round. Each full review pass is one round. Continue until every delegate is satisfied. Fight scope creep: decline what the change does not need. 5. Run `council_review.py table` and put the section in the PR description, before the model and harness line. Edit the open PR's description; do not post a comment. With no PR, put it in the final reply. A delegate with no score shows `no score`; never invent one. ## Scripts Run each from the root of your working checkout with the plugin locator, for example: ```bash ( for d in "${CLAUDE_PLUGIN_ROOT}" "${COPILOT_PLUGIN_ROOT}" .claude; do if [ -n "$d" ] && [ -f "$d/lib/ptk_run.py" ]; then exec python3 "$d/lib/ptk_run.py" skills/llm-council/scripts/council.py "QUESTION" --mode design; fi; done; echo "ptk_run: no plugin root holds lib/ptk_run.py" >&2; exit 2 ) ``` When no plugin root holds `lib/ptk_run.py` (for example Codex, or a plain checkout), run the script beside this SKILL.md directly. Each script finds its own plugin's `lib/`: ```bash python3 "/scripts/council.py" "QUESTION" --mode design ``` ### `council.py` Arguments: `QUESTION`, `--mode`, `--models`, `--chairman`, `--cheap`, `--out`, `--quiet`, `--min-responses` (default 3, at least 1, clamped to the council size). The default output is `/council/runs/.md`. Stderr shows progress; stdout prints the chairman synthesis. | Exit | Meaning | |---|---| | `0` | Synthesis written (a failed chairman is recorded in the artifact) | | `1` | A GitHub Models member needs `gh auth login` | | `2` | Fewer councillors answered than `--min-responses` | ### `council_steps.py` Subcommands `init`, `rank`, `chair`, `finish` over one `--run-dir`. Each prints JSON naming the prompt to send and where each reply goes. | Exit | Meaning | |---|---| | `0` | Step done | | `1` | Fewer councillors answered than the minimum | | `2` | Config error: missing file, empty question, a reused run directory, `chair` before `rank`, or a bad `--unavailable` value | ### `council_review.py` `table --review-dir DIR --members m1,m2,m3` prints the `## Council review` section from `DIR/round-/.md`, reading each `SCORE: N/10`. | Exit | Meaning | |---|---| | `0` | Table printed | | `2` | No `round-` directories, no members, or an unreadable directory | `council_core.py` and `council_backends.py` are shared modules, never run alone. ### Model IDs Copilot CLI (charges AI Credits): `claude-opus-5.5`, `claude-sonnet-5`, `claude-opus-4.7`, `claude-sonnet-4.6`, `claude-haiku-4.5`, `gpt-6-astra`, `gpt-6-sol`, `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.3-codex`, `gpt-5-mini`, `gemini-3.8-flash`, `grok-4.5`, `kimi-k3`. GitHub Models REST (free tier): `meta/llama-3.3-70b-instruct`, `deepseek/deepseek-v3-0324`, `deepseek/deepseek-r1`, `mistral-ai/mistral-medium-2505`, `openai/gpt-4.1`, `openai/gpt-4.1-mini`, `microsoft/phi-4`. The script routes any id in its Copilot list through `copilot -p` and everything else through GitHub Models. ## Artifact - Chairman synthesis (the verdict) - Borda ranking with model attribution - Stage 1: every councillor's full response, labeled and attributed - Stage 2: every ranker's raw output - Errors: each member that failed or was unavailable, with the reason When one response wins consistently, that is the strongest take. When rankings scatter, the council disagreed and the synthesis is doing real bridging work. ## Honest limits - **Self-voting bias is real but bounded.** Anonymous labels prevent direct self-recognition, but models still recognize their own style. The chairman may also be a councillor; it is not an independent judge. - **Unanimity is not maximal confidence.** Borda rewards agreement, but blind consensus can be correlated failure: models that share training data converge on the same plausible but wrong frame (the ensemble "popularity trap" in arXiv:2510.21513). Read the Stage 1 reasoning, not only the ranking. Agreement reached by different routes is strong; agreement on one framing is suspect. - **Copilot CLI adds 22k to 35k input tokens of harness overhead per call** on the script route, so each Copilot councillor takes 15 to 20 seconds. - **The question must be self-contained** on the script route: councillors cannot read your files. Put the context in the question. Questions over 24 KB may exceed a model's context. - **No streaming.** You get the synthesis when it is done. ## Anti-Patterns - Swapping in another model when a member is missing. Report it instead. - Treating a unanimous council as proof. - Making the chairman a councillor when you need an unbiased verdict. Pass a `--chairman` that is not on the council. - Running several frontier councils in a row: 4 runs burn about 250 AI Credits on the script route. Use `--cheap` for low-stakes questions. ## Pitfalls - **Generic output** when the question lacks context. - **Borda parse failures** when a ranker skips the exact `FINAL RANKING:` block. That ranker is excluded; if 2 or more fail, treat the Borda result as suspect. - **GitHub Models rate limits** after 3 or more councils in 5 minutes. - **Copilot CLI wedges** on a permission prompt. The script passes `--allow-all-tools` to avoid it; `--available-tools view` and its deny rules keep the call from writing. If a run hangs, check `pgrep -af 'copilot -p'`. ## Verification - [ ] Every member ran, or is listed as unavailable with a reason; none was replaced. - [ ] At least 3 councillors answered (or the whole council, if smaller). - [ ] The artifact holds the synthesis, the Borda ranking, and both stages. - [ ] Unparsed rankers were named.