# Harness capability map — peer validation log Rubric (unchanged since v1): **2** = first-party documented page/command · **1** = reachable only via a plugin, MCP server, community tool, or a preview/partial surface · **0** = not found in that project's own docs. A correction counts only if it survives a check of the cited URL by the compiler. ## Round 1 — posted 2026-09-15 (the team channel) → v2 compiled 2026-09-15 → v3 compiled 2026-09-15 ### Antigravity (CLI + IDE) — @Gemini, 26 cells confirmed + 1 correction ACCEPTED | cell | v1 | v2 | evidence | |---|---|---|---| | Generates images / audio / video | 1 | **2** | https://antigravity.google/docs/sdk/tools — built-in tools table: `generate_image` = `BuiltinTools.GENERATE_IMAGE`, "Generate or edit images"; https://antigravity.google/docs/models — Nano Banana 2, "used by the generative image tool when the Agent wants to produce a UI mockup … or other generative image tasks" | - **Citation fix:** @Gemini cited `https://antigravity.google/docs/tools`. Checked: that page is *Task Groups* (planning mode, subtasks, pending steps) and never mentions image generation. The claim is correct; the URL was not. - **Model-id nuance:** @Gemini said `gemini-3.1-flash-lite-image`. The docs name *Nano Banana 2*; Google's forum thread says Antigravity defaults to *Gemini 3.1 Flash Image*. Immaterial to the cell. - **URL bug found while checking:** four Antigravity source links were built from the row root (`/docs/overview`) and pointed nowhere real. Repinned to `/docs/models`, `/docs/sdk/mcp`, `/docs/sdk/tools`, `/docs/cli/commands/voice`. All 42 Antigravity + Gemini CLI source URLs re-checked → HTTP 200. ### Gemini CLI — @Gemini, CONFIRMED, no cell changes 30/54 confirmed on its own cells (BYO/local models 1, browser/screen 0, scheduled runs 0, exposes-over-MCP 0, orchestration 1, no desktop/web/mobile/remote/chat/voice/media/IoT, all listed 2s held). Landed at 29 after the creative-row rubric fix below (not a new verdict on Gemini CLI). ### OpenCode — @Honey, 4 up / 1 down / 1 compiler decision (36 → 39) — *revised again in v3.1, see below* | cell | v1 | v2 | evidence | |---|---|---|---| | Lifecycle hooks | 1 | **2** | https://opencode.ai/docs/plugins#events — real event table: `tool.execute.before/after`, `session.idle`, `permission.asked`, `shell.env`, `message.*`, `tui.*` | | Scheduled / recurring runs | 0 | **2** | https://opencode.ai/docs/github#supported-events — `schedule` (cron) + `workflow_dispatch` with a cron example. Caveat kept in the cell note: the clock is GitHub Actions, not a local scheduler. Same shape as Cursor's cloud-agent automations (2). | | Remote-drive a live session | 1 | **2** | https://opencode.ai/docs/server — "supports multiple clients", `opencode attach`, `/tui/*` endpoints drive a running session | | Exposes itself over MCP | 1 | **0** | https://opencode.ai/docs/mcp-servers — client-only; serve surfaces are HTTP/SDK/ACP. A generic MCP host cannot drive it. Aligns with Cursor/Gemini (0). | | Desktop GUI app | 0 | **1** | Compiler decision (reviewer said 2, floor 1): the docs Intro page names "terminal-based interface, desktop app, or IDE extension" and the desktop build is a beta, but there is no docs page for it. Documented ≠ a one-line mention, so 1. A page flips it to 2. **→ 2 in v3.1 on @Honey's evidence; the flip rule was applied to one cell only — see below.** | ### Claude Code — @GLM, 1 correction ACCEPTED (44, net 0 after rubric fix) - Voice in / out **0 → 1**: https://code.claude.com/docs/en/voice-dictation.md — first-party `/voice` hold-to-record / tap-to-record dictation in the CLI. Input only, so 1, not 2. - Logged, not counted: `mcpsrv` 1 (Agent SDK in-process MCP server) stays 1. **Closed:** Cursor `remote` 2 → no downgrade (see below). ### Cursor (+ CLI) — @GLM, 1 correction ACCEPTED (43, net 0 after rubric fix) - Chat-platform presence **1 → 2**: https://cursor.com/docs/integrations/microsoft-teams.md (+ `integrations/slack.md`) — first-party Slack and Teams for cloud agents. Same evidence tier that earned Codex its 2. The old note ("Slack only, not a general bot platform") was wrong on the facts. #### Cursor `remote` — open question raised by @GLM, CLOSED: 2 holds, citation fixed @GLM read `cursor.com/docs/sdk/bridge.md` + the `cursor/sdk-bridge` README and reported that the bridge is a loopback SDK gateway: an adapter spawns or attaches to it and the bridge calls out to the Cursor API. Correct — and it does not attach to an already-running interactive session the way `opencode attach` or Claude `--resume` do. But the bridge is not this cell's mechanism, so it neither earns nor costs the score. Verified by the compiler: **the row is satisfied by Cursor's first-party Remote Control**, which is a live-session handoff. | cell | v2 | v3 | evidence | |---|---|---|---| | Remote-drive a live session | 2 | **2** (holds) | https://cursor.com/docs/cloud-agent/mobile.md#remote-control — "take an agent you're running on your computer and keep directing it from your phone. The agent loop moves to the cloud while its tools keep running on your machine." Enabled with `/remote-control` in the Agents Window; Cursor 3.9.8+; Pro/Pro+/Ultra/Teams/Enterprise | - **Citation fix (the real defect in this cell):** Cursor's 2 was resting on the bare row root `https://cursor.com/docs` with the boilerplate note "First-party documented capability." Repinned to `cloud-agent/mobile.md#remote-control` with a real note. Same class of bug the Antigravity pass caught, now fixed on three cells. - **@GLM's second finding recorded, not absorbed:** first-party *remote-originated* driving also exists — Cloud Agents API over plain HTTP (https://cursor.com/docs/cloud-agent/api/endpoints.md) and TS/Python SDK **cloud** runs that "survive the caller disconnecting", reattachable by `Agent.resume(agentId)` from anywhere. That is a `headless` / `sched` surface, where Cursor already scores. Not this row. - **Row semantics tightened** so the question cannot recur: description now reads "Attach to or steer an already-running session from another device or chat channel — not remote-starting a fresh cloud run." - **No split recommended.** @GLM offered a split (cloud-run scripting vs live-session attach). Declined for now: the row's plain definition already covers both legs (another *device* — Cursor Remote Control, Claude remote-control, Antigravity `agy remote-control`; another *chat channel* — Hermes gateway, and Cursor's own Slack/Teams). Splitting changes the rubric mid-round for no score movement. One-line revert if the group wants it. - **Audit fallout — two more boilerplate citations on this row, both scores unchanged:** `remote.codex` → https://learn.chatgpt.com/docs/remote.md ("Start, guide, approve, and review Codex tasks on a connected computer from your phone") and `remote.antigravity` → https://antigravity.google/docs/remote-control (`agy remote-control start`, machine registers in the Remote Control hub, background agents and shell commands keep running on the host). Both were previously the row root + boilerplate note. - **Bridge verdict kept for the record, because it is the right answer to a different question:** the SDK Bridge is a programmatic-SDK gateway for languages without a first-party SDK (Go, Rust, Java, C#), served on a loopback port over Connect/HTTP/1.1 — classic gRPC over HTTP/2 will not connect. The docs say so plainly: use the TS or Python SDK unless you need a language they do not cover. It is not an attach mechanism and was never going to be one. ### Codex — @Luna, 27 cells walked: 2 corrections, 25 confirmed (49 → 47) | cell | v2 | v3 | evidence | |---|---|---|---| | Exposes itself over MCP | 2 | **0** | https://learn.chatgpt.com/docs/codex-manual.md, `#### codex mcp-server`: "The `codex mcp-server` command and standalone `codex-mcp-server` binary have been removed. Use the Codex app server instead." | | Drives external creative apps | 1 | **1** (already) | https://github.com/DigitalWorkflowCompany/resolve-mcp — same community MCP server; corroborates the v2 rubric fix rather than adding a second deduction | - **Verification of the removal claim (compiler, 2026-09-15):** the CLI reference in the current docs bundle is explicit — command *and* standalone binary removed, with the app server named as the replacement. The old `https://learn.chatgpt.com/docs/mcp-server` page still resolves, but as a **tombstone**: its header reads "Deprecated. Use the Codex app server or the Codex plugin for Claude Code", body copy says "This page documents the deprecated command for existing integrations", and the docs index (`llms.txt`) files it under **Deprecated**. A tombstone is not a documented capability, so the cell is 0 — and the row finally has a zero in it, which is what the v1 challenge asked for. - **Why the replacement does not hold the cell at 2:** the app server is a JSON-RPC protocol (`thread/start`, `turn/*`, `mcpServer/tool/call` — https://learn.chatgpt.com/docs/app-server.md). It *consumes* MCP servers; it does not present Codex as one. A generic MCP host cannot attach to Codex any more. - **Tombstone nuance logged:** `codex mcp-server` printed a deprecation warning on stable before removal, and the removal commit sat on the alpha line first. Both are consistent with the current manual; neither changes v3. - **Open question, not counted:** `npx @openai/codex-security mcp add` registers the *Codex Security* CLI as an MCP server (https://learn.chatgpt.com/docs/security/cli/reference.md) — but it "exposes only the read-only `info` metadata command", and Codex Security is a sibling product, not the Codex agent loop. If the group decides a sibling CLI counts, this cell is **1**, not 2. Deliberately not absorbed into the score. - **Arithmetic:** @Luna's reported figure (47/54 · Where 16 · How 17 · Beyond 9 · Guard 5) matches the compiler's independent recompute exactly. ### Rubric fix — raised by @Honey, APPLIED to all rows *Drives external creative apps* was 2 for every MCP-capable harness, sourced to one community MCP server (https://github.com/DigitalWorkflowCompany/resolve-mcp). Our own definition of **1** is "reachable only via a plugin, MCP server, community tool" — so by the published rubric that row is **1**, and the v1 note ("First-party documented capability") was simply false for that cell. Re-scored to 1 for the 8 MCP harnesses; Aider stays 0. The flat-row claim survives and reads better: 8 of 9 tie at 1 because nobody ships it. An audit of every 2 in the table found no other cell resting on a third-party source. Reversal is one line if the group prefers the loose reading — say so on the board. ### Post-v3 — OpenCode *Desktop GUI app* **1 → 2**: compiler reversal, @Honey's evidence verified (OpenCode 39 → 40) | cell | v3 | v3.1 | evidence | |---|---|---|---| | Desktop GUI app | 1 | **2** | https://opencode.ai/download — page title *Download OpenCode Desktop*, per-platform artifacts (macOS Apple-Silicon + Intel DMG, Windows x64 NSIS, Linux `.deb`/`.rpm`/`.AppImage`) and `brew install --cask opencode-desktop` / Scoop. The repo README carries the same matrix under `### Desktop App (BETA)`: https://github.com/anomalyco/opencode | - **Why the earlier 1 was wrong — and it is not only about the URL.** The compiler had stated the flip rule as "a docs page flips it to 2", then held the cell at 1 because `/download` is not inside the `/docs/*` tree. But that docs-tree rule was **never applied table-wide**: Antigravity's *Desktop GUI app* has been a **2** since v1, sourced to a product page (`antigravity.google/product/antigravity-2`), not to the docs tree. Enforcing it here would have meant inventing a stricter standard for exactly one cell — the same class of error as the creative-row slip, in the other direction. - **What the cell actually tests:** whether the *vendor* documents and ships the surface. A download page with per-platform installers and package-manager casks is *stronger* evidence than a docs page carrying one line of copy — which was the original, and still valid, objection to scoring 2 off the Intro-page mention alone. - **Beta is a caveat, not a deduction.** It stays pinned in the cell note. This rubric measures documented breadth, not maturity or stability; a first-party, downloadable, documented surface is a 2. - **Published rubric clarification, so this cannot recur** (now in the Method tab and the checklist header): *"a first-party page" means any vendor-published page — product / download / install pages count, not only `/docs/*` paths.* The 0 / 1 / 2 definitions themselves are unchanged. - **Concurrency note:** a sibling run was writing this same log while the reversal landed (its own section on @GLM's Cursor `remote` question, above). Both edits are additive; neither regressed the other. Two writers on one artifact set is a collision risk — flagged to the group rather than silently merged. ## v2 board ``` Codex 49/54 Where 16 · How 19 · Beyond 9 · Guard 5 Hermes Agent 49/54 Where 13 · How 20 · Beyond 11 · Guard 5 Antigravity (CLI + IDE) 45/54 Where 14 · How 19 · Beyond 7 · Guard 5 Claude Code 44/54 Where 14 · How 19 · Beyond 6 · Guard 5 Cursor (+ CLI) 43/54 Where 14 · How 18 · Beyond 6 · Guard 5 OpenCode 39/54 Where 14 · How 17 · Beyond 2 · Guard 6 Gemini CLI 29/54 Where 8 · How 15 · Beyond 1 · Guard 5 Grok Build 14/54 Where 6 · How 5 · Beyond 2 · Guard 1 Aider 12/54 Where 6 · How 2 · Beyond 2 · Guard 2 ``` ## v3 board ``` Hermes Agent 49/54 Where 13 · How 20 · Beyond 11 · Guard 5 Codex 47/54 Where 16 · How 17 · Beyond 9 · Guard 5 (was 49 — 1 correction) Antigravity (CLI + IDE) 45/54 Where 14 · How 19 · Beyond 7 · Guard 5 Claude Code 44/54 Where 14 · How 19 · Beyond 6 · Guard 5 Cursor (+ CLI) 43/54 Where 14 · How 18 · Beyond 6 · Guard 5 OpenCode 39/54 Where 14 · How 17 · Beyond 2 · Guard 6 Gemini CLI 29/54 Where 8 · How 15 · Beyond 1 · Guard 5 Grok Build 14/54 Where 6 · How 5 · Beyond 2 · Guard 1 Aider 12/54 Where 6 · How 2 · Beyond 2 · Guard 2 ``` ## v3.1 board ``` Hermes Agent 49/54 Where 13 · How 20 · Beyond 11 · Guard 5 Codex 47/54 Where 16 · How 17 · Beyond 9 · Guard 5 Antigravity (CLI + IDE) 45/54 Where 14 · How 19 · Beyond 7 · Guard 5 Claude Code 44/54 Where 14 · How 19 · Beyond 6 · Guard 5 Cursor (+ CLI) 43/54 Where 14 · How 18 · Beyond 6 · Guard 5 OpenCode 40/54 Where 15 · How 17 · Beyond 2 · Guard 6 (was 39 — compiler reversal) Gemini CLI 29/54 Where 8 · How 15 · Beyond 1 · Guard 5 Grok Build 14/54 Where 6 · How 5 · Beyond 2 · Guard 1 Aider 12/54 Where 6 · How 2 · Beyond 2 · Guard 2 ``` Only OpenCode moved (+1, one cell). The creative-row rubric fix stands as applied — @Honey re-affirmed the strict reading, so there is no revert to make. **What moved, and why it is worth more than the total:** Codex was the row the v1 post dared people to break ("no zeros at all"). It broke on a *protocol* cell rather than a feature cell. The widest product surface in the table — desktop, web, mobile, computer use, Slack/Linear/GitHub/GitLab — also turned out to be the one that withdrew from serving itself over MCP, while the smallest row that still does (`claude mcp serve`, `hermes mcp serve`) kept it. Downward corrections are the ones that make the rest of the count credible. Still open: Grok Build's DM reply (xAI row, 23 non-2 cells) · rows not yet owner-verified with a runtime — Claude Code, Cursor, Aider, Hermes (the two web-research passes here are @GLM's spot checks, which are docs-based, not runtime) · the `codex-security mcp` open question above · six coarse Codex citations below. ## Citation repairs — compiler sweep, 2026-09-15 (no score changes) @Luna flagged five stale Codex citation paths alongside the Codex verdict. Confirmed, fixed — and the sweep was widened to **every** source URL in the table (110 unique) instead of the five reported. **Codex — five root-joined paths that 404'd.** Four were built by joining a docs path onto the row root (`developers.openai.com/codex/cli/` + `/features/voice.md` → `…/cli//features/voice.md`, which redirects into a `learn.chatgpt.com/docs/cli/…` 404). Repinned to the canonical first-party pages: | cell | was | now | |---|---|---| | Voice in / out | `…/codex/cli//features/voice.md` (404) | https://learn.chatgpt.com/docs/features/voice.md | | Generates images / audio / video | `…/codex/cli//image-generation.md` (404) | https://learn.chatgpt.com/docs/image-generation.md | | Chat-platform presence | `…/codex/cli//third-party/slack.md` (404) | https://learn.chatgpt.com/docs/third-party/slack.md | | Bring-your-own / local models | `…/codex/cli//amazon-bedrock.md` (404) | https://learn.chatgpt.com/docs/amazon-bedrock.md | | Multi-agent orchestration | `…/codex/cli//agent-configuration/subagents.md` (404) | https://learn.chatgpt.com/docs/agent-configuration/subagents.md | - The last one is the same bug on a cell that was not in @Luna's list — which is the argument for sweeping rather than patching the reported five. - `remote` needed nothing: it was already pinned to https://learn.chatgpt.com/docs/remote.md, and that page ("Codex Remote — start, guide, and review coding tasks from your phone") is what holds the cell at 2. - **Voice note tightened, score unchanged.** `/docs/features/voice` is *ChatGPT Voice* in the ChatGPT desktop app, and it documents starting a voice chat inside a Codex task — so the 2 stands, but the old note ("a ChatGPT surface, not a Codex CLI feature") undersold the Codex path. Rewritten to say what the page says. **Three more 404s the sweep found outside the Codex row, repaired the same way (links only, no scores touched):** | cell | was | now | |---|---|---| | Claude Code · Exposes itself over MCP | `…/claude-code/overview/agent-sdk/overview` (404) | https://docs.claude.com/en/docs/claude-code/agent-sdk/overview | | Claude Code · Chat-platform presence | `…/claude-code/overview/slack` (404) | https://docs.claude.com/en/docs/claude-code/slack | | Hermes Agent · Remote-drive + Chat-platform presence | `…/user-guide/messaging/index` (404) | https://hermes-agent.nousresearch.com/docs/user-guide/messaging | - Same root-join bug across three vendors, so it was a **builder** defect, not a Codex defect — fixed in `build.py` (score arrays, `NOTES` tuples, `SRC` dict) and in the rendered output, in that order. - **Sweep result: 110 unique source URLs → 109 × HTTP 200, plus one transient `000` on `antigravity.google/docs/sdk/tools` that returned 200 on retry.** Re-running the sweep is now part of a release. **Coarse-but-live citations (logged, not repaired):** six Codex cells still fall back to the row root (`developers.openai.com/codex/cli/`, HTTP 200) instead of a cell-specific page — `cli`, `mobile`, `oss`, `customtools`, `iot`, `enterprise`. Not broken, just under-pinned: `mobile` in particular earns its 2 from the ChatGPT mobile app, not from the CLI index. Candidates exist (`/docs/remote.md` for mobile, `/docs/developer-commands.md?surface=cli` for custom tools) but pinning them is a per-cell citation decision, so it waits for the next pass rather than being guessed. ## v3.2 — Pi added as a 10th column (compiler, 2026-09-15) **Not a peer correction: a scope addition, requested by @Erich in #the team channel.** Pi (https://pi.dev/, `@earendil-works/pi-coding-agent`, MIT, Mario Zechner / Earendil Inc.) was scored from its own docs tree — `packages/coding-agent/docs/*` (32 pages, listed via the GitHub contents API) plus the first-party package catalog on pi.dev/packages — not from the marketing page. Lands at **26/54**: Hermes 49 · Codex 47 · Antigravity 45 · Claude 44 · Cursor 43 · OpenCode 40 · Gemini 29 · **Pi 26** · Grok Build 14 · Aider 12. No existing cell moved. **The column caveat ships with the column.** Pi is the only harness here that deliberately ships almost nothing and documents the absence: no MCP, no sub-agents, no plan mode, no permission popups, no to-dos, no background bash (`docs/usage.md`: *"It intentionally does not include built-in MCP, sub-agents, permission popups, plan mode, to-dos, or background bash"*). Its count is a count of **built-ins**; its reach lives in a 5,400-package catalog where MCP (`pi-mcp-adapter`, 866K installs/mo), sub-agents (`pi-subagents`, 412K/mo) and workflow scripting are one `pi install` away. Read the column as *ships less*, not *can do less* — it is the cleanest worked example of why an unweighted count is a breadth map. **Scoring rule applied to 1s, stated so it can be argued with:** a cell is **1** when a documented route exists in Pi's own docs *or* in the first-party package catalog with real adoption, and **0** when the only route is to write it yourself or to use a separate product. Under that rule: mobile 1 (Termux install — the same standard already applied to the Hermes row), remote 1 (rpc/sdk can steer a running process; no live multi-device attach), mcp 1, subagents 1, orchestrate 1, computer 1, creative 1 (double hop: Pi ships no MCP client), sandbox 1 (`security.md` says outright there is no built-in sandbox and no permission popups; isolation comes from containers or extensions — literal rubric reading). Zeros: IDE (no ACP either — contrast Grok Build's 1), desktop, web client, chat platforms, voice, media generation, IoT, sched, exposes-over-MCP, enterprise. OpenClaw drives Pi from chat channels via the SDK but is a separate product, so `messaging` stays 0. **Two cells flagged, not decided alone:** `memory` (AGENTS.md at `~/.pi/agent/` + per-project SYSTEM.md — durable instructions, first-party documented, but instruction files are not a fact store, so 1 is arguable) and `mcpsrv` (the third-party `pi-cursor-sdk` bridges Pi's tools into Cursor over a local MCP server — logged, not counted, the same handling the `codex-security mcp` question got on the Codex row). **Timeline tab added in the same revision** (requested in the same message): *Public releases* — monthly stable releases from each project's own public release feed, cumulated: Claude 219, Codex 159, Gemini 315, OpenCode 869, Pi 259, Hermes 34, Aider 93; no public feed exists for Cursor, Antigravity or Grok Build (grok-build publishes neither releases nor tags), so those rows are absent by fact. Aider's feed stops **2025-08-09**. *Board score* — this map's own revisions, with v1 reconstructed from the recorded per-cell before-values. Neither series is a capability-introduction date and the method tab says so; a per-cell first-documented date would need a vendor-changelog harvest and is not faked here. **Citation sweep for the new row:** 19 Pi source URLs + the row root, all checked → HTTP 200. Pi's docs pages are cited from the repo tree (`github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/.md`) because `pi.dev/docs/latest/` is a rendered mirror of the same files. **Naming drift logged** (same trap as Grok Build ≠ Grok Bot): older write-ups cite `@mariozechner/pi-coding-agent` and `github.com/badlogic/pi-mono`; the npm scope moved to `@earendil-works/` and the domain was donated by exe.dev. Cite the current package. ## Verification receipt — OpenClaw 2.0 verdict (independent check, 2026-09-15) Second agent, same board. The OpenClaw 2.0 verdict was checked against the cited pages rather than taken on trust. Both claims hold; two citations were repaired; no score moved beyond the one correction that was already submitted. **1. `Exposes itself over MCP` 0 → 2 — ACCEPTED, and the page says it plainly.** https://docs.openclaw.ai/cli/mcp/serve opens with *"This page covers the `openclaw mcp serve` path: OpenClaw acting as an MCP server over stdio, its tools, its event model, and its limits."* The mechanics are documented end to end: the MCP client spawns `openclaw mcp serve`, the bridge connects to a local or remote Gateway over WebSocket, routed sessions become MCP conversations plus transcript/history tools, and live events queue for the client, with approval tools (page keyword census: `conversation` ×23, `event` ×21, `message` ×19, `approval` ×13, `transcript` ×10, `stdio` ×11). That is a serving surface, not a client config. Worth recording next to it: **in the same round the Codex row lost its serve surface while OpenClaw 2.0 gained one** — the map now has exactly two rows with a documented `… mcp serve` path (Hermes, OpenClaw 2.0) and one that removed it (Codex). **2. Chat-matrix attack — REBUTTAL STANDS, and the evidence is stronger than the rebuttal claims.** The channels index (https://docs.openclaw.ai/channels) enumerates **all 23 platforms the reviewer named** plus more (QQ bot, Yuanbao, A2A, ClickClack, Raft, Reef, Zalo personal, WebChat), each with its own page and an explicit tier label — *included in core* / *bundled plugin* / *official plugin* / *external plugin*, with setup, access-control and troubleshooting sub-pages. That is a documented matrix, not stub copy. The caveats the reviewer listed are the tier labels and per-channel limits, i.e. real, and they do not touch the cell: **19 of the 23 are core/bundled/official**, so `Chat-platform presence` = 2 holds even under the strict "plugin-only = 1" reading — only the four *external-plugin* channels (WeChat, WeCom, Yuanbao, Zalo ClawBot) would move, and the cell counts the class, not one channel. Logged, not re-scored. **3. `Home automation / IoT` = 0 — CONFIRMED, and now pinned to the boundary.** The nearest path is https://docs.openclaw.ai/automation/cron-jobs/webhooks; a full-text scan of that page for `Home Assistant`, `MQTT`, `Matter`, `Zigbee`, `sensor`, `smart home`, `IoT` returns **zero** hits — the automation surface is generic webhooks, so 0 stands with a citation that shows why instead of the row root. **Citation repairs on this row (links only):** - `Voice in / out`: was pinned to https://docs.openclaw.ai/tools/tts, which is speech **out** only and undersold a 2. Repinned to https://docs.openclaw.ai/nodes/talk — *"continuous speech conversations across local STT/TTS and realtime voice"* — with the inbound voice-note page (`/nodes/audio`) and the voice-call plugin noted as the telephony path. Same score, better evidence. - `Home automation / IoT`: row root → the webhooks page above. - Left alone deliberately: the reviewer offered `tools/screen` and `tools/media-overview` where the row carries `tools/browser` and `tools/video-generation`. Both pairs are live and both document their cell, so the existing pins stand rather than being churned; the umbrella pages are the broader citation if anyone prefers it. **Sweep:** all 25 distinct URLs in the OpenClaw 2.0 verdict → HTTP 200, verified before counting anything. **Served state (re-fetched over HTTP after publish, not read from disk):** 324 rows = 12 harnesses × 27; OpenClaw 2.0 **49/54** (Where 15 · How 20 · Beyond 9 · Guard 5) — identical to the reviewer's own recompute; OpenClaw 1.x 47; board otherwise unchanged. Published artifacts had drifted ~15 minutes behind `build.py` when this check started, so `checklist.*`, `data.json`, `index.html` and `preview.png` were regenerated and republished as part of the same pass. Third artifact collision of the night between two writers on one pipeline — the flag from v3.1 stands: whoever holds the row should publish, and the verifier should verify. ## Round 2 — posted 2026-09-15 23:44 (the team channel) → v4.2 compiled 2026-09-16 Two new columns: **OpenClaw 2.0** (release v2026.8.1, 2026-08-30 — 16,000+ PRs, roughly half of every PR ever merged) and **OpenClaw 1.x** (last release v2026.7.1, 2026-07-13). Board is 12 harnesses x 27 capabilities = 324 cells. The headline under attack: *16,000 PRs moved the breadth score by one point* (47 vs 46), explained as "rebuild, not expansion". ### @Gemini — falsification of the headline. 2 of 3 cell corrections accepted, 1 rejected. | cell | we said | verdict | evidence | |---|---|---|---| | OpenClaw 2.0 · Exposes itself over MCP | 0 | **2 — accepted** | https://docs.openclaw.ai/cli/mcp/serve — a dedicated first-party page for `openclaw mcp serve`: a stdio MCP server exposing routed conversations (`conversations_list`, `conversation_get`), transcripts (`messages_read`, `attachments_fetch`), live event waiting (`events_poll`, `events_wait`), `messages_send` and approval tools, with a Docker smoke test. We had flagged this exact cell as the one that moves if a serve surface existed. It exists. | | OpenClaw 1.x · Generates images / audio / video | 1 | **2 — accepted** | https://github.com/openclaw/openclaw/releases/tag/v2026.4.5 (2026-04-06) ships the built-in `video_generate` tool and the built-in `music_generate` tool (Google Lyria, MiniMax) — four months before 2.0. The v2026.7.1 notes then harden "OpenAI video generation", "Google Veo video generation", BytePlus and OpenRouter paths. Our published condition was: date it to a pre-2026.8.1 release and the cell becomes 2. | | OpenClaw 1.x · Multi-agent orchestration | 2 | **2 — correction rejected** | The claim ("Swarm, Fleets and TaskFlows did not exist in v2026.7.1 — 0 matches") fails on Task Flow. Grep of the v2026.7.1 notes: `swarm` 0, `fleet` 0, but `TaskFlow`/`taskflow` **3** and `Workboard` **13**. Hits include "TaskFlow-only validation" (#102036), "recoverable TaskFlow state" (#97358), "preventing linked TaskFlows from falsely reporting them missing" (#103946). The current Task Flow page titles itself **"Task Flow (formerly ClawFlow)"** — a 1.x-era name, i.e. direct evidence of pre-2.0 lineage. A board plus a flow-orchestration layer is a 2 under our own rubric (Hermes scored 2 on a kanban board). What 2.0 added is **Swarm** (collector children, `agents_wait`, bounded fan-out) and **Fleets** *on top of* an orchestration layer that already existed. | ### @GLM — OpenClaw 1.x row: evidence accepted, verdict overridden by the published rule @GLM dated video/music generation to **v2026.4.5 (2026-04-06)** — earlier than @Gemini's v2026.7.1 — and then said "cell stays at 1". Both reviewers agree on the date; our posted rule says a dated pre-2.0 release moves the cell to 2. The rule decides: **1.x media generation 1 → 2**. (If the intent was "this cell is about what 2.0 added", that is a different question from what the row scores.) **Addendum — @GLM withdraws the objection; the cell is now peer-confirmed, not overridden.** Their follow-up states the disagreement was a scoring error, not a rubric gap: the cell was silently anchored to the existing 1.x subtotal (46/54) instead of being evaluated against the cell's own content, and they ask for **no** rubric entry. Verified by the compiler on the current artifact before recording it: `mediagen.openclaw1` = 2 and OpenClaw 1.x totals 47/54, so the published figure they endorse is the one on the board. - **Load-bearing citation re-read, not taken on trust:** https://github.com/openclaw/openclaw/releases/tag/v2026.4.5 (06 Apr, signed tag) reads "add the built-in `video_generate` tool … return the generated media directly in the reply" and "add the built-in `music_generate` tool with bundled Google (Lyria) and MiniMax providers plus workflow-backed Comfy support". Built-in, first-party, pre-2.0. The cell stands on the release notes. - **#77244 verified and filed as corroboration only** (https://github.com/openclaw/openclaw/issues/77244 — "Generation tools Not Live despite provider API keys … on v2026.5.3"): it independently shows `image_generate`, `video_generate` and `music_generate` present in the **1.x** catalog on another pre-2.0 release, which is real evidence these are not 2.0 additions. It is *not* promoted to the citation: it is a user bug report about those tools being absent from `tools.effective` on one install, so it corroborates 1.x-era *existence*, not the built-in-and-working claim — and the rubric wants a first-party page/command for a 2. Keep it secondary. ### The four "new capability categories" — 1 verified, 1 partly verified, 2 do not survive 1. **Multi-agent orchestration engine (Swarm, Fleets, TaskFlow)** — *partly verified.* Swarm and Fleets are new (0 mentions in the 1.x notes; their PRs — #111379, #111411, #111505, #112989, #123831 — all sit above the ~106k PR ceiling of the v2026.7.1 notes). TaskFlow is **not** new. Verdict: a genuine new coordination architecture added on top of an existing orchestration layer. 2. **Native OS desktop screen control** — *does not survive as cited.* https://docs.openclaw.ai/tools/screen documents a **Control UI layout/navigation** tool and says outright: "It is a typed layout and navigation surface, not screenshot capture or browser automation." It cannot read pixels, screenshot, or click page content. The v2026.7.1 notes also already reference **Codex Computer Use** desktop controls (#96730, #96818, #97955), so "1.x only had remote browser control" is wrong on its face. `/tools/computer-use` returns 404. Not counted. 3. **Claws packaging** — *unproven and off-rubric.* The cited PR #102406 sits **below** the 1.x-era PR ceiling, so it cannot date the feature; `claws` appears in the 1.x notes only as contributor handles (@clawsean). Packaging is also not a capability category in this rubric — skills and plugins are already 2 in both columns. **Citation repair that fell out of this:** 2.0's *Browser and screen control* was sourced to `/tools/screen`; repinned to https://docs.openclaw.ai/tools/browser, which documents real browser control. 4. **Autonomous mailbox watch / IMAP triggers** — *verified.* `imap`/`mailbox` = 0 in the 1.x notes; the 2026.8.1 automations page documents the bundled IMAP watcher (disabled by default, inbound-only, sender allowlisting, cannot send or modify mail) with PRs #130230 and #131060–#131178 — all far above the 1.x ceiling. New inbound trigger surface. No score change (`sched` is 2 in both columns), but it is real expansion. ### v4.2 board ``` Hermes Agent 49/54 OpenCode 40/54 OpenClaw 2.0 49/54 Gemini CLI 29/54 Codex 47/54 Pi 26/54 OpenClaw 1.x 47/54 Grok Build 14/54 Antigravity (CLI + IDE) 45/54 Aider 12/54 Claude Code 44/54 Cursor (+ CLI) 43/54 ``` **Net effect on the claim:** the corrected delta is **two points** (49 vs 47), not one — and it is two points for a reason that cuts both ways: **one under-credited cell in each column** (2.0's MCP serve mode, 1.x's video/music generation). The *explanation* offered for the one-point figure — "rebuild, not expansion: depth, not new capability categories" — does **not** survive: Swarm/Fleets is a new coordination architecture and the IMAP watcher is a new inbound-trigger surface, both absent from the 1.x release notes. Of the four categories claimed as new, two hold and two do not. ## Verification receipt 2 — chat-platform counts, version-pinned (independent check, 2026-09-15) The chat-platform walk was re-counted from the reviewer's own version-pinned primary sources rather than from the docs site, because a version-pinned file is the only way to make a *count* mean the same thing twice. | tag | linked entries in `docs/channels/index.md` → Supported channels | tier census | |---|---|---| | `v2026.8.1` (2.0) | **33** including Voice Call → **32** excluding it | 3 bundled · 25 official-plugin · 1 included-in-core · 4 external-plugin | | `v2026.7.1` (1.x) | **28** including Voice Call → **27** excluding it | 22 official-plugin · 3 included-in-core · 3 external-plugin | Both of the reviewer's numbers reproduce **exactly** (32 / 28, +1 with Voice Call), and the bundle/official/external tier labels the cell note claimed are real and present in both tags. Delta between the two tags, computed name by name: **2.0 added A2A, Buzz, ClickClack, Reef, WeCom; nothing was dropped.** (Correction to our own earlier claim: "Zalo personal" is not a 2.0 addition — it exists in 1.x as "Zalo Personal"; only the capitalisation changed.) **Pins upgraded:** both rows now cite the version-pinned raw files (`.../openclaw/openclaw/v2026.8.1/docs/channels/index.md`, `.../v2026.7.1/docs/channels/index.md`) instead of the moving docs page and a release note. No score changed. ### Open question — the "30+" is a note, not a rule (raised here, not absorbed) The walk reads the cell as having a **30-documented-pages threshold** and therefore concludes 1.x (28) does not clear it. That would take OpenClaw 1.x *Chat-platform presence* 2 → 1 and the row 47 → 46. It is logged, **not applied**, for one reason: **our published rubric has no page-count threshold.** The wording "30+ documented channel pages" entered the board as *descriptive copy in the 2.0 cell note*, and the Method tab says only 2 = first-party documented page/command · 1 = plugin/MCP/community/preview · 0 = not found. Turning a description into a rejection rule after the fact is criterion drift — the same failure mode as the creative row scoring 2 off a community MCP server. It is also not a one-cell change. Applied consistently, a 30-page threshold would move **Hermes** *Chat-platform presence* (19 gateway platforms) 2 → 1, would fail **Claude Code** and **Cursor** outright (Slack + Teams, two integrations each), and would re-open every row whose cell rests on a documented surface rather than a page count. If the group wants breadth thresholds, they belong in the Method tab, published before the next round, with every messaging cell re-scored in one pass — a v4 rubric change, not a correction to one row. Until then: **OpenClaw 2.0 messaging 2, OpenClaw 1.x messaging 2**, on 32 vs 27 documented channel pages, both first-party. Recorded in the served checklist as cell notes so the reasoning travels with the numbers. ## v4.3 — clarity / public-readiness pass (peer clarity review, 2026-09-15 21:07; forwarded by @ailona 21:09 "review this thread") Reviewer's scope was presentation and interpretation, not cells. Every item was checked against the artifact before being accepted or rejected; no cell, score or harness moves in this revision. Accepted (some modified) / rejected: **Accepted, live in v4.3** - **Metric stated before anything else.** The Counts view rendered an ordered bar list with a `★` on the top scorer, which does read as a benchmark leaderboard. The star is gone (it carried no information the total doesn't) and the header now leads with *"Metric: documented operational breadth — how much surface a project documents and ships. Not coding quality, not output quality, and not a ranking: a deliberately lean core scores lower on purpose."* (Pi and Aider are the rows that sentence exists for.) - **Group split readable.** `Where 13 · How 20 · Beyond 9 · Guard 5` → full group names with their own maxima: `Where it runs 13/16 · How you extend it 20/20 · Beyond code 9/12 · Guardrails & model choice 5/6`. - **Matrix legibility.** Added group filter chips (All + the four groups) and a horizontal *"Columns, in order: 1. Claude Code … 12. Pi"* key above the grid, because twelve vertical headers are hard to decode. Zero cells now render `0` instead of `·` (`·` read as missing data), and the zero-cell colour moved `#4a5a6e` → `#8b9bb1` — **2.18:1 → 5.44:1**. The reviewer's "~2.3:1, fails AA" was accurate. - **Export links.** Header now carries *Download CSV (324 cells)*, *Markdown checklist* and *Audit trail*. All three were already served from the same origin (checked 200 before linking, not after). - **Desktop layout (>900px).** Nav becomes a sticky top bar and the evidence sheet becomes a right-hand inspector instead of a 62%-height bottom drawer. - **Timeline labels.** *Board score* → *Board revisions*; axes now read *"cumulative public releases — publish cadence, not capability count"* and *"board score out of 54 — the map converging, not the products"*. - **Method copy.** The stale example numbers ("A 51 is not better than a 44") are generic now, and the page footer no longer carries the internal room name — it points at this file instead. - **Factual repair — the finding that mattered.** The OpenClaw 1.x row note still ended *"2.0 moved depth, not categories,"* which contradicts the same round's own record (Swarm, Workboard, Fleet and Task Flow are new coordination surfaces; the two OpenClaw rows differ by two points, not one). Corrected in the row note. **Discipline note:** this was nearly dismissed as false because the first check looked in cell notes (`notes.*`) and the phrase lives in the row note (`harnesses[].note`). Search both fields before calling a reviewer's finding wrong. **Rejected, or corrected** - **"Board score plots v1 → v4.2, compiled across ~24h — viewers think agents evolved overnight."** The figure is wrong: every revision including v4.3 was compiled on 2026-09-15. The x axis is revision labels, not time, and the footnote already said the series shows the map converging. Took the relabel; declined removing the mode — it *is* the audit trail, and hiding it is one click. - **"Rotate the matrix headers to 45°."** Declined for this pass, and not on merit: the machine that builds this has no browser or headless Chromium, so a rotated-header layout cannot be visually verified before publishing. Left vertical labels plus the horizontal column key instead. Revisit when a real browser check is available. - **"Filter chips: Open Source Only / IDE-Based / CLI-Only / Diff Only."** The data model has no per-harness tags of that kind (oss/ide/cli exist as capability *rows*, not as row attributes) and "Diff Only" has no definition on a capability map. Shipped group filters, which the data does support. - **"Sanitize 're-run the harvest'."** Kept — it is the maintenance contract, not jargon: the harvest is dated precisely because the numbers rot. The room reference in the footer was dropped per the review. Publication metadata (publisher / license line) stays an owner decision; claiming MIT would be false, since this artifact declares no license. - **Poster defect the review would have hit next:** at 12 columns the heatmap header labels were wider than the ~63px cell pitch and overlapped (read as `OpenClawOpenClawCursor`). Short labels (OC 2.0 · OC 1.x · AGY · Grok …) and a re-render; verified clean. ## Round 3 — Claude Code column, reviewed from inside a running Claude Code session (Opus), 2026-09-16 → v4.4 Handed over by the owner as a document; the compiler re-verified every claim before applying it (the review itself asked for that, and it was right to). Verification: all 14 cited URLs probed independently, the `claude mcp serve` section read on the live page, the redirect claim reproduced. ### Accepted | # | item | change | evidence | |---|---|---|---| | 1 | Exposes itself over MCP **1 → 2** | **score** | `## Use Claude Code as an MCP server` is a dedicated section on the first-party MCP page: `claude mcp serve`, stdio transport, paste-ready `claude_desktop_config.json` client block. Under the rubric (2 = first-party documented page/command) this is a 2; the cell had been scored off the Agent SDK overview, where `createSdkMcpServer` runs the other direction. Claude Code 44 → **45/54**. | | 2 | Browser & screen control 2 | citation repair | The cited computer-use page self-labels a research preview, macOS-only, Pro/Max, not on Team or Enterprise — a 1 under our own preview rule. The 2 rests on Claude in Chrome (GA). Re-pinned to `/docs/en/chrome` and the note now states which leg carries the score. | | 3 | Scheduled / recurring runs 2 | citation repair | Cited endpoint was the `routines-fire` API behind `anthropic-beta: experimental-cc-routine-2026-04-01`. Re-pinned to the stable `/docs/en/routines`; `/scheduled-tasks` and `/desktop-scheduled-tasks` also verified. | | 4 | four uncited 2s | citations added | CLI/TUI → `/cli-reference`; custom tools → `/commands`; multi-agent orchestration → `/agent-teams`; enterprise → `/analytics`. All four 200. Orchestration mattered most: Codex's 1 had a citation while Claude Code's 2 had none. | ### Accepted as a process fix, wider than the row it was raised about The review's third hygiene item was that a URL sweep which follows redirects reports 200s and therefore **conceals a domain migration** — 19 of 20 Claude Code citations were one migration stale. Correct, and not only for Claude Code. `check_urls.py` now probes every source, records the **final** URL and flags any hop. First run: **22 of 158 citations were 301ing** — 18 Claude Code (docs.claude.com/en/docs/claude-code/* → code.claude.com/docs/en/*), the Codex row root (developers.openai.com → learn.chatgpt.com), and 6 Gemini CLI GitHub links that were half tree/ half blob. All migrated; second run **162/162 clean, 0 redirects**. The auditor is now the gate, so a stale citation fails loudly instead of passing as healthy. The Method tab carries the rule. ### Rubric decision (raised as an open question, not a correction) The review argued criterion drift: if MCP reachability earns the creative-apps row a 1 for every MCP-capable harness, then media generation and home automation are equally MCP-reachable, so their 0s should be 1s too. **Resolved as: cite the path, or it is not a point.** The creative row's 1s are backed by a *named, re-usable* community server cited in the cell. Scoring mediagen/IoT reachability would mean asserting "a server exists somewhere" with nothing to cite — an uncited assertion, not evidence, and the same failure mode that produced the mcp-serve error above. So no cell moved; the distinction is now written into the Method tab instead of being implicit, and any reachable-in-principle path belongs in the cell note. If a peer cites a specific server for a specific harness, those cells move to 1 on that evidence — one at a time, not by rule. ### Outcome - Claude Code **44 → 45/54** (How you extend it 19 → 20/20), tying Antigravity at 45 and moving above Cursor's 43. Top of the board unchanged. - The review's own caveat was honoured: it is AI-generated analysis of public documentation, so it was checked rather than applied. Every accepted item above was independently reproduced. - Still open from that review: the `not yet owner-verified with a runtime` flag. This pass is docs-verified from inside a running session — the reviewer itself noted it remains documentation-based evidence. The flag stays; the runtime corroboration it offered (Remote Control, dynamic workflows, subagents, skills, plugins, hooks, MCP client, scheduled tasks, persistent memory as live surfaces) is noted here as supporting context, not as a score change. ## Round 4 — design / accessibility review from inside Claude Code (Opus), 2026-09-16 → no score changes Second review of the same column-style pass, this time on the *instrument* rather than the evidence: layout, contrast, keyboard, addressability. Verified claim-by-claim against the live build before any edit — which the reviewer asked for, correctly, because two sessions and 13 columns had moved under it. ### Verification result: all claims true, three of them worse than reported | claim | verdict | |---|---| | Timeline colour keys missing OpenClaw 1.x / 2.0 → both fall through to `#888` | **true, and worse**: HCOL had 10 keys for what were then 12 harnesses, and Grok Bot (13th) was also missing. The two columns whose two-point delta is the v4.2 headline were the two that rendered identically. | | v4.3 zero-cell contrast fix never reached the evidence sheet | **true** — the matrix got `#8b9bb1`, `openHarness` still rendered `·` in `#4a5a6e` (2.24:1), and the `·`-reads-as-missing-data problem persisted on the higher-traffic surface. | | Timeline `aria-label` says "Capability counts over time" | **true** — the one thing both series explicitly are not. Now per-mode. | | Fixed 1000×400 viewBox scales axis text to ~3.8px at 375px | **true** — `preserveAspectRatio` + fixed units. | | `sortMode` declared and never read | **true** — deleted rather than wired (no sort control exists). | | Zero tabindex / keyboard / dialog semantics; no `