Agentation

Name: Agentation
Author: neondatabase

109

This skill provides a visual UI feedback bridge for AI agents, allowing users to easily capture, annotate, and target specific UI elements with CSS selectors, bounding boxes, or React component trees. It supports interactive and automated workflows for UI/UX review, design critique, and agent-based auto-fixing, primarily for desktop environments. Suitable for developers and designers working with AI agents in React or various platform integrations, it offers local and server-based modes for seamless annotation and feedback management.

npx skills add https://github.com/supercent-io/skills-template --skill agentation

agentation — Visual UI Feedback Bridge for AI Agents

The missing link between human eyes and agent code.

Instead of describing "the blue button in the sidebar," you hand the agent .sidebar > button.primary. It can grep for that directly.

When to use this skill

Human needs to point at a UI element and give feedback — without writing selectors
Running iterative UI/UX review cycles between human and coding agent
Building a watch-loop where agent auto-fixes every annotation a human leaves
Capturing CSS selectors, bounding boxes, and React component trees for precise code targeting
Autonomous design critique via agent-browser + self-driving pattern
Integrating visual feedback into agent hooks so annotations auto-appear in agent context

1. Architecture

agentation (monorepo)
├── agentation          → npm: agentation (React toolbar component)
│   └── src/index.ts   → exports Agentation component + types + utilities
└── agentation-mcp      → npm: agentation-mcp (MCP server + CLI)
    ├── src/cli.ts      → agentation-mcp CLI (init, server, doctor)
    └── src/server/     → HTTP REST API (port 4747) + SSE events + MCP stdio tools

Two modes of operation: Mode How it works Copy-Paste Human annotates → clicks Copy → pastes markdown into agent chat Agent Sync endpoint prop connects toolbar to MCP server → agent uses agentation_watch_annotations loop

2. Installation

npm install agentation -D
# or: pnpm add agentation -D  /  yarn add agentation -D  /  bun add agentation -D

Requirements: React 18+, desktop browser, zero runtime deps beyond React (desktop only — no mobile)

🔗 Local-first by design: Annotations are stored locally and auto-sync when connected to the MCP server.

Offline operation — Annotations can be created without a server

Session continuity — Same session persists after page refresh, no duplicates

Agent-first — resolve/dismiss is handled by the agent

2.2 MCP Server — Universal Setup (Recommended)

Fastest method — Auto-detects all installed agents and configures them (Claude Code, Cursor, Codex, Windsurf, and 9+ more agents):

npx add-mcp "npx -y agentation-mcp server"

Or install manually:

npm install agentation-mcp -D
npx agentation-mcp server          # HTTP :4747 + MCP stdio
npx agentation-mcp server --port 8080   # custom port
npx agentation-mcp doctor          # verify setup

2.3 Claude Code — Official Skill (Minimal Setup)

Recommended for Claude Code users — automatically handles framework detection, package installation, and layout integration:

npx skills add benjitaylor/agentation
# then in Claude Code:
/agentation

3. React Component Setup

Basic (Copy-Paste mode — no server needed)

import { Agentation } from 'agentation';
function App() {
  return (
    <>
      <YourApp />
      {process.env.NODE_ENV === 'development' && <Agentation />}
    </>
  );
}

Next.js App Router

// app/layout.tsx
import { Agentation } from 'agentation';
export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html>
      <body>
        {children}
        {process.env.NODE_ENV === 'development' && (
          <Agentation endpoint="http://localhost:4747" />
        )}
      </body>
    </html>
  );
}

Next.js Pages Router

// pages/_app.tsx
import { Agentation } from 'agentation';
export default function App({ Component, pageProps }) {
  return (
    <>
      <Component {...pageProps} />
      {process.env.NODE_ENV === 'development' && (
        <Agentation endpoint="http://localhost:4747" />
      )}
    </>
  );
}

Full Props Reference

Prop Type Default Description endpoint string — MCP server URL for Agent Sync mode sessionId string — Pre-existing session ID to join onAnnotationAdd (a: Annotation) => void — Callback when annotation created onAnnotationDelete (a: Annotation) => void — Callback when annotation deleted onAnnotationUpdate (a: Annotation) => void — Callback when annotation edited onAnnotationsClear (a: Annotation[]) => void — Callback when all cleared onCopy (markdown: string) => void — Callback with markdown on copy onSubmit (output: string, annotations: Annotation[]) => void — On "Send Annotations" click copyToClipboard boolean true Set false to suppress clipboard write onSessionCreated (sessionId: string) => void — Called on new session creation webhookUrl string — Webhook URL to receive annotation events

4. MCP Server Setup — All Platforms

Fastest method — Universal (auto-detects 9+ agents):

npx add-mcp "npx -y agentation-mcp server"

add-mcp auto-detects Claude Code, Cursor, Codex, Windsurf, and more, writing directly to the correct config. Start server / verify:

npx agentation-mcp server          # HTTP :4747 + MCP stdio
npx agentation-mcp server --port 8080   # custom port
npx agentation-mcp doctor          # verify setup

Claude Code (`.claude/`)

Minimal setup — Official Claude Code Skill (Recommended):

npx skills add benjitaylor/agentation
# In Claude Code:
/agentation

Universal MCP auto-setup (Claude Code + 9+ agents):

npx add-mcp "npx -y agentation-mcp server"

Interactive wizard (Claude Code only):

npx agentation-mcp init

Option A — CLI (recommended):

claude mcp add agentation -- npx -y agentation-mcp server

Option B — config file (~/.claude/claude_desktop_config.json for global, or .claude/mcp.json for project-level):

{
  "mcpServers": {
    "agentation": {
      "command": "npx",
      "args": ["-y", "agentation-mcp", "server"]
    }
  }
}

Interactive wizard (Claude Code only):

npx agentation-mcp init

UserPromptSubmit hook — auto-inject pending annotations on every message. Add to .claude/settings.json (project) or ~/.claude/settings.json (global):

{
  "hooks": {
    "UserPromptSubmit": [
      {
        "type": "command",
        "command": "curl -sf --connect-timeout 1 http://localhost:4747/pending 2>/dev/null | python3 -c \"import sys,json;d=json.load(sys.stdin);c=d['count'];exit(0)if c==0 else[print(f'\\n=== AGENTATION: {c} UI annotations ===\\n'),*[print(f\\\"[{i+1}] {a['element']} ({a['elementPath']})\\n    {a['comment']}\\n\\\")for i,a in enumerate(d['annotations'])],print('=== END ===\\n')]\" 2>/dev/null;exit 0"
      }
    ]
  }
}

Codex CLI (`~/.codex/`)

Add to ~/.codex/config.toml:

# Agentation MCP Server
[[mcp_servers]]
name = "agentation"
command = "npx"
args = ["-y", "agentation-mcp", "server"]
# Optional: teach Codex about watch-loop
developer_instructions = """
When user says "watch mode" or "agentation watch", call agentation_watch_annotations in a loop.
For each annotation: acknowledge it, fix the code using the elementPath CSS selector, resolve with summary.
"""

Restart Codex CLI after editing config.toml.

Gemini CLI (`~/.gemini/`)

Option A — CLI:

gemini mcp add agentation npx -y agentation-mcp server
# or with explicit scope
gemini mcp add -s user agentation npx -y agentation-mcp server

Option B — config file (~/.gemini/settings.json for global, .gemini/settings.json for project):

{
  "mcpServers": {
    "agentation": {
      "command": "npx",
      "args": ["-y", "agentation-mcp", "server"]
    }
  }
}

AfterAgent hook — trigger annotation check after each agent turn:

{
  "mcpServers": {
    "agentation": {
      "command": "npx",
      "args": ["-y", "agentation-mcp", "server"]
    }
  },
  "hooks": {
    "AfterAgent": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "curl -sf --connect-timeout 1 http://localhost:4747/pending 2>/dev/null | python3 -c \"import sys,json;d=json.load(sys.stdin);c=d.get('count',0);[print(f'[agentation] {c} pending annotations'),exit(1)]if c>0 else exit(0)\" 2>/dev/null;exit 0",
            "description": "Check for pending agentation annotations"
          }
        ]
      }
    ]
  }
}

OpenCode (`~/.config/opencode/`)

Add to ~/.config/opencode/opencode.json:

{
  "mcp": {
    "agentation": {
      "type": "local",
      "command": ["npx", "-y", "agentation-mcp", "server"]
    }
  }
}

With environment variables:

{
  "mcp": {
    "agentation": {
      "type": "local",
      "command": ["npx", "-y", "agentation-mcp", "server"],
      "environment": {
        "AGENTATION_STORE": "sqlite",
        "AGENTATION_EVENT_RETENTION_DAYS": "7"
      }
    }
  }
}

Restart OpenCode after editing. MCP tools (agentation_*) will be available immediately.

Universal (npx add-mcp)

Works for any MCP-compatible agent:

npx add-mcp "npx -y agentation-mcp server"

Quick-Setup Script

Save and run bash setup-agentation-mcp.sh [--all | --claude | --codex | --gemini | --opencode]:

#!/usr/bin/env bash
# setup-agentation-mcp.sh — Register agentation MCP for all agent platforms
set -euo pipefail
SETUP_CLAUDE=false; SETUP_CODEX=false; SETUP_GEMINI=false; SETUP_OPENCODE=false
while [[ $# -gt 0 ]]; do
  case "$1" in
    --claude) SETUP_CLAUDE=true ;;
    --codex) SETUP_CODEX=true ;;
    --gemini) SETUP_GEMINI=true ;;
    --opencode) SETUP_OPENCODE=true ;;
    --all) SETUP_CLAUDE=true; SETUP_CODEX=true; SETUP_GEMINI=true; SETUP_OPENCODE=true ;;
  esac
  shift
done
[[ "$SETUP_CLAUDE$SETUP_CODEX$SETUP_GEMINI$SETUP_OPENCODE" == "falsefalsefalsefalse" ]] && \
  SETUP_CLAUDE=true && SETUP_CODEX=true && SETUP_GEMINI=true && SETUP_OPENCODE=true
MCP_JSON='"agentation": {"command": "npx", "args": ["-y", "agentation-mcp", "server"]}'
# Claude Code
if [[ "$SETUP_CLAUDE" == "true" ]]; then
  mkdir -p ~/.claude
  CFG=~/.claude/claude_desktop_config.json
  if [[ -f "$CFG" ]] && command -v jq &>/dev/null; then
    jq ".mcpServers += {$MCP_JSON}" "$CFG" > "$CFG.tmp" && mv "$CFG.tmp" "$CFG"
  else
    echo "{\"mcpServers\": {$MCP_JSON}}" > "$CFG"
  fi
  echo "✅ Claude Code: $CFG"
fi
# Codex CLI
if [[ "$SETUP_CODEX" == "true" ]]; then
  mkdir -p ~/.codex
  CFG=~/.codex/config.toml
  if ! grep -q "agentation" "$CFG" 2>/dev/null; then
    printf '\n[[mcp_servers]]\nname = "agentation"\ncommand = "npx"\nargs = ["-y", "agentation-mcp", "server"]\n' >> "$CFG"
  fi
  echo "✅ Codex CLI: $CFG"
fi
# Gemini CLI
if [[ "$SETUP_GEMINI" == "true" ]]; then
  mkdir -p ~/.gemini
  CFG=~/.gemini/settings.json
  if [[ -f "$CFG" ]] && command -v jq &>/dev/null; then
    jq ".mcpServers += {$MCP_JSON}" "$CFG" > "$CFG.tmp" && mv "$CFG.tmp" "$CFG"
  else
    echo "{\"mcpServers\": {$MCP_JSON}}" > "$CFG"
  fi
  echo "✅ Gemini CLI: $CFG"
fi
# OpenCode
if [[ "$SETUP_OPENCODE" == "true" ]]; then
  mkdir -p ~/.config/opencode
  CFG=~/.config/opencode/opencode.json
  ENTRY='"agentation": {"type": "local", "command": ["npx", "-y", "agentation-mcp", "server"]}'
  if [[ -f "$CFG" ]] && command -v jq &>/dev/null; then
    jq ".mcp += {$ENTRY}" "$CFG" > "$CFG.tmp" && mv "$CFG.tmp" "$CFG"
  else
    echo "{\"mcp\": {$ENTRY}}" > "$CFG"
  fi
  echo "✅ OpenCode: $CFG"
fi
echo ""
echo "Done. Restart your agent(s) and run: npx agentation-mcp server"

5. MCP Tools (Agent API)

Tool Parameters Description agentation_list_sessions none List all active annotation sessions agentation_get_session sessionId: string Get session with all annotations agentation_get_pending sessionId: string Get pending annotations for a session agentation_get_all_pending none Get pending annotations across ALL sessions agentation_acknowledge annotationId: string Mark annotation as acknowledged (agent is working on it) agentation_resolve annotationId: string, summary?: string Mark as resolved with optional summary agentation_dismiss annotationId: string, reason: string Dismiss with required reason agentation_reply annotationId: string, message: string Add reply to annotation thread agentation_watch_annotations sessionId?: string, batchWindowSeconds?: number (default 10, max 60), timeoutSeconds?: number (default 120, max 300) Block until new annotations arrive — core watch-loop tool

6. Workflow Patterns

Pattern 1: Copy-Paste (Simplest, No Server)

1. Human opens app in browser
2. Clicks agentation toolbar → activates
3. Clicks element → adds comment → clicks Copy
4. Pastes markdown output into agent chat
5. Agent receives CSS selectors, elementPath, boundingBox
6. Agent greps/edits code using selector

Pattern 2: MCP Watch Loop (Recommended for iterative review)

Agent: agentation_watch_annotations (blocks up to 120s)
  → Human adds annotation in browser
  → Agent receives batch immediately
  → Agent: agentation_acknowledge(annotationId)
  → Agent makes code changes using elementPath as grep target
  → Agent: agentation_resolve(annotationId, "Changed button color to #3b82f6")
  → Agent: agentation_watch_annotations (loops again)

CLAUDE.md / GEMINI.md / Codex developer_instructions — add for automated watch:

When I say "watch mode" or "agentation watch", call agentation_watch_annotations in a loop.
For each annotation received:
  1. Call agentation_acknowledge(annotationId)
  2. Use elementPath to locate the code: Grep(elementPath) or search codebase for CSS class
  3. Make the minimal change described in the comment
  4. Call agentation_resolve(annotationId, "<brief summary of what was changed>")
Continue watching until I say stop, or until timeout.

Pattern 3: Platform-Specific Hook (Passive Injection)

The hook from Section 4 auto-appends pending annotations to every agent message — no "watch mode" needed. Works across all platforms.

Pattern 4: Autonomous Self-Driving Critique

Two-agent setup for fully autonomous UI review cycles: Session 1 (Critic — uses agent-browser):

# Start headed browser pointing at your dev server
agent-browser open http://localhost:3000
agent-browser snapshot -i
# Agent navigates, clicks elements via agentation toolbar, adds critique
# Annotations flow to agentation MCP server automatically

Session 2 (Fixer — watches MCP):

agentation_watch_annotations → receives critique → acknowledge → edit → resolve → loop

Pattern 5: Webhook Integration

<Agentation webhookUrl="https://your-server.com/webhook" />
# or env var:
# AGENTATION_WEBHOOK_URL=https://your-server.com/webhook

7. Annotation Type (Full Schema)

type Annotation = {
  // Core
  id: string;
  x: number;            // % of viewport width (0-100)
  y: number;            // px from document top
  comment: string;      // User's feedback text
  element: string;      // Tag name: "button", "div", etc.
  elementPath: string;  // CSS selector: "body > main > button.cta"  ← grep target
  timestamp: number;
  // Context
  selectedText?: string;
  boundingBox?: { x: number; y: number; width: number; height: number };
  nearbyText?: string;
  cssClasses?: string;
  nearbyElements?: string;
  computedStyles?: string;
  fullPath?: string;
  accessibility?: string;
  reactComponents?: string;  // "App > Dashboard > Button"  ← component grep target
  isMultiSelect?: boolean;
  isFixed?: boolean;
  // Lifecycle (server-synced)
  sessionId?: string;
  url?: string;
  intent?: "fix" | "change" | "question" | "approve";
  severity?: "blocking" | "important" | "suggestion";
  status?: "pending" | "acknowledged" | "resolved" | "dismissed";
  thread?: ThreadMessage[];
  createdAt?: string;
  updatedAt?: string;
  resolvedAt?: string;
  resolvedBy?: "human" | "agent";
};

Annotation lifecycle:

pending → acknowledged → resolved
                      ↘ dismissed (requires reason)

8. HTTP REST API (port 4747)

# Sessions
POST   /sessions                     # Create session
GET    /sessions                     # List all sessions
GET    /sessions/:id                 # Get session + annotations
# Annotations
POST   /sessions/:id/annotations     # Add annotation
GET    /annotations/:id              # Get annotation
PATCH  /annotations/:id              # Update annotation
DELETE /annotations/:id              # Delete annotation
GET    /sessions/:id/pending         # Pending for session
GET    /pending                      # ALL pending across sessions
# Events (SSE streaming)
GET    /sessions/:id/events          # Session stream
GET    /events                       # Global stream (?domain=filter)
# Health
GET    /health
GET    /status

9. Environment Variables

Variable Description Default AGENTATION_STORE memory or sqlite sqlite AGENTATION_WEBHOOK_URL Single webhook URL — AGENTATION_WEBHOOKS Comma-separated webhook URLs — AGENTATION_EVENT_RETENTION_DAYS Days to keep events 7 SQLite storage: ~/.agentation/store.db

10. Programmatic Utilities

import {
  identifyElement, identifyAnimationElement,
  getElementPath, getNearbyText, getElementClasses,
  isInShadowDOM, getShadowHost, closestCrossingShadow,
  loadAnnotations, saveAnnotations, getStorageKey,
  type Annotation, type Session, type ThreadMessage,
} from 'agentation';

11. Platform Support Matrix

Platform Config File MCP Key Hook Claude Code ~/.claude/claude_desktop_config.json mcpServers hooks.UserPromptSubmit in settings.json Codex CLI ~/.codex/config.toml [[mcp_servers]] (TOML) developer_instructions + notify Gemini CLI ~/.gemini/settings.json mcpServers hooks.AfterAgent in settings.json OpenCode ~/.config/opencode/opencode.json mcp (type: "local") Skills system (no hook needed) Cursor / Windsurf .cursor/mcp.json / .windsurf/mcp.json mcpServers —

Best practices

Always gate <Agentation> with NODE_ENV === 'development' — never ship to production
Use MCP watch-loop over copy-paste for iterative cycles — eliminates context switching
Call agentation_acknowledge immediately when starting a fix — signals human
Include a summary in agentation_resolve — gives human traceability
Process severity: "blocking" annotations first in the watch loop
Use elementPath as the primary grep/search target in code — it's a valid CSS selector
Use reactComponents field when the codebase is React — matches component names directly
Add the appropriate hook for your platform (Section 4) for zero-friction passive injection
For autonomous self-driving, use agent-browser in headed mode with agentation mounted

12. jeo Integration (annotate keyword)

agentation integrates as the VERIFY_UI phase of the jeo skill. This follows the same pattern as plannotator operating in planui / ExitPlanMode. annotate is the primary keyword. agentui is kept as a backward-compatible alias.

How it works

plannotator (planui):         agentation (annotate):
Write plan.md                   Mount <Agentation> in app UI
    ↓ blocking                       ↓ blocking
Run plannotator             agentation_watch_annotations
    ↓                              ↓
Approve/Feedback in UI        Create annotation in UI
    ↓                              ↓
Confirm approved:true          annotation ack→fix→resolve
    ↓                              ↓
Enter EXECUTE                 Next step or loop

Trigger Keywords

Keyword Platform Action annotate Claude Code agentation_watch_annotations MCP blocking call annotate Codex ANNOTATE_READY signal → jeo-notify.py HTTP polling annotate Gemini GEMINI.md instruction: HTTP REST polling pattern /jeo-annotate OpenCode opencode.json mcp.agentation + instructions agentui (deprecated) All platforms Same behavior as above — backward-compatible alias UI review All platforms Same as annotate

Using with jeo

# 1. agentation auto-registered when installing jeo
bash .agent-skills/jeo/scripts/install.sh --with-agentation
# Or full install:
bash .agent-skills/jeo/scripts/install.sh --all
# 2. Mount agentation component in app
# app/layout.tsx or pages/_app.tsx:
#   <Agentation endpoint="http://localhost:4747" />
# 3. Start MCP server
npx agentation-mcp server
# 4. Enter annotate keyword in agent → watch loop starts (agentui also works as backward-compatible alias)
# Claude Code: direct MCP tool call
# Codex: output ANNOTATE_READY (or AGENTUI_READY) → notify hook auto-polls
# Gemini: GEMINI.md HTTP polling pattern
# OpenCode: /jeo-annotate slash command (or /jeo-agentui — deprecated)

Separation from plannotator (Phase Guard)

plannotator and agentation use the same blocking loop pattern but only operate in different phases: Tool Allowed phase Hook Guard plannotator plan only jeo-state.json → phase === "plan" agentation verify / verify_ui only jeo-state.json → phase === "verify_ui" Each platform's hook script checks the phase field in jeo-state.json to prevent execution in the wrong phase. Without this guard, both tools could run simultaneously in Gemini's AfterAgent hook.

Pre-flight Check

3-step check before entering VERIFY_UI:

Server status: GET /health — whether agentation-mcp server is running
Session exists: GET /sessions — whether <Agentation> component is mounted
Pending annotations: GET /pending — number of annotations to process After passing, set phase in jeo-state.json to "verify_ui" and agentation.active to true.

Loop Verification Test

# Run agentation watch loop integration test
bash .agent-skills/agentation/scripts/verify-loop.sh
# Quick test (skip error cases)
bash .agent-skills/agentation/scripts/verify-loop.sh --quick

4-step verification: Server Health → Annotation CRUD → ACK-RESOLVE Cycle → Error Cases

Evaluation Flow (jeo VERIFY_UI phase)

jeo "<task>"
    │
[1] PLAN (plannotator loop)    ← approve plan.md
[2] EXECUTE (team/bmad)
[3] VERIFY
    ├─ agent-browser snapshot
    ├─ Pre-flight check (server + session + pending)
    └─ annotate → VERIFY_UI (agentation loop)   ← this phase (agentui also backward-compatible)
        ├─ ACK → FIND → FIX → RESOLVE
        ├─ RE-SNAPSHOT (agent-browser)  ← re-check after fix
        └─ update agentation fields in jeo-state.json
[4] CLEANUP

For detailed jeo integration: see jeo SKILL.md Section 3.3.1 detailed workflow

References

Metadata

Version: 1.1.0
Source: benjitaylor/agentation (PolyForm Shield 1.0.0)
Packages: agentation@2.2.1, agentation-mcp@1.2.0
Last updated: 2026-03-05
Scope: UI annotation bridge for human-agent feedback loops — Claude Code, Codex, Gemini CLI, OpenCode

GitHub Owner

Owner: neondatabase

GitHub Links

Website: https://neon.tech
Twitter: https://twitter.com/neondatabase
YouTube: https://www.youtube.com/@neondatabase
Verified domains: neondatabase, neon.tech

Files