Updated today
by UiPath
coder_eval
Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.
151 GitHub stars (151)
+10 in 20 days
| Period | New stars |
|---|---|
| Sep 27 | 0 |
| Sep 28 | 0 |
| Sep 29 | 0 |
| Sep 30 | +3 |
| Oct 1 | +1 |
| Oct 2 | 0 |
| Oct 3 | +3 |
| Oct 4 | 0 |
| Oct 5 | 0 |
| Oct 6 | 0 |
| Oct 7 | +1 |
| Oct 8 | 0 |
| Oct 9 | +2 |
| Oct 10 | 0 |
| Oct 11 | 0 |
| Oct 12 | 0 |
| Oct 13 | 0 |
| Oct 14 | 0 |
| Oct 15 | 0 |
| Oct 16 | 0 |
- 6 skills (6 skills found in its SKILL.md files)
- 5 forks (5 forks)
- Today last update (Last updated today)
Install
Skills CLI
npx skills add UiPath/coder_eval
Asks which skills and agents to install. Just one: npx skills add UiPath/coder_eval --skill analyze — add -g for every project.
Claude Code
git clone https://github.com/UiPath/coder_evalcp -r "coder_eval/plugins/coder-eval/skills/analyze" ~/.claude/skills/
Copies the analyze skill into ~/.claude/skills/ for every project (use .claude/skills/ for one project). Repeat for the other skills listed below.
Plugin
/plugin marketplace add UiPath/coder_eval
Run inside Claude Code if the repository ships a plugin marketplace, then pick plugins from /plugin.
Skills can run code on your machine — read the SKILL.md first.
What it does
In the author's own words, from each skill's SKILL.md.
-
analyze
Analyze a finished coder-eval run and write analysis.md — cluster failures into systemic patterns and recommend fixes. Use when the user wants to know why a run failed, what regressed or got worse since a previous run, what to fix, or what a run says about their tasks.
plugins/coder-eval/skills/analyze/SKILL.md on GitHub (opens in a new tab) -
check-skill
Generate and run a coder-eval activation suite for a Claude Code skill. Use when the user asks whether a skill triggers, wants to test skill activation, or worries a skill has silently stopped firing.
plugins/coder-eval/skills/check-skill/SKILL.md on GitHub (opens in a new tab) -
ci
Generate a GitHub Actions workflow that runs a coder-eval suite as a CI gate or on a schedule, using the published composite action — with the agent runtime, credentials, JUnit output and the run reports wired correctly.
plugins/coder-eval/skills/ci/SKILL.md on GitHub (opens in a new tab) -
init
Set up coder-eval in this repository — scan for what is worth evaluating (Claude Code skills, an MCP server, a CLI), then scaffold a task directory with one real task and the command to run it.
plugins/coder-eval/skills/init/SKILL.md on GitHub (opens in a new tab) -
lint-tasks
Review existing coder-eval task YAML — find criteria that cannot fail, prompts that leak the answer, and near-duplicate tasks, each with a fix. Read-only. Use when the user wants existing tasks reviewed, linted, audited, or checked for gaps.
plugins/coder-eval/skills/lint-tasks/SKILL.md on GitHub (opens in a new tab) -
task
Turn a natural-language description into coder-eval task YAML — minimal prompts, weighted criteria that check output content, validated with `coder-eval plan`. Use when the user wants to write, add, or generate an evaluation task.
plugins/coder-eval/skills/task/SKILL.md on GitHub (opens in a new tab)
No skill matches that filter.
Review before you install
A skill is a set of instructions and scripts your agent will follow, and it can run code on your machine. Read the SKILL.md and any scripts first — this listing is not a security review.
Similar skills
-
cc-thinking-skillsby tjboudreaux 28 eval-informed mental models and critical-thinking skills for Claude Code, GitHub Copilot, Codex, Cursor, and other Agent Skills-compatib…
-
ECCby affaan-m The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Cod…
-
ponytailby DietrichGebert Makes your AI agent think like the laziest senior dev in the room. The best code is the code you never wrote.
-
cavemanby JuliusBrussee 🪨 why use many token when few token do trick. Viral skill + proxy for coding agents that cuts 65% of tokens by talking like a caveman.
-
awesome-claude-skillsby ComposioHQ A curated list of awesome Claude Skills, resources, and tools for customizing Claude AI workflows
-
humanizerby blader Agent skill that removes signs of AI-generated writing from text