slop-eval

Quickstart•CLI reference•Library API•MCP Server•Comparison•FAQ

npx slop-eval-cli score --screenshot ./preview.png --json

slop-eval-cliis live on bothnpmandPyPI(packageslop_eval). The Python port is a genuine, independent implementation, built and tested (60/60 tests, verified in this pass) against the same rubric and Anthropic judge prompt as the TypeScript original. Seepython/README.mdfor Python-specific usage.

- Impeccable(pbakaus/impeccable, 54,000+ stars, Apache 2.0) ships a CLI that flags 59 specific visual tells of AI-generated UI (gradient palettes, glassmorphism, side-stripe borders, WCAG contrast violations), all enabled by default with no model call; a separateimpeccable critiquecommand adds further, opt-in LLM-based judgments on top. Core detection stays fast because it doesn't need a model for any of its default checks. It has grown well beyond a slop detector into a full design-language skill for Claude Code, Cursor, and Codex, with 23 commands total.
- aislop(MIT, 500+ stars) does the deterministic, rule-based equivalent for AI-generatedcode(not UI): 50+ regex/AST rules across 8 languages, no LLM in the runtime path, positioned exactly as a CI quality gate.

- Three rubric categories, each with mandatory cited evidence.src/rubric/v1.jsonscores layout novelty, visual-identity distinctiveness, and component-pattern novelty, 0-10 each. A finding with no specific citation is treated as a bug, not a valid score (seesrc/sources/RuleSource.ts).
- LLM judge via forced tool-call, returning structured JSON.LLMJudgeSourcecalls the Anthropic API withtool_choicelocked to asubmit_slop_scoresschema: the response comes back as reliably structured JSON instead of a chat reply that has to be regexed apart.
- --jsonmode for CI and agents.Every run can emit a parseable{ target, rubric, compositeScore, findings[], summary, disclaimer }object on stdout, on both success and error paths, so a script or agent never has to branch on shape to find an error string.
- Real exit-code contract.0success (no threshold, or score at/above--fail-below),1success but below threshold,2usage error or unrecoverable failure. Verified directly against the built CLI and the real npm/PyPI packages this session; seeCLI reference.
- Content-hash caching.src/cache/judge-cache.tshashes the input bytes and skips the API call entirely on a repeat run against unchanged input. That's a correctness guarantee as much as a cost saver: an unchanged PR can't flap a CI gate from LLM run-to-run variance.
- ComposableRuleSourceplugin interface.src/sources/RuleSource.tsis the boundary every scoring source implements. Today that's one real source (LLMJudgeSource) and one documented stub (ScreenshotDiffSource, honestly reported asnot_scoreduntil a real labeled corpus exists), so a future rule catalog or a second LLM provider slots in without touching the composite scorer.
- Screenshot input (real visual read) or--urlfallback.--screenshotsends the actual rendered image to the judge.--urlis a documented v0.1 limitation: no bundled headless browser, so it fetches raw HTML/text and the judge reasons over markup and copy instead of layout.
- GitHub Action that leads with the specific flag, then the score.action/action.ymlposts a PR comment headed by the single most specific flagged finding, followed by the composite score, giving a reviewer the reasoning behind the number.
- Versioned, public rubric.Every score names the rubric version (v1today) that produced it. Rubric changes ship as a new file, never a silent edit to an existing one.
- A real, agent-native library API alongside the CLI.Both distributions export a programmatic entry point (score_compositeand friends in Python,runScore/scoreCompositein TypeScript) so an agent framework can call slop-eval in-process instead of shelling out. SeeLibrary API.

Requires Node.js 18+ (npm) or Python 3.9+ (PyPI), and anANTHROPIC_API_KEY(BYO key; get one atconsole.anthropic.com).

npx slop-eval-cli score --screenshot ./preview.png --json

$ npx --yes slop-eval-cli@latest score --screenshot ./preview.png --json { "error": "ANTHROPIC_API_KEY environment variable is not set.\nslop-eval calls the Anthropic API to run the LLM judge, and is BYO-key (bring your own key) -- there is no default or shared key baked into this tool. Set your key and try again:\n\n export ANTHROPIC_API_KEY=\"sk-ant-...\"\n\nGet a key at https://console.anthropic.com/" } # exit code 2

git clone https://github.com/RudrenduPaul/slop-eval.git cd slop-eval npm install npm run build export ANTHROPIC_API_KEY="sk-ant-..." ./dist/cli.js score --screenshot ./test/fixtures/sample.png

./dist/cli.js score --screenshot ./test/fixtures/sample.png --json

{ "target": "./test/fixtures/sample.png", "rubric": "v1", "compositeScore": 62, "findings": [ { "ruleId": "llm-judge.layout-novelty", "category": "Layout novelty", "score": 4, "evidence": "Matches a common hero + 3-card grid + footer CTA pattern.", "status": "flag" } ], "summary": { "pass": 1, "flagged": 1, "notScored": 1 }, "disclaimer": "This score is a heuristic quality signal from an LLM judge, not a certification..." }

Captured directly from./dist/cli.js score --helpon the built CLI this session, word for word:

Usage: slop-eval score [options] Score a URL or screenshot for AI-UI genericness against a versioned rubric. Note on --url mode (v0.1 limitation): this tool does not bundle a headless browser. If --url is given, the raw HTML/text response is fetched and given to the judge as a fallback input, instead of a rendered screenshot -- the judge can reason about markup and copy, but not the actual visual layout. For the stronger, layout-aware signal, render the page yourself and pass --screenshot. Options: --url <url> URL to score (fetched as raw HTML/text -- see limitation note above) --screenshot <path> path to a screenshot image to score (preferred over --url) --rubric <name> rubric version to use, reads src/rubric/<name>.json (default: "v1") --json output structured JSON instead of a human-readable report (default: false) --fail-below <n> exit code 1 if the composite score is below this threshold (0-100); no threshold by default -h, --help display help for command

Exit codes:0success (no threshold, or score at/above--fail-below),1success but below threshold,2usage error or unrecoverable failure (missing API key, unreadable file, malformed rubric, mutually exclusive--url/--screenshot).

--urland--screenshotare mutually exclusive; passing both or neither is a usage error (exit 2) in either output mode. Both verified directly against the built CLI this session.

[!NOTE]--urlis a v0.1 limitation, by design: no bundled headless browser. It fetches raw HTML/text and hands it to the judge as a text fallback, reasoning over markup and copy rather than the rendered layout.--screenshotis the stronger signal; render the page yourself (Playwright, Puppeteer, or your CI's existing preview-screenshot step) and pass the image.

The Python CLI (slop-evalconsole script, installed viapip install slop-eval-cli) exposes the identical flag set and exit-code contract, confirmed against its own--helpoutput this session.

Both distributions export a real, documented programmatic entry point in addition to the CLI. This is the interface an agent framework or CI script calls in-process instead of shelling out.

from slop_eval import score_composite, ScoreInput, LLMJudgeSource, ScreenshotDiffSource sources = [LLMJudgeSource("v1"), ScreenshotDiffSource()] result = score_composite(sources, ScoreInput(screenshot_path="./preview.png")) print(result.composite_score, result.findings)

score_composite(sources: List[RuleSource], score_input: ScoreInput) -> CompositeResultruns everyRuleSourcein list order, flattens their findings, and returns aCompositeResultwithcomposite_score: float(0-100) andfindings: List[RuleFinding]. Also exported:RuleFinding,RuleFindingStatus,RuleSource,Rubric,RubricCategory,load_rubric,build_json_report,render_human_report,print_report,print_error,MissingApiKeyError,RubricLoadError.

TypeScript(src/cli.ts, exported from the package'smain/typesentry):runScore(options: ScoreOptions, buildSources?) => Promise<number>andbuildProgram(): Commandare the two exported entry points, along with theScoreOptionsinterface.scoreComposite(fromsrc/scorer/composite.ts) is the same composite-scoring function the CLI calls internally. These exist primarily so the test suite can drive the CLI in-process; the Python package's__init__.pyis the more deliberately documented "agent-native" library surface of the two.

slop-eval ships a Model Context Protocol server, so an MCP-compatible agent (Claude Desktop, Claude Code, Cursor, an orchestrator) can call slop-eval directly as a tool instead of shelling out to the CLI and parsing stdout itself.

pip install "slop-eval-cli[mcp]"

{ "mcpServers": { "slop-eval": { "command": "slop-eval-mcp", "env": { "ANTHROPIC_API_KEY": "sk-ant-..." } } } }

The server exposes one tool,run(args: list[str]) -> dict, a generic wrapper around the CLI: pass it the same argv you'd pass on the command line (minus the leadingslop-eval), and it returns the CLI's parsed JSON output, or a structured{"error": ...}dict on a non-zero exit, a timeout, or a subprocess failure -- the tool call itself never raises. Example:

run(["score", "--screenshot", "./preview.png", "--json"]) # -> {"result": {"target": "./preview.png", "rubric": "v1", "compositeScore": 62.0, "findings": [...], ...}}

Start it directly withslop-eval-mcp(stdio transport). Requires Python 3.9+ for the base package; themcpextra itself needsmcp>=2.0.0.

- uses: RudrenduPaul/slop-eval/action@main with: url: ${{ steps.deploy.outputs.preview_url }} fail-below: 50 env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Posts a PR comment leading with the most specific flagged finding, then the composite score. Requirespermissions: pull-requests: writein the calling workflow. Full input/output reference inaction/README.md.

Want fast, deterministic, zero-cost checks for known AI-UI tells? Impeccable's tool is the better fit today, and by star count and scope it's the more established project by far. For a holistic judgment call on layout and component novelty that a fixed rule set can't easily encode, that's what slop-eval adds. Nothing stops you from running both in the same CI job.

On speed:slop-eval is genuinely slower than Impeccable's core checks and aislop, because an LLM call sits in the critical path. Real, measured CLI-overhead numbers from a fresh clone and build, taken this session (--helpand error paths, no scoring call):

The actual scored-run latency (a real LLM-judge call, fresh vs. cached) requires a liveANTHROPIC_API_KEYthis environment doesn't have, so these two numbers aretargets pending a real measured run: under 10 seconds fresh, under 1 second on a cache hit for identical input. The cache-hit number is guaranteed by the content-hash cache logic insrc/cache/judge-cache.ts; the fresh-run number is an estimate. We would rather label a target as a target than assert a number we can't reproduce.

A slop-eval score is a heuristic quality signal from one LLM's read of your UI against a stated rubric. It is not a certification that something is or isn't AI-generated, and a clean score doesn't mean the UI is good by every measure, only that this rubric, at this version, didn't flag it.

Every score is graded againstsrc/rubric/v1.json, a real, versioned file you can open and read directly. Read it, propose changes, or pin a specific version with--rubric. A rubric version is never edited in place; a change ships as a new file so a historical score always records which rubric produced it.

- v0.1 (this release):LLM-judge scoring, CLI, GitHub Action, content-hash caching,--jsonmode, library API on both distributions.
- v0.2:ScreenshotDiffSourcebecomes real once a genuine labeled corpus exists. An Impeccable-catalog adapter, pending a license check. Explicitrescore --rubric v2command so a rubric bump is never silent.

ANTHROPIC_API_KEYis read from the environment only, is never logged, and is never written to the content-hash cache -- seeSECURITY.mdfor the full policy and the private disclosure process.

What is slop-eval, and how is it different from a linter?It's a CLI, GitHub Action, and library that scores AI-generated UI for genericness ("slop") using an Anthropic LLM judge against a versioned rubric (src/rubric/v1.json), instead of a fixed set of deterministic pattern checks. It's built to catch the "this looks like every other AI-built app" read a human reviewer gives on sight, and to run alongside a deterministic linter in the same CI job or agent loop.

Do I need an API key?Yes. slop-eval is bring-your-own-key against the Anthropic API; there's no shared or hosted key. Nothing is sent anywhere except Anthropic's API.

How do I install it, and what platforms does it support?Two independent distributions, both verified installable and runnable this session. npm:npx slop-eval-cli score ...(no install) ornpm install -g slop-eval-cli, requiring Node.js 18+ (seeenginesinpackage.json). PyPI:pip install slop-eval-cli, requiring Python 3.9-3.13 (see the classifiers inpython/pyproject.toml). Neither package has a native binary or a platform-specific build step, so both install the same way on macOS, Linux, and Windows.

How does slop-eval compare to Impeccable specifically?See theHonest comparisontable above for the full breakdown. In short: Impeccable's core is 59 deterministic checks, all enabled by default, that need no API key and run near-instantly, and the project itself has grown into a much larger design-language skill (54,000+ stars, 23 commands) beyond just slop detection; a separatecritiquecommand adds further LLM judgments on top of the deterministic set. slop-eval is a single LLM-judge call that needs a BYO Anthropic key and is slower by design, because a real model call sits in the critical path, in exchange for holistic layout/component judgment a fixed rule can't easily encode. They're built to run together in the same CI job.

Build and deploy full-stack Next.js apps with 98 tools for React, AWS, and MongoDB