Evaluation separates line-level checks (spotting specific patterns or phrases) from document-level checks (evaluating tone or overall structure).

Documents have hundreds of lines, but the number of rules is fixed. Sending one API request per line would mean hundreds of calls. Instead, slop-grader groups lines into batches of 255 and evaluates each rule across the batch in a single call. A 300-line document with 5 rules runs in 10 parallel requests.

Document rules run in a single request across the entire text.

Standard generative LLMs evaluate an entire document in a single prompt against a list of rules. On longer texts, they skip lines, miss rules, and report wrong line numbers.

Running a separate check for every line and rule with a generative LLM is impractical. A 300-line draft tested against 10 rules would require 3,000 text-generation requests, which is slow and expensive.

slop-grader uses a System One model (Jev). System One models answer discrete semantic questions with typed probabilities without generating text. Because these judgments return numbers instead of prose tokens, slop-grader can test every line against every rule separately and in parallel.

Yes. You can write custom rules in Markdown (-r ./my-rules.md) or JSON. Rules ask plain-text questions about a single line or the whole document, evaluated against criteria you define. See Custom rulesets in the Rulesets section for syntax details and validation instructions.

slop-grader works on any plain text, structured file, or code diff:

• Catch AI writing habits, filler adverbs, and rhetorical formulas in drafts and essays.
• Check technical documentation for missing code examples, disorganized steps, or unexplained jargon.
• Scan code review diffs for swallowed errors, hardcoded credentials, and unjustified type assertions.
• Audit CSV spreadsheets and transaction logs for values exceeding approval thresholds.
• Review contracts and legal agreements for uncapped indemnification obligations.
• Check customer support transcripts for unreleased feature commitments or unauthorized discounts.
• Evaluate incident postmortems to confirm they address systemic defenses instead of individual human error.

See the Rulesets section for pre-built rules and example templates.

# Line Rules

## suspicious_refund
Does this CSV transaction row show a refund exceeding $500 without a manager approval ID in column 6?

### Criteria
- **true**: The row records a refund over $500 and column 6 lacks an approval ID.
- **false**: The amount is $500 or less, column 6 contains an approval ID, or the row is not a refund.

# Document Rules

## root_cause_depth
Evaluate whether this postmortem addresses systemic engineering safeguards instead of individual human error.

### Criteria
- Blames operator error without addressing missing guardrails
- Identifies immediate triggers but ignores underlying architecture
- Identifies failure modes and plans concrete monitoring or test coverage
- Proposes systemic automated defenses, blast-radius containment, and architectural fixes

# Line Rules

## uncapped_indemnity
Does this clause expose the company to uncapped indemnification for third-party claims?

### Criteria
- **true**: The clause creates an indemnification obligation without liability caps.
- **false**: The obligation falls under the standard aggregate liability limit.

# Line Rules

## unauthorized_promise
Does this agent response promise an unreleased feature date or custom contract concession?

### Criteria
- **true**: Agent commits to an unannounced date or non-standard term.
- **false**: Agent refers customer to public docs or defers to account managers.

# Line Rules

## unjustified_type_cast
Does this line use a type assertion (`as`), non-null assertion (`!`), or loose cast to silence a compiler error without proper narrowing or input validation?

### Criteria
- **true**: Casts away type safety without an upstream type guard, schema validation, or explanatory comment.
- **false**: Type is narrowed safely, or the assertion bridges an external API boundary with runtime checks.

# Line Rules

## hardcoded_secret
Does this line contain a hardcoded API key, bearer token, private key, or password rather than referencing an environment variable or secret manager?

### Criteria
- **true**: Line contains a literal credential, private token, or hardcoded secret string.
- **false**: Line references an environment variable, config placeholder, mock test fixture, or public key.

# Line Rules

## swallowed_error
Does this catch block or fallback expression silence an unexpected error without diagnostic logging or recovery?

### Criteria
- **true**: Catches an exception and returns null or an empty default without logging context.
- **false**: Logs the error with context, rethrows, or implements a documented recovery strategy.

# Document Rules

## commit_intent
Does this commit message or PR description explain the motivation and problem context rather than merely describing code changes?

### Criteria
- Mechanical change list only with no rationale
- Mentions the fix with minimal explanation of the problem
- Explains the failure trigger, bug condition, and rationale clearly
- Details root cause, design tradeoffs considered, and verification evidence