LLM Evaluation: From MMLU to Real Business Metrics

LLM evaluation is one of the hardest unsolved problems in AI.

If you can’t measure quality, you can’t improve it.

This piece covers: academic benchmarks, real-world evaluation strategies, and the hybrid systems used in production.

Why LLM Evaluation Is Hard

Unlike classifiers (“right” / “wrong”), LLM outputs are:

• Open-ended — many “correct” responses
• Multi-dimensional — accuracy + safety + helpfulness + style
• Context-dependent — what’s good in one app is bad in another
• Subjective — what does “good writing” mean?

This is why “how to evaluate LLMs” has its own research subfield.

Tier 1: Academic Benchmarks

These are what model card papers report. Common ones:

Benchmark	Tests	Notes
MMLU	57 subjects, multi-choice	The “general knowledge” benchmark
HumanEval	Code completion	Functional correctness
GSM8K / MATH	Math word problems	Step-by-step reasoning
HellaSwag	Common sense	Continue stories sensibly
MT-Bench	Multi-turn chat	LLM-as-judge
SWE-bench	Real GitHub bugs	Realistic coding
GPQA	Graduate-level questions	Harder than MMLU
AIME	Math olympiad	Reasoning capability

Critical limits:

• Contamination: many benchmarks leak into training data → inflated scores
• Gameability: train on similar examples = score goes up, ability doesn’t
• Doesn’t transfer: 90% on MMLU ≠ good at your actual task

Tier 2: LLM-as-Judge

Use another LLM (often GPT-4 or Claude) as the judge:

def llm_judge(prompt, response_a, response_b):
    eval_prompt = f"""
    Compare these two responses to the same prompt.
    Rate each on helpfulness (1-10), accuracy (1-10), and clarity (1-10).
    Pick the better one.

    Prompt: {prompt}
    Response A: {response_a}
    Response B: {response_b}
    """
    return claude.invoke(eval_prompt)

Pros: scalable, fast, cheap Cons:

• Bias (LLM judges often prefer wordy responses)
• Position bias (prefer first option shown)
• Not always accurate on technical correctness

Used by: MT-Bench, AlpacaEval, Arena.

Tier 3: Human Eval

The gold standard. Pay humans to rate model outputs.

Cost: $0.5-5 per rating Throughput: limited by labelers Quality: best for subjective tasks (writing, art, safety)

Used by: Chatbot Arena (ELO ranking from millions of human votes).

Tier 4: Business Metrics

The actual test: does it work for your users?

• Conversion rate
• Customer satisfaction (CSAT)
• Task completion rate
• Time saved
• Revenue impact

This is where most production LLM apps end up. The benchmark scores are a starting point — but the real measure is does this improve the user’s outcome.

Building an Eval Suite

For a production LLM app:

1. Golden test set

• 50-200 hand-curated examples
• Covers important use cases
• Includes edge cases
• Run on every release

2. Regression tests

• “Did this still work after the update?”
• Auto-run in CI

3. A/B testing

• Real users see different versions
• Measure conversion / engagement
• Statistically rigorous

4. Production monitoring

• Sample 1% of real responses
• Score with LLM-as-judge
• Track quality over time

Tools

• Promptfoo — open-source eval framework
• LangSmith — eval + monitoring
• HumanLoop — human + auto
• DeepEval — pytest-style LLM tests
• Inspect AI (UK AISI) — safety-focused

Common Pitfalls

1. Over-relying on one metric

Trust no single benchmark. Use multiple. Cross-check.

2. Training to the test

If your eval set leaks into training, scores explode but real quality doesn’t.

3. Ignoring failure modes

A model with 90% accuracy is wonderful… until the 10% includes the most-asked questions.

4. Static evals on dynamic problems

Your users’ questions evolve. Your evals must too.

A Real Eval Pipeline

[New model version]
        ↓
[Run on golden set]
   - 200 examples
   - LLM-as-judge for soft tests
   - Exact match for hard tests
        ↓
[Compare to current production]
   - Side-by-side ELO
   - Win-rate by category
        ↓
[Smoke test in staging]
   - 1% real traffic
   - Monitor latency, errors, user feedback
        ↓
[Gradual rollout]
   - 10%, 50%, 100%
   - Pause + roll back if quality drops

Next recommended: L4-09 Tool Use or L4-10 Multi-Agent.