Skip to content

Why deterministic tests are not enough in the world of AI: An Introduction to AI Evals – AI for Dummies #7

Published by Carlos Rodrigues on August 12, 2026

If you are a software engineer stepping into the world of LLMs and AI engineering, you have likely encountered this or a very similar frustrating moment:

You build a nice LLM pipeline that extracts structured JSON data from messy customer emails (or any other LLM-backed workload really).

You write standard unit tests, asserting that the returned dict matches your hard-coded dictionary.

It passes on Monday. On Tuesday, OpenAI or Anthropic (or really any other model provider) updates their model weights, or you slightly tweak your system prompt, let’s say, to improve date formatting. Suddenly, your unit test fails. Not because the AI got the answer wrong, but because it returned a synonym of the term you were expecting, or formatted "1,250.00" as 1250.0.

Traditional software testing relies on deterministic equality: Input A + Logic B = Exact Output C. AI systems, on the other hand, are non-deterministic and probabilistic: Input A + Prompt B = Range of Semantically Valid Outputs.

In this first post of our AI Evals Series (part of the #AI for dummies series) we will look into what AI Evals are, why traditional unit tests with deterministic statements break. We’ll see why LLM extractors fail in 4 main ways, and how to shift our brains into thinking in fuzzy, probabilistic assertions.

Let’s see an example: automated Invoice Extraction

As usual in this series, we will work with a concrete, real(ish)-world scenario that most companies face: Automating Expense & Invoice Data Extraction.

Imagine your company receives hundreds of vendor emails every day like this:

Hi Accounts Payable,
Please find attached invoice #INV-9921 from Acme Corp for $1,250.00 (which includes $150 tax).
Services rendered: Software Licensing, Cloud Setup Support.
Due date: 2026-09-15.
Thanks!

(I know, your company has a better system and does not receive messy emails like this, but bear with me for this series 🙂 )

Our LLM pipeline takes this raw email text and extracts structured JSON:

{
  "vendor_name": "Acme Corporation",
  "invoice_id": "INV-9921",
  "total_amount": 1250.00,
  "tax_amount": 150.00,
  "due_date": "2026-09-15",
  "line_items": ["Software Licensing", "Cloud Setup Support"]
}

The Exam vs. Recipe Analogy

If you’ve been following this series you know I love analogies. Here is another one to help you frame your thinking.

To understand why traditional testing breaks, think of code testing vs. AI testing as following a recipe, step by step, exactly as written vs. grading an essay exam, as a whole:

  • Traditional Code Unit Test (Following a Recipe): If a recipe calls for 2 eggs and 100g of flour, and you put in 3 eggs and 500g of sugar, you failed the recipe. There is only one correct sequence of operations.
  • AI Evaluation (Grading an Essay): If a student is asked “Who was Julius Caesar?”, one student might write “A Roman general and statesman who played a critical role in the demise of the Roman Republic.” Another might write “The famous Roman dictator assassinated in 44 BC.” Both answers earn an A+. An assert student_answer == "A Roman general" test would give one of them a zero.

The 4 Main Ways AI Extractors Fail in Production

When building LLM extractors, failure isn’t just a thrown exception or a syntax error. LLMs fail in subtler, human-like ways:

  1. Hallucination (Making things up): Inventing a $150.00 amount when the original receipt mentioned the value as tax. LLMs are known to do this kind of nice surprises.
  2. Schema & Type Drift: Returning string "1250" instead of a float 1250.00, or wrapping JSON in markdown backticks (```json ... ) when your API parser expects raw JSON.
  3. Omission (Skipping data): Extracting only 1 line item when 4 items were listed in the email.
  4. Semantic Alteration: Changing "Acme Corporation" to "Acme Inc." (which might break foreign key lookups in your relational database) – Although these are semantically equivalent for humans, they are not for deterministic code, and may cause failures in downstream systems.

Deterministic vs. Fuzzy Evals

Does this mean we throw away standard code checks entirely? 

No! The secret to robust AI Evals is combining Deterministic Structural Assertions with Fuzzy Semantic Assertions.

1. Deterministic Checks (Fast & Free)

Before spending money on LLM calls to grade your LLM, check the basics programmatically:

  • Is the output valid JSON?
  • Are required keys (vendor_name, total_amount) present?
  • Are total_amount and tax_amount positive numbers?
  • Is due_date formatted as YYYY-MM-DD?
  • Are any other deterministic constraints that can be checked? Do it

2. Fuzzy & Semantic Checks (The Eval Engine)

Once structural validity passes, evaluate the semantic quality:

  • Fuzzy String Containment: Is "acme" contained in vendor_name.lower()?
  • Numerical Range Tolerance: Is abs(extracted_amount - true_amount) < 0.01?
  • Set Similarity: Did the model extract all expected line items, regardless of exact word order?

Ok, that’s nice, but how does it work in practice?

Let’s look at how this looks in Python using pytest:

import pytest
from extractor import extract_invoice_data_mock

RAW_EMAIL = """
Hi Accounts Payable,
Please find attached invoice #INV-9921 from Acme Corp for $1,250.00 (which includes $150 tax).
Services rendered: Software Licensing, Cloud Setup Support.
Due date: 2026-09-15.
"""

def test_traditional_exact_match_fails():
    """Traditional exact string matching fails on minor variations."""
    result = extract_invoice_data_mock(RAW_EMAIL)
    
    # Expected dictionary with exact string 'Acme Corp'
    expected_exact = {
        "vendor_name": "Acme Corp", # LLM returned 'Acme Corporation'
        "invoice_id": "INV-9921",
        "total_amount": 1250.00,
        "tax_amount": 150.00,
        "due_date": "2026-09-15",
        "line_items": ["Software Licensing", "Cloud Setup Support"]
    }
    
    # Fails in traditional testing!
    # assert result == expected_exact 

def test_ai_eval_fuzzy_matching_passes():
    """Proper AI Eval: Asserts structural, numerical, and semantic rules."""
    
    #perform real LLM call
    result = extract_invoice_data_mock(RAW_EMAIL)
    
    # 1. Semantic Vendor check (handles 'Acme Corp' vs 'Acme Corporation')
    assert "acme" in result["vendor_name"].lower()
    
    # 2. Strict ID check
    assert result["invoice_id"] == "INV-9921"
    
    # 3. Floating point tolerance check
    assert abs(result["total_amount"] - 1250.00) < 0.01
    assert abs(result["tax_amount"] - 150.00) < 0.01
    
    # 4. Collection item coverage check
    assert len(result["line_items"]) == 2
    assert any("software" in item.lower() for item in result["line_items"])

The combination of both deterministic and fuzzy checks is the foundation for robust LLM evaluations and will yield much more reliable results than traditional unit tests alone.

By running this on every prompt, model or configuration change in your CI/CD pipeline, you’ll catch regressions early and ensure that your LLM system is production-ready.

Is that it?

Basic fuzzy checks work great for single fields, but what happens when you need to evaluate complex multi-sentence summaries, subtle hallucinations, or nuanced reasoning?

That’s where LLM-as-a-Judge comes in.

In the next articles of this series, we will build a custom LLM-as-a-Judge evaluator from scratch in pure Python, and explore how to build Golden Datasets using both Synthetic Generation and Human Curation.

Hope this helps clearing a bit the “fuzziness” of what evals are.

Cheers

Published inAIML

Be First to Comment

Leave a Reply