AI Call Scoring Is Theatre Without a Knowledge Layer

AI call scoring that runs on a strong LLM with a well-written rubric produces results that look accurate. Scores arrive, categories get assigned, coaching flags surface. The problem shows up when you test those results against what actually happened: a rep misquoted a price tier and the scoring said the commercial conversation was handled well. A buyer was clearly outside your ICP and the discovery score came back positive. A competitor claim the rep made has been inaccurate for months, and the competitive positioning field keeps returning green. The evaluation was never wrong from the model's perspective. It was wrong from yours.

The failure mode is the same across every dimension: the model evaluates against what it can infer a good sales call looks like, not against what your company has actually defined. Without access to your documents, it falls back to training data assumptions. Those assumptions may be plausible enough that you don't notice they're wrong until a deal slips, a compliance complaint surfaces, or a rep who scores well keeps missing quota.

What a knowledge layer actually does

Knowledge grounding means attaching your organisation's documents to the evaluation so that scores become checks against your defined standards rather than inferences from training data. The distinction matters because it changes what the evaluation is actually doing. Without grounding, a field like “commercial discussion handled well” is an impression: the model assessed whether the conversation sounded competent against a general sense of what competent looks like. With grounding, the same field becomes a set of verifiable claims: was the price correct, was the value case made in the terms your methodology defines, was the scope of what's included accurately described. Impressions are provisional. Verifiable claims are auditable.

The conversation intelligence fails on knowledge, not calls framing covers why this distinction matters for everything downstream of the score. The transcript isn't the problem. What the evaluation has access to when reading it is.

What depth within a single area looks like

The real value of a knowledge layer isn't catching one kind of error. It's that any commercial dimension worth assessing has multiple facets, and each facet requires its own grounded evidence to evaluate properly. Take a single area as an example. The question “how did the commercial discussion go?” isn't one question. It includes whether the price stated was accurate, whether the case for that price was made in terms the buyer cared about, whether the rep correctly described what the buyer receives for that investment, and how the rep handled pressure when the buyer pushed back. Each of those is a distinct evaluable question. Each needs a different document to answer it reliably.

A rate card tells you what prices are. It doesn't tell you whether the rep articulated the return the buyer could expect, or whether they correctly framed what the price actually covers. A value framework tells you what commercial outcomes the offering is meant to deliver, and against what. It doesn't tell you what the approved boundaries are when a buyer pushes on price. A packaging guide defines the scope of each tier. A negotiation policy defines what flexibility exists and under what conditions. Each document has a specific evidential purpose. Without that specificity, you end up with one document that tries to cover everything, which means the evaluation can't check anything precisely. The Bricks that produce useful scoring are the ones grounded in documents with a clear, bounded purpose.

This pattern holds across every assessment area. ICP qualification isn't one check: it's firmographic criteria, use case match, stakeholder seniority, and exclusion conditions, each answerable only against the document that defines the standard. Methodology adherence isn't one field: it's stage-specific requirements, each grounded in what your playbook says is required at that point. When you build assessment areas as structured sets of grounded checks rather than single rubric scores, the scoring is precise because each question is specific, and the evidence is interpretable because each answer comes from a document with a defined purpose.

Why prompt engineering can't substitute for this

The instinct when scoring is wrong is to improve the prompt: more specific rubric language, more detailed instruction, more context in the system message. This helps at the margins but doesn't solve the underlying problem. You can't pack your pricing rules, value framework, packaging scope, negotiation policy, ICP criteria, stage requirements, and competitive standards into a prompt and expect the model to apply each of them precisely across hundreds of calls. The context competes with itself, and the model generalises where it should be checking.

The signal that you're still in prompt-dependent territory is score drift. If rewriting the evaluation instructions produces meaningfully different scores on the same calls, the evaluation is interpreting rather than checking. A grounded evaluation produces the same result regardless of how the question is phrased, because the answer comes from the document. If your scores shift when your prompts shift, the knowledge isn't in the system.

What this enables for RevOps

Comprehensive grounded scoring changes what you can trust when it reaches your CRM, forecast model, or coaching workflow. A field that assesses a commercial dimension as a single score is a liability: it can be wrong in ways invisible until a deal slips, because the single score averaged across five distinct facets may look fine even when two of them failed. Fields that assess each facet separately, grounded in the document that defines the standard, give you evidence at the level where the problem actually occurred. That evidence is what makes a field usable downstream rather than decorative.

Treat ungrounded scores as provisional inputs. Use them for directional trend detection and early-stage coaching signals. For anything that feeds a CRM field, a pipeline risk flag, a performance review, or a compliance audit, the evaluation needs to be checking against documents with defined purposes, not inferring from training data against a generic rubric.

How Semarize is built for this

Semarize supports knowledge grounding at the Kit level, and the Brick architectureis what makes multi-dimensional, document-grounded assessment possible at scale. Each Brick asks one specific question and accesses only the knowledge relevant to that question. A Brick checking whether the value case was made reads your value framework. A Brick checking whether the scope was correctly described reads your packaging guide. A Brick checking how price pressure was handled reads your negotiation policy. Because each Brick has a focused, bounded knowledge scope, the checks don't interfere with each other. Adding more Bricks to cover more facets of an area doesn't dilute the accuracy of the existing ones.

This is what makes it possible to go both broad (multiple assessment areas covered in one Kit) and deep (multiple grounded Bricks per area) without the accuracy tradeoff that comes from packing everything into a single prompt. The output is consistent structured JSON across the full Kit: one named field per Brick, each with a typed value, confidence, reason, and supporting evidence from the transcript. Because Kits are versioned in the app, scoring behaviour doesn't change when the underlying model updates. When your standards change, you update the document. The scoring reflects your current reality without a prompt rewrite. See the knowledge grounding documentation for the full setup.

Semarize grounds every Brick in a document with a defined purpose, across as many facets of your evaluation as the assessment requires.

Start building →

Common questions