04. ML Metrics Taxonomy — Full Reference + Production Application

"Choosing the right metric is as important as choosing the right model. I've seen teams optimize for accuracy when they should have optimized for recall, and measure F1 when the business needed conversion rate. Here's the full taxonomy of ML metrics I used, why each matters, and which ones actually drove production decisions."

Overview: Metric Categories in MangaAssist

graph TD
    subgraph "Classification Metrics"
        C1[Accuracy]
        C2[Precision]
        C3[Recall]
        C4[F1 Score]
        C5[Confusion Matrix]
        C6[AUC-ROC]
        C7[AUC-PR]
        C8[Log Loss]
    end

    subgraph "Ranking & Retrieval Metrics"
        R1["Recall@K"]
        R2["Precision@K"]
        R3[MRR]
        R4[NDCG]
        R5[MAP]
        R6[Hit Rate]
    end

    subgraph "Embedding Quality Metrics"
        E1[Cosine Similarity Distribution]
        E2[Embedding Clustering Quality]
        E3[Nearest Neighbor Accuracy]
        E4[Alignment & Uniformity]
    end

    subgraph "Reranking Metrics"
        RR1[Reranking Lift]
        RR2["NDCG@K Improvement"]
        RR3[Pairwise Accuracy]
    end

    C1 --> Applied1[Intent Classifier]
    R1 --> Applied2[RAG Pipeline]
    E1 --> Applied3[Embedding Model]
    RR1 --> Applied4[Cross-Encoder Reranker]

Metric Selection Decision Guide

"Which metric should I use?" — Use this decision tree to pick the right metric for your scenario.

Common Metric Mistakes to Avoid

Part 1: Classification Metrics (Intent Classifier)

These metrics evaluated our fine-tuned DistilBERT intent classifier, which categorized every user message into one of 8 intent classes.

1.1 Accuracy

Definition: Percentage of correctly classified messages out of all messages.

$$\text{Accuracy} = \frac{\text{Correct Predictions}}{\text{Total Predictions}}$$

MangaAssist Application: Overall intent classification accuracy across all 8 intents.

Why it matters but isn't enough: Accuracy hides class imbalance. If chitchat is 40% of messages and the model always predicts chitchat, accuracy could be 40% while being completely useless for every other intent. That's why I paired it with per-class metrics.

When I used it: Weekly dashboard metric, gate for model deployment (must be ≥ 90% on test set).

1.2 Precision

Definition: Of all messages predicted as a given intent, what percentage actually were that intent?

$$\text{Precision}_c = \frac{\text{True Positives}_c}{\text{True Positives}_c + \text{False Positives}_c}$$

MangaAssist Application: Per-intent precision tells me how "trustworthy" each intent classification is.

Why precision matters for escalation: Every false positive escalation sends a user to a human agent unnecessarily — costing ~$5 per escalation. At 500K messages/day, even 1% false positive escalations = 5,000 unnecessary escalations = $25K/day wasted.

When I used it: Flagged when per-class precision dropped below 0.85. Drove training data augmentation for confused classes.

1.3 Recall

Definition: Of all messages that actually belong to a given intent, what percentage did the model correctly identify?

$$\text{Recall}_c = \frac{\text{True Positives}_c}{\text{True Positives}_c + \text{False Negatives}_c}$$

MangaAssist Application: Per-intent recall tells me how many messages I'm "missing" for each intent.

Why recall matters for return_request: A missed return request means the user doesn't get routed to the returns API. They get a generic LLM response that can't process the return — leading to escalation and frustration.

Precision-Recall tradeoff by intent:

1.4 F1 Score

Definition: Harmonic mean of precision and recall. Balances both.

$$F1_c = 2 \times \frac{\text{Precision}_c \times \text{Recall}_c}{\text{Precision}_c + \text{Recall}_c}$$

MangaAssist Application: Used as the single per-class metric for model comparison.

Deployment gate: All per-class F1 scores must be ≥ 0.85. If any class drops below, the model update is blocked until training data for that class is augmented.

Macro vs. Weighted F1: - Macro F1 (unweighted average across classes): Used for DS model comparison — ensures rare classes aren't ignored. - Weighted F1 (weighted by class frequency): Used for production monitoring — reflects actual user impact.

1.5 Confusion Matrix

Definition: A matrix showing predicted vs. actual class distributions.

MangaAssist Application: I used confusion matrices to identify specific class-pair confusions:

                     Predicted
                 rec   prod   faq   order  return  esc   promo  chat
Actual  rec     264    18      5      0      0      3      8      2
        prod     22   269      8      0      0      1      0      0
        faq       3    12    255      0      5      5     15      5
        order     0     0      0    282      8      5      0      5
        return    0     2      5      5    270      8      5      5
        esc       2     0      3      3      5    276      1     10
        promo     8     0     18      0      3      2    246     23
        chat      0     0      2      3      2      8     10    275

Key insight from the confusion matrix: promotion ↔ faq and promotion ↔ chitchat were the most confused pairs. Queries like "Are there any deals?" could be a promotion query or a casual FAQ. This drove us to: 1. Add more promotion-specific training examples. 2. Consider merging promotion into faq (decided against it — promotion queries route to the Promotions Service API, while FAQ queries use RAG).

1.6 AUC-ROC (Area Under the Receiver Operating Characteristic Curve)

Definition: Measures the model's ability to distinguish between classes at all classification thresholds. AUC = 1.0 means perfect separation; AUC = 0.5 means random.

$$\text{AUC-ROC} = \int_0^1 \text{TPR}(t) \, d(\text{FPR}(t))$$

MangaAssist Application: I computed per-class AUC-ROC (one-vs-rest):

When I used it: Primarily for DS model comparison (offline). Not a production monitoring metric — the operating threshold was fixed in production.

1.7 AUC-PR (Area Under the Precision-Recall Curve)

Definition: Like AUC-ROC but focused on the positive class. More informative for imbalanced classes.

MangaAssist Application: escalation intent was only ~5% of messages (imbalanced). AUC-ROC looked great (0.97) because the model could easily identify non-escalation messages. AUC-PR was more honest:

Key insight: For rare intents, AUC-PR was the better metric. It revealed that our escalation classifier was weaker than AUC-ROC suggested.

1.8 Log Loss (Cross-Entropy Loss)

Definition: Measures the quality of predicted probability distributions, not just the final classification. Penalizes confident wrong predictions more harshly.

$$\text{Log Loss} = -\frac{1}{N} \sum_{i=1}^{N} \sum_{c=1}^{C} y_{i,c} \log(p_{i,c})$$

MangaAssist Application: I monitored log loss as a proxy for model confidence calibration:

Why calibration matters: I used the classifier's confidence score to decide whether to use the rule-based path or fall back to BERT. If the model was overconfident (low log loss but wrong), it wouldn't fall back when it should.

Part 2: Ranking & Retrieval Metrics (RAG Pipeline)

These metrics evaluated the RAG retrieval pipeline: given a user query, did we retrieve the right documents from OpenSearch?

2.1 Recall@K

Definition: Percentage of queries where the correct document appears in the top K retrieved results.

$$\text{Recall@K} = \frac{\text{Number of queries with relevant doc in top K}}{\text{Total queries}}$$

MangaAssist Application:

Target: Recall@3 ≥ 80%. We achieved 86% after embedding fine-tuning (up from 72% baseline).

Why Recall@3: We injected exactly 3 chunks into the LLM prompt (token budget constraint). If the correct document wasn't in those 3, the LLM couldn't generate a grounded answer.

2.2 Precision@K

Definition: Of the K retrieved documents, what fraction are actually relevant?

$$\text{Precision@K} = \frac{\text{Relevant docs in top K}}{K}$$

MangaAssist Application:

Why precision matters for RAG: Low precision means irrelevant documents are injected into the LLM prompt, which: 1. Wastes tokens (each irrelevant chunk costs ~500 tokens × $3/M = wasted money). 2. Confuses the LLM — irrelevant context can lead to hallucinations or off-topic responses.

2.3 MRR (Mean Reciprocal Rank)

Definition: Average of the reciprocal rank of the first relevant result. MRR = 1.0 means the first result is always relevant.

$$\text{MRR} = \frac{1}{|Q|} \sum_{i=1}^{|Q|} \frac{1}{\text{rank}_i}$$

MangaAssist Application:

When I used it: MRR was the go-to metric for comparing retrieval strategies. Higher MRR = the LLM sees the most relevant chunk first, which matters because LLMs tend to weight early context more heavily.

2.4 NDCG (Normalized Discounted Cumulative Gain)

Definition: Measures ranking quality, accounting for the position of relevant results. Higher-ranked relevant results contribute more to the score.

$$\text{DCG@K} = \sum_{i=1}^{K} \frac{2^{rel_i} - 1}{\log_2(i + 1)}$$

$$\text{NDCG@K} = \frac{\text{DCG@K}}{\text{IDCG@K}}$$

MangaAssist Application: NDCG was particularly useful for multi-relevance queries. For a recommendation query like "dark fantasy manga," multiple documents were relevant but with varying degrees:

Relevance levels:
3 = Highly relevant (directly answers the query)
2 = Relevant (related but incomplete)
1 = Partially relevant (tangentially related)
0 = Irrelevant

When I used it: Primarily in the weekly RAG evaluation pipeline. More nuanced than Recall@K because it rewards putting the best documents first.

2.5 MAP (Mean Average Precision)

Definition: Mean of the average precision scores across all queries. Average precision rewards ranking relevant documents higher.

$$\text{AP} = \frac{1}{|R|} \sum_{k=1}^{K} \text{Precision@k} \times rel(k)$$

MangaAssist Application: MAP was used as a single-number summary of retrieval quality for weekly reporting. Our MAP was 0.78 (hybrid + reranking), up from 0.64 (vector only).

2.6 Hit Rate

Definition: Simplest retrieval metric — was there at least one relevant document in the top K?

$$\text{Hit Rate@K} = \frac{\text{Queries with ≥ 1 relevant doc in top K}}{\text{Total queries}}$$

MangaAssist Application: Hit Rate@3 = 89%. This was the "hard floor" metric — if we couldn't even find one relevant document, the RAG pipeline was functionally broken for that query.

Part 3: Embedding Quality Metrics

3.1 Cosine Similarity Distribution

What I measured: The distribution of cosine similarity scores between query embeddings and their top retrieved documents.

Baseline Titan Embeddings:
  - Avg similarity for relevant pairs: 0.72
  - Avg similarity for irrelevant pairs: 0.45
  - Separation gap: 0.27

Fine-tuned Adapter:
  - Avg similarity for relevant pairs: 0.84
  - Avg similarity for irrelevant pairs: 0.38
  - Separation gap: 0.46

A wider separation gap = easier for the retrieval system to distinguish relevant from irrelevant documents. The fine-tuned adapter nearly doubled the gap.

3.2 Nearest Neighbor Accuracy

Definition: For a set of query-document pairs, is the correct document the nearest neighbor in embedding space?

MangaAssist Application: Nearest neighbor accuracy improved from 58% to 76% after fine-tuning — particularly for manga-specific vocabulary where the base model struggled.

3.3 Alignment & Uniformity

Two properties of good embedding spaces: - Alignment: Semantically similar items should be close. Measured as average distance between positive pairs. - Uniformity: Embeddings should be spread across the space (not all clustered). Measured as log of pairwise Gaussian potential.

These were DS-internal metrics used during embedding adaptation training — not production monitoring metrics.

Part 4: Reranking Metrics

4.1 Reranking Lift

Definition: Improvement in retrieval quality from the cross-encoder reranker vs. raw vector search results.

The reranker added ~50ms of latency but significantly improved retrieval quality. The reranking lift justified the latency cost.

4.2 Pairwise Accuracy

Definition: Given two documents (one relevant, one irrelevant), does the reranker rank the relevant one higher?

MangaAssist Application: Pairwise accuracy = 93%. The 7% failure rate occurred mostly when both documents were partially relevant (relevance 1 vs. 2), not when comparing relevant vs. clearly irrelevant.

Part 5: Metrics I Actually Tracked in Production (Focused Section)

"Out of the 20+ metrics above, these are the ones on my production dashboard that I checked daily and used to drive decisions."

Tier 1: Daily Dashboard (Alerted On)

Tier 2: Weekly Review (DS Sync)

Tier 3: Monthly Deep Dive (Model Release Gate)

Key Takeaways for Interviews

1. "Accuracy alone is misleading" — I always pair accuracy with per-class F1 and confusion matrices. A 92% accurate model can have a 0.78 F1 on a critical intent class.

2. "Different metrics for different stakeholders" — DS cares about AUC-ROC and F1. Engineering cares about latency and throughput. Business cares about escalation rate and conversion. The shared metrics (Tier 1) bridge these worlds.

3. "Recall@3 was our north star retrieval metric" — Because we injected exactly 3 chunks, Recall@3 directly translated to "probability the LLM has good source material."

4. "Precision-recall priority depends on the intent" — For escalation, recall matters more (never miss a frustrated user). For recommendation, precision matters more (don't route real questions to the reco engine).

5. "AUC-PR is more honest than AUC-ROC for rare classes" — Our escalation AUC-ROC was 0.97 (misleading) vs. AUC-PR of 0.82 (honest).

6. "Embedding quality metrics showed the impact of fine-tuning" — The cosine similarity gap nearly doubled after domain-specific fine-tuning, and this translated directly to +14% Recall@3.

Related Documents

• 02-data-scientist-collaboration.md — How DS and I jointly defined metric thresholds
• 03-tradeoffs-decisions.md — How metrics drove tradeoff decisions
• 13-metrics.md — Business, UX, AI quality, and operational metrics framework
• Challenges/real-world-challenges.md §7 — RAG Quality — Retrieval challenges and solutions