Off-the-shelf, generic evaluation metrics (e.g. a pre-built hallucination score) are often not worth reporting as results — they're too generic to reflect what actually matters for a specific product. But they still have a legitimate secondary use: as a sorting device to direct manual Error Analysis (Manual Trace Review) toward the most promising traces.
The pattern: score every trace with the generic metric, sort by that score, and manually review the highest- and/or lowest-scoring traces to see whether a real, product-specific failure pattern emerges. If it does, that pattern becomes a candidate for a purpose-built Code-Based (Reference-Based) Evaluation or a validated LLM-as-judge — see Eval Investment Triage for how that decision gets made. The generic score itself is discarded; only the traces it surfaced are kept.