Benchmark Methodology
Each evaluation input is analyzed using the same production configuration used by the API. Ranked outputs are compared with the expected labels. The benchmark suite publishes versioned natural-language evaluation sets per analyzer with internal consistency validation; analyzers without published per-language results are marked pending rather than attributed coverage. The catalog keeps ranked and production-gated metrics separate so benchmark rankings are not represented as calibrated probabilities.
Test Suite
Generated natural-language samples per analyzer and language, plus internal consistency validation for each versioned configuration.
Evaluation Rules
Production Configuration
Each evaluation input is analyzed using the same production configuration used by the API. Ranked outputs are compared with the expected labels.
Natural Top-3
A generated natural sentence passes when its expected concept appears in the top three ranked results.
Production Gate
The production metric additionally requires the expected signal to meet the quality gate applied to live responses.
Internal Consistency Validation
Reference artifacts pass configuration-integrity checks before each benchmark release.
Evaluation Sequence
Candidate scoring
Every candidate signal for the input receives a score under the pinned production configuration.
Filtering
The production gate filters candidates exactly as in live API responses. Filtering happens after scoring and before ranking - it affects which candidates may appear, never which cases count.
Ranking
Remaining candidates are ordered by score. The ranked diagnostic uses the unfiltered order over the same cases.
Truncation
The ordered list is truncated to the top 3.
Label comparison
A case is a hit if and only if its expected label appears in the truncated list.
Same denominator
Every reported rate for a language divides by the full evaluated sample for that language (the published sample count). Filtering and gating affect only the numerator - a filtered case is still an evaluated case.
Abstentions and errors
When the production gate returns zero signals for a case (abstention), that case counts as a miss in the production metric. Cases that error during evaluation also count as misses - no evaluated case is dropped silently.
Conditional quality and coverage
Quality conditional on returning a result is not published without its coverage figure beside it. Coverage is the share of evaluated cases returning at least one gated signal.
Scoring
Pure ranking evidence without applying the production quality gate.
Ranking evidence after applying the quality gate used in production responses. The predeclared primary metric.
Reference artifacts pass configuration-integrity checks.
Production Top-3 is the single predeclared public primary metric - chosen before evaluation, not after. Ranked Top-3 is published alongside it as a diagnostic; the primary is never whichever number is higher.
Engine Parameters
| engine | Deterministic, versioned, non-generative analysis engine - fixed per configuration ID. |
| quality_gate | The production quality gate applied to live responses; benchmark target is 85.0% Top-3 Hit Rate. |
| evidence_source | Metrics are generated reproducibly from version-controlled evaluation artifacts. |
Models
| evaluation_data | Evaluation datasets contain controlled synthetic, curated, licensed, and/or reviewed examples as documented for each benchmark release. |
| analysis | Versioned, proprietary analysis engine (fixed per configuration ID) |
| catalog_version | bcat_Q7K2N |
| enterprise | Full dataset provenance - source, licensing, and lineage - is documented per benchmark release and disclosed to Enterprise customers under NDA. |
Analyzer Results
| Analyzer | Languages | Samples | Ranked Top-3 | Production Top-3 | Integrity |
|---|---|---|---|---|---|
| Trust & Safety | 16 | 1,544 | 95.7–100.0% | 95.7–100.0% | Pass |
| Cybersecurity | 15 | 2,237 | 85.3–95.9% | 85.3–96.6% | Pass |
| Discrimination Analysis | 16 | 2,399 | 85.3–96.7% | 88.0–98.7% | Pass |
| Crisis Intelligence | 21 | 3,117 | 85.3–93.3% | 86.7–96.7% | Pass |
| Disinformation Analysis | 16 | 2,400 | 85.3–95.3% | 87.3–100.0% | Pass |
| Market Intelligence | 8 | 1,081 | 85.2–89.8% | 85.4–93.2% | Pass |
| Emotion Analysis | 16 | 1,008 | 85.7–95.2% | 87.3–95.2% | Pass |
| Customer Service Intelligence | 18 | 2,700 | 85.3–98.7% | 87.3–96.7% | Pass |
Reproducibility
Metrics are generated reproducibly from version-controlled evaluation artifacts. The portal never edits published metrics by hand.
Benchmark Results
Per-language Top-3 Hit Rate
vs LLM Moderation
Sentiqra vs Perspective, GPT-4o
Playground
Test detection live - no API key
Reference Architecture
Migrating from LLM moderation
Indicator Dictionary
Browse analyzer indicator dictionaries
Quickstart Guide
First API call in under 5 minutes
Pricing
From Free to Enterprise
Legal Disclaimer
Benchmark methodology claims