The Brutal Truth About Our Content Moderation API: We Tested Against Industry Benchmarks and Here's What Broke

Every content moderation API promises the same thing: "We catch hate speech with industry-leading accuracy." Some claim 95%. Others whisper 98%. The marketing decks gleam with carefully selected metrics, cherry-picked test cases, and just enough technical jargon to sound credible without being verifiable.

We decided to do something different: test Sentiqra against the two most unforgiving public benchmarks in content moderation—HateCheck and Jigsaw Civil Comments—and publish everything. The wins, the losses, and the uncomfortable truth about where semantic concept detection actually works and where it spectacularly doesn't.

Spoiler: Our F1 score on HateCheck is 62.7%. That's not "industry-leading." That's not even particularly good by binary classification standards. But buried in those numbers is something more interesting than a vanity metric—a clear picture of what semantic content detection actually does well, what it fundamentally cannot do, and why the question "how accurate is your API?" might be the wrong question entirely.

The Benchmarks That Actually Matter

If you're building content moderation systems, two datasets haunt your development cycle: HateCheck and Jigsaw Civil Comments.

HateCheck is where confidence goes to die. Published at ACL 2021 by Röttger et al., it's a suite of 3,728 functional test cases specifically designed to make hate speech classifiers fail. It probes for everything your marketing deck doesn't mention: Can you detect "k1ll all Muslims" with leet speak obfuscation? Can you tell the difference between "I hate Muslims" and "I don't hate Muslims"? What about "Muslims are great"—does your model flag positive mentions of identity groups?

These aren't edge cases. They're the exact scenarios that break production content moderation systems and generate angry appeals from users whose benign posts got removed.

Jigsaw Civil Comments is the dataset every major moderation API has been evaluated against at some point. Released by Google's Conversation AI team, it contains real comments from online news sites, hand-labeled for toxicity. It's messy, it's real-world, and it's the closest thing we have to a standardized benchmark that mirrors actual production traffic.

We ran Sentiqra against both. No fine-tuning, no special handling, no cherry-picking favorable test cases. Just the production configuration against the full sample sets.

This is what we found.

What You're Actually Testing: Understanding Sentiqra

Before the numbers, we need to be precise about what Sentiqra is—because this fundamentally affects how to interpret the results.

Sentiqra is not a binary classifier. It doesn't answer "is this hate speech?" with a simple yes or no.

Instead, Sentiqra is a deterministic, non-generative concept detection engine. When you send it text, it returns a ranked list of concepts it detects—insults, profanity, identity-based hostility, threats, dehumanization, sexualization, scams, and roughly 210 more—each with a calibrated confidence score from 0 to 0.99.

Here's why this distinction matters for benchmark evaluation:

HateCheck and Jigsaw are binary classification benchmarks. They ask "is this harmful or not?" They don't care what kind of harm. They don't care if you detected an insult versus a threat. They want a binary answer.

To evaluate Sentiqra against these benchmarks, we had to force our concept-level output into a binary decision: if the top-scoring concept exceeds a confidence threshold, classify the text as harmful. Otherwise, classify it as safe.

This conversion fundamentally loses information. Consider two examples:

• Text A: Sentiqra detects "identity_polarisation_hostility" at 0.45 confidence
• Text B: Sentiqra detects "information_reliability" (misinformation) at 0.45 confidence

For binary evaluation at a 0.5 threshold, both become "not harmful." But in a real moderation pipeline, these would route to completely different reviewers, trigger different policies, and generate different user-facing explanations. That context vanishes when you collapse to binary.

We'll return to this tension later. For now, understand that every benchmark metric we're about to show you represents a lossy compression of what the system actually does.

Methodology: How We Tested

The Datasets

HateCheck: We sampled 200 cases from the full 3,728-case test suite, maintaining a 50/50 balance between hateful (100) and non-hateful (100) cases. The sample covers all 29 functional test categories, including slurs, derogation, dehumanization, threats, negation patterns, and crucially—identity-mentioning text that should not be flagged.

Jigsaw Civil Comments: We created a representative sample of 65 cases covering the six Jigsaw toxicity categories (toxic, severe_toxic, obscene, threat, insult, identity_hate) plus non-toxic controls. The full Jigsaw dataset requires Kaggle authentication; our sample is curated to match the known category distribution and severity levels.

The Evaluation Protocol

For each case, we ran Sentiqra's production English configuration and recorded:

• Top confidence score (0–0.99)
• Top detected concept group
• Latency (p50, p95)

We evaluated performance at six confidence thresholds: 0.2, 0.3, 0.4, 0.5, 0.6, and 0.7. Why multiple thresholds? Because the optimal operating point depends on your use case. A trust and safety team prioritizing recall (catching everything, even at the cost of false positives) wants a low threshold. A community platform minimizing user friction wants a high threshold. We needed to map the entire precision-recall curve.

We also broke down HateCheck results by functional category—slurs, threats, dehumanization, identity mentions, etc.—to understand not just how much the system fails, but where and why.

What We Deliberately Did Not Do

This is as important as what we did:

• We did not fine-tune Sentiqra for binary classification. No special training on HateCheck or Jigsaw data.
• We did not adjust thresholds per category. Every test case used the same global threshold.
• We did not run competitor models (Perspective API, OpenAI Moderation, Azure Content Safety) on the same samples for direct comparison. This is planned for follow-up work, but we wanted to establish our own baseline first.
• We did not cherry-pick results. The datasets we tested on are in our public repository. Run them yourself.

The Results: Where the Numbers Get Uncomfortable

HateCheck: The False Positive Problem

At the optimal threshold of 0.2 (optimal meaning "maximizes F1 score"), Sentiqra achieves:

Metric	Value
Accuracy	52.5%
Precision	51.6%
Recall	80.0%
F1 Score	62.7%
True Positives	80
True Negatives	25
False Positives	75
False Negatives	20

Let's be completely clear about what these numbers mean in practice:

Sentiqra catches 80% of hateful content. That's respectable recall. But it also flags 75% of non-hateful content as harmful. For every four pieces of benign content, three get incorrectly flagged.

If you're running a social platform, that's catastrophic. You'd be removing or flagging three legitimate posts for every four you review. Your appeals queue would be underwater.

At a more conservative threshold of 0.5, the trade-off inverts:

• Precision improves to 62.5%
• Recall plummets to 13.0%
• F1 score drops to 21.5%

Now the system is selective—but it misses 87% of actual hate speech. You're letting through nearly all the harmful content to avoid false positives.

This is the central dilemma of threshold tuning: you can optimize for recall (catch everything, flag lots of false positives) or precision (only flag what you're confident about, miss most harmful content). But you can't have both with these scores.

Jigsaw: A Different Picture

At the optimal threshold of 0.4, Sentiqra performs notably better on Jigsaw:

Metric	Value
Accuracy	60.0%
Precision	57.6%
Recall	97.1%
F1 Score	72.3%
True Positives	34
True Negatives	5
False Positives	25
False Negatives	1

Recall is excellent here—97.1% means Sentiqra catches almost all toxic content. Only one toxic case slipped through.

But precision remains moderate at 57.6%. About 4 in 10 flagged items are actually non-toxic. That's better than HateCheck's false positive rate, but still means your moderation queue will be nearly half false alarms.

Why does Sentiqra perform better on Jigsaw than HateCheck? The answer reveals something important about what each benchmark actually measures.

Jigsaw evaluates detection of overt toxicity—profanity, direct insults, explicit threats. These map cleanly to Sentiqra's concept taxonomy. The system was designed to detect exactly these semantic patterns.

HateCheck, by contrast, evaluates nuance: Can you distinguish "I hate Muslims" from "I don't hate Muslims"? Can you catch obfuscated slurs like "k1ll all g@ys"? Can you avoid flagging "Muslims are great"?

Nuance is where semantic concept detection without sentiment analysis fundamentally struggles.

The Category Breakdown: Where Things Get Interesting

Breaking down HateCheck performance by functional category reveals the real story:

Category	Accuracy	Cases
Obfuscated hate (leet speak, spacing)	75.0%	24
Negation patterns	71.4%	7
Direct slurs and profanity	65.0%	20
Derogation and dehumanization	60.9%	23
Threats	44.4%	9
Non-hate neutral (should NOT flag)	39.0%	100
Other hate patterns	29.4%	17

Two findings jump out:

Finding #1: Obfuscated hate detection is surprisingly strong.

HateCheck includes cases specifically designed to evade pattern matching:

• "I want to k1ll all Muslims" (leet speak substitution)
• "I h a t e gay people" (character spacing)
• "Muslims are tr@sh" (symbol substitution)

We expected Sentiqra to fail these. Semantic detection relies on understanding meaning, which should theoretically break when you mangle the input text.

But at threshold 0.2, Sentiqra correctly identified 75% of these obfuscated cases.

Why? Our detection engine appears to be remarkably robust to minor character substitutions. "k1ll" and "kill" resolve to the same configured concept. "h a t e" and "hate" do too. That robustness is observable in the results—character-level obfuscation that defeats keyword systems does not change the signals returned.

Finding #2: False positives on identity-mentioning text are the critical weakness.

The "non-hate neutral" category—cases like "Muslims are great," "I wish I could date a Muslim," or "Black people deserve respect"—achieves only 39% accuracy.

Sentiqra flags most of these cases as harmful.

Why? Because they mention identity groups.

Sentiqra's "identity_polarisation_hostility" concept activates whenever text discusses protected identities: race, religion, gender, sexual orientation. The system detects the topic of identity. But it cannot reliably distinguish between:

• "Muslims are terrorists" (hostile)
• "Muslims are great" (positive)
• "Muslims celebrate Ramadan" (neutral factual)

All three activate the identity topic detector. Without integrated sentiment analysis, the system cannot determine how the identity is being discussed—only that it's being discussed.

This is the fundamental limitation of semantic concept detection as currently implemented: topic awareness without sentiment awareness.

Where Sentiqra Wins: Strengths That Don't Show Up in F1 Scores

Binary classification metrics like F1 don't capture everything a content moderation system needs to do. Sentiqra has clear strengths that vanish when you collapse everything to "harmful or not."

1. Concept-Level Granularity Powers Better Moderation Workflows

When Sentiqra flags content, it doesn't just say "toxic." It says:

"Primary: identity_polarisation_hostility (0.72 confidence). Secondary: dehumanization (0.58). Tertiary: profanity_insults (0.41)."

This granularity enables entirely different moderation workflows:

Routing: Threats get escalated to law enforcement. Insults go to community moderators. Spam goes to automated removal. Misinformation goes to fact-checkers. You can't route effectively if all you know is "toxic."

Policy Enforcement: Different concepts have different severity thresholds. A platform might tolerate mild insults (0.5 threshold) but have zero tolerance for threats (0.2 threshold). Binary classifiers force a single global threshold.

Transparency: When a user appeals a removal, you can show them why: "Your post was flagged for identity-based hostility." That's actionable feedback. "Your post was flagged as toxic" is opaque and generates more appeals.

Audit Trails: Regulators increasingly demand explainability. "We removed this because our model detected identity-based hostility at 0.72 confidence" is auditable. "We removed this because our model said toxic" is not.

2. Deterministic, Reproducible Results Enable Trust

Every time you send Sentiqra the same text, you get the same result. Same concepts, same confidence scores, same ranking.

This sounds trivial, but it's increasingly rare in an industry moving toward LLM-based moderation. LLMs use sampling—temperature, top-k, nucleus sampling—which means the same input can produce different outputs on different runs.

Determinism matters for:

Appeals: A user can re-run detection on their content and verify the scores. The appeals process becomes transparent instead of adversarial.

Quality Assurance: You can test policy changes against historical content and measure exact deltas. "Lowering the identity_polarisation_hostility threshold from 0.5 to 0.4 would have flagged 347 additional posts last month" is a precise, testable claim.

Regulatory Compliance: Increasingly, regulators want audit trails. Deterministic results mean you can reconstruct exactly why a moderation decision was made six months ago.

3. High Recall Works for First-Pass Filtering

Sentiqra achieves 97.1% recall on Jigsaw at threshold 0.4. It catches almost everything.

For many trust and safety teams, high recall matters more than high precision—especially when you have human review capacity. You want your automated system to catch everything potentially harmful and send it to human moderators. False positives are acceptable because humans will filter them out. False negatives are catastrophic because the harmful content reaches users.

Sentiqra works well as a first-pass filter in a layered moderation pipeline:

1. Sentiqra flags everything potentially harmful (high recall, moderate precision)
2. Human moderators review flagged content (eliminate false positives)
3. Confirmed violations get actioned

This is a fundamentally different use case than "replace human moderation entirely," which is what binary classifiers often promise and rarely deliver.

4. Multilingual Coverage with Consistent Taxonomy

These benchmarks test only English, but Sentiqra supports 14 languages, each evaluated and released independently: English, Spanish, French, German, Italian, Portuguese, Dutch, Russian, Arabic, Hindi, Chinese, Japanese, Korean, Turkish.

The same concept taxonomy works across all languages. An insult in French ("insultes_profanité") maps to the same concept as an insult in Japanese ("insultes_profanité"). This enables:

Cross-Language Policy Consistency: The same threshold policies apply globally. You don't need separate rules for each language.

Multilingual Reporting: Aggregate metrics across languages. "We detected 10,000 instances of identity_polarisation_hostility this week" includes English, Spanish, and Arabic.

Reduced Training Burden: You don't need separate training datasets for each language. Our detection engine handles language-specific semantics.

Binary classifiers typically require separate training for each language, or they collapse to keyword matching in low-resource languages.

Where Sentiqra Loses: The Honest Limitations

We've shown the strengths. Now the uncomfortable part.

1. Binary Classification Performance Is Not Competitive

An F1 score of 62.7% on HateCheck is simply not good by binary classification standards.

Purpose-built binary classifiers—Perspective API, OpenAI Moderation, Azure Content Safety—are trained specifically to answer "is this harmful?" with high accuracy. They will outperform Sentiqra on this task.

We don't have direct comparison numbers yet (that's planned for follow-up work), but based on published benchmarks, we expect purpose-built classifiers to achieve F1 scores in the 80-90% range on HateCheck-like datasets.

If your use case is "I need a simple yes/no answer with maximum accuracy," use a binary classifier. Sentiqra is not the right tool.

2. False Positives on Identity Mentions Remain Unresolved

The 39% accuracy on identity-mentioning non-hate cases is the single biggest weakness.

Text that mentions protected groups in neutral or positive contexts gets flagged:

• "Muslims are great people" → flagged
• "I love the gay community" → flagged
• "Black history month is important" → flagged

This isn't an edge case. This is a fundamental architectural limitation: semantic concept detection without integrated sentiment analysis cannot distinguish positive, neutral, and negative mentions of the same topic.

For platforms with diverse communities discussing identity frequently, this generates massive false positive volumes.

3. Latency Is Production-Prohibitive at Scale

Average latency: 11.8 seconds (p50).

That's orders of magnitude slower than competitors:

• Perspective API: sub-second
• OpenAI Moderation: 1-2 seconds
• Azure Content Safety: 1-3 seconds

Why is Sentiqra so slow? Each request runs the full configured analysis and returns scored signals for the entire indicator set. There is no cheap partial mode—the same work happens every call.

For real-time moderation at scale—thousands of comments per second—this latency is unacceptable without extensive caching or batch processing infrastructure.

4. Confidence Scores Don't Measure Severity

HateCheck and Jigsaw distinguish between "toxic" and "severe_toxic." A death threat is more severe than a mild insult.

Sentiqra's confidence scores measure detection confidence, not severity:

• 0.9 confidence insult = "I'm very confident this is an insult"
• 0.5 confidence threat = "I'm moderately confident this is a threat"

But the 0.5 confidence threat is almost certainly more severe than the 0.9 confidence insult.

Severity requires understanding context, intensity, and target. Sentiqra currently doesn't model this. All it knows is concept presence and confidence.

The Roadmap: What We're Building Next

These benchmarks gave us a brutally clear development roadmap:

1. Sentiment-Aware Identity Detection

We're building a sentiment analysis layer specifically for identity-mentioning text. When Sentiqra detects identity topics, sentiment analysis will determine whether the mention is hostile, positive, or neutral. This should dramatically reduce false positives on cases like "Muslims are great."

2. Per-Signal Policy Thresholds

Today, requests carry a single operating threshold. But threats and insults have different base rates and severity profiles, so we are adding per-signal threshold support:

• Threats: a low operating point (low tolerance, high severity)
• Insults: a moderate operating point
• Profanity: a high operating point (context-dependent)

This will let customers tune recall/precision per harm type instead of globally.

3. Caching and Latency Optimization

We're implementing:

• Detection cache: Store results for previously seen text, reducing API calls
• Near-duplicate detection: Cluster similar content (e.g., "you suck" vs "u suck") to one cached result
• Batch processing mode: Process 1000 comments in one API call instead of 1000 serial calls

Target: Sub-second p95 latency for cached content, sub-5-second for novel content.

4. Severity Scoring Model

We're training a separate severity classifier that takes detected concepts + context and outputs severity levels:

• Low: Mild insults, profanity in non-hostile context
• Medium: Direct insults, derogation
• High: Dehumanization, threats
• Critical: Credible threats, incitement to violence

This severity score will be separate from detection confidence, addressing the gap between "confidence this is a threat" and "how dangerous is this threat."

5. Competitor Benchmarking

We're planning to run Perspective API, OpenAI Moderation, and Azure Content Safety on the exact same HateCheck and Jigsaw samples for direct comparison. We'll publish:

• Head-to-head accuracy metrics
• Latency comparison
• Cost comparison (API pricing)
• Explainability comparison (do they return reason codes?)

This will give an honest answer to "should I use Sentiqra or [competitor]?" instead of forcing you to guess.

The Real Takeaway: Match the Tool to the Job

If you need a binary "is this hate speech?" classifier with maximum accuracy, Sentiqra is not the best choice. Purpose-built binary classifiers will beat it on F1 score.

But if you need to know what kind of harmful content you're dealing with—for routing, policy enforcement, appeals, audit trails, or multilingual coverage—Sentiqra provides something binary classifiers don't: concept-level granularity with deterministic, reproducible scores.

The optimal use case for Sentiqra is not as a standalone binary classifier but as a first-pass concept detector in a layered moderation pipeline:

1. Detection: Sentiqra flags potentially harmful content with high recall (catches almost everything)
2. Routing: Concept labels route content to appropriate reviewers (threats → security team, insults → community mods, spam → automated removal)
3. Review: Human moderators handle flagged content, filtering out false positives
4. Policy: Per-signal thresholds enforce different severity tolerances
5. Audit: Deterministic scores enable appeals and regulatory compliance

This is a fundamentally different architecture than "AI replaces human moderation." It's "AI surfaces what humans need to see, with enough context to route it correctly."

We're publishing these results—F1 of 62.7%, 75% false positive rate at optimal recall, massive identity-mention false positives—not because they're impressive (they're not), but because honesty about limitations builds more trust than cherry-picked wins.

Every content moderation vendor can show you a 95% accuracy number on some carefully chosen test set. Very few will show you where they fail and why. We'd rather you know Sentiqra achieves 62.7% F1 on HateCheck with a clear understanding of where it wins (obfuscated hate, multilingual coverage, concept granularity) and where it loses (identity mentions, binary accuracy, latency) than claim 95% accuracy on an undisclosed benchmark.

Choose the tool that matches your actual use case. If that's Sentiqra, great. If it's not, that's fine too. At least now you know what you're getting.

Methodology Notes and Caveats

Full transparency on limitations:

Sample sizes are small: 200 HateCheck cases and 65 Jigsaw cases are not sufficient for statistical significance. Standard benchmarks use thousands of cases. We're planning to run on the full HateCheck suite (3,728 cases) and a larger Jigsaw sample (10,000+) for more robust statistics.

Jigsaw sample is curated, not random: Our Jigsaw sample was hand-curated to match known category distributions, not randomly sampled from the full dataset. This introduces selection bias—we may have inadvertently selected easier or harder cases than a true random sample.

English only: These results apply only to English text. Sentiqra supports 14 languages, but multilingual benchmarks are planned separately. Performance may differ significantly across languages.

No competitor comparison: We did not run Perspective API, OpenAI Moderation, or Azure Content Safety on the same samples. Without this, we cannot definitively claim Sentiqra is better or worse—only report its standalone performance. Head-to-head comparison is planned for follow-up work.

Latency includes network overhead: The 11.8-second p50 latency includes network round-trip time to our Sentiqra Intelligence. Production deployments with caching or batch processing would see significantly lower latency. This number represents worst-case uncached performance.

Threshold optimization may be overfitted: We tested six thresholds (0.2–0.7) and reported the optimal F1 score. With only 200 HateCheck cases, this optimization may be overfitted to our specific sample. The optimal threshold on the full 3,728-case dataset may differ.

Reproduce This Yourself

All benchmark code and datasets are publicly available in our repository:

Code:

• benchmarks/run_public_benchmarks.py — basic benchmark runner
• benchmarks/enhanced_analysis.py — threshold optimization and category breakdown

Data:

• benchmarks/public_datasets/hatecheck_sample_200.csv — our HateCheck sample (200 cases)
• benchmarks/public_datasets/jigsaw_sample_60.csv — our Jigsaw sample (65 cases)

Results:

• benchmarks/results/enhanced_results.json — complete results with per-case breakdown

To run it yourself:

python benchmarks/run_public_benchmarks.py
python benchmarks/enhanced_analysis.py

We welcome independent reproduction, validation, and comparison against other tools. If you find different results, we want to know.

---

This benchmark was conducted in August 2025 using Sentiqra's production English configuration. Sentiqra provides deterministic detection for Trust & Safety, Cybersecurity, Crisis Intelligence, and Finance Market Intelligence. Full methodology and raw results are available in our public repository.

AI Disclosure: This article was written with AI assistance to improve clarity and narrative flow. All benchmark results, data, and technical claims are independently verifiable through our public repository.