TRUST & SAFETY
Generally availableCatch the specific policy violation - not a generic toxicity score.
210 policy-specific labels across 16 languages, each returned with the configuration ID and receipt you need to reproduce the call - even months after a user appeals. No generative model is used to produce results.
210 labels · 21 policy families · 16 languages · p95 target <500ms · No generative model is used to produce results
Re-run with the same pinned version → identical result
What this analyzer does and doesn't do
- Detect 210 labeled harmful-content behaviors in text
- Return confidence-ranked signals your policy engine can route on
- Cover 16 languages with per-language evaluation
- ✕Analyze images, video, or audio
- ✕Determine whether content is illegal
- ✕Access user history, thread context, or metadata
- ✕Replace your moderators or make the final call
Sentiqra returns structured signals. Your policy, reviewers, and models make the decision. See the full boundary statement →
WHAT YOU'LL DETECT
Not "toxic: 0.91". The specific behavior - with the label your policy already knows.
Hate & Identity Attack
dehumanization
“You’re a subhuman” → very high
Threats & Intimidation
death_threat
“I hope you don’t wake up tomorrow” → very high
Child Safety & Predatory Language
grooming_language
Detected - example withheld
Coercive & Controlling Abuse
blackmail_threat
“If you don’t send it I’ll…” → very high
Evasion & Coded Abuse
dogwhistle_hate
Detected - example withheld
95.7% – 100% Production Top-3 Hit Rate across 16 languages.
Benchmarks vary by language - the range above is the honest spread, not the best case. Top-3 Hit Rate means the expected label appears among the 3 highest-confidence signals returned; it measures ranking quality, not whether a thresholded accept/reject decision was right.
WHERE IT GETS USED
Three jobs this analyzer does well.
Live chat moderation for gaming platforms
A high-DAU gaming platform routes every in-chat message through Trust & Safety before it reaches the recipient. Because every result is pinned to a config ID, appealed bans can be reviewed against the exact detection that fired - not a re-generated opinion. Appeal reviews stop being arguments about what the model "meant".
Fit signals: high-volume · multilingual · adversarial users · appeals matter
Marketplace listing screening
Every new listing runs through Trust & Safety before publication. Scam, fraud, and impersonation labels route directly to review; hate speech and explicit content get blocked. The moderator queue shrinks to the categories that actually need a human.
Fit signals: pre-publish screening · structured category output · moderator scarcity
Appeals and audit trails for regulated markets
A marketplace subject to DSA obligations stores the config ID and signal signature with every moderation action. When a user appeals or a regulator asks, the exact original analysis is reproducible. No "the model changed" caveats.
Fit signals: DSA/regulator exposure · audit obligations · adversarial disputes
Send text. Get ranked labels + a receipt you can replay.
POST /v1/analyze
Authorization: Bearer sq_live_your_api_key_here
{
"text": "You're a worthless subhuman who deserves to be erased.",
"language": "en",
"analyzers": ["trust-safety"]
}{
"language": "en",
"signals": [
{ "analyzer": "trust-safety", "code": "insults_derogation", "label": "Insults & Derogation", "score": 0.9, "confidence": "very_high", "level": "category" },
{ "analyzer": "trust-safety", "code": "insult", "label": "Insult", "score": 0.9, "confidence": "very_high", "level": "label", "parent": "insults_derogation" },
{ "analyzer": "trust-safety", "code": "belittling", "label": "Belittling", "score": 0.9, "confidence": "very_high", "level": "label", "parent": "insults_derogation" }
],
"meta": { "analyzers": [{ "name": "trust-safety", "status": "completed", "signal_count": 3, "billed": true }], "partial": false },
"usage": { "analyzer_runs": 1, "billed": true },
"verification": {
"configuration": "sqcfg_ZZW77NM9",
"fingerprint": "fp_8a72...19f",
"input_commitment": "ic_31bd...a42",
"signature": "sig_ed25519_4c1f...e2a"
}
}READING THE OUTPUT
What a Trust & Safety result tells you - and what it doesn't.
Intended task
Tell your moderation pipeline which documented harm categories a piece of text matches - the policy routing decision stays with you.
Input requirements
text- the only field analyzed; up to 4,096 Unicode code points. Over the limit is rejected withinput_too_long, never truncated.language- must be a language this analyzer supports. Unsupported codes are rejected with400 unsupported_language, never analyzed as a different language.analyzers- include"trust-safety"; up to 2 analyzers per request, one billed run each.- No context beyond
text- no user IDs, history, profiles, or attachments are read or inferred. - Works on the submitted text alone - the analyzer never sees the author's account, history, or reputation.
Ordinary positive
“You are an idiot and everyone on this server hates you. Keep talking and I will post your home address.”
Safe inference: The text matched two documented harm categories. You can route it to your harassment/doxxing policy lane and cite the exact labels in a statement of reasons.
Confusing negative · Quotation
“The commenter told her "I will post your home address" - our report on doxxing in gaming forums.”
What happens: A privacy/doxxing signal can still fire. The analyzer scores the submitted text as written - quoted abuse language inside reporting or commentary can match the same documented pattern.
Why it's tricky: A signal means "this text contains language matching the documented pattern," not "the author intended harm." Whether quotation context matters is a policy decision - keep human review in the loop for reported-speech surfaces.
What you can safely infer
- Which documented indicators the text matched, and at what signal-match strength.
- That the same input on the same pinned configuration replays identically - the receipt records which configuration produced the result.
- That a category-level signal with label-level children gives you both the policy area and the specific behavior.
What you must not infer
- ✕That the author intended harm - intent is not in the text.
- ✕That quoted or reported abuse means the author endorses it.
- ✕That an empty signals list clears the content - "not detected" is never "safe."
- ✕Anything about the author's account or history - the analyzer never sees it.
Trust & Safety is on every plan. Even Free.
Free
5,000 runs/mo
Build
$149/mo · 50,000 runs/mo
Growth
$349/mo · 150,000 runs/mo
Production
$799/mo · 500,000 runs/mo
Scale
$2,999/mo · 2,500,000 runs/mo
Enterprise
By agreement · SOC 2 Type II in progress
Deterministic by construction, provable by receipt.
A signed verification receipt - configuration ID (sqcfg_...), result fingerprint (fp2_...), input commitment (ic3_...), and Ed25519 signature (sig_ed25519_...) - ships with every successful analysis result. Pin it to the stored decision and a later replay under the same supported configuration and output profile yields the identical canonical result - the difference between "the model said so" and evidence.
Where it fits in your pipeline
Often paired with: Discrimination for protected-class specificity, Cybersecurity for phishing inside messages, Emotion Analysis for escalation scoring.
Used in: Content Moderation and Marketplace & Messaging Safety
Try it on your hardest moderation cases.
5,000 analyzer runs/month · No card required
Analyzer FAQ
New to Sentiqra? It's a deterministic text-analysis API - you send text, you get labeled, confidence-scored signals plus a replayable receipt. Trust & Safety is the harmful language detection analyzer.
What does Trust & Safety detect?
Detect harmful language, harassment, abuse, scams, and safety policy signals in multilingual text.
Which languages does it cover?
16 supported languages, each with a published per-language benchmark. Coverage is per-analyzer, not platform-wide - check the language list on this page before assuming parity with other analyzers.
How do I call it?
POST /v1/analyze with "analyzers": ["trust-safety"]. Same endpoint, auth, and response shape as every other analyzer - you can pair it with one more analyzer in the same request.
What does a call cost?
One analyzer run per call, billed against your plan's included runs (overage per plan). All GA analyzers are on every plan, including Free - see pricing.
How do I evaluate it on my data?
Label a few hundred of your own examples, run them through the free tier, and compare our signals to your labels. The evaluation playbook walks through building that test set - it's the same method we use internally.