Dangerous Behavior & Harm Encouragement

Intended task

Tell your moderation pipeline which documented harm categories a piece of text matches - the policy routing decision stays with you.

Input requirements

• text - the only field analyzed; up to 4,096 Unicode code points. Over the limit is rejected with input_too_long, never truncated.
• language - must be a language this analyzer supports. Unsupported codes are rejected with 400 unsupported_language, never analyzed as a different language.
• analyzers - include "trust-safety"; up to 2 analyzers per request, one billed run each.
• No context beyond text - no user IDs, history, profiles, or attachments are read or inferred.
• Works on the submitted text alone - the analyzer never sees the author's account, history, or reputation.

Ordinary positive

“You are an idiot and everyone on this server hates you. Keep talking and I will post your home address.”

Safe inference: The text matched two documented harm categories. You can route it to your harassment/doxxing policy lane and cite the exact labels in a statement of reasons.

Confusing negative · Quotation

“The commenter told her "I will post your home address" - our report on doxxing in gaming forums.”

What happens: A privacy/doxxing signal can still fire. The analyzer scores the submitted text as written - quoted abuse language inside reporting or commentary can match the same documented pattern.

Why it's tricky: A signal means "this text contains language matching the documented pattern," not "the author intended harm." Whether quotation context matters is a policy decision - keep human review in the loop for reported-speech surfaces.