Contract sentiqra/prepare/v1

Input preparation

POST /v1/analyze analyzes short text. A document, thread, or conversation needs a transformation first — extraction, normalization, chunking — and that transformation is yours. Sentiqra's reproducibility guarantee covers the exact bytes it received. This page defines a versioned, deterministic contract for everything before that boundary, plus a manifest you keep that records it.

Why an informal "just split it" is not enough

Two customers who split the same document differently produce different requests, different signals, and different receipts — with no way to show the difference came from preparation rather than detection. Arbitrary splitting also loses meaning: a "not" severed from its clause, a quoted line attributed to the wrong speaker, a signature dropped to fit the limit.

Chunk receipts verify the snippet, not the document. A receipt proves what Sentiqra analyzed and under which configuration — for the derived text of that request. It says nothing about pages you never submitted.
Chunk signals are not document conclusions. A signal present in one chunk describes that chunk. Any document-level claim is a customer-side aggregation and must be qualified as such.
The source stays with you. The manifest records source identifiers and offsets so you can bind receipts back to your material — Sentiqra never needs the original document.

The versioned contract

sentiqra-prepare.py implements sentiqra/prepare/v1. Any behavior-affecting change — extraction rules, segmentation, normalization, chunking — requires a new prepare version. Reprocessing the same source under the same version produces byte-identical requests and manifest; any preparation change produces a distinguishable manifest.

Stage	Contract
extract	`plain` (paragraph blocks) or `threaded` (headers, `>` quotes with depth, `Name:` speaker turns, signature delimiter). Rule-set version `b1`.
normalize	NFC — canonical composition, explicitly. Not "clean text"; not NFKC. Compatibility normalization folds distinctions that change meaning (e.g. ligature ﬁ → fi), so it is never applied. Also: CRLF/CR→LF, trailing spaces stripped, blank runs capped at two. Every transformation is counted into the manifest.
segment	Deterministic sentence rules `s1` — boundaries only at sentence-final punctuation, protected abbreviations, decimals, and ellipses. A sentence is atomic: it is never split across chunks, so negations and qualifiers keep their clause.
chunk	Greedy fill up to `--max-chars` code points. One chunk draws from exactly one block: different speakers, quoted material, and own statements never share a chunk. Signatures and quoted replies are included by default; explicit `--drop-*` flags record every omission in `dropped_blocks`.
overlap	`--overlap-sentences N` carries N trailing sentences forward as marked context (`context_sentences`), so a boundary sentence keeps its surroundings. Context sentences are analyzed with the chunk but counted once at aggregation.
language	Explicit only: a `[lang:xx]` directive at a block start wins, else your declared `--language`, else `und` (the request omits `language` and the API default applies). Never inferred silently. `language_source` is recorded per chunk.
aggregate	`sentiqra/prepare-aggregate/v1`: a union of per-chunk signal codes with a `chunk_hits` count, de-duplicated over overlap context. Output is labeled a customer-side statistic — never a detector prediction on the whole document.

Using it

curl -O https://sentiqra.tech/eval-toolkit/sentiqra-prepare.py
python3 sentiqra-prepare.py --selftest   # determinism, boundaries, overlap, multilingual

python3 sentiqra-prepare.py prepare contract.txt \
    --format plain --analyzers emotions \
    --language en --source-id case-0042 \
    --pin {alias} \
    --manifest-out manifest.json --requests-out requests.jsonl

Each requests.jsonl line is the exact body to send for that chunk, keyed by request_sha256. Send them (respecting the API's per-request limits and your quota), then attach what came back:

python3 sentiqra-prepare.py update --manifest manifest.json --receipts receipts.json
python3 sentiqra-prepare.py aggregate --manifest manifest.json --results results.jsonl

receipts.json maps each request_sha256 to the returned analysis_config_alias, receipt_fingerprint, and request_id. A receipt that names a request the manifest never prepared is rejected — the binding stays honest.

The manifest — customer-held, never uploaded

{
  "schema": "sentiqra/prepare-manifest/v1",
  "prepare_version": "1.0.0",
  "contract": {"id": "sentiqra/prepare/v1",
               "normalization": "unicode=NFC; line_endings=LF; …"},
  "source": {"source_id": "case-0042",            // your identifier — stays with you
             "source_sha256": "…",                // binds to your original bytes
             "normalized_sha256": "…",
             "offsets_relative_to": "normalized_source"},
  "chunking": {"max_chars": 4000, "overlap_sentences": 1, …},
  "chunks":  [{"index", "chunk_id", "offset_start", "offset_end",
               "kind", "speaker", "role", "language", "language_source",
               "context_sentences", "status", "request_sha256"}],
  "requests": [{"chunk_id", "request_sha256",
                "analysis_config_alias", "receipt_fingerprint", "request_id"}],
  "dropped_blocks": []                            // every explicit omission, with reason
}

Reproducibility: the same source + version + options → the same manifest and requests. Verify before and after a change to see exactly what differs.
Provenance: offset_start/offset_end locate each chunk inside your normalized source; source_sha256 binds the manifest to material that never left your environment.
Receipts bind to requests: request_sha256 is a canonical digest of the exact JSON sent. A returned receipt verifies that submission — never the original document.
Oversize: a single sentence over the limit becomes skipped_oversize — recorded, never silently truncated or dropped.

Reference architecture

your environment ────────────────────────────────
  source docs ──► sentiqra-prepare.py ──► manifest.json  (keep)
                              └────────► requests.jsonl ──► POST /v1/analyze
                                                              │
                              ◄────────── receipts ◄──────────┘
                  update → manifest  ·  aggregate → document statistic
───────────────────────────────────────────────────────────────
Sentiqra sees: derived request bodies only — not sources, IDs, or manifests

Store the manifest with the source (or its content address). It is your audit link between retained originals and the derived requests Sentiqra signed.
Pin --pin so every chunk runs under the same configuration alias — otherwise per-chunk receipts can resolve to different configurations.
If you implement your own preparer, keep the same rule: version the transform, record the manifest, qualify aggregation.

Explicit non-claims

A chunk receipt or signature verifies the derived snippet Sentiqra received — it does not verify your source document.
Aggregated chunk signals are a customer-side statistic, not a Sentiqra prediction about the whole document.
The library does not call the API, does not upload manifests, and does not send your source_id anywhere — those stay in your environment by construction.
NFC preserves canonical equivalence only; it will not fix extraction errors from PDFs or OCR — inspect normalization.counts and sample chunks.