Together AI

Together AI chat batches in Batchwork with secure presigned JSONL upload, preprocessing, and normalized result collection.

Together AI uses a specialized file-backed chat flow. Batchwork initiates an upload, sends JSONL to a validated presigned storage URL, preprocesses the file, then creates the batch.

At a glance

Example

from batchwork import BatchRequest, Batchwork

async with Batchwork() as client:
    job = await client.batch(
        model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo",
        requests=[BatchRequest(custom_id="summary", prompt="Summarize this report.")],
    )

togetherai/model is accepted as an alias for together/model.

Lifecycle

1. Requests become body-only JSONL lines: {custom_id, body}.
2. Batchwork initiates /files using multipart metadata with purpose batch-api, filename batchwork.jsonl, and type jsonl.
3. Together returns a file ID and presigned storage URL.
4. Batchwork performs a pinned HTTPS PUT containing the JSONL.
5. /files/{id}/preprocess prepares the upload.
6. /batches creates the chat batch with optional metadata.
7. Output and error files are streamed and normalized.

The JSONL bytes are not sent in the multipart initiation body.

Upload security

The presigned URL must use HTTPS, contain no embedded credentials or control characters, and resolve entirely to globally routable addresses. Batchwork validates DNS, pins the connection to approved addresses while preserving Host/SNI, disables redirects, and does not forward provider credentials, injected auth, cookies, or unrelated client headers to the storage host.

Media

Together chat serialization supports:

• Image URLs and inline images
• Inline PDF data
• UTF-8 text-file data
• Inline WAV and MP3 audio

PDF and audio URLs are not emitted directly; remote sources are generally downloaded and converted to supported inline forms before Together serialization.

Provider options

Known options rename user, reasoning effort, and text verbosity fields. Other Together provider-option keys, including strictJsonSchema, are forwarded unchanged to the wire body, so treat them as trusted provider configuration.

Assistant reasoning is serialized as reasoning_content.

CLI capability contract

Only chat-completions is a valid explicit --endpoint. Together is the sole documented request-root passthrough exception: unrecognized exact keys are forwarded unchanged after validation of protected fields. Treat every passthrough value as trusted provider request configuration.

The reserved keys custom_id, customId, input, messages, model, prompt, and stream always fail. Provider options cannot change identity, model, endpoint, method, authentication, input ownership, or forced non-streaming behavior. Canonical values collide with passthrough frequency_penalty, max_tokens, presence_penalty, seed, stop, temperature, tool_choice, and top_p; specifying both fails. Canonical top_k is unsupported.

Batch metadata is supported and forwarded. Package hard bounds are 50,000 requests, 20 MiB per serialized request, and 200 MiB aggregate, subject to lower Together/model limits.

Results and limitations

Together uses output/error file normalization for text, usage, and item errors. Provider-specific fields remain in result.response.

Batchwork does not expose Together embeddings or image-generation output. Audio described above is text-request input, not an audio-generation or transcription batch API.

Official references

• Together Batch Inference
• Together vision

See Text and messages, Security, and Results.