Batches and files
Submit up to 50,000 chat completion requests at once, inline or as a JSONL file, then poll for status and fetch the results.
On this page
Batches run many chat completions asynchronously. Use them for bulk work such as re-scoring a backlog of findings.
/api/v1/llm/v1/files file). Maximum 50 MiB by default.Scope: llm:batch, llm:chat or llm:access
/api/v1/llm/v1/batches requests or an input_file_id./api/v1/llm/v1/batches limit 1 to 100, default 20)./api/v1/llm/v1/batches/{batch_id} /api/v1/llm/v1/batches/{batch_id}/cancel /api/v1/llm/v1/batches/{batch_id}/results /api/v1/llm/v1/files/{file_id}/content Batches need an organization-bound user and the llm:batch, llm:chat or llm:access scope. They belong to the user and organization that created them: any key or console session of that user in that organization can see and cancel them, and anyone else gets 404.
Input format
Each line is one request to /v1/chat/completions:
{"custom_id": "finding-101", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "nebula-4.5", "messages": [{"role": "user", "content": "Rate the severity of: open redirect on /logout"}]}}
{"custom_id": "finding-102", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "nebula-4.5", "messages": [{"role": "user", "content": "Rate the severity of: stack trace on 500 page"}]}}
A batch can hold up to 50,000 requests.
Run a batch
batch_input = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=batch_input.id,
endpoint="/v1/chat/completions",
completion_window="24h",
)
print(batch.id, batch.status)
Poll GET /v1/batches/{id} until status is final, then fetch the results.
| Status | Meaning |
|---|---|
pending |
Accepted, not started. |
running |
In progress. request_counts shows completed and failed lines. |
done |
Finished. Results are ready. |
failed |
The batch could not run. |
expired |
It did not finish within its completion window. |
canceled |
You cancelled it. |
Results and billing
GET /v1/batches/{id}/results returns one entry per request, keyed by your custom_id.
- A batch is charged once, at the full Nebula price, the first time its results are fetched in a final state. There is no batch discount.
- If the charge has not settled yet, the endpoint returns
503withRetry-After: 30. Retry after the delay.