BotHub Launches Batch API With 45% Lower Token Costs

We've launched Batch API in BotHub, bringing asynchronous batch processing to leading AI models from OpenAI, Anthropic, and Google.The new feature lets you process thousands of AI requests in the background while reducing token costs by 45%. It's designed for developers, businesses, and teams handling large-scale workloads that don't require immediate responses.

What Is Batch API?

Batch API allows you to submit thousands of AI requests in a single file instead of sending them individually and waiting for each response.We process these requests asynchronously and provide an output file with the results once the batch job is complete.Our implementation is fully compatible with the OpenAI Batch API specification, allowing you to use official SDKs without changing how you prepare your requests.

How Does It Work?

Getting started takes three steps:

Prepare your requests: сreate a .jsonl file containing your prompts, with one Chat Completions API request per line.
Submit a batch job: upload the file and start processing through the API.
Retrieve your results: once processing is complete, download the output file containing the responses.

Each batch job has a processing window of up to 24 hours. Actual completion times may vary depending on the provider and workload.

Key Benefits

45% Lower Token Costs

We offer a 45% discount on batch processing with supported OpenAI, Anthropic, and Google models.This helps make large-scale AI workloads more affordable, especially for businesses processing substantial volumes of text.

Fewer Rate-Limit Bottlenecks

Traditional API workflows can encounter requests-per-minute limits when processing large datasets.With Batch API, you can submit large workloads for background processing without relying on the same synchronous request flow.We don't impose additional batch-processing limits beyond those enforced by the underlying model providers.

Simplified Request Management

Batch processing reduces the need to maintain individual connections, manage concurrent requests, and handle timeouts across large workloads.Instead, you can submit a batch job and retrieve the results when processing is complete, reducing the amount of custom infrastructure your application needs.

Use Cases

Batch API is particularly useful for businesses, developers, data analysts, SEO specialists, and content teams working with large datasets.Common use cases include:

E-commerce: generate or translate tens of thousands of product descriptions.
SEO and content production: create SEO articles and generate meta tags at scale.
Data extraction and classification: extract structured information from text and categorize customer reviews.
Document processing: summarize large collections of documents and call transcripts.
AI development: prepare, organize, and label datasets for training AI models.

Batch processing is best suited for workloads where cost efficiency and scalability matter more than real-time responses.

Availability and Limitations

Batch API is available in BotHub for supported models from OpenAI, Anthropic, and Google.A few things to keep in mind:

Batch jobs are processed asynchronously within a window of up to 24 hours.
Provider-specific processing limits still apply.
Model availability depends on batch-processing support from the underlying providers.

You can retrieve the current list of supported models dynamically through the API.

Get Started

Ready to integrate Batch API into your workflow? Explore the documentation and start processing large-scale AI workloads at 45% lower token costs Our Batch API documentation includes setup instructions and code examples in JavaScript, Python, and cURL.