Limits
| Feature | Limit |
|---|---|
| Maximum requests per second, per pipeline | 20,000 default (configurable up to 100,000) |
| Maximum payload per request | 1 MB |
| Maximum data throughput per pipeline | 20 MB / s default (configurable up to 100 MB / s) |
| Maximum batch size | 100 MB |
| Maximum batch records | 10,000,000 |
| Maximum batch duration | 300s |
If you consistently exceed the requests per second or throughput limits, your pipeline might not be able to keep up with the load. The pipeline will communicate backpressure by returning a 429 response to HTTP requests, or throwing an error if using the Workers API.
If you are consistently seeing backpressure from your pipeline, consider the following strategies:
- Increase the shard count, to increase the maxiumum throughput of your pipeline.
- Send data to a second pipeline if you receive an error. You can setup multiple pipelines to write to the same R2 bucket.