Rate limits

Rate limits control how frequently users can make requests to our LLM API within specific time periods. Understanding and working within these limits is essential for optimal API usage.

​

1. Understanding Rate Limits

​

What are Rate Limits?

Rate limits restrict the number of API requests that can be made within defined time periods. They help:

• Prevent API abuse and misuse;
• Ensure fair resource distribution among users;
• Maintain consistent API performance and reliability;
• Protect the stability of our services.

​

Default Rate Limits

Every account has a default rate limit for model calls, with RPM limiting the number of requests per model per minute.

More details (RPM):

​

2. Handling Rate Limits

​

How to Monitor Rate Limits?

When you exceed the rate limit, the API will return:

• HTTP Status Code: 429 (Too Many Requests);
• A rate limit exceeded message in the response body.

​

Best Practices

To avoid hitting rate limits:

1. Implement request throttling in your application;
2. Add exponential backoff for retries;
3. Monitor your API usage patterns.

​

When You Hit Rate Limits

If you receive a 429 error, you can:

1. Retry Later: Wait a short period before retrying your request;
2. Optimize Requests: Reduce request frequency;
3. Rate Limits Increase: For higher rate limits, you can:
   • Contact us through Discord
   • or Book a call with our sales team

If you have any questions, please reach out to us on Discord.