AWS throttling exception fix using exponential back off

Throttling Exceptions in AWS and ways to resolve them

Implement exponential back off for AWS services using boto3

Image for post
Photo by Max Chen on Unsplash

In various situations, we may have to make repeated calls to any AWS service, say SageMaker via boto3 SDK. For instance, you may need to get the number of jupyter notebooks available every 20 seconds.

Here’s how you could create a throttling exception:

import boto3client = boto3.client('sagemaker',region_name = UR_REGION)
for i in range(200):
print(" The retry number is {} ".format(i))
response = client.list_notebook_instances()
print(response)
Output: The retry number is 0
{'NotebookInstances': [{'NotebookInstanceName': 'dfdasf'... }
The retry number is 1
{'NotebookInstances': [{'NotebookInstanceName': 'dfdasf'...}
The retry number is 2
{'NotebookInstances': [{'NotebookInstanceName': 'dfdasf'...}
.
.
.

The very first time the throttling exception is raised approximately between 50 and 70 retrials and the number of retries reduces gradually on consecutive executions which in turn increases the number of requests again.

API calls to any AWS service can’t exceed the maximum allowed API request rate per account and per AWS Region. It doesn’t matter if the calls come from an application, the AWS Command Line Interface (AWS CLI), or the AWS Management Console. If API requests exceed the maximum rate, you receive a “Rate Exceeded” error, and API calls are then throttled.

Not only this stalls the operations via API but also any operation to that particular AWS service temporarily.

To resolve this, we may need to do the following

  1. Reduce the rate at which the API hits an AWS service
  2. Implement exponential back off

Each AWS SDK implements automatic retry logic and exponential backoff algorithms.

import boto3
from botocore.config import Config
# the default max_attempts is set to 4config = Config(retries=dict(max_attempts=10))
client = boto3.client('sagemaker', config=config)
for i in range(200):
print(" The retry number is {} ".format(i))
response = client.list_notebook_instances()
print(response)

After increasing the maximum attempts to 10(it could be a value of your choice based on the number of retries) the throttling exception is raised after ~190 retries. However, performance has to be given a thought while increasing the number of retries. The consecutive new requests may get delayed based on the number of retries for an individual request.

References:

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html

https://aws.amazon.com/blogs/messaging-and-targeting/how-to-handle-a-throttling-maximum-sending-rate-exceeded-error/

Python Developer, AWS certified solutions architect associate | CSM | Django | Flask | www.linkedin.com/in/dineshkumarkb

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store