in HOWTOs

Multithreading boto3

If you see this error in your multithreaded boto3 python script:

Error:
ValueError: Unknown component: endpoint_resolver

then you might be using boto3 incorrectly for threading.

By default, boto3 uses the default session, which is not thread safe if you have more than one thread.

The fix is to ensure each thread starts its own session.

All explained quite well with a great example in the Boto3 Documentation:
https://boto3.readthedocs.org/en/latest/guide/resources.html#multithreading

In short, each thread should do it’s own:

session = boto3.session.Session()
s3 = session.resource('s3')

Write a Comment

Comment