boto
Mocking boto3 S3 client method Python
Botocore has a client stubber you can use for just this purpose: docs. Here’s an example of putting an error in: import boto3 from botocore.stub import Stubber client = boto3.client(‘s3’) stubber = Stubber(client) stubber.add_client_error(‘upload_part_copy’) stubber.activate() # Will raise a ClientError client.upload_part_copy() Here’s an example of putting a normal response in. Additionally, the stubber can now … Read more
How to save S3 object to a file using boto3
There is a customization that went into Boto3 recently which helps with this (among other things). It is currently exposed on the low-level S3 client, and can be used like this: s3_client = boto3.client(‘s3’) open(‘hello.txt’).write(‘Hello, world!’) # Upload the file to S3 s3_client.upload_file(‘hello.txt’, ‘MyBucket’, ‘hello-remote.txt’) # Download the file from S3 s3_client.download_file(‘MyBucket’, ‘hello-remote.txt’, ‘hello2.txt’) print(open(‘hello2.txt’).read()) … Read more
How to upload a file to directory in S3 bucket using boto
NOTE: This answer uses boto. See the other answer that uses boto3, which is newer. Try this… import boto import boto.s3 import sys from boto.s3.key import Key AWS_ACCESS_KEY_ID = ” AWS_SECRET_ACCESS_KEY = ” bucket_name = AWS_ACCESS_KEY_ID.lower() + ‘-dump’ conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) bucket = conn.create_bucket(bucket_name, location=boto.s3.connection.Location.DEFAULT) testfile = “replace this with an actual filename” print … Read more
Amazon S3 boto – how to create a folder?
There is no concept of folders or directories in S3. You can create file names like “abc/xys/uvw/123.jpg”, which many S3 access tools like S3Fox show like a directory structure, but it’s actually just a single file in a bucket.
What is the difference between the AWS boto and boto3
The boto package is the hand-coded Python library that has been around since 2006. It is very popular and is fully supported by AWS but because it is hand-coded and there are so many services available (with more appearing all the time) it is difficult to maintain. So, boto3 is a new version of the … Read more
How to write a file or data to an S3 object using boto3
In boto 3, the ‘Key.set_contents_from_’ methods were replaced by Object.put() Client.put_object() For example: import boto3 some_binary_data = b’Here we have some data’ more_binary_data = b’Here we have some more data’ # Method 1: Object.put() s3 = boto3.resource(‘s3’) object = s3.Object(‘my_bucket_name’, ‘my/key/including/filename.txt’) object.put(Body=some_binary_data) # Method 2: Client.put_object() client = boto3.client(‘s3′) client.put_object(Body=more_binary_data, Bucket=”my_bucket_name”, Key=’my/key/including/anotherfilename.txt’) Alternatively, the binary … Read more
Boto3 Error: botocore.exceptions.NoCredentialsError: Unable to locate credentials
try specifying keys manually s3 = boto3.resource(‘s3′, aws_access_key_id=ACCESS_ID, aws_secret_access_key= ACCESS_KEY) Make sure you don’t include your ACCESS_ID and ACCESS_KEY in the code directly for security concerns. Consider using environment configs and injecting them in the code as suggested by @Tiger_Mike. For Prod environments consider using rotating access keys: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_RotateAccessKey