codekingpro/portable-devtools

1List objects in an Amazon S3 bucket2^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^3 4The following example shows how to use an Amazon S3 bucket resource to list5the objects in the bucket.6 7.. code-block:: python8 9    import boto310 11    s3 = boto3.resource('s3')12    bucket = s3.Bucket('my-bucket')13    for obj in bucket.objects.all():14        print(obj.key)15 16 17List top-level common prefixes in Amazon S3 bucket18^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^19 20This example shows how to list all of the top-level common prefixes in an21Amazon S3 bucket:22 23.. code-block:: python24 25    import boto326 27    client = boto3.client('s3')28    paginator = client.get_paginator('list_objects')29    result = paginator.paginate(Bucket='my-bucket', Delimiter='/')30    for prefix in result.search('CommonPrefixes'):31        print(prefix.get('Prefix'))32 33 34Restore Glacier objects in an Amazon S3 bucket35^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^36 37The following example shows how to initiate restoration of glacier objects in38an Amazon S3 bucket, determine if a restoration is on-going, and determine if a39restoration is finished.40 41.. code-block:: python42 43    import boto344 45    s3 = boto3.resource('s3')46    bucket = s3.Bucket('glacier-bucket')47    for obj_sum in bucket.objects.all():48        obj = s3.Object(obj_sum.bucket_name, obj_sum.key)49        if obj.storage_class == 'GLACIER':50            # Try to restore the object if the storage class is glacier and51            # the object does not have a completed or ongoing restoration52            # request.53            if obj.restore is None:54                print('Submitting restoration request: %s' % obj.key)55                obj.restore_object(RestoreRequest={'Days': 1})56            # Print out objects whose restoration is on-going57            elif 'ongoing-request="true"' in obj.restore:58                print('Restoration in-progress: %s' % obj.key)59            # Print out objects whose restoration is complete60            elif 'ongoing-request="false"' in obj.restore:61                print('Restoration complete: %s' % obj.key)62 63 64Uploading/downloading files using SSE KMS65^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^66 67This example shows how to use SSE-KMS to upload objects using68server side encryption with a key managed by KMS.69 70We can either use the default KMS master key, or create a71custom key in AWS and use it to encrypt the object by passing in its72key id.73 74With KMS, nothing else needs to be provided for getting the75object; S3 already knows how to decrypt the object.76 77 78.. code-block:: python79 80    import boto381    import os82 83    BUCKET = 'your-bucket-name'84    s3 = boto3.client('s3')85    keyid = '<the key id>'86 87    print("Uploading S3 object with SSE-KMS")88    s3.put_object(Bucket=BUCKET,89                  Key='encrypt-key',90                  Body=b'foobar',91                  ServerSideEncryption='aws:kms',92                  # Optional: SSEKMSKeyId93                  SSEKMSKeyId=keyid)94    print("Done")95 96    # Getting the object:97    print("Getting S3 object...")98    response = s3.get_object(Bucket=BUCKET,99                             Key='encrypt-key')100    print("Done, response body:")101    print(response['Body'].read())102 103 104Uploading/downloading files using SSE Customer Keys105^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^106 107This example shows how to use SSE-C to upload objects using108server side encryption with a customer provided key.109 110First, we'll need a 32 byte key.  For this example, we'll111randomly generate a key but you can use any 32 byte key112you want.  Remember, you must the same key to download113the object.  If you lose the encryption key, you lose114the object.115 116Also note how we don't have to provide the SSECustomerKeyMD5.117Boto3 will automatically compute this value for us.118 119 120.. code-block:: python121 122    import boto3123    import os124 125    BUCKET = 'your-bucket-name'126    KEY = os.urandom(32)127    s3 = boto3.client('s3')128 129    print("Uploading S3 object with SSE-C")130    s3.put_object(Bucket=BUCKET,131                  Key='encrypt-key',132                  Body=b'foobar',133                  SSECustomerKey=KEY,134                  SSECustomerAlgorithm='AES256')135    print("Done")136 137    # Getting the object:138    print("Getting S3 object...")139    # Note how we're using the same ``KEY`` we140    # created earlier.141    response = s3.get_object(Bucket=BUCKET,142                             Key='encrypt-key',143                             SSECustomerKey=KEY,144                             SSECustomerAlgorithm='AES256')145    print("Done, response body:")146    print(response['Body'].read())147 148 149Downloading a specific version of an S3 object150^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^151 152This example shows how to download a specific version of an153S3 object.154 155.. code-block:: python156 157    import boto3158    s3 = boto3.client('s3')159 160    s3.download_file(161        "bucket-name", "key-name", "tmp.txt",162        ExtraArgs={"VersionId": "my-version-id"}163    )164 165 166Filter objects by last modified time using JMESPath167^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^168 169This example shows how to filter objects by last modified time170using JMESPath.171 172.. code-block:: python173 174    import boto3175    s3 = boto3.client("s3")176 177    s3_paginator = s3.get_paginator('list_objects_v2')178    s3_iterator = s3_paginator.paginate(Bucket='your-bucket-name')179 180    filtered_iterator = s3_iterator.search(181        "Contents[?to_string(LastModified)>='\"2022-01-05 08:05:37+00:00\"'].Key"182    )183 184    for key_data in filtered_iterator:185        print(key_data)186