S3 Direct Access

You can retrieve temporary S3 credentials at the /s3credentials endpoint when authenticated via Earthdata Login. These credentials will only be valid for 1 hour due to role chaining and can be used make in-region s3:ListBucket and s3:GetObject requests. Your code must handle expired tokens and request new ones as needed for sessions that exceed this 1 hour limit.

Request

Credentials are retrieved through an HTTP GET request to the /s3credentials endpoint. The request must be authenticated with either a JWT token for TEA or by using EDL Bearer Tokens. Unauthenticated requests will be redirected to EDL.

Headers:

(optional) app-name: An arbitrary string to include in the generated role session name for metric reporting purposes. It can only contain characters that are valid in a RoleSessionName see the AssumeRole documentation. It is recommended to include this header when making requests on users behalf from another cloud service. The generated role session name is username@app-name.

Example:

import requests

resp = requests.get(
    "https://your-tea-host/s3credentials",
    headers={"app-name": "my-application"},
    cookies={"asf-urs": "<your jwt token>"}
)
print(resp.json())

Response

The response is your temporary credentials as returned by Amazon STS. See the AWS Credentials reference for more details.

Example:

{
    "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
    "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "sessionToken": "LONGSTRINGOFCHARACTERS.../HJLgV91QJFCMlmY8slIEOjrOChLQYmzAqrb5U1ekoQAK6f86HKJFTT2dONzPgmJN9ZvW5DBwt6XUxC9HAQ0LDPEYEwbjGVKkzSNQh/",
    "expiration": "2021-01-27 00:50:09+00:00"
}

Using Temporary Credentials

To use the credentials you must configure your AWS SDK library with the returned access key, secret and token. Note that the credentials are only valid for in-region requests, so using them with your AWS CLI will not work! You must make your requests from an AWS service such as Lambda or EC2 in the same region as the source bucket you are pulling from. See Using temporary credentials with AWS resources for more information on how to use your temporary credentials.

Example:

This example lambda function uses EDL Bearer Tokens to obtain s3 credentials and stream an object from one bucket to another. The lambda execution role will need s3:PutObject permissions on the destination bucket.

import boto3
import json
import urllib.request


def lambda_handler(event, context):
    # Get temporary download credentials
    tea_url = event["CredentialsEndpoint"]
    bearer_token = event["BearerToken"]
    req = urllib.request.Request(
        url=tea_url,
        headers={"Authorization": f"Bearer {bearer_token}"}
    )
    with urllib.request.urlopen(req) as f:
        creds = json.loads(f.read().decode())

    # Set up separate boto3 clients for download and upload
    download_client = boto3.client(
        "s3",
        aws_access_key_id=creds["accessKeyId"],
        aws_secret_access_key=creds["secretAccessKey"],
        aws_session_token=creds["sessionToken"]
    )
    # Lambda needs to have permission to upload to destination bucket
    upload_client = boto3.client("s3")

    # Stream from the source bucket to the destination bucket
    resp = download_client.get_object(
        Bucket=event["Source"]["Bucket"],
        Key=event["Source"]["Key"],
    )
    upload_client.upload_fileobj(
        resp["Body"],
        event["Dest"]["Bucket"],
        event["Dest"].get("Key") or event["Source"]["Key"],
    )

The example can be invoked with an event payload as follows:

{
    "CredentialsEndpoint": "https://your-tea-host/s3credentials",
    "BearerToken": "your bearer token",
    "Source": {
        "Bucket": "S3 bucket name from CMR link",
        "Key": "S3 key from CMR link"
    },
    "Dest": {
        "Bucket": "S3 bucket name to copy to"
    }
}