Python Boto3 Sessions: Credentials and Profiles

Python boto3 sessions credentials and profiles control how your code authenticates to AWS. A boto3 Session is the configuration object that holds the region, profile, and credential resolution logic. Every client or resource you create from that session inherits its settings. Understanding how a session resolves credentials prevents the most common authentication failures in AWS SDK for Python code.

How Boto3 Sessions Relate to Credentials and Profiles

A Session in boto3 is not a network connection. It is a container for configuration and credential state. When you call boto3.client('s3') without creating a session explicitly, boto3 uses a default session. That default session is created from environment variables, shared configuration files, and the current IAM role if you are running on EC2 or ECS.

When you create a client or resource from a session, boto3 uses the session's credential resolution. The credentials may be fetched lazily on the first API call, but you can force resolution with session.get_credentials().

import boto3

session = boto3.Session()
credentials = session.get_credentials()
print(credentials.access_key)

get_credentials() returns a Credentials object with access_key, secret_key, and token. The token is present only when using temporary credentials such as an STS assume role session or an EC2 instance role.

Credential Resolution Order in a Session

Boto3 checks credential sources in a specific order. The first source that provides a complete set of credentials wins. The order is:

1. Environment variables
2. Shared credential file
3. Shared config file
4. ECS container credentials
5. EC2 instance metadata

Environment variables take precedence. The relevant variables are AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. If only the access key and secret key are set, boto3 treats the credentials as long-term and leaves the token empty.

The shared credential file is typically ~/.aws/credentials. The shared config file is typically ~/.aws/config. Both files can contain profile sections. A profile section in the credentials file looks like this:

[default]
aws_access_key_id = AKIA...
aws_secret_access_key = ...

In the config file, the same profile is written as [profile default] or [profile dev]. The profile prefix is required in the config file, but not in the credentials file.

The order matters when the same key exists in multiple places. If AWS_ACCESS_KEY_ID is set in the environment, boto3 will not look at the shared files, even if the profile specifies different credentials.

Using Profiles with Sessions

A profile is a named set of configuration and credential data. You can select a profile when creating a session.

import boto3

session = boto3.Session(profile_name='dev')
s3 = session.client('s3')

The profile_name argument tells boto3 which section to use from the shared files. If the profile is not found, boto3 raises ProfileNotFound.

You can also set the AWS_PROFILE environment variable. When you create a Session() without arguments, boto3 reads AWS_PROFILE and uses that profile. AWS_DEFAULT_PROFILE is an older alias that is still respected when AWS_PROFILE is not set.

A profile can contain more than credentials. In the config file, a profile can define region, output, role_arn, source_profile, sso_start_url, and other settings. The session uses those values for clients created from it.

[profile dev]
region = us-east-1
output = json

When you create a client from a session with profile_name='dev', the client uses the region from that profile unless you pass region_name explicitly.

Creating a Session with Explicit Credentials

Sometimes you need to bypass the provider chain entirely. You can pass credentials directly to the Session constructor.

import boto3

session = boto3.Session(
    aws_access_key_id='AKIA...',
    aws_secret_access_key='...',
    aws_session_token='...',
    region_name='us-west-2'
)

Explicit credentials take precedence over environment variables and profile files. This is useful when credentials come from a vault, a configuration service, or a short-lived token generated at runtime.

If you pass only aws_access_key_id and aws_secret_access_key, boto3 treats them as long-term credentials. If you pass aws_session_token, it treats them as temporary credentials. Do not mix long-term keys with a token from a different session.

Explicit credentials also work with boto3.client directly, but using a session keeps the credentials in one place and makes it easier to create multiple clients with the same identity.

Assume Role Profiles and Source Profiles

A common pattern is to use a profile that assumes an IAM role. The profile in the config file references another profile for the initial authentication.

[profile base]
aws_access_key_id = AKIA...
aws_secret_access_key = ...

[profile admin]
role_arn = arn:aws:iam::123456789012:role/AdminRole
source_profile = base
region = us-east-1

When you create a session with profile_name='admin', boto3 reads the source_profile, loads the base credentials, and calls sts:AssumeRole with the role_arn. The resulting temporary credentials are used for the session.

import boto3

session = boto3.Session(profile_name='admin')
sts = session.client('sts')
identity = sts.get_caller_identity()
print(identity['Arn'])

The get_caller_identity call verifies which role the session assumed. If the role requires an external ID or MFA, the profile must include external_id and mfa_serial. Boto3 will prompt for the MFA token when the profile is used.

Assume role profiles can also chain. A profile can use another assume role profile as its source. Boto3 resolves the chain until it reaches a profile with static credentials or another supported source.

Credential Caching, Refresh, and Thread Safety

Boto3 caches credentials internally. For an assume role profile, the session stores the temporary credentials and reuses them until they expire. When the credentials are near expiration, boto3 refreshes them by calling AssumeRole again. This behavior is handled by the credential provider, so your code does not need to manage the refresh cycle.

For EC2 instance metadata credentials, boto3 also caches and refreshes them. The refresh happens when the cached credentials expire or are about to expire.

Thread safety depends on how you use the session. A Session object is not guaranteed to be thread-safe for concurrent mutation, such as changing region_name or profile_name while another thread creates clients. However, a boto3 client created from a session is thread-safe for making API calls. The common pattern is to create one session per configuration and then create clients from it. If you need different credentials in different threads, create a separate session for each credential set.

session_a = boto3.Session(profile_name='dev')
session_b = boto3.Session(profile_name='prod')

client_a = session_a.client('s3')
client_b = session_b.client('s3')

Sharing client_a across threads is safe for concurrent calls. Sharing the session itself across threads while creating clients is less predictable, so avoid that pattern.

Common Failure Modes and How to Diagnose

Authentication failures usually come from the credential resolution chain, not from the AWS service. The most common errors are:

NoCredentialsError means boto3 could not find any credentials in the chain. Check whether AWS_ACCESS_KEY_ID is set, whether the profile exists, and whether the instance metadata service is reachable.

PartialCredentialsError means boto3 found an access key but no secret key, or vice versa. This often happens when only one environment variable is set.

ProfileNotFound means the profile name does not exist in either the credentials file or the config file. Check for typos and for the [profile ...] prefix in the config file.

ExpiredToken means the temporary credentials in use have expired. This happens when you reuse a session that cached credentials from an assume role call and the role session duration is short. Creating a new session or forcing a refresh resolves it.

InvalidClientTokenId usually means the access key is not valid for the requested service or the key was deleted. This is different from an expired token.

When diagnosing, start by printing the resolved profile and credentials source.

import boto3

session = boto3.Session(profile_name='dev')
print(session.profile_name)
print(session.region_name)
print(session.get_credentials())

The get_credentials() output shows whether a token is present. If the token is missing but the profile uses an assume role, the role assumption failed or the source profile did not resolve.

Choosing the Right Credential Strategy for Your Session

The choice between environment variables, profiles, and explicit credentials depends on where your code runs and how often the credentials change.

Use environment variables when you need a quick override in a local shell or in a CI pipeline. They are simple, but they leak into every process started from that shell, so do not use them for long-lived production workloads.

Use shared profiles when you have multiple AWS environments or when you need to assume different roles. Profiles keep the configuration outside the code and make it easy to switch contexts without editing source files.

Use explicit credentials when credentials are retrieved from a secret manager, a vault, or a runtime source. This keeps the credentials out of the shared files and gives you full control over the credential lifecycle.

For production code running on EC2 or ECS, prefer IAM roles instead of static keys. The session will resolve the role automatically from the instance metadata or container credentials, and the credentials rotate automatically. This removes the need to store secrets in the environment or in files.

When you need to assume a role from a production service, use an assume role profile or call sts.assume_role directly. Direct calls give you more control over session duration and external ID, but require you to handle the refresh cycle yourself. Boto3's profile-based assume role handles refresh for you.

The important rule is to keep the credential source consistent with the session. If you create a session with profile_name, do not also set AWS_ACCESS_KEY_ID in the environment unless you intend the environment variable to win. The resolution order is deterministic, but mixing sources makes debugging harder.