RDRYUSLOG DEV
Back to Blog
Python

Python aiohttp: Headers, Authentication, and Cookies

python aiohttp headers authentication and cookies: Learn how to set headers, implement authentication, and manage cookies with aiohttp for robust HTTP clients and serv...

aiohttpHTTP authenticationcookiesheadersPython HTTP client
A diagram showing an aiohttp request with labeled header fields, a key icon for authentication, and a cookie jar icon.

When building HTTP clients with aiohttp, you often need to send custom headers, authenticate with tokens, and maintain cookies across requests. The ClientSession object is the core of this behavior, and understanding how headers, authentication, and cookies interact will save you from subtle bugs in production. This article covers the practical patterns for python aiohttp headers authentication and cookies on both the client and server side.

Setting Headers on aiohttp Requests

Every aiohttp request accepts a headers parameter that overrides or adds to the session's default headers. The simplest way to send a custom header is to pass a dictionary directly to the request method:

import aiohttp
import asyncio

async def fetch_with_header():
    async with aiohttp.ClientSession() as session:
        async with session.get(
            'https://api.example.com/data',
            headers={'X-API-Key': 'my-secret-key'}
        ) as resp:
            return await resp.text()

For repeated requests, define headers at the session level. This avoids repeating the same dictionary on every call and keeps your code DRY:

async def session_with_default_headers():
    headers = {'User-Agent': 'MyApp/1.0', 'Accept': 'application/json'}
    async with aiohttp.ClientSession(headers=headers) as session:
        async with session.get('https://api.example.com/endpoint') as resp:
            print(resp.status)

When you pass headers both to the session and to an individual request, the request-level headers take precedence for keys that overlap. This is useful for overriding a default header for a specific call without mutating the session.

Authentication Patterns: Basic, Bearer, and Custom

Authentication typically involves setting an Authorization header. aiohttp provides a helper for Basic auth, but you can also construct the header manually for Bearer tokens or custom schemes.

Basic Authentication

Use aiohttp.BasicAuth to encode credentials and set the header automatically:

async def basic_auth_example():
    auth = aiohttp.BasicAuth('username', 'password')
    async with aiohttp.ClientSession(auth=auth) as session:
        async with session.get('https://api.example.com/secure') as resp:
            print(resp.status)

The auth parameter is passed to the session, and aiohttp applies it to every request. If you need different credentials for specific requests, pass auth to the request method instead.

Bearer Token Authentication

For token-based APIs, set the Authorization header directly:

async def bearer_auth_example(token):
    headers = {'Authorization': f'Bearer {token}'}
    async with aiohttp.ClientSession(headers=headers) as session:
        async with session.get('https://api.example.com/profile') as resp:
            return await resp.json()

If the token changes frequently, avoid storing it in the session headers. Instead, pass it per request or update the session headers dynamically using session.headers.update(...).

Custom Authentication Schemes

Some APIs use non-standard schemes like ApiKey or X-API-Key. These are just headers, so you can set them like any other header. The key is to keep the authentication logic centralized, perhaps in a helper function that returns the headers for a given request.

Managing Cookies with aiohttp

Cookies are essential for maintaining state across requests, especially for login-based flows. aiohttp's ClientSession automatically stores cookies from responses and sends them on subsequent requests to the same domain.

async def cookie_flow():
    async with aiohttp.ClientSession() as session:
        # First request logs in and receives a session cookie
        async with session.post('https://example.com/login', data={'user': 'a', 'pass': 'b'}) as resp:
            await resp.text()  # Ensure response is read to capture cookies
        # Subsequent requests automatically include the cookie
        async with session.get('https://example.com/dashboard') as resp:
            print(resp.status)

You can also manually set cookies using the cookies parameter of ClientSession:

async def manual_cookie():
    jar = aiohttp.CookieJar()
    jar.update_cookies({'session_id': 'abc123'})
    async with aiohttp.ClientSession(cookie_jar=jar) as session:
        async with session.get('https://example.com/') as resp:
            print(resp.status)

To inspect cookies after a response, access session.cookie_jar. This is useful for debugging or for extracting tokens that the server sets.

Server-Side: Reading Headers and Cookies

On the server side, aiohttp handlers receive a Request object with .headers and .cookies properties. You can access authentication information from the headers and validate it.

from aiohttp import web

async def handler(request):
    auth_header = request.headers.get('Authorization', '')
    if auth_header.startswith('Bearer '):
        token = auth_header[7:]
        # Validate token
    user_cookie = request.cookies.get('session_id')
    return web.json_response({'status': 'ok'})

When building a server, you often need to set cookies in the response. Use web.Response and its set_cookie method:

async def login_handler(request):
    resp = web.Response(text='Logged in')
    resp.set_cookie('session_id', 'new-session-token', max_age=3600, httponly=True)
    return resp

Setting httponly=True prevents JavaScript from accessing the cookie, which is a basic security measure.

Handling Authentication Errors and Redirects

Authentication often fails with 401 Unauthorized or 403 Forbidden. Your client should handle these statuses explicitly rather than assuming success. A common pattern is to retry once with fresh credentials if the token has expired.

async def request_with_retry(session, url, token):
    headers = {'Authorization': f'Bearer {token}'}
    async with session.get(url, headers=headers) as resp:
        if resp.status == 401:
            # Refresh token and retry
            new_token = await refresh_token()
            headers['Authorization'] = f'Bearer {new_token}'
            async with session.get(url, headers=headers) as retry_resp:
                return retry_resp
        return resp

Redirects can also affect authentication. By default, aiohttp follows redirects, and the Authorization header is stripped when redirecting to a different host. This is a security feature, but it can break flows that rely on cross-domain redirects. Use allow_redirects=False if you need to handle redirects manually.

Security Considerations: TLS, Cookie Flags, and Session Reuse

When dealing with authentication and cookies, security is not optional. Always use HTTPS in production to prevent credentials from being intercepted. aiohttp verifies TLS certificates by default; disable verification only for local testing with ssl=False, and never in production.

Cookie flags matter. When setting cookies server-side, use secure=True to send them only over HTTPS, and httponly=True to prevent client-side script access. The samesite attribute can mitigate CSRF attacks. On the client side, be careful when storing cookies from untrusted domains; aiohttp's CookieJar can be configured with a policy, but the default is safe for most use cases.

Session reuse is another concern. If you are using a ClientSession for a long-lived process, periodically rotate the authentication token if the server supports it. Also, avoid logging headers that contain sensitive tokens. Use repr() carefully and redact Authorization and Cookie headers in logs.

Common Pitfalls and Debugging

Several issues commonly trip up developers when working with aiohttp headers, authentication, and cookies.

Header Case and Duplicate Headers

HTTP headers are case-insensitive, but aiohttp normalizes them to lowercase. If you rely on a specific case, use request.headers['authorization'] or request.headers.get('Authorization'); both work because the internal storage is case-insensitive. Duplicate headers are combined into a comma-separated string, which can be surprising for Set-Cookie (though aiohttp handles cookies separately).

Cookies Not Being Sent

If cookies are not sent on subsequent requests, ensure the response body is fully read before the session context exits. aiohttp captures cookies when the response is processed, so await resp.text() or await resp.read() is necessary. Also, check that the domain and path of the cookie match the request URL.

Authentication Header Overwritten

If you set Authorization at the session level and then pass a different Authorization in a request, the request-level value wins. If you need to remove the header for a specific request, set it to an empty string, but be aware that some servers may reject empty values.

Debugging with Trace

For complex issues, enable aiohttp's debug logging to see the actual headers and cookies sent:

import logging
logging.basicConfig(level=logging.DEBUG)

This prints request and response headers, which helps verify that your authentication and cookie logic is working as expected.

Advanced Usage: Custom Cookie Jar and Header Injection

For advanced scenarios, you can subclass aiohttp.CookieJar to implement custom cookie policies, such as ignoring cookies from certain domains. Similarly, you can create a custom TraceConfig to inject headers into every request before it is sent. This is useful for adding correlation IDs or dynamic tokens without repeating code.

from aiohttp import ClientSession, TraceConfig

async def on_request_start(session, trace_config_ctx, params):
    params.headers['X-Correlation-ID'] = 'generated-id'

trace_config = TraceConfig()
trace_config.on_request_start.append(on_request_start)

async def main():
    async with ClientSession(trace_configs=[trace_config]) as session:
        async with session.get('https://example.com') as resp:
            print(resp.status)

This pattern keeps your request logic clean and centralizes cross-cutting concerns like authentication and tracing. Just be careful not to override headers that are set explicitly on the request method, as the trace callback runs before the request is sent and may conflict with per-request headers.

python aiohttp headers authentication and cookies: Practical | RYUSLOG DEV