Python async with vs with: Choosing the Right Context Manager

python async with vs with requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When writing asynchronous Python code, you'll encounter both with and async with. The choice affects whether your context manager's setup and teardown can await operations. This article explains the difference and when to use each, focusing on the practical implications for asyncio-based applications.

What with Does in Python

The with statement invokes the __enter__ and __exit__ methods of a context manager. These methods are synchronous by definition. When you write:

with open('file.txt') as f:
    data = f.read()

Python calls open(...).__enter__() before the block and __exit__() after it. Both methods run synchronously. If the context manager performs I/O, such as opening a file or acquiring a lock, that I/O blocks the calling thread. In an asyncio application, blocking the thread also blocks the event loop, preventing other coroutines from running.

What async with Adds

The async with statement is designed for asynchronous context managers. It calls __aenter__ and __aexit__, which are coroutine functions. This allows the setup and teardown to await on I/O operations without blocking the event loop. For example:

import asyncio

class AsyncResource:
    async def __aenter__(self):
        await asyncio.sleep(1)  # Simulate async setup
        return self

    async def __aexit__(self, exc_type, exc, tb):
        await asyncio.sleep(0.5)  # Simulate async cleanup

async def main():
    async with AsyncResource() as res:
        print('Using resource')

asyncio.run(main())

Here, __aenter__ and __aexit__ are coroutines, so the event loop remains responsive during the sleep operations.

When to Use with in Async Code

You can use with inside an async function as long as the context manager's __enter__ and __exit__ do not perform blocking I/O. For example, a simple in-memory lock or a context manager that just sets a flag is fine:

import threading

lock = threading.Lock()

async def task():
    with lock:
        # Critical section
        await asyncio.sleep(0.1)

Even though lock is a threading lock, its __enter__ and __exit__ are non-blocking if the lock is uncontended. However, if the lock is contended, with lock will block the event loop while waiting. In such cases, consider using an asyncio lock (asyncio.Lock) with async with.

The rule of thumb: use with when the context manager's setup and teardown are synchronous and fast. If they involve any I/O, network calls, or other operations that would block, you need async with.

When async with Is Required

async with is required when the context manager implements __aenter__ and __aexit__ instead of __enter__ and __exit__. Many asyncio-aware libraries provide async context managers. For example, an HTTP client session in aiohttp:

import aiohttp
import asyncio

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

Trying to use with on an async context manager raises TypeError: 'async with' requires an object with __aenter__ and __aexit__ methods. Conversely, using async with on a synchronous context manager raises a similar error because it lacks __aenter__.

Runtime Behavior: Blocking the Event Loop

The most critical difference is how each affects the asyncio event loop. When you use with inside a coroutine, the __enter__ and __exit__ methods execute synchronously. If they perform blocking I/O, the entire event loop stalls. This defeats the purpose of async programming.

Consider a file read using the standard open() inside an async function:

async def read_file():
    with open('large_file.txt') as f:
        data = f.read()  # Blocks the event loop
    return data

While read_file is running, no other coroutine can execute. If you have multiple concurrent tasks, they all wait until the file read completes. To avoid this, use an async file library like aiofiles:

import aiofiles
import asyncio

async def read_file_async():
    async with aiofiles.open('large_file.txt') as f:
        data = await f.read()
    return data

nHere, aiofiles provides an async context manager that yields control to the event loop during I/O.

Error Handling Differences

The exception handling behavior differs subtly. With with, if __enter__ raises an exception, __exit__ is not called. The same applies to async with: if __aenter__ raises, __aexit__ is not called. However, because __aenter__ is a coroutine, an exception inside it may be raised at the await point, which is handled by the event loop. You should catch exceptions around the async with block as you would with a regular with.

Another difference is how exceptions propagate from the body. Both __exit__ and __aexit__ receive the exception type, value, and traceback. They can suppress exceptions by returning True (or True in the async case). The coroutine version must be awaited, but the semantics are identical.

Practical Example: File I/O vs Async HTTP Client

To see the difference in a realistic scenario, consider a function that needs to read a configuration file and then make an HTTP request. Using with for the file read blocks the event loop, but using async with for the HTTP client keeps it responsive:

import asyncio
import aiohttp
import aiofiles

async def process():
    # Blocking file read
    with open('config.json') as f:
        config = f.read()

    # Non-blocking HTTP request
    async with aiohttp.ClientSession() as session:
        async with session.get('https://api.example.com') as resp:
n            data = await resp.json()
    return data

If you have many concurrent process() calls, the file reads will serialize and block the loop. Replacing the file read with aiofiles and async with allows the loop to handle other tasks while the file is being read.

Choosing Between with and async with

The decision depends on the context manager's implementation and the nature of the operation:

Use with when the context manager is synchronous and its setup/teardown are fast or non-blocking. Use async with when the context manager provides async methods, or when you need to await during setup or teardown.

Common Mistakes and How to Avoid Them

A frequent mistake is using with on an async context manager, which raises a TypeError. Always check the library's documentation to see whether it provides __aenter__ or __enter__. Another mistake is using async with on a synchronous context manager, which also fails. If you control the context manager class, you can implement both sets of methods to support both usage patterns, but that is rare.

Another issue is mixing with and async with in the same code path without understanding the blocking implications. Even if a synchronous context manager is fast, if it does I/O, it will block. For high-concurrency applications, prefer async context managers for any I/O-bound resource.

Finally, remember that async with is only valid inside a coroutine or async generator. You cannot use it in a synchronous function. If you need to use an async context manager from synchronous code, you must run the coroutine with asyncio.run() or similar, which creates a new event loop.

When you implement your own context managers, decide whether they need to be async based on whether their setup or teardown involves I/O. If they do, implement __aenter__ and __aexit__ as coroutines. If they don't, keep them synchronous to avoid unnecessary overhead. The key is to match the context manager's behavior to the execution context, ensuring the event loop stays responsive.