python future object: Practical Usage and Code Examples

When you submit a task to a thread or process pool in Python, you receive a Future object that represents the eventual result of that task. The same concept appears in asyncio for coroutines. Understanding the python future object is essential for writing concurrent code that is both correct and maintainable.

The Future is not the result itself. It is a placeholder that will hold the result once the task completes, or an exception if the task fails. This lets you submit work, continue doing other things, and later ask the future for its outcome. The behavior is consistent across concurrent.futures and asyncio, though there are important differences in how they are created and used.

The Purpose of the Future Object

A future decouples the submission of work from the collection of its result. Instead of blocking until a task finishes, you get a handle immediately. This handle provides methods to check whether the task is done, wait for it, retrieve the result, or attach callbacks that run when the task completes.

Consider a simple example with ThreadPoolExecutor:

from concurrent.futures import ThreadPoolExecutor

def square(n):
    return n * n

with ThreadPoolExecutor() as executor:
    future = executor.submit(square, 5)
    print(future.result())

Here future is a Future instance. Calling result() blocks until the task finishes and returns 25. The future object is what allows the executor to return control to your code immediately after submission.

Creating a Future with concurrent.futures

In concurrent.futures, you rarely create a Future directly. Instead, you obtain one from an executor's submit method or from map (though map returns an iterator of results, not futures). The Future is created internally by the executor and associated with the submitted callable.

You can also create a Future manually using concurrent.futures.Future(), but that is uncommon. Manual creation is useful for testing or when you need to integrate a callback-based API with the executor model.

from concurrent.futures import Future

future = Future()
future.set_result(42)
print(future.result())  # 42

Setting a result manually is allowed, but you must be careful not to set it twice. Attempting to set a result on an already resolved future raises InvalidStateError.

Retrieving Results and Handling Exceptions

The result() method blocks until the future is resolved. You can pass a timeout to avoid waiting indefinitely:

future = executor.submit(slow_task)
try:
    value = future.result(timeout=2)
except TimeoutError:
    print("Task did not finish in time")
except Exception as exc:
    print(f"Task raised: {exc}")

If the callable raises an exception, result() re-raises that exception in the calling thread. This is important: the exception is not lost; it is stored in the future and re-raised when you call result(). You can inspect the exception directly with future.exception() without re-raising it.

future = executor.submit(divide, 1, 0)
if future.exception() is not None:
    print(f"Error: {future.exception()}")

Future States and Callbacks

A future has a state that changes as the task progresses. The states are PENDING, RUNNING, CANCELLED, and FINISHED. You can check whether a future is done with done(), which returns True for both FINISHED and CANCELLED states.

Callbacks are functions that run when the future completes. They are executed in the thread that resolves the future, which is the worker thread in the case of an executor. You add a callback with add_done_callback:

future = executor.submit(square, 7)

def on_done(fut):
    print(f"Result: {fut.result()}")

future.add_done_callback(on_done)

The callback receives the future as its only argument. This is a convenient way to chain operations without blocking the main thread, but it runs in the worker thread, so you must be careful about thread safety when touching shared state.

Differences Between concurrent.futures.Future and asyncio.Future

asyncio has its own Future class, which is designed for event-loop-based concurrency. The core concept is the same, but the API differs in a few ways.

First, asyncio.Future is not thread-safe. It must be used from the same event loop thread, or with careful synchronization if accessed from other threads. concurrent.futures.Future is designed to be used across threads.

Second, asyncio.Future integrates with await. You can await an asyncio.Future directly, which suspends the coroutine until the future is resolved. concurrent.futures.Future is not awaitable.

Third, asyncio.Future has an async-friendly way to add callbacks: add_done_callback still works, but you often use await instead of callbacks for cleaner code.

import asyncio

async def main():
    loop = asyncio.get_running_loop()
    future = loop.create_future()
    loop.call_soon(future.set_result, 10)
    result = await future
    print(result)  # 10

asyncio.run(main())

You rarely create asyncio.Future manually. Instead, you get it from asyncio.ensure_future or from awaiting a coroutine. The asyncio.Task class is a subclass of Future and is what you actually get when you schedule a coroutine with asyncio.create_task.

Cancelling a Future and Timeout Behavior

Both future types support cancellation. For concurrent.futures.Future, calling cancel() returns True if the task was not yet started and was successfully cancelled. If the task is already running, cancel() returns False and the task continues.

future = executor.submit(slow_task)
cancelled = future.cancel()
print(cancelled)  # False if already running

For asyncio.Future, cancellation is more cooperative. You call future.cancel() to request cancellation. This sets the future to CANCELLED and schedules a CancelledError to be thrown into the awaiting coroutine. The coroutine can catch it and perform cleanup, or re-raise it.

Timeout handling also differs. With concurrent.futures, you pass a timeout to result(). With asyncio, you use asyncio.wait_for to wrap a coroutine or future with a timeout.

async def main():
    try:
        result = await asyncio.wait_for(slow_coro(), timeout=2)
    except asyncio.TimeoutError:
        print("Timed out")

Using Futures with ThreadPoolExecutor and ProcessPoolExecutor

The most common way to get a Future is through an executor. ThreadPoolExecutor uses threads, while ProcessPoolExecutor uses separate processes. The future object behaves the same in both cases, but there are important differences in what can be passed to the worker.

With ThreadPoolExecutor, the callable and its arguments must be picklable only if you use ProcessPoolExecutor. Threads share memory, so objects are passed by reference. Processes require pickling, so the callable, arguments, and return value must be picklable.

from concurrent.futures import ProcessPoolExecutor

def compute(x):
    return x ** 2

with ProcessPoolExecutor() as executor:
    future = executor.submit(compute, 4)
    print(future.result())  # 16

This limitation affects how you design tasks for process pools. You cannot pass lambdas or local functions easily because they are not picklable. You need to use module-level functions or objects that support pickling.

Common Pitfalls and Maintainability Considerations

One common mistake is calling result() on a future inside a callback that was added to the same future. This can deadlock if the callback runs in the same thread that is waiting for the result. In concurrent.futures, callbacks run in the worker thread, so calling result() on the same future inside the callback will raise InvalidStateError because the future is already finished.

Another pitfall is ignoring exceptions. If you never call result() on a future that raised an exception, the exception is silently swallowed. This can hide bugs. Always ensure you either call result() or attach a callback that inspects the exception.

When using asyncio.Future, a common issue is forgetting to await the future. If you create a task and never await it, the task may be garbage collected and cancelled. Use asyncio.create_task and keep a reference to the task, or use asyncio.gather to manage multiple futures.

For maintainability, prefer asyncio.gather over manually adding callbacks when you need to run several coroutines concurrently. It provides a single future that completes when all inputs complete, and it propagates the first exception by default.

async def main():
    results = await asyncio.gather(coro1(), coro2(), coro3())

Finally, be aware that concurrent.futures.Future is not designed for very fine-grained coordination. For complex dependency graphs, consider using asyncio or a dedicated library. The future object is a low-level primitive; it gives you a handle on a single asynchronous operation, not a full workflow engine.

Understanding the python future object means knowing when to use it and when to use higher-level abstractions. In most production code, you will interact with futures indirectly through executor.submit or asyncio.gather. But when you need fine control over cancellation, timeouts, or callbacks, the future object gives you that control without forcing you to manage threads or event loops directly.