python pymongo transactions: Practical Usage and Code Exampl

Using python pymongo transactions requires a MongoDB deployment that supports multi-document transactions. In practice, that means a replica set or a sharded cluster whose shards are replica sets. A standalone mongod instance does not support transactions, so the first step is confirming your server topology.

What MongoDB Transactions Require

MongoDB transactions are available starting in version 4.0 for replica sets and 4.2 for sharded clusters. PyMongo exposes transaction support through the session API. The key requirement is that every operation you want to include in a transaction must be executed within the same session.

You also need to consider the storage engine. The WiredTiger storage engine, which is the default since MongoDB 3.2, supports transactions. If your deployment uses an older engine like MMAPv1, you must migrate to WiredTiger before using transactions.

Finally, transactions have a default 60-second timeout. This is configurable via transactionLifetimeLimitSeconds on the server, but you should design your operations to complete quickly to avoid exceeding the limit.

Starting a Session and Transaction

In PyMongo, you create a client, then start a session using client.start_session(). Inside that session, you call session.start_transaction(). All operations that participate in the transaction must be executed with the session argument passed to the collection methods.

from pymongo import MongoClient

client = MongoClient("mongodb://localhost:27017")

with client.start_session() as session:
    with session.start_transaction():
        db = client.test_db
        db.accounts.update_one(
            {"user": "alice"},
            {"$inc": {"balance": -100}},
            session=session,
        )
        db.accounts.update_one(
            {"user": "bob"},
            {"$inc": {"balance": 100}},
            session=session,
        )

The with blocks ensure that the session is closed and the transaction is aborted if an exception occurs. If both updates succeed, the transaction is committed when the with block exits normally. If any operation raises an exception, the transaction is aborted automatically.

Committing and Aborting a Transaction

You can also manage the transaction explicitly instead of relying on the context manager. The start_transaction() method returns a transaction object, and you can call commit_transaction() or abort_transaction() on the session.

session = client.start_session()
session.start_transaction()
try:
    db.accounts.update_one(
        {"user": "alice"},
        {"$inc": {"balance": -100}},
        session=session,
    )
    db.accounts.update_one(
        {"user": "bob"},
        {"$inc": {"balance": 100}},
        session=session,
    )
    session.commit_transaction()
except Exception:
    session.abort_transaction()
finally:
    session.end_session()

When you commit, MongoDB writes all changes atomically. If you abort, none of the operations are applied. Note that commit_transaction() can raise a ConnectionFailure if the primary changes during the commit. In that case, you may need to retry the entire transaction.

Handling Errors and Retry Logic

Transactions can fail for several reasons: transient network issues, write conflicts, or timeouts. PyMongo provides a Transaction class that can be used with a retry loop. The recommended pattern is to catch OperationFailure and check for a TransientTransactionError label.

from pymongo.errors import OperationFailure

def run_transaction_with_retry(session, fn):
    while True:
        try:
            with session.start_transaction():
                fn(session)
            return
        except OperationFailure as exc:
            if exc.has_error_label("TransientTransactionError"):
                continue
            raise

This loop retries the entire transaction when the server indicates a transient error. For commit errors with the UnknownTransactionCommitResult label, you should retry the commit operation itself, not the whole transaction.

def commit_with_retry(session):
    while True:
        try:
            session.commit_transaction()
            return
        except OperationFailure as exc:
            if exc.has_error_label("UnknownTransactionCommitResult"):
                continue
            raise

Transaction Options: Read and Write Concern

You can specify read concern, write concern, and read preference for a transaction. These are set when calling start_transaction(). The default read concern is "snapshot", which gives you a consistent view of the data. Write concern defaults to the cluster's setting.

from pymongo import ReadPreference
from pymongo.write_concern import WriteConcern

with client.start_session() as session:
    with session.start_transaction(
        read_concern={"level": "majority"},
        write_concern=WriteConcern(w="majority", j=True),
        read_preference=ReadPreference.PRIMARY,
    ):
        # operations here

Using majority write concern ensures that the transaction is durable only after the data is replicated to a majority of nodes. This increases safety but adds latency. The read preference must be PRIMARY for transactions; you cannot use secondary reads inside a transaction.

Performance and Operational Considerations

Transactions have overhead compared to single-document operations. They require additional coordination between the server and the client, and they hold locks on the documents involved. To minimize contention, keep transactions short and avoid performing network calls or external I/O inside the transaction.

Another operational concern is the transactionLifetimeLimitSeconds setting. If a transaction exceeds this limit, the server aborts it. You can adjust this value, but doing so affects all clients. Monitoring the currentOp command can help you identify long-running transactions.

Also, transactions are not compatible with all MongoDB features. For example, you cannot create or drop collections inside a transaction. DDL operations like createIndex are also prohibited. The operations allowed are limited to CRUD and a few others.

Common Pitfalls and Limitations

One common mistake is forgetting to pass the session argument to every operation. If an operation does not use the session, it executes outside the transaction, breaking atomicity. Always verify that every insert, update, delete, or find that should be part of the transaction includes session=session.

Another pitfall is using a transaction on a standalone server. The client will raise an OperationFailure with the message "Transaction numbers are only allowed on a replica set member or mongos". Always confirm your deployment topology before writing transaction code.

Finally, be aware that transactions in MongoDB are not isolated at the snapshot level by default if you change the read concern. If you set read_concern to "local", you lose snapshot isolation, which can lead to anomalies. Only use "local" if you understand the consistency implications.

When you need to move data between collections or perform multi-document updates, transactions give you a reliable way to maintain consistency. The key is to use sessions correctly, handle retryable errors, and keep your operations short to avoid contention and timeouts.