Python Cyclic References: How to Detect and Break Them

Python cyclic references occur when two or more objects hold references to each other, forming a cycle. In CPython, the reference counter cannot collect these cycles by itself, so the cyclic garbage collector must step in. Understanding how these cycles form and how the collector handles them is essential for avoiding memory leaks in long-running Python applications.

What Creates a Cyclic Reference

A cyclic reference appears when objects reference each other directly or indirectly. The simplest case is two objects that point to each other:

class Node:
    def __init__(self, name):
        self.name = name
        self.other = None

a = Node("a")
b = Node("b")
a.other = b
b.other = a

Now a references b and b references a. The reference count of each object is at least 1 because the other object holds a reference. When you delete the external references a and b, the objects still reference each other, so their reference counts never drop to zero. Without a cyclic garbage collector, these objects would leak.

Cycles can also form through containers like lists, dictionaries, or sets, and through objects that hold references in attributes. Even indirect cycles through a chain of objects are possible.

How CPython's Garbage Collector Handles Cycles

CPython uses two mechanisms for memory management: reference counting and a cyclic garbage collector. The reference counter frees objects immediately when their reference count reaches zero. The cyclic collector runs periodically and finds groups of objects that reference each other but are not reachable from the outside.

The collector uses a generational approach. It tracks objects in three generations, with new objects in generation 0. When a generation fills up, the collector runs on that generation and promotes surviving objects to the next generation. The threshold for each generation can be adjusted with gc.set_threshold().

The collector identifies cycles by computing the set of objects that are reachable from a root set. Objects that are part of a cycle and not reachable from any root are collected. This process works for most container types and objects with __dict__, but there are important exceptions.

When Cyclic References Cause Memory Leaks

The cyclic collector is good at finding unreachable cycles, but it cannot collect cycles that contain objects with __del__ methods. When a cycle includes a finalizer, the collector cannot determine a safe order to run the finalizers, so it leaves the entire cycle in an uncollectable state. This is a classic source of memory leaks.

Consider a class that defines __del__:

class Resource:
    def __del__(self):
        print("Cleaning up", self.name)

If two Resource objects reference each other and become unreachable, the collector will not collect them because it cannot decide which __del__ to call first. The objects remain in memory, and the finalizers never run.

Another problematic pattern is holding references to objects that are expensive to create, such as database connections or large caches, inside a cycle. Even if the collector eventually collects the cycle, the objects remain alive longer than necessary, increasing memory pressure.

Detecting Cyclic References with the gc Module

The gc module provides tools to inspect and control the collector. You can list all tracked objects and identify cycles.

gc.get_objects() returns a list of all objects tracked by the collector. You can filter for objects of a specific type, but this can be slow in production. gc.is_tracked(obj) tells you whether an object is being tracked.

To find cycles, you can use gc.get_referrers(obj) and gc.get_referents(obj) to walk the reference graph. A simple debugging approach is to run the collector with gc.collect() and then check gc.garbage for uncollectable objects.

import gc

gc.collect()
print(gc.garbage)  # list of uncollectable objects

If gc.garbage contains objects after a collection, you have cycles with finalizers or other uncollectable patterns.

For a more systematic check, you can temporarily disable the collector and use gc.get_objects() to find objects that are still alive but have no external references. This is a common technique in memory profiling.

Breaking Cycles with weakref

The weakref module lets you create weak references to objects. A weak reference does not increase the object's reference count, so it does not keep the object alive. When the object is garbage collected, the weak reference simply returns None when called.

Using weak references is the standard way to break cycles without changing the design of your classes. For example, in a parent-child relationship, the parent can hold a strong reference to the child, but the child should hold a weak reference to the parent.

import weakref

class Parent:
    def __init__(self):
        self.children = []

    def add_child(self, child):
        self.children.append(child)
        child.parent = weakref.ref(self)

class Child:
    def __init__(self, name):
        self.name = name
        self.parent = None

Now the child does not keep the parent alive. When the parent is no longer referenced externally, it can be collected even if the child still exists.

Weak references are also useful for caches and callbacks. A callback that holds a strong reference to an object can prevent that object from being collected. Using weakref.ref or weakref.WeakValueDictionary avoids this problem.

Performance and Runtime Considerations

The cyclic collector adds overhead to every allocation that creates a tracked object. The more objects you create, the more often the collector runs. In performance-critical code, you can adjust the thresholds or disable the collector entirely, but doing so requires that you are certain your code does not create cycles.

gc.disable() stops the automatic collection. This can improve performance in short-lived scripts, but it risks memory leaks if cycles are created. You can manually call gc.collect() at safe points, such as after a large batch of work.

The gc.set_threshold(threshold0, threshold1, threshold2) method controls how often each generation is collected. Increasing the thresholds reduces collection frequency but allows more garbage to accumulate. Decreasing them makes collection more aggressive but increases overhead.

For long-running services, it is usually better to keep the collector enabled and rely on weak references to prevent cycles from forming in the first place.

Common Pitfalls with Finalizers and Cycles

The interaction between __del__ and cycles is a frequent source of bugs. If you must use __del__, avoid creating cycles that include the object. If a cycle is unavoidable, consider using weakref.finalize instead of __del__. The finalize mechanism registers a callback that runs when the object is collected, and it handles cycles more gracefully.

import weakref

class Resource:
    def __init__(self, name):
        self.name = name
        weakref.finalize(self, self._cleanup)

    def _cleanup(self):
        print("Cleaning up", self.name)

Because finalize does not rely on the object's __del__, the collector can process cycles that include the object without being blocked.

Another common mistake is storing a strong reference to an object in a global cache without ever removing it. Even if the object is part of a cycle, the cache keeps it alive. Using weakref.WeakValueDictionary for caches prevents this.

Designing Classes to Avoid Unnecessary Cycles

The best way to deal with cyclic references is to avoid creating them when possible. In many cases, you can restructure your object graph to use one-directional references. For example, a tree can have children that know their parent, but the parent does not need to reference the children if you store them in a list. The children can hold a weak reference to the parent.

When a cycle is inherent to the domain model, use weak references for the back-reference. This is common in observer patterns, where subjects hold a list of observers, and observers hold a weak reference to the subject to avoid keeping it alive.

In practice, you should profile your application's memory usage and use gc.get_objects() to identify unexpected cycles. The tracemalloc module can also help track allocations, but it does not directly show reference cycles.

The key takeaway is that cyclic references are not inherently bad, but they require the garbage collector to work. By understanding when cycles form and using weak references appropriately, you can keep your Python applications memory-efficient and predictable.