Using python dataclass slots for memory efficiency
Learn how python dataclass slots reduce memory overhead and speed up attribute access, and understand the tradeoffs and limitations.
The default dataclass stores instance attributes in a per-object dictionary named __dict__. That makes attribute access flexible, but it costs memory and adds a layer of indirection. Setting slots=True on a dataclass changes the generated class to use __slots__, which replaces the dictionary with a fixed set of descriptors. This article explains how python dataclass slots work, what they change at runtime, and where they introduce constraints.
Enabling Slots on a Dataclass
Adding slots to a dataclass is a one-line change. The @dataclass decorator accepts a slots parameter that, when True, generates a class with __slots__ instead of the usual __dict__.
from dataclasses import dataclass @dataclass(slots=True) class Point: x: float y: float
This Point class behaves like a normal dataclass for most purposes: __init__, __repr__, and equality comparisons are still generated. The difference is internal. Instances of Point no longer carry a __dict__; attribute values are stored in fixed slots defined on the class.
What Changes Under the Hood
Without slots=True, a dataclass instance stores its attributes in a dictionary. Each instance has a __dict__ attribute that maps field names to values. That dictionary is flexible—you can add new attributes at runtime—but it also consumes memory even for a small number of fields.
With slots=True, the class gets a __slots__ definition listing the field names. Python then uses descriptors to manage attribute access. Each slot is a fixed storage location on the instance, and there is no __dict__. Attempting to assign an attribute not listed in __slots__ raises an AttributeError. This is the core behavioral difference: the instance is closed to new attributes.
Memory and Performance Effects
The main benefit of using slots is reduced memory usage per instance. A dictionary has overhead for the hash table, keys, and values. Slots store values directly in a compact C-level structure, eliminating that overhead. For programs that create millions of dataclass instances—for example, when processing large datasets or building in-memory caches—the savings can be substantial.
Attribute access also becomes slightly faster. A dictionary lookup involves hashing the key and probing the table. A slot access is a direct descriptor lookup, which is a simpler operation. The difference is small for a single access but can add up in tight loops that read or write fields frequently.
These effects are structural, not micro-optimizations. The exact numbers depend on the Python interpreter, the number of fields, and the access pattern, but the mechanism is consistent: no dictionary means less memory and fewer indirections.
Limitations and Compatibility Issues
Slots come with several limitations that you need to account for.
- No dynamic attributes: Because there is no
__dict__, you cannot add new attributes to an instance after creation. This breaks patterns that rely on attaching temporary state. - No weak references by default: Instances of a class with
__slots__do not support weak references unless you explicitly include__weakref__in the slots list. If you needweakref.ref()on your dataclass instances, you must add that slot manually. - Inheritance complications: If a base class uses slots, derived classes must also define slots or they will get a
__dict__anyway. Mixing slotted and non-slotted classes requires careful design. - Pickling and copying: Some serialization libraries rely on
__dict__to inspect or modify objects. Whilepickleandcopycan work with slotted classes, third-party tools may not handle them correctly without additional support.
Slots with Inheritance
When you inherit from a slotted dataclass, the child class must also define slots=True to avoid reintroducing a __dict__. If the child does not set slots=True, Python will add a __dict__ to the child instances, negating the memory benefit.
@dataclass(slots=True) class Point3D(Point): z: float
In this example, Point3D inherits the __slots__ from Point and adds its own z slot. The combined slots list includes x, y, and z. If you omit slots=True on Point3D, instances will have a __dict__ in addition to the inherited slots, which defeats the purpose.
A more subtle issue arises when a slotted class inherits from a non-slotted class. The non-slotted base provides a __dict__, so the derived slotted class will still have that dictionary. The memory savings only apply to the slots defined in the derived class, not to the base attributes. In practice, it is best to use slots consistently across an entire inheritance hierarchy.
Practical Considerations: When to Use Slots
Use slots=True when you have a well-defined schema, create many instances, and do not need to attach extra attributes. Typical candidates are data transfer objects, configuration records, and value objects in domain models. The memory savings are most noticeable when the number of instances is large.
Avoid slots when you rely on dynamic attribute assignment, use weak references frequently, or work with libraries that introspect __dict__. Also, if your dataclass has many fields and you rarely instantiate more than a few objects, the memory benefit is negligible and the added rigidity is not worth it.
Common Pitfalls and Workarounds
One common pitfall is forgetting that slotted classes cannot have attributes added later. If you need to attach metadata temporarily, consider using a separate dictionary keyed by the object, or use a non-slotted dataclass for that specific case.
For weak reference support, you can add __weakref__ to the slots list manually. The dataclass decorator does not do this automatically, so you must include it explicitly.
from dataclasses import dataclass @dataclass(slots=True) class Node: value: int __weakref__: object = None
This adds a slot for the weak reference pointer, allowing weakref.ref() to work on Node instances. The field is not part of the dataclass fields; it is just a slot declaration.
Another issue is that default values and field ordering still work normally with slots, but you cannot use class-level mutable defaults without field(default_factory=...). This is the same rule as regular dataclasses, not specific to slots.
Finally, remember that slots=True is not compatible with __dict__-based features like vars(instance) or instance.__dict__. If your code relies on those, you will need to refactor or avoid slots.
When you need to choose between a slotted dataclass and a regular one, weigh the memory and access speed benefits against the loss of flexibility. For long-lived processes with many instances, slots are a practical optimization. For short scripts or objects that evolve at runtime, the default dataclass is often the better fit.