Python __new__ Metaclass: Class Creation Explained

The python **new** metaclass pattern is a common source of confusion because __new__ in a metaclass operates at a different stage than __new__ in a regular class. When you define a metaclass, you can override __new__ to intercept the creation of a class object itself, before the class body is fully initialized. This gives you a hook to modify the class's attributes, methods, or even its bases before it becomes usable.

What __new__ Does in a Metaclass

In a metaclass, __new__ is responsible for creating the class object. The signature is:

class Meta(type):
    def __new__(mcls, name, bases, namespace, **kwargs):
        cls = super().__new__(mcls, name, bases, namespace)
        return cls

Here mcls is the metaclass itself, name is the class name, bases is a tuple of base classes, and namespace is the class body's namespace (usually a dict). The **kwargs are any extra keyword arguments passed in the class definition, such as class MyClass(metaclass=Meta, some_option=True).

__new__ returns the newly created class object. If you don't override it, type.__new__ does the work. When you override it, you can inspect or modify the namespace before passing it to super().__new__(), or you can even return a completely different object.

How Class Creation Works with __new__ and __init__

When Python executes a class definition, it follows a specific order:

1. The class body is executed, populating the namespace.
2. The metaclass's __new__ is called to create the class object.
3. The metaclass's __init__ is called to initialize the class object, if __new__ returns an instance of the metaclass.

This means __new__ runs before __init__. If __new__ does not return an instance of the metaclass, __init__ is not called. This separation is useful when you need to construct the class object itself, rather than just configure it after creation.

A Minimal Metaclass Using __new__

Here's a minimal example that adds a class attribute automatically:

class AutoAttributeMeta(type):
    def __new__(mcls, name, bases, namespace, **kwargs):
        namespace.setdefault('created_by', 'AutoAttributeMeta')
        return super().__new__(mcls, name, bases, namespace)

class MyClass(metaclass=AutoAttributeMeta):
    pass

print(MyClass.created_by)  # AutoAttributeMeta

In this example, __new__ modifies the namespace dict before the class is created. The setdefault call ensures the attribute is only added if it doesn't already exist. This is a clean way to inject defaults without touching the class body.

Using __new__ to Modify the Class Before Creation

Because __new__ receives the namespace as a mutable dict, you can perform more complex transformations. For example, you might want to wrap all methods with a decorator, or enforce naming conventions:

class UpperMethodMeta(type):
    def __new__(mcls, name, bases, namespace, **kwargs):
        for key, value in list(namespace.items()):
            if callable(value) and not key.startswith('__'):
                namespace[key] = lambda *args, _f=value, **kw: _f(*args, **kw) * 2
        return super().__new__(mcls, name, bases, namespace)

class Math(metaclass=UpperMethodMeta):
    def double(self, x):
        return x * 2

m = Math()
print(m.double(3))  # 12, because the method is wrapped

This is a contrived example, but it shows that __new__ can alter the class's methods before the class exists. The key is that you are working with the raw namespace dict, so any changes you make are reflected in the final class.

Common Mistakes and Pitfalls

One common mistake is forgetting to call super().__new__() and returning a non-class object. If you return a plain object, Python will not create a class, and __init__ will not run. This can lead to confusing errors.

Another pitfall is relying on __new__ to modify the class after it has been created. If you need to set attributes that depend on the class object itself, use __init__ instead. For example, computing a class-level property that references the class is better done in __init__ because the class already exists there.

Also, be careful with keyword arguments. If you pass extra keyword arguments in the class definition, you must handle them in __new__ or __init__. Otherwise, type.__new__ will raise a TypeError because it doesn't accept arbitrary **kwargs.

When to Use __new__ Instead of __init__ in a Metaclass

Use __new__ when you need to:

• Modify the namespace before the class object is constructed.
• Change the bases or the metaclass itself.
• Return an object that is not an instance of the metaclass (rare).

Use __init__ when you need to configure the class after it exists, such as setting attributes that depend on the class object or validating the class's structure.

In practice, most metaclass customization can be done in __init__, but __new__ is essential when you need to alter the class body before it is turned into a class. For example, a metaclass that automatically registers subclasses might use __init__ to add the class to a registry, while a metaclass that rewrites method signatures would need __new__.

Compatibility and Maintainability Considerations

Metaclasses are a powerful metaprogramming tool, but they add complexity. Overriding __new__ in a metaclass makes the code harder to follow because the class creation flow is no longer standard. Future maintainers must understand the metaclass to reason about the class's behavior.

Performance is rarely a concern because metaclass __new__ runs only once per class definition, not per instance. However, if your metaclass performs heavy processing in __new__, it can slow down module import time. Keep the logic minimal and avoid side effects that depend on external state.

Compatibility across Python versions is generally stable, but be aware that the exact behavior of namespace may differ in edge cases, such as when using __slots__ or when the class body contains annotations. Always test your metaclass with the Python versions you support.

A final practical note: if you find yourself using __new__ in a metaclass, consider whether a class decorator or a simpler factory function would achieve the same result with less magic. Metaclasses are appropriate when you need to apply behavior across many unrelated classes, but for a single class, a decorator is often clearer.