Python Hash Function: Usage and Customization

When you store an object as a key in a Python dictionary or add it to a set, Python calls that object's hash() function to compute an integer bucket index. The built-in hash() function returns an integer based on the object's identity or value, and its behavior determines how fast lookups, inserts, and membership tests work. This article explains how python hash function behaves, how to implement custom hashing for your own classes, and where the built-in function is not appropriate.

How the Built-in hash() Works

hash() is a built-in function that takes an object and returns an integer. For most immutable built-in types, the hash is derived from the object's value. For example, hash(42) returns 42, and hash("hello") returns an integer that is stable within a single Python process but varies across runs due to hash randomization. The randomization applies to strings, bytes, and some other types to prevent denial-of-service attacks that exploit predictable hash collisions.

The hash value is used internally by dictionaries and sets to place entries in a table. When you look up a key, Python computes its hash, finds the bucket, and then checks equality only for objects in that bucket. This means that two objects that compare equal must have the same hash value, otherwise the container cannot find them consistently.

print(hash(42))       # 42
print(hash("hello"))  # e.g., 1234567890 (varies between runs)

Because hash() is deterministic within a process, you can rely on it for the lifetime of that process. However, you should not persist hash values or use them across processes, because the randomization seed changes.

The Relationship Between hash() and Equality

Python's data model requires that if a == b is True, then hash(a) == hash(b) must also be True. This invariant is what allows dictionaries and sets to function correctly. When you insert an object, Python stores its hash. When you look up a key, it computes the hash and then checks equality only among objects with the same hash. If two equal objects have different hashes, the container will treat them as distinct, breaking the expected behavior.

For immutable built-in types like int, str, and tuple, this invariant holds by construction. For custom classes, you are responsible for maintaining it. If you override __eq__ without overriding __hash__, Python sets __hash__ to None, making the object unhashable. This is a deliberate safety measure: if equality is value-based, the default identity-based hash would violate the invariant for equal objects.

Implementing Custom hash and eq

To make instances of your class usable as dictionary keys or set members, you must define both __hash__ and __eq__. The hash should be computed from the same attributes that define equality. A common pattern is to create a tuple of those attributes and call hash() on it.

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __eq__(self, other):
        return isinstance(other, Point) and self.x == other.x and self.y == other.y

    def __hash__(self):
        return hash((self.x, self.y))

Now two Point instances with the same coordinates compare equal and produce the same hash, so they can be used interchangeably as dictionary keys.

p1 = Point(1, 2)
p2 = Point(1, 2)
d = {p1: "origin"}
print(d[p2])  # "origin"

If you define __eq__ but not __hash__, the class becomes unhashable. If you define __hash__ but not __eq__, the default identity-based equality means two distinct objects with the same hash are not equal, which is allowed but often not what you want for value objects.

Mutable Objects and Hashability

A critical rule is that hashable objects must be immutable. If an object's hash changes after it has been inserted into a dictionary or set, the container will not be able to find it. For example, if you use a list as a key, Python raises a TypeError because lists are mutable and unhashable. The same logic applies to custom classes: if you allow attributes that participate in __hash__ to change, you break the contract.

Consider a class with a mutable attribute that is part of the hash:

class BadKey:
    def __init__(self, value):
        self.value = value

    def __hash__(self):
        return hash(self.value)

    def __eq__(self, other):
        return isinstance(other, BadKey) and self.value == other.value

If you mutate value after inserting the object into a dictionary, the hash changes, and the dictionary will no longer locate the key. This leads to subtle bugs. To avoid this, either make the class immutable (e.g., use namedtuple or frozen dataclass) or ensure that the hashed attributes never change after the object is used as a key.

Hash Collisions and Performance

Hash collisions occur when two distinct objects produce the same hash value. Python's dictionary and set implementations handle collisions by comparing keys with == within the same bucket. A high collision rate degrades performance because the container must perform more equality checks. For built-in types, Python uses a randomized hash function that spreads values well, but custom hashes can be poorly distributed.

A common mistake is to return a constant from __hash__, such as return 1. This is valid but causes all objects to land in the same bucket, turning lookups into linear scans. For small collections this may be acceptable, but for large ones it defeats the purpose of hashing. A better approach is to combine the hashes of the object's fields using a tuple, as shown earlier. Python's hash() on a tuple already combines the element hashes in a way that is designed to produce good distribution.

If you need a custom combination for performance reasons, you can use bitwise XOR or multiplication, but the tuple approach is usually sufficient and less error-prone.

Hash Randomization and Security

Python enables hash randomization by default for strings and bytes. The environment variable PYTHONHASHSEED controls the random seed. If you set it to a fixed integer, hash values become predictable across runs. This is sometimes needed for reproducible debugging, but it weakens protection against denial-of-service attacks that exploit collision attacks. In production, leave the seed random unless you have a specific reason to fix it.

The randomization applies only to certain built-in types, not to custom objects. If your custom __hash__ relies on strings, those strings are randomized, so the resulting hash is also randomized across runs. This is fine as long as you do not persist hash values.

When to Use hashlib Instead of hash()

The built-in hash() is not a cryptographic hash. It is designed for use in hash tables and makes no guarantees about collision resistance or reversibility. If you need a secure hash for password storage, message digests, or integrity checks, use the hashlib module, which provides algorithms like SHA-256.

import hashlib

def sha256_digest(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()

hashlib functions are deterministic across processes and platforms, and they are designed to be one-way and collision-resistant. Do not use hash() for security purposes; its output is randomized and not guaranteed to be stable even within a single process if the object's type is subject to randomization.

Practical Considerations for Production Code

When designing classes that will be used as dictionary keys, prefer immutable data structures like namedtuple or frozen dataclasses. These automatically generate correct __hash__ and __eq__ methods based on fields, saving you from manual implementation errors.

from dataclasses import dataclass

@dataclass(frozen=True)
class Point:
    x: int
    y: int

This Point is hashable and immutable. If you need to customize the hash, you can still override __hash__, but the frozen dataclass ensures that the object cannot be mutated after creation.

Another production concern is that hash values should not be stored in a database or used as a primary key, because they are not guaranteed to be stable across Python versions or process runs. Use a cryptographic hash from hashlib for persistent identifiers.

Finally, be aware that the performance of dictionary lookups depends on the quality of your hash function. If you have a class with many fields, hashing a tuple of all fields is usually fast enough. If you observe performance issues, profile your code before optimizing the hash; the bottleneck is often elsewhere.