python nested dictionary: Practical Usage and Code Examples

When working with a python nested dictionary, you are essentially storing dictionaries inside other dictionaries. This structure is common when handling JSON payloads, configuration files, or hierarchical data. The main challenge is that accessing a deeply nested value requires every intermediate key to exist, otherwise Python raises a KeyError. Understanding how to read, modify, and construct these structures safely is a core skill for any Python developer.

How Nested Dictionaries Are Structured

A nested dictionary is simply a dictionary where some values are themselves dictionaries. For example:

config = {
    "database": {
        "host": "localhost",
        "port": 5432,
        "credentials": {
            "user": "admin",
            "password": "secret"
        }
    },
    "cache": {
        "ttl": 300
    }
}

The outer dictionary has two keys: database and cache. The value for database is another dictionary with its own keys. The credentials key holds yet another dictionary. This nesting can go arbitrarily deep, though practical limits are usually set by readability and data size.

Accessing Values in a Nested Dictionary

The most direct way to access a nested value is to chain square brackets. To get the password from the example above:

password = config["database"]["credentials"]["password"]
print(password)  # secret

Each [] lookup must succeed. If any key is missing, Python raises a KeyError. For instance, config["database"]["hostname"] would fail because hostname is not a key in the database dictionary. This behavior is often undesirable when the data comes from an external source, such as a JSON API, where the shape may vary.

Using `.get()` for Safer Access

The dict.get() method returns a default value instead of raising an error when a key is missing. You can chain .get() calls, but each intermediate result must be a dictionary or None. If you call .get() on None, you get an AttributeError. A common pattern is:

user = config.get("database", {}).get("credentials", {}).get("user", "anonymous")
print(user)  # admin

The second argument {} provides an empty dictionary as a fallback, so the next .get() call works even if the previous key is absent. This avoids exceptions but can become verbose when the nesting is deep.

Updating Values in a Nested Dictionary

Updating a nested value follows the same chained assignment syntax. To change the port:

config["database"]["port"] = 5433

If the intermediate keys exist, the assignment works. If they do not, you get a KeyError. To add a new nested key, you must ensure the intermediate dictionaries exist. For example, to add a timeout key inside database:

if "database" not in config:
    config["database"] = {}
config["database"]["timeout"] = 30

This is safe but requires explicit checks. A more concise approach uses setdefault():

config.setdefault("database", {}).setdefault("timeout", 30)

setdefault() returns the value for the key if it exists, or inserts the provided default and returns it. Chaining setdefault() ensures that intermediate dictionaries are created as needed. This is a clean way to build a nested dictionary incrementally.

Building a Nested Dictionary from Scratch

There are several ways to construct a nested dictionary. The most straightforward is to define it literally, as in the first example. For dynamic construction, you can start with an empty dictionary and add levels as needed.

Using a Loop

Suppose you have a list of user records with fields like name and city. You want to group users by city, and within each city, store a dictionary of user IDs to names.

users = [
    {"id": 1, "name": "Alice", "city": "London"},
    {"id": 2, "name": "Bob", "city": "Paris"},
    {"id": 3, "name": "Carol", "city": "London"},
]

by_city = {}
for user in users:
    city = user["city"]
    if city not in by_city:
        by_city[city] = {}
    by_city[city][user["id"]] = user["name"]

print(by_city)
# {'London': {1: 'Alice', 3: 'Carol'}, 'Paris': {2: 'Bob'}}

The manual if city not in by_city check ensures the inner dictionary exists before adding a key. This pattern is common and easy to read.

Using `defaultdict`

The collections.defaultdict class can simplify this. A defaultdict calls a factory function for missing keys, so you can avoid explicit checks.

from collections import defaultdict

by_city = defaultdict(dict)
for user in users:
    by_city[user["city"]][user["id"]] = user["name"]

print(dict(by_city))

Here defaultdict(dict) creates an empty dictionary for each new city. The inner assignment works without checking. Note that by_city remains a defaultdict, which may be fine, but if you need a plain dictionary, wrap it with dict().

Using a Recursive Function to Navigate Arbitrary Depth

Sometimes the nesting depth is not fixed. For example, you might have a tree-like structure where nodes can contain children. A recursive function can traverse the dictionary safely.

def get_path(data, path):
    """Return the value at a dotted path, or None if missing."""
    current = data
    for key in path.split("."):
        if isinstance(current, dict) and key in current:
            current = current[key]
        else:
            return None
    return current

Usage:

value = get_path(config, "database.credentials.user")
print(value)  # admin

This approach avoids chained .get() calls and handles missing keys gracefully. It also works with any depth. The function checks that each intermediate value is a dictionary before attempting a key lookup, preventing AttributeError on non-dict values.

Merging Nested Dictionaries

Merging two nested dictionaries requires care because a simple update() only merges top-level keys. If you want to merge nested structures recursively, you need a custom function.

def deep_merge(base, override):
    result = base.copy()
    for key, value in override.items():
        if key in result and isinstance(result[key], dict) and isinstance(value, dict):
            result[key] = deep_merge(result[key], value)
        else:
            result[key] = value
    return result

This function recursively merges dictionaries, preserving values from override when keys conflict. For example:

a = {"db": {"host": "localhost", "port": 5432}}
b = {"db": {"port": 5433, "user": "admin"}}
merged = deep_merge(a, b)
print(merged)
# {'db': {'host': 'localhost', 'port': 5433, 'user': 'admin'}}

The port value is overwritten, while host and user are preserved. This is useful when combining configuration files or API responses.

Performance and Memory Considerations

Nested dictionaries are convenient but have some performance implications. Each dictionary lookup is O(1) on average, but accessing a deeply nested value requires multiple lookups. In practice, this is rarely a bottleneck unless you are performing millions of lookups per second.

Memory usage is another concern. Each dictionary has overhead for its internal table, so deeply nested structures with many small dictionaries can consume more memory than a flat dictionary with tuple keys. For example, storing coordinates as {"x": 1, "y": 2} for thousands of points uses more memory than using a tuple (1, 2). If memory is critical, consider a flat dictionary with composite keys like (x, y).

When dealing with large JSON payloads, you might want to avoid copying nested structures. The deep_merge function above copies the base dictionary, which can be expensive for large trees. If you need to mutate in place, modify the function to update base directly.

Common Pitfalls and How to Avoid Them

The most frequent mistake is assuming that a nested key exists. Always validate the structure when the data comes from an untrusted source. Use .get() with a default or a recursive helper.

Another pitfall is using a mutable default value in a function definition. For example:

def add_user(user_db, user_id, name):
    user_db[user_id] = name

This is fine, but if you write def add_user(user_db={}):, the default dictionary is shared across all calls, leading to unexpected behavior. Use None and create a new dictionary inside the function.

Finally, when using defaultdict, be aware that accessing a missing key creates an entry. This can cause silent side effects if you are only checking for existence. Use in or .get() if you do not want to insert.

Handling JSON Data with Nested Dictionaries

Nested dictionaries are the natural representation of JSON in Python. The json module converts JSON objects to dictionaries, preserving nesting. When you parse a JSON response from an API, you often get a nested dictionary. The same access and update patterns apply.

import json

payload = '{"user": {"name": "Alice", "address": {"city": "London"}}}'
data = json.loads(payload)

city = data.get("user", {}).get("address", {}).get("city")
print(city)  # London

When serializing back to JSON, json.dumps() will handle nested dictionaries correctly. Just be aware that dictionary keys must be strings for JSON compatibility. If your dictionary uses integer keys, they will be converted to strings in the output.

When a Nested Dictionary Is Not the Best Choice

Nested dictionaries are not always the ideal data structure. If you need to query the structure frequently by path, a flat dictionary with tuple keys may be more efficient and easier to iterate. For example:

flat = {
    ("database", "host"): "localhost",
    ("database", "port"): 5432,
    ("database", "credentials", "user"): "admin"
}

This allows you to access values with flat[("database", "host")]. It also makes it easy to iterate over all leaf values. However, it loses the hierarchical grouping that nested dictionaries provide, which can make the code less readable.

For tree-like data with variable depth, a custom class or a library like attr or dataclasses might be more appropriate if you need methods and validation. But for most configuration and JSON handling, a python nested dictionary is a straightforward and practical solution.