Python PyYAML: Dictionaries, Lists, and Multiple Documents

python pyyaml dictionaries lists and multiple documents requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When working with Python PyYAML, dictionaries, lists, and multiple documents are the three structures you'll encounter most often. This article covers how to load YAML into native Python types, how to navigate nested data, and how to handle streams that contain more than one document.

Loading YAML into Python Dictionaries and Lists

PyYAML's safe_load is the standard way to parse a single YAML document into Python objects. A YAML mapping becomes a dict, and a YAML sequence becomes a list. For example:

import yaml

data = yaml.safe_load("""
name: api-server
ports:
  - 8080
  - 8081
env:
  LOG_LEVEL: debug
""")

After this call, data is a dictionary with keys name, ports, and env. The ports value is a list of integers, and env is a nested dictionary. PyYAML maps YAML types to Python types automatically: strings, integers, floats, booleans, and None are converted according to the YAML 1.1 specification.

This direct mapping is why PyYAML is often used for configuration files. You can access values with normal dictionary and list indexing, or use dict.get when a key might be missing.

Accessing and Modifying Nested Data

Nested YAML structures produce nested Python collections. Consider a configuration that contains a list of service definitions:

config = yaml.safe_load("""
services:
  - name: auth
    replicas: 2
  - name: billing
    replicas: 3
""")

You can iterate over config["services"] and read each service's fields. Modifying the loaded Python objects does not automatically write back to a file; you would need to dump them with yaml.dump or yaml.safe_dump. If you plan to serialize the data again, keep in mind that PyYAML may reorder dictionary keys. By default, safe_dump sorts keys alphabetically, which can be surprising when you need to preserve insertion order. You can disable sorting with sort_keys=False.

Handling Multiple Documents in One Stream

A single YAML stream can contain several documents separated by a line containing exactly ---. PyYAML provides safe_load_all to iterate over each document:

import yaml

stream = """
name: alpha
---
name: beta
---
name: gamma
"""

for doc in yaml.safe_load_all(stream):
    print(doc["name"])

safe_load_all returns a generator that yields one Python object per document. This is useful when a file contains multiple independent configurations, such as Kubernetes manifests or CI pipeline stages. The generator is lazy, so documents are parsed one at a time. If you need all documents in memory at once, you can wrap it in list().

Why safe_load Is Safer Than load

PyYAML's load function can construct arbitrary Python objects from untrusted YAML, which can lead to code execution. The safe_load function restricts construction to basic Python types: dictionaries, lists, strings, numbers, booleans, and None. For any YAML that is not fully trusted, always use safe_load or safe_load_all. This is not a theoretical concern; crafted YAML can invoke Python constructors when using load. The PyYAML documentation has warned about this for years. If you need to load custom types, use a custom SafeLoader subclass with explicit constructors rather than the default load.

Common Pitfalls with Type Conversion and Keys

YAML's type resolution can produce unexpected Python types. For example, the string "yes" is interpreted as a boolean True in YAML 1.1, which PyYAML follows. If you need a literal string, quote it: "yes". Similarly, keys in YAML mappings are converted to Python keys; if a key looks like a number, it becomes an integer. This can cause subtle bugs when you expect all keys to be strings. You can force string keys by quoting them in the YAML source.

Another pitfall is duplicate keys. PyYAML does not raise an error for duplicate keys in a mapping; the last value silently overwrites earlier ones. If you need to detect duplicates, you must parse the YAML with a custom loader that tracks seen keys.

Performance and Memory Considerations for Large Files

safe_load reads the entire stream and builds the complete Python object graph. For large configuration files, this can consume significant memory. If you only need a subset of the data, consider streaming with safe_load_all if the file is split into documents, or use a lower-level parser like yaml.compose to inspect the node tree without constructing all Python objects. For very large files, PyYAML's pure Python implementation is slower than the C-based _yaml extension, which is used automatically when available. You can check yaml.__with_libyaml__ to see whether the C extension is active. If performance matters, ensure the extension is installed.

Using Custom Constructors for Complex Types

When your YAML contains custom tags, such as !timestamp, you can extend the SafeLoader to construct specific Python types. This keeps the safety of safe_load while allowing controlled deserialization. For example:

import datetime
import yaml

class TimestampLoader(yaml.SafeLoader):
    pass

def construct_timestamp(loader, node):
    return datetime.datetime.fromisoformat(loader.construct_scalar(node))

TimestampLoader.add_constructor("!timestamp", construct_timestamp)

data = yaml.load(stream, Loader=TimestampLoader)

This approach is safer than using the default load because you explicitly allow only the tags you define. It also makes the expected data types visible in one place, which improves maintainability when the YAML schema evolves.