Python orjson: Serializing Dataclasses with datetime, numpy, and pydantic

When you need to serialize a Python dataclass that contains datetime objects or numpy arrays, the standard json module often falls short. orjson steps in as a fast, correct JSON library that natively understands dataclasses, datetime, and numpy types. This article covers how to combine python orjson datetime dataclass numpy and pydantic in a practical way, including how to use orjson with pydantic models for validation and serialization.

Why orjson for Dataclass Serialization?

The built-in json module requires you to write a custom default function for every non-serializable type. Dataclasses are not serializable by default, datetime objects raise a TypeError, and numpy arrays are completely unsupported. orjson eliminates most of that boilerplate by providing built-in support for these types through its OPT_SERIALIZE_DATACLASS and OPT_SERIALIZE_NUMPY options. It also serializes datetime objects to ISO 8601 strings without a custom encoder.

For a typical API response or event payload that includes a dataclass with a timestamp and a numeric vector, orjson reduces the serialization code to a single function call. This matters in performance-sensitive paths where the standard library's json module becomes a bottleneck.

Serializing Dataclasses with orjson

orjson does not serialize dataclasses by default. You must enable the OPT_SERIALIZE_DATACLASS flag when calling orjson.dumps(). Consider this dataclass:

from dataclasses import dataclass
from datetime import datetime
import numpy as np
import orjson

@dataclass
class SensorReading:
    sensor_id: str
    timestamp: datetime
    values: np.ndarray

To serialize it, pass the option:

reading = SensorReading(
    sensor_id="temp-01",
    timestamp=datetime(2025, 3, 14, 12, 30, 0),
    values=np.array([21.5, 22.1, 21.9]),
)

json_bytes = orjson.dumps(reading, option=orjson.OPT_SERIALIZE_DATACLASS)

The output is a bytes object containing JSON. The dataclass fields are serialized recursively. Nested dataclasses, lists, and dictionaries also work as long as the same option is set. If you forget the flag, orjson raises a TypeError because it does not know how to handle the dataclass instance.

Handling datetime Objects

orjson serializes datetime, date, and time objects to ISO 8601 strings by default. This behavior is consistent and does not require any special option. For the SensorReading example, the timestamp field becomes "2025-03-14T12:30:00".

If you need a different format, such as a Unix timestamp, you must provide a custom default function. orjson calls this function only for types it does not already recognize. For example:

def default(obj):
    if isinstance(obj, datetime):
        return obj.timestamp()
    raise TypeError

json_bytes = orjson.dumps(reading, option=orjson.OPT_SERIALIZE_DATACLASS, default=default)

Note that default is not called for dataclass fields that orjson can handle natively. In this case, the timestamp is a datetime, which orjson normally serializes as ISO. To override it, you must also disable the default ISO handling. A simpler approach is to convert the datetime to a numeric value before serialization, or to use a custom serializer for the entire dataclass.

Serializing numpy Arrays and Scalars

numpy arrays are not JSON-serializable by default. orjson provides the OPT_SERIALIZE_NUMPY flag to convert arrays to lists, and it also handles numpy scalar types like np.float64 and np.int64. Enable both dataclass and numpy options together:

json_bytes = orjson.dumps(
    reading,
    option=orjson.OPT_SERIALIZE_DATACLASS | orjson.OPT_SERIALIZE_NUMPY,
)

The values field becomes [21.5, 22.1, 21.9]. orjson converts multidimensional arrays to nested lists. This is often the expected JSON representation for numerical data, but it can be memory-intensive for large arrays. If you need to preserve the array's shape or dtype, you must implement a custom encoding, such as base64-encoding the raw bytes.

orjson also serializes numpy scalars without the numpy option? Actually, it does require OPT_SERIALIZE_NUMPY for scalars as well. Without it, a np.float64 raises a TypeError. So always include the flag when your data contains any numpy type.

Combining orjson with pydantic Models

Pydantic v2 models have their own JSON serialization via model_dump_json(), which uses the standard library by default. You can replace that with orjson to gain speed and consistent handling of datetime and numpy types. The cleanest way is to override the model's model_dump_json method or use a custom encoder.

For example, define a pydantic model that includes a datetime field and a numpy array field. Pydantic v2 does not natively support numpy types unless you add a validator. A common pattern is to store the array as a list in the model and convert it to numpy when needed. But if you want to serialize a model that already contains a numpy array, you can use orjson directly on the model's __dict__ or use a custom serializer.

A more practical approach is to use pydantic for validation and then convert the model to a dataclass or dictionary before calling orjson.dumps. For instance:

from pydantic import BaseModel

class ReadingModel(BaseModel):
    sensor_id: str
    timestamp: datetime
    values: list[float]

model = ReadingModel(
    sensor_id="temp-01",
    timestamp=datetime(2025, 3, 14, 12, 30, 0),
    values=[21.5, 22.1, 21.9],
)

json_bytes = orjson.dumps(model.model_dump())

Here model_dump() returns a plain dictionary, which orjson serializes without any special options. This works well when you control the model fields and do not need to serialize raw numpy objects.

If you must serialize a pydantic model that contains a numpy array, you can add a custom field serializer or use a default function with orjson that converts the model to a dictionary. For example:

def orjson_default(obj):
    if isinstance(obj, BaseModel):
        return obj.model_dump()
    if isinstance(obj, np.ndarray):
        return obj.tolist()
    raise TypeError

json_bytes = orjson.dumps(model, default=orjson_default)

This approach gives you pydantic's validation and orjson's speed, but it bypasses pydantic's own serialization logic. For most use cases, this is acceptable because you are only serializing the validated data.

Performance and Runtime Considerations

orjson is significantly faster than the standard json module for typical workloads, especially when serializing dataclasses and numpy arrays. The performance gain comes from a C-based encoder and optimized type handling. However, the exact speedup depends on the data shape and the options you enable.

Enabling OPT_SERIALIZE_DATACLASS and OPT_SERIALIZE_NUMPY adds minimal overhead because orjson inspects types directly. The main cost is converting numpy arrays to Python lists, which allocates new objects. If you serialize large arrays frequently, consider whether the JSON output needs to be a list or if a more compact binary format would be better.

When using orjson with pydantic, avoid calling model_dump_json() and then parsing it back to a dictionary. Instead, call model_dump() and pass the result to orjson.dumps(). This avoids double serialization and reduces CPU usage.

Another runtime consideration is that orjson.dumps() returns bytes, not a string. If your code expects a str, you must decode it: json_str = json_bytes.decode('utf-8'). This is a common source of subtle bugs when integrating with frameworks that expect strings.

Edge Cases and Error Handling

orjson raises TypeError for types it cannot serialize, even when a default function is provided. The default function must handle all unsupported types or raise TypeError itself. If you forget to enable OPT_SERIALIZE_DATACLASS, you get a TypeError that is not always intuitive. Always test with a small sample before deploying.

datetime objects with timezone information are serialized with the offset, e.g., "2025-03-14T12:30:00+00:00". Naive datetimes have no offset. This is consistent with ISO 8601 but may differ from your API's expected format. If you need a specific timezone, normalize the datetime before serialization.

numpy arrays with dtype=object or structured dtypes may not serialize correctly. orjson handles numeric and string arrays well, but object arrays can raise errors. In those cases, convert the array to a list of Python objects first.

Pydantic models with custom validators may produce fields that are not JSON-serializable by orjson if they are not standard types. The default function approach works, but it can become complex when models contain nested models. A cleaner alternative is to use pydantic's model_dump(mode='json'), which converts all fields to JSON-compatible types, and then pass that to orjson.dumps(). This ensures compatibility without a custom default function.

For production systems, wrap serialization in a try-except block to handle unexpected types gracefully. Log the offending object type to aid debugging. This is especially important when data comes from external sources and may not match the expected schema.