Python memoryview vs bytes: Zero-Copy Buffers

When working with binary data in Python, the choice between bytes and memoryview directly affects memory usage and performance. The python memoryview vs bytes distinction is central to Python's buffer protocol: a memoryview exposes an object's internal buffer without copying, while bytes always creates an immutable sequence of bytes. This article explains the practical differences, shows when zero-copy access matters, and provides decision criteria for choosing between them.

The Core Difference: Copying vs Viewing

A bytes object is an immutable sequence of integers in the range 0–255. When you create a bytes object from another buffer-like object, Python copies the underlying data into a new allocation. For example:

original = bytearray(b'hello world')
b = bytes(original)  # copies the entire buffer

Here, b is a new object with its own memory. Modifying original later does not affect b. This copy guarantees immutability and isolation, but it costs time and memory proportional to the data size.

A memoryview, on the other hand, is a view into an existing buffer. It does not own the data; it references the buffer of another object, such as a bytearray, array.array, or a bytes object. Creating a memoryview does not copy the underlying data:

original = bytearray(b'hello world')
mv = memoryview(original)  # no copy, just a view

Now mv reflects the current contents of original. If you modify original, the view sees the change. This zero-copy behavior is the primary reason to use memoryview when you need to work with large buffers without duplicating them.

How bytes and memoryview Handle Data Differently

bytes is a concrete type that owns its memory. It supports slicing, concatenation, and various methods, but every operation that produces a new bytes object creates a copy. For example, slicing a bytes object:

data = b'0123456789'
part = data[2:5]  # creates a new bytes object with b'234'

The slice copies three bytes into a new allocation. This is fine for small data, but for large buffers, repeated slicing can become expensive.

memoryview also supports slicing, but the slice is another memoryview that references the same underlying buffer. No data is copied:

data = bytearray(b'0123456789')
mv = memoryview(data)
part = mv[2:5]  # a new memoryview, but no copy

This makes memoryview slices nearly free in terms of memory and time. However, memoryview objects are not as feature-rich as bytes. They lack methods like find, startswith, or split that are available on bytes. You can convert a memoryview to bytes when you need those methods, but that conversion copies the data.

Another difference is mutability. bytes is immutable, so you cannot change individual bytes. A memoryview can be read-only or writable depending on the underlying object. If the source is a bytearray, the memoryview is writable by default, allowing in-place modification:

buf = bytearray(b'hello')
mv = memoryview(buf)
mv[0] = ord('H')
print(buf)  # b'Hello'

If the source is a bytes object, the memoryview is read-only and any attempt to assign will raise TypeError.

Zero-Copy Slicing with memoryview

Slicing is where memoryview shines. Consider a protocol that reads a fixed-size header followed by a payload. With bytes, each slice copies data:

def parse_bytes(data: bytes):
    header = data[:4]      # copy
    payload = data[4:]     # copy
    return header, payload

With memoryview, the slices are views into the original buffer:

def parse_memoryview(data: memoryview):
    header = data[:4]      # view, no copy
    payload = data[4:]     # view, no copy
    return header, payload

This is especially beneficial when processing large binary files or network packets. The original buffer remains referenced as long as any view exists, so the data is not freed prematurely. However, this also means that keeping a small slice prevents the entire original buffer from being garbage-collected. If you need to retain only a small portion of a large buffer, converting that slice to bytes may be more memory-efficient in the long run.

Practical Example: Parsing a Binary File

Let's apply these concepts to a realistic scenario: reading a binary file that contains a sequence of records, each with a 4-byte length prefix and a payload. We'll compare a bytes-based approach with a memoryview-based approach.

First, using bytes:

def read_records_bytes(data: bytes):
    records = []
    offset = 0
    while offset < len(data):
        length = int.from_bytes(data[offset:offset+4], 'big')
        offset += 4
        payload = data[offset:offset+length]
        records.append(payload)
        offset += length
    return records

Each slice data[offset:offset+4] and data[offset:offset+length] creates a new bytes object. For many records, this results in many small allocations.

Using memoryview:

def read_records_memoryview(data: memoryview):
    records = []
    offset = 0
    while offset < len(data):
        length = int.from_bytes(data[offset:offset+4], 'big')
        offset += 4
        payload = data[offset:offset+length]
        records.append(payload)  # stores a memoryview slice
        offset += length
    return records

Here, payload is a memoryview slice that references the original data buffer. No copies are made. If you need the payload as a bytes object (e.g., to pass to an API that requires bytes), you can call payload.tobytes(), but that copies. The trade-off is clear: memoryview avoids copies during parsing, but the records remain tied to the original buffer.

Performance and Memory Implications

The performance benefit of memoryview comes from avoiding copies. Copying a large buffer takes time proportional to its size. When you slice a bytes object, Python allocates new memory and copies the bytes. Repeated slicing in a loop can lead to many allocations and increased memory pressure. memoryview slices are cheap: they create a new view object that stores a pointer and length, not a copy of the data.

However, memoryview is not always faster. Creating a memoryview object itself has a small overhead. For tiny buffers, the overhead may outweigh the copy cost. Also, operations that require a contiguous bytes object, such as many C extensions or functions that expect a bytes argument, may force a conversion and thus a copy anyway.

Memory usage also differs. A bytes object owns its data; when it is no longer referenced, the memory is freed. A memoryview keeps the underlying buffer alive. If you create a memoryview of a large bytearray and then keep only a small slice, the entire large buffer remains in memory. In such cases, it is better to copy the small slice to bytes and let the large buffer be garbage-collected.

Another consideration is thread safety. bytes is immutable, so it can be safely shared between threads without locking. memoryview may be writable if the underlying object is mutable, and concurrent writes can cause data races. If you need to share data across threads, either use bytes or ensure proper synchronization.

When to Use memoryview vs bytes

The choice depends on your specific use case. Use memoryview when:

• You are working with large buffers and need to slice or access sub-parts without copying.
• You want to modify data in place through a writable view.
• You are implementing a protocol parser or binary format reader where many slices are created.
• You need to pass a buffer to a C extension that supports the buffer protocol and can work with a view directly.

Use bytes when:

• You need a simple, immutable, self-contained object that can be safely stored or transmitted.
• You need methods like find, split, startswith, or replace that are not available on memoryview.
• The data is small enough that copying is negligible.
• You want to avoid keeping a large underlying buffer alive just to retain a small slice.
• You are writing code that must be compatible with older Python versions (memoryview has been available since Python 2.7, but its API has evolved; Python 3.3 added many improvements).

A common pattern is to use memoryview during parsing and then convert the final result to bytes when you need to store or return it. This gives you the zero-copy benefit during processing while avoiding the memory-retention issue.

Common Pitfalls and Compatibility Notes

One pitfall is assuming that a memoryview is always writable. Check the readonly attribute:

mv = memoryview(b'immutable')
print(mv.readonly)  # True

Attempting to write to a read-only view raises TypeError.

Another pitfall is using memoryview with non-contiguous buffers. Some objects, like certain array.array instances or multi-dimensional arrays, may have non-contiguous memory layouts. Slicing such a view can produce a view that is not C-contiguous, and some operations may require a contiguous buffer. You can call tobytes() to get a contiguous copy, but that defeats the zero-copy purpose.

Compatibility is also worth noting. The memoryview object has evolved across Python versions. For example, the cast() method was added in Python 3.3, and the ability to use memoryview with struct and array modules has improved over time. If you are writing code that must run on Python 2, be aware that memoryview behaves differently and lacks some features. For modern Python 3, memoryview is stable and well-supported.

Finally, remember that memoryview does not own the data. If the underlying object is resized (e.g., a bytearray that is appended to), the view may become invalid. In CPython, resizing a bytearray invalidates existing views, and accessing them can raise BufferError. Always ensure that the underlying buffer is not modified in a way that changes its size while views are active.

In summary, the python memoryview vs bytes decision is about balancing copy overhead against memory retention and API convenience. For large, performance-sensitive binary processing, memoryview offers a clear advantage. For small data or when you need the full bytes API, stick with bytes.