Python Binary Integer: Bits and Bitwise Operations

When you work with a Python binary integer, you are interacting with a fixed-width conceptual model that Python implements with arbitrary precision. Unlike languages like C or Java, Python integers can grow beyond 64 bits, but the underlying bit operations follow the same two's complement rules. Understanding this representation is essential for writing correct bit manipulation code, especially when dealing with negative numbers or shifting operations.

How Python Stores Integers in Binary

Python's int type uses a variable-length array of digits in base 2^30 or 2^15 internally, depending on the platform. From a developer's perspective, however, you can treat an integer as an infinite sequence of bits in two's complement form. Positive numbers have leading zeros, and negative numbers have leading ones. This abstraction is consistent across all bitwise operations.

For example, the integer 5 is represented as ...000101, while -5 is ...111011 in two's complement. Python hides the infinite leading bits, but they become visible when you use the bin() function or perform shifts.

Reading and Writing Binary Literals

You can write integer literals directly in binary using the 0b prefix. This is useful for defining bit masks or flags without converting from decimal.

mask = 0b11110000
flags = 0b00001111

The bin() function returns a string representation with a 0b prefix, and you can use int() with a base to parse binary strings.

value = 0b1010
print(bin(value))  # 0b1010
print(int('1010', 2))  # 10

When you print a negative integer with bin(), Python includes a minus sign rather than showing the two's complement bits. For example, bin(-5) returns -0b101. To see the actual two's complement bit pattern, you need to mask the value to a fixed width.

Bitwise Operators and Their Behavior

Python provides six bitwise operators: & (AND), | (OR), ^ (XOR), ~ (NOT), << (left shift), and >> (right shift). These operate on the binary representation of integers.

a = 0b1100
b = 0b1010
print(a & b)  # 0b1000 (8)
print(a | b)  # 0b1110 (14)
print(a ^ b)  # 0b0110 (6)
print(~a)     # -13 (inverts all bits)

The ~ operator returns the bitwise complement, which in Python is -x - 1 because of the infinite leading bits. For example, ~5 is -6. This behavior can surprise developers coming from languages with fixed-width integers.

Two's Complement and Negative Numbers

Python's bitwise operations assume an infinite two's complement representation. This means that shifting right on a negative number preserves the sign bit (arithmetic shift), not a logical shift that fills with zeros.

print(-8 >> 1)  # -4
print(-8 >> 2)  # -2

Left shifts behave as expected, but they can produce large positive numbers because Python integers grow without overflow. For example, 1 << 100 is a valid integer with 101 bits.

To simulate a fixed-width two's complement representation, you can apply a mask after an operation. For instance, to get the 8-bit representation of -1:

value = -1
mask = 0xFF
print(bin(value & mask))  # 0b11111111

This is a common pattern when working with hardware registers or network protocols that expect a specific bit width.

Practical Bit Manipulation: Flags and Masks

Bitwise operations are often used to pack multiple boolean flags into a single integer. This reduces memory usage and can improve cache locality when many flags are stored in a list or array.

READ = 0b001
WRITE = 0b010
EXECUTE = 0b100

permission = READ | WRITE  # 0b011

if permission & READ:
    print("read allowed")

To toggle a bit, use XOR with a mask. To clear a bit, use AND with the complement of the mask.

permission ^= WRITE  # toggle write
permission &= ~EXECUTE  # clear execute

These operations are atomic and avoid the overhead of integer arithmetic when you only need to manage a few independent flags.

Performance and Memory Considerations

Bitwise operations are among the fastest operations in Python because they map directly to C-level integer operations. However, creating large integers through repeated shifts can allocate memory for the variable-length digit array. For example, 1 << 1000000 creates an integer with over 125 KB of internal storage. If you need to handle extremely large bit sets, consider using the bitarray library or Python's built-in bytearray for more memory-efficient storage.

For typical flag manipulation, the overhead is negligible. The main performance concern is not the bitwise operation itself but the conversion between integers and strings or bytes. Use int.from_bytes() and int.to_bytes() when working with binary data from files or network streams.

data = b'\x01\x02'
value = int.from_bytes(data, byteorder='big')
print(value)  # 258

This approach avoids manual bit shifting for each byte and is more readable.

Common Pitfalls with Bitwise Operations

One frequent mistake is assuming that right shift on a negative number fills with zeros. In Python, it always preserves the sign bit. Another pitfall is using ~ on a positive number and expecting a fixed-width result. For example, ~0b1010 is -11, not 0b0101. If you need a fixed-width bitwise NOT, you must mask the result:

value = 0b1010
width = 4
not_value = (~value) & ((1 << width) - 1)
print(bin(not_value))  # 0b0101

Also, be careful when mixing signed and unsigned interpretations. Python has no unsigned integer type, so a value like 0xFFFFFFFF is a positive integer, not -1. If you are porting code from C, you may need to convert using value - (1 << bits) when a value exceeds the signed range.

Finally, remember that bitwise operators have lower precedence than arithmetic operators but higher than comparisons. Always use parentheses when mixing expressions to avoid subtle bugs.

# Without parentheses, this is parsed as (a & b) == c, not a & (b == c)
if a & b == c:
    pass

Understanding these details helps you write reliable bit manipulation code that behaves predictably across Python versions and platforms.