Python Requests File Upload with Multipart Form Data

python requests file upload multipart form data requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

Uploading files with the Python requests library is a common task when integrating with REST APIs that accept multipart form data. The files parameter is the key to this functionality. It allows you to send one or more files in a POST request, along with optional form fields, without manually constructing the multipart body. This article focuses on the practical mechanics of using requests for file uploads, covering syntax, behavior, and common pitfalls.

How the files Parameter Works

The requests library abstracts the multipart encoding for you. When you pass a dictionary to the files parameter, it encodes the request body as multipart/form-data. Each key in the dictionary becomes a form field name, and the value can be a file-like object, a tuple, or a string. The library handles the Content-Type header and the multipart boundary automatically.

A minimal upload looks like this:

import requests

url = "https://api.example.com/upload"
files = {"file": open("report.pdf", "rb")}
response = requests.post(url, files=files)

The open() call returns a file object that requests reads to populate the multipart body. After the request completes, you should close the file explicitly, or use a context manager to ensure it is closed even if an exception occurs.

Basic File Upload with a Context Manager

Using a with statement guarantees the file is closed after the request, even if the request fails. This is a safer pattern for production code:

import requests

url = "https://api.example.com/upload"
with open("report.pdf", "rb") as f:
    response = requests.post(url, files={"file": f})
print(response.status_code)

The file is opened in binary mode ("rb"). This is required because multipart uploads must read the file as bytes, not text. If you open in text mode, the request may fail or corrupt the upload depending on the content.

Sending Multiple Files in One Request

The files dictionary can hold multiple entries, each representing a separate file field. The server receives each file under its corresponding field name.

import requests

url = "https://api.example.com/upload"
files = {
    "document": open("report.pdf", "rb"),
    "image": open("photo.jpg", "rb"),
}
response = requests.post(url, files=files)

Alternatively, you can send a list of tuples when you need to repeat the same field name multiple times, such as when the API expects multiple files under the same key:

files = [
    ("files", open("a.pdf", "rb")),
    ("files", open("b.pdf", "rb")),
]
response = requests.post(url, files=files)

This produces a multipart body with two files fields, each containing a different file.

Including Additional Form Fields

Multipart uploads often include non-file fields alongside the file. You can pass them using the data parameter, which requests merges into the same multipart body.

import requests

url = "https://api.example.com/upload"
files = {"file": open("report.pdf", "rb")}
data = {"description": "Q3 report", "author": "Alice"}
response = requests.post(url, files=files, data=data)

The data dictionary is encoded as form fields, while files provides the file parts. This is the standard way to send metadata with an upload.

Streaming Large Files with a Generator

For very large files, reading the entire file into memory before sending is inefficient. The requests library allows you to pass a generator that yields chunks of data, streaming the upload without loading the whole file into RAM. This is useful for files that exceed available memory or when you want to reduce memory footprint.

import requests

def file_chunk_generator(file_path, chunk_size=8192):
    with open(file_path, "rb") as f:
        while chunk := f.read(chunk_size):
            yield chunk

files = {"file": ("large.bin", file_chunk_generator("large.bin"))}
response = requests.post("https://api.example.com/upload", files=files)

The generator yields binary chunks, and requests consumes them as part of the multipart encoding. This avoids loading the entire file into memory at once. However, note that requests still builds the multipart body incrementally, so the memory savings depend on how the server handles the request. For true streaming uploads, consider using a lower-level HTTP client or a library that supports chunked transfer encoding.

Common Pitfalls and Error Handling

Several issues commonly arise when using requests for file uploads.

Forgetting to Close Files

If you open a file with open() and do not close it, the file descriptor leaks. This can exhaust system resources in long-running processes. Always use a context manager or close the file explicitly after the request.

Incorrect File Mode

Opening a file in text mode ("r"``) instead of binary mode ("rb"`) can cause encoding errors, especially for non-text files. Always use binary mode for uploads.

Overriding the Content-Type Header

When you pass files, requests sets the Content-Type header to multipart/form-data with a boundary. If you manually set Content-Type in the headers, it will override the boundary and break the request. Let requests manage the header.

Handling HTTP Errors

A non-2xx status code does not raise an exception by default. Check response.status_code or use response.raise_for_status() to detect failures. The server may return an error message in the response body, so inspect response.text for details.

try:
    response = requests.post(url, files=files)
    response.raise_for_status()
except requests.exceptions.RequestException as e:
    print(f"Upload failed: {e}")

Performance and Memory Considerations

For large uploads, memory usage is the primary concern. The default files approach reads the entire file into memory when building the multipart body, which can be problematic for multi-gigabyte files. The generator approach reduces memory usage by streaming chunks, but it still requires the server to accept chunked uploads. If you control the server, ensure it supports streaming. For very large files, consider using a dedicated upload library or an object storage service that handles multipart uploads natively.

Another performance factor is connection reuse. The requests library uses a session object to reuse the underlying TCP connection, which reduces latency for multiple uploads to the same host. Use requests.Session() when uploading many files in a loop.

Security Considerations for File Uploads

When your application accepts file uploads from users, the multipart handling is only one part of the security picture. You must validate file types, size limits, and content before processing. The requests library does not perform any validation; it simply sends whatever you provide. On the server side, enforce limits and sanitize filenames. On the client side, avoid uploading files from untrusted sources without checking them first.

If you are building an upload client, consider using a session with a timeout to prevent hanging requests. Set a reasonable timeout value based on the expected file size and network conditions.

s = requests.Session()
s.timeout = 30

This prevents the client from waiting indefinitely if the server stops responding.

Handling File-Like Objects and Custom Filenames

The files parameter accepts file-like objects that have a read() method, not just real files. This allows you to upload data from memory, such as a BytesIO object. You can also specify a custom filename and content type by using a tuple as the value.

from io import BytesIO
import requests

file_data = BytesIO(b"some binary content")
files = {"file": ("custom_name.bin", file_data, "application/octet-stream")}
response = requests.post(url, files=files)

The tuple format is (filename, file_handle, content_type). This is useful when the file name on the server should differ from the local file name, or when you are generating content in memory.