Python OpenAI File Upload: How to Upload Files with the API

Uploading files to the OpenAI API is a common task when you need to provide training data for fine-tuning, attach documents to an assistant, or use the Files API for other workflows. In Python, the openai package exposes a files.create method that handles multipart uploads. This article covers the practical details of python openai file upload: setup, parameters, error handling, and how to use the uploaded file ID in downstream API calls.

Prerequisites and Setup

Before you can upload a file, you need the openai Python package installed and an API key configured. The package is available on PyPI and can be installed with pip:

pip install openai

The library requires Python 3.7 or later. After installation, create a client instance with your API key. Avoid hardcoding the key in source code; use an environment variable instead:

export OPENAI_API_KEY="your-api-key"

Then in your Python code:

import os
from openai import OpenAI

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

If the key is missing, the client raises an OpenAIError when you make a request. For local development, you can also use a .env file with the python-dotenv package, but the environment variable approach is the most portable.

Basic File Upload with files.create

The core method for uploading a file is client.files.create. It expects a file object opened in binary mode and a purpose string that tells the API how the file will be used. Here is the minimal example:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from environment

with open("training_data.jsonl", "rb") as f:
    response = client.files.create(
        file=f,
        purpose="fine-tune"
    )

print(response.id)

The response is a FileObject containing the id, purpose, filename, bytes, and created_at fields. The id is the value you will use later when creating a fine-tuning job or attaching the file to an assistant message.

If you are uploading a file for an assistant, change the purpose to "assistants". The API accepts a limited set of purposes, and using an invalid one returns a 400 error.

Understanding File Upload Parameters

files.create accepts several parameters beyond the mandatory file and purpose:

The filename parameter is particularly useful when you are uploading a file from memory or when the local name contains characters that are not allowed. For example, you can upload a BytesIO object and set a custom name:

from io import BytesIO

content = b'{"prompt": "What is AI?", "completion": "Artificial Intelligence"}\n'
file_obj = BytesIO(content)

response = client.files.create(
    file=file_obj,
    purpose="fine-tune",
    filename="custom_name.jsonl"
)

Note that the API enforces file size limits and supported file types. The exact limits change over time, so check the official OpenAI documentation for the current values. The library does not validate these constraints locally; it relies on the server to reject invalid uploads.

Handling Upload Errors and Retries

Network failures, rate limits, and invalid inputs can cause uploads to fail. The openai library raises exceptions that you can catch and handle explicitly. The most common ones are:

• openai.APIError: Base class for all API errors.
• openai.APIConnectionError: Network issues, such as a dropped connection.
• openai.RateLimitError: You have exceeded the allowed request rate.
• openai.AuthenticationError: Invalid API key.
• openai.BadRequestError: The file or parameters are invalid.

A robust upload function should retry transient errors and surface permanent ones. Here is an example that retries on connection errors and rate limits with exponential backoff:

import time
from openai import OpenAI, APIConnectionError, RateLimitError

client = OpenAI()

def upload_file(file_path, purpose, max_retries=3):
    for attempt in range(max_retries):
        try:
            with open(file_path, "rb") as f:
                return client.files.create(file=f, purpose=purpose)
        except (APIConnectionError, RateLimitError) as e:
            if attempt == max_retries - 1:
                raise
            wait_time = 2 ** attempt
            print(f"Upload failed, retrying in {wait_time}s: {e}")
            time.sleep(wait_time)
    raise RuntimeError("Unreachable")

For permanent errors like BadRequestError, you should not retry because the request will never succeed. Inspect the exception message to understand what went wrong—it often includes the exact field that failed validation.

Using Uploaded Files in Fine-Tuning and Assistants

The file ID returned from files.create is the key to using the file in other API calls. For fine-tuning, you pass it to the fine_tuning.jobs.create method:

job = client.fine_tuning.jobs.create(
    training_file=response.id,
    model="gpt-3.5-turbo"
)

For assistants, you attach the file to a message using the attachments parameter. The exact syntax depends on the API version, but a common pattern is:

message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Please analyze this document.",
    attachments=[{"file_id": response.id, "tools": [{"type": "file_search"}]}]
)

In both cases, the file must have been uploaded with the correct purpose. A file uploaded with purpose="fine-tune" cannot be attached to an assistant message, and vice versa. If you need to use the same data for both, upload it twice with different purposes.

Performance and Operational Considerations

Uploading large files can take time and consume memory. The openai library reads the entire file into memory when you pass a file object, so for very large files, consider streaming or splitting the data. The API has a maximum file size, and exceeding it results in a 400 error. You can check the file size before uploading to fail fast:

import os

file_size = os.path.getsize("training_data.jsonl")
if file_size > MAX_ALLOWED_SIZE:  # check current limit from docs
    raise ValueError("File too large")

For concurrent uploads, the library is thread-safe, but you should be mindful of rate limits. If you need to upload many files, use a semaphore or a worker pool to limit concurrency. The openai library also supports automatic retries via the max_retries parameter on the client, but it is better to handle retries at the application level when you need custom backoff logic.

Security and API Key Management

Never expose your API key in client-side code or commit it to version control. Use environment variables or a secrets manager. When uploading files, be aware that the content is stored on OpenAI's servers. Do not upload sensitive data unless you have reviewed OpenAI's data usage policies and your own compliance requirements. The openai library does not encrypt files locally; encryption is handled by OpenAI's infrastructure during transmission and at rest.

Common Pitfalls and Edge Cases

One frequent mistake is opening the file in text mode instead of binary. The API expects a binary file object; passing a text-mode file can cause encoding errors or corrupted uploads. Always use "rb".

Another issue is using an invalid purpose string. The API is strict about allowed values. If you are unsure, check the documentation or the error message returned by the server.

Finally, remember that file IDs are not permanent. They may be deleted after a certain period or when you explicitly delete them. For long-term storage, keep a local copy of the file and its ID mapping. When you need to reuse a file, re-upload it if the ID is no longer valid.

Handling these edge cases ensures that your file uploads work reliably in production. The files.create method is straightforward, but the surrounding details—parameter validation, error handling, and operational constraints—determine whether your integration is robust.