Python OpenAI Responses API for Text Generation

python openai responses api text generation requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

The OpenAI Responses API provides a single endpoint for generating text from a model. In Python, the openai client library exposes it through client.responses.create(). This article shows how to use it for text generation, covering request structure, response parsing, streaming, and production considerations. If you're already familiar with the Chat Completions API, the Responses API introduces a cleaner interface and a unified response format, but it also has its own quirks.

Making Your First Text Generation Request

To start, install the OpenAI Python SDK and set your API key. The SDK is the official client and handles authentication, request serialization, and response deserialization. The minimal call to generate text looks like this:

from openai import OpenAI

client = OpenAI(api_key="your-api-key")

response = client.responses.create(
    model="gpt-4o",
    input="Explain the difference between a list and a tuple in Python."
)

print(response.output_text)

The output_text attribute is a convenience property that concatenates the text from all output items. It's the quickest way to get the generated content, but if you need structured access to individual parts, you'll parse the output list as described later.

This request uses the default instructions (none) and no tools. It's the simplest possible invocation, and it's a good starting point for understanding the API's behavior.

Understanding the Request Parameters

The responses.create method accepts several parameters that control the generation. The most important are:

• model: The model identifier, such as gpt-4o or gpt-4o-mini. The model determines capability, speed, and cost.
• input: The user message or a list of messages. For a simple text prompt, a string works. For multi-turn conversations, you pass a list of message objects with role and content.
• instructions: A system-level instruction that guides the model's behavior. This is equivalent to the system prompt in Chat Completions.
• max_output_tokens: The maximum number of tokens to generate. This controls response length and cost.
• temperature: Controls randomness. Lower values produce more deterministic output, higher values more creative.
• top_p: Nucleus sampling parameter. It's an alternative to temperature.
• tools: A list of tools the model can call, such as functions or a web search.
• store: Whether to store the response for later retrieval. Defaults to True in the Responses API, but you can set it to False to avoid data retention.

Here's a more complete example that uses instructions and limits output length:

response = client.responses.create(
    model="gpt-4o",
    instructions="You are a concise technical writer.",
    input="Summarize the Python GIL in two sentences.",
    max_output_tokens=100,
    temperature=0.2
)

The instructions parameter is especially useful when you need consistent behavior across many calls. It keeps the system prompt separate from the user input, which simplifies logging and debugging.

Parsing the Response Object

The response object contains more than just the generated text. The output attribute is a list of items, each with a type such as message or function_call. For a simple text generation, you'll typically see one message item with content that contains a list of parts. The output_text property flattens this into a single string.

If you need to inspect the structure, you can iterate over the output:

for item in response.output:
    if item.type == "message":
        for content in item.content:
            if content.type == "output_text":
                print(content.text)

This is useful when the model emits multiple content parts, such as when it interleaves text with tool calls. In a pure text generation scenario, output_text is sufficient.

Another important field is usage, which reports token counts:

print(response.usage)

It returns an object with input_tokens, output_tokens, and total_tokens. Monitoring this helps you estimate costs and debug token limit issues.

Handling Errors and Retries

The OpenAI SDK raises exceptions for network issues, invalid requests, and rate limits. The most common ones are openai.APIError, openai.APIConnectionError, openai.RateLimitError, and openai.AuthenticationError. A robust implementation should catch these and retry with exponential backoff.

import time
from openai import OpenAI, RateLimitError, APIConnectionError

client = OpenAI()

for attempt in range(3):
    try:
        response = client.responses.create(
            model="gpt-4o",
            input="Write a haiku about Python."
        )
        print(response.output_text)
        break
    except RateLimitError:
        wait = 2 ** attempt
        time.sleep(wait)
    except APIConnectionError:
        wait = 2 ** attempt
        time.sleep(wait)

Rate limits are per-organization and per-model. If you hit a rate limit, the RateLimitError includes a retry_after value in some cases. The SDK also has a built-in retry mechanism that you can configure with the max_retries parameter when creating the client:

client = OpenAI(max_retries=3)

This retries on transient failures like connection errors and 429 responses, but it doesn't handle all errors. For production, you'll want a custom retry loop that respects the retry_after header when present.

Streaming Text Generation

For long outputs, streaming reduces perceived latency and lets you display text as it's generated. The Responses API supports streaming via the stream parameter. When set to True, the method returns an iterator of events instead of a single response object.

stream = client.responses.create(
    model="gpt-4o",
    input="Explain the benefits of async programming in Python.",
    stream=True
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="")

Each event has a type field. The response.output_text.delta event contains the incremental text. You can accumulate these deltas to build the full output or display them directly in a UI.

Streaming is particularly useful for chat interfaces or when you want to show progress. It also lets you cancel generation early if the user interrupts, saving tokens.

Responses API vs Chat Completions

The Responses API is the newer interface, designed to replace Chat Completions for many use cases. The key differences affect how you structure requests and handle responses.

If you're starting a new project, the Responses API is the recommended path. It simplifies the request structure and provides a more consistent response model. However, if you have existing code that uses Chat Completions, migrating requires changing both the call and the parsing logic.

One notable difference is that the Responses API automatically stores responses by default. This can be useful for debugging, but it also means data is retained on OpenAI's servers. Set store=False if you don't need that.

Production Considerations for Text Generation

When moving from a script to a production service, several factors become important. The first is cost control. Token usage varies significantly based on model and prompt length. Setting max_output_tokens prevents runaway costs from unexpected long responses. You should also monitor the usage field in each response to track spending.

The second is concurrency. The OpenAI client is thread-safe, so you can share a single client instance across multiple requests. However, you need to respect rate limits. Use a semaphore or a connection pool to limit concurrent requests, and implement retries with backoff as shown earlier.

The third is error handling beyond simple retries. For example, if the model returns a response that violates your content policy, the API may raise a ContentPolicyViolationError. You should catch that separately and handle it gracefully, perhaps by returning a fallback message.

Finally, consider the store parameter. In production, you often don't want to store every response. Set store=False to avoid unnecessary data retention and to comply with stricter privacy requirements. The trade-off is that you lose the ability to retrieve past responses via the API, but you can log them yourself if needed.

Streaming also has production implications. When streaming, you must handle partial output carefully. If a user disconnects mid-stream, you should cancel the stream to stop token generation. The SDK's stream object can be closed with close(), and you can use a try/finally block to ensure cleanup.

These considerations are not exhaustive, but they address the most common issues developers face when deploying text generation with the Responses API. The API is still evolving, so always check the official documentation for the latest parameter changes and deprecations.