RDRYUSLOG DEV
Back to Blog
Python

Python Requests Timeout Exception Handling and raise_for_status

python requests timeout exception handling and raise_for_status: Set connect and read timeouts in Python requests, catch Timeout exceptions, and use raise_for_status()...

pythonrequeststimeoutexception-handlingraise-for-status
Illustration of a Python requests call with a timeout clock and an HTTP status error indicator

A bare requests.get() call can hang indefinitely, and a response with a 404 status code raises nothing by default. Handling both problems requires explicit configuration: a timeout on the request and a status check on the response. This article covers python requests timeout exception handling and raise_for_status — how to set timeouts, catch the exceptions they produce, and combine that with raise_for_status() so transport failures and HTTP error statuses are both handled predictably.

Why a Request Without a Timeout Can Hang

When you call requests.get(url) without a timeout argument, the underlying socket has no deadline. If the server accepts the connection but never sends a response, the call blocks until the OS-level TCP timeout eventually fires, which can take minutes. In a web application, that blocked thread consumes a worker, and under enough concurrent hangs the entire service stalls.

The requests library does not apply a default timeout. The documentation is explicit about this: a missing timeout means the request waits indefinitely. For scripts that run once this may be acceptable, but for anything that serves traffic or runs in a scheduled job, an unbounded wait is a reliability bug.

Setting a Timeout on the Request

The timeout parameter accepts either a single float or a tuple of two floats:

import requests

# Single value: applies to both connect and read phases
response = requests.get("https://api.example.com/data", timeout=5)

# Tuple: (connect timeout, read timeout)
response = requests.get("https://api.example.com/data", timeout=(3, 10))

The single-value form applies the same number to both phases. The tuple form separates them: the first value is the maximum time to establish the connection, and the second is the maximum time to wait between bytes received from the server. The read timeout is not a total request duration; it resets each time data arrives. A slow-but-progressing download can therefore run much longer than the read timeout value.

Choosing values depends on the service. A public API reached over the internet typically needs a connect timeout of a few seconds and a read timeout of 10–30 seconds. An internal service on the same network can use tighter values. There is no universal correct number, but a missing timeout is almost always worse than a too-tight one, because a too-tight timeout fails fast and can be tuned upward.

Catching Timeout Exceptions

When a timeout fires, requests raises an exception from the requests.exceptions module. The base class is Timeout, which inherits from RequestException. Two subclasses distinguish the failure phase:

  • ConnectTimeout — the connection could not be established within the connect timeout.
  • ReadTimeout — the server stopped sending data for longer than the read timeout.

Catching the base Timeout class covers both:

import requests
from requests.exceptions import Timeout

try:
    response = requests.get("https://api.example.com/data", timeout=(3, 10))
except Timeout:
    # Log and either retry or degrade gracefully
    print("The request timed out")

Catching Timeout is usually sufficient unless you need different behavior for each phase. A ConnectTimeout often means the host is unreachable, in which case retrying is pointless until the network issue is resolved. A ReadTimeout may mean the server is overloaded or the response is genuinely slow, so a retry with a longer read timeout can be justified.

What raise_for_status() Actually Does

raise_for_status() checks the response status code and raises requests.exceptions.HTTPError if the code indicates a client or server error. It does not raise for 3xx redirects, and it does not inspect the response body. It also does nothing about transport-level failures — a timeout or connection error is raised before a response object even exists.

import requests
from requests.exceptions import HTTPError

response = requests.get("https://api.example.com/data", timeout=5)
response.raise_for_status()

If the server returns 404 or 500, raise_for_status() raises an HTTPError whose .response attribute holds the original response object, so the status code and body remain available for logging.

The method is useful because requests otherwise treats a 404 as a successful request. Without raise_for_status(), code that assumes response.json() will succeed can fail later with an unrelated ValueError when the body is an error page instead of JSON. Calling raise_for_status() moves the failure to the point where the status is known.

Combining Timeout Handling with raise_for_status()

The two mechanisms cover different failure domains: timeouts happen before a response exists, and HTTP error statuses happen after one arrives. A complete handler catches both:

import requests
from requests.exceptions import HTTPError, Timeout

try:
    response = requests.get("https://api.example.com/data", timeout=(3, 10))
    response.raise_for_status()
except Timeout:
    # Transport-level failure: no response was received
    handle_timeout()
except HTTPError:
    # The server responded with 4xx or 5xx
    status = response.status_code
    handle_http_error(status)

The except order matters. HTTPError and Timeout are siblings under RequestException, so either order works here, but if you later add a broader except requests.exceptions.RequestException, it must come last. Catching the base RequestException covers timeouts, connection errors, HTTP errors, and invalid URL errors, which is useful at the outermost boundary of a request function:

import requests
from requests.exceptions import RequestException

try:
    response = requests.get("https://api.example.com/data", timeout=5)
    response.raise_for_status()
except RequestException as exc:
    # Covers Timeout, ConnectionError, HTTPError, and others
    log_error(exc)

The tradeoff is that RequestException hides which failure mode occurred. For logging and metrics, catching the specific classes separately gives more information; for a single fallback path where any failure means the same user-facing error, the base class keeps the handler simple.

Retrying After a Timeout

A timeout is not necessarily a permanent failure. The server may have processed the request and simply taken too long to respond. This makes retrying dangerous for non-idempotent operations: a POST that creates a resource could succeed server-side even though the client timed out, and a retry would create it twice.

For idempotent operations — GET, PUT, DELETE where the operation is naturally repeatable — retrying with a backoff is reasonable. The urllib3 Retry class, which requests uses under the hood, can be configured on an HTTPAdapter:

from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry_strategy = Retry(
    total=3,
    backoff_factor=0.5,
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["GET", "PUT", "DELETE"],
)

session = requests.Session()
session.mount("https://", HTTPAdapter(max_retries=retry_strategy))

This retries on the listed status codes and on transport errors, but only for the allowed methods. POST is excluded because it is not assumed safe to repeat. The backoff_factor produces sleep times of 0.5, 1.0, and 2.0 seconds before the three retries.

A manual retry loop gives finer control over which exceptions trigger a retry:

import time
import requests
from requests.exceptions import Timeout

for attempt in range(3):
    try:
        response = requests.get("https://api.example.com/data", timeout=5)
        response.raise_for_status()
        break
    except Timeout:
        if attempt == 2:
            raise
        time.sleep(0.5 * (2 ** attempt))

The key rule is to retry only when the operation is safe to repeat, and to re-raise after the final attempt so the caller knows the operation failed.

Distinguishing Timeout from Other Transport Errors

Timeout is one of several exceptions under RequestException. ConnectionError covers DNS failures, refused connections, and dropped connections. TooManyRedirects covers redirect loops. InvalidURL covers malformed URLs. A handler that catches only Timeout will let these propagate, which is often correct — a DNS failure and a timeout warrant different responses.

The requests.exceptions hierarchy mirrors urllib3's exceptions, and the mapping is stable across recent versions. If you need to distinguish a timeout from a connection error in one handler, catch both explicitly:

from requests.exceptions import ConnectionError, Timeout

try:
    response = requests.get("https://api.example.com/data", timeout=5)
except Timeout:
    # Server accepted the connection but stopped responding
except ConnectionError:
    # Host unreachable, DNS failure, or connection refused

In practice, a ReadTimeout is more likely to be worth retrying than a ConnectionError, because the latter often indicates an infrastructure problem that a retry will not fix.

Production Considerations for Timeout Handling

In production code, the distinction between a timeout and an HTTP error should appear in logs and metrics separately. A high rate of ReadTimeout on one endpoint points to a slow upstream service; a high rate of ConnectTimeout points to network or DNS problems. If both are logged as generic RequestException, the signal is lost.

The response object attached to an HTTPError is worth logging:

import requests
from requests.exceptions import HTTPError

try:
    response = requests.get("https://api.example.com/data", timeout=5)
    response.raise_for_status()
except HTTPError as exc:
    status = exc.response.status_code
    body_preview = exc.response.text[:200]
    log_error(f"HTTP {status}: {body_preview}")

The body preview is truncated because error pages can be large and logging the full body wastes storage. The status code alone is usually enough to route the alert, but the body often contains the upstream error message that explains the cause.

Timeout values should be configuration, not hardcoded constants, because they depend on the environment. A batch job that calls a slow reporting endpoint may need a 60-second read timeout, while an interactive API call should fail fast at 5 seconds. Putting these in environment variables or a settings module makes the behavior tunable without a code change.

python requests timeout exception handling and raise_for_sta | RYUSLOG DEV