Python Playwright Network Interception and Blocking Requests

When automating a browser with Python Playwright, the page often loads resources you don't need—analytics scripts, tracking pixels, large images, or third-party fonts. These requests slow down your script and can introduce flaky behavior. Playwright gives you a route-based interception API to block, modify, or fulfill network requests before they reach the network. This article focuses on python playwright network interception and blocking requests, showing you how to filter and abort requests cleanly, and where the approach has limits.

Understanding Playwright's Route Interception Model

Playwright intercepts network requests at the browser level using the route method on a page or browser context. When you register a route handler, Playwright intercepts every request that matches the URL pattern and gives your handler a route object. The handler can then:

• route.abort() to cancel the request
• route.continue() to let it proceed unchanged
• route.fulfill() to return a synthetic response without hitting the network

This model is synchronous in the sense that the handler runs in the same process as the browser automation, but it can be asynchronous if you use the async API. The key is that the handler must call exactly one of these methods; otherwise the request hangs forever.

Here's a minimal example that blocks all requests to example.com:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.route("**://example.com/**", lambda route: route.abort())
    page.goto("https://example.com")
    browser.close()

The route pattern uses glob syntax. ** matches any number of path segments, and * matches a single segment. The pattern above matches any request whose URL contains example.com as the host, regardless of scheme or path.

Blocking Requests by URL and Resource Type

In practice you rarely want to block an entire domain. You might want to block only certain resource types, like images or stylesheets, or block requests to specific third-party domains while allowing the main site. Playwright's route handler receives a request object that exposes the URL and the resource type.

To block all image requests across the page:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.route("**/*", lambda route: route.abort() if route.request.resource_type == "image" else route.continue())
    page.goto("https://news.ycombinator.com")
    browser.close()

The resource_type property returns values like document, stylesheet, image, media, font, script, xhr, fetch, and other. This is useful when you want to strip heavy resources from a page to speed up scraping or to avoid loading tracking scripts.

You can also block requests based on the URL using a regex or a custom predicate. The route pattern is a glob, but you can use a regex by passing a compiled pattern:

import re
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.route(re.compile(r".*\\.(png|jpg|jpeg|gif)$"), lambda route: route.abort())
    page.goto("https://example.com")
    browser.close()

This blocks all requests whose URL ends with a common image extension. Combining URL and resource type checks gives you fine-grained control over what the browser loads.

Modifying Requests and Responses

Interception isn't limited to blocking. You can also modify request headers, change the request method, or rewrite the URL before continuing. For example, to add a custom header to every request:

from playwright.sync_api import sync_playwright

def add_header(route):
    headers = route.request.headers
    headers["X-Custom-Header"] = "my-value"
    route.continue_(headers=headers)

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.route("**/*", add_header)
    page.goto("https://example.com")
    browser.close()

Note that continue_ is the method name in Python because continue is a reserved keyword. This is a common source of confusion for developers coming from JavaScript.

You can also fulfill a request with a synthetic response, which is useful for mocking API endpoints or returning static data without a server:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.route("**/api/users", lambda route: route.fulfill(status=200, content_type="application/json", body='{"users": []}'))
    page.goto("https://example.com")
    browser.close()

This technique is powerful for testing front-end behavior without depending on a backend.

Async and Context-Level Interception

The examples so far use the sync API. Playwright also provides an async API where route handlers can be async functions. This is useful when you need to perform asynchronous work inside the handler, such as fetching data from a database or making another network call.

import asyncio
from playwright.async_api import async_playwright

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        async def block_analytics(route):
            if "analytics" in route.request.url:
                await route.abort()
            else:
                await route.continue_()
        await page.route("**/*", block_analytics)
        await page.goto("https://example.com")
        await browser.close()

asyncio.run(main())

When you register a route on a browser context, it applies to all pages in that context. This is more efficient than registering the same handler on every page individually. Context-level interception is also useful for setting up a consistent environment across multiple pages in a scraping session.

Performance and Operational Considerations

Blocking requests has a direct performance benefit: fewer network round trips, less bandwidth usage, and faster page load. This is especially noticeable when scraping pages that load heavy media or third-party scripts. However, the route handler itself runs in the Node.js process that drives the browser, and every request goes through it. If you register a route for **/* and perform complex logic for every request, the overhead can become significant.

For high-throughput scraping, keep the handler as lightweight as possible. Use glob patterns to narrow the scope of interception rather than catching everything and then filtering. For example, if you only want to block images, use a pattern like **/*.png or **/*.jpg instead of **/* and then checking resource_type. This reduces the number of requests that invoke your handler.

Another consideration is that aborting a request can affect page behavior. Some pages may wait for a resource to load and then time out if it's aborted. In practice, browsers handle aborted requests gracefully, but you should test your specific target sites to ensure the page still reaches the state you need.

Common Pitfalls and Debugging

A frequent mistake is forgetting to call continue_ for requests you don't want to block. If a route handler returns without calling any method, the request hangs and the page never finishes loading. Always ensure every code path in your handler calls abort, continue_, or fulfill.

Another pitfall is using the wrong method name. In the Python API, it's continue_ with a trailing underscore, not continue. Using continue will raise a syntax error.

Route patterns are glob-based, not regex. If you try to use regex syntax directly in the pattern string, it will be interpreted as a glob. For regex matching, pass a compiled regex object as shown earlier.

When debugging, you can inspect the request object inside the handler to see the URL, method, headers, and post data. You can also log to the console to trace which requests are being intercepted. Playwright's trace viewer records route interception details, which is helpful for understanding why a request was blocked or modified.

Finally, remember that route handlers are per-context or per-page. If you set up a route in one page and then navigate to a new page in the same context, the route still applies. If you need different behavior per page, register the route on the page instead of the context.

Advanced Usage: Conditional Blocking Based on Response

Sometimes you need to decide whether to block a request based on the response it would return. Playwright doesn't allow you to inspect the response before continuing, but you can use route.fetch() to perform the request yourself and then decide how to respond. This is more expensive because it makes the network call, but it gives you full control.

from playwright.sync_api import sync_playwright

def conditional_block(route):
    response = route.fetch()
    if response.status == 200 and "tracking" in response.body.decode("utf-8", errors="ignore"):
        route.abort()
    else:
        route.fulfill(response=response)

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.route("**/script.js", conditional_block)
    page.goto("https://example.com")
    browser.close()

This pattern is useful when you want to block scripts that contain certain identifiers but allow others. The route.fetch() method returns a APIResponse object that you can inspect and then either abort or fulfill with the original response. This adds latency because the request is made twice (once by fetch, once by the browser if you continue), but it's the only way to make decisions based on response content.

Use this technique sparingly, as it defeats the performance benefit of blocking. For most use cases, URL and resource type filtering is sufficient.