Python Watchdog: Monitor File and Directory Changes

python watchdog monitor file and directory changes requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When you need to react to changes in files and directories, the Python watchdog library provides a straightforward way to monitor file and directory changes without polling. Instead of repeatedly scanning the filesystem, watchdog uses OS-level notifications where available, making it both efficient and responsive. This article walks through the core components, common usage patterns, and practical considerations for building a reliable file watcher.

Setting Up Watchdog for File and Directory Monitoring

To get started, install the library from PyPI:

pip install watchdog

Watchdog is built around two main abstractions: an Observer that watches the filesystem and an event handler that defines what to do when changes occur. The observer runs in a separate thread and dispatches events to your handler as they happen. This separation lets you keep your main program responsive while the watcher works in the background.

Handling File System Events with Event Handlers

The core of a watchdog-based watcher is a subclass of FileSystemEventHandler. This class defines methods that are called for different event types: on_modified, on_created, on_deleted, on_moved, and on_any_event. You override the methods you care about.

from watchdog.events import FileSystemEventHandler

class MyHandler(FileSystemEventHandler):
    def on_modified(self, event):
        print(f"Modified: {event.src_path}")

    def on_created(self, event):
        print(f"Created: {event.src_path}")

Each event object contains attributes like src_path, is_directory, and for move events, dest_path. The event_type attribute tells you which method triggered the call. When you override on_any_event, you can inspect event.event_type to distinguish between creation, modification, deletion, and movement.

Monitoring Recursively with Patterns and Filters

By default, an observer watches a single directory non-recursively. To watch subdirectories, set recursive=True. You can also filter which files trigger events using the patterns and ignore_patterns parameters when scheduling the observer.

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class Handler(FileSystemEventHandler):
    def on_modified(self, event):
        print(f"Modified: {event.src_path}")

observer = Observer()
observer.schedule(Handler(), path='.', recursive=True, patterns=['*.py', '*.txt'])
observer.start()

The patterns argument accepts a list of glob-style patterns. Only files matching these patterns will generate events. This is useful when you want to watch only source files or logs, avoiding noise from temporary files. Similarly, ignore_patterns lets you exclude files that match certain patterns, such as editor swap files or build artifacts.

Running the Observer in a Background Thread

The observer runs in its own thread, so your main program can continue doing other work. This is essential for long-running applications. When you call observer.start(), the thread begins monitoring. You can keep the main thread alive with a loop or by using observer.join() if you want to block until the observer stops.

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class Handler(FileSystemEventHandler):
    def on_modified(self, event):
        print(f"Modified: {event.src_path}")

observer = Observer()
observer.schedule(Handler(), path='.', recursive=True)
observer.start()
try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()
observer.join()

The stop() method halts the observer, and join() waits for the thread to finish cleanly. This pattern is common in scripts that need to run until interrupted.

Common Pitfalls and How to Avoid Them

One frequent issue is receiving duplicate events for a single file change. This happens because some editors write files by renaming a temporary file, triggering both on_created and on_modified. You can deduplicate by tracking recent paths and timestamps, or by using on_any_event and filtering based on event.event_type.

Another pitfall is missing events when the directory is replaced or when the observer is not started before changes occur. Ensure the observer is started before you expect events, and consider using a short time.sleep after start() to allow the observer to initialize its underlying watch mechanisms.

Performance and Resource Considerations

Watchdog uses OS-level notifications where available, so it is more efficient than polling. However, handling a high volume of events can still saturate your event loop. If you are processing many files, consider batching events with a queue and processing them in a separate thread or process. Also, be mindful of recursive watches on large directory trees, as they consume memory and file descriptors.

For long-running processes, periodically check that the observer is still alive and restart it if it fails. The observer may silently stop if the underlying filesystem becomes unavailable. You can inspect observer.is_alive() to detect this condition.

Stopping the Observer Gracefully

Stopping the observer cleanly is important to avoid leaving orphan threads. The recommended approach is to use a context manager or a try/finally block. Watchdog provides a watchdog.observers.Observer that implements the context manager protocol, so you can use it with with:

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class Handler(FileSystemEventHandler):
    def on_modified(self, event):
        print(f"Modified: {event.src_path}")

with Observer() as observer:
    observer.schedule(Handler(), path='.', recursive=True)
    observer.start()
    # ... do work

When the with block exits, stop() and join() are called automatically. This ensures the observer thread is cleaned up even if an exception occurs. Using this pattern makes your watcher robust and easy to integrate into larger applications.