python psutil process monitoring and termination: Practical

When you need to inspect running processes from Python, the psutil library is the most direct way to get CPU, memory, and status information without shelling out to system commands. It also provides a clean API for terminating or killing processes, which is useful for cleanup scripts, resource managers, and test harnesses. This article covers the core patterns for python psutil process monitoring and termination, with code examples you can adapt to your own tooling.

Getting Started with psutil

psutil is a cross-platform library that exposes system and process information through a consistent Python interface. Install it with pip:

pip install psutil

The central object is psutil.Process. You can create one from a PID, or use psutil.process_iter() to iterate over all running processes. A minimal monitoring script looks like this:

import psutil

for proc in psutil.process_iter(['pid', 'name', 'cpu_percent', 'memory_percent']):
    print(proc.info)

The process_iter method accepts a list of fields to preload, which avoids a separate system call for each attribute. This is important when you are scanning many processes.

Listing and Filtering Processes

Often you need to find processes that match a name, command line, or other criteria. process_iter returns an iterator of Process objects, and you can filter them in a loop:

import psutil

def find_processes_by_name(name):
    matches = []
    for proc in psutil.process_iter(['pid', 'name']):
        if proc.info['name'] == name:
            matches.append(proc)
    return matches

For more complex filtering, use proc.cmdline() to inspect the full command line. This is useful when multiple executables share the same process name but have different arguments. For example, to find a specific Python script:

def find_script_process(script_name):
    for proc in psutil.process_iter(['pid', 'name']):
        try:
            cmdline = proc.cmdline()
        except (psutil.NoSuchProcess, psutil.AccessDenied):
            continue
        if any(script_name in arg for arg in cmdline):
            return proc
    return None

Note that cmdline() can raise exceptions if the process disappears or if you lack permission, so always handle those cases.

Monitoring CPU and Memory Usage

To monitor a process's resource usage, call cpu_percent() and memory_percent(). The first call to cpu_percent() returns 0.0 because it needs a baseline; subsequent calls return the average CPU usage since the last call. This is a common gotcha.

import psutil
import time

proc = psutil.Process(1234)
print(proc.cpu_percent(interval=None))  # 0.0 on first call
print(proc.memory_percent())            # percentage of system memory

If you need a meaningful CPU reading, either pass an interval or call the method twice with a delay. For a snapshot of the current state, use proc.cpu_times() to get cumulative CPU time, which does not require a baseline.

cpu_times = proc.cpu_times()
print(f"User time: {cpu_times.user}, System time: {cpu_times.system}")

For ongoing monitoring, you can sample at regular intervals. The following loop tracks a process's CPU and memory usage over time:

proc = psutil.Process(pid)
proc.cpu_percent()  # prime the baseline

for _ in range(10):
    print(proc.cpu_percent(interval=1), proc.memory_percent())
    time.sleep(1)

Terminating and Killing Processes

psutil.Process provides two methods for stopping a process: terminate() and kill(). terminate() sends a SIGTERM on Unix or calls TerminateProcess on Windows, giving the process a chance to clean up. kill() sends SIGKILL on Unix or forces termination on Windows, which cannot be caught by the process.

proc = psutil.Process(pid)
proc.terminate()   # graceful request
proc.wait(timeout=5)  # wait for exit

If the process does not exit within the timeout, you may need to escalate to kill():

if proc.is_running():
    proc.kill()

A robust termination helper should handle the case where the process is already gone:

def safe_terminate(pid, timeout=5):
    try:
        proc = psutil.Process(pid)
        proc.terminate()
        proc.wait(timeout=timeout)
    except psutil.NoSuchProcess:
        pass
    except psutil.TimeoutExpired:
        proc.kill()

This pattern is common in scripts that need to clean up child processes or stop background workers.

Handling Permissions and Errors

Process operations are subject to operating system permissions. psutil.AccessDenied is raised when you try to access a process owned by another user or a protected system process. For example, on Linux, reading the environment of another user's process often fails. Always catch this exception:

try:
    proc = psutil.Process(pid)
    print(proc.environ())
except psutil.AccessDenied:
    print("Permission denied")
except psutil.NoSuchProcess:
    print("Process no longer exists")

When iterating over all processes, it is common to encounter both NoSuchProcess and AccessDenied because processes can exit mid-iteration or be protected. Use a try/except inside the loop to skip those entries gracefully.

Another subtle issue is the difference between terminate() and kill() on Windows. On Windows, terminate() calls TerminateProcess immediately, so there is no graceful shutdown. If you need a graceful shutdown on Windows, you must use a different mechanism, such as sending a CTRL_BREAK_EVENT or using a library like subprocess to manage the child process from the start.

Performance and Operational Considerations

Calling psutil methods for every process in a tight loop can be expensive because each attribute access may trigger a system call. When you only need a few fields, pass them to process_iter to batch the request. Avoid calling proc.cmdline() for every process unless you really need it, as it is one of the slower operations.

For long-running monitoring, be careful with cpu_percent(interval=None). If you call it without an interval, it returns the CPU usage since the last call, which is useful for sampling. But if you call it on many processes in a loop, the baseline is per-process and the results may be inconsistent. For aggregate CPU usage, consider using psutil.cpu_percent(interval=None) at the system level.

When terminating processes, be aware that a process may have children. terminate() only stops the target process; its children may become orphaned. To terminate a process tree, you can use psutil.Process.children(recursive=True) and terminate them in reverse order. This is a common pattern for cleaning up after a test suite or a build script.

def terminate_tree(pid):
    try:
        parent = psutil.Process(pid)
        children = parent.children(recursive=True)
        for child in children:
            child.terminate()
        parent.terminate()
        gone, alive = psutil.wait_procs(children + [parent], timeout=5)
        for p in alive:
            p.kill()
    except psutil.NoSuchProcess:
        pass

psutil.wait_procs is a convenient utility that waits for a list of processes and returns two lists: those that exited and those still alive after the timeout. This is more efficient than calling wait() on each process individually.

Monitoring a Specific Process by PID

Sometimes you already know the PID and just need to check its status. The status() method returns a string like running, sleeping, zombie, or stopped. This is useful for detecting hung processes.

proc = psutil.Process(pid)
status = proc.status()
if status == psutil.STATUS_ZOMBIE:
    print("Process is a zombie")

Zombie processes on Unix have already exited but their entry remains in the process table because the parent has not called wait(). psutil can detect them, but you cannot terminate a zombie; you must wait for the parent to reap it. If your script is the parent, use proc.wait() to clean it up.

For a process that is not responding, you may want to check its memory_info() or num_threads() to diagnose the issue. These attributes are cheap to retrieve and give you a snapshot of the process's resource footprint.

Combining these techniques, you can build a process watchdog that monitors a specific PID and terminates it if it exceeds a memory threshold:

import psutil
import time

def watchdog(pid, max_memory_mb, check_interval=2):
    proc = psutil.Process(pid)
    while True:
        try:
            mem = proc.memory_info().rss / (1024 * 1024)
            if mem > max_memory_mb:
                proc.terminate()
                proc.wait(timeout=5)
                print(f"Terminated PID {pid} (memory {mem:.1f} MB)")
                break
        except psutil.NoSuchProcess:
            print("Process already exited")
            break
        time.sleep(check_interval)

This kind of script is useful in CI pipelines or as a simple resource guard when you cannot rely on the application to manage its own memory limits.