python loguru structured json logging: Practical Usage and C

Loguru's default output is human-readable text with colors and formatting. For centralized logging systems like Elasticsearch, Loki, or CloudWatch, you need structured key-value pairs. JSON is the common interchange format. So you need to configure Loguru to emit JSON lines. This article shows how to implement python loguru structured json logging without external plugins.

Why Loguru Needs a JSON Formatter

Loguru's default sink prints a formatted string with time, level, message, and other fields. That works for local debugging but breaks down when logs are shipped to a log aggregator. A JSON line is self-describing: each field has a name, and the structure is consistent across records. Aggregators can index fields without custom parsing rules. The tradeoff is that JSON is more verbose and slightly harder to read in a terminal, but that is acceptable when the logs are consumed by machines.

A Minimal JSON Sink

Loguru lets you replace the default sink with a custom function that receives a record dictionary. Inside that function, you build a dictionary and serialize it with json.dumps. Here is a minimal implementation:

import json
import sys
from loguru import logger

def json_sink(record):
    log_entry = {
        "time": record["time"].isoformat(),
        "level": record["level"].name,
        "message": record["message"],
        "module": record["name"],
        "function": record["function"],
        "line": record["line"],
    }
    print(json.dumps(log_entry), file=sys.stderr)

logger.remove()
logger.add(json_sink)

The logger.remove() call clears the default stderr sink. The custom sink receives a record that contains all the metadata Loguru collects. The time field is a datetime object, so we call isoformat() to get a string. The level field is a Level object; .name gives the uppercase level name. The message is the formatted message after interpolation.

This sink writes to stderr to match Loguru's default behavior. If you want stdout, change the file argument to sys.stdout. The key point is that each log call produces one JSON object on its own line.

Capturing Contextual Fields with bind and extra

Loguru's bind method attaches extra fields to a logger instance. These fields end up in record["extra"]. To include them in the JSON output, merge the extra dictionary into the log entry:

import json
import sys
from loguru import logger

def json_sink(record):
    log_entry = {
        "time": record["time"].isoformat(),
        "level": record["level"].name,
        "message": record["message"],
        **record["extra"],
    }
    print(json.dumps(log_entry), file=sys.stderr)

logger.remove()
logger.add(json_sink)

logger.bind(user_id=123, request_id="abc").info("user logged in")

The output is a single JSON line with user_id and request_id alongside the standard fields. The **record["extra"] unpacking works because extra is a regular dictionary. If you need to override a standard field with a contextual one, place the unpacking after the standard keys, as shown. This lets you add request IDs, user IDs, or any domain-specific context without changing the sink.

Serializing Exceptions and Tracebacks

When you call logger.exception or logger.opt(exception=True), Loguru captures the active exception and stores it in record["exception"]. This field is a tuple containing the exception type, value, and a traceback object. To serialize it, you need to convert it to a JSON-friendly structure:

import json
import sys
from loguru import logger

def json_sink(record):
    log_entry = {
        "time": record["time"].isoformat(),
        "level": record["level"].name,
        "message": record["message"],
        **record["extra"],
    }
    if record["exception"] is not None:
        exc_type, exc_value, exc_tb = record["exception"]
        log_entry["exception"] = {
            "type": exc_type.__name__,
            "value": str(exc_value),
            "traceback": "".join(exc_tb.format_traceback()).strip(),
        }
    print(json.dumps(log_entry), file=sys.stderr)

logger.remove()
logger.add(json_sink)

try:
    1 / 0
except ZeroDivisionError:
    logger.exception("division failed")

Here exc_type is the class, exc_value is the instance, and exc_tb is a Traceback object. The format_traceback() method returns a list of strings, which we join into a single string. The strip() removes trailing newlines. This structure is easy to query in a log aggregator: you can filter by exception.type or search for a substring in exception.traceback.

Filtering and Multiple Sinks

You often want JSON output for production and human-readable output for local development. Loguru allows multiple sinks with different filters. The filter parameter can be a callable that receives the record and returns True if the record should be processed by that sink. For example:

import json
import sys
from loguru import logger

def json_sink(record):
    log_entry = {
        "time": record["time"].isoformat(),
        "level": record["level"].name,
        "message": record["message"],
        **record["extra"],
    }
    print(json.dumps(log_entry), file=sys.stderr)

logger.remove()
logger.add(sys.stderr, format="{time} {level} {message}", filter=lambda r: r["level"].name == "DEBUG")
logger.add(json_sink, filter=lambda r: r["level"].name != "DEBUG")

In this setup, DEBUG messages go to the console in a readable format, while INFO and above go to the JSON sink. The filter receives the full record, so you can also filter on module name, thread ID, or any field in record["extra"]. This is useful for separating noisy debug logs from operational logs.

Performance and Serialization Overhead

JSON serialization adds CPU cost per log call. For moderate log volumes, json.dumps is sufficient. For high throughput, consider using orjson if you can add a dependency. orjson is faster and natively serializes datetime objects. However, the sink function itself is called synchronously, so a slow serializer blocks the calling thread. If you need non-blocking logging, you can push records to a queue and process them in a background thread, but that adds complexity.

Another concern is the serializability of values in record["extra"]. If you bind a custom object, json.dumps will fail. Use default=str in json.dumps to fall back to string conversion, or ensure that all bound values are primitives. For example:

print(json.dumps(log_entry, default=str), file=sys.stderr)

This prevents a TypeError when an unexpected object appears. The cost is that the object's string representation may not be what you want, so it is better to explicitly convert non-serializable values before binding them.

Operational Considerations for JSON Logs

A JSON log line must be a single line for most log shippers. If a message contains a newline, it will break the JSON structure. Replace newlines in the message before serialization:

log_entry["message"] = record["message"].replace("\n", "\\n")

Similarly, ensure that any field from extra does not contain raw newlines. You can apply the same replacement to all string values, but that is rarely necessary if you control the bound values.

Log rotation is another concern. Loguru's rotation and retention parameters work with any sink, including a custom function. For example, logger.add(json_sink, rotation="10 MB", retention="30 days") rotates the sink's output based on file size. However, a custom sink that writes to print does not manage files; you would need to use a file sink instead. If you want JSON logs written to a file, define a sink that opens a file and writes JSON lines, and use Loguru's built-in rotation on that sink.

Finally, ensure that the sink writes to the correct stream for your deployment. In containers, stdout is often the standard for logs. Using sys.stdout makes the logs appear in docker logs or kubectl logs. If you use a logging library that captures stderr, adjust accordingly. The choice affects how your orchestrator collects logs, so it is worth matching the platform convention.