Python lxml: XML Namespaces and Validation

When you combine python lxml xml namespaces and validation in the same pipeline, the namespace handling usually fails first. lxml reports namespaced element tags as Clark-notation strings like {http://example.com/orders}order, so a plain find('order') returns nothing, and an XSD with a target namespace refuses a document that omits it. Both behaviors are correct, but they require explicit namespace handling in every lookup and every schema check.

The Namespace Problem in lxml

lxml represents every element tag as either a plain local name or a Clark-notation string {namespace}localname. When an XML document declares a default namespace, all unprefixed elements inherit it, so lxml reports their tags with the namespace prefix:

<?xml version="1.0"?>
<order xmlns="http://example.com/orders">
  <item id="1">
    <name>Keyboard</name>
  </item>
</order>

Parsing this with etree.fromstring gives:

from lxml import etree

root = etree.fromstring(xml)
print(root.tag)          # {http://example.com/orders}order
print(root.find('item')) # None

find('item') returns None because the element's actual tag is {http://example.com/orders}item. Every namespace-aware lookup must use the full Clark notation or an XPath namespace map.

Reading Namespaced Elements

The direct way to access a namespaced element is to include the namespace in the tag:

NS = 'http://example.com/orders'
item = root.find(f'{{{NS}}}item')
name = item.find(f'{{{NS}}}name')

The doubled {{ and }} escape the braces in an f-string. For repeated lookups, define the namespace as a module-level constant so the Clark-notation strings stay consistent across the codebase.

The nsmap attribute on an element shows the namespace declarations in scope:

print(root.nsmap)  # {None: 'http://example.com/orders'}

None is the key for the default namespace. Prefixed declarations appear as their string keys.

Using XPath with Namespace Maps

XPath is the more practical route when the document is deeply nested. xpath() takes a namespaces argument that maps prefixes to URIs:

root.xpath('//o:item/o:name', namespaces={'o': 'http://example.com/orders'})

The prefix in the XPath expression does not need to match the prefix used in the document. Only the URI matters. The same expression works whether the source document used o:, ord:, or a default namespace.

When you do not care which namespace an element belongs to, use a wildcard:

root.xpath('//*[local-name() = "item"]')

This matches item in any namespace, which is useful for heterogeneous feeds but should be avoided when the namespace is known, because it can silently match an unrelated element from a different vocabulary.

findall() and find() accept the same namespace map through a second argument:

root.findall('o:item', namespaces={'o': 'http://example.com/orders'})

Creating Namespaced Documents

When building a document that must be validated later, declare the namespace on the root element so the serializer writes the correct xmlns attributes:

nsmap = {'o': 'http://example.com/orders'}
order = etree.Element('{http://example.com/orders}order', nsmap=nsmap)
item = etree.SubElement(order, '{http://example.com/orders}item')
item.text = 'Keyboard'

Serializing this tree produces:

<o:order xmlns:o="http://example.com/orders">
  <o:item>Keyboard</o:item>
</o:order>

If the document should use a default namespace instead of a prefix, pass None as the key:

nsmap = {None: 'http://example.com/orders'}

For output with a specific prefix, etree.register_namespace(prefix, uri) sets the prefix used during serialization across the process. This matters when the downstream consumer expects a particular prefix in the XML text.

Validating Against an XSD Schema

lxml validates documents against XSD schemas through the XMLSchema class:

schema_doc = etree.parse('orders.xsd')
schema = etree.XMLSchema(schema_doc)

doc = etree.parse('order.xml')
print(schema.validate(doc))  # True or False

validate() returns a boolean and does not raise. To surface the error, use assertValid():

try:
    schema.assertValid(doc)
except etree.XMLSchemaValidateError as exc:
    print(exc)

The error message includes the element path and the reason, for example a missing required attribute or an element that is not allowed at that position.

The schema's target namespace must match the document's namespace. An XSD that declares targetNamespace="http://example.com/orders" validates documents whose root element is in that namespace. A document with no namespace, or with a different namespace, fails even if the local names match. When the schema has no targetNamespace, it validates only documents that also have no default namespace.

Validating Against a DTD

DTD validation is simpler but less expressive:

dtd = etree.DTD('orders.dtd')
print(dtd.validate(doc))

validate() returns a boolean. dtd.error_log contains the last validation error with line and column information:

if not dtd.validate(doc):
    print(dtd.error_log.last_error)

DTDs cannot express datatype constraints the way XSD can, and they have no notion of a target namespace in the same sense. A DTD declares element and attribute lists globally, so namespace prefixes in the document are treated as part of the names unless the DTD accounts for them. For new integrations, XSD is usually the better choice; DTD validation is most relevant for legacy formats that ship with a DTD.

Common Failure Modes

The most frequent errors in namespace-aware lxml code:

Undeclared prefix in XPath. Calling xpath('//o:item') without passing namespaces raises XPathEvalError: Undefined namespace prefix. The prefix must be declared in the namespaces argument, not inferred from the document.

Missing namespace in a lookup. find('item') on a document with a default namespace returns None silently. There is no exception, so the bug appears as a later AttributeError on None.

Schema and document namespace mismatch. An XSD with targetNamespace will not validate a document that omits the namespace, and a document with a namespace will not validate against a schema without one. Check both the document root's namespace and the schema's target namespace when validation fails unexpectedly.

Prefix mismatch during serialization. Two documents with identical namespace URIs but different prefixes are equivalent to a parser, but a downstream consumer that naively matches on the prefix string may reject them. Use register_namespace to control the emitted prefix.

Performance and Production Considerations

Compiling an XMLSchema object parses and processes the entire XSD, which is noticeably more expensive than parsing a single document. In a request handler, build the schema once at module load and reuse it for every document:

_SCHEMA = etree.XMLSchema(etree.parse('orders.xsd'))

def validate_order(doc):
    _SCHEMA.assertValid(doc)

The same applies to DTD objects. Recreating them per request repeats work that does not change between calls.

Validation cost scales with document size and schema complexity. For large feeds, validate once per document rather than per element, and let the schema error log point to the failing path instead of locating it manually.

When the input is untrusted, parse with a parser that does not resolve external entities unless the format requires it:

parser = etree.XMLParser(resolve_entities=False, no_network=True)
doc = etree.parse(source, parser)

This limits the parser's exposure to external entity resolution while keeping namespace and schema handling intact.