How to Automate EPUB 3 Navigation Documents from Content Headings?

EPUB specifications (W3C Recommendation 25 May 2023) mandate the inclusion of an EPUB Navigation Document conforming to the application/xhtml+xml media type and declared in the package manifest (content.opf) with the properties="nav" attribute. Modern commercial distributors, however, maintain dual-reading system constraints: legacy e-readers and low-overhead firmware engines (e.g., Adobe Mobile SDK integrations, older Kindle sideloading targets, older Kobo models) continue to require the EPUB 2 Navigation Control File for XML (toc.ncx) declared via the <spine toc="ncx"> attribute.

Dual-navigation maintenance routinely breaks continuous integration (CI) pipelines due to three critical divergence failure modes:

Manifest and PlayOrder Desynchronization: An entry declared in toc.ncx with an integer playOrder that diverges from the structural sequence of the <nav epub:type="toc"> ordered list triggers severe EPUBCheck validation errors (e.g., RSC-005 or reading-order navigation faults).
Structural Ill-Formedness: EPUB 3.3 strictly dictates that the navigation document’s TOC structure must consist exclusively of nested <ol> containers wrapped inside <li> nodes. Inserting a direct child <ol> inside another <ol> without an enclosing parent <li> violates the EPUB Navigation Document Schema (W3C EPUB 3.3 § 6.2).
Broken Landmark and Fragment Binding: While toc.ncx targets bare filenames or URI fragments, EPUB 3 requires distinct semantic partitions—<nav epub:type="toc">, <nav epub:type="landmarks">, and optionally <nav epub:type="page-list">. Missing or non-existent fragment IDs inside target content documents manifest as broken reference errors (RSC-007 and RSC-012).

Manual synchronization across dual files during fast-paced editorial revisions introduces drift. The deterministic solution is to treat content XHTML files as single sources of truth, extracting structural heading hierarchies (<h1> through <h6>) and generating both nav.xhtml and toc.ncx programmatically.

+-------------------------------------------------------------+
|               XHTML Source Documents (OEBPS/*.xhtml)        |
|  - Parse DOM (H1-H6)                                        |
|  - Resolve ID attributes (inject deterministic IDs if none) |
+------------------------------+------------------------------+
                               |
                               v
+-------------------------------------------------------------+
|              Heading Tree Parser & Normalizer               |
|  - Construct absolute tree with hierarchical nesting        |
|  - Flatten playOrder sequences                              |
+------------------------------+------------------------------+
                               |
               +---------------+---------------+
               v                               v
+-----------------------------+ +-----------------------------+
|  W3C EPUB 3.3 `nav.xhtml`   | |     EPUB 2 `toc.ncx`        |
|  - <nav epub:type="toc">    | |  - <navMap>                 |
|  - <nav epub:type="landmarks| |  - <navPoint id=...         |
|  - Pure nested <ol>/<li>    | |    playOrder=N>             |
+-----------------------------+ +-----------------------------+

Executable Implementation

The following production script, generate_epub_nav.py, parses an exploded EPUB’s content documents using lxml, builds an internal heading tree, assigns predictable fragment identifiers where omitted, and renders fully synchronized nav.xhtml and toc.ncx artifacts.

#!/usr/bin/env python3
"""
generate_epub_nav.py
Generates W3C-compliant nav.xhtml (EPUB 3.3) and toc.ncx (EPUB 2 fallback)
from exploded EPUB content documents.
"""

import sys
from pathlib import Path
from lxml import etree

XHTML_NS = "http://www.w3.org/1999/xhtml"
EPUB_NS = "http://www.idpf.org/2007/ops"
NCX_NS = "http://www.daisy.org/z3986/2005/ncx/"

NSMAP_NAV = {None: XHTML_NS, "epub": EPUB_NS}
NSMAP_NCX = {None: NCX_NS}


class NavNode:
    def __init__(self, title, href, level):
        self.title = title.strip()
        self.href = href
        self.level = level
        self.children = []


def extract_headings_from_document(xhtml_path: Path, base_dir: Path):
    parser = etree.XMLParser(recover=False, remove_blank_text=True)
    tree = etree.parse(str(xhtml_path), parser)
    root = tree.getroot()

    rel_href = xhtml_path.relative_to(base_dir).as_posix()
    headings = []
    modified = False

    for idx, elem in enumerate(root.xpath("//*[re:test(local-name(), '^h[1-6]$')]",
                                          namespaces={"re": "http://exslt.org/regular-expressions"})):
        tag_name = etree.QName(elem).localname
        level = int(tag_name[1])
        elem_id = elem.get("id")

        if not elem_id:
            elem_id = f"autogen_heading_{level}_{idx + 1}"
            elem.set("id", elem_id)
            modified = True

        title_text = "".join(elem.itertext()).strip()
        if not title_text:
            continue

        headings.append(NavNode(title_text, f"{rel_href}#{elem_id}", level))

    if modified:
        xhtml_path.write_bytes(etree.tostring(tree, xml_declaration=True, encoding="utf-8", pretty_print=True))

    return headings


def assemble_hierarchy(flat_nodes):
    root_nodes = []
    stack = []

    for node in flat_nodes:
        while stack and stack[-1].level >= node.level:
            stack.pop()

        if not stack:
            root_nodes.append(node)
        else:
            stack[-1].children.append(node)
        stack.append(node)

    return root_nodes


def build_nav_ol(parent_xml, nodes):
    ol = etree.SubElement(parent_xml, f"{{{XHTML_NS}}}ol")
    for node in nodes:
        li = etree.SubElement(ol, f"{{{XHTML_NS}}}li")
        a = etree.SubElement(li, f"{{{XHTML_NS}}}a", href=node.href)
        a.text = node.title
        if node.children:
            build_nav_ol(li, node.children)


def generate_nav_xhtml(hierarchy, output_path: Path):
    html = etree.Element(f"{{{XHTML_NS}}}html", nsmap=NSMAP_NAV)
    head = etree.SubElement(html, f"{{{XHTML_NS}}}head")
    title = etree.SubElement(head, f"{{{XHTML_NS}}}title")
    title.text = "Navigation"

    body = etree.SubElement(html, f"{{{XHTML_NS}}}body")
    nav_toc = etree.SubElement(body, f"{{{XHTML_NS}}}nav", attrib={f"{{{EPUB_NS}}}type": "toc", "id": "toc"})
    h1 = etree.SubElement(nav_toc, f"{{{XHTML_NS}}}h1")
    h1.text = "Table of Contents"

    build_nav_ol(nav_toc, hierarchy)

    # Required landmarks structure for production reading systems
    nav_landmarks = etree.SubElement(body, f"{{{XHTML_NS}}}nav", attrib={f"{{{EPUB_NS}}}type": "landmarks", "hidden": "hidden"})
    ol_landmarks = etree.SubElement(nav_landmarks, f"{{{XHTML_NS}}}ol")
    if hierarchy:
        li_start = etree.SubElement(ol_landmarks, f"{{{XHTML_NS}}}li")
        a_start = etree.SubElement(li_start, f"{{{XHTML_NS}}}a", href=hierarchy[0].href, attrib={f"{{{EPUB_NS}}}type": "bodymatter"})
        a_start.text = "Start of Content"

    output_path.write_bytes(etree.tostring(html, xml_declaration=True, encoding="utf-8", pretty_print=True))


def generate_toc_ncx(flat_nodes, output_path: Path, book_uid: str = "urn:uuid:code-zeba-default"):
    ncx = etree.Element(f"{{{NCX_NS}}}ncx", version="2005-1", nsmap=NSMAP_NCX)
    head = etree.SubElement(ncx, f"{{{NCX_NS}}}head")
    etree.SubElement(head, f"{{{NCX_NS}}}meta", name="dtb:uid", content=book_uid)
    etree.SubElement(head, f"{{{NCX_NS}}}meta", name="dtb:depth", content="3")
    etree.SubElement(head, f"{{{NCX_NS}}}meta", name="dtb:totalPageCount", content="0")
    etree.SubElement(head, f"{{{NCX_NS}}}meta", name="dtb:maxPageNumber", content="0")

    doc_title = etree.SubElement(ncx, f"{{{NCX_NS}}}docTitle")
    etree.SubElement(doc_title, f"{{{NCX_NS}}}text").text = "Table of Contents"

    nav_map = etree.SubElement(ncx, f"{{{NCX_NS}}}navMap")

    for order, node in enumerate(flat_nodes, start=1):
        nav_point = etree.SubElement(nav_map, f"{{{NCX_NS}}}navPoint", id=f"navpoint-{order}", playOrder=str(order))
        nav_label = etree.SubElement(nav_point, f"{{{NCX_NS}}}navLabel")
        etree.SubElement(nav_label, f"{{{NCX_NS}}}text").text = node.title
        etree.SubElement(nav_point, f"{{{NCX_NS}}}content", src=node.href)

    output_path.write_bytes(etree.tostring(ncx, xml_declaration=True, encoding="utf-8", pretty_print=True))


def main(oebps_dir_str: str):
    oebps_dir = Path(oebps_dir_str).resolve()
    xhtml_files = sorted([f for f in oebps_dir.glob("*.xhtml") if f.name != "nav.xhtml"])

    flat_nodes = []
    for xhtml in xhtml_files:
        flat_nodes.extend(extract_headings_from_document(xhtml, oebps_dir))

    hierarchy = assemble_hierarchy(flat_nodes)

    generate_nav_xhtml(hierarchy, oebps_dir / "nav.xhtml")
    generate_toc_ncx(flat_nodes, oebps_dir / "toc.ncx")
    print(f"Generated {len(flat_nodes)} navigation nodes across nav.xhtml and toc.ncx.")


if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.exit("Usage: python3 generate_epub_nav.py <path_to_exploded_oebps_dir>")
    main(sys.argv[1])

Automated Validation

Integrate deterministic validation into the publishing workflow. Execute XML well-formedness verification, schema constraints check, and strict EPUB validation assertions:

# Assert valid XML structure for generated nav.xhtml
xmllint --noout --nonet OEBPS/nav.xhtml

# Assert valid XML structure for generated fallback toc.ncx
xmllint --noout --nonet OEBPS/toc.ncx

# Repackage the EPUB according to ISO/IEC 21320-1 (uncompressed mimetype first)
zip -X0 output.epub mimetype
zip -Xr9D output.epub META-INF OEBPS

# Execute strict preflight validation with EPUBCheck CLI
epubcheck output.epub --mode exp --severity error

epubcheck must execute with a return code of 0. An exit status greater than zero flags unlinked fragments, missing landmarks, or content.opf property declaration defects.

Production Gotchas

Bare Self-Closing Tags in Namespaced Nav Elements: When generating nav.xhtml using naive string interpolation or non-XML-aware templating engines (such as Jinja2 configured for raw HTML), <nav epub:type="toc"></nav> is frequently rendered as <nav epub:type="toc" />. Some e-ink rendering engines (e.g., older NetFront implementations) drop self-closing non-void XHTML nodes entirely, resulting in blank navigation panes and unnavigable files.
Missing hidden Attribute on Structural Landmarks: EPUB 3.3 § 6.2.2 permits the <nav epub:type="landmarks"> element to clarify reading boundaries (cover, bodymatter, toc). If the hidden="" (or hidden="hidden") attribute is omitted from this landmark block, reading systems render duplicate, non-styled navigation links visibly at the top or bottom of the reader viewport.
Implicit Heading Level Skips: Direct structural jumps from <h1> to <h3> inside an arbitrary content XHTML chapter break the recursive nesting logic of downstream navigation parsers. If the hierarchy engine does not automatically promote or nest phantom nodes, the output XML produces orphaned <ol> lists that violate schema constraints and trigger epubcheck rule RSC-005.

Executable Implementation

Automated Validation

Production Gotchas

You may also like

How to Automate EPUB 3 Navigation Documents from Content Headings?

How to Validate JATS Contrib Affiliations and Persistent IDs?

Structuring Chapter Metadata in BITS 2.1 Monographs