Manual validation within desktop authoring environments such as Oxygen XML Editor introduces catastrophic latency and human error into automated journal ingestion pipelines. When publishing workflows rely on manual validation steps, malformed XML, unresolvable public identifiers, invalid element sequences, and editorial rule violations inevitably bypass gatekeeping layers. These corrupt payloads trigger downstream failures during Crossref DOI registration, PubMed Central (PMC) automated deposit, and archival packaging.
Production-grade journal ingest requires deterministic, non-interactive validation against two structural standards:
- Syntactic and Structural Compliance: NISO JATS 1.3 (ANSI/NISO Z39.96-2021) Journal Archiving and Interchange DTD. The XML instance must strictly adhere to the declared element nesting models, attribute specifications, and parameter entity constraints.
- Semantic and Business-Rule Compliance: ISO/IEC 19757-3:2020 (Information technology Document Schema Definition Languages (DSDL) Part 3: Rule-based validation using Schematron) and JATS4R (JATS for Reuse) recommendations. This layer checks contextual requirements beyond DTD capabilities, such as enforcing specific
@pub-id-typeattributes for DOIs, validating ISO 8601 publication dates, and ensuring required metadata relationships between<contrib>and<aff>nodes.
To eliminate GUI dependencies, this pipeline executes entirely in headless continuous integration, utilizing xmllint with local OASIS XML catalogs for offline DTD resolution, coupled with a Python-driven XSLT processing step to evaluate ISO Schematron rules against standard JATS instances.
Pipeline Architecture
Incoming JATS XML Ingestion Batch
│
├── [Stage 1: DTD Syntax & Entity Resolution]
│ ├── xmllint CLI invocation
│ ├── OASIS XML Catalog Resolution (Local DTD caching; network disabled)
│ └── Exit Code Assertion (Non-zero halts pipeline)
│
└── [Stage 2: ISO Schematron Semantic Validation]
├── Python ISO Schematron Processor (Saxon-HE / lxml pipeline)
│ ├── Compile Schematron (.sch) -> XSLT Validator (ISO Skeleton)
│ └── Transform JATS XML -> SVRL (Schematron Validation Report Language)
├── SVRL AST Parsing (Filter: //svrl:failed-assert | //svrl:successful-report)
└── Exit Code Assertion (Emits SARIF/JUnit diagnostics; fails if violations exist)The pipeline executes as a zero-network, multi-stage gatekeeper inside GitHub Actions:
- Network-Isolated Catalog Resolution: Network calls (
--nonet) are explicitly forbidden. An OASIS catalog routes NISO public and system identifiers directly to pinned local DTD definitions, blocking external entity injection (XXE) and network timeout flakiness. - SVRL Extraction: ISO Schematron rules are compiled via the official ISO Schematron reference implementation XSLT pipelines into an evaluation stylesheet. The Python engine evaluates the target XML against this stylesheet to generate a Schematron Validation Report Language (SVRL) XML document (ISO/IEC 19757-3 Annex D).
- Structured Failure Reporting: Any presence of
svrl:failed-assertorsvrl:successful-reportyields a non-zero exit code, formatting line numbers, context XPath expressions, and violation flags directly into GitHub Actions error annotations.
Executable Implementation
- OASIS XML Catalog (
catalog.xml): Store this file at the repository root alongside the pinned NISO JATS 1.3 DTD schemas:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE catalog PUBLIC "-//OASIS//DTD XML Catalogs V1.1//EN"
"http://www.oasis-open.org/committees/entity/release/1.1/catalog.dtd">
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
<public publicId="-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.3 20210610//EN"
uri="schemas/jats-1.3/JATS-archivearticle1-3.dtd"/>
<public publicId="-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN"
uri="schemas/jats-1.3/JATS-journalpublishing1-3.dtd"/>
<system systemId="JATS-archivearticle1-3.dtd"
uri="schemas/jats-1.3/JATS-archivearticle1-3.dtd"/>
<system systemId="JATS-journalpublishing1-3.dtd"
uri="schemas/jats-1.3/JATS-journalpublishing1-3.dtd"/>
</catalog>- Headless Schematron Runner (
scripts/validate_schematron.py): This script usessaxonche(Saxon-HE for Python) to execute XSLT 2.0/3.0 ISO Schematron pipelines deterministically:
#!/usr/bin/env python3
"""
validate_schematron.py
Compiles ISO Schematron rules to XSLT and executes assertions against JATS XML instances.
Emits GitHub Actions log formatting and exits non-zero on failed assertions.
"""
import sys
import os
import argparse
from pathlib import Path
import xml.etree.ElementTree as ET
from saxonche import PySaxonProcessor
SVRL_NS = {"svrl": "http://purl.oclc.org/dsdl/svrl"}
def run_schematron_validation(xml_path: Path, compiled_xslt_path: Path) -> int:
with PySaxonProcessor(license=False) as proc:
xslt_proc = proc.new_xslt30_processor()
# Load compiled Schematron validator XSLT
executable = xslt_proc.compile_stylesheet(stylesheet_file=str(compiled_xslt_path))
# Transform XML directly to SVRL in memory
svrl_output = executable.transform_to_string(source_file=str(xml_path))
root = ET.fromstring(svrl_output)
failed_asserts = root.findall(".//svrl:failed-assert", SVRL_NS)
successful_reports = root.findall(".//svrl:successful-report", SVRL_NS)
violations = failed_asserts + successful_reports
if not violations:
print(f"[PASS] Schematron assertions passed: {xml_path.name}")
return 0
print(f"::group::Schematron Violations: {xml_path.name}", file=sys.stderr)
for v in violations:
location = v.attrib.get("location", "unknown-xpath")
role = v.attrib.get("role", "error")
text_node = v.find("svrl:text", SVRL_NS)
diagnostic = text_node.text.strip() if text_node is not None and text_node.text else "No message provided."
# Emit native GitHub Actions workflow commands
level = "error" if role.lower() in ("error", "fatal") else "warning"
print(f"::{level} file={xml_path},title=Schematron Assertion::{diagnostic} at {location}", file=sys.stderr)
print("::endgroup::", file=sys.stderr)
return len(violations)
def main():
parser = argparse.ArgumentParser(description="Headless JATS Schematron Preflight CLI")
parser.add_argument("--xml", required=True, type=Path, help="Target JATS XML document")
parser.add_argument("--compiled-xslt", required=True, type=Path, help="Pre-compiled Schematron XSLT")
args = parser.parse_args()
if not args.xml.is_file():
print(f"Error: Target file {args.xml} not found.", file=sys.stderr)
sys.exit(2)
exit_code = run_schematron_validation(args.xml, args.compiled_xslt)
sys.exit(1 if exit_code > 0 else 0)
if __name__ == "__main__":
main()- GitHub Actions Pipeline Configuration (
.github/workflows/jats-preflight.yml):
name: JATS Automated Production Preflight
on:
pull_request:
paths:
- 'articles/**/*.xml'
push:
branches:
- main
paths:
- 'articles/**/*.xml'
jobs:
validate-jats:
runs-on: ubuntu-latest
env:
XML_CATALOG_FILES: ${{ github.workspace }}/catalog.xml
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Install Prepress Tooling Dependencies
run: |
sudo apt-get update
sudo apt-get install --no-install-recommends -y libxml2-utils
python3 -m pip install --upgrade pip
pip install saxonche
- name: Stage JATS 1.3 DTD Schemas
run: |
mkdir -p schemas/jats-1.3
curl -sSL "https://jats.nlm.nih.gov/archiving/1.3/JATS-archive-oasis-article1-3-mathml3-dtd.zip" -o jats.zip
unzip -q jats.zip -d schemas/jats-1.3/
- name: Compile ISO Schematron Pipeline to XSLT
run: |
mkdir -p build/schematron
# Leverage ISO standard pipeline skeleton to transform .sch to executable .xsl
python3 -c '
from saxonche import PySaxonProcessor
with PySaxonProcessor(license=False) as proc:
xslt = proc.new_xslt30_processor()
# Step 1: Include expansion
step1 = xslt.compile_stylesheet(stylesheet_file="rules/iso-schematron/iso_dsdl_include.xsl")
res1 = step1.transform_to_string(source_file="rules/jats4r.sch")
# Step 2: Abstract pattern expansion
step2 = xslt.compile_stylesheet(stylesheet_file="rules/iso-schematron/iso_abstract_expand.xsl")
res2 = step2.transform_to_string(source_text=res1)
# Step 3: Produce validation XSLT
step3 = xslt.compile_stylesheet(stylesheet_file="rules/iso-schematron/iso_svrl_for_xslt2.xsl")
final_xslt = step3.transform_to_string(source_text=res2)
with open("build/schematron/jats4r-compiled.xsl", "w") as f:
f.write(final_xslt)
'
- name: Stage 1 - Batch DTD Validation (xmllint)
run: |
FAILURES=0
for xml_file in articles/**/*.xml; do
[ -e "$xml_file" ] || continue
echo "::group::Validating DTD for ${xml_file}"
xmllint --noout --valid --catalogs --nonet "$xml_file" || ((FAILURES++))
echo "::endgroup::"
done
if [ $FAILURES -ne 0 ]; then
echo "::error::DTD Validation failed on $FAILURES files."
exit 1
fi
- name: Stage 2 - Batch Schematron Semantic Preflight
run: |
FAILURES=0
for xml_file in articles/**/*.xml; do
[ -e "$xml_file" ] || continue
python3 scripts/validate_schematron.py \
--xml "$xml_file" \
--compiled-xslt "build/schematron/jats4r-compiled.xsl" || ((FAILURES++))
done
if [ $FAILURES -ne 0 ]; then
echo "::error::Schematron preflight failed on $FAILURES files."
exit 1
fiAutomated Validation
To locally verify an individual XML asset without internet access, run the validation sequence directly in the terminal:
# 1. Assert DTD compliance using offline catalog
XML_CATALOG_FILES="./catalog.xml" xmllint --noout --valid --catalogs --nonet articles/PMC99281.xml
echo "DTD Exit Code: $?"
# 2. Assert ISO Schematron compliance and extract SVRL
python3 scripts/validate_schematron.py \
--xml articles/PMC99281.xml \
--compiled-xslt build/schematron/jats4r-compiled.xsl
echo "Schematron Exit Code: $?"An execution that satisfies all production requirements returns zero output for xmllint and logs the validation pass:
[PASS] Schematron assertions passed: PMC99281.xml
Schematron Exit Code: 0If an author leaves off an @assigning-authority or formats an invalid DOI, the pipeline traps the error and pinpoints the node’s XPath:
::group::Schematron Violations: PMC99281.xml
::error file=articles/PMC99281.xml,title=Schematron Assertion::DOI must start with '10.' and contain a directory slash at /article[1]/front[1]/article-meta[1]/article-id[1]
::endgroup::Production Gotchas
- Catalog Path Misconfiguration and Silent Fallbacks: When
XML_CATALOG_FILESpoints to an incorrect path or has improper URI mappings,xmllintattempts to fetch the DTD from[http://jats.nlm.nih.gov/](http://jats.nlm.nih.gov/)...over the network. If the--nonetflag is omitted, the CI runner hangs waiting on network timeouts, or passes conditionally while masking local schema errors. Always enforce--nonetto catch unresolved entities, missing DTD files, and broken catalog URIs before reaching deployment. - Saxon-HE vs. Libxml2 XSLT 1.0 Processor Drift: Standard Linux utilities rely on
xsltproc(which is strictly tied to libxslt and XSLT 1.0). Modern ISO Schematron schemas (especially JATS4R rules and Schematron 2016+ features) heavily utilize regular expressions viamatches(), sequence manipulation viacurrent-group(), and XPath 2.0 functions. Running modern Schematron files throughxsltprocfails with syntax exceptions during rule compilation. Production pipelines must pin an XSLT 2.0/3.0 engine such assaxoncheto maintain compatibility with modern standard Schematron implementations. - XML Character Entity Decomposition (Unicode Form Bugs): Incoming files authored across divergent operating systems frequently introduce Unicode normalization mismatches (NFC vs. NFD), or contain undeclared legacy MathML HTML-like entities (
±,α) that are not present in standard XML. Unless the DTD explicitly defines these character entities or pre-processing normalizes files into standard UTF-8 NFC codepoints,xmllintthrows fatal XML parsing errors on entity references before Schematron processing can begin.
