[sr-dev] Why fetch docbook stylesheets remotely?

[Jan Janak]

On Fri, Oct 16, 2009 at 5:18 PM, Jan Janak <jan at ryngle.com> wrote:
> On Fri, Oct 16, 2009 at 4:32 PM, Olle E. Johansson <oej at edvina.net> wrote:
>>>
>>> The catalog file contains references for Docbook 4.3, have you changed
>>> the docbook version number in documentation files perhaps? If yes then
>>> this could be the reason why the build system tries to download XSL
>>> stylesheets from the internet.
>>
>> Checking modules directory I see that we have both 4.2 and 4.4 used. Maybe
>> we should standardize
>> and make sure that we have the 4.4 files referenced.
>
> Changing docbook version only makes sense if you need to use any of
> the bug-fixes or improvements done between 4.2 and 4.4 in XML source
> files, that means in docbook sources. Those changes fix minor bugs in
> the schema and sometimes allow some elements to be used in new
> contexts. In other words, you only need to change docbook version if
> you want to use something that has been specified after 4.2 and it
> makes your docbook documents invalid.
>
> Changing Docbook version does not change the version of XSL
> stylesheets that are used to generate various output formats, such as
> HTML or READMEs. These are configured in catalog files and in *.xsl
> files in sip-router/docbook.
>
> If you want to change docbook versions anyway then the latest
> compatible version is 4.5. There is also version 5.0, but that would
> require changes in source XML files because this version is not
> backwards compatible anymore.
>
> Also, if you do the change, please wait until the master branch is unfrozen.

After second reading of emails in this thread I realized that my
earlier explanation of the Docbook system is not only confusing, but
also incorrect. I apologize, please forget what I wrote in the email
sent yesterday and let me try again.

The usual text at the beginning of Docbook source documents

<!DOCTYPE section PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
   "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">

determines the particular version of the DTD (Document Type
Definition) the Docbook document tries stay compatible with. This
information is only used if you try to validate your Docbook document
against the schema, for example by running make check in sip-router or
if you use an editor which performs document validation on the fly as
you edit the document.

The HTTP URL in the example above refers to the DTD file online, this
is the recommended way of writing DTD declarations in Docbook XML
files. If you actually try to download the document then you get the
DTD file for Docbook version 4.2.

If you try to validate a Docbook document in sip-router by typing
"make check" in a doc directory then the makefile system runs xmllint
to validate the document. Xmllint uses the document type declaration
presented above to locate the DTD file needed for the validation. But
before it tries to download it using the HTTP URL above, the tool
consults the file sip-router/docbook/catalog.xml to see if there is
perhaps a local copy of the file already on a local disk.

The file catalog.xml is a XML catalog file. XML catalog files are used
to map document type declarations (like the one above) to local files.
XML catalogs can map both type of identifiers, the textual one
"//OASIS//DTD.." and the HTTP URL as well. If the catalog file has a
reference to  a local DTD file then Xmllint uses the local file and
does not try to download it from the HTTP URL. If there is no local
copy of the DTD file then Xmllint downloads it using the HTTP URL
above.

The XML catalog file in sip-router/docbook/catalog.xml refers to
system-wide "standard" catalog files by default. On Debian (and
perhaps other Linux/GNU based systems) the system-wide catalog is
/etc/xml/catalog.

On Debian systems the system-wide catalog file is updated by the
package manager. That means if you install the package docbook-xml,
which contains Docbook DTD files for all 4.x versions, the DTD files
from that package become automatically available to the Docbook build
system in sip-router and it never has to download anything from the
internet.

On systems that do not have automatically updated system-wide catalog
files you can edit the file sip-router/docbook/catalog.xml and map the
document type declaration presented above to some local file that you
downloaded manually.

XML catalog files conveniently collect all the information about
locally available DTD and XSL files in one place, so that you do not
have to edit multiple Docbook source files when the configuration of
your system changes.

That was validation which we normally do not use much, only when
someone edits a Docbook file and wants to make sure that the document
structure is valid. Having a valid Docbook document is good before you
try to convert it to other formats, but not strictly required.

The Docbook build system works slightly differently if you try to
convert a Docbook source file to other formats, such as module
READMEs, plain-text or HTML files. To convert Docbook documents to
other formats you need Docbook XSL stylesheets. This is a set of XML
files that describe how individual elements in Docbook sources can be
converted to HTML, XHTML, and FO (Formatting Objects--used to generate
PDFs).

Because Docbook source files contain no information about version and
whereabouts of Docbook XSL stylesheets that can be used to convert
them to other formats, we need to put this information somewhere else
in our Docbook build system. That "somewhere else" is in files with
xsl extension in sip-router/docbook. There is one xsl file for each
output format we support in the build system. When you type make
README then the build system looks for file
sip-router/docbook/readme.xsl, when you type make html then it looks
for file sip-router/docbook/html.xsl, and so on.

If you actually look at the content of the file
sip-router/docbook/readme.xsl, you will see that it is very simple.
The file only contains a bunch of configuration options (Docbook XSL
stylesheets have configuration options that can be used to fine-tune
the output) customized to make README files look nice. The file also
imports other XSL stylesheets and one of them is identified by the
following HTTP URL:

http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl

This is the canonical URL of the current version of Docbook XSL
stylesheets. And here again, we use the XML catalog system to map that
HTTP URL to a local file. XSL stylesheets are processed by the tool
xsltproc and when the tool sees a URL like that, it is configured to
look into the catalog file sip-router/docbook/catalog.xml to see if it
refers to a local copy of the XSL stylesheet.

On Debian systems Docbook XSL stylesheets are available in the package
docbook-xsl. The package manager again updates the system-wide catalog
file if you install the package, so it becomes instantly available to
our Docbook build system. On other systems you can download the latest
copy of the XSL stylesheets manually and enter its location into the
sip-router catalog file.

Note that Docbook XSL stylesheets are big, the whole package comprises
many files and that's why it takes such a long time to download them
from the internet if you do not have a local copy.

Also note that there is no strict relationship between the version of
Docbook schema and the version of Docbook XSL stylesheets. There is no
such thing as XSL stylesheets for Docbook 4.2, XSL stylesheets for
Docbook 4.5, and so on. They are the same for all Docbook schema
versions. You should always use the latest version of Docbook XSL
stylesheets available, regardless of the Docbook schema version your
documents use.

  -- Jan