Managing large projects

XInclude

Instructions for large fonts and font families may be split into parts so as to speed compilation for testing purposes, distribute work among members of a team, or create libraries of shared functions. Xgridfit's mechanism for splitting projects is XInclude, the W3C "XML Inclusions" specification. XInclude is extremely flexible and simple to use: it allows you to structure a complex project in a number of ways.

To use XInclude in an Xgridfit file requires just two steps. First, include the XInclude namespace declaration in the <xgridfit> element:

    <xgridfit xmlns="http://xgridfit.sourceforge.net/Xgridfit2"
              xmlns:xi="http://www.w3.org/2001/XInclude">

Next, to merge part of another file into the file you're editing, add an <xi:include> element:

    <xi:include href="Junicode-Regular-Basic.xgf"
                xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                xpointer(/x:xgridfit/x:glyph)"/>

Here the href attribute (a URI) points to a file. The xpointer attribute points to the element or elements that we want to import into the current file; it is an XPointer (usually a simple XPath expression). In the present example, the XPointer expression pulls in all the <glyph> elements from the file Junicode-Regular-Basic.xgf.

It is necessary to specify the Xgridfit namespace for each <xi:include> element, and this makes the <xi:include> tag rather verbose; but much of this is boilerplate which can be copied from tag to tag. Indeed, the only variable part of the xpointer attribute is inside the final set of parentheses.

Several Xgridfit elements can take an xml:id attribute, and can be included by referencing that id: <xgridfit>, <pre-program>, <function>, <macro>, <glyph>. To fetch this <function> element

    <function name="myfunc" xml:id="jun-reg-myfunc">
       . . .
    </function>

use one of these <xi:include> elements:

    <xi:include href="Junicode-Regular-common.xgf" xpointer="jun-reg-myfunc"/>

    <xi:include href="Junicode-Regular-common.xgf#jun-reg-myfunc"/>

Notice that you do not need to worry about namespaces when you use the xml:id method.

Some XSLT engines can handle XInclude and others cannot. The XInclude capability in xsltproc (the engine invoked by the xgridfit shell script) is turned on by simply including the parameter --xinclude on the command line (this is done for you in the shell script). For some other engines, turning on XInclude capability is an arcane and difficult matter: you must check the documentation for your preferred engine. The xgridfit shell script uses xmllint when necessary to work around difficulties with XSLT engines and XML validators.

Example of a large project

XInclude is flexible enough to permit you to structure your project in many different ways (you must just remember to avoid recursive includes). Here is a simple example involving the Junicode font (complete source code available at the download site).

Instructions for Junicode-Regular are compiled by running xgridfit against a master file, Junicode-Regular.xgf, which contains little besides <xi:include> elements:

    <?xml version="1.0"?>
    <xgridfit xmlns:xi="http://www.w3.org/2001/XInclude">
      <xi:include href="Junicode-Regular-common.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:infile)"/>
      <xi:include href="Junicode-Regular-common.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:outfile)"/>
      <xi:include href="Junicode-Regular-common.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:constant)"/>
      <xi:include href="Junicode-Regular-common.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(//x:control-value)"/>
      <xi:include href="Junicode-Regular-common.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(//x:function)"/>
      <xi:include href="Junicode-Regular-common.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(//x:macro)"/>
      <xi:include href="Junicode-Regular-common.xgf" xpointer="jun-reg-prep"/>
      <xi:include href="Junicode-Regular-Basic.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-Latin1.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-LatExtA.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-LatExtA-dotlessi.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-LatExtB.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-IPA.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-SpacMod.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-CombDiac.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                              xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-LatExtAdd.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-GenPunct.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-Greek.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-PUA.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
      <xi:include href="Junicode-Regular-NoEncode.xgf"
                  xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                            xpointer(/x:xgridfit/x:glyph)"/>
    </xgridfit>

The file Junicode-Regular-common.xgf contains no <glyph> elements, but only global elements: <infile>, <outfile>, <constant>, <control-value>, <function>, <macro>, <pre-program>. These are included with the first seven xi:include elements (the <pre-program> is included by xml:id, just by way of illustration). The other files contain only <glyph> elements organized by Unicode range. These are included with a specific XPointer, /x:gridfit/x:glyph, rather than //x:glyph, to avoid pulling in <glyph> elements inside of <no-compile> elements. Each file validates against the Relax NG schema xgridfit.rnc.

When working with any particular Unicode range, most or all of the other <xi:include> elements can be commented out in the master file, speeding compilation. If more than one developer were working on this font, the file Junicode-Regular-common.xgf might be controlled by the lead developer so as to avoid conflicts introduced by individual contributors, who when necessary would submit patches on that file for approval.

Shared functions and libraries

You very likely have some favorite functions that you insert in all your fonts. In addition, similar members of a font family are likely to share functions. For a library of shared functions, simply create an Xgridfit program file in which the only children of the <xgridfit> element are <function> elements. An Xgridfit program file can very easily merge the functions from this library anywhere in the sequence of your own functions:

    <function name="local-function">
      . . .
    </function>

    <xi:include href="My-Library.xgf#xpointer(//function)"/>

    <function name="another-local-function">
      . . .
    </function>

Alternatively, you can merge a single function from the library if you have supplied that function with an xml:id attribute:

    <function name="local-function">
      . . .
    </function>

    <xi:include href="My-Library.xgf#set-left-sidebearing"/>

    <function name="another-local-function">
      . . .
    </function>

Note that the included function must be called by its required name attribute and not by the xml:id attribute, which is needed only when a function is merged, before it gets compiled. The name and xml:id attributes should probably be the same when both are present.

Separate compilation of source files

Though an Xgridfit file is destined to be included in a master file, it can also be compiled separately. This may be convenient when writing instructions for a specific range of glyphs. To prepare a file for separate compilation, simply use XInclude to pull in global elements from the common file. For example, one of the Junicode source files containing only glyphs has these XIncludes:

  <xi:include href="Junicode-Regular-common.xgf"
	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
			xpointer(/x:xgridfit/x:constant)"/>
  <xi:include href="Junicode-Regular-common.xgf"
	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
			xpointer(//x:control-value)"/>
  <xi:include href="Junicode-Regular-common.xgf"
	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
			xpointer(//x:function)"/>
  <xi:include href="Junicode-Regular-common.xgf"
	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
			xpointer(//x:pre-program)"/>
  <xi:include href="Junicode-Regular-common.xgf"
	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
			xpointer(//x:macro)"/>

If any glyph program in the file needs to refer to a glyph program in another file, this can also be pulled in via XInclude. <xi:include> elements for this purpose should be placed in a <no-compile> element, which makes them visible to all the programming in the file but prevents their being compiled. This is from Junicode-Regular-Latin1.xgf:

  <no-compile>
    <xi:include href="Junicode-Regular-Basic.xgf"
                xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                          xpointer(/x:xgridfit/x:glyph)"/>
    <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapAcute"/>
    <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapCircumflex"/>
    <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapDieresis"/>
    <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapGrave"/>
    <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapTilde"/>
    <xi:include href="Junicode-Regular-SpacMod.xgf" xpointer="circumflex"/>
    <xi:include href="Junicode-Regular-SpacMod.xgf" xpointer="tilde"/>
    <xi:include href="Junicode-Regular-LatExtA-dotlessi.xgf" xpointer="dotlessi"/>
  </no-compile>

Note that any of these included glyphs will be compiled if specified in a <glyph-select> element or with the -g command-line option.

Remember that you must either take care to avoid recursive XIncludes or comment out the XIncludes in a file before including elements from it it in another file.

Another strategy for speeding development is to create a quick-and-dirty Makefile for compiling and displaying a single file of glyph programs. The following example uses merge-mode, in which it is necessary first to compile the file containing the font's common elements (functions, control-values, etc.) and run that script. The Makefile compiles the common file only when it has changed.

FONTNAME = MyFont
RANGE = IPA

COMMON = $(FONTNAME)-common
COMMONSCRIPT = $(COMMON).py
COMMONXGF = $(COMMON).xgf

GLYPH = $(FONTNAME)-$(RANGE)
GLYPHSCRIPT = $(GLYPH).py
GLYPHXGF = $(GLYPH).xgf

SOURCEFONT = $(FONTNAME).sfd

OUTPUTFONT = $(FONTNAME).ttf

INTERMEDIATEFONT = temporary.sfd

$(COMMONSCRIPT) : $(COMMONXGF)
	xgridfit -m -D -c yes -O $(COMMONSCRIPT) -o $(INTERMEDIATEFONT) \
	    -i $(SOURCEFONT) $(COMMONXGF)

$(GLYPHSCRIPT) : $(GLYPHXGF)
	xgridfit -m -c no -O $(GLYPHSCRIPT) -o $(OUTPUTFONT) \
	    -i $(INTERMEDIATEFONT) $(GLYPHXGF)

$(INTERMEDIATEFONT) : $(COMMONSCRIPT) $(SOURCEFONT)
	fontforge -script $(COMMONSCRIPT)

$(OUTPUTFONT) : $(COMMONSCRIPT) $(GLYPHSCRIPT) $(INTERMEDIATEFONT)
	fontforge -script $(GLYPHSCRIPT)

When you type make MyFont.ttf, this Makefile does the following:

Compiles MyFont-common.xgf to make MyFont-common.py
Compiles MyFont-IPA.xgf to make MyFont-IPA.py
Runs the script MyFont-common.py against MyFont.sfd to create temporary.sfd
Runs the script MyFont-IPA.py against temporary.sfd to make MyFont.ttf

This font contains all the glyphs in MyFont.sfd, but TrueType instructions only for the glyphs in the IPA range. To do the same with a file containing glyphs for another range, you need only change the RANGE variable near the top of the file.