SME Server:Documentation:Developers Manual:Chapter8
Configuration file templates
Design of the template system
Every piece of software has its own configuration format, and writing parsers for each one is a complex, time-consuming and error-prone process. The SME Server software avoids the whole issue by using templates which generate the correct configuration.
In most cases, SME Server configuration files are over-written when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file (e.g. /etc/fstab) is also automatically updated by some other process.
Templates are stored under /etc/e-smith/templates/ in a directory hierarchy which matches the standard filesystem. For example, the template for /etc/inittab is stored in the /etc/e-smith/templates/etc/inittab/ directory. Each template is stored as a directory of template fragments and processed by the Perl Text::Template module.
The template fragments are concatenated together in ASCIIbetical order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of the SME Server's modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.
Note: It is also possible to store templates as single files, rather than as a directory of fragments. This method is preserved for backwards compatibility, but does not provide the extensibility of directory based templates. Directory templates should be used for all new templates, even if that directory only contains a single fragment.
The Text::Template module
The Text::Template module allows arbitary Perl code to be embedded in a template file by surrounding it in braces ("{" and "}"). The code inside the braces is interpreted and its return value replaces the section between, and including, the braces. For instance:
The answer is { 2 + 2 }
becomes
The answer is 4
Variables can be passed in from the program which is expanding the template, hence:
Shopping list:
{
$OUT = '';
for my $item ( qw(bread milk bananas) )
{
$OUT .= "* $item\n";
}
}
would expand to:
Shopping list: * bread * milk * bananas
The SME Server template system uses this mechanism to automatically pass in global configuration variables from the configuration database which can then be used to fill out the configuration files.
For example, the /etc/hosts template is fairly simple and composed of two fragments:
[gordonr@smebuild hosts]$ pwd /etc/e-smith/templates/etc/hosts [gordonr@smebuild hosts]$ ls 10localhost 20hostname
Let's look at those fragments. The first is a piece of static text, which Text::Template will include verbatim:
127.0.0.1 localhost
The second is more complex and relies on values from the configuration database:
{
$OUT .= "$LocalIP\t";
$OUT .= " ${SystemName}.${DomainName}";
$OUT .= " ${SystemName}";
}
Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code. When this template is expanded, it results in the following configuration file:
#------------------------------------------------------------ # !!DO NOT MODIFY THIS FILE!! # # Manual changes will be lost when this file is regenerated. # # Please read the developer's guide, which is available # at http://www.contribs.org/development/ # # Copyright (C) 1999-2006 Mitel Networks Corporation #------------------------------------------------------------ 127.0.0.1 localhost 192.168.10.1 smebuild.gormand.com.au smebuild
The header block comes "for free" as part of the template system, courtesy of an optional file template-begin, which is always processed as the first fragment. If it isn't provided, the text shown with # comments is included.
The other lines are provided by the two fragments shown above. Note the use of the configuration database variables: $LocalIP, $SystemName and $DomainName. All simple entries in the configuration database are provided as global variables to the templates.
Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments.
The complex entries in the configuration database are also provided as global variables to the templates. However, they are provided as Perl hashes instead of simple scalars. For example, here is how you might configure the Network Time Protocol (NTP) server /etc/ntp.conf file:
server { $ntpd{NTPServer} }
driftfile /etc/ntp/drift
authenticate no
The NTPServer setting is stored in the ntpd configuration database record, and so can be accessed via the hash accessor $ntpd{NTPServer}.
template-begin and template-end
Each template directory can contain two optional files template-begin and template-end . The template-begin file is always processed as the first file of the template, and the template-end file is always processed as the last file.
If the directory does not contain a template-begin file, the contents of /etc/e-smith/templates-default/template-begin is used automatically.
If the directory does not contain a template-end , nothing is appended to the template output. It is mostly used to provide the closing block for configuration files written in languages such as HTML and PHP, through a link to an entry in the templates-default/ directory.
/etc/e-smith/templates-default
The /etc/e-smith/templates-default directory contains a set of template-begin and template-end files for various languages. For example, if your template generates a perl script, you would link template-begin to /etc/e-smith/templates-default/template-begin-perl and automatically get the #!/usr/bin/perl -w line and a comment containing the contents of the default template-begin file.
[gordonr@sevendev1 devguide]$ ls /etc/e-smith/templates-default/ template-begin template-begin-perl template-end-php template-begin-html template-begin-php template-begin-pam template-begin-shell
Note: You may also need a templates.metadata configuration file if your generated file needs to be executable.
Template fragment ordering
Template fragments are assembled in ASCII-betical order, with two exceptions: template-begin always comes first, and template-end always comes last. Template fragments are often named to start with a two digit number to make the ordering obvious, but this is not required.
Note: The number of fragments and the order of those fragments within a template directory is subject to change between releases.
Templates for user home directories: templates-user
Most of the templates on the system map to single, fixed output files, such as /etc/hosts. However, templates are also used to generate configuration files such as mail delivery instructions for users. These templates are stored in the /etc/e-smith/template-user/ tree.
For example, the template for the .qmail file in user home directories (which details how mail is to be handled), is stored under /etc/e-smith/template-user/.qmail/. As these templates have a variable output filename, they are expanded using small pieces of Perl code in action scripts.
Local site overrides: templates-custom and templates-user-custom
It is possible that the standard templates are not correct for a particular installation, and so the local system administrator can override the extsing templates by placing files in the templates-custom tree. This is a parallel tree to the normal templates hierarchy, and is normally empty. There is also a template-user-custom tree for overriding entries in the templates-user tree.
Note: Never edit the standard templates. Your changes will be overwritten when packages are upgraded.
Note: The template-custom trees should be reserved for local system overrides. Software should not install files in this tree.
If a templates-custom entry exists for a template, it is merged with the standard templates directory during template expansion, using the following rules:
- If a fragment of the same name exists in both templates and templates-custom, the one from templates-custom is used, and the one from the standard templates tree is ignored.
- If the fragments in templates-custom have different names from those in templates, they are merged into the template as if they were in the templates directory.
- If the templates-custom entry is a file, rather than a directory, it completely overrides the standard template.
To make this concrete, let's assume we have the following template structure:
/etc/e-smith/templates/etc/book.conf: 10intro 30chapter3 40chapter4 80synopsis
and
/etc/e-smith/templates-custom/etc/book.conf: 30chapter3 50chapter5
The resulting template would be processed in this order:
- template-begin from /etc/e-smith/templates-default
- 10intro from /etc/e-smith/templates/etc/book.conf
- 30chapter3 from /etc/e-smith/templates-custom/etc/book.conf
- 40chapter4 from /etc/e-smith/templates/etc/book.conf
- 50chapter5 from /etc/e-smith/templates-custom/etc/book.conf
- 80synopsis from /etc/e-smith/templates/etc/book.conf
- template-end (empty), nominally from /etc/e-smith/templates-default
How to resolve conflicts with standard templates
It is possible that the standard templates may specify behaviour which is not appropriate for your application. In many cases the templates will be driven by configuration database settings which allow their behaviour to be customized, which should be the first thing to check.
In many cases, your application only needs to extend the behaviour of the template by adding one or more fragments. This should be your second option and can be achieved by simply adding your fragment in the correct place in the list of fragments.
In rare cases the standard template specifies a behaviour which conflicts with your application. In these cases, you should do all of the following:
- Create a templates-custom directory to match the existing one in the templates hierachy.
- Copy the conflicting fragment, and only that fragment, to the templates-custom directory. The fragment should have the same name in both directories. At this point you have not changed the behaviour of the system as the templates-custom entry will be preferred, but will behave identically.
- Modify the copy in templates-custom to suit your required behaviour.
- Raise a New Feature Request here: http://www.contribs.org/bugzilla/. Please attach your modified template (or even better, a patch file) and provide details of why you think that the standard template should be changed.
Subdirectory templates
It is also possible to split templates into further subdirectories. This can be very useful for evaluating the same fragments in a loop, for example for each virtual domain in httpd.conf or each ibay in smb.conf.
Two examples of this can be found in /etc/e-smith/templates/etc/httpd/conf/httpd.conf/80VirtualHosts which loops over the /etc/e-smith/templates/etc/httpd/conf/httpd.conf/VirtualHosts/ directory, and /etc/e-smith/templates/etc/smb.conf/90ibays which performs a similar loop over the /etc/e-smith/templates/etc/smb.conf/ibays/ directory.
Template expansion
Mapping templates to events: templates2expand
The SME Server is designed to ensure consistent and reliable operation, without requiring command-line access. Whenever an event is signalled, the relevant templates for that event are expanded and the services are notified of the configuration changes.
Requesting expansion of a template in an event is a simple matter of creating an empty file under the templates2expand hierarchy for that event. For example, here are the templates which are expanded during an ip-change event:
[gordonr@smebuild templates2expand]$ pwd /etc/e-smith/events/ip-change/templates2expand [gordonr@smebuild templates2expand]$ find . -type f ./etc/services ./etc/pam.d/passwd ./etc/dhcpd.conf ./etc/pptpd.conf ./etc/securetty ./etc/hosts.deny ./etc/shells ./etc/proftpd.conf ./etc/fetchmail ./etc/ppp/options.pptpd ./etc/ppp/ip-down.local ./etc/ppp/ip-up.local ./etc/hosts.allow ./etc/startmail ./var/qmail/alias/.qmail-localdelivery-default ./var/qmail/alias/.qmail-default ./var/qmail/control/concurrencylocal ./var/qmail/control/me ./var/qmail/control/virtualdomains ./var/qmail/control/smtproutes ./var/qmail/control/plusdomain ./var/qmail/control/doublebounceto ./var/qmail/control/rcpthosts ./var/qmail/control/badhelo ./var/qmail/control/databytes ./var/qmail/control/mailrules.default ./var/qmail/control/helohost ./var/qmail/control/bouncehost ./var/qmail/control/envnoathost ./var/qmail/control/defaultdomain ./var/qmail/control/locals ./var/qmail/control/bouncefrom ./var/qmail/control/defaulthost ./var/qmail/control/concurrencyremote ./home/e-smith/.qmail
It is important to note that any package can request a template expansion for an event. The list shown above has been contributed by a number of packages, and some of those packages have requested expansion of more than one template:
[gordonr@smebuild templates2expand]$ find . -type f|xargs rpm -qf | sort | uniq e-smith-base-4.15.6-01 e-smith-email-4.15.4-01 e-smith-pptpd-1.11.0-18 e-smith-proftpd-1.11.0-25 e-smith-qmail-1.9.0-11 smeserver-qpsmtpd-1.0.1-09
Template permissions and ownership: templates.metadata
Templates are normally expanded to be owned by root and are not executable, which is a reasonable default for most configuration files. However, templates may need to generate configuration files which are owned by a different user, or which need to be executable or have other special permissions. This can be done by creating a templates.metadata file which defines the additional attributes for the expansion.
For example, here is the metadata file /etc/e-smith/templates.metadata/etc/ppp/ip-up.local:
UID="root" GID="daemon" PERMS=0755
which sets the group to daemon and makes the script executable. Note that the file is readable by members of the daemon group, but it is not writable by anyone but root. It is also possible to use the same template to generate multiple output files, such as in this example:
TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX"
OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1"
MORE_DATA={ THIS_DEVICE => "eth1" }
FILTER=sub { $_[0] =~ /^#/ ? : $_[0] } # Remove comments
The templates.metadata file for route-eth0 just uses eth0 instead of eth1 on the second and third lines. Note also the FILTER setting which allows post-processing of the generated template.
There are many examples under /etc/e-smith/templates.metadata/ and the full list of options can be seen with:
perldoc esmith::templates
Manual testing: expand-template
It is sometimes useful to expand templates manually during testing, which can be done with the expand-template command. The syntax of this command is simply:
expand-template filename
where filename is the name of the configuration file you want to generate, e.g. /etc/hosts.
Perl API: processTemplate
In rare circumstances you may need to call processTemplate directly. Explicit calls to processTemplate are typically only used when the output filename is variable, such as when processing the .qmail files for each group:
use esmith::templates;
foreach my $group (@groups)
{
my $groupName = $group->key;
[...]
processTemplate(
{
CONFREF =>
{
Members => $members,
},
TEMPLATE_PATH =>
"/var/qmail/alias/.qmail-group",
OUTPUT_FILENAME => "/var/qmail/alias/.qmail-$groupName",
}
);
[...]
}
Note: Software which was written for SME Server before release 7 will have a number of scripts which call processTemplate. In almost all cases, these can be replaced with simple flag files in the templates2expand/ directory of the relevant events. The new method is far more efficient as a single invocation is perl is used to expand all template files.