2,411
edits
Changes
From SME Server
Jump to navigationJump to searchSME Server:Documentation:Developers Manual:Chapter8 (view source)
Revision as of 11:50, 25 April 2008
, 11:50, 25 April 2008Splitting the SME Server Developers Guide (and relocating)
<div class="CHAPTER">
=Configuration file templates=
<div class="SECT1">
==Design of the template system==
Every piece of software has its own configuration format, and writing parsers for each one is a complex, time-consuming and error-prone process. The SME Server software avoids the whole issue by using templates which <span class="emphasis">''generate''</span> the correct configuration.
In most cases, SME Server configuration files are <span class="emphasis">''over-written''</span> when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file (e.g. <tt class="FILENAME">/etc/fstab</tt>) is also automatically updated by some other process.
Templates are stored under <tt class="FILENAME">/etc/e-smith/templates/</tt> in a directory hierarchy which matches the standard filesystem. For example, the template for <tt class="FILENAME">/etc/inittab</tt> is stored in the <tt class="FILENAME">/etc/e-smith/templates/etc/inittab/</tt> directory. Each template is stored as a directory of template fragments and processed by the Perl <code class="CLASSNAME">Text::Template</code> module.
The template fragments are concatenated together in <span class="emphasis">''ASCIIbetical''</span> order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of the SME Server's modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.
<div class="NOTE"><blockquote class="NOTE">
'''Note: ''' It is also possible to store templates as single files, rather than as a directory of fragments. This method is preserved for backwards compatibility, but does not provide the extensibility of directory based templates. Directory templates should be used for all new templates, even if that directory only contains a single fragment.
</blockquote></div></div><div class="SECT1">
----
==The Text::Template module==
The <code class="CLASSNAME">Text::Template</code> module allows arbitary Perl code to be embedded in a template file by surrounding it in braces (<var class="LITERAL">"{"</var> and <var class="LITERAL">"}"</var>). The code inside the braces is interpreted and its return value replaces the section between, and including, the braces. For instance:
The answer is { 2 + 2 }
becomes
The answer is 4
Variables can be passed in from the program which is expanding the template, hence:
<nowiki>Shopping list:
{
$OUT = '';
for my $item ( qw(bread milk bananas) )
{
$OUT .= "* $item\n";
}
}</nowiki>
would expand to:
<nowiki>Shopping list:
* bread
* milk
* bananas</nowiki>
The SME Server template system uses this mechanism to automatically pass in global configuration variables from the <tt class="FILENAME">configuration</tt> database which can then be used to fill out the configuration files.
For example, the /etc/hosts template is fairly simple and composed of two fragments:
[gordonr@smebuild hosts]$ pwd
/etc/e-smith/templates/etc/hosts
[gordonr@smebuild hosts]$ ls
10localhost 20hostname
Let's look at those fragments. The first is a piece of static text, which <code class="CLASSNAME">Text::Template</code> will include verbatim:
127.0.0.1 localhost
The second is more complex and relies on values from the configuration database:
{
$OUT .= "$LocalIP\t";
$OUT .= " ${SystemName}.${DomainName}";
$OUT .= " ${SystemName}";
}
Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code. When this template is expanded, it results in the following configuration file:
<nowiki>#------------------------------------------------------------
# !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------
127.0.0.1 localhost
192.168.10.1 smebuild.gormand.com.au smebuild</nowiki>
The header block comes "for free" as part of the template system, courtesy of an optional file <tt class="FILENAME">template-begin</tt>, which is always processed as the first fragment. If it isn't provided, the text shown with # comments is included.
The other lines are provided by the two fragments shown above. Note the use of the configuration database variables: <var class="LITERAL">$LocalIP</var>, <var class="LITERAL">$SystemName</var> and <var class="LITERAL">$DomainName</var>. All simple entries in the configuration database are provided as global variables to the templates.
Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments.
The complex entries in the configuration database are also provided as global variables to the templates. However, they are provided as Perl hashes instead of simple scalars. For example, here is how you might configure the Network Time Protocol (NTP) server <tt class="FILENAME">/etc/ntp.conf</tt> file:
server { $ntpd{NTPServer} }
driftfile /etc/ntp/drift
authenticate no
The <var class="LITERAL">NTPServer</var> setting is stored in the <var class="LITERAL">ntpd</var> configuration database record, and so can be accessed via the hash accessor <var class="LITERAL">$ntpd{NTPServer}</var>.
<div class="SECT2">
----
===template-begin and template-end===
Each template directory can contain two optional files <tt class="FILENAME">template-begin</tt> and <tt class="FILENAME">template-end</tt> . The template-begin file is always processed as the first file of the template, and the template-end file is always processed as the last file.
If the directory does not contain a <tt class="FILENAME">template-begin</tt> file, the contents of <tt class="FILENAME">/etc/e-smith/templates-default/template-begin</tt> is used automatically.
If the directory does not contain a <tt class="FILENAME">template-end</tt> , nothing is appended to the template output. It is mostly used to provide the closing block for configuration files written in languages such as HTML and PHP, through a link to an entry in the <tt class="FILENAME">templates-default/</tt> directory.
</div><div class="SECT2">
----
===/etc/e-smith/templates-default===
The <tt class="FILENAME">/etc/e-smith/templates-default</tt> directory contains a set of template-begin and template-end files for various languages. For example, if your template generates a perl script, you would link <tt class="FILENAME">template-begin</tt> to <tt class="FILENAME">/etc/e-smith/templates-default/template-begin-perl</tt> and automatically get the <var class="LITERAL"><nowiki>#!/usr/bin/perl -w</nowiki></var> line and a comment containing the contents of the default template-begin file.
[gordonr@sevendev1 devguide]$ ls /etc/e-smith/templates-default/
template-begin template-begin-perl template-end-php
template-begin-html template-begin-php
template-begin-pam template-begin-shell
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''You may also need a <tt class="FILENAME">templates.metadata</tt> configuration file if your generated file needs to be executable.
</blockquote></div></div><div class="SECT2">
----
===Template fragment ordering===
Template fragments are assembled in ASCII-betical order, with two exceptions: template-begin always comes first, and template-end always comes last. Template fragments are often named to start with a two digit number to make the ordering obvious, but this is not required.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''The number of fragments and the order of those fragments within a template directory is subject to change between releases.
</blockquote></div></div><div class="SECT2">
----
===Templates for user home directories: templates-user===
Most of the templates on the system map to single, fixed output files, such as <tt class="FILENAME">/etc/hosts</tt>. However, templates are also used to generate configuration files such as mail delivery instructions for users. These templates are stored in the <tt class="FILENAME">/etc/e-smith/template-user/</tt> tree.
For example, the template for the <tt class="FILENAME">.qmail</tt> file in user home directories (which details how mail is to be handled), is stored under <tt class="FILENAME">/etc/e-smith/template-user/.qmail/</tt>. As these templates have a variable output filename, they are expanded using small pieces of Perl code in action scripts.
</div><div class="SECT2">
----
===Local site overrides: templates-custom and templates-user-custom===
It is possible that the standard templates are not correct for a particular installation, and so the local system administrator can override the extsing templates by placing files in the <tt class="FILENAME">templates-custom</tt> tree. This is a parallel tree to the normal templates hierarchy, and is normally empty. There is also a <tt class="FILENAME">template-user-custom</tt> tree for overriding entries in the templates-user tree.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''<span class="emphasis">''Never edit the standard templates.''</span> Your changes will be overwritten when packages are upgraded.
</blockquote></div><div class="NOTE"><blockquote class="NOTE">
'''Note: '''The template-custom trees should be reserved for local system overrides. <span class="emphasis">''Software should not install files in this tree.''</span>
</blockquote></div>
If a templates-custom entry exists for a template, it is merged with the standard templates directory during template expansion, using the following rules:
* If a fragment of the same name exists in both templates and templates-custom, the one from templates-custom is used, and the one from the standard templates tree is ignored.
* If the fragments in templates-custom have different names from those in templates, they are merged into the template as if they were in the templates directory.
* If the templates-custom entry is a file, rather than a directory, it completely overrides the standard template.
To make this concrete, let's assume we have the following template structure:
/etc/e-smith/templates/etc/book.conf:
10intro
30chapter3
40chapter4
80synopsis
and
/etc/e-smith/templates-custom/etc/book.conf:
30chapter3
50chapter5
The resulting template would be processed in this order:
* template-begin from /etc/e-smith/templates-default
* 10intro from /etc/e-smith/templates/etc/book.conf
* 30chapter3 from /etc/e-smith/templates-custom/etc/book.conf
* 40chapter4 from /etc/e-smith/templates/etc/book.conf
* 50chapter5 from /etc/e-smith/templates-custom/etc/book.conf
* 80synopsis from /etc/e-smith/templates/etc/book.conf
* template-end (empty), nominally from /etc/e-smith/templates-default
</div><div class="SECT2">
----
===How to resolve conflicts with standard templates===
It is possible that the standard templates may specify behaviour which is not appropriate for your application. In many cases the templates will be driven by configuration database settings which allow their behaviour to be customized, which should be the first thing to check.
In many cases, your application only needs to <span class="emphasis">''extend''</span> the behaviour of the template by adding one or more fragments. This should be your second option and can be achieved by simply adding your fragment in the correct place in the list of fragments.
In rare cases the standard template specifies a behaviour which conflicts with your application. In these cases, you should do <span class="emphasis">''all''</span> of the following:
* Create a templates-custom directory to match the existing one in the templates hierachy.
* Copy the conflicting fragment, and only that fragment, to the templates-custom directory. The fragment should have the same name in both directories. At this point you have not changed the behaviour of the system as the templates-custom entry will be preferred, but will behave identically.
* Modify the copy in templates-custom to suit your required behaviour.
* Raise a New Feature Request here: http://www.contribs.org/bugzilla/. Please attach your modified template (or even better, a patch file) and provide details of why you think that the standard template should be changed.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''You should not release RPMs which install templates in the <tt class="FILENAME">templates-custom</tt> directories. If the behaviour of a base template needs to be changed, please raise a bug to discuss the change.
</blockquote></div></div><div class="SECT2">
----
===Subdirectory templates===
It is also possible to split templates into further subdirectories. This can be very useful for evaluating the same fragments in a loop, for example for each virtual domain in <tt class="FILENAME">httpd.conf</tt> or each ibay in <tt class="FILENAME">smb.conf</tt>.
Two examples of this can be found in <tt class="FILENAME">/etc/e-smith/templates/etc/httpd/conf/httpd.conf/80VirtualHosts</tt> which loops over the <tt class="FILENAME">/etc/e-smith/templates/etc/httpd/conf/httpd.conf/VirtualHosts/</tt> directory, and <tt class="FILENAME">/etc/e-smith/templates/etc/smb.conf/90ibays</tt> which performs a similar loop over the <tt class="FILENAME">/etc/e-smith/templates/etc/smb.conf/ibays/</tt> directory.
</div></div><div class="SECT1">
----
==Template expansion==
<div class="SECT2">
===Mapping templates to events: templates2expand===
The SME Server is designed to ensure consistent and reliable operation, without requiring command-line access. Whenever an event is signalled, the relevant templates for that event are expanded and the services are notified of the configuration changes.
Requesting expansion of a template in an event is a simple matter of creating an empty file under the <var class="LITERAL">templates2expand</var> hierarchy for that event. For example, here are the templates which are expanded during an <var class="LITERAL">ip-change</var> event:
[gordonr@smebuild templates2expand]$ pwd
/etc/e-smith/events/ip-change/templates2expand
[gordonr@smebuild templates2expand]$ find . -type f
./etc/services
./etc/pam.d/passwd
./etc/dhcpd.conf
./etc/pptpd.conf
./etc/securetty
./etc/hosts.deny
./etc/shells
./etc/proftpd.conf
./etc/fetchmail
./etc/ppp/options.pptpd
./etc/ppp/ip-down.local
./etc/ppp/ip-up.local
./etc/hosts.allow
./etc/startmail
./var/qmail/alias/.qmail-localdelivery-default
./var/qmail/alias/.qmail-default
./var/qmail/control/concurrencylocal
./var/qmail/control/me
./var/qmail/control/virtualdomains
./var/qmail/control/smtproutes
./var/qmail/control/plusdomain
./var/qmail/control/doublebounceto
./var/qmail/control/rcpthosts
./var/qmail/control/badhelo
./var/qmail/control/databytes
./var/qmail/control/mailrules.default
./var/qmail/control/helohost
./var/qmail/control/bouncehost
./var/qmail/control/envnoathost
./var/qmail/control/defaultdomain
./var/qmail/control/locals
./var/qmail/control/bouncefrom
./var/qmail/control/defaulthost
./var/qmail/control/concurrencyremote
./home/e-smith/.qmail
It is important to note that any package can request a template expansion for an event. The list shown above has been contributed by a number of packages, and some of those packages have requested expansion of more than one template:
[gordonr@smebuild templates2expand]$ find . -type f|xargs rpm -qf | sort | uniq
e-smith-base-4.15.6-01
e-smith-email-4.15.4-01
e-smith-pptpd-1.11.0-18
e-smith-proftpd-1.11.0-25
e-smith-qmail-1.9.0-11
smeserver-qpsmtpd-1.0.1-09
</div><div class="SECT2">
----
===Template permissions and ownership: templates.metadata===
Templates are normally expanded to be owned by <var class="LITERAL">root</var> and are not executable, which is a reasonable default for most configuration files. However, templates may need to generate configuration files which are owned by a different user, or which need to be executable or have other special permissions. This can be done by creating a <var class="LITERAL">templates.metadata</var> file which defines the additional attributes for the expansion.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''Configuration files should generally <span class="emphasis">''not''</span> be writable by any user other than root. In particular, configuration files should not normally be writable the <span class="emphasis">''www''</span> user as this poses a significant security risk. Installation advice which says "chmod 777" is almost invariably wrong.
</blockquote></div>
For example, here is the metadata file <tt class="FILENAME">/etc/e-smith/templates.metadata/etc/ppp/ip-up.local</tt><nowiki>:</nowiki>
UID="root"
GID="daemon"
PERMS=0755
which sets the group to <var class="LITERAL">daemon</var> and makes the script executable. Note that the file is readable by members of the <var class="LITERAL">daemon</var> group, but it is not writable by anyone but root. It is also possible to use the same template to generate multiple output files, such as in this example:
<nowiki>TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX"
OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1"
MORE_DATA={ THIS_DEVICE => "eth1" }
FILTER=sub { $_[0] =~ /^#/ ? '' : $_[0] } # Remove comments</nowiki>
The templates.metadata file for route-eth0 just uses <var class="LITERAL">eth0</var> instead of <var class="LITERAL">eth1</var> on the second and third lines. Note also the <var class="LITERAL">FILTER</var> setting which allows post-processing of the generated template.
There are many examples under <tt class="FILENAME">/etc/e-smith/templates.metadata/</tt> and the full list of options can be seen with:
perldoc esmith::templates
</div><div class="SECT2">
----
===Manual testing: expand-template===
It is sometimes useful to expand templates manually during testing, which can be done with the '''expand-template''' command. The syntax of this command is simply:
expand-template <var class="REPLACEABLE">filename</var>
where <var class="REPLACEABLE">filename</var> is the name of the configuration file you want to generate, e.g. <tt class="FILENAME">/etc/hosts</tt>.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''<var class="LITERAL">expand-template</var> is designed for testing, and not as the standard way to expand templates. The correct way to ensure that a template is expanded is to create the <var class="LITERAL">templates2expand</var> files in the relevant events, along with any <var class="LITERAL">templates.metadata</var> files which may be required.
</blockquote></div></div><div class="SECT2">
----
===Perl API: processTemplate===
In rare circumstances you may need to call '''processTemplate''' directly. Explicit calls to '''processTemplate''' are typically only used when the output filename is variable, such as when processing the <var class="LITERAL">.qmail</var> files for each group:
use esmith::templates;
foreach my $group (@groups)
{
my $groupName = $group->key;
[...]
processTemplate(
{
CONFREF =>
{
Members => $members,
},
TEMPLATE_PATH =>
"/var/qmail/alias/.qmail-group",
OUTPUT_FILENAME => "/var/qmail/alias/.qmail-$groupName",
}
);
[...]
}
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''Software which was written for SME Server before release 7 will have a number of scripts which call '''processTemplate'''. In almost all cases, these can be replaced with simple flag files in the <tt class="FILENAME">templates2expand/</tt> directory of the relevant events. The new method is <span class="emphasis">''far''</span> more efficient as a single invocation is perl is used to expand all template files.
</blockquote></div></div></div></div>
=Configuration file templates=
<div class="SECT1">
==Design of the template system==
Every piece of software has its own configuration format, and writing parsers for each one is a complex, time-consuming and error-prone process. The SME Server software avoids the whole issue by using templates which <span class="emphasis">''generate''</span> the correct configuration.
In most cases, SME Server configuration files are <span class="emphasis">''over-written''</span> when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file (e.g. <tt class="FILENAME">/etc/fstab</tt>) is also automatically updated by some other process.
Templates are stored under <tt class="FILENAME">/etc/e-smith/templates/</tt> in a directory hierarchy which matches the standard filesystem. For example, the template for <tt class="FILENAME">/etc/inittab</tt> is stored in the <tt class="FILENAME">/etc/e-smith/templates/etc/inittab/</tt> directory. Each template is stored as a directory of template fragments and processed by the Perl <code class="CLASSNAME">Text::Template</code> module.
The template fragments are concatenated together in <span class="emphasis">''ASCIIbetical''</span> order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of the SME Server's modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.
<div class="NOTE"><blockquote class="NOTE">
'''Note: ''' It is also possible to store templates as single files, rather than as a directory of fragments. This method is preserved for backwards compatibility, but does not provide the extensibility of directory based templates. Directory templates should be used for all new templates, even if that directory only contains a single fragment.
</blockquote></div></div><div class="SECT1">
----
==The Text::Template module==
The <code class="CLASSNAME">Text::Template</code> module allows arbitary Perl code to be embedded in a template file by surrounding it in braces (<var class="LITERAL">"{"</var> and <var class="LITERAL">"}"</var>). The code inside the braces is interpreted and its return value replaces the section between, and including, the braces. For instance:
The answer is { 2 + 2 }
becomes
The answer is 4
Variables can be passed in from the program which is expanding the template, hence:
<nowiki>Shopping list:
{
$OUT = '';
for my $item ( qw(bread milk bananas) )
{
$OUT .= "* $item\n";
}
}</nowiki>
would expand to:
<nowiki>Shopping list:
* bread
* milk
* bananas</nowiki>
The SME Server template system uses this mechanism to automatically pass in global configuration variables from the <tt class="FILENAME">configuration</tt> database which can then be used to fill out the configuration files.
For example, the /etc/hosts template is fairly simple and composed of two fragments:
[gordonr@smebuild hosts]$ pwd
/etc/e-smith/templates/etc/hosts
[gordonr@smebuild hosts]$ ls
10localhost 20hostname
Let's look at those fragments. The first is a piece of static text, which <code class="CLASSNAME">Text::Template</code> will include verbatim:
127.0.0.1 localhost
The second is more complex and relies on values from the configuration database:
{
$OUT .= "$LocalIP\t";
$OUT .= " ${SystemName}.${DomainName}";
$OUT .= " ${SystemName}";
}
Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code. When this template is expanded, it results in the following configuration file:
<nowiki>#------------------------------------------------------------
# !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------
127.0.0.1 localhost
192.168.10.1 smebuild.gormand.com.au smebuild</nowiki>
The header block comes "for free" as part of the template system, courtesy of an optional file <tt class="FILENAME">template-begin</tt>, which is always processed as the first fragment. If it isn't provided, the text shown with # comments is included.
The other lines are provided by the two fragments shown above. Note the use of the configuration database variables: <var class="LITERAL">$LocalIP</var>, <var class="LITERAL">$SystemName</var> and <var class="LITERAL">$DomainName</var>. All simple entries in the configuration database are provided as global variables to the templates.
Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments.
The complex entries in the configuration database are also provided as global variables to the templates. However, they are provided as Perl hashes instead of simple scalars. For example, here is how you might configure the Network Time Protocol (NTP) server <tt class="FILENAME">/etc/ntp.conf</tt> file:
server { $ntpd{NTPServer} }
driftfile /etc/ntp/drift
authenticate no
The <var class="LITERAL">NTPServer</var> setting is stored in the <var class="LITERAL">ntpd</var> configuration database record, and so can be accessed via the hash accessor <var class="LITERAL">$ntpd{NTPServer}</var>.
<div class="SECT2">
----
===template-begin and template-end===
Each template directory can contain two optional files <tt class="FILENAME">template-begin</tt> and <tt class="FILENAME">template-end</tt> . The template-begin file is always processed as the first file of the template, and the template-end file is always processed as the last file.
If the directory does not contain a <tt class="FILENAME">template-begin</tt> file, the contents of <tt class="FILENAME">/etc/e-smith/templates-default/template-begin</tt> is used automatically.
If the directory does not contain a <tt class="FILENAME">template-end</tt> , nothing is appended to the template output. It is mostly used to provide the closing block for configuration files written in languages such as HTML and PHP, through a link to an entry in the <tt class="FILENAME">templates-default/</tt> directory.
</div><div class="SECT2">
----
===/etc/e-smith/templates-default===
The <tt class="FILENAME">/etc/e-smith/templates-default</tt> directory contains a set of template-begin and template-end files for various languages. For example, if your template generates a perl script, you would link <tt class="FILENAME">template-begin</tt> to <tt class="FILENAME">/etc/e-smith/templates-default/template-begin-perl</tt> and automatically get the <var class="LITERAL"><nowiki>#!/usr/bin/perl -w</nowiki></var> line and a comment containing the contents of the default template-begin file.
[gordonr@sevendev1 devguide]$ ls /etc/e-smith/templates-default/
template-begin template-begin-perl template-end-php
template-begin-html template-begin-php
template-begin-pam template-begin-shell
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''You may also need a <tt class="FILENAME">templates.metadata</tt> configuration file if your generated file needs to be executable.
</blockquote></div></div><div class="SECT2">
----
===Template fragment ordering===
Template fragments are assembled in ASCII-betical order, with two exceptions: template-begin always comes first, and template-end always comes last. Template fragments are often named to start with a two digit number to make the ordering obvious, but this is not required.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''The number of fragments and the order of those fragments within a template directory is subject to change between releases.
</blockquote></div></div><div class="SECT2">
----
===Templates for user home directories: templates-user===
Most of the templates on the system map to single, fixed output files, such as <tt class="FILENAME">/etc/hosts</tt>. However, templates are also used to generate configuration files such as mail delivery instructions for users. These templates are stored in the <tt class="FILENAME">/etc/e-smith/template-user/</tt> tree.
For example, the template for the <tt class="FILENAME">.qmail</tt> file in user home directories (which details how mail is to be handled), is stored under <tt class="FILENAME">/etc/e-smith/template-user/.qmail/</tt>. As these templates have a variable output filename, they are expanded using small pieces of Perl code in action scripts.
</div><div class="SECT2">
----
===Local site overrides: templates-custom and templates-user-custom===
It is possible that the standard templates are not correct for a particular installation, and so the local system administrator can override the extsing templates by placing files in the <tt class="FILENAME">templates-custom</tt> tree. This is a parallel tree to the normal templates hierarchy, and is normally empty. There is also a <tt class="FILENAME">template-user-custom</tt> tree for overriding entries in the templates-user tree.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''<span class="emphasis">''Never edit the standard templates.''</span> Your changes will be overwritten when packages are upgraded.
</blockquote></div><div class="NOTE"><blockquote class="NOTE">
'''Note: '''The template-custom trees should be reserved for local system overrides. <span class="emphasis">''Software should not install files in this tree.''</span>
</blockquote></div>
If a templates-custom entry exists for a template, it is merged with the standard templates directory during template expansion, using the following rules:
* If a fragment of the same name exists in both templates and templates-custom, the one from templates-custom is used, and the one from the standard templates tree is ignored.
* If the fragments in templates-custom have different names from those in templates, they are merged into the template as if they were in the templates directory.
* If the templates-custom entry is a file, rather than a directory, it completely overrides the standard template.
To make this concrete, let's assume we have the following template structure:
/etc/e-smith/templates/etc/book.conf:
10intro
30chapter3
40chapter4
80synopsis
and
/etc/e-smith/templates-custom/etc/book.conf:
30chapter3
50chapter5
The resulting template would be processed in this order:
* template-begin from /etc/e-smith/templates-default
* 10intro from /etc/e-smith/templates/etc/book.conf
* 30chapter3 from /etc/e-smith/templates-custom/etc/book.conf
* 40chapter4 from /etc/e-smith/templates/etc/book.conf
* 50chapter5 from /etc/e-smith/templates-custom/etc/book.conf
* 80synopsis from /etc/e-smith/templates/etc/book.conf
* template-end (empty), nominally from /etc/e-smith/templates-default
</div><div class="SECT2">
----
===How to resolve conflicts with standard templates===
It is possible that the standard templates may specify behaviour which is not appropriate for your application. In many cases the templates will be driven by configuration database settings which allow their behaviour to be customized, which should be the first thing to check.
In many cases, your application only needs to <span class="emphasis">''extend''</span> the behaviour of the template by adding one or more fragments. This should be your second option and can be achieved by simply adding your fragment in the correct place in the list of fragments.
In rare cases the standard template specifies a behaviour which conflicts with your application. In these cases, you should do <span class="emphasis">''all''</span> of the following:
* Create a templates-custom directory to match the existing one in the templates hierachy.
* Copy the conflicting fragment, and only that fragment, to the templates-custom directory. The fragment should have the same name in both directories. At this point you have not changed the behaviour of the system as the templates-custom entry will be preferred, but will behave identically.
* Modify the copy in templates-custom to suit your required behaviour.
* Raise a New Feature Request here: http://www.contribs.org/bugzilla/. Please attach your modified template (or even better, a patch file) and provide details of why you think that the standard template should be changed.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''You should not release RPMs which install templates in the <tt class="FILENAME">templates-custom</tt> directories. If the behaviour of a base template needs to be changed, please raise a bug to discuss the change.
</blockquote></div></div><div class="SECT2">
----
===Subdirectory templates===
It is also possible to split templates into further subdirectories. This can be very useful for evaluating the same fragments in a loop, for example for each virtual domain in <tt class="FILENAME">httpd.conf</tt> or each ibay in <tt class="FILENAME">smb.conf</tt>.
Two examples of this can be found in <tt class="FILENAME">/etc/e-smith/templates/etc/httpd/conf/httpd.conf/80VirtualHosts</tt> which loops over the <tt class="FILENAME">/etc/e-smith/templates/etc/httpd/conf/httpd.conf/VirtualHosts/</tt> directory, and <tt class="FILENAME">/etc/e-smith/templates/etc/smb.conf/90ibays</tt> which performs a similar loop over the <tt class="FILENAME">/etc/e-smith/templates/etc/smb.conf/ibays/</tt> directory.
</div></div><div class="SECT1">
----
==Template expansion==
<div class="SECT2">
===Mapping templates to events: templates2expand===
The SME Server is designed to ensure consistent and reliable operation, without requiring command-line access. Whenever an event is signalled, the relevant templates for that event are expanded and the services are notified of the configuration changes.
Requesting expansion of a template in an event is a simple matter of creating an empty file under the <var class="LITERAL">templates2expand</var> hierarchy for that event. For example, here are the templates which are expanded during an <var class="LITERAL">ip-change</var> event:
[gordonr@smebuild templates2expand]$ pwd
/etc/e-smith/events/ip-change/templates2expand
[gordonr@smebuild templates2expand]$ find . -type f
./etc/services
./etc/pam.d/passwd
./etc/dhcpd.conf
./etc/pptpd.conf
./etc/securetty
./etc/hosts.deny
./etc/shells
./etc/proftpd.conf
./etc/fetchmail
./etc/ppp/options.pptpd
./etc/ppp/ip-down.local
./etc/ppp/ip-up.local
./etc/hosts.allow
./etc/startmail
./var/qmail/alias/.qmail-localdelivery-default
./var/qmail/alias/.qmail-default
./var/qmail/control/concurrencylocal
./var/qmail/control/me
./var/qmail/control/virtualdomains
./var/qmail/control/smtproutes
./var/qmail/control/plusdomain
./var/qmail/control/doublebounceto
./var/qmail/control/rcpthosts
./var/qmail/control/badhelo
./var/qmail/control/databytes
./var/qmail/control/mailrules.default
./var/qmail/control/helohost
./var/qmail/control/bouncehost
./var/qmail/control/envnoathost
./var/qmail/control/defaultdomain
./var/qmail/control/locals
./var/qmail/control/bouncefrom
./var/qmail/control/defaulthost
./var/qmail/control/concurrencyremote
./home/e-smith/.qmail
It is important to note that any package can request a template expansion for an event. The list shown above has been contributed by a number of packages, and some of those packages have requested expansion of more than one template:
[gordonr@smebuild templates2expand]$ find . -type f|xargs rpm -qf | sort | uniq
e-smith-base-4.15.6-01
e-smith-email-4.15.4-01
e-smith-pptpd-1.11.0-18
e-smith-proftpd-1.11.0-25
e-smith-qmail-1.9.0-11
smeserver-qpsmtpd-1.0.1-09
</div><div class="SECT2">
----
===Template permissions and ownership: templates.metadata===
Templates are normally expanded to be owned by <var class="LITERAL">root</var> and are not executable, which is a reasonable default for most configuration files. However, templates may need to generate configuration files which are owned by a different user, or which need to be executable or have other special permissions. This can be done by creating a <var class="LITERAL">templates.metadata</var> file which defines the additional attributes for the expansion.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''Configuration files should generally <span class="emphasis">''not''</span> be writable by any user other than root. In particular, configuration files should not normally be writable the <span class="emphasis">''www''</span> user as this poses a significant security risk. Installation advice which says "chmod 777" is almost invariably wrong.
</blockquote></div>
For example, here is the metadata file <tt class="FILENAME">/etc/e-smith/templates.metadata/etc/ppp/ip-up.local</tt><nowiki>:</nowiki>
UID="root"
GID="daemon"
PERMS=0755
which sets the group to <var class="LITERAL">daemon</var> and makes the script executable. Note that the file is readable by members of the <var class="LITERAL">daemon</var> group, but it is not writable by anyone but root. It is also possible to use the same template to generate multiple output files, such as in this example:
<nowiki>TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX"
OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1"
MORE_DATA={ THIS_DEVICE => "eth1" }
FILTER=sub { $_[0] =~ /^#/ ? '' : $_[0] } # Remove comments</nowiki>
The templates.metadata file for route-eth0 just uses <var class="LITERAL">eth0</var> instead of <var class="LITERAL">eth1</var> on the second and third lines. Note also the <var class="LITERAL">FILTER</var> setting which allows post-processing of the generated template.
There are many examples under <tt class="FILENAME">/etc/e-smith/templates.metadata/</tt> and the full list of options can be seen with:
perldoc esmith::templates
</div><div class="SECT2">
----
===Manual testing: expand-template===
It is sometimes useful to expand templates manually during testing, which can be done with the '''expand-template''' command. The syntax of this command is simply:
expand-template <var class="REPLACEABLE">filename</var>
where <var class="REPLACEABLE">filename</var> is the name of the configuration file you want to generate, e.g. <tt class="FILENAME">/etc/hosts</tt>.
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''<var class="LITERAL">expand-template</var> is designed for testing, and not as the standard way to expand templates. The correct way to ensure that a template is expanded is to create the <var class="LITERAL">templates2expand</var> files in the relevant events, along with any <var class="LITERAL">templates.metadata</var> files which may be required.
</blockquote></div></div><div class="SECT2">
----
===Perl API: processTemplate===
In rare circumstances you may need to call '''processTemplate''' directly. Explicit calls to '''processTemplate''' are typically only used when the output filename is variable, such as when processing the <var class="LITERAL">.qmail</var> files for each group:
use esmith::templates;
foreach my $group (@groups)
{
my $groupName = $group->key;
[...]
processTemplate(
{
CONFREF =>
{
Members => $members,
},
TEMPLATE_PATH =>
"/var/qmail/alias/.qmail-group",
OUTPUT_FILENAME => "/var/qmail/alias/.qmail-$groupName",
}
);
[...]
}
<div class="NOTE"><blockquote class="NOTE">
'''Note: '''Software which was written for SME Server before release 7 will have a number of scripts which call '''processTemplate'''. In almost all cases, these can be replaced with simple flag files in the <tt class="FILENAME">templates2expand/</tt> directory of the relevant events. The new method is <span class="emphasis">''far''</span> more efficient as a single invocation is perl is used to expand all template files.
</blockquote></div></div></div></div>
