Difference between revisions of "SME Server:Documentation:Developers Manual:Chapter2/de"

From SME Server
Jump to navigationJump to search
m (Replaced Template:drawBox* (deprecated) with Template:* box and made language independant)
 
(11 intermediate revisions by one other user not shown)
Line 13: Line 13:
 
Jeder Eintrag in der Konfigurationsdatenbank ist entweder ein einfaches Schlüssel/Wert-Paar oder ein Schlüssel und eine Zusammenstellung dazu passender Eigenschaften bzw. Werte-Paare.
 
Jeder Eintrag in der Konfigurationsdatenbank ist entweder ein einfaches Schlüssel/Wert-Paar oder ein Schlüssel und eine Zusammenstellung dazu passender Eigenschaften bzw. Werte-Paare.
  
{{DrawBoxNote/de|Inhalt=Dieser Abschnitt beschreibt die generelle Struktur der Konfigurations-Datenbank. Aktuelle Einträge und Eigenschaften können sich zwischen den Versionen des SME Servers ändern.}}
+
{{Note box|type=Anmerkung|Dieser Abschnitt beschreibt die generelle Struktur der Konfigurations-Datenbank. Aktuelle Einträge und Eigenschaften können sich zwischen den Versionen des SME Servers ändern.}}
  
  
Line 55: Line 55:
 
  LocalIP=192.168.1.100
 
  LocalIP=192.168.1.100
  
{{DrawBoxNote/de|Inhalt=Der Begriff ''Konfigurations-Datenbank'' wird mehrfach benutzt. Er beschreibt sowohl die ''Master''-Konfiguration des Gesamtsystems als auch die Zusammenfassung mehrerer Konfigurations-Datenbanken, in denen die einzelnen Benutzerkonten, Netzwerkeinstellungen usw. enthalten sind.}}
+
{{Note box|type=Anmerkung|Der Begriff ''Konfigurations-Datenbank'' wird mehrfach benutzt. Er beschreibt sowohl die ''Master''-Konfiguration des Gesamtsystems als auch die Zusammenfassung mehrerer Konfigurations-Datenbanken, in denen die einzelnen Benutzerkonten, Netzwerkeinstellungen usw. enthalten sind.}}
  
 
Sie können auch mit dem ''db''-Befehl auf die Konfigurations-Datenbanken zugreifen. Um beispielsweise Details für das Benutzerkonto ''admin'' anzeigen zu lassen, geben Sie ein:
 
Sie können auch mit dem ''db''-Befehl auf die Konfigurations-Datenbanken zugreifen. Um beispielsweise Details für das Benutzerkonto ''admin'' anzeigen zu lassen, geben Sie ein:
Line 126: Line 126:
 
* Den Wert eines Datenbank-Eintrags mit einem neuen Wert überschreiben
 
* Den Wert eines Datenbank-Eintrags mit einem neuen Wert überschreiben
  
This design allows each package to provide part of the system configuration, or migrate the system configuration values as required. Note that a single database property can only be "owned" by one package. Database initialization is run during system install, system upgrade and after new software has been installed.
+
Dieses Design erlaubt jedem Softwarepaket, Teile des Gesamtsystems zu konfigurieren oder die Konfigurationswerte, auf gewünschte Werte zu ändern.  
  
If you examine the /etc/e-smith/db/configuration/ directory you will see three subdirectories: defaults/, force/ and migrate/ to match the three options above. A similar structure exists for each of the other databases. A new database can be created by populating a new directory tree under the /etc/e-smith/db/ directory.
+
{{Note box|type=Anmerkung|"Besitzer" einer Datenbank-Eigenschaft ist immer ein einzelnes Softwarepaket.Ein einzelnes(einziges) Datenbankeigentum(-besitz) nur von einem Paket im Besitz sein kann. Die Datenbank-Initialisierung wird während des Systems-Erstinstallation, beim System-Upgrade und nach der Installation neuer Softwarepakete durchgeführt.}}
  
[root@gsxdev1 db]# cd /etc/e-smith/db
+
Das Verzeichnis ''/etc/e-smith/db/configuration/'' besteht aus drei Unterverzeichnissen: defaults/, force/ und migrate/, mit denen die Datenbank-Initialisierung unterstützt wird. Eine ähnliche Struktur finden Sie auch für jede der anderen Datenbanken. Eine neue Datenbank wird über ein neues Verzeichnis im Verzeichnisbaum ''/etc/e-smith/db/'' angelegt.
[root@gsxdev1 db]# ls
 
accounts      domains      networks      yum_installed
 
backups        hosts        spamassassin  yum_repositories
 
configuration  mailpatterns  yum_available  yum_updates
 
  
[root@gsxdev1 db]# ls configuration/
+
[root@gsxdev1 db]# cd /etc/e-smith/db
defaults force migrate
+
[root@gsxdev1 db]# ls
 +
accounts      domains      networks      yum_installed
 +
backups        hosts        spamassassin  yum_repositories
 +
configuration  mailpatterns  yum_available yum_updates
  
Defaults files
+
[root@gsxdev1 db]# ls configuration/
 +
defaults  force  migrate
  
Defaults files are simple text files. If the corresponding database key/property already exists, it is skipped. Otherwise, the key/property is created and the value loaded. For example, this file:
+
=====Dateien unter ''defaults''=====
  
[root@gsxdev1 db]# cat configuration/defaults/sshd/status
+
Die Dateien unter ''defaults'' sind einfache Textdateien und beihalten die für den Server voreingestellten Werte. Falls das Schlüssel/Werte-Paar in der korrespondierenden Datenbank bereits existiert, wird es bei der Initialisierung üpergangen. Ansonsten wird das Schlüssel/Werte-Paar erzeugt und der ''default''-Wert geladen. Das Beispiel
disabled
 
  
would create the sshd database entry if it doesn't already exist, create the status property for that entry, again if it doesn't already exist, and finally set the status property to disabled.
+
[root@gsxdev1 db]# cat configuration/defaults/sshd/status
Force files
+
disabled
  
Force files are just like defaults files, except they overwrite the existing value. So, this file:
+
würde einen Eintrag in der ''sshd''-Datenbank, seinen zugehörigen Status erzeugen und diesen auf ''disabled'' setzen, wenn dieser Eintrag noch nicht existiert.
  
[root@gsxdev1 db]# cat configuration/force/sysconfig/ReleaseVersion
+
=====Daten unter ''force''=====
7.0rc2
 
  
would create the ReleaseVersion property of the sysconfig entry and unconditionally set its value to 7.0rc2
+
Die Dateien unter ''force'' sind ähnlich wie diejenigen unter ''defaults'', sind aber dafür gedacht, existierende Einträge zu überschreiben. So würde die Datei
Migrate fragments
 
  
Migrate fragments are small pieces of Perl text which can be used to perform more complex migrations than is possible with defaults and force files. They would normally be used to replace database keys or properties with new names, or to adjust policy settings during an upgrade.
+
[root@gsxdev1 db]# cat configuration/force/sysconfig/ReleaseVersion
 +
7.0rc2
  
Each fragment is passed a reference to the current database in the $DB variable. This variable is an instance of the appropriate esmith::DB subclass, e.g. esmith::AccountsDB when the accounts database migrate fragments are being executed. This means that you can use the methods of that subclass, for example esmith::AccountsDB->users().
+
die Eigenschaft ''ReleaseVersion'' im Eintrag von ''sysconfig'' ändern und den Wert auf ''7.0rc2'' setzen.
  
Here is an example of a migrate fragment, which replaces the outdated popd entry with the new name pop3:
+
=====Codefragmente=====
  
{
+
Codefragmente sind kleine Stückchen aus Perl-Texten und werden für komplexere Änderungen benutzt, die nicht mit den ''default'' oder ''force''-Dateien durchgeführt werden können. Normalerweise werden Codefragmente benutzt, um Datenbank-Schlüssel oder -Eigenschaften mit neuen Namen zu ersetzen o der um Regeln für die Einstellungen während eines Upgrades zu aktualisieren.
 +
 
 +
Jedes Fragment enthält einen Verweis auf die passende Konfigurations-Datenbank in der $DB Variable. Diese Variable fungiert als Instanz der passenden ''esmith::DB'' subclass, z.B. ''esmith::AccountsDB'', in der Codefragmente für die Konfigurations-Datenbank der Benutzerkonten ausgeführt werden können. Damit sind die Methode subclass verwendbar. Zum Beispiel für die subclass ''esmith::AccountsDB->users()''.
 +
 
 +
Hier ist ein Beispiel für Codefragmente, die den veralteten Eintrag für ''popd'' mit dem neuen Namen ''pop3'' ersetzen:
 +
{
 
     my $popd = $DB->get("popd") or return;
 
     my $popd = $DB->get("popd") or return;
 
 
     my $pop3 = $DB->get("pop3") ||
 
     my $pop3 = $DB->get("pop3") ||
 
         $DB->new_record("pop3", { type => "service" });
 
         $DB->new_record("pop3", { type => "service" });
 
 
     $pop3->merge_props($popd->props);
 
     $pop3->merge_props($popd->props);
 
 
     $popd->delete;
 
     $popd->delete;
}
+
}
  
This fragment checks whether the database (the configuration database in this case) has a popd entry. If that entry does not exist, the migrate fragment returns immediately. If the popd entry exists, we need to convert it, so we retrieve the pop3 entry (or create it if it doesn't already exist). We then merge the properties from the popd entry into the pop3 entry and finally delete the popd entry.
+
Mit diesem Fragment wird überprüft, ob die Datenbank (in diesem Fall die Konfigurations-Datenabnk) einen Eintrag für den Dienst ''popd'' enthält. Falls der Eintrag nicht exisiert, wird das Fragment sofort beendet. Ist der Eintrag ''popd'' vorhanden, muss er geändert werden, indem der Eintrag für ''pop3'' abgefragt wird (oder neu erstellt wird, wenn er noch nicht vorhanden ist). Die Eigenschaften der beiden Einträge ''popd'' und ''pop3'' werden anschließend zusammengeführt und schließlich der Eintrag ''popd'' gelöscht.
  
If this migrate fragment is run again, it will return immediately as the popd entry has already been deleted.
+
Bei erneutem Start dieses Migrations-Fragments wird dann festgestellt, dass der ''popd'' Eintrag bereits gelöscht wurde und das Fragment dann sofort beendet.  
Important notes about migrate fragments
 
  
    *
+
{{Note box|type=Anmerkung|Wichtige Anmerkungen zu Migrations-Fragmenten
  
      Please be careful with migrate fragments. Although they should only modify entries within the current database, there are no restrictions placed on what they can do. The ability to open and even modify other databases may be required to perform a migration.
+
Gehen Sie vorsichtig mit Migrations-Fragmenten um. Auch wenn diese nur die Einträge der aktuellen Datenbank ändern, gibt es keine Beschränkungen darin, was änderbar ist. Um eine Migration durchführen zu können, kann es erforderlich sein, andere Datenbanken öffnen und sogar ändern zu können.
    *
 
  
      Migrate fragments must be safe to run multiple times. They should migrate the value when required and do nothing in other cases.
+
Migrate fragments must be safe to run multiple times. They should migrate the value when required and do nothing in other cases.
    *
 
  
      Migrate fragments should never call croak or die. This will cause the database migration to stop. If an error is detected, call carp or warn to note the error in the logs.
+
Migrate fragments should never call croak or die. This will cause the database migration to stop. If an error is detected, call carp or warn to note the error in the logs.
    *
 
  
      Migrate fragments should be owned by the package requiring the migration so that the migration only occurs when that package is installed.
+
Migrate fragments should be owned by the package requiring the migration so that the migration only occurs when that package is installed.
    *
 
  
      Migrate fragments should be self-contained and ideally perform only one migration per fragment.
+
Migrate fragments should be self-contained and ideally perform only one migration per fragment.
    *
 
  
      It is also possible to initialize and migrate database values in action scripts, but creation of migrate fragments is strongly preferred. Creating defaults is a simple matter of creating text files and migrate fragments require far less code than action scripts.
+
It is also possible to initialize and migrate database values in action scripts, but creation of migrate fragments is strongly preferred. Creating defaults is a simple matter of creating text files and migrate fragments require far less code than action scripts.}}
  
Evaluation order: migrate, defaults, force
+
=====Evaluation order: migrate, defaults, force=====
  
 
When a database is loaded:
 
When a database is loaded:
Line 213: Line 207:
  
 
This order allows migration of old format entries to occur prior to loading of new default values. Remember, defaults will not change an existing database property.
 
This order allows migration of old format entries to occur prior to loading of new default values. Remember, defaults will not change an existing database property.
Forcing database initialization
+
 
 +
=====Forcing database initialization=====
  
 
The database is initialized during a number of events, including console-save, so a call to signal-event console-save will evaluate all of the database fragments.
 
The database is initialized during a number of events, including console-save, so a call to signal-event console-save will evaluate all of the database fragments.
Line 221: Line 216:
 
     It is an SME Server requirement that all database entries and configuration files must be correctly configured after a "reconfiguration reboot". This is available from the console and server manager and performs the post-upgrade and reboot events. Packages should also provide links in other events (e.g. "email-update" for email related changes) to provide reconfiguration without the reboot.
 
     It is an SME Server requirement that all database entries and configuration files must be correctly configured after a "reconfiguration reboot". This is available from the console and server manager and performs the post-upgrade and reboot events. Packages should also provide links in other events (e.g. "email-update" for email related changes) to provide reconfiguration without the reboot.
  
Important notes about the configuration databases
+
=====Important notes about the configuration databases=====
  
 
     *
 
     *

Latest revision as of 12:15, 29 September 2007


height=48px SME Server Handbücher

Administration
Benutzer
Entwickler

height=48px Zusätzliche Dokumentation

Technikerhandbuch
Häufig gestellte Fragen
HowTo-Anleitungen

height=48px Erweiterungen
Individuelle SME Server Erweiterungen

Kapitel 2: Der SME Server intern

Konfigurations-Datenbank

Überblick

Alle SME Server-Parameter, die durch Benutzer geändert werden können, sind in der Konfigurations-Datenbank gespeichert. Diese Werte werden dann benutzt, um Systemkonfigurationsdateien zu generieren, die im Verzeichnis /etc/ gespeichert werden.

Die Konfigurations-Datenbank kann durch verschiedene Programme geändert werden, über den Server-Manager, die Serverkonsole oder Skripte, die ein Administrator aus der SME Server Shell heraus aufruft.

Jeder Eintrag in der Konfigurationsdatenbank ist entweder ein einfaches Schlüssel/Wert-Paar oder ein Schlüssel und eine Zusammenstellung dazu passender Eigenschaften bzw. Werte-Paare.


Important.png Anmerkung
Dieser Abschnitt beschreibt die generelle Struktur der Konfigurations-Datenbank. Aktuelle Einträge und Eigenschaften können sich zwischen den Versionen des SME Servers ändern.



Einfache Einträge

Einfache Einträge in der Konfigurations-Datenbank als Schlüssel/Wert-Paar:

[root@gsxdev1 ~]# config show AccessType
AccessType=dedicated
[root@gsxdev1 ~]# config show ConsoleMode
ConsoleMode=login
[root@gsxdev1 ~]# config show TimeZone
TimeZone=Australia/NSW

Komplexe Einträge

Komplexere Einträge bestehen aus einen Schlüssel, einem Typ und einer Zusammenstellung dazu passender Eigenschaften bzw. Werte-Paare:

[root@gsxdev1 ~]# config show atalk
atalk=service
   MaxClients=20
   status=enabled
[root@gsxdev1 ~]# config show dhcpd
dhcpd=service
   end=192.168.1.250
   start=192.168.1.65
   status=disabled


In den meisten Fällen werden die komplexeren Einträge gegenüber den einfachen Einträgen bevorzugt, weil damit zusätzliche Eigenschaften gespeichert werden können, um die Flexibilität des Gesamtsystems zu erhöhen.


Zugriff von der Kommandozeile

Sie können auf die Konfigurations-Datenbank von der SME Server Shell aus über die Kommandozeile zugreifen, in dem Sie die Befehle config oder db verwenden. Dabei ist der config-Befehl die abgekürzte Form. Folgende Befehle sind äquivalent:

[root@gsxdev1 ~]# config show LocalIP
LocalIP=192.168.1.100
[root@gsxdev1 ~]# db configuration show LocalIP
LocalIP=192.168.1.100


Important.png Anmerkung
Der Begriff Konfigurations-Datenbank wird mehrfach benutzt. Er beschreibt sowohl die Master-Konfiguration des Gesamtsystems als auch die Zusammenfassung mehrerer Konfigurations-Datenbanken, in denen die einzelnen Benutzerkonten, Netzwerkeinstellungen usw. enthalten sind.


Sie können auch mit dem db-Befehl auf die Konfigurations-Datenbanken zugreifen. Um beispielsweise Details für das Benutzerkonto admin anzeigen zu lassen, geben Sie ein:

[root@gsxdev1 ~]# db accounts show admin
admin=system
   EmailForward=local
   FirstName=Local
   ForwardAddress=
   LastName=Administrator
   Lockable=no
   PasswordSet=yes
   Removable=no
   Shell=/sbin/e-smith/console
   VPNClientAccess=no

Die Dokumentation für den db-Befehl wird Ihnen angezeigt, wenn Sie den db-Befel ohne weitere Argumente benutzen:

[root@gsxdev1 ~]# db
usage:
   /sbin/e-smith/db dbfile keys
   /sbin/e-smith/db dbfile print [key]
   /sbin/e-smith/db dbfile show [key]
   /sbin/e-smith/db dbfile get key
   /sbin/e-smith/db dbfile set key type [prop1 val1] [prop2 val2] ...
   /sbin/e-smith/db dbfile setdefault key type [prop1 val1] [prop2 val2] ...
   /sbin/e-smith/db dbfile delete key
   /sbin/e-smith/db dbfile printtype [key]
   /sbin/e-smith/db dbfile gettype key
   /sbin/e-smith/db dbfile settype key type
   /sbin/e-smith/db dbfile printprop key [prop1] [prop2] [prop3] ...
   /sbin/e-smith/db dbfile getprop key prop
   /sbin/e-smith/db dbfile setprop key prop1 val1 [prop2 val2] [prop3 val3] ...
   /sbin/e-smith/db dbfile delprop key prop1 [prop2] [prop3] ...

Zugriff über die Perl API

Sie können auch mit Perl-Programmen auf die Konfigurations-Datenbanken zugreifen, wenn Sie esmith::ConfigDB und zugehörige Perl Module verwenden, die Abstraktionen für die esmith::DB module sind.

Zum Beispiel können Sie Details zum Benutzerkonto admin folgendermaßen anfordern und anzeigen lassen:

use esmith::AccountsDB;
my $db = esmith::AccountsDB->open or die "Couldn't open AccountsDB\n";
my $admin = $db->get("admin") or die "admin account missing from AccountsDB\n";
print $admin->show();

Mit diesem Code-Fragment würden die gleichen Informationen angezeigt wie mit dem Befehl db accounts show admin aus dem vorigen Abschnitt.

admin
    EmailForward = local
       FirstName = Local
  ForwardAddress =
        LastName = Administrator
        Lockable = no
     PasswordSet = yes
       Removable = no
           Shell = /sbin/e-smith/console
 VPNClientAccess = no
            type = system

Die Perl API wird noch deutlich tiefer in den späteren Übungen dieses Handbuchs beschrieben. Die Dokumentation dieser API finden Sie über die SME Server Shell mit dem perldoc-Befehl:

perldoc esmith::ConfigDB
perldoc esmith::AccountsDB
perldoc esmith::HostsDB
perldoc esmith::NetworksDB
perldoc esmith::DB

Datenbank-Initialisierung

Die Konfigurations-Datenbanken werden aus dem Dateibaum /etc/e-smith/db/ initialisiert. Darin enthaltene Dateien können jeweils eine von drei verschiedenen Aktionen ausführen:

  • Erstellen eines Datenbank-Eintrags mit einem voreingestellten Wert, falls der Eintrag nicht schon existiert
  • Erzwingen eines spezifischen Wertes für einen Datenbank-Eintrag unabhängig von der aktuellen Einstellung
  • Den Wert eines Datenbank-Eintrags mit einem neuen Wert überschreiben

Dieses Design erlaubt jedem Softwarepaket, Teile des Gesamtsystems zu konfigurieren oder die Konfigurationswerte, auf gewünschte Werte zu ändern.


Important.png Anmerkung
"Besitzer" einer Datenbank-Eigenschaft ist immer ein einzelnes Softwarepaket.Ein einzelnes(einziges) Datenbankeigentum(-besitz) nur von einem Paket im Besitz sein kann. Die Datenbank-Initialisierung wird während des Systems-Erstinstallation, beim System-Upgrade und nach der Installation neuer Softwarepakete durchgeführt.


Das Verzeichnis /etc/e-smith/db/configuration/ besteht aus drei Unterverzeichnissen: defaults/, force/ und migrate/, mit denen die Datenbank-Initialisierung unterstützt wird. Eine ähnliche Struktur finden Sie auch für jede der anderen Datenbanken. Eine neue Datenbank wird über ein neues Verzeichnis im Verzeichnisbaum /etc/e-smith/db/ angelegt.

[root@gsxdev1 db]# cd /etc/e-smith/db
[root@gsxdev1 db]# ls
accounts       domains       networks       yum_installed
backups        hosts         spamassassin   yum_repositories
configuration  mailpatterns  yum_available  yum_updates
[root@gsxdev1 db]# ls configuration/
defaults  force  migrate
Dateien unter defaults

Die Dateien unter defaults sind einfache Textdateien und beihalten die für den Server voreingestellten Werte. Falls das Schlüssel/Werte-Paar in der korrespondierenden Datenbank bereits existiert, wird es bei der Initialisierung üpergangen. Ansonsten wird das Schlüssel/Werte-Paar erzeugt und der default-Wert geladen. Das Beispiel

[root@gsxdev1 db]# cat configuration/defaults/sshd/status
disabled

würde einen Eintrag in der sshd-Datenbank, seinen zugehörigen Status erzeugen und diesen auf disabled setzen, wenn dieser Eintrag noch nicht existiert.

Daten unter force

Die Dateien unter force sind ähnlich wie diejenigen unter defaults, sind aber dafür gedacht, existierende Einträge zu überschreiben. So würde die Datei

[root@gsxdev1 db]# cat configuration/force/sysconfig/ReleaseVersion
7.0rc2

die Eigenschaft ReleaseVersion im Eintrag von sysconfig ändern und den Wert auf 7.0rc2 setzen.

Codefragmente

Codefragmente sind kleine Stückchen aus Perl-Texten und werden für komplexere Änderungen benutzt, die nicht mit den default oder force-Dateien durchgeführt werden können. Normalerweise werden Codefragmente benutzt, um Datenbank-Schlüssel oder -Eigenschaften mit neuen Namen zu ersetzen o der um Regeln für die Einstellungen während eines Upgrades zu aktualisieren.

Jedes Fragment enthält einen Verweis auf die passende Konfigurations-Datenbank in der $DB Variable. Diese Variable fungiert als Instanz der passenden esmith::DB subclass, z.B. esmith::AccountsDB, in der Codefragmente für die Konfigurations-Datenbank der Benutzerkonten ausgeführt werden können. Damit sind die Methode subclass verwendbar. Zum Beispiel für die subclass esmith::AccountsDB->users().

Hier ist ein Beispiel für Codefragmente, die den veralteten Eintrag für popd mit dem neuen Namen pop3 ersetzen:

{
   my $popd = $DB->get("popd") or return;
   my $pop3 = $DB->get("pop3") ||
       $DB->new_record("pop3", { type => "service" });
   $pop3->merge_props($popd->props);
   $popd->delete;
}

Mit diesem Fragment wird überprüft, ob die Datenbank (in diesem Fall die Konfigurations-Datenabnk) einen Eintrag für den Dienst popd enthält. Falls der Eintrag nicht exisiert, wird das Fragment sofort beendet. Ist der Eintrag popd vorhanden, muss er geändert werden, indem der Eintrag für pop3 abgefragt wird (oder neu erstellt wird, wenn er noch nicht vorhanden ist). Die Eigenschaften der beiden Einträge popd und pop3 werden anschließend zusammengeführt und schließlich der Eintrag popd gelöscht.

Bei erneutem Start dieses Migrations-Fragments wird dann festgestellt, dass der popd Eintrag bereits gelöscht wurde und das Fragment dann sofort beendet.


Important.png Anmerkung
Wichtige Anmerkungen zu Migrations-Fragmenten

Gehen Sie vorsichtig mit Migrations-Fragmenten um. Auch wenn diese nur die Einträge der aktuellen Datenbank ändern, gibt es keine Beschränkungen darin, was änderbar ist. Um eine Migration durchführen zu können, kann es erforderlich sein, andere Datenbanken öffnen und sogar ändern zu können.

Migrate fragments must be safe to run multiple times. They should migrate the value when required and do nothing in other cases.

Migrate fragments should never call croak or die. This will cause the database migration to stop. If an error is detected, call carp or warn to note the error in the logs.

Migrate fragments should be owned by the package requiring the migration so that the migration only occurs when that package is installed.

Migrate fragments should be self-contained and ideally perform only one migration per fragment.

It is also possible to initialize and migrate database values in action scripts, but creation of migrate fragments is strongly preferred. Creating defaults is a simple matter of creating text files and migrate fragments require far less code than action scripts.


Evaluation order: migrate, defaults, force

When a database is loaded:

   *
     migrate scripts are run first
   *
     then defaults are loaded
   *
     and finally any force files are loaded.

This order allows migration of old format entries to occur prior to loading of new default values. Remember, defaults will not change an existing database property.

Forcing database initialization

The database is initialized during a number of events, including console-save, so a call to signal-event console-save will evaluate all of the database fragments.

   Note: The console-save event is not a "reconfigure everything" event, and only changes items which can be configured from the text-mode console. It is convenient in this case as it performs database initialization and migration.
   It is an SME Server requirement that all database entries and configuration files must be correctly configured after a "reconfiguration reboot". This is available from the console and server manager and performs the post-upgrade and reboot events. Packages should also provide links in other events (e.g. "email-update" for email related changes) to provide reconfiguration without the reboot.
Important notes about the configuration databases
   *
     The configuration databases should only be modified using the tools and APIs provided.
   *
     The order of the entries and the order of properties is undefined.
   *
     The keys and property names are currently treated in a case-sensitive manner, though this may change in the future. Do not create keys or property names which differ only by their case.
   *
     Underscores and hyphens are valid in key and property names, but should normally be avoided.
   *
     Do not "overload" an existing property with a new value. If the existing values do not meet your requirements, discuss your implementation with the developers. Values which are not known by the base may cause serious issues on upgrade. If the existing panels have three choices, do not invent new choices without enhancing the panel to support them.
   *
     The type pseudo-property is used internally and is reserved.
   *
     By convention, database keys are lower case, and property names are stored in mixed case. The type, status and access properties are exceptions to this convention.
   *
     The storage location and internals of the databases is subject to change.
   *
     The configuration databases are currently stored as pipe-delimited flat text files in the /home/e-smith/db/ directory.

The configuration databases Configuration

The most important database is the (master) configuration database. This database describes how the system should operate; the type of Internet access to use, how email should be handled, and so on.

The configuration database contains a mix of simple and complex entries, although all new entries are complex entries. Accounts

Account details are stored in the accounts database, as complex entries. We classify accounts into several types, including:

   *
     User accounts: These are accounts created for individual users at the local organization. Each account has a POP/IMAP mailbox and an area for storing files.
   *
     Groups: Groups of users, which can be used for configuring permissions on storage areas and automatically provide a group e-mail address.
   *
     Information bays: These accounts correspond to information bays defined in the system. These storage areas can be accessed via filesharing, FTP and the web.
   *
     System accounts: Linux system accounts which are reserved by installed software packages.
   *
     URL accounts: Portions of the Web namespace which are reserved for system use. For example, the server-manager account is reserved as it is used for redirecting web access to the server manager.
   *
     Pseudonyms: Alternate names for existing accounts. For example, fred.frog could be a pseudonym for the account ffrog, allowing email to be sent to either address.
   *
     Printers: Network shared printers share the same namespace as other accounts so that they can be made visible to the local network.

Domains

The domains database shows the domains handled by this server, including information about how to handle web requests, and the DNS servers for the domain. Networks

The networks database details the networks which should be treated as local by this server. Local networks have additional access rights which are denied for other networks. Hosts

The hosts database decribes all hosts/machines known to this server and is used to generate DHCP and DNS configuration. Other configuration databases

There are several other configuration databases stored with the ones listed above, and the system design allows for additional databases to be created as required. Namespace issues

All entries in a single database share the same namespace. Users, groups, information bays, printers, and other entries in the accounts database currently all share one namespace. This means that you cannot have a user with the same name as an information bay, group or other entry in the accounts database.

However, it would be possible to have a host named fredfrog as well as a user named fredfrog as they are stored in separate databases and thus different namespaces.

Actions und Events

Actions

An action is a program, frequently written in a scripting language, which performs a single task. It is typically an encapsulation of a task usually done by a system administrator, such as editing a configuration file or reconfiguring a service. Actions are not called directly; they are always called by signalling an event.

The actions are stored in the /etc/e-smith/events/actions/ directory. These actions are then linked into the relevant events as the same action may need to be performed in more than one event.. To create a new action called myaction you simply create a program to perform the action myaction and save it as /etc/e-smith/events/actions/myaction . Actions can be written in any programming language, although additional platform support is provided for Perl code.

An example action script is set-external-ip which is called when the external IP address changes. Here's the body of that script (at time of writing):

package esmith;

use strict; use Errno; use esmith::ConfigDB;

my $db = esmith::ConfigDB->open or die "Couldn't open ConfigDB\n";

my $event = $ARGV[0]; my $newip = $ARGV[1];

$db->set_value('ExternalIP', $newip); $db->set_prop('ExternalInterface', 'IPAddress', $newip);

exit 0;

This script sets the ExternalIP value and the IPAddress property of the ExternalInterface record in the configuration database to the value provided as a parameter. The $event parameter is not used in this particular script.

   Note: The two records exist due to an partial migration from simple to complex entries in the configuration database. Setting both values in this script avoids the need to perform database migration in the ip-change event.

Action script parameters

Action scripts are always called with at least one parameter; the name of the current event. Many action scripts, such as set-external-ip , are called with a single additional parameter. This parameter is usually a configuration database key, for example the username being modified or the new IP address.

Action scripts rarely require more than two parameters.The details should be stored in the configuration database(s) and only the key should be passed to the action scripts. Events are not meant to be used as function calls. All configuration details must be stored in the configuration databases and the database key passed as the parameter to the action. This allows other scripts to be added to the event.

Since the SME Server passes the name of the current event as the first parameter, it is often beneficial to write action scripts which are polymorphic based on the event name. For example, the code to create a user and the code to modify an existing user may be only slightly different and may well benefit from being in a single script. Events

Events are a mechanism which allows the system to trigger a set of actions in response to actual events that happen on the system. When one of the users interfaces modifies the configuration databases, it must signal an event to regenerate the various server application configuration files according to the new configuration. The user interface must never modify configuration files directly.

Each event is associated with a list of actions which should be performed when that event occurs and is defined as a subdirectory of /etc/e-smith/events/ containing symbolic links to the appropriate actions, loosely modelled after the System V init mechanism for starting servers. For example, if you examine the /etc/e-smith/events/ip-change directory:

lrwxrwxrwx 1 root root 26 S15set-external-ip -> ../actions/set-external-ip* lrwxrwxrwx 1 root root 21 S85update-dns -> ../actions/update-dns* drwxr-xr-x 2 root root 4096 services2adjust/ drwxr-xr-x 5 root root 4096 templates2expand/

The symbolic links are given prefixes such as S15, S85, etc. to specify the order in which the actions should be executed in a similar manner to the System V init mechanism.

You can change the actions performed by an event by changing the links in the event directory. You can also create a new event by creating another subdirectory of /etc/e-smith/events/. Implicit actions: services2adjust and templates2expand

Most events contain two common tasks: expanding various templates and adjusting (e.g. restarting) the relevant services. For this reason, two implicit actions are included in all events. These implicit actions mean that additional code does not need to be written to perform these common tasks. The implicit actions are represented by entries in the services2adjust/ and templates2expand/ subdirectories. services2adjust

The services2adjust/ directory contains links mapping a specific service to the action to perform on that service. For example, if signalling the event in question requires that the ntpd service is restarted, you simply include the link ntpd -> restart in the services2adjust directory. The implicit action services2adjust would then restart the ntpd service. As an example, the services2adjust/ directory for the ip-change event is shown below:

lrwxrwxrwx 1 root root 6 masq -> adjust lrwxrwxrwx 1 root root 7 ntpd -> restart lrwxrwxrwx 1 root root 7 pptpd -> sigterm lrwxrwxrwx 1 root root 6 qmail -> sighup lrwxrwxrwx 1 root root 7 tinydns -> sigusr2

templates2expand

The templates2expand/ directory contains a list of the configuration files which need to be regenerated from their templates. This list consists of a collection of empty files with the same file name as the configuration file to be expanded and in a heirarchy mirroring their location on the system. For example, to expand templates for the /etc/samba/smb.conf configuration file, simply include the empty file etc/samba/smb.conf in the templates2expand/ directory of the relevant event(s). For more detail, see the Section called Mapping templates to events: templates2expand in Chapter 8. Order of implicit actions

The implicit actions are implemented by inserting the action script generic_template_expand early in the list of actions to be run in an event and the adjust-services action near the end of the list.

You should normally link your action scripts in the range S10 to S80 so that they occur after templates2expand and before services2adjust.

   Note: The generic_template_expand action is currently run at S05 and adjust-services is run at S90. The order of action scripts within an event is subject to change between releases.

Signalling events

The signal-event program takes an event name as an argument, and executes all of the actions in that event, providing the event name as the first parameter and directing all output to the system log. It works by listing the entries in the event directory and executing them in sequence. So for example, the command:

signal-event console-save

will perform all the actions associated with the console-save event, which is defined by the contents of the /etc/e-smith/events/console-save/ directory. This is exactly what the console user interface does when you select save at the end of the console configuration wizard. Events with arguments

So far we have described the following general principle throughout the SME Server; changes are made by altering the configuration files, then signalling events. The actions triggered by each event typically regenerate entire configuration files, taking into account the latest configuration information.

However, some changes are best made incrementally. For example, consider the user-create event. One of its actions updates the LDAP directory, which it could do by deleting all of the users and recreating them based on the updated accounts database. However, this is inefficient and would lose any additional LDAP attributes which may have been stored. It would be better to simply add the new user incrementally, using the default LDAP schema.

But how is the action code to know which user was just added? The new username is passed as an argument to the user-create event. This way the action programs triggered by the user-create event have a choice. They can either ignore the username argument and regenerate their output based on the updated list of accounts, or they can pay attention to the username argument, retrieve the rest of the information about the new user from the accounts database, and perform the incremental work to add the user.

   Note: Reminder: action scripts should normally take at most two arguments. The first is always the event name. The second optional argument is a key into one of the databases. Events are not function calls.
   Events are not currently serialized. In most cases overlapping events will not cause issues, but caution should be exercised when events are signalled from programs.

Standard events and their arguments

The table below summarises the key SME Server events and their argument if required. Remember, each action script is always called with the event name as the first argument. The arguments listed in this table are provided as the second argument.

   Note: Events which are not listed in this table are subject to change and may not appear in future releases of the SME Server.

Table 7-1. SME Server standard events Event Argument Description bootstrap-console-save (none) Expands all templates in the system. It is a requirement that all templates are correct after a combination of post-upgrade/reboot. Called after the initial console wizard, after system upgrades, and as part of a reconfiguration reboot. console-save (none) Expands templates and reconfigures services which can be changed from the text-mode console and which do not require a reboot. Services which do require a reboot for configuration will be handled by bootstrap-console-save. The console-save event is not a general "reconfigure everything" event. email-update (none) Reconfigures services listed on the e-mail panel. group-create, group-delete, group-modify Group - key into accounts database Called when a group is created/deleted/modified. halt (none) Called when the system is being shutdown prior to power off. host-create, host-delete, host-modify Host - key into hosts database Called when a host is created, deleted or modified. ibay-create, ibay-delete, ibay-modify Ibay - key into accounts database Called when an information bay is created/deleted/modified. ip-change New external IP address Called when the external IP address changes, e.g. through a new PPPoE connection or DHCP lease. local (none) Called after each reboot. Customisations which would normally require modification of the /etc/rc.local file should instead be installed as individual scripts in the /etc/e-smith/events/local/ event directory. network-create, network-delete Network - key into networks database Called when a local network is created or deleted. password-modify User - key into accounts database Called when a user password is modified, including when the account is unlocked. post-upgrade (and post-install) (none) Called as final step of the CD upgrade (install). This event must be immediately followed by a reboot. The bootstrap-console-save event is then called after the reboot to complete the reconfiguration. The only changes which should occur in this event are ones which must be performed prior to the reboot (e.g. configuring the boot loader). The post-install event is only called once, from the CD installer. pre-backup, post-backup Cause - type of backup being performed (e.g. "tape") The pre-backup event creates consistent system state for the backup. For example, it creates an ASCII dump of the MySQL databases. If the pre-backup event fails, the backup is not run. The post-backup is called if the backup is successful and removes the state files generated by pre-backup. pseudonym-create, pseudonym-delete, pseudonym-modify Pseudonym - key into accounts database Called when a pseudonym is created/deleted/modified. reboot (none) Called when the system is being shutdown prior to a reboot. remoteaccess-update (none) Reconfigures services listed on the Remote Access panel and updates the firewall rules for all services. user-create, user-delete, user-modify User - key into accounts database Called when a user is created/deleted/modified. user-lock User - key into accounts database Called when a user account is locked. Handling deletions

When adding a user, the user is created in the accounts database, and various actions, such as creating the Linux account, are performed in the user-create event. However, when deleting a user, we want to maintain the accounts database entry for as long as possible, in case there is information which the actions in the user-delete event might need in order to cleanly delete the users.

The SME Server convention for handling deletions is:

   *
     Change the type of the entry to mark it as being in the process of being deleted e.g. a user entry becomes a user-deleted entry.
   *
     Signal the relevant deletion event - e.g. user-delete
   *
     Remove the entry from the database, but only if the event succeeds.

With this approach, the action scripts can decide whether to ignore the user-deleted entries when performing their tasks. Event logs

All events, and all actions run by the event, are logged to the messages system log. Here is an example action log, which has been formatted onto multiple lines to enhance readability:

Feb 2 13:22:33 gsxdev1 esmith::event[4525]:

 S65sshd-conf=action|
 Event|remoteaccess-update|
 Action|S65sshd-conf|
 Start|1138846952 730480|
 End|1138846953 66768|
 Elapsed|0.336288

From this single log, we can see the action script name, which event it was called in, when it started, ended and how long it took (0.34 seconds). Now, let's add an action script which always fails and signal the event again:

Feb 2 16:11:54 gsxdev1 esmith::event[4787]:

 S99false=action|
 Event|remoteaccess-update|
 Action|S99false|
 Start|1138857114 58910|
 End|1138857114 81920|
 Elapsed|0.02301|
 Status|256

Note that this log has a new field Status, which is added if the action script returns a false (non-zero) exit status. Suppressing the Status field when it is zero (success) makes it much easier to find failed actions in the logs. Failed events

If an action script fails, the entire event fails. The other actions scripts in the event are run, but the whole event is marked as having failed.

By convention, if a delete event fails, the user interface does not delete the entry from the relevant database. So, if the user-delete event fails, a "stray" user-deleted entry will appear in the accounts database. The event logs with Status properties can be matched with the user-deleted entries to determine which action script failed so it can be corrected in the future. This user-deleted entry will also block the creation of another account with that name until the issue is corrected.

Vorlagen für Konfigurationsdateien

Design of the template system

Every piece of software has its own configuration format, and writing parsers for each one is a complex, time-consuming and error-prone process. The SME Server software avoids the whole issue by using templates which generate the correct configuration.

In most cases, SME Server configuration files are over-written when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file (e.g. /etc/fstab) is also automatically updated by some other process.

Templates are stored under /etc/e-smith/templates/ in a directory hierarchy which matches the standard filesystem. For example, the template for /etc/inittab is stored in the /etc/e-smith/templates/etc/inittab/ directory. Each template is stored as a directory of template fragments and processed by the Perl Text::Template module.

The template fragments are concatenated together in ASCIIbetical order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of the SME Server's modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.

   Note: It is also possible to store templates as single files, rather than as a directory of fragments. This method is preserved for backwards compatibility, but does not provide the extensibility of directory based templates. Directory templates should be used for all new templates, even if that directory only contains a single fragment. 

The Text::Template module

The Text::Template module allows arbitary Perl code to be embedded in a template file by surrounding it in braces ("{" and "}"). The code inside the braces is interpreted and its return value replaces the section between, and including, the braces. For instance:

The answer is { 2 + 2 }

becomes

The answer is 4

Variables can be passed in from the program which is expanding the template, hence:

Shopping list: {

   $OUT = ;
   for my $item ( qw(bread milk bananas) )
   {
       $OUT .= "* $item\n";
   }

}

would expand to:

Shopping list:

  • bread
  • milk
  • bananas

The SME Server template system uses this mechanism to automatically pass in global configuration variables from the configuration database which can then be used to fill out the configuration files.

For example, the /etc/hosts template is fairly simple and composed of two fragments:

[gordonr@smebuild hosts]$ pwd /etc/e-smith/templates/etc/hosts

[gordonr@smebuild hosts]$ ls 10localhost 20hostname

Let's look at those fragments. The first is a piece of static text, which Text::Template will include verbatim:

127.0.0.1 localhost

The second is more complex and relies on values from the configuration database:

{

   $OUT .= "$LocalIP\t";
   $OUT .= " ${SystemName}.${DomainName}";
   $OUT .= " ${SystemName}";

}

Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code. When this template is expanded, it results in the following configuration file:

  1. ------------------------------------------------------------
  2. !!DO NOT MODIFY THIS FILE!!
  3. Manual changes will be lost when this file is regenerated.
  4. Please read the developer's guide, which is available
  5. at http://wiki.contribs.org/development/
  6. Copyright (C) 1999-2006 Mitel Networks Corporation
  7. ------------------------------------------------------------

127.0.0.1 localhost 192.168.10.1 smebuild.gormand.com.au smebuild

The header block comes "for free" as part of the template system, courtesy of an optional file template-begin, which is always processed as the first fragment. If it isn't provided, the text shown with # comments is included.

The other lines are provided by the two fragments shown above. Note the use of the configuration database variables: $LocalIP, $SystemName and $DomainName. All simple entries in the configuration database are provided as global variables to the templates.

Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments.

The complex entries in the configuration database are also provided as global variables to the templates. However, they are provided as Perl hashes instead of simple scalars. For example, here is how you might configure the Network Time Protocol (NTP) server /etc/ntp.conf file:

server { $ntpd{NTPServer} } driftfile /etc/ntp/drift authenticate no

The NTPServer setting is stored in the ntpd configuration database record, and so can be accessed via the hash accessor $ntpd{NTPServer}. template-begin and template-end

Each template directory can contain two optional files template-begin and template-end . The template-begin file is always processed as the first file of the template, and the template-end file is always processed as the last file.

If the directory does not contain a template-begin file, the contents of /etc/e-smith/templates-default/template-begin is used automatically.

If the directory does not contain a template-end , nothing is appended to the template output. It is mostly used to provide the closing block for configuration files written in languages such as HTML and PHP, through a link to an entry in the templates-default/ directory. /etc/e-smith/templates-default

The /etc/e-smith/templates-default directory contains a set of template-begin and template-end files for various languages. For example, if your template generates a perl script, you would link template-begin to /etc/e-smith/templates-default/template-begin-perl and automatically get the #!/usr/bin/perl -w line and a comment containing the contents of the default template-begin file.

[gordonr@sevendev1 devguide]$ ls /etc/e-smith/templates-default/ template-begin template-begin-perl template-end-php template-begin-html template-begin-php template-begin-pam template-begin-shell

   Note: You may also need a templates.metadata configuration file if your generated file needs to be executable.

Template fragment ordering

Template fragments are assembled in ASCII-betical order, with two exceptions: template-begin always comes first, and template-end always comes last. Template fragments are often named to start with a two digit number to make the ordering obvious, but this is not required.

   Note: The number of fragments and the order of those fragments within a template directory is subject to change between releases.

Templates for user home directories: templates-user

Most of the templates on the system map to single, fixed output files, such as /etc/hosts. However, templates are also used to generate configuration files such as mail delivery instructions for users. These templates are stored in the /etc/e-smith/template-user/ tree.

For example, the template for the .qmail file in user home directories (which details how mail is to be handled), is stored under /etc/e-smith/template-user/.qmail/. As these templates have a variable output filename, they are expanded using small pieces of Perl code in action scripts. Local site overrides: templates-custom and templates-user-custom

It is possible that the standard templates are not correct for a particular installation, and so the local system administrator can override the extsing templates by placing files in the templates-custom tree. This is a parallel tree to the normal templates hierarchy, and is normally empty. There is also a template-user-custom tree for overriding entries in the templates-user tree.

   Note: Never edit the standard templates. Your changes will be overwritten when packages are upgraded.
   Note: The template-custom trees should be reserved for local system overrides. Software should not install files in this tree.

If a templates-custom entry exists for a template, it is merged with the standard templates directory during template expansion, using the following rules:

   *
     If a fragment of the same name exists in both templates and templates-custom, the one from templates-custom is used, and the one from the standard templates tree is ignored.
   *
     If the fragments in templates-custom have different names from those in templates, they are merged into the template as if they were in the templates directory.
   *
     If the templates-custom entry is a file, rather than a directory, it completely overrides the standard template. 

To make this concrete, let's assume we have the following template structure:

/etc/e-smith/templates/etc/book.conf: 10intro 30chapter3 40chapter4 80synopsis

and

/etc/e-smith/templates-custom/etc/book.conf: 30chapter3 50chapter5

The resulting template would be processed in this order:

   *
     template-begin from /etc/e-smith/templates-default
   *
     10intro from /etc/e-smith/templates/etc/book.conf
   *
     30chapter3 from /etc/e-smith/templates-custom/etc/book.conf
   *
     40chapter4 from /etc/e-smith/templates/etc/book.conf
   *
     50chapter5 from /etc/e-smith/templates-custom/etc/book.conf
   *
     80synopsis from /etc/e-smith/templates/etc/book.conf
   *
     template-end (empty), nominally from /etc/e-smith/templates-default

How to resolve conflicts with standard templates

It is possible that the standard templates may specify behaviour which is not appropriate for your application. In many cases the templates will be driven by configuration database settings which allow their behaviour to be customized, which should be the first thing to check.

In many cases, your application only needs to extend the behaviour of the template by adding one or more fragments. This should be your second option and can be achieved by simply adding your fragment in the correct place in the list of fragments.

In rare cases the standard template specifies a behaviour which conflicts with your application. In these cases, you should do all of the following:

   *
     Create a templates-custom directory to match the existing one in the templates hierachy.
   *
     Copy the conflicting fragment, and only that fragment, to the templates-custom directory. The fragment should have the same name in both directories. At this point you have not changed the behaviour of the system as the templates-custom entry will be preferred, but will behave identically.
   *
     Modify the copy in templates-custom to suit your required behaviour.
   *
     Raise a New Feature Request here: http://bugs.contribs.org/. Please attach your modified template (or even better, a patch file) and provide details of why you think that the standard template should be changed.
   Note: You should not release RPMs which install templates in the templates-custom directories. If the behaviour of a base template needs to be changed, please raise a bug to discuss the change.

Subdirectory templates

It is also possible to split templates into further subdirectories. This can be very useful for evaluating the same fragments in a loop, for example for each virtual domain in httpd.conf or each ibay in smb.conf.

Two examples of this can be found in /etc/e-smith/templates/etc/httpd/conf/httpd.conf/80VirtualHosts which loops over the /etc/e-smith/templates/etc/httpd/conf/httpd.conf/VirtualHosts/ directory, and /etc/e-smith/templates/etc/smb.conf/90ibays which performs a similar loop over the /etc/e-smith/templates/etc/smb.conf/ibays/ directory. Template expansion Mapping templates to events: templates2expand

The SME Server is designed to ensure consistent and reliable operation, without requiring command-line access. Whenever an event is signalled, the relevant templates for that event are expanded and the services are notified of the configuration changes.

Requesting expansion of a template in an event is a simple matter of creating an empty file under the templates2expand hierarchy for that event. For example, here are the templates which are expanded during an ip-change event:

[gordonr@smebuild templates2expand]$ pwd /etc/e-smith/events/ip-change/templates2expand

[gordonr@smebuild templates2expand]$ find . -type f ./etc/services ./etc/pam.d/passwd ./etc/dhcpd.conf ./etc/pptpd.conf ./etc/securetty ./etc/hosts.deny ./etc/shells ./etc/proftpd.conf ./etc/fetchmail ./etc/ppp/options.pptpd ./etc/ppp/ip-down.local ./etc/ppp/ip-up.local ./etc/hosts.allow ./etc/startmail ./var/qmail/alias/.qmail-localdelivery-default ./var/qmail/alias/.qmail-default ./var/qmail/control/concurrencylocal ./var/qmail/control/me ./var/qmail/control/virtualdomains ./var/qmail/control/smtproutes ./var/qmail/control/plusdomain ./var/qmail/control/doublebounceto ./var/qmail/control/rcpthosts ./var/qmail/control/badhelo ./var/qmail/control/databytes ./var/qmail/control/mailrules.default ./var/qmail/control/helohost ./var/qmail/control/bouncehost ./var/qmail/control/envnoathost ./var/qmail/control/defaultdomain ./var/qmail/control/locals ./var/qmail/control/bouncefrom ./var/qmail/control/defaulthost ./var/qmail/control/concurrencyremote ./home/e-smith/.qmail

It is important to note that any package can request a template expansion for an event. The list shown above has been contributed by a number of packages, and some of those packages have requested expansion of more than one template:

[gordonr@smebuild templates2expand]$ find . -type f|xargs rpm -qf | sort | uniq e-smith-base-4.15.6-01 e-smith-email-4.15.4-01 e-smith-pptpd-1.11.0-18 e-smith-proftpd-1.11.0-25 e-smith-qmail-1.9.0-11 smeserver-qpsmtpd-1.0.1-09

Template permissions and ownership: templates.metadata

Templates are normally expanded to be owned by root and are not executable, which is a reasonable default for most configuration files. However, templates may need to generate configuration files which are owned by a different user, or which need to be executable or have other special permissions. This can be done by creating a templates.metadata file which defines the additional attributes for the expansion.

   Note: Configuration files should generally not be writable by any user other than root. In particular, configuration files should not normally be writable the www user as this poses a significant security risk. Installation advice which says "chmod 777" is almost invariably wrong.

For example, here is the metadata file /etc/e-smith/templates.metadata/etc/ppp/ip-up.local:

UID="root" GID="daemon" PERMS=0755

which sets the group to daemon and makes the script executable. Note that the file is readable by members of the daemon group, but it is not writable by anyone but root. It is also possible to use the same template to generate multiple output files, such as in this example:

TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX" OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1" MORE_DATA={ THIS_DEVICE => "eth1" } FILTER=sub { $_[0] =~ /^#/ ?  : $_[0] } # Remove comments

The templates.metadata file for route-eth0 just uses eth0 instead of eth1 on the second and third lines. Note also the FILTER setting which allows post-processing of the generated template.

There are many examples under /etc/e-smith/templates.metadata/ and the full list of options can be seen with:

perldoc esmith::templates

Manual testing: expand-template

It is sometimes useful to expand templates manually during testing, which can be done with the expand-template command. The syntax of this command is simply:

expand-template filename

where filename is the name of the configuration file you want to generate, e.g. /etc/hosts.

   Note: expand-template is designed for testing, and not as the standard way to expand templates. The correct way to ensure that a template is expanded is to create the templates2expand files in the relevant events, along with any templates.metadata files which may be required.

Perl API: processTemplate

In rare circumstances you may need to call processTemplate directly. Explicit calls to processTemplate are typically only used when the output filename is variable, such as when processing the .qmail files for each group:

use esmith::templates;

foreach my $group (@groups) {

   my $groupName = $group->key;
   [...]
   processTemplate(
           {
               CONFREF =>
                   {
                       Members => $members,
                   },
               TEMPLATE_PATH =>
                   "/var/qmail/alias/.qmail-group",
               OUTPUT_FILENAME => "/var/qmail/alias/.qmail-$groupName",
           }
       );
   [...]

}

   Note: Software which was written for SME Server before release 7 will have a number of scripts which call processTemplate. In almost all cases, these can be replaced with simple flag files in the templates2expand/ directory of the relevant events. The new method is far more efficient as a single invocation is perl is used to expand all template files.

Prozesse starten, steuern und stoppen

Process startup

In typical Linux systems, services (processes) are started at boot time through a mechanism such as System V init. When the system administrator needs to change the settings, they modify the configuration files and then restart the service or notify the process that it needs to re-read the configuration.

It is usually assumed that processes which have been started will continue to run, and only require intervention during configuration changes. There are a number of problems with this model, which are addressed by the SME Server:

   *
     Processes do occasionally fail through software errors, memory exhaustion and accidental finger poking by the system administrator.
   *
     Some startup scripts and processes do not gracefully handle server crashes, such as power outages. The startup scripts and processes often use process identifier (PID) files to determine whether the process is running. Reliable handling of PID files is impossible to achieve under all failure cases.
   *
     Many processes do not deal properly with rapid invocation of stop and start requests. This is often, but not always, due to "PID file race" conditions. 

Process supervision: runit (and supervise)

The SME Server addresses these issues by running processes under the runit process supervision environment, which:

   *
     runs each process under control of its own supervisor process
   *
     imposes process limits
   *
     restarts the process if it fails
   *
     provides a consistent mechanism for controlling the underlying process 
   Note: Gerrit Pape's runit came from previous work by Dan Bernstein on the supervise supervision environment. runit provides additional features, and has been released under a free software license.

The runit process tree

When a Linux system boots, it starts the init process, which then starts all other processes. When init enters "run-level 7", it starts /etc/runit/2 from an entry in /etc/inittab.

/etc/runit/2 starts the runsvdir master supervision process, which scans the /service/ directory for work to do. If the runsvdir command happened to fail, it would be restarted by init.

The runsvdir command looks for subdirectories under the /service/ directory, and starts a runsv process to manage that directory. If any of the runsv processes fail, they will be restarted by runsvdir.

Each runsv process looks for a run script under the directory it is managing. runsv runs the run script and keeps a connection to the process started by that script. If the process dies, it is restarted.

If the directory also has a log subdirectory, runsv runs run script in that directory and connects the output of the main program to the input of the "logger" process.

This produces a process tree which looks something like this:

[root@gsxdev1 events]# pstree 1 init-+-acpid

    |-md1_raid1
    |-md2_raid1
    | ...
    |-runsvdir-+-runsv-+-multilog
    |          |       `-ulogd
    |          |-6*[runsv---multilog]
    |          |-runsv-+-multilog
    |          |       `-ntpd
    |          |-runsv-+-multilog
    |          |       `-tinydns
    |          |-runsv-+-cvm-unix
    |          |       `-multilog
    |          |-runsv-+-multilog
    |          |       `-mysqld
    |          |-5*[runsv-+-multilog]
    |          |          `-tcpsvd]
    |          |-runsv-+-multilog
    |          |       `-oidentd
    |          |-runsv-+-multilog
    |          |       `-smtp-auth-proxy
    |          |-runsv-+-multilog
    |          |       `-smbd---smbd
    |          |-runsv---httpd---10*[httpd]

This looks like a complex process tree, but is a critical part of the SME Server's design for reliability. Each process is independent, has a consistent management interface, has process limits imposed on it, and will restart if it happens to fail.

   Note: For the curious, if init fails, the system reboots.

For further documentation on runit, refer to the runit manual page. Run-level 7 and the e-smith-service wrapper

The SME Server runs in the normally unused run-level 7. This ensures that the only software running on the SME Server is software that we have chosen to run, and it is started and stopped in a consistent way. If we need to replace a standard startup script with one which runs the process under supervise, we can do so without modifying the original package.

In order to run a process under run-level 7, all you need to do is provide a link in the /etc/rc.d/rc7.d/ directory to your startup script. However, in most cases your process should only start if it is enabled in the configuration database.

If you look at the /etc/rc.d/rc7.d/ directory. you will see that it contains a large number of links to the /etc/rc.d/init.d/e-smith-service script.

S00microcode_ctl -> /etc/rc.d/init.d/e-smith-service S05syslog -> /etc/rc.d/init.d/e-smith-service S06cpuspeed -> /etc/rc.d/init.d/e-smith-service S15nut -> ../init.d/e-smith-service S15raidmonitor -> /etc/rc.d/init.d/e-smith-service S26apmd -> /etc/rc.d/init.d/e-smith-service S35bootstrap-console -> /etc/rc.d/init.d/e-smith-service [...]

This script is key to ensuring that services start when they are enabled and do not start when they are disabled, as it:

   *
     Checks the name of the link, e.g. S05syslog
   *
     Removes the S05 prefix, leaving syslog
   *
     Checks to see whether syslog is defined in the configuration database, and whether it has its status set to enabled.
   *
     If so, it runs the /etc/init.d/syslog script with the argument start.
   *
     If the service is not enabled, it exits without starting the service. 
   Note: If a script exists in the /etc/init.d/supervise/ directory, e-smith-service will use that in preference to the one in the /etc/init.d/ directory. This allows us to install our own supervised startup scripts without modifying the original package.

Der Server-Manager als Web-Frontend

The user interfaces to the SME Server (the web based server-manager and the text mode console interface) perform their work by modifying the master system configuration database to describe the new system configuration, then regenerating the various application configuration files by signalling an event.

This decoupling of the user interfaces from the system configuration allows packages to be added and removed without modifying the user interface code. It also allows all actions performed by the manager to be scripted, if this is desired. For example, if a new package needs to expand a template when users are created, it can just create the appropriate links in the user-create event. The web directory

The primary files which make up the SME Server manager are kept in the /etc/e-smith/web/ directory. These files define the layout of the web functions and require auxiliary files which provide translations and the implementation of the functions.

Table 10-1. Web interface directories Name Description /etc/e-smith/web/common/ Common files such as images and page headers. /etc/e-smith/web/functions/ Screen definitions, written in FormMagick XML. The scripts in this directory are linked into the cgi-bin directory of the panels in which they should appear. /etc/e-smith/web/panels/ Top-level directory for panel definitions. Each panel is a collection of screens, presented as a single user interface. ./manager/{cgi-bin,common,html}/ Subdirectories for the HTML, CGI and common files for the "manager" panel, which is accessed by the /server-manager/ URL. ./password/{cgi-bin,common,html}/ Subdirectories for the "password" panel, which is accessed by the /user-password/ URL. /etc/e-smith/locale/ Top-level directory for all panel localizations. ./en-us/etc/e-smith/web/functions/ Subdirectory containing localization into US English. ./fr/etc/e-smith/web/functions/ Subdirectory containing localization into French. /usr/lib/perl5/site_perl/ Top-level directory for all Perl modules. ./esmith/FormMagick/Panel/ Subdirectory containing Perl modules which provide the implementations to support the panel definitions. Web function scripts

The functions subdirectory contains all of the screen definitions for all panels. Each screen definition is a CGI script which displays the screen and also handles the CGI form submission. The scripts are written using the CGI::FormMagick toolkit, which separates the screen layout from the panel implementation code, facilitates form validation and provides full support for localisation of the manager. An overview of FormMagick Layout of a FormMagick script

This section describes the FormMagick panel which is used in the Section called Exercise 5: Adding a user interface screen in Chapter 12. A typical FormMagick web function starts with the script preamble, which notes it as a perl script and informs the vi editor that the majority of the file is XML, rather than perl.

  1. !/usr/bin/perl -wT
  2. vim: ft=xml:

This is followed by the navigation settings metadata, which determine where the script should appear in the manager menu bar.

  1. ----------------------------------------------------------------------
  2. heading : Demo
  3. description : Logger
  4. navigation : 1000 1000
  5. ----------------------------------------------------------------------

Next is a small number of lines of perl which create a FormMagick object and then call the display method to draw the page.

use strict; use warnings;

use esmith::FormMagick::Panel::loggerdemo;

my $f = esmith::FormMagick::Panel::loggerdemo->new(); $f->display();

And finally there is the FormMagick XML page description, which starts at the __DATA__ marker and continues to the end of file. We will examine that in the next section. The FormMagick XML description

The FormMagick XML is divided into a preamble and then a set of pages. The preamble contains references to the title, header and footer of the page. These are usually the same on all pages so that a consistent header and footer is displayed.

<form

   title="FORM_TITLE"
   header="/etc/e-smith/web/common/head.tmpl"
   footer="/etc/e-smith/web/common/foot.tmpl">

The upper-case word FORM_TITLE is a placemarker token for a phrase which needs to be localised. There is an associated lexicon file which provides the translation of this token into the appropriate language for the user accessing the panel, as specified by their browser settings. For example, here is the English lexicon entry for that token:

   <entry>
       <base>FORM_TITLE</base>
       <trans>Logger demo</trans>
   </entry>

If the user browses the panel with English as their chosen language, the panel will display in English. If they choose French, French will be displayed. If an unsupported language is chosen, FormMagick will fall back to US English. Adding another language is basically a matter of providing the lexicon for that language.

The rest of the XML description is a series of pages. In this example there is a single page. Each page starts with a page tag, which gives the page a name for later reference and can optionally specify a pre-event and post-event.

   <page name="First" pre-event="print_status_message()"
       post-event="change_settings">

The pre-event is a reference to a function in the panel implementation (described later) and called before the page is loaded. The post-event is called after the user submits the information on the page, for example by pressing the Save button.

Each page is then composed of a number of fields

       <field
           type="select"
           id="loggerdemo_Interval"
           options="10,20,30,40,50"
           value="get_interval()">
           <label>LABEL_LOGGERDEMO_INTERVAL</label>
       </field>
       <field
           type="select"
           id="loggerdemo_status"
           options="'disabled' => 'DISABLED', 'enabled' => 'ENABLED'"
           value="get_status()">
           <label>LABEL_LOGGERDEMO_STATUS</label>
       </field>

Each field describes a user interface widget (e.g. a select box) and provides the data required for that widget. These data may be static lists (the options of the first field above), a set of key/value pairs (the options of the second field above) or dyanamic data returned from a subroutine (the value parameters in each of the fields).

The command perldoc CGI::FormMagick provides detailed documentation about the supported field types.

It is also possible to call subroutines which generate the required HTML for a section of a page. For example, buttons are often added by calling the print_button routine:

       <subroutine src="print_button('SAVE')" />
   Note: Buttons should be part of the FormMagick XML description, and hopefully will be in the future. The print_button routine is a workaround for the lack of a button widget.

Each page must finish with a closing page tag:

   </page>

After all of the pages have been described there is a single XML tag to close the form.

</form>

Navigation metadata

The web manager's navigation frame is generated automatically by examining the contents of the /etc/e-smith/web/functions/ directory.

In order to be listed in the navigation frame, your CGI script must contain heading, description and navigation lines, usually at the top of the script:

  1. heading : Configuration
  2. description : E-mail
  3. navigation : 6000 6700

These define the category heading under which your add-on's admin interface should be listed, the title it should have, and the priority it should have in the listing order. The first number gives the priority of the heading (usually a multiple of 1000) and the second number gives the priority of this particular item within that heading group. In other words, a heading with a priority of 1000 will come before one with 6000 in the navigation panel, and within that heading category the individual items are listed in order from highest to lowest.

To figure out what numbers to give your own script, figure out where you want it to appear in the navigation panel then check source code for the scripts which appear before and after where you want to be. For instance, if you want your item to appear before "Remote Access" and after "Local Networks" in the navigation menu, you would look at /etc/e-smith/web/functions/remoteaccess and /etc/e-smith/web/functions/localnetworks and find the following:

  1. heading : Security
  2. description : Remote access
  3. navigation : 5000 5200
  1. heading : Security
  2. description : Local networks
  3. navigation : 5000 5300

You might then put something these lines in your own script:

  1. heading : Security
  2. description : Advanced security
  3. navigation : 5000 5250
   Tip: When naming your script, use a name which closely resembles the description (and hence the name in the navigation panel). This makes it easier to correlate menu items to Perl scripts. Just take the descriptive name and remove capital letters, punctuation and spaces. For instance, "Advanced security" might become /etc/e-smith/web/functions/advancedsecurity

Permissions and security

The CGI scripts must have elevated permissions (setuid root) in order to write to the configuration database, since they will be run by the web server (which runs as user www). To ensure that these scripts can only be run by system administrators, the permissions on the parent directory and the scripts are set so that only the members of the admin group can run them. These panels are also restricted in the web server configuration so that only the admin user can access them. Common files

The common subdirectory contains any static files (such as images) which are used by multiple panels. Panel definitions

The panels directory contains the panel definitions. There is one subdirectory for each panel. Each panel must have html and cgi-bin subdirectories. The cgi-bin subdirectory should contain only symbolic links to the actual CGI scripts in the functions directory, and the html directory should contain the main index.html file for the panel, as well as any required navigation links.

   Note: The word panel is also sometimes used to refer to an individual web manager web function.

Keeping the CGI scripts for all panels in a shared directory makes it much easier to create auxiliary panels with slightly different options and permissions. You can just copy the entire panel directory, then customize the access permissions and navigation links. For example, it would be very straightforward to create a password-protected panel which only allowed the creation and deletion of user accounts. That task could be delegated to administrative staff.