SME Server:Documentation:Developers Manual:Chapter6

From SME Server
Revision as of 22:56, 9 February 2009 by Snoble (talk | contribs) (→‎Important notes about migrate fragments: bug 4997)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

II. SME Server internals

Configuration database

Overview

All user-modifiable configuration parameters on the SME Server are stored in the configuration database. These values are used to generate the system configuration files, such as those found in the /etc/ directory.

The configuration databases may be modified by various programs on the system, including the SME Server manager, the SME Server console, or scripts run from the command line by a system administrator.

Each entry in the database is either a simple key/value pair or a key and a collection of related property/value pairs.

Important.png Note:
The section describes the general structure of the configuration database. The actual entries and properties are subject to change between releases.

Simple entries

Simple configuration database entries take the form of a key/value pair: 

[root@gsxdev1 ~]# config show AccessType
AccessType=dedicated

[root@gsxdev1 ~]# config show ConsoleMode
ConsoleMode=login

[root@gsxdev1 ~]# config show TimeZone
TimeZone=Australia/NSW

Complex entries

More complex entries consist of a key, a type, and a collection of property/value pairs: 

[root@gsxdev1 ~]# config show atalk
atalk=service
    MaxClients=20
    status=enabled

[root@gsxdev1 ~]# config show dhcpd
dhcpd=service
    end=192.168.1.250
    start=192.168.1.65
    status=disabled

In most cases, complex entries are used in preference to simple entries. The complex entries allow additional properties to be stored for an entry, which enhances the system's flexibility.

Access from the command line

You can access configuration database entries from the command line using the config command, as shown above, or the db command. The config command provides a shorthand for accessing the configuration database. The following commands are equivalent: 

[root@gsxdev1 ~]# config show LocalIP
LocalIP=192.168.1.100

[root@gsxdev1 ~]# db configuration show LocalIP
LocalIP=192.168.1.100
Important.png Note:
The term configuration database is used both to refer to the "master" configuration database and to refer collectively to the set of configuration databases, which includes the individual accounts, networks and configuration databases.

The db allows you to access all of the databases. For example to show the details of the admin entry from accounts 

[root@gsxdev1 ~]# db accounts show admin
admin=system
    EmailForward=local
    FirstName=Local
    ForwardAddress=
    LastName=Administrator
    Lockable=no
    PasswordSet=yes
    Removable=no
    Shell=/sbin/e-smith/console
    VPNClientAccess=no

Documentation for the db command is displayed if you run it without providing any arguments: 

[root@gsxdev1 ~]# db
usage:
    /sbin/e-smith/db dbfile keys
    /sbin/e-smith/db dbfile print [key]
    /sbin/e-smith/db dbfile show [key]
    /sbin/e-smith/db dbfile get key
    /sbin/e-smith/db dbfile set key type [prop1 val1] [prop2 val2] ...
    /sbin/e-smith/db dbfile setdefault key type [prop1 val1] [prop2 val2] ...
    /sbin/e-smith/db dbfile delete key
    /sbin/e-smith/db dbfile printtype [key]
    /sbin/e-smith/db dbfile gettype key
    /sbin/e-smith/db dbfile settype key type
    /sbin/e-smith/db dbfile printprop key [prop1] [prop2] [prop3] ...
    /sbin/e-smith/db dbfile getprop key prop
    /sbin/e-smith/db dbfile setprop key prop1 val1 [prop2 val2] [prop3 val3] ...
    /sbin/e-smith/db dbfile delprop key prop1 [prop2] [prop3] ...

Access via the Perl API

You can also access configuration database entries programmatically using the esmith::ConfigDB and related Perl modules, which are abstractions for the esmith::DB module.

For example, we can retrieve and show the admin account details like this: 

use esmith::AccountsDB;

my $db = esmith::AccountsDB->open or die "Couldn't open AccountsDB\n";

my $admin = $db->get("admin") or die "admin account missing from AccountsDB\n";

print $admin->show();

This code fragment would display the same information as running the db accounts show admin command we saw previously. 

admin
     EmailForward = local
        FirstName = Local
   ForwardAddress =
         LastName = Administrator
         Lockable = no
      PasswordSet = yes
        Removable = no
            Shell = /sbin/e-smith/console
  VPNClientAccess = no
             type = system

The Perl API will be covered in more depth in the exercises later in this manual. For documentation on the API, log into the SME Server and browse the documentation using the perldoc command: 

perldoc esmith::ConfigDB
perldoc esmith::AccountsDB
perldoc esmith::HostsDB
perldoc esmith::NetworksDB

perldoc esmith::DB

Database initialization

The configuration databases are initialized from files in the /etc/e-smith/db/ hierarchy. These files can perform one of three actions:

  • Create a database entry and set it to a default value, if the entry does not already exist.
  • Force a database entry to a specific value, regardless of its current setting.
  • Migrate an entry from a previous value to a new value.

This design allows each package to provide part of the system configuration, or migrate the system configuration values as required. Note that a single database property can only be "owned" by one package. Database initialization is run during system install, system upgrade and after new software has been installed.

If you examine the /etc/e-smith/db/configuration/ directory you will see three subdirectories: defaults/, force/ and migrate/ to match the three options above. A similar structure exists for each of the other databases. A new database can be created by populating a new directory tree under the /etc/e-smith/db/ directory. 

[root@gsxdev1 db]# cd /etc/e-smith/db
[root@gsxdev1 db]# ls
accounts       domains       networks       yum_installed
backups        hosts         spamassassin   yum_repositories
configuration  mailpatterns  yum_available  yum_updates

[root@gsxdev1 db]# ls configuration/
defaults  force  migrate
Defaults files

Defaults files are simple text files. If the corresponding database key/property already exists, it is skipped. Otherwise, the key/property is created and the value loaded. For example, this file: 

[root@gsxdev1 db]# cat configuration/defaults/sshd/status
disabled

would create the sshd database entry if it doesn't already exist, create the status property for that entry, again if it doesn't already exist, and finally set the status property to disabled.

Force files

Force files are just like defaults files, except they overwrite the existing value. So, this file: 

[root@gsxdev1 db]# cat configuration/force/sysconfig/ReleaseVersion
7.0rc2

would create the ReleaseVersion property of the sysconfig entry and unconditionally set its value to 7.0rc2

Migrate fragments

Migrate fragments are small pieces of Perl text which can be used to perform more complex migrations than is possible with defaults and force files. They would normally be used to replace database keys or properties with new names, or to adjust policy settings during an upgrade.

Each fragment is passed a reference to the current database in the $DB variable. This variable is an instance of the appropriate esmith::DB subclass, e.g. esmith::AccountsDB when the accounts database migrate fragments are being executed. This means that you can use the methods of that subclass, for example esmith::AccountsDB->users().

Here is an example of a migrate fragment, which replaces the outdated popd entry with the new name pop3: 

{
    my $popd = $DB->get("popd") or return;

    my $pop3 = $DB->get("pop3") ||
        $DB->new_record("pop3", { type => "service" });

    $pop3->merge_props($popd->props);

    $popd->delete;
}

This fragment checks whether the database (the configuration database in this case) has a popd entry. If that entry does not exist, the migrate fragment returns immediately. If the popd entry exists, we need to convert it, so we retrieve the pop3 entry (or create it if it doesn't already exist). We then merge the properties from the popd entry into the pop3 entry and finally delete the popd entry.

If this migrate fragment is run again, it will return immediately as the popd entry has already been deleted.

Important notes about migrate fragments
  • Please be careful with migrate fragments. Although they should only modify entries within the current database, there are no restrictions placed on what they can do. The ability to open and even modify other databases may be required to perform a migration.
  • Migrate fragments must be safe to run multiple times. They should migrate the value when required and do nothing in other cases.
  • Migrate fragments should never call croak or die. This will cause the database migration to stop. If an error is detected, call carp or warn to note the error in the logs.
  • Migrate fragments should call good termination with return(0) rather than exit(0).
  • Migrate fragments should be owned by the package requiring the migration so that the migration only occurs when that package is installed.
  • Migrate fragments should be self-contained and ideally perform only one migration per fragment.
  • It is also possible to initialize and migrate database values in action scripts, but creation of migrate fragments is strongly preferred. Creating defaults is a simple matter of creating text files and migrate fragments require far less code than action scripts.
Evaluation order: migrate, defaults, force

When a database is loaded:

  • migrate scripts are run first
  • then defaults are loaded
  • and finally any force files are loaded.

This order allows migration of old format entries to occur prior to loading of new default values. Remember, defaults will not change an existing database property.

Forcing database initialization

The database is initialized during a number of events, including console-save, so a call to signal-event console-save will evaluate all of the database fragments.

Important.png Note:
The console-save event is not a "reconfigure everything" event, and only changes items which can be configured from the text-mode console. It is convenient in this case as it performs database initialization and migration.

It is an SME Server requirement that all database entries and configuration files must be correctly configured after a "reconfiguration reboot". This is available from the console and server manager and performs the post-upgrade and reboot events. Packages should also provide links in other events (e.g. "email-update" for email related changes) to provide reconfiguration without the reboot.


Important notes about the configuration databases

  • The configuration databases should only be modified using the tools and APIs provided.
  • The order of the entries and the order of properties is undefined.
  • The keys and property names are currently treated in a case-sensitive manner, though this may change in the future. Do not create keys or property names which differ only by their case.
  • Underscores and hyphens are valid in key and property names, but should normally be avoided.
  • Do not "overload" an existing property with a new value. If the existing values do not meet your requirements, discuss your implementation with the developers. Values which are not known by the base may cause serious issues on upgrade. If the existing panels have three choices, do not invent new choices without enhancing the panel to support them.
  • The type pseudo-property is used internally and is reserved.
  • By convention, database keys are lower case, and property names are stored in mixed case. The type, status and access properties are exceptions to this convention.
  • The storage location and internals of the databases is subject to change.
  • The configuration databases are currently stored as pipe-delimited flat text files in the /home/e-smith/db/ directory.

The configuration databases

Configuration

The most important database is the (master) configuration database. This database describes how the system should operate; the type of Internet access to use, how email should be handled, and so on.

The configuration database contains a mix of simple and complex entries, although all new entries are complex entries.

Accounts

Account details are stored in the accounts database, as complex entries. We classify accounts into several types, including:

  • User accounts: These are accounts created for individual users at the local organization. Each account has a POP/IMAP mailbox and an area for storing files.
  • Groups: Groups of users, which can be used for configuring permissions on storage areas and automatically provide a group e-mail address.
  • Information bays: These accounts correspond to information bays defined in the system. These storage areas can be accessed via filesharing, FTP and the web.
  • System accounts: Linux system accounts which are reserved by installed software packages.
  • URL accounts: Portions of the Web namespace which are reserved for system use. For example, the server-manager account is reserved as it is used for redirecting web access to the server manager.
  • Pseudonyms: Alternate names for existing accounts. For example, fred.frog could be a pseudonym for the account ffrog, allowing email to be sent to either address.
  • Printers: Network shared printers share the same namespace as other accounts so that they can be made visible to the local network.

Domains

The domains database shows the domains handled by this server, including information about how to handle web requests, and the DNS servers for the domain.

Networks

The networks database details the networks which should be treated as local by this server. Local networks have additional access rights which are denied for other networks.

Hosts

The hosts database decribes all hosts/machines known to this server and is used to generate DHCP and DNS configuration.

Other configuration databases

There are several other configuration databases stored with the ones listed above, and the system design allows for additional databases to be created as required.

Namespace issues

All entries in a single database share the same namespace. Users, groups, information bays, printers, and other entries in the accounts database currently all share one namespace. This means that you cannot have a user with the same name as an information bay, group or other entry in the accounts database.

However, it would be possible to have a host named fredfrog as well as a user named fredfrog as they are stored in separate databases and thus different namespaces.

Retrieved from "https://wiki.koozali.org/index.php?title=SME_Server:Documentation:Developers_Manual:Chapter6&oldid=12277"