1,346
edits
Changes
From SME Server
Jump to navigationJump to searchSME Server:Documentation:Developers Manual:Chapter3/de (view source)
Revision as of 19:19, 17 May 2007
, 19:19, 17 May 2007New page: {{Languages|SME_Server:Documentation:Developers_Manual:Chapter3}} {{Overview/de}} =Kapitel 3: SME Server Pakete erstellen - Schritt für Schritt= ==Beginn== The best way to get started i...
{{Languages|SME_Server:Documentation:Developers_Manual:Chapter3}}
{{Overview/de}}
=Kapitel 3: SME Server Pakete erstellen - Schritt für Schritt=
==Beginn==
The best way to get started is to install an SME Server and start experimenting with it. Download a copy from http://mirror.contribs.org/smeserver/releases/7/iso/ and burn your own CD.
If you (or any developers at your organization) have multiple computers on a home network, a cablemodem, DSL, or dialup connection, and an old Pentium machine that you don't need, we recommend installing the SME Server software on the old Pentium machine, and using it as a home gateway and firewall.
Note: The SME Server software erases all data from the PC on which it is installed, to turn it into a dedicated server that can run 24x7. Do not install it on a PC unless you are prepared to erase all of its data!
Alternatively, you can install the SME Server on a corporate LAN in server/gateway mode (creating a small private network behind a firewall that occupies a single IP address on the Internet) or in server-only mode - in which the SME Server provides network services to other computers as a peer on the network.
Note: Warning! The server-only mode is designed for LAN environments that already have a firewall/gateway.
In addition to feeling comfortable installing and using the SME Server software, you should also have a working knowledge of Linux, including use of the command line tools.
You should also be familiar with the perl programming language. Most of the SME Server software is written in perl, and the configuration template mechanism is based on perl.
It is strongly recommended that you obtain and read a copy of the book Maximum RPM (ISBN 067231 1054) or study the on-line version available at http://www.rpm.org/max-rpm/.
You also need to know how to use one of the Linux text editors such as vi, nano or pico. It is also possible to edit files on a remote machine and copy them to the server. However, it is important that the files are converted to Unix text format.
Creating a development environment
Packages which do not require compilation, for example shell and perl scripts, can be built on the SME Server platform. All of the examples in this documentation can be performed on a standard SME Server installation.
Before attempting to compile any software, you should check whether the package is available from one of the many well-maintained RPM repositories. Using these RPMs will ensure compatibility with the other RPMs on the SME Server. You are likely to find the package you want in either the CentOS or Dag Wieers repositories.
If an RPM does not already exist, you should install a CentOS developer workstation or server for SME Server development. You will also need to install the e-smith-devtools packages which can be found on the SME Server CD.
Note: We strongly recommend against installing development tools, such as compilers, on any production servers, especially those which are accessible from the Internet.
==Customizing des SME Servers==
Once you have studied the architecture of the SME Server, it is best to try to make some small customizations to become comfortable with the concepts. The number one rule to remember is: customizations always involve adding files to the server, rather than modifying existing files. This is very important, as it enables customizations to be easily packaged, and mixed and matched. The unique architecture of the SME Server enables virtually anything to be customized by adding a file in the correct location.
Exercise 1: Changing a configuration template
Let us say that you wish to customize your server so that it runs a specified program every twenty minutes. To simplify the problem, let us assume that this program simply adds a line of dots to the log file (/var/log/messages), i.e.:
/usr/bin/logger -t "Demo" "......"
Normally you would accomplish this by adding a line to the /etc/crontab file, which is the standard Linux mechanism for running scheduled jobs. However, the default /etc/crontab file looks something like this on an SME Server:
#------------------------------------------------------------
# !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://wiki.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
# logrotate
12 1 */7 * * root /sbin/e-smith/signal-event logrotate
Note the auto-generated comment block which reminds you not to edit the file. If you do, your changes will be overwritten when the template is next expanded by a system event. We want to append a new line that looks like this (read the Linux crontab documentation to understand the format of crontab entries):
*/20 * * * * root /usr/bin/logger -t "Demo" "......"
Remember that we cannot simply edit the /etc/crontab file. The rule is that we must perform this customization by adding a new file to the system. To get an idea how to do this, have a look at the contents of the template for /etc/crontab:
[gordonr@smebuild crontab]$ pwd
/etc/e-smith/templates/etc/crontab
[gordonr@smebuild crontab]$ ls
00setup 10runparts 20statusreport 65_logrotate email
Each of the files in that directory is a template fragment. The SME Server builds the /etc/crontab file by assembling those fragments and running them through the template processor.
To make your customization, create your own additional fragment by creating a file in this directory called 25templatedemo with the following contents:
# Template demo crontab entry:
*/20 * * * * root /usr/bin/logger -t "Demo" "......"
Next time the SME Server regenerates the /etc/crontab file, it will contain your additional fragment. Starting the name with the prefix "25" forces the template fragment to go between the "20statusreport" and "65_logrotate" fragments. Force the /etc/crontab file to be generated immediately by typing the command:
expand-template /etc/crontab
If you look at the /etc/crontab file now, you should see your new fragment at the appropriate place, and your customization will take effect immediately (as cron notices when its configuration file has been changed). Check /var/log/messages to see the results.
To package this customization, you will need to create an RPM package that contains this single file, and ensures that the /etc/crontab template is expanded in the relevant events. You should also call one of these events in the RPM post-install section to ensure that the template is expanded without further action. Installing that RPM on any SME Server will cause the customization to occur, and will start printing the line of dots to the /var/log/messages logfile every 20 minutes.
The final point to note here is that if you remove your new file 25templatedemo and re-expand the /etc/crontab template, the crontab will go back to the way it was, and your customization will disappear cleanly. Therefore you should put a post-uninstallation script into your RPM package that runs the appropriate events to expand the templates once more. That will result in a package that installs and uninstalls cleanly.
Remember that for testing you can call expand-template directly in the post-install and post-uninstall sections, but released software should use the templates2expand mechanism to request template expansion in the relevant events.
Exercise 2: The magic of templates
For the next exercise, let us build on the first one. You have already created an exciting (o.k. not that exciting) new capability - the ability of the server to write dots into the log file. Let us now take advantage of the fact that the template processor can fill in values from the configuration database.
Edit the /etc/e-smith/templates/etc/crontab/25templatedemo file again, this time with the following contents:
# Template demo crontab entry:
*/20 * * * * root /usr/bin/logger -t "Demo3" "... {
use esmith::AccountsDB;
$adb = esmith::AccountsDB->open_ro;
$adb->get_prop('admin', 'ForwardAddress') || 'admin';
}
Once again, regenerate the template by typing:
expand-template /etc/crontab
If you look at the new /etc/crontab file, you will see that the template processor has replaced the block between the braces with the actual email address of the administrator, which is defined in the accounts database.
Note: Be careful with the placement of the braces in the example. We want two lines of output - the comment and the line starting with the asterisk. If you move the opening brace onto a new line, you will end up with three lines of output.
Whitespace is not signifcant and the code within braces should be formatted "nicely". However, whitespace outside braces is significant and will be copied literally to the output file.
You could check that value with the db accounts show admin command. With this change to the template, the message which will go to the /var/log/messages log file every 20 minutes will contain the actual current administrative email address, rather than a hardcoded text message.
Now here is the exciting part. Click on the E-mail function in the server manager user interface, and change the administrative email address. If you look at /etc/crontab, you will see that it has been updated with the new email address! Every 20 minutes you will receive a new entry in the messages log which automatically reflects the new setting of the administrator email address.
So by simply creating a single file 25templatedemo, you have created quite a sophisticated customization that changes behaviour based on the system settings. And you have done so without affecting the rest of the system or requiring additional changes to the user interface.
The reason this works is subtle, but follows from the overall architecture. It is critical to study this example until you understand it thoroughly. If you understand exactly how this example works, you understand a substantial part of the SME Server architecture.
To understand this example in detail, let's start from the top by studying the user interface. The implementation section of the E-mail function can be found in the file /usr/lib/perl5/site_perl/esmith/FormMagick/Panel/emailsettings.pm. If you study the part of this script that gets executed when the user clicks Save, you will find some Perl code that saves the administrator e-mail address into the configuration database and signals the email-update event - thus informing the system that the email settings have changed:
sub change_settings_delivery
{
my ($fm) = @_;
my $q = $fm->{'cgi'};
[...]
my $admin_email = $q->param('AdminEmail');
my $admin = $accounts->get('admin');
if ($admin_email)
{
$admin->merge_props(
EmailForward => 'forward',
ForwardAddress => $admin_email,
);
}
else
{
$admin->merge_props(
EmailForward => 'local',
ForwardAddress => '',
);
}
[...]
unless ( system( "/sbin/e-smith/signal-event", "email-update" ) == 0 )
{
$fm->error('ERROR_UPDATING');
return undef;
}
$fm->success('SUCCESS');
}
When the email-update event is signalled, the SME Server executes all of the action scripts in the /etc/e-smith/events/email-update/ directory. The event also expands all templates as noted in the templates2expand/ hierarchy, including the /etc/crontab template.
That may all seem rather complicated, but it boils down to this: changing the administrator email address automatically rebuilds the /etc/crontab template - and if you have customized that template, your customization will automatically pick up the current administrator email address.
Note that if you wanted additional programs to execute whenever the email settings were changed, you would put all of those programs in the /etc/e-smith/events/actions/ directory, then create symbolic links to them from the /etc/e-smith/events/email-update/ directory. We use symbolic links because you may want your program triggered by other events as well as email-update, and so you create links from all of the relevant event directories back to your action program.
This system is, by design, extensible. For example, you could use this exact same type of customization to send an email message every hour containing the current IP address. This is left as an exercise to the reader.
Exercise 3: Using events and actions
In the SME Server, events are like callbacks in a programming language. The system signals an event whenever something interesting happens (e.g. a user is added, the IP address changes, etc.), which automatically executes all programs in the event directory. Therefore, any applications which need to know when a certain event is happening simply create a symbolic link from the event directory to a handler program, which will get executed whenever the event occurs.
In the previous exercise, we relied on the predefined events and actions in the SME Server to keep the /etc/crontab file up to date. In this example, we will create a new action script that will track the user accounts in the system. This script can be used as a template for any type of application that has its own notion of user accounts and needs to be driven by the existing user interface for adding, deleting, or modifying users.
Start by creating a new file called /etc/e-smith/events/actions/demo-user-tracking with the following contents:
#!/usr/bin/perl -w
# Set up Perl environment and libraries
package esmith;
use strict;
use warnings;
use esmith::ConfigDB;
use esmith::AccountsDB;
# Prepare to access configuration databases
my $db = esmith::ConfigDB->open_ro
or die "Couldn't open ConfigDB\n";
my $accounts = esmith::AccountsDB->open_ro
or die "Couldn't open AccountsDB\n";
# Read domain name from configuration database
my $domain = $db->get_value('DomainName');
# Read command line arguments
my $event = $ARGV [0];
my $id = $ARGV [1];
# If no command line arguments, assume this is the initial setup
# of all users. Process all accounts of type "user" (ignore groups,
# information bays, printers, etc.)
unless ($event and $id)
{
for my $user ($accounts->users)
{
my $key = $user->key;
my $first = $user->prop('FirstName');
my $last = $user->prop('LastName');
system ("/usr/bin/logger", "-t", "Demo3",
"Initializing user $key ($first $last) in domain $domain");
}
exit 0;
}
# If command line arguments are present, then this is a create, modify,
# or delete event, signalled by the SME Server event/action system.
my $user = $accounts->get($id)
or die "User $id does not exist\n";
my $first = $user->prop('FirstName');
my $last = $user->prop('LastName');
if ($event eq 'user-create')
{
system ("/usr/bin/logger", "-t", "Demo3",
"Creating user $id ($first $last) in domain $domain");
}
elsif ($event eq 'user-modify')
{
system ("/usr/bin/logger", "-t", "Demo3",
"Changing user $id to ($first $last) in domain $domain");
}
elsif ($event eq 'user-delete')
{
system ("/usr/bin/logger", "-t", "Demo3",
"Deleting user $id in domain $domain");
}
else
{
system ("/usr/bin/logger", "-t", "Demo3", "Ignoring $event event");
}
exit 0;
Make sure the permissions are correct:
chmod +x /etc/e-smith/events/actions/demo-user-tracking
Now create symbolic links so that this program is executed whenever a user is created, modified, or deleted. Make three symbolic links; one for each event directory:
cd /etc/e-smith/events
ln -s ../actions/demo-user-tracking user-create/S90demo-user-tracking
ln -s ../actions/demo-user-tracking user-modify/S90demo-user-tracking
ln -s ../actions/demo-user-tracking user-delete/S90demo-user-tracking
The S90 prefix ensures that the program will be executed after the standard actions (which typically have prefixes ranging from S15 to S80).
Study this program carefully. It uses many different SME Server capabilities. If invoked from the command line with no arguments, it will read all user accounts from the user database, fetch all the data fields associated with each user, and print this information to the log file. If invoked as an event, the SME Server will automatically pass it the event name and user id as command line arguments; in this case the program will print messages to the log file explaining that it is adding, modifying, or dropping the user.
Trying watching the log file by running the command:
tail -F /var/log/messages
or with the "View log files" panel in the server-manager. Use the standard SME Server user interface to add, modify, or remove users. You should see a stream of comments in the log file.
If you were creating an application that had its own database of user information, you would replace the logfile-writing code with your own code that initialized new users, modified them, etc. Then you would arrange for the RPM post-install script to execute:
/etc/e-smith/events/actions/demo-user-tracking
with no command line arguments. When the application is initially installed, it will immediately read all users from the database and set them up in your application. Then, as users are added, modified, or dropped over time - your code will be invoked each time, to update the application's private user database.
Tip: It is almost always better to extend the existing accounts database with additional properties for your application than to maintain another database.
Exercise 4: Adding new configuration database parameters
New system configuration parameters can be spontaneously invented and added to the configuration database at any time. For example, let us return to our earlier exercise and parameterize the time interval for the log messages by introducing a new parameter called LogInterval.
You can write that parameter into the configuration database, as though it had always existed. For example, type this on the command line:
config set LogInterval 30
You can use config show LogInterval to show that it was set as intended. You can now edit the 25templatedemo file and replace the hardcoded number 20 with the LogInterval parameter. The resulting file will read:
# Template demo crontab entry:
*/{ $LogInterval } * * * * root /usr/bin/logger -t "Demo4" "... {
use esmith::AccountsDB;
$adb = esmith::AccountsDB->open_ro;
$adb->get_prop('admin', 'ForwardAddress');
} ..."
Now you can change the logger interval at any time by typing the following (replace the number 20 with whatever logger interval you want):
config set LogInterval 20
expand-template /etc/crontab
This ability to spontaneously introduce new configuration parameters is very important in the SME Server architecture. The configuration database is a high-level specification of how the overall system is supposed to behave for the end user. Configuration settings are like knobs on a stereo system. The templates, events, and actions, are the underlying machinery to carry out the user's wishes. When adding a new application to the system, it is important to be able to add new knobs on demand.
Now let us say that you want to introduce a parameter to enable or disable this logging function. At this point, you might start thinking of this logging activity as a service that you should be able to enable or disable. In this case the convention is to create a single service entry in the configuration database to manage both parameters.
To implement this, first delete the LogInterval parameter, which will no longer be needed:
db configuration delete LogInterval
Now create a service entry:
db configuration set loggerdemo service status enabled Interval 20
If you examine the configuration database you will see the new entry looks like this:
[root@gsxdev1 ~]# config show loggerdemo
loggerdemo=service
Interval=20
status=enabled
Note: By convention, database keys are lower case, and property names are mixed case (e.g. Interval). The status property is an exception here, and is stored lower case.
Now edit the 25templatedemo file to look like this:
# Template demo crontab entry
{
my $status = $loggerdemo{status} || "disabled";
return "# loggerdemo service is disabled."
unless ($status eq "enabled");
use esmith::AccountsDB;
$adb = esmith::AccountsDB->open_ro;
my $admin_email = $adb->get_prop('admin', 'ForwardAddress')
|| 'admin';
my $interval = $loggerdemo{Interval} || 10;
$OUT = "*/$interval * * * * root /usr/bin/logger";
$OUT .= " -t \"Demo4\" \"... $admin_email ...\"\n";
}
Note: The variable $OUT is a special variable used for output from a template. It is documented in the Text::Template documentation. For now, just think about it as the return value from the template, so set it to the value you want to print from this fragment. Note also that a template fragment can return static text without setting $OUT directly, as shown on the return line above.
This is more complicated than the original template, but it is also more flexible. Note that the initial comment (# Template demo crontab entry) is hardcoded, but the line that follows is generated from the configuration database parameters. The code in the template retrieves the loggerdemo service entry, retrieves the required properties, and returns the appropriate output. To experiment, try different combinations of parameters:
db configuration setprop loggerdemo Interval 10
expand-template /etc/crontab
and
config setprop loggerdemo status disabled
expand-template /etc/crontab
and so on.
Exercise 5: Adding a user interface screen
Let us add a nice user interface screen to adjust the logger interval. Create a new file called /etc/e-smith/web/functions/loggerdemo, with the following contents:
#!/usr/bin/perl -wT
# vim: ft=xml:
#----------------------------------------------------------------------
# heading : Demo
# description : Logger
# navigation : 1000 1000
#----------------------------------------------------------------------
use strict;
use warnings;
use esmith::FormMagick::Panel::loggerdemo;
my $f = esmith::FormMagick::Panel::loggerdemo->new();
$f->display();
__DATA__
<form
title="FORM_TITLE"
header="/etc/e-smith/web/common/head.tmpl"
footer="/etc/e-smith/web/common/foot.tmpl">
<page name="First" pre-event="print_status_message()"
post-event="change_settings">
<field
type="select"
id="loggerdemo_Interval"
options="10,20,30,40,50"
value="get_interval()">
<label>LABEL_LOGGERDEMO_INTERVAL</label>
</field>
<field
type="select"
id="loggerdemo_status"
options="'disabled' => 'DISABLED', 'enabled' => 'ENABLED'"
value="get_status()">
<label>LABEL_LOGGERDEMO_STATUS</label>
</field>
<subroutine src="print_button('SAVE')" />
</page>
</form>
The file above describes the panel layout, which is written in FormMagick XML. Further details about FormMagick can be fiound in the Section called An overview of FormMagick in Chapter 10.
Another file provides the implementation, which goes into /usr/lib/perl5/site_perl/esmith/FormMagick/Panel/loggerdemo.pm:
#!/usr/bin/perl -w
package esmith::FormMagick::Panel::loggerdemo;
use strict;
use warnings;
use esmith::ConfigDB;
use esmith::FormMagick;
our @ISA = qw(esmith::FormMagick Exporter);
our @EXPORT = qw();
our $VERSION = sprintf '%d.%03d', q$Revision: 1.1 $ =~ /: (\d+).(\d+)/;
our $db = esmith::ConfigDB->open or die "Couldn't open ConfigDB\n";
sub get_status
{
return $db->get_prop("loggerdemo", "status");
}
sub get_interval
{
return $db->get_prop("loggerdemo", "Interval");
}
sub change_settings
{
my $fm = shift;
my $q = $fm->{'cgi'};
$db->set_prop('loggerdemo', 'status', $q->param("loggerdemo_status"));
$db->set_prop('loggerdemo', 'Interval', $q->param("loggerdemo_Interval"));
unless ( system ("/sbin/e-smith/expand-template", "/etc/crontab") == 0 )
{
$fm->error('ERROR_UPDATING');
return undef;
}
$fm->success('SUCCESS');
}
1;
Note: This code example calls expand-template. This is used for illustrative purposes only. All templates should be expanded by signalling events.
Similarly to events and actions, the /etc/e-smith/web/functions/ directory is a repository of potentially available functions. To make the new function actually show up in the user interface, create a symbolic link to it from the web manager cgi-bin directory, as follows:
cd /etc/e-smith/web/panels/manager/cgi-bin
ln -s ../../../functions/loggerdemo loggerdemo
Now, make sure the permissions and ownership are correct so that the web server can run your new function:
cd /etc/e-smith/web/functions
chown root:admin loggerdemo
chmod 4750 loggerdemo
We also need to run a program to rebuild the navigation bar. This is only required when adding or removing functions from the manager, and is normally handled automatically. Let's do it manually for now:
/etc/e-smith/events/actions/navigation-conf
Try the server manager now, reloading the navigation bar in your browser, if necessary. You will see a new category called Demo and a new function within it called Logger. It will look a bit bare with some upper-case words which signify phrases which have not been localised. We'll fix that in a moment.
Try experimenting with it - it is a nice little user interface for playing with the logger customization. Every time you change the settings, notice that the /etc/crontab file is updated appropriately.
Adding localizations
The SME Server is designed to support localization into any language. This is done by small files which describe the mapping from the upper case tags (seen above) to the appropriate words for the local language. Enter the following into /etc/e-smith/locale/en-us/etc/e-smith/web/functions/loggerdemo
<lexicon lang="en-us">
<!-- vim: ft=xml:
-->
<entry>
<base>FORM_TITLE</base>
<trans>Logger demo</trans>
</entry>
<entry>
<base>Demo</base>
<trans>Demo</trans>
</entry>
<entry>
<base>Logger</base>
<trans>Logger</trans>
</entry>
<entry>
<base>LABEL_LOGGERDEMO_STATUS</base>
<trans>Status</trans>
</entry>
<entry>
<base>ENABLED</base>
<trans>Enabled</trans>
</entry>
<entry>
<base>DISABLED</base>
<trans>Disabled</trans>
</entry>
<entry>
<base>SAVE</base>
<trans>Save</trans>
</entry>
</lexicon>
Now, re-select the Logger function and the tags should now be replaced by English phrases. We can very easily add translations for other languages by adding new locale files in the same hierarchy.
Note: The LABEL_LOGGERDEMO_INTERVAL tags has intentionally been left untranslated. Why don't you fix it now?
Exercise 6: Adding a new event type
Let us continue building on this example. Let us say that you want to add a hook to the logger demo, enabling other third party applications to receive a notification whenever the logger settings are changed. We need a new event type for this. Let us create a new event called loggerdemo-update:
mkdir -p /etc/e-smith/events/loggerdemo-update
The idea is that we will arrange for our user interface function to signal this new event whenever the settings are changed, instead of directly expanding the /etc/crontab file. Edit the /usr/lib/perl5/site_perl/esmith/FormMagick/Panels/loggerdemo.pm file and replace the line:
unless (system ("/sbin/e-smith/expand-template", "/etc/crontab") == 0)
with the line:
unless (system ("/sbin/e-smith/signal-event", "loggerdemo-update") == 0)
Now the loggerdemo user interface signals the new event whenever it saves the new settings to the configuration database. Next, we have to make sure that the event does what we need it to do - rebuild the /etc/crontab file.
cd /etc/e-smith/events/loggerdemo-update
mkdir -p templates2expand/etc
touch templates2expand/etc/crontab
Now the example should work just as before. You can edit the loggerdemo settings in the web manager, and see that the /etc/crontab file changes. But now other applications can also receive notifications of the loggerdemo-update event, by creating symbolic links from the /etc/e-smith/events/loggerdemo-update directory.
Note: Panel implementation code should always signal events, and should never expand templates or modify files directly. These modifications should only be peformed in events.
Exercise 7: Thought experiment - adding a new server application
You have now learned most of the machinery required for integrating a new server application into the SME Server. Consider a hypothetical chat server, with a configuration file called /etc/chatserv.conf.
You would first decide if the chat server needs any new settings. If so, you can create a new service entry in the configuration database to hold those settings.
You would then create a templated version of /etc/chatserv.conf, by creating the directory /etc/e-smith/templates/etc/chatserv.conf with at least one template fragment. The template should generate the correct version of /etc/chatserv.conf based on the configuration database settings. Experiment with different combinations of settings and manually running expand-template /etc/chatserv.conf.
Then create a web manager function (if one is required) enabling users to edit the settings. An event, for example chatserv-update should be signalled whenever the settings are changed. You would normally link into an existing event, but this is a thought experiment.
Then create the file /etc/e-smith/events/chatserv-update/templates2expand/etc/chatserv.conf to ensure that the configuration file is updated. You may also need to restart the chatserv program after changing its configuration, so you'll need a /etc/e-smith/events/chatserver-update/services2adjust/chatserv link which contains the word restart.
Finally, figure out when you'll need these scripts to be run. At a minimum, you'll want to run them whenever the event is signalled indicating that the user has saved new settings from the web manager. A symbolic link should be created, so that the new event triggers your new action scripts. Let's say you also need to reconfigure and restart your server each time a user is added. In that case, you would also create symlinks from the user-create event to your action scripts.
We are done - the new server has been integrated into the system. Note that every single one of these steps involved creating new files and directories, and reading/writing from the configuration database. No existing files were edited! You can easily imagine packaging these new files and directories into an RPM.
Customization guidelines
When creating applications:
*
You can create new configuration parameters.
*
You can add new configuration templates under /etc/e-smith/templates/.
*
You can add fragments to any of the existing templates under /etc/e-smith/templates/.
*
You can arrange for actions to be triggered upon package pre-install, post-install, pre-uninstall, or post-uninstall.
*
You can link new actions into the events listed in the Section called Standard events and their arguments in Chapter 7.
*
You can create new events to allow your feature set to be expanded. Do not create events for any other reason. They are not a replacement for function calls.
*
You can arrange for new server programs to be started up at boot time.
*
Typically you would expand a template at application post-install time, application post-uninstall time, and one or more of the other events.
*
You can add new web functions to the navigation bar. Remember, panels should only retrieve and modify database values, perform validation, and signal events.
That is all! Applications should not make any extensions to the system other than these. For example, an application should not:
*
Change the kernel or add new kernel modules.
*
Edit configuration files directly - templates must be used.
*
Link actions into any events other than the ones listed above or new events that you create. All built-in events other than the ones listed above are subject to change without notice in new SME Server releases.
*
Directly access the per-user email store (i.e. the Maildir and related subdirectories within each user's home directory). This access should be performed via the IMAP server as the location and format of a user's home directory may change between releases.
*
Take over the function of existing servers (i.e. shut down qmail and Apache and take over ports 25 and 80). The SME Server has features for proxying email and web requests to other servers on the system.
Do not expand templates at boot time. The only thing that should be happening during a normal system startup is to start servers. Templates should be expanded when it is necessary to change the system configuration (i.e. when a setting is changed, when the IP address changes, etc.) A normal shutdown or reboot should not trigger configuration changes. The bootstrap-console-save event will be run after a system reconfiguration, but will not run if the system does not require reconfiguration.
==Eigene RPM-Pakete erstellen==
Once you have created a customization for your SME Server by adding new files, directories, and symbolic links (for your actions, events, etc.) - and perhaps also triggering an action to initialize your customization - you are ready to package your customization into an RPM.
A quick introduction to RPMs
All SME Server software packages are distributed as RPM packages. This is the format used by CentOS and other major Linux distributions for distributing applications and other collections of files. The RPM system provides the ability to install, upgrade, remove and (importantly) verify the contents of installed packages.
An RPM essentially consists of an archive of all the files required by a piece of software. Additionally, it includes meta-information describing the software, and scripts which must be run to install or uninstall the software.
Meta-information stored in an RPM includes:
1.
summary and description of the software
2.
package name
3.
version number
4.
copyright information
5.
category/group to which the software belongs
6.
name and email address of the packager
7.
pre-requisites to installing this package
8.
... and more
Selecting and creating RPMs for your application
Your application will typically depend on several components:
1.
Software packages that are shipped as a standard part of the SME Server. You do not need to include any of these packages; they are always present in the runtime environment.
2.
Software packages that are not a standard part of the SME Server, but that are required by your software, and would also be of general use in the runtime environment. For example: a Java runtime environment, libraries that enable communication with devices, etc. If possible, these packages should be made into separate packages, rather than being included in your application. This makes it easier to share them with other applications and enforces version compatiblity.
3.
Software packages that are not a standard part of the SME Server, and that are specific to your software application (i.e. not generally useful in the runtime environment). This is the raw Linux version of your application without any specific SME Server integration code. For example, if your application is already available for Linux in the form of RPMs - these RPMs are what we are referring to. These are referred to as the application RPMs.
4.
Any new files that you have created specifically in order to integrate your application into the SME Server runtime environment - should be packaged into a single RPM, as explained in the next section. This is referred to as the integration RPM.
So, if your application is based on Linux software that has already been packaged into RPMs, then you will need to create one new RPM:the integration RPM.
If, on the other hand, your application is based on Linux software that has not yet been packaged into RPMs, then you will probably need to create at least two RPMs: one or more application RPMs, and the integration RPM.
Finally, for simple customizations (such as the loggerdemo example earlier in this manual) there may be no application RPM at all. This would be typical if the point of the application is to change the server configuration without really adding a new software package. In this case you need only the integration RPM which contains the new template fragments, user interface screens, etc..
All files on the system, except for user data, must be installed by RPMs.
Setting up your RPM development environment
If you haven't done so already, set up an RPM development environment. If you are using an SME Server as your development environment, you will need to alter your user account to enable regular login. If you want to enable account "joe", then you would type the following commands from the root account:
chsh -s /bin/bash joe
db accounts setprop joe Shell /bin/bash
Note: Shell/login access is disabled by default to enhance the security of the SME Server. Shell access should only be provided to users who require it and who can be trusted to maintain system security.
Then you should be able to log in to the server as user "joe", and get a Linux command line prompt. Log in, then type the following commands to set up your RPM work area:
cd ~/
mkdir -p rpms/{SRPMS,BUILD,SOURCES,SPECS,RPMS,lib}
mkdir -p rpms/RPMS/{i386,noarch}
echo "%_topdir $HOME/rpms" > ~/.rpmmacros
You will now find that you have a directory called rpms in which you will do your work. Under this are the following subdirectories:
SOURCES
The base material from which RPMs are built -- source code, tarballs, etc.
BUILD
Working area used by the rpmbuild program during RPM creation
SPECS
Specification files for building RPMs
SRPMS
Source RPMS (created by build process)
RPMS
Binary RPMS (created by build process). Has subdirectories noarch and i386 for architecture independent and x86 platforms respectively.
As you prepare software to turn into RPMs, you will place files in these directories as appropriate. The following sections will describe what goes where as each item is covered.
Tip: As you start work on an RPM for version x.y.z of a package, create a subdirectory rpms/SOURCES/yourpackage-x.y.z/ to work in.
mkdir rpms/SOURCES/yourpackage-x.y.z
Under this directory there should be a subdirectory called root, under which is an image of the file hierarchy that will be installed by the RPM.
mkdir rpms/SOURCES/yourpackage-x.y.z/root
Building an RPM
This section describes the process for building an RPM - step by step.
1.
Choose a name and version number for your package. We are going to package the complete loggerdemo example and will use loggerdemo and 1.0.0 as the name and version number.
2.
Collect all of the files which have been created in the previous sections into the /tmp/ directory. There is one additional file createlinks which looks like this:
#!/usr/bin/perl -w
use esmith::Build::CreateLinks qw(:all);
use File::Basename;
my $panel = "manager";
panel_link("loggerdemo", $panel);
3.
Create the directory hierarchy required for building the RPM. This is very close to the hierarchy on the installed system.
# Change to the SOURCES directory
cd ~/rpms/SOURCES
# Remove old files (check that you don't need anything here!)
rm -rf loggerdemo-1.0.0
# Create new directory
mkdir loggerdemo-1.0.0
cd loggerdemo-1.0.0
# The crontab template fragment
mkdir -p root/etc/e-smith/templates/etc/crontab
cp /tmp/25templatedemo root/etc/e-smith/templates/etc/crontab
# The web panel description
mkdir -p root/etc/e-smith/web/functions
cp -p /tmp/loggerdemo !$
# The web panel implementation
mkdir -p root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel
cp -p /tmp/loggerdemo.pm !$
# The web panel English localisation
mkdir -p root/etc/e-smith/locale/en-us/etc/e-smith/web/functions
cp -p /tmp/loggerdemo-en !$
# The createlinks auxiliary file
cp -p /tmp/createlinks .
Your directory structure should now look like this:
[gordonr@sevendev1 loggerdemo-1.0.0]$ find . -type f
./root/etc/e-smith/templates/etc/crontab/25templatedemo
./root/etc/e-smith/locale/en-us/etc/e-smith/web/functions/loggerdemo
./root/etc/e-smith/web/functions/loggerdemo
./root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/loggerdemo.pm
./createlinks
4.
Package the directory into a tarball:
cd ~/rpms/SOURCES
tar zcvf loggerdemo-1.0.0.tar.gz loggerdemo-1.0.0
5.
Create the RPM specification "SPEC" file ~/rpms/SPECS/loggerdemo.spec which looks like this:
%define name loggerdemo
%define version 1.0.0
%define release 01
Summary: SME Server logger demo
Name: %{name}
Version: %{version}
Release: %{release}
License: GPL
Group: Networking/Daemons
Source: %{name}-%{version}.tar.gz
Packager: Fred Frog <red@example.com>
BuildRoot: /var/tmp/%{name}-%{version}-%{release}-buildroot
BuildArchitectures: noarch
%description
Logger Demo sample application.
%changelog
* Thu Feb 2 2006 Fred Frog <fred@example.com>
- 1.0.0-01
- Original version
%prep
%setup
%build
perl createlinks
DEFAULTS=root/etc/e-smith/db/configuration/defaults/loggerdemo
mkdir -p $DEFAULTS
echo "service" > $DEFAULTS/type
echo "enabled" > $DEFAULTS/status
echo "10" > $DEFAULTS/Interval
%install
rm -rf $RPM_BUILD_ROOT
(cd root ; find . -depth -print | cpio -dump $RPM_BUILD_ROOT)
rm -f %{name}-%{version}-filelist
/sbin/e-smith/genfilelist $RPM_BUILD_ROOT > %{name}-%{version}-filelist
%clean
rm -rf $RPM_BUILD_ROOT
%post
/etc/e-smith/events/actions/initialize-default-databases
/etc/e-smith/events/actions/navigation-conf
/sbin/e-smith/expand-template /etc/crontab
true
%postun
/sbin/e-smith/expand-template /etc/crontab
true
%files -f %{name}-%{version}-filelist
%defattr(-,root,root)
Note the %post (post-installation) and %postun (post-uninstallation) statements which expand the /etc/crontab template after installing or uninstalling the RPM.
6.
Check that your RPM will build OK with "build prepare":
cd ~/rpms/SPECS
rpmbuild -bp loggerdemo.spec
The last line of output should be + exit 0 if rpmbuild is successful.
7.
Run the rpmbuild command again to actually create your RPM with the "build all" options:
rpmbuild -ba loggerdemo.spec
If everything was successful, the last line of output should again be + exit 0.
8.
The RPMs should have been generated and put into ~/rpms/RPMS/noarch/ as this program can run equally well on any platform. A source RPM should also exist in ~/rpms/SRPMS/.
9.
Test your RPM by installing it on an SME Server test box.
Note: RPMs need to be installed as root, but you should not log in as the root user. Instead, you should create a normal user and provide them with 'root' privileges via the sudo command. To provide full sudo rights to the user joe, use the su - -c /usr/sbin/visudo and add the following line to the end of the file:
joe ALL=(ALL) ALL
You can then use the sudo to run commands as root when required. You will be prompted for your password (not the root password) which ensures that someone else isn't using your terminal.
[joe@sevendev1 SPECS]$ sudo rpm -Uvh \
~/rpms/RPMS/noarch/loggerdemo-1.0.0-01.noarch.rpm
Preparing... ########################################### [100%]
1:loggerdemo ########################################### [100%]
Migrating existing database mailpatterns
Migrating existing database hosts
Migrating existing database configuration
Migrating existing database yum_repositories
Migrating existing database networks
Migrating existing database yum_updates
Migrating existing database yum_installed
Migrating existing database spamassassin
Migrating existing database accounts
Migrating existing database backups
Migrating existing database yum_available
Migrating existing database domains
Note: In almost all instances you should use the -U (upgrade) option to rpm instead of the -i (install) option. The -i option should be reserved for special operations, such as installing kernels.
The customization should be fully installed, and the /etc/crontab file should show the customization. Then remove the customization:
# remove the RPM
sudo rpm -e loggerdemo
The customization should be completely gone, and the /etc/crontab file should look the way it did before.
The createlinks script
The source tarballs of an RPM should not include symbolic links as they are difficult to store under many version control systems and cause issues when generating patches. Since the SME Server uses many symbolic links, there are simple methods for creating the ones required. This is done through the createlinks script which is called from the %build section of the SPEC file. Let's examine one. It starts with the standard Perl script header and an import of the required module:
#!/usr/bin/perl -w
use esmith::Build::CreateLinks qw(:all);
The templates2events function can be used to create the appropriate templates2expand links in various events:
my $imap = "/var/service/imap";
my $imaps = "/var/service/imaps";
templates2events("/etc/dovecot.conf", qw(bootstrap-console-save console-save));
templates2events("$imap/config", qw(bootstrap-console-save email-update));
templates2events("$imaps/config", qw(bootstrap-console-save email-update));
Note that the first argument is a filename and the second argument is a list of events in which to create the link. The safe_symlink function can be used to create a generic symbolic link, as well as the directory hierarchy enclosing that link:
for my $event (qw(
email-update
ldap-update
network-create
network-delete
))
{
safe_symlink("sigusr1", "root/etc/e-smith/events/$event/services2adjust/imap");
}
safe_symlink("daemontools", "root/etc/rc.d/init.d/imap");
The event_link function is used to create the links from the event directories to the generic actions directory. For example:
for my $event (qw(post-upgrade))
{
event_link("imap-relocate-maildirs", $event, "05");
}
creates a symbolic link S05imap-relocate-maildirs in the post-upgrade event. The target of the symbolic link will be the imap-relocate-maildirs script in the /etc/e-smith/events/actions/ directory.
Finally, the service_link_enhanced function makes it simple to create the /etc/rc.d/rc7.d and similar startup symlinks:
service_link_enhanced("imap", "S55", "7");
service_link_enhanced("imap", "K45", "6");
service_link_enhanced("imap", "K45", "0");
service_link_enhanced("imap", "K45", "1");
More documentation on this module can be seen with the command perldoc esmith::Build::CreateLinks.
==Die SME-Entwicklungsumgebung==
Configuring your development environment
The SME Server source code is checked into CVS at SourceForge. The project has a SourceForge page at https://sourceforge.net/projects/smeserver/. All packages are built using mezzanine, and an overview of mezzanine is provided here: https://wiki.caosity.org/tiki-index.php?page=Package+Maintenance
Reminder: The SME Server is released under the GPL
The SME Server source code is released under the GPL. You must release the source code to all modifications. If you make improvements, please raise a bug and attach a patch so the change can be discussed and pulled back into the base for everyone to share.
Do I need a SourceForge account?
Not really. Only developers who are going to put patches back into CVS and build new packages need SourceForge CVS access. The sources are freely available and patches are gratefully received. Just follow the instructions in this section and attach the patch(es) to the Bugzilla entry, explaining why the change should be made.
Add some useful aliases
Mezzanine needs to know to use the SourceForge CVS repository in order to checkout/get or import a package. Once a package is checked out from SourceForge, mezzanine/CVS will remember where it came from and check it back into the same CVS repository. You should not set the CVSROOT environment variable as it makes it too easy to accidentally import code into the wrong CVS repository when working with multiple repositories.
Add the following lines to ~/.bashrc (or a file such as /etc/profile.d/smebuild.sh to configure it for all users on this machine):
export CVS_RSH=ssh # tell CVS to use ssh
# DO NOT set CVSROOT
sf_cvsroot=smeserver.cvs.sourceforge.net:/cvsroot/smeserver
sf_cvsroot_anon=:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver
alias mzgetsf="mzget --dir $sf_cvsroot"
alias mzimportsf="mzimport --dir $sf_cvsroot"
alias mzgetsf_anon="mzget --dir $sf_cvsroot_anon"
You can now use the mzgetsf_anon (or msgetsf if you have a developer account) to retrieve a package from SourceForge. If you use mzget, you will get an error message to remind you that CVSROOT has not been set, and should not be set. Note that you can use the other mezzanine commands, such as mzclean and mzsync normally as the CVS repository is already known.
[gordonr@smebuild tmp]$ mzget smeserver-yum
cvs checkout: No CVSROOT specified! Please use the `-d' option
cvs [checkout aborted]: or set the CVSROOT environment variable.
An unknown error must have occured, because the command returned 256
Create your work area
It is a good idea to perform all of your checkouts in a standard directory hierarchy. This makes it easier to move around between the packages. On a shared build server, it also makes it easier for people to recover your work if required (for example when you are on holidays). If you want to use filesharing to edit files from a networked PC, you might like to use ~/home/smeserver/. On a shared build server, you might even use something like /builds/users/$LOGNAME/smeserver/ so the files are not in your home directory and are accessible to all developers.
Note: Reminder: All files must be checked in using Unix text format.
[gordonr@smebuild ~]$ mkdir ~/smeserver
[gordonr@smebuild ~]$ cd ~/smeserver
Modifying an SME Server package
Raise a Bugzilla entry
Before you make any changes to a package, you need to have a Bugzilla entry which specifies the problem and preferably proposes a fix. Raising the bug before you do the work allows others to comment on the proposed approach and can save significant time when you go to submit the changes. The change should also be approved by the Development Manager if it is meant for near-term release. You will need the Bugzilla bugid when you check in the changes.
All changes must have an associated Bugzilla entry. The bug tracker is here: http://bugs.contribs.org/
If a relevant bug does not exist, raise one. If the bug exists, assign it to yourself to show that you are working on it:
For this exercise, let's look at bug 1174 "yum-import-keys should not import duplicates" http://bugs.contribs.org/show_bug.cgi?id=1174.
Choose the package(s) to modify
If you are modifying an existing file, the simplest way to determine the package is to install the relevant version and run rpm -qf on the file to be modified:
[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys
smeserver-yum-1.1.2-05
and so, we want to modify the smeserver-yum package.
Note: You can view a complete list of the packages checked into SourceForge CVS http://smeserver.cvs.sourceforge.net/smeserver/. On rare occasions, the sources of particular packages may be slightly out of date as there is a small delay (typically only a few hours) between changes to the developer CVS and the anonymous CVS. In practice, this rarely matters and at worst it requires a merging of your changes with any other recent changes to the package.
All packages on the SME Server ISO/CD must be checked into SourceForge CVS. The only exceptions are packages which come from the following upstream repositories: CentOS and dag.
mzgetsf_anon: SourceForge anonymous CVS checkout
You can now retrieve one of the packages from SourceForge. In this case, we want to modify the smeserver-yum package, so let's retrieve it from SourceForge:
[gordonr@smebuild smeserver]$ mzgetsf_anon smeserver-yum
U smeserver-yum/ChangeLog
U smeserver-yum/F/smeserver-yum.spec
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch2
U smeserver-yum/P/smeserver-yum-1.2.0-ModifyUpdateDBs.patch
U smeserver-yum/S/smeserver-yum-1.2.0.tar.gz
Note: Use mzgetsf if you have a SME Server developer account.
Mezzanine package hierarchy
[gordonr@smebuild smeserver]$ cd smeserver-yum/
[gordonr@smebuild smeserver-yum]$ ls
ChangeLog CVS F P S smeserver-yum
If you examine the checked out package, you will see the following directories:
*
The SME Server project does not use the ChangeLog file as we maintain all of our changes in the %changelog section of the SPEC file.
*
The CVS directory contains CVS state information, such as the location of the CVS repository:
[gordonr@smebuild smeserver-yum]$ ls CVS
Entries Entries.Log Repository Root
*
The F directory contains the SPEC for this package:
[gordonr@smebuild smeserver-yum]$ ls F
CVS smeserver-yum.spec
*
The P directory contains any patches which are required to update the source in the tarball(s) to the current version:
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
*
The S directory contains the source tarball(s):
[gordonr@smebuild smeserver-yum]$ ls S
CVS smeserver-yum-1.2.0.tar.gz
Note: The smeserver-yum subdirectory is a leftover from an incorrect package import. It can be ignored and will be removed when maintenance is next performed on the CVS repository.
mzclean: Clean out modified files
Before making changes to a package, you should ensure that you have a clean copy of the latest sources. Remember - other people may also be working on the package and making changes in SourceForge. You can perform a "merge" when you check in your changes, but it is easiest to start with an up-to-date copy. It is even possible that someone else has already fixed the bug you are working on!
[gordonr@smebuild smeserver-yum]$ mzclean
Cleaning and resyncing smeserver-yum, please wait....
Removing work....
Cleanup of smeserver-yum complete.
[gordonr@smebuild smeserver-yum]$ ls
ChangeLog CVS F P S
Tip: Ensure that you have copied any modified files out of your work tree prior to runing mzclean.
Note that the stray smeserver-yum subdirectory has been removed.
mzprep: Build working tree
The mzprep command runs rpmbuild -bp on your SPEC file, sources and patches to generate a working tree. You can make changes in the working tree and use mezzanine to generate the patches between the previous version and your new files.
[gordonr@smebuild smeserver-yum]$ mzprep
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
You may now chdir to work to make changes.
Use "mzpatch -n <patch_name>" to generate a patch when done.
[gordonr@smebuild smeserver-yum]$ ls
ChangeLog CVS F P S work
The only obvious change is the creation of a work/ subdirectory. Let's have a look in there. Under the work/ directory is smeserver-yum-1.2.0/ which contains the patched sources for this version of smeserver-yum.
[gordonr@smebuild smeserver-yum]$ ls work/
smeserver-yum-1.2.0
Under smeserver-yum-1.2.0/ are the createlinks script, the root hierarchy and a few generated files, explained below.
[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/
createlinks debugsources.list smeserver-yum-1.1.2-26-filelist
debugfiles.list root
SME Server package directory layout
Table 14-1. SME Server package directory layout
Directory/File Description
createlinks Builds action, panel and initscript links.
po Package translations, if gettext is used.
root The top-level directory of the package, which will be the / directory of the installed system. So, root/usr/lib/perl5/site_perl becomes /usr/lib/perl5/site_perl on the installed system. Note that the somewhat confusing root/root is the /root directory of the installed system.
update-po Helper script to generate translation binary files, if gettext is used.
debugsources.list debugfiles.list Generated by RPM build - lists this files which are built with debug flags
smeserver-yum-1.1.2-26-filelist Generated by RPM build - lists the files to be packaged, and their permissions and ownership
The root directory hierarchy
The work/smeserver-yum-1.2.0/root/ hierarchy contains the files which will be installed on the target machine.
[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/root/
etc sbin service usr var
Modifying a file
OK - now we can actually make a change. We want to modify the file which will be installed as /etc/e-smith/events/actions/yum-import-keys, so the actual file we need to modify is: work/smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys.
Use your favourite editor (in Unix text mode) to modify the file, adding a comment where the change needs to be made. Here's a diff showing such a comment:
[gordonr@smebuild tmp]$ diff -u yum-import-keys.orig yum-import-keys
--- yum-import-keys.orig 2006-05-22 16:28:51.734609534 +1000
+++ yum-import-keys 2006-05-22 16:28:39.006814950 +1000
@@ -30,6 +30,7 @@
for my $key ( grep {!/^\./} readdir(DIR) )
{
warn "Importing key $key\n";
+ # TODO: Skip keys which have previously been imported
system("rpm", "--import", $key) == 0 or
warn "Couldn't rpm --import $key\n";
mzpatch: Create a patch
Mezzanine automates the creation of patches through the mzpatch command.
Note: Always run mzpatch from the top-level directory of your package (e.g. ~/smeserver/smeserver-yum/).
[gordonr@smebuild actions]$ cd ~/smeserver/smeserver-yum/
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
You will see from the listing above that the patches should be named package-version-patchname.patch. You should also ensure that the patch name you are about to choose doesn't already exist, or you may overwrite an older version of the patch.
[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs [server aborted]: "add" requires write access to the repository
You do not have write access to the master repository.
srctool: Error: Addition of P/smeserver-yum-1.2.0-ImportKeysComment.patch failed.
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ImportKeysComment.patch
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
Mezzanine has created the patch, but it has not committed it to CVS. This allows you to revise the patch until it works, prior to committing it. If you do not have a developer account you will get an error when mezzanine attempts to reserve the patch name in CVS, but the patch will still be created in the P directory.
The patch looks quite similar to the one we saw previously, except for the pathname to the modified file:
[gordonr@smebuild smeserver-yum]$ cat P/smeserver-yum-1.2.0-ImportKeysComment.patch
diff -Nur -x '*.orig' -x '*.rej' smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys
--- smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2005-09-26 12:30:54.000000000 +1000
+++ mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2006-05-22 16:35:57.752811633 +1000
@@ -30,6 +30,7 @@
for my $key ( grep {!/^\./} readdir(DIR) )
{
warn "Importing key $key\n";
+ # TODO: Skip keys which have previously been imported
system("rpm", "--import", $key) == 0 or
warn "Couldn't rpm --import $key\n";
Building a new package, with the patch
The next step is to change the package SPEC file so that the patch is applied.
Update the Release tag
The first change you should make is to update the "Release" tag in the SPEC file. You should do this by appending your initials and sequence number to the Release number. In most SME Server SPEC files, the Release number is set via the release macro, and so you should set it there.
Summary: YUM, an rpm updater
%define name smeserver-yum
Name: %{name}
%define version 1.2.0
- %define release 05
+ %define release 05ff01
Please just append your initials and a sequence number to the release tag. Do not change the package name (e.g. to smeserver-yum-fred) or the version number. This policy makes it obvious that your update is a patch based on smeserver-yum-1.2.0-05.
If a change has been approved by the Development Manager, it is given an official version/release number (e.g. smeserver-yum-1.2.0-06). Always tag unofficial changes with your initials and sequence number.
If an upstream (e.g. CentOS) package needs to be patched and it is not possible to wait for the patch to come from the upstream source, the release number should be modified with the special tag sme and a sequence number (e.g. clamav-0.88.2-1sme01). The sme tag is reserved for changes approved by the Development Manager.
Add a changelog entry
The SME Server project maintains history of packages changes in the %changelog section of the SPEC file. The changelog entry should be a simple description of the change of behaviour and must include a reference to an SME Server project Bugzilla entry (1174) and the new version number (1.2.0-05ff01):
%changelog
+ * Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
+ - Initial work on avoiding duplicate RPM keys [SME: 1174]
* Mon May 1 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
- Remove stray yum.pm.orig file. [SME: 1350]
The first line of the changelog entry shows when the change was made, by whom and notes the new version number. The other lines describe the change and refer to the Bugzilla entry or entries, using the format [SME: 1174] as a shorthand.
Add a patch reference
You now need to tell RPM to retrieve and apply the new patch. This needs to be done in two places, which is annoying, but allows for patches to be carried around, but not applied (e.g. because they are not quite finished). The first change is near the top of the file:
Patch0: smeserver-yum-1.2.0-DisplayStatus.patch
Patch1: smeserver-yum-1.2.0-ModifyUpdateDBs.patch
Patch2: smeserver-yum-1.2.0-DisplayStatus.patch2
+ Patch3: smeserver-yum-1.2.0-ImportKeysComment.patch
The second change is in the %prep section:
%prep
%setup
%patch0 -p1
%patch1 -p1
%patch2 -p1
rm root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/yum.pm.orig
+ %patch3 -p1
mzbuild: Build a new package
With all that done, you should be able to build a new package using mzbuild:
[gordonr@smebuild smeserver-yum]$ mzbuild
[...]
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.35037
+ umask 022
+ cd /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/build.mezz/BUILD
+ cd smeserver-yum-1.2.0
+ /bin/rm -rf /var/tmp/mezzanine-buildroot.18508
+ exit 0
Package build succeeded. Output files are:
smeserver-yum-1.2.0-05ff01.src.rpm
smeserver-yum-1.2.0-05ff01.noarch.rpm
Check the package
You should now perform some simple checks on the package before installing it for testing.
*
Does the built package have your initials in the release tag? We never want to have two packages with the same name and version number.
[gordonr@smebuild smeserver-yum]$ ls *.rpm
smeserver-yum-1.2.0-05ff01.noarch.rpm smeserver-yum-1.2.0-05ff01.src.rpm
*
Did you update the changelog? It's easy to forget, but vital to do.
[gordonr@smebuild smeserver-yum]$ rpm -qp --changelog \
smeserver-yum-1.2.0-05ff01.noarch.rpm | head -8
* Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
- Initial work on avoiding duplicate RPM keys [SME: 1174]
* Mon May 01 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
- Remove stray yum.pm.orig file. [SME: 1350]
*
Did you remember to apply the patch? It happens, and leads to head scratching "I knew I made that change".
[gordonr@smebuild smeserver-yum]$ mzbuild | grep patch
+ echo 'Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):'
Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):
+ patch -p1 -s
+ echo 'Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):'
Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):
+ patch -p1 -s
+ echo 'Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):'
Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):
+ patch -p1 -s
+ echo 'Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):'
Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):
+ patch -p1 -s
Test your new package
You should now upgrade your test system:
[gordonr@sevendev1 tmp]$ sudo rpm -Uvh smeserver-yum-1.2.0-05ff01.noarch.rpm
Preparing... ########################################### [100%]
1:smeserver-yum ########################################### [100%]
and test that you have fixed the bug.
Lather, rinse, repeat
If you haven't fixed the bug, repeat the cycle: make changes, build a new patch, update the release number (e.g. to 05ff02, 05ff03, etc.), build a new package, test. Don't worry about using up release numbers.
Note: Actually fixing bug 1174 is left as an exercise for the reader. Fame and fortune (or at least thanks) await the person who does.
Attach your patch(es) to the bug
You should now attach your patch(es) and SPEC file changes to the appropriate Bugzilla entry. If the change is the simple application of a patch, you do not need to attach the whole SPEC file - just the changelog entry.
Please attach the patch(es) and ensure that they are in Unix text format. Patches make it obvious which sections of code have been changed and attaching them ensures that their formatting is preserved.
SourceForge developer CVS access
If you are going to work on many bugs, or wish to help with the many tasks involved in a release, you should create a SourceForge developer account.
You do not need a SourceForge account in order to do development. You can contribute very effectively by producing patches and attaching them to the Bugzilla entries.
Create a SourceForge account
*
Create a SourceForge user id
*
Contact the Development Manager (or one of the other project admins) listed on the SourceForge project page, detailing the packages you would like to work on
*
The project admin will add you as a developer
*
Upload an SSH2 key to SourceForge, following the SourceForge documentation: https://sourceforge.net/docs/F02/en/#key_posting
*
You will also need to generate another SSH2 key for access to the smebuild build server
mzgetsf: SourceForge developer CVS checkout
Once you have SourceForge CVS access, you can use mzgetsf (instead of mzgetsf_anon) to retrieve the sources. Your copy will be writable and you will be able to put changes back to CVS. When you run mzpatch you should not see an error during the cvs add step.
[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs add: use 'cvs commit' to add this file permanently
Patch added to SCM. Use 'mzput' to upload to repository.
mzput: Put the changes back to CVS
Instead of attaching the patch to the Bugzilla entry, you will be able to put it back to CVS. Mezzanine normally edits the ChangeLog, but we don't need to do that as we keep the changes in the %changelog section of the SPEC file. The -m option to mzput allows a simple comment to be appended, which is all that is required:
[gordonr@smebuild smeserver-yum]$ mzput -m 'See changelog'
Build the official package
Official packages have unadorned release tags (e.g. smeserver-yum-1.2.0-05) and must be built on the official build servers. There are three steps involved - build the RPMs (mzbuild), sign the RPMs (rpm --addsign *.rpm) and release the RPMs (release_rpms *.rpm).
mzimportsf: SourceForge package import
If a package has not already been checked into SourceForge CVS, you can use mzimportsf to import it. You should be careful to run this command from a directory which does not contain an imported package or mezzanine may import the package underneath the existing package.
mzmerge: Merge changed source RPM
If a new source RPM has been built without corresponding changes in SourceForge CVS, you can use mzmerge to merge in the changes. This can also be used to merge in changes from a modified upstream package. Care should be exercised with this command, especially if you want to maintain previous patches.
{{Overview/de}}
=Kapitel 3: SME Server Pakete erstellen - Schritt für Schritt=
==Beginn==
The best way to get started is to install an SME Server and start experimenting with it. Download a copy from http://mirror.contribs.org/smeserver/releases/7/iso/ and burn your own CD.
If you (or any developers at your organization) have multiple computers on a home network, a cablemodem, DSL, or dialup connection, and an old Pentium machine that you don't need, we recommend installing the SME Server software on the old Pentium machine, and using it as a home gateway and firewall.
Note: The SME Server software erases all data from the PC on which it is installed, to turn it into a dedicated server that can run 24x7. Do not install it on a PC unless you are prepared to erase all of its data!
Alternatively, you can install the SME Server on a corporate LAN in server/gateway mode (creating a small private network behind a firewall that occupies a single IP address on the Internet) or in server-only mode - in which the SME Server provides network services to other computers as a peer on the network.
Note: Warning! The server-only mode is designed for LAN environments that already have a firewall/gateway.
In addition to feeling comfortable installing and using the SME Server software, you should also have a working knowledge of Linux, including use of the command line tools.
You should also be familiar with the perl programming language. Most of the SME Server software is written in perl, and the configuration template mechanism is based on perl.
It is strongly recommended that you obtain and read a copy of the book Maximum RPM (ISBN 067231 1054) or study the on-line version available at http://www.rpm.org/max-rpm/.
You also need to know how to use one of the Linux text editors such as vi, nano or pico. It is also possible to edit files on a remote machine and copy them to the server. However, it is important that the files are converted to Unix text format.
Creating a development environment
Packages which do not require compilation, for example shell and perl scripts, can be built on the SME Server platform. All of the examples in this documentation can be performed on a standard SME Server installation.
Before attempting to compile any software, you should check whether the package is available from one of the many well-maintained RPM repositories. Using these RPMs will ensure compatibility with the other RPMs on the SME Server. You are likely to find the package you want in either the CentOS or Dag Wieers repositories.
If an RPM does not already exist, you should install a CentOS developer workstation or server for SME Server development. You will also need to install the e-smith-devtools packages which can be found on the SME Server CD.
Note: We strongly recommend against installing development tools, such as compilers, on any production servers, especially those which are accessible from the Internet.
==Customizing des SME Servers==
Once you have studied the architecture of the SME Server, it is best to try to make some small customizations to become comfortable with the concepts. The number one rule to remember is: customizations always involve adding files to the server, rather than modifying existing files. This is very important, as it enables customizations to be easily packaged, and mixed and matched. The unique architecture of the SME Server enables virtually anything to be customized by adding a file in the correct location.
Exercise 1: Changing a configuration template
Let us say that you wish to customize your server so that it runs a specified program every twenty minutes. To simplify the problem, let us assume that this program simply adds a line of dots to the log file (/var/log/messages), i.e.:
/usr/bin/logger -t "Demo" "......"
Normally you would accomplish this by adding a line to the /etc/crontab file, which is the standard Linux mechanism for running scheduled jobs. However, the default /etc/crontab file looks something like this on an SME Server:
#------------------------------------------------------------
# !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://wiki.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
# logrotate
12 1 */7 * * root /sbin/e-smith/signal-event logrotate
Note the auto-generated comment block which reminds you not to edit the file. If you do, your changes will be overwritten when the template is next expanded by a system event. We want to append a new line that looks like this (read the Linux crontab documentation to understand the format of crontab entries):
*/20 * * * * root /usr/bin/logger -t "Demo" "......"
Remember that we cannot simply edit the /etc/crontab file. The rule is that we must perform this customization by adding a new file to the system. To get an idea how to do this, have a look at the contents of the template for /etc/crontab:
[gordonr@smebuild crontab]$ pwd
/etc/e-smith/templates/etc/crontab
[gordonr@smebuild crontab]$ ls
00setup 10runparts 20statusreport 65_logrotate email
Each of the files in that directory is a template fragment. The SME Server builds the /etc/crontab file by assembling those fragments and running them through the template processor.
To make your customization, create your own additional fragment by creating a file in this directory called 25templatedemo with the following contents:
# Template demo crontab entry:
*/20 * * * * root /usr/bin/logger -t "Demo" "......"
Next time the SME Server regenerates the /etc/crontab file, it will contain your additional fragment. Starting the name with the prefix "25" forces the template fragment to go between the "20statusreport" and "65_logrotate" fragments. Force the /etc/crontab file to be generated immediately by typing the command:
expand-template /etc/crontab
If you look at the /etc/crontab file now, you should see your new fragment at the appropriate place, and your customization will take effect immediately (as cron notices when its configuration file has been changed). Check /var/log/messages to see the results.
To package this customization, you will need to create an RPM package that contains this single file, and ensures that the /etc/crontab template is expanded in the relevant events. You should also call one of these events in the RPM post-install section to ensure that the template is expanded without further action. Installing that RPM on any SME Server will cause the customization to occur, and will start printing the line of dots to the /var/log/messages logfile every 20 minutes.
The final point to note here is that if you remove your new file 25templatedemo and re-expand the /etc/crontab template, the crontab will go back to the way it was, and your customization will disappear cleanly. Therefore you should put a post-uninstallation script into your RPM package that runs the appropriate events to expand the templates once more. That will result in a package that installs and uninstalls cleanly.
Remember that for testing you can call expand-template directly in the post-install and post-uninstall sections, but released software should use the templates2expand mechanism to request template expansion in the relevant events.
Exercise 2: The magic of templates
For the next exercise, let us build on the first one. You have already created an exciting (o.k. not that exciting) new capability - the ability of the server to write dots into the log file. Let us now take advantage of the fact that the template processor can fill in values from the configuration database.
Edit the /etc/e-smith/templates/etc/crontab/25templatedemo file again, this time with the following contents:
# Template demo crontab entry:
*/20 * * * * root /usr/bin/logger -t "Demo3" "... {
use esmith::AccountsDB;
$adb = esmith::AccountsDB->open_ro;
$adb->get_prop('admin', 'ForwardAddress') || 'admin';
}
Once again, regenerate the template by typing:
expand-template /etc/crontab
If you look at the new /etc/crontab file, you will see that the template processor has replaced the block between the braces with the actual email address of the administrator, which is defined in the accounts database.
Note: Be careful with the placement of the braces in the example. We want two lines of output - the comment and the line starting with the asterisk. If you move the opening brace onto a new line, you will end up with three lines of output.
Whitespace is not signifcant and the code within braces should be formatted "nicely". However, whitespace outside braces is significant and will be copied literally to the output file.
You could check that value with the db accounts show admin command. With this change to the template, the message which will go to the /var/log/messages log file every 20 minutes will contain the actual current administrative email address, rather than a hardcoded text message.
Now here is the exciting part. Click on the E-mail function in the server manager user interface, and change the administrative email address. If you look at /etc/crontab, you will see that it has been updated with the new email address! Every 20 minutes you will receive a new entry in the messages log which automatically reflects the new setting of the administrator email address.
So by simply creating a single file 25templatedemo, you have created quite a sophisticated customization that changes behaviour based on the system settings. And you have done so without affecting the rest of the system or requiring additional changes to the user interface.
The reason this works is subtle, but follows from the overall architecture. It is critical to study this example until you understand it thoroughly. If you understand exactly how this example works, you understand a substantial part of the SME Server architecture.
To understand this example in detail, let's start from the top by studying the user interface. The implementation section of the E-mail function can be found in the file /usr/lib/perl5/site_perl/esmith/FormMagick/Panel/emailsettings.pm. If you study the part of this script that gets executed when the user clicks Save, you will find some Perl code that saves the administrator e-mail address into the configuration database and signals the email-update event - thus informing the system that the email settings have changed:
sub change_settings_delivery
{
my ($fm) = @_;
my $q = $fm->{'cgi'};
[...]
my $admin_email = $q->param('AdminEmail');
my $admin = $accounts->get('admin');
if ($admin_email)
{
$admin->merge_props(
EmailForward => 'forward',
ForwardAddress => $admin_email,
);
}
else
{
$admin->merge_props(
EmailForward => 'local',
ForwardAddress => '',
);
}
[...]
unless ( system( "/sbin/e-smith/signal-event", "email-update" ) == 0 )
{
$fm->error('ERROR_UPDATING');
return undef;
}
$fm->success('SUCCESS');
}
When the email-update event is signalled, the SME Server executes all of the action scripts in the /etc/e-smith/events/email-update/ directory. The event also expands all templates as noted in the templates2expand/ hierarchy, including the /etc/crontab template.
That may all seem rather complicated, but it boils down to this: changing the administrator email address automatically rebuilds the /etc/crontab template - and if you have customized that template, your customization will automatically pick up the current administrator email address.
Note that if you wanted additional programs to execute whenever the email settings were changed, you would put all of those programs in the /etc/e-smith/events/actions/ directory, then create symbolic links to them from the /etc/e-smith/events/email-update/ directory. We use symbolic links because you may want your program triggered by other events as well as email-update, and so you create links from all of the relevant event directories back to your action program.
This system is, by design, extensible. For example, you could use this exact same type of customization to send an email message every hour containing the current IP address. This is left as an exercise to the reader.
Exercise 3: Using events and actions
In the SME Server, events are like callbacks in a programming language. The system signals an event whenever something interesting happens (e.g. a user is added, the IP address changes, etc.), which automatically executes all programs in the event directory. Therefore, any applications which need to know when a certain event is happening simply create a symbolic link from the event directory to a handler program, which will get executed whenever the event occurs.
In the previous exercise, we relied on the predefined events and actions in the SME Server to keep the /etc/crontab file up to date. In this example, we will create a new action script that will track the user accounts in the system. This script can be used as a template for any type of application that has its own notion of user accounts and needs to be driven by the existing user interface for adding, deleting, or modifying users.
Start by creating a new file called /etc/e-smith/events/actions/demo-user-tracking with the following contents:
#!/usr/bin/perl -w
# Set up Perl environment and libraries
package esmith;
use strict;
use warnings;
use esmith::ConfigDB;
use esmith::AccountsDB;
# Prepare to access configuration databases
my $db = esmith::ConfigDB->open_ro
or die "Couldn't open ConfigDB\n";
my $accounts = esmith::AccountsDB->open_ro
or die "Couldn't open AccountsDB\n";
# Read domain name from configuration database
my $domain = $db->get_value('DomainName');
# Read command line arguments
my $event = $ARGV [0];
my $id = $ARGV [1];
# If no command line arguments, assume this is the initial setup
# of all users. Process all accounts of type "user" (ignore groups,
# information bays, printers, etc.)
unless ($event and $id)
{
for my $user ($accounts->users)
{
my $key = $user->key;
my $first = $user->prop('FirstName');
my $last = $user->prop('LastName');
system ("/usr/bin/logger", "-t", "Demo3",
"Initializing user $key ($first $last) in domain $domain");
}
exit 0;
}
# If command line arguments are present, then this is a create, modify,
# or delete event, signalled by the SME Server event/action system.
my $user = $accounts->get($id)
or die "User $id does not exist\n";
my $first = $user->prop('FirstName');
my $last = $user->prop('LastName');
if ($event eq 'user-create')
{
system ("/usr/bin/logger", "-t", "Demo3",
"Creating user $id ($first $last) in domain $domain");
}
elsif ($event eq 'user-modify')
{
system ("/usr/bin/logger", "-t", "Demo3",
"Changing user $id to ($first $last) in domain $domain");
}
elsif ($event eq 'user-delete')
{
system ("/usr/bin/logger", "-t", "Demo3",
"Deleting user $id in domain $domain");
}
else
{
system ("/usr/bin/logger", "-t", "Demo3", "Ignoring $event event");
}
exit 0;
Make sure the permissions are correct:
chmod +x /etc/e-smith/events/actions/demo-user-tracking
Now create symbolic links so that this program is executed whenever a user is created, modified, or deleted. Make three symbolic links; one for each event directory:
cd /etc/e-smith/events
ln -s ../actions/demo-user-tracking user-create/S90demo-user-tracking
ln -s ../actions/demo-user-tracking user-modify/S90demo-user-tracking
ln -s ../actions/demo-user-tracking user-delete/S90demo-user-tracking
The S90 prefix ensures that the program will be executed after the standard actions (which typically have prefixes ranging from S15 to S80).
Study this program carefully. It uses many different SME Server capabilities. If invoked from the command line with no arguments, it will read all user accounts from the user database, fetch all the data fields associated with each user, and print this information to the log file. If invoked as an event, the SME Server will automatically pass it the event name and user id as command line arguments; in this case the program will print messages to the log file explaining that it is adding, modifying, or dropping the user.
Trying watching the log file by running the command:
tail -F /var/log/messages
or with the "View log files" panel in the server-manager. Use the standard SME Server user interface to add, modify, or remove users. You should see a stream of comments in the log file.
If you were creating an application that had its own database of user information, you would replace the logfile-writing code with your own code that initialized new users, modified them, etc. Then you would arrange for the RPM post-install script to execute:
/etc/e-smith/events/actions/demo-user-tracking
with no command line arguments. When the application is initially installed, it will immediately read all users from the database and set them up in your application. Then, as users are added, modified, or dropped over time - your code will be invoked each time, to update the application's private user database.
Tip: It is almost always better to extend the existing accounts database with additional properties for your application than to maintain another database.
Exercise 4: Adding new configuration database parameters
New system configuration parameters can be spontaneously invented and added to the configuration database at any time. For example, let us return to our earlier exercise and parameterize the time interval for the log messages by introducing a new parameter called LogInterval.
You can write that parameter into the configuration database, as though it had always existed. For example, type this on the command line:
config set LogInterval 30
You can use config show LogInterval to show that it was set as intended. You can now edit the 25templatedemo file and replace the hardcoded number 20 with the LogInterval parameter. The resulting file will read:
# Template demo crontab entry:
*/{ $LogInterval } * * * * root /usr/bin/logger -t "Demo4" "... {
use esmith::AccountsDB;
$adb = esmith::AccountsDB->open_ro;
$adb->get_prop('admin', 'ForwardAddress');
} ..."
Now you can change the logger interval at any time by typing the following (replace the number 20 with whatever logger interval you want):
config set LogInterval 20
expand-template /etc/crontab
This ability to spontaneously introduce new configuration parameters is very important in the SME Server architecture. The configuration database is a high-level specification of how the overall system is supposed to behave for the end user. Configuration settings are like knobs on a stereo system. The templates, events, and actions, are the underlying machinery to carry out the user's wishes. When adding a new application to the system, it is important to be able to add new knobs on demand.
Now let us say that you want to introduce a parameter to enable or disable this logging function. At this point, you might start thinking of this logging activity as a service that you should be able to enable or disable. In this case the convention is to create a single service entry in the configuration database to manage both parameters.
To implement this, first delete the LogInterval parameter, which will no longer be needed:
db configuration delete LogInterval
Now create a service entry:
db configuration set loggerdemo service status enabled Interval 20
If you examine the configuration database you will see the new entry looks like this:
[root@gsxdev1 ~]# config show loggerdemo
loggerdemo=service
Interval=20
status=enabled
Note: By convention, database keys are lower case, and property names are mixed case (e.g. Interval). The status property is an exception here, and is stored lower case.
Now edit the 25templatedemo file to look like this:
# Template demo crontab entry
{
my $status = $loggerdemo{status} || "disabled";
return "# loggerdemo service is disabled."
unless ($status eq "enabled");
use esmith::AccountsDB;
$adb = esmith::AccountsDB->open_ro;
my $admin_email = $adb->get_prop('admin', 'ForwardAddress')
|| 'admin';
my $interval = $loggerdemo{Interval} || 10;
$OUT = "*/$interval * * * * root /usr/bin/logger";
$OUT .= " -t \"Demo4\" \"... $admin_email ...\"\n";
}
Note: The variable $OUT is a special variable used for output from a template. It is documented in the Text::Template documentation. For now, just think about it as the return value from the template, so set it to the value you want to print from this fragment. Note also that a template fragment can return static text without setting $OUT directly, as shown on the return line above.
This is more complicated than the original template, but it is also more flexible. Note that the initial comment (# Template demo crontab entry) is hardcoded, but the line that follows is generated from the configuration database parameters. The code in the template retrieves the loggerdemo service entry, retrieves the required properties, and returns the appropriate output. To experiment, try different combinations of parameters:
db configuration setprop loggerdemo Interval 10
expand-template /etc/crontab
and
config setprop loggerdemo status disabled
expand-template /etc/crontab
and so on.
Exercise 5: Adding a user interface screen
Let us add a nice user interface screen to adjust the logger interval. Create a new file called /etc/e-smith/web/functions/loggerdemo, with the following contents:
#!/usr/bin/perl -wT
# vim: ft=xml:
#----------------------------------------------------------------------
# heading : Demo
# description : Logger
# navigation : 1000 1000
#----------------------------------------------------------------------
use strict;
use warnings;
use esmith::FormMagick::Panel::loggerdemo;
my $f = esmith::FormMagick::Panel::loggerdemo->new();
$f->display();
__DATA__
<form
title="FORM_TITLE"
header="/etc/e-smith/web/common/head.tmpl"
footer="/etc/e-smith/web/common/foot.tmpl">
<page name="First" pre-event="print_status_message()"
post-event="change_settings">
<field
type="select"
id="loggerdemo_Interval"
options="10,20,30,40,50"
value="get_interval()">
<label>LABEL_LOGGERDEMO_INTERVAL</label>
</field>
<field
type="select"
id="loggerdemo_status"
options="'disabled' => 'DISABLED', 'enabled' => 'ENABLED'"
value="get_status()">
<label>LABEL_LOGGERDEMO_STATUS</label>
</field>
<subroutine src="print_button('SAVE')" />
</page>
</form>
The file above describes the panel layout, which is written in FormMagick XML. Further details about FormMagick can be fiound in the Section called An overview of FormMagick in Chapter 10.
Another file provides the implementation, which goes into /usr/lib/perl5/site_perl/esmith/FormMagick/Panel/loggerdemo.pm:
#!/usr/bin/perl -w
package esmith::FormMagick::Panel::loggerdemo;
use strict;
use warnings;
use esmith::ConfigDB;
use esmith::FormMagick;
our @ISA = qw(esmith::FormMagick Exporter);
our @EXPORT = qw();
our $VERSION = sprintf '%d.%03d', q$Revision: 1.1 $ =~ /: (\d+).(\d+)/;
our $db = esmith::ConfigDB->open or die "Couldn't open ConfigDB\n";
sub get_status
{
return $db->get_prop("loggerdemo", "status");
}
sub get_interval
{
return $db->get_prop("loggerdemo", "Interval");
}
sub change_settings
{
my $fm = shift;
my $q = $fm->{'cgi'};
$db->set_prop('loggerdemo', 'status', $q->param("loggerdemo_status"));
$db->set_prop('loggerdemo', 'Interval', $q->param("loggerdemo_Interval"));
unless ( system ("/sbin/e-smith/expand-template", "/etc/crontab") == 0 )
{
$fm->error('ERROR_UPDATING');
return undef;
}
$fm->success('SUCCESS');
}
1;
Note: This code example calls expand-template. This is used for illustrative purposes only. All templates should be expanded by signalling events.
Similarly to events and actions, the /etc/e-smith/web/functions/ directory is a repository of potentially available functions. To make the new function actually show up in the user interface, create a symbolic link to it from the web manager cgi-bin directory, as follows:
cd /etc/e-smith/web/panels/manager/cgi-bin
ln -s ../../../functions/loggerdemo loggerdemo
Now, make sure the permissions and ownership are correct so that the web server can run your new function:
cd /etc/e-smith/web/functions
chown root:admin loggerdemo
chmod 4750 loggerdemo
We also need to run a program to rebuild the navigation bar. This is only required when adding or removing functions from the manager, and is normally handled automatically. Let's do it manually for now:
/etc/e-smith/events/actions/navigation-conf
Try the server manager now, reloading the navigation bar in your browser, if necessary. You will see a new category called Demo and a new function within it called Logger. It will look a bit bare with some upper-case words which signify phrases which have not been localised. We'll fix that in a moment.
Try experimenting with it - it is a nice little user interface for playing with the logger customization. Every time you change the settings, notice that the /etc/crontab file is updated appropriately.
Adding localizations
The SME Server is designed to support localization into any language. This is done by small files which describe the mapping from the upper case tags (seen above) to the appropriate words for the local language. Enter the following into /etc/e-smith/locale/en-us/etc/e-smith/web/functions/loggerdemo
<lexicon lang="en-us">
<!-- vim: ft=xml:
-->
<entry>
<base>FORM_TITLE</base>
<trans>Logger demo</trans>
</entry>
<entry>
<base>Demo</base>
<trans>Demo</trans>
</entry>
<entry>
<base>Logger</base>
<trans>Logger</trans>
</entry>
<entry>
<base>LABEL_LOGGERDEMO_STATUS</base>
<trans>Status</trans>
</entry>
<entry>
<base>ENABLED</base>
<trans>Enabled</trans>
</entry>
<entry>
<base>DISABLED</base>
<trans>Disabled</trans>
</entry>
<entry>
<base>SAVE</base>
<trans>Save</trans>
</entry>
</lexicon>
Now, re-select the Logger function and the tags should now be replaced by English phrases. We can very easily add translations for other languages by adding new locale files in the same hierarchy.
Note: The LABEL_LOGGERDEMO_INTERVAL tags has intentionally been left untranslated. Why don't you fix it now?
Exercise 6: Adding a new event type
Let us continue building on this example. Let us say that you want to add a hook to the logger demo, enabling other third party applications to receive a notification whenever the logger settings are changed. We need a new event type for this. Let us create a new event called loggerdemo-update:
mkdir -p /etc/e-smith/events/loggerdemo-update
The idea is that we will arrange for our user interface function to signal this new event whenever the settings are changed, instead of directly expanding the /etc/crontab file. Edit the /usr/lib/perl5/site_perl/esmith/FormMagick/Panels/loggerdemo.pm file and replace the line:
unless (system ("/sbin/e-smith/expand-template", "/etc/crontab") == 0)
with the line:
unless (system ("/sbin/e-smith/signal-event", "loggerdemo-update") == 0)
Now the loggerdemo user interface signals the new event whenever it saves the new settings to the configuration database. Next, we have to make sure that the event does what we need it to do - rebuild the /etc/crontab file.
cd /etc/e-smith/events/loggerdemo-update
mkdir -p templates2expand/etc
touch templates2expand/etc/crontab
Now the example should work just as before. You can edit the loggerdemo settings in the web manager, and see that the /etc/crontab file changes. But now other applications can also receive notifications of the loggerdemo-update event, by creating symbolic links from the /etc/e-smith/events/loggerdemo-update directory.
Note: Panel implementation code should always signal events, and should never expand templates or modify files directly. These modifications should only be peformed in events.
Exercise 7: Thought experiment - adding a new server application
You have now learned most of the machinery required for integrating a new server application into the SME Server. Consider a hypothetical chat server, with a configuration file called /etc/chatserv.conf.
You would first decide if the chat server needs any new settings. If so, you can create a new service entry in the configuration database to hold those settings.
You would then create a templated version of /etc/chatserv.conf, by creating the directory /etc/e-smith/templates/etc/chatserv.conf with at least one template fragment. The template should generate the correct version of /etc/chatserv.conf based on the configuration database settings. Experiment with different combinations of settings and manually running expand-template /etc/chatserv.conf.
Then create a web manager function (if one is required) enabling users to edit the settings. An event, for example chatserv-update should be signalled whenever the settings are changed. You would normally link into an existing event, but this is a thought experiment.
Then create the file /etc/e-smith/events/chatserv-update/templates2expand/etc/chatserv.conf to ensure that the configuration file is updated. You may also need to restart the chatserv program after changing its configuration, so you'll need a /etc/e-smith/events/chatserver-update/services2adjust/chatserv link which contains the word restart.
Finally, figure out when you'll need these scripts to be run. At a minimum, you'll want to run them whenever the event is signalled indicating that the user has saved new settings from the web manager. A symbolic link should be created, so that the new event triggers your new action scripts. Let's say you also need to reconfigure and restart your server each time a user is added. In that case, you would also create symlinks from the user-create event to your action scripts.
We are done - the new server has been integrated into the system. Note that every single one of these steps involved creating new files and directories, and reading/writing from the configuration database. No existing files were edited! You can easily imagine packaging these new files and directories into an RPM.
Customization guidelines
When creating applications:
*
You can create new configuration parameters.
*
You can add new configuration templates under /etc/e-smith/templates/.
*
You can add fragments to any of the existing templates under /etc/e-smith/templates/.
*
You can arrange for actions to be triggered upon package pre-install, post-install, pre-uninstall, or post-uninstall.
*
You can link new actions into the events listed in the Section called Standard events and their arguments in Chapter 7.
*
You can create new events to allow your feature set to be expanded. Do not create events for any other reason. They are not a replacement for function calls.
*
You can arrange for new server programs to be started up at boot time.
*
Typically you would expand a template at application post-install time, application post-uninstall time, and one or more of the other events.
*
You can add new web functions to the navigation bar. Remember, panels should only retrieve and modify database values, perform validation, and signal events.
That is all! Applications should not make any extensions to the system other than these. For example, an application should not:
*
Change the kernel or add new kernel modules.
*
Edit configuration files directly - templates must be used.
*
Link actions into any events other than the ones listed above or new events that you create. All built-in events other than the ones listed above are subject to change without notice in new SME Server releases.
*
Directly access the per-user email store (i.e. the Maildir and related subdirectories within each user's home directory). This access should be performed via the IMAP server as the location and format of a user's home directory may change between releases.
*
Take over the function of existing servers (i.e. shut down qmail and Apache and take over ports 25 and 80). The SME Server has features for proxying email and web requests to other servers on the system.
Do not expand templates at boot time. The only thing that should be happening during a normal system startup is to start servers. Templates should be expanded when it is necessary to change the system configuration (i.e. when a setting is changed, when the IP address changes, etc.) A normal shutdown or reboot should not trigger configuration changes. The bootstrap-console-save event will be run after a system reconfiguration, but will not run if the system does not require reconfiguration.
==Eigene RPM-Pakete erstellen==
Once you have created a customization for your SME Server by adding new files, directories, and symbolic links (for your actions, events, etc.) - and perhaps also triggering an action to initialize your customization - you are ready to package your customization into an RPM.
A quick introduction to RPMs
All SME Server software packages are distributed as RPM packages. This is the format used by CentOS and other major Linux distributions for distributing applications and other collections of files. The RPM system provides the ability to install, upgrade, remove and (importantly) verify the contents of installed packages.
An RPM essentially consists of an archive of all the files required by a piece of software. Additionally, it includes meta-information describing the software, and scripts which must be run to install or uninstall the software.
Meta-information stored in an RPM includes:
1.
summary and description of the software
2.
package name
3.
version number
4.
copyright information
5.
category/group to which the software belongs
6.
name and email address of the packager
7.
pre-requisites to installing this package
8.
... and more
Selecting and creating RPMs for your application
Your application will typically depend on several components:
1.
Software packages that are shipped as a standard part of the SME Server. You do not need to include any of these packages; they are always present in the runtime environment.
2.
Software packages that are not a standard part of the SME Server, but that are required by your software, and would also be of general use in the runtime environment. For example: a Java runtime environment, libraries that enable communication with devices, etc. If possible, these packages should be made into separate packages, rather than being included in your application. This makes it easier to share them with other applications and enforces version compatiblity.
3.
Software packages that are not a standard part of the SME Server, and that are specific to your software application (i.e. not generally useful in the runtime environment). This is the raw Linux version of your application without any specific SME Server integration code. For example, if your application is already available for Linux in the form of RPMs - these RPMs are what we are referring to. These are referred to as the application RPMs.
4.
Any new files that you have created specifically in order to integrate your application into the SME Server runtime environment - should be packaged into a single RPM, as explained in the next section. This is referred to as the integration RPM.
So, if your application is based on Linux software that has already been packaged into RPMs, then you will need to create one new RPM:the integration RPM.
If, on the other hand, your application is based on Linux software that has not yet been packaged into RPMs, then you will probably need to create at least two RPMs: one or more application RPMs, and the integration RPM.
Finally, for simple customizations (such as the loggerdemo example earlier in this manual) there may be no application RPM at all. This would be typical if the point of the application is to change the server configuration without really adding a new software package. In this case you need only the integration RPM which contains the new template fragments, user interface screens, etc..
All files on the system, except for user data, must be installed by RPMs.
Setting up your RPM development environment
If you haven't done so already, set up an RPM development environment. If you are using an SME Server as your development environment, you will need to alter your user account to enable regular login. If you want to enable account "joe", then you would type the following commands from the root account:
chsh -s /bin/bash joe
db accounts setprop joe Shell /bin/bash
Note: Shell/login access is disabled by default to enhance the security of the SME Server. Shell access should only be provided to users who require it and who can be trusted to maintain system security.
Then you should be able to log in to the server as user "joe", and get a Linux command line prompt. Log in, then type the following commands to set up your RPM work area:
cd ~/
mkdir -p rpms/{SRPMS,BUILD,SOURCES,SPECS,RPMS,lib}
mkdir -p rpms/RPMS/{i386,noarch}
echo "%_topdir $HOME/rpms" > ~/.rpmmacros
You will now find that you have a directory called rpms in which you will do your work. Under this are the following subdirectories:
SOURCES
The base material from which RPMs are built -- source code, tarballs, etc.
BUILD
Working area used by the rpmbuild program during RPM creation
SPECS
Specification files for building RPMs
SRPMS
Source RPMS (created by build process)
RPMS
Binary RPMS (created by build process). Has subdirectories noarch and i386 for architecture independent and x86 platforms respectively.
As you prepare software to turn into RPMs, you will place files in these directories as appropriate. The following sections will describe what goes where as each item is covered.
Tip: As you start work on an RPM for version x.y.z of a package, create a subdirectory rpms/SOURCES/yourpackage-x.y.z/ to work in.
mkdir rpms/SOURCES/yourpackage-x.y.z
Under this directory there should be a subdirectory called root, under which is an image of the file hierarchy that will be installed by the RPM.
mkdir rpms/SOURCES/yourpackage-x.y.z/root
Building an RPM
This section describes the process for building an RPM - step by step.
1.
Choose a name and version number for your package. We are going to package the complete loggerdemo example and will use loggerdemo and 1.0.0 as the name and version number.
2.
Collect all of the files which have been created in the previous sections into the /tmp/ directory. There is one additional file createlinks which looks like this:
#!/usr/bin/perl -w
use esmith::Build::CreateLinks qw(:all);
use File::Basename;
my $panel = "manager";
panel_link("loggerdemo", $panel);
3.
Create the directory hierarchy required for building the RPM. This is very close to the hierarchy on the installed system.
# Change to the SOURCES directory
cd ~/rpms/SOURCES
# Remove old files (check that you don't need anything here!)
rm -rf loggerdemo-1.0.0
# Create new directory
mkdir loggerdemo-1.0.0
cd loggerdemo-1.0.0
# The crontab template fragment
mkdir -p root/etc/e-smith/templates/etc/crontab
cp /tmp/25templatedemo root/etc/e-smith/templates/etc/crontab
# The web panel description
mkdir -p root/etc/e-smith/web/functions
cp -p /tmp/loggerdemo !$
# The web panel implementation
mkdir -p root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel
cp -p /tmp/loggerdemo.pm !$
# The web panel English localisation
mkdir -p root/etc/e-smith/locale/en-us/etc/e-smith/web/functions
cp -p /tmp/loggerdemo-en !$
# The createlinks auxiliary file
cp -p /tmp/createlinks .
Your directory structure should now look like this:
[gordonr@sevendev1 loggerdemo-1.0.0]$ find . -type f
./root/etc/e-smith/templates/etc/crontab/25templatedemo
./root/etc/e-smith/locale/en-us/etc/e-smith/web/functions/loggerdemo
./root/etc/e-smith/web/functions/loggerdemo
./root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/loggerdemo.pm
./createlinks
4.
Package the directory into a tarball:
cd ~/rpms/SOURCES
tar zcvf loggerdemo-1.0.0.tar.gz loggerdemo-1.0.0
5.
Create the RPM specification "SPEC" file ~/rpms/SPECS/loggerdemo.spec which looks like this:
%define name loggerdemo
%define version 1.0.0
%define release 01
Summary: SME Server logger demo
Name: %{name}
Version: %{version}
Release: %{release}
License: GPL
Group: Networking/Daemons
Source: %{name}-%{version}.tar.gz
Packager: Fred Frog <red@example.com>
BuildRoot: /var/tmp/%{name}-%{version}-%{release}-buildroot
BuildArchitectures: noarch
%description
Logger Demo sample application.
%changelog
* Thu Feb 2 2006 Fred Frog <fred@example.com>
- 1.0.0-01
- Original version
%prep
%setup
%build
perl createlinks
DEFAULTS=root/etc/e-smith/db/configuration/defaults/loggerdemo
mkdir -p $DEFAULTS
echo "service" > $DEFAULTS/type
echo "enabled" > $DEFAULTS/status
echo "10" > $DEFAULTS/Interval
%install
rm -rf $RPM_BUILD_ROOT
(cd root ; find . -depth -print | cpio -dump $RPM_BUILD_ROOT)
rm -f %{name}-%{version}-filelist
/sbin/e-smith/genfilelist $RPM_BUILD_ROOT > %{name}-%{version}-filelist
%clean
rm -rf $RPM_BUILD_ROOT
%post
/etc/e-smith/events/actions/initialize-default-databases
/etc/e-smith/events/actions/navigation-conf
/sbin/e-smith/expand-template /etc/crontab
true
%postun
/sbin/e-smith/expand-template /etc/crontab
true
%files -f %{name}-%{version}-filelist
%defattr(-,root,root)
Note the %post (post-installation) and %postun (post-uninstallation) statements which expand the /etc/crontab template after installing or uninstalling the RPM.
6.
Check that your RPM will build OK with "build prepare":
cd ~/rpms/SPECS
rpmbuild -bp loggerdemo.spec
The last line of output should be + exit 0 if rpmbuild is successful.
7.
Run the rpmbuild command again to actually create your RPM with the "build all" options:
rpmbuild -ba loggerdemo.spec
If everything was successful, the last line of output should again be + exit 0.
8.
The RPMs should have been generated and put into ~/rpms/RPMS/noarch/ as this program can run equally well on any platform. A source RPM should also exist in ~/rpms/SRPMS/.
9.
Test your RPM by installing it on an SME Server test box.
Note: RPMs need to be installed as root, but you should not log in as the root user. Instead, you should create a normal user and provide them with 'root' privileges via the sudo command. To provide full sudo rights to the user joe, use the su - -c /usr/sbin/visudo and add the following line to the end of the file:
joe ALL=(ALL) ALL
You can then use the sudo to run commands as root when required. You will be prompted for your password (not the root password) which ensures that someone else isn't using your terminal.
[joe@sevendev1 SPECS]$ sudo rpm -Uvh \
~/rpms/RPMS/noarch/loggerdemo-1.0.0-01.noarch.rpm
Preparing... ########################################### [100%]
1:loggerdemo ########################################### [100%]
Migrating existing database mailpatterns
Migrating existing database hosts
Migrating existing database configuration
Migrating existing database yum_repositories
Migrating existing database networks
Migrating existing database yum_updates
Migrating existing database yum_installed
Migrating existing database spamassassin
Migrating existing database accounts
Migrating existing database backups
Migrating existing database yum_available
Migrating existing database domains
Note: In almost all instances you should use the -U (upgrade) option to rpm instead of the -i (install) option. The -i option should be reserved for special operations, such as installing kernels.
The customization should be fully installed, and the /etc/crontab file should show the customization. Then remove the customization:
# remove the RPM
sudo rpm -e loggerdemo
The customization should be completely gone, and the /etc/crontab file should look the way it did before.
The createlinks script
The source tarballs of an RPM should not include symbolic links as they are difficult to store under many version control systems and cause issues when generating patches. Since the SME Server uses many symbolic links, there are simple methods for creating the ones required. This is done through the createlinks script which is called from the %build section of the SPEC file. Let's examine one. It starts with the standard Perl script header and an import of the required module:
#!/usr/bin/perl -w
use esmith::Build::CreateLinks qw(:all);
The templates2events function can be used to create the appropriate templates2expand links in various events:
my $imap = "/var/service/imap";
my $imaps = "/var/service/imaps";
templates2events("/etc/dovecot.conf", qw(bootstrap-console-save console-save));
templates2events("$imap/config", qw(bootstrap-console-save email-update));
templates2events("$imaps/config", qw(bootstrap-console-save email-update));
Note that the first argument is a filename and the second argument is a list of events in which to create the link. The safe_symlink function can be used to create a generic symbolic link, as well as the directory hierarchy enclosing that link:
for my $event (qw(
email-update
ldap-update
network-create
network-delete
))
{
safe_symlink("sigusr1", "root/etc/e-smith/events/$event/services2adjust/imap");
}
safe_symlink("daemontools", "root/etc/rc.d/init.d/imap");
The event_link function is used to create the links from the event directories to the generic actions directory. For example:
for my $event (qw(post-upgrade))
{
event_link("imap-relocate-maildirs", $event, "05");
}
creates a symbolic link S05imap-relocate-maildirs in the post-upgrade event. The target of the symbolic link will be the imap-relocate-maildirs script in the /etc/e-smith/events/actions/ directory.
Finally, the service_link_enhanced function makes it simple to create the /etc/rc.d/rc7.d and similar startup symlinks:
service_link_enhanced("imap", "S55", "7");
service_link_enhanced("imap", "K45", "6");
service_link_enhanced("imap", "K45", "0");
service_link_enhanced("imap", "K45", "1");
More documentation on this module can be seen with the command perldoc esmith::Build::CreateLinks.
==Die SME-Entwicklungsumgebung==
Configuring your development environment
The SME Server source code is checked into CVS at SourceForge. The project has a SourceForge page at https://sourceforge.net/projects/smeserver/. All packages are built using mezzanine, and an overview of mezzanine is provided here: https://wiki.caosity.org/tiki-index.php?page=Package+Maintenance
Reminder: The SME Server is released under the GPL
The SME Server source code is released under the GPL. You must release the source code to all modifications. If you make improvements, please raise a bug and attach a patch so the change can be discussed and pulled back into the base for everyone to share.
Do I need a SourceForge account?
Not really. Only developers who are going to put patches back into CVS and build new packages need SourceForge CVS access. The sources are freely available and patches are gratefully received. Just follow the instructions in this section and attach the patch(es) to the Bugzilla entry, explaining why the change should be made.
Add some useful aliases
Mezzanine needs to know to use the SourceForge CVS repository in order to checkout/get or import a package. Once a package is checked out from SourceForge, mezzanine/CVS will remember where it came from and check it back into the same CVS repository. You should not set the CVSROOT environment variable as it makes it too easy to accidentally import code into the wrong CVS repository when working with multiple repositories.
Add the following lines to ~/.bashrc (or a file such as /etc/profile.d/smebuild.sh to configure it for all users on this machine):
export CVS_RSH=ssh # tell CVS to use ssh
# DO NOT set CVSROOT
sf_cvsroot=smeserver.cvs.sourceforge.net:/cvsroot/smeserver
sf_cvsroot_anon=:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver
alias mzgetsf="mzget --dir $sf_cvsroot"
alias mzimportsf="mzimport --dir $sf_cvsroot"
alias mzgetsf_anon="mzget --dir $sf_cvsroot_anon"
You can now use the mzgetsf_anon (or msgetsf if you have a developer account) to retrieve a package from SourceForge. If you use mzget, you will get an error message to remind you that CVSROOT has not been set, and should not be set. Note that you can use the other mezzanine commands, such as mzclean and mzsync normally as the CVS repository is already known.
[gordonr@smebuild tmp]$ mzget smeserver-yum
cvs checkout: No CVSROOT specified! Please use the `-d' option
cvs [checkout aborted]: or set the CVSROOT environment variable.
An unknown error must have occured, because the command returned 256
Create your work area
It is a good idea to perform all of your checkouts in a standard directory hierarchy. This makes it easier to move around between the packages. On a shared build server, it also makes it easier for people to recover your work if required (for example when you are on holidays). If you want to use filesharing to edit files from a networked PC, you might like to use ~/home/smeserver/. On a shared build server, you might even use something like /builds/users/$LOGNAME/smeserver/ so the files are not in your home directory and are accessible to all developers.
Note: Reminder: All files must be checked in using Unix text format.
[gordonr@smebuild ~]$ mkdir ~/smeserver
[gordonr@smebuild ~]$ cd ~/smeserver
Modifying an SME Server package
Raise a Bugzilla entry
Before you make any changes to a package, you need to have a Bugzilla entry which specifies the problem and preferably proposes a fix. Raising the bug before you do the work allows others to comment on the proposed approach and can save significant time when you go to submit the changes. The change should also be approved by the Development Manager if it is meant for near-term release. You will need the Bugzilla bugid when you check in the changes.
All changes must have an associated Bugzilla entry. The bug tracker is here: http://bugs.contribs.org/
If a relevant bug does not exist, raise one. If the bug exists, assign it to yourself to show that you are working on it:
For this exercise, let's look at bug 1174 "yum-import-keys should not import duplicates" http://bugs.contribs.org/show_bug.cgi?id=1174.
Choose the package(s) to modify
If you are modifying an existing file, the simplest way to determine the package is to install the relevant version and run rpm -qf on the file to be modified:
[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys
smeserver-yum-1.1.2-05
and so, we want to modify the smeserver-yum package.
Note: You can view a complete list of the packages checked into SourceForge CVS http://smeserver.cvs.sourceforge.net/smeserver/. On rare occasions, the sources of particular packages may be slightly out of date as there is a small delay (typically only a few hours) between changes to the developer CVS and the anonymous CVS. In practice, this rarely matters and at worst it requires a merging of your changes with any other recent changes to the package.
All packages on the SME Server ISO/CD must be checked into SourceForge CVS. The only exceptions are packages which come from the following upstream repositories: CentOS and dag.
mzgetsf_anon: SourceForge anonymous CVS checkout
You can now retrieve one of the packages from SourceForge. In this case, we want to modify the smeserver-yum package, so let's retrieve it from SourceForge:
[gordonr@smebuild smeserver]$ mzgetsf_anon smeserver-yum
U smeserver-yum/ChangeLog
U smeserver-yum/F/smeserver-yum.spec
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch2
U smeserver-yum/P/smeserver-yum-1.2.0-ModifyUpdateDBs.patch
U smeserver-yum/S/smeserver-yum-1.2.0.tar.gz
Note: Use mzgetsf if you have a SME Server developer account.
Mezzanine package hierarchy
[gordonr@smebuild smeserver]$ cd smeserver-yum/
[gordonr@smebuild smeserver-yum]$ ls
ChangeLog CVS F P S smeserver-yum
If you examine the checked out package, you will see the following directories:
*
The SME Server project does not use the ChangeLog file as we maintain all of our changes in the %changelog section of the SPEC file.
*
The CVS directory contains CVS state information, such as the location of the CVS repository:
[gordonr@smebuild smeserver-yum]$ ls CVS
Entries Entries.Log Repository Root
*
The F directory contains the SPEC for this package:
[gordonr@smebuild smeserver-yum]$ ls F
CVS smeserver-yum.spec
*
The P directory contains any patches which are required to update the source in the tarball(s) to the current version:
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
*
The S directory contains the source tarball(s):
[gordonr@smebuild smeserver-yum]$ ls S
CVS smeserver-yum-1.2.0.tar.gz
Note: The smeserver-yum subdirectory is a leftover from an incorrect package import. It can be ignored and will be removed when maintenance is next performed on the CVS repository.
mzclean: Clean out modified files
Before making changes to a package, you should ensure that you have a clean copy of the latest sources. Remember - other people may also be working on the package and making changes in SourceForge. You can perform a "merge" when you check in your changes, but it is easiest to start with an up-to-date copy. It is even possible that someone else has already fixed the bug you are working on!
[gordonr@smebuild smeserver-yum]$ mzclean
Cleaning and resyncing smeserver-yum, please wait....
Removing work....
Cleanup of smeserver-yum complete.
[gordonr@smebuild smeserver-yum]$ ls
ChangeLog CVS F P S
Tip: Ensure that you have copied any modified files out of your work tree prior to runing mzclean.
Note that the stray smeserver-yum subdirectory has been removed.
mzprep: Build working tree
The mzprep command runs rpmbuild -bp on your SPEC file, sources and patches to generate a working tree. You can make changes in the working tree and use mezzanine to generate the patches between the previous version and your new files.
[gordonr@smebuild smeserver-yum]$ mzprep
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
You may now chdir to work to make changes.
Use "mzpatch -n <patch_name>" to generate a patch when done.
[gordonr@smebuild smeserver-yum]$ ls
ChangeLog CVS F P S work
The only obvious change is the creation of a work/ subdirectory. Let's have a look in there. Under the work/ directory is smeserver-yum-1.2.0/ which contains the patched sources for this version of smeserver-yum.
[gordonr@smebuild smeserver-yum]$ ls work/
smeserver-yum-1.2.0
Under smeserver-yum-1.2.0/ are the createlinks script, the root hierarchy and a few generated files, explained below.
[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/
createlinks debugsources.list smeserver-yum-1.1.2-26-filelist
debugfiles.list root
SME Server package directory layout
Table 14-1. SME Server package directory layout
Directory/File Description
createlinks Builds action, panel and initscript links.
po Package translations, if gettext is used.
root The top-level directory of the package, which will be the / directory of the installed system. So, root/usr/lib/perl5/site_perl becomes /usr/lib/perl5/site_perl on the installed system. Note that the somewhat confusing root/root is the /root directory of the installed system.
update-po Helper script to generate translation binary files, if gettext is used.
debugsources.list debugfiles.list Generated by RPM build - lists this files which are built with debug flags
smeserver-yum-1.1.2-26-filelist Generated by RPM build - lists the files to be packaged, and their permissions and ownership
The root directory hierarchy
The work/smeserver-yum-1.2.0/root/ hierarchy contains the files which will be installed on the target machine.
[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/root/
etc sbin service usr var
Modifying a file
OK - now we can actually make a change. We want to modify the file which will be installed as /etc/e-smith/events/actions/yum-import-keys, so the actual file we need to modify is: work/smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys.
Use your favourite editor (in Unix text mode) to modify the file, adding a comment where the change needs to be made. Here's a diff showing such a comment:
[gordonr@smebuild tmp]$ diff -u yum-import-keys.orig yum-import-keys
--- yum-import-keys.orig 2006-05-22 16:28:51.734609534 +1000
+++ yum-import-keys 2006-05-22 16:28:39.006814950 +1000
@@ -30,6 +30,7 @@
for my $key ( grep {!/^\./} readdir(DIR) )
{
warn "Importing key $key\n";
+ # TODO: Skip keys which have previously been imported
system("rpm", "--import", $key) == 0 or
warn "Couldn't rpm --import $key\n";
mzpatch: Create a patch
Mezzanine automates the creation of patches through the mzpatch command.
Note: Always run mzpatch from the top-level directory of your package (e.g. ~/smeserver/smeserver-yum/).
[gordonr@smebuild actions]$ cd ~/smeserver/smeserver-yum/
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
You will see from the listing above that the patches should be named package-version-patchname.patch. You should also ensure that the patch name you are about to choose doesn't already exist, or you may overwrite an older version of the patch.
[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs [server aborted]: "add" requires write access to the repository
You do not have write access to the master repository.
srctool: Error: Addition of P/smeserver-yum-1.2.0-ImportKeysComment.patch failed.
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ImportKeysComment.patch
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
Mezzanine has created the patch, but it has not committed it to CVS. This allows you to revise the patch until it works, prior to committing it. If you do not have a developer account you will get an error when mezzanine attempts to reserve the patch name in CVS, but the patch will still be created in the P directory.
The patch looks quite similar to the one we saw previously, except for the pathname to the modified file:
[gordonr@smebuild smeserver-yum]$ cat P/smeserver-yum-1.2.0-ImportKeysComment.patch
diff -Nur -x '*.orig' -x '*.rej' smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys
--- smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2005-09-26 12:30:54.000000000 +1000
+++ mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2006-05-22 16:35:57.752811633 +1000
@@ -30,6 +30,7 @@
for my $key ( grep {!/^\./} readdir(DIR) )
{
warn "Importing key $key\n";
+ # TODO: Skip keys which have previously been imported
system("rpm", "--import", $key) == 0 or
warn "Couldn't rpm --import $key\n";
Building a new package, with the patch
The next step is to change the package SPEC file so that the patch is applied.
Update the Release tag
The first change you should make is to update the "Release" tag in the SPEC file. You should do this by appending your initials and sequence number to the Release number. In most SME Server SPEC files, the Release number is set via the release macro, and so you should set it there.
Summary: YUM, an rpm updater
%define name smeserver-yum
Name: %{name}
%define version 1.2.0
- %define release 05
+ %define release 05ff01
Please just append your initials and a sequence number to the release tag. Do not change the package name (e.g. to smeserver-yum-fred) or the version number. This policy makes it obvious that your update is a patch based on smeserver-yum-1.2.0-05.
If a change has been approved by the Development Manager, it is given an official version/release number (e.g. smeserver-yum-1.2.0-06). Always tag unofficial changes with your initials and sequence number.
If an upstream (e.g. CentOS) package needs to be patched and it is not possible to wait for the patch to come from the upstream source, the release number should be modified with the special tag sme and a sequence number (e.g. clamav-0.88.2-1sme01). The sme tag is reserved for changes approved by the Development Manager.
Add a changelog entry
The SME Server project maintains history of packages changes in the %changelog section of the SPEC file. The changelog entry should be a simple description of the change of behaviour and must include a reference to an SME Server project Bugzilla entry (1174) and the new version number (1.2.0-05ff01):
%changelog
+ * Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
+ - Initial work on avoiding duplicate RPM keys [SME: 1174]
* Mon May 1 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
- Remove stray yum.pm.orig file. [SME: 1350]
The first line of the changelog entry shows when the change was made, by whom and notes the new version number. The other lines describe the change and refer to the Bugzilla entry or entries, using the format [SME: 1174] as a shorthand.
Add a patch reference
You now need to tell RPM to retrieve and apply the new patch. This needs to be done in two places, which is annoying, but allows for patches to be carried around, but not applied (e.g. because they are not quite finished). The first change is near the top of the file:
Patch0: smeserver-yum-1.2.0-DisplayStatus.patch
Patch1: smeserver-yum-1.2.0-ModifyUpdateDBs.patch
Patch2: smeserver-yum-1.2.0-DisplayStatus.patch2
+ Patch3: smeserver-yum-1.2.0-ImportKeysComment.patch
The second change is in the %prep section:
%prep
%setup
%patch0 -p1
%patch1 -p1
%patch2 -p1
rm root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/yum.pm.orig
+ %patch3 -p1
mzbuild: Build a new package
With all that done, you should be able to build a new package using mzbuild:
[gordonr@smebuild smeserver-yum]$ mzbuild
[...]
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.35037
+ umask 022
+ cd /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/build.mezz/BUILD
+ cd smeserver-yum-1.2.0
+ /bin/rm -rf /var/tmp/mezzanine-buildroot.18508
+ exit 0
Package build succeeded. Output files are:
smeserver-yum-1.2.0-05ff01.src.rpm
smeserver-yum-1.2.0-05ff01.noarch.rpm
Check the package
You should now perform some simple checks on the package before installing it for testing.
*
Does the built package have your initials in the release tag? We never want to have two packages with the same name and version number.
[gordonr@smebuild smeserver-yum]$ ls *.rpm
smeserver-yum-1.2.0-05ff01.noarch.rpm smeserver-yum-1.2.0-05ff01.src.rpm
*
Did you update the changelog? It's easy to forget, but vital to do.
[gordonr@smebuild smeserver-yum]$ rpm -qp --changelog \
smeserver-yum-1.2.0-05ff01.noarch.rpm | head -8
* Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
- Initial work on avoiding duplicate RPM keys [SME: 1174]
* Mon May 01 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
- Remove stray yum.pm.orig file. [SME: 1350]
*
Did you remember to apply the patch? It happens, and leads to head scratching "I knew I made that change".
[gordonr@smebuild smeserver-yum]$ mzbuild | grep patch
+ echo 'Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):'
Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):
+ patch -p1 -s
+ echo 'Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):'
Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):
+ patch -p1 -s
+ echo 'Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):'
Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):
+ patch -p1 -s
+ echo 'Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):'
Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):
+ patch -p1 -s
Test your new package
You should now upgrade your test system:
[gordonr@sevendev1 tmp]$ sudo rpm -Uvh smeserver-yum-1.2.0-05ff01.noarch.rpm
Preparing... ########################################### [100%]
1:smeserver-yum ########################################### [100%]
and test that you have fixed the bug.
Lather, rinse, repeat
If you haven't fixed the bug, repeat the cycle: make changes, build a new patch, update the release number (e.g. to 05ff02, 05ff03, etc.), build a new package, test. Don't worry about using up release numbers.
Note: Actually fixing bug 1174 is left as an exercise for the reader. Fame and fortune (or at least thanks) await the person who does.
Attach your patch(es) to the bug
You should now attach your patch(es) and SPEC file changes to the appropriate Bugzilla entry. If the change is the simple application of a patch, you do not need to attach the whole SPEC file - just the changelog entry.
Please attach the patch(es) and ensure that they are in Unix text format. Patches make it obvious which sections of code have been changed and attaching them ensures that their formatting is preserved.
SourceForge developer CVS access
If you are going to work on many bugs, or wish to help with the many tasks involved in a release, you should create a SourceForge developer account.
You do not need a SourceForge account in order to do development. You can contribute very effectively by producing patches and attaching them to the Bugzilla entries.
Create a SourceForge account
*
Create a SourceForge user id
*
Contact the Development Manager (or one of the other project admins) listed on the SourceForge project page, detailing the packages you would like to work on
*
The project admin will add you as a developer
*
Upload an SSH2 key to SourceForge, following the SourceForge documentation: https://sourceforge.net/docs/F02/en/#key_posting
*
You will also need to generate another SSH2 key for access to the smebuild build server
mzgetsf: SourceForge developer CVS checkout
Once you have SourceForge CVS access, you can use mzgetsf (instead of mzgetsf_anon) to retrieve the sources. Your copy will be writable and you will be able to put changes back to CVS. When you run mzpatch you should not see an error during the cvs add step.
[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs add: use 'cvs commit' to add this file permanently
Patch added to SCM. Use 'mzput' to upload to repository.
mzput: Put the changes back to CVS
Instead of attaching the patch to the Bugzilla entry, you will be able to put it back to CVS. Mezzanine normally edits the ChangeLog, but we don't need to do that as we keep the changes in the %changelog section of the SPEC file. The -m option to mzput allows a simple comment to be appended, which is all that is required:
[gordonr@smebuild smeserver-yum]$ mzput -m 'See changelog'
Build the official package
Official packages have unadorned release tags (e.g. smeserver-yum-1.2.0-05) and must be built on the official build servers. There are three steps involved - build the RPMs (mzbuild), sign the RPMs (rpm --addsign *.rpm) and release the RPMs (release_rpms *.rpm).
mzimportsf: SourceForge package import
If a package has not already been checked into SourceForge CVS, you can use mzimportsf to import it. You should be careful to run this command from a directory which does not contain an imported package or mezzanine may import the package underneath the existing package.
mzmerge: Merge changed source RPM
If a new source RPM has been built without corresponding changes in SourceForge CVS, you can use mzmerge to merge in the changes. This can also be used to merge in changes from a modified upstream package. Care should be exercised with this command, especially if you want to maintain previous patches.
